Manipulating Vertices by drag and drop

Hi Caroline and others,

I hope you are well.

I want to import an obj model, but be able to manipulate vertices on the screen. How is it best to do that?

Thanks a lot


Funnily enough, we were talking about this the other day - how to deal with taps and selection:

Conceptually, OpenGL is much the same as Metal, and this suggests using colors:


Hi Omar, I was chatting to a few friends at Apple and Nvidia who all recommend the color way as it has less overhead regarding implementation( you dont need the bsp trees for objects and triangles ). I would suggest giving the colour method a try first.


Thanks. That is great advice. I will have some homework to do :slight_smile: :smile:

The next step I will need to do when I click on a vertex ist to reposition it. But I want to stretch the adhering vertices. I guess this is best done with a tensor, like clothes? Or should I apply a scaling matrix with variable scaling factor?

@rumor thanks. I guess you could introduce me to your friends :smiley:

That I wouldn’t know. It would be fun to try to do it like cloth.

What do you mean by stretch? Just translate the vert into a new position? Soft selection translate or kinematic simulation translation, where every vert tries to follow every other vert and maintain its distance between all its neighbours?( Very complicated)

For Cloth, take a look into Verlet cloth implementation. Here is a pretty good article covering the topic

thanks @rumor . I made a small model in Matlab, with a very simplified solution:

  1. Imagine the model is a shirt.

2)I selected the vertices in the arm. In Matlab I did it with identifying the vertices in a polygon. In Metal I could do it with a sub-mesh.

  1. I calculated a line , originating from the middle of the shoulder and continuing to the middle of the sleeve. Which I will cal backbone

  2. When scaling, I calculate the distance of the vertex to this backbone and then multiply it with the calculated scale to be done.


Now: I even do not know how to do that in metal, second: The drag thing should make it more organic. Theoratically, I know which vertices to pull, and which not. So if I drag the lowest point of the sleeve, I could do the same differential geometry equations to a certain set of vertices. The thing is in Matlab I can make a matrix holding the vertices I want to move. In Metal I do not. But does this make it easier?

I’m afraid I can’t help you, Omar, but if you can do it in Matlab with matrices, it seems to me that you should be looking at Accelerate (CPU):

and perhaps Metal Performance Shaders (GPU):

Hi @omarfakhr,

What you have described is very common in 3D world. What you describe is usually called a soft selection, or soft fall-off.

There are multiple ways to implement this.

  • the most common is allow the user to input a distance and then specify a curve that goes from 1 to 0. 1 being the selected vertices 0 being the distance from the selected vertex.
  • best to look up falloff distances or linear interpolated fall offs, as there so many custom implementation.
  • Alot of 3D applications allow the user to paint a texture map(0…1) onto the surface and then use that map by the translation delta.

Implementatino needed off the top of my head:

  • import obj/usd
  • Vert selection
  • Gizmo (rotate,translate,scale)
  • falloff/soft selection modifier on a manipulator/gizmo
  • Texture painting(could be stored per a vert, and cleared when the user is happy)
  • Updating the CPU verts data with the GPU vert data and keeping it all in sync.

I hope that helps give you a bit more information. there quite a few implementation steps to get it working.


@rumor thanks Si,
I managed to import an obj in an understandable fashion as described here:

Is there a possibility to implement this data in the draw function? or convert it to mdlmesh and mtkmesh?

I think I used the same code to get the vert data from the import buffer and then stored it in a custom data structure. I did not do this in a draw function as you would be getting alot of data over and over for each frame.

I would suggest storing it in a variable after load time. and then using the stored variable for all your drawing / alteration needs.

Thanks @rumor
How do I transport the variable back to SceneKit? I mean make a frame with the variable? Any suggestions/code ?

Sorry @omarfakhr, I have not done any SceneKit dev work at this current time. I have been doing everything in Metal at a low level.