Manipulating Vertices by drag and drop

[omarfakhr]

Hi Caroline and others,

I hope you are well.

I want to import an obj model, but be able to manipulate vertices on the screen. How is it best to do that?

Thanks a lot

vest
Omar

[caroline]

Funnily enough, we were talking about this the other day - how to deal with taps and selection:

Detect tap on a mesh

To pick or “click” on a geometry items, you need to cast a ray (line-segment) out from the camera and then test against every triangle in the scene. This can be very expensive depending on the amount of geo / tries in a scene. To get around this there is a tree structure called an OctTree which is used to query regions in space. I would suggest looking into projection from screen space into world space, this will help with the creation of a line segment from camera into the world. Then loop ov…

Conceptually, OpenGL is much the same as Metal, and this suggests using colors:

How to select a vertex or triangle of a mesh in modern OpenGl (3.3 above)?

[rumor]

Hi Omar, I was chatting to a few friends at Apple and Nvidia who all recommend the color way as it has less overhead regarding implementation( you dont need the bsp trees for objects and triangles ). I would suggest giving the colour method a try first.

[omarfakhr]

Thanks. That is great advice. I will have some homework to do

The next step I will need to do when I click on a vertex ist to reposition it. But I want to stretch the adhering vertices. I guess this is best done with a tensor, like clothes? Or should I apply a scaling matrix with variable scaling factor?

@rumor thanks. I guess you could introduce me to your friends

[caroline]

@omarfakhr

I guess this is best done with a tensor, like clothes? Or should I apply a scaling matrix with variable scaling factor?

That I wouldn’t know. It would be fun to try to do it like cloth.

[rumor]

What do you mean by stretch? Just translate the vert into a new position? Soft selection translate or kinematic simulation translation, where every vert tries to follow every other vert and maintain its distance between all its neighbours?( Very complicated)

[rumor]

For Cloth, take a look into Verlet cloth implementation. Here is a pretty good article covering the topic

Simulate Tearable Cloth and Ragdolls With Simple Verlet Integration

[omarfakhr]

thanks @rumor . I made a small model in Matlab, with a very simplified solution:

1. Imagine the model is a shirt.
   

2)I selected the vertices in the arm. In Matlab I did it with identifying the vertices in a polygon. In Metal I could do it with a sub-mesh.

3. I calculated a line , originating from the middle of the shoulder and continuing to the middle of the sleeve. Which I will cal backbone

4. When scaling, I calculate the distance of the vertex to this backbone and then multiply it with the calculated scale to be done.

Now: I even do not know how to do that in metal, second: The drag thing should make it more organic. Theoratically, I know which vertices to pull, and which not. So if I drag the lowest point of the sleeve, I could do the same differential geometry equations to a certain set of vertices. The thing is in Matlab I can make a matrix holding the vertices I want to move. In Metal I do not. But does this make it easier?

[caroline]

I’m afraid I can’t help you, Omar, but if you can do it in Matlab with matrices, it seems to me that you should be looking at Accelerate (CPU):

https://developer.apple.com/documentation/accelerate

and perhaps Metal Performance Shaders (GPU):

https://developer.apple.com/documentation/metalperformanceshaders

[rumor]

Hi @omarfakhr,

What you have described is very common in 3D world. What you describe is usually called a soft selection, or soft fall-off.

There are multiple ways to implement this.

• the most common is allow the user to input a distance and then specify a curve that goes from 1 to 0. 1 being the selected vertices 0 being the distance from the selected vertex.
• best to look up falloff distances or linear interpolated fall offs, as there so many custom implementation.
• Alot of 3D applications allow the user to paint a texture map(0…1) onto the surface and then use that map by the translation delta.

Implementatino needed off the top of my head:

• import obj/usd
• Vert selection
• Gizmo (rotate,translate,scale)
• falloff/soft selection modifier on a manipulator/gizmo
• Texture painting(could be stored per a vert, and cleared when the user is happy)
• Updating the CPU verts data with the GPU vert data and keeping it all in sync.

I hope that helps give you a bit more information. there quite a few implementation steps to get it working.

Cheers,
Si

[omarfakhr]

@rumor thanks Si,
I managed to import an obj in an understandable fashion as described here:

printing vertex positions of an MDLMesh

Is there a possibility to implement this data in the draw function? or convert it to mdlmesh and mtkmesh?

[rumor]

I think I used the same code to get the vert data from the import buffer and then stored it in a custom data structure. I did not do this in a draw function as you would be getting alot of data over and over for each frame.

I would suggest storing it in a variable after load time. and then using the stored variable for all your drawing / alteration needs.

[omarfakhr]

Thanks @rumor
How do I transport the variable back to SceneKit? I mean make a frame with the variable? Any suggestions/code ?

[rumor]

Sorry @omarfakhr, I have not done any SceneKit dev work at this current time. I have been doing everything in Metal at a low level.