Metal by tutorials book 2?

📚 Official Books Metal by Tutorials
1

is there going to be another book?
this book is great!
I would love to learn mouse picking on metal, hybrid rendering, ARKit + cubemap reflections

2 Likes
2

I’m so glad you’re enjoying it!

I would be great to have another book with extra topics, but at the moment, I don’t believe one is in the pipeline. That’s not to say that there won’t be, and your feedback definitely helps :blush:.

We had a post about picking: Detect tap on a mesh - as I wrote there, when I’ve needed to, I have assigned objects a color and then written the objects to a texture, then work out coordinates on a screen touch and read the color from the texture which gives the object hit. rumour had another method with ray casting.

2 Likes
3

@caroline

Thank you!
this method does it have a name? and for a 1 object 50k triangles will it work which triangle was hit? would I render each triangle with unique different color onto a texture → framebuffer?

4

It’s a fairly common technique. Also called hit testing.

Warren Moore wrote about ray picking: Picking and Hit-Testing in Metal – Metal by Example

There’s this thread about object id picking on the Apple forum: https://developer.apple.com/forums/thread/68587

Although, reading that made me think perhaps instead of doing a color texture, you could use primitive id somehow. I haven’t used primitive id yet, as it only arrived into Metal last year.

About the 50k triangles - you’d have to test for speed. I don’t know. Would you be rendering all 50k triangles at the same time? Surely some would be overdrawn? On the A14/M1 chip, the overdrawn triangles would be caught during tile binning, so they wouldn’t overload the fragment phase.

5

Thank you,
I was able to render a shoe mesh(50k triangles) texture with primitive id
tempImagelFvs9q

and I guess my next question is
how can I know the position of the triangle that I hit/select on the color texture, so I can place a sphere on top of that selected triangle?

1 Like
6

Does the primitive id match the position in the vertex buffer?

| position0 | normal0 | position1 | normal1 | position2 | normal2 |

You could index into the vertex buffer and extract the position?

Oh wait - do I mean index into the index buffer and then get the vertex indexes from that to index into the vertex buffer?

I’m looking at an index buffer in the Metal debugger, and it’s a uint3, so it should give you the three vertex indices.

Does the primitive id match the indices in the index buffer? You might have to experiment with a cube mesh rather than 50ks and look at the Metal debugger to see what’s going on in the buffers

1 Like
7

@Caroline I’m getting very close…it worked for a cube, but for a high poly its weird
but I’m not getting the right position what do you think I’m doing wrong?

                // Access the indices buffer and get the indices of the vertices
                let indexPointer = mesh.submeshes.first!.mtkSubmesh.indexBuffer.buffer.contents().advanced(
                    by:((MemoryLayout<UInt32>.size+MemoryLayout<UInt32>.size+MemoryLayout<UInt32>.size) * Int(selectedPrimitive)))
                
                let triangleIndices = indexPointer.assumingMemoryBound(to: (UInt32, UInt32, UInt32).self).pointee
                
                let vertexPointer = mesh.mtkMesh.vertexBuffers[0].buffer.contents().advanced(
                    by:(ModelVertex.size() * Int(triangleIndices.0)))
                
                let vert = vertexPointer.assumingMemoryBound(to: (Float, Float, Float).self).pointee
                
                vertex = float3(vert.0,vert.1,vert.2)

Update:
Finally It works! my kernel texture2d had to be float instead of half

Thank you for everything!!!

8

I’m so glad you got it working!

I just did a different sort of picker for demonstration using the primitive ID:

1 Like