Using a triangle as environment map

[mk2019]

Hi all,
I’m trying out this thing where I use a triangle instead of a cube as an environment map in the background. But I cannot get it to work, the triangle is not visible where it should be.

In the code where there is a it should be 1,1 for the z and w in the float4() – but then I don’t see the triangle…?
I set w=4 just as an example to see the triangle as smaller and in front of everything to see that it is there and working.

So my question is why w:1 does not show the triangle as it should be in clip space at the far end behind the spinning torus?

With this technique, the idea is that I don’t have to worry about any object being clipped i 3D-space, since the env-map is always behind everything.

Link: Dropbox - EnvMap1.mov - Simplify your life

Here is my shader code:

#include <metal_stdlib>
using namespace metal;

struct FullscreenVertexOut 
{
    float4 position [[position]];
    float4 clipPosition;
};

vertex FullscreenVertexOut EnvironmentMappingNode_VertexMain(uint index [[vertex_id]])
{
    // triangle vertices ◸
    const float2 positions[] = {
        { -1, 1 },  // left-top
        { -1, -3 }, // left-bottom
        { 3, 1 }    // right-top
    };
    FullscreenVertexOut out;
    out.position = float4(positions[index], 1, 4);  // 👋
    out.clipPosition = out.position;
    return out;
}

fragment float4 EnvironmentMappingNode_FragmentMain(FullscreenVertexOut in [[stage_in]],
                                                    constant float4x4 &inverseViewProjectionMatrix [[buffer(0)]],
                                                    texturecube<float, access::sample> cubeEnvTex [[texture(0)]],
                                                    sampler cubeSampler [[sampler(0)]])
{
    const float4 worldPosition = (inverseViewProjectionMatrix * in.clipPosition);
    const float3 worldDirection = normalize(worldPosition.xyz);
    return (cubeEnvTex.sample(cubeSampler, worldDirection) * 1.0);
}

[caroline]

Hi @mk2019 - sorry for the late answer - I’ve been on break.

It’s really hard to diagnose problems like this without the full code. Your triangle appears to rendering correctly, but I don’t know what the other code is doing.

I would suggest you investigate the in and out vertex positions for your objects using the Geometry resources in the GPU debugger.

[mk2019]

Hi @caroline

The camera:
fovDegrees = 65
near = 0.1
far = 100

There is nothing much other than the standard setup on the Swift side, and this is the update/draw function:

override func updateAndDraw(with renderEncoder: MTLRenderCommandEncoder, sceneUniform: DCMTLUniform) -> DCMTLUniform
    {
        let cameraMatrix = sceneUniform.viewMatrix
        let viewMatrix = cameraMatrix//.inverse

        var viewDirectionMatrix = viewMatrix
        viewDirectionMatrix.columns.3 = SIMD4<Float>(0, 0, 0, 1)

        var clipToViewDirectionTransform = (sceneUniform.projectionMatrix * viewDirectionMatrix).inverse

        //

        renderEncoder.setFrontFacing(.counterClockwise)
        renderEncoder.setCullMode(.none)

        renderEncoder.setRenderPipelineState(mRenderPipelineState)

            renderEncoder.setFragmentBytes(&clipToViewDirectionTransform, length: MemoryLayout<float4x4>.size, index: 0)
            renderEncoder.setFragmentTexture(mEnvironmentTexture, index: 0)
            renderEncoder.setFragmentSamplerState(mSamplerState, index: 0)

            renderEncoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)

        return sceneUniform
    }

I found out that if I set the:

out.position = float4(positions[index], 0.98, 1);

then I see the torus half cut into the environment triangle

and if I set the value to 0.99 then I fully see the torus in front of the background.
Same for 0.99999997, but 0.99999998 to 1.0 makes the background disappear.

In the debugger, this is what 0.99999997 looks like:

and 0.99999998

They both look the same in numbers and say 1.000 instead of the value set, but look to be in the “far” position.

The torus is close to the camera:

Seems like some very sensitive z-axis position happening here…

[caroline]

Thank you for providing more detail. It’s still very hard for me to see what’s going on and test out possible answers. You aren’t able to upload a small zip file?

Have you experimented with different depth stencil states?

Remember that NDC (Normalised device coordinates) for Metal are -1 to 1 in the x and y axes and 0 to 1 in the z axis.

This means that if something is positioned at z = 0.98 and the w is 1, it will be very close to the far position. If something is positioned at z = 98 and the w is 100, it will be at the same position. (Divide by w happens after the vertex function.)

[mk2019]

Ahh I found the issue.

I was using
descriptor.depthCompareFunction = .less

but if I change it to
descriptor.depthCompareFunction = .lessEqual

then it works as expected where both z & w are 1.

The dangers of changing something at some point and thinking – what could go wrong

[tured1966]

Thank you for sharing this.