Error loading texture from HEIC image

🎤 General Discussion Swift
1

I’m writing a Multiplatform app in SwiftUI that uses Metal textures to create kaleidoscopes.

I have code that lets the user select an image and attempts to load that image as a texture.

If the image data I attempt to load is from an iPhone HEIC image, it only works if the image is in certain orientations. In other orientations, it fails with the error “Error loading texture: Error Domain=MTKTextureLoaderErrorDomain Code=0 “Unsupported image orientation”.

The code looks like this:

        let loader = MTKTextureLoader(device: device)
        do {
            let options: [MTKTextureLoader.Option: Any] = [.origin:MTKTextureLoader.Origin.bottomLeft, .generateMipmaps: true]
            let tex = try loader.newTexture(data: imageData, options: options)
            Task { @MainActor in
                scopeState.texture = tex
            }
            let hasAlpha =
            tex.pixelFormat == .rgba8Unorm ||
            tex.pixelFormat == .rgba8Unorm_srgb ||
            tex.pixelFormat == .bgra8Unorm ||
            tex.pixelFormat == .bgra8Unorm_srgb ||
            tex.pixelFormat == .rgba16Float ||
            tex.pixelFormat == .rgba32Float
        } catch {
            print("Error loading texture: \(error)")
        }

(Where device is my MTLDevice and imageData is the Data I read from the HEIC file.)

I get the same result on iOS, if I load the image from the user’s photo library, or on MacOS, if I load the image from a file using Data(contentsOf: url)

Only an image in landscape left seems to load. Images shot in all other orientations fail to load with the above error.

How do I get HEIC images to reliably load as Metal textures, regardless of their orientation?

2

@caroline may be able to help