Chapter 11: Facial Blend Shapes - Reality Composer Notify Triggers broken with iOS 15.5+?

fasatae · August 3, 2022, 2:35am

Hello,

I’ve gone through most of the examples and really appreciate the work done in the book.

Under the following section, where you’re adding lasers to the robot that trigger when the jaw is opened, the behavior triggers once and never repeats.

Section IV: RealityKit & Face Tracking → 11. Facial Blend Shapes → 11.14 Sending & Receiving Notifications

This happens on iOS 15.6 (and also iOS 15.5). I tested with a device running iOS 15.1 and it still works as expected.

The onAction method is not executed at the bottom of the session(_:didUpdate) method, confirmed by debugger breakpoint:

robot.actions.lasersDone.onAction = { _ in
  self.isLasersDone = true
}

This failure occurs in both the project github code (from a clean build), and the example as written in the book:

github.com

raywenderlich/apr-materials/blob/2012e7b4997bd14f96691a36f8d991079faf310f/11-facial-blend-shapes/projects/final/ARFunnyFace/ContentView.swift#L197


      
                    angle: Deg2Rad((-90 * browRight!) - (-30 * browInnerUp!)),
                    axis: [0, 0, 1]))
                
                robot.jaw?.orientation = simd_quatf(
                  angle: Deg2Rad(-100 + (60 * jawOpen!)),
                  axis: [1, 0, 0])
                
                if (self.isLasersDone == true && jawOpen! > 0.9) {
                  self.isLasersDone = false
                  robot.notifications.showLasers.post()
                  robot.actions.lasersDone.onAction = { _ in
                    self.isLasersDone = true
                  }
                }
              }
              
              func Deg2Rad(_ value: Float) -> Float {
                return value * .pi / 180
              }
            }
          }

I’ve verified that the generated code in Experience.swift is valid:

public private(set) lazy var lasersDone = Experience.NotifyAction(identifier: "LasersDone", root: root)

The Foundation observer is defined here in Experience.swift:

Foundation.NotificationCenter.default.addObserver(self, selector: #selector(actionDidFire(notification:)), name: Foundation.NSNotification.Name(rawValue: "RealityKit.NotifyAction"), object: nil)

Maybe there’s something going on with Reality Composer’s integration with Foundation NotificationCenter as of iOS 15.5? Apple Developer Documentation

robertomachorro · August 13, 2022, 4:06pm

@chrislanguage any ideas on what may be happening?