professorkamrankhan@gmail.com 
If Apple ever does bring Face ID to the Mac, newly revealed research shows how the company could also use its LiDAR to recognize a user’s hand gestures.
Face ID first came to Apple devices in 2017 with the launch of the iPhone X. It’s now on every iPhone, and many iPads, but it has not come to the Mac.
This means that while a newly-revealed Apple patent application about gesture detection promises much for the future, there’s an unstated extra element. The technology described would scan users, looking for gestures, and that means a camera sensor on the iMac, the MacBook Pro, or perhaps the MacBook Air.
While the patent does not mention Face ID or even just the word lock once in its more than 12,000 words of description, it does detail the use of cameras. A front-facing camera, it says, could be used to detect gestures.
There is no guarantee that such gesture detection would use the same technology as Face ID, and Apple’s illustrations suggest the camera would be at the base of the MacBook screen. But they would still be mounted so that they can observe a user facing the Mac, and while in use, they would constantly be scanning for those gestures.
Detail from the patent application showing a system detecting hand gestures — image credit: Apple
Apple’s interest here is in how what it describes as a pair of cameras can observe what’s in front of them.
“Hand movements and other gestures may involve relatively quick movements,” says the patent application. “Accordingly, gesture recognition systems that use images from traditional frame-based cameras [alone] may lack accuracy and efficiency or otherwise be limited by the relatively slow frame rates that such cameras use.”
So Apple proposes using what it calls an event camera, broadly speaking a video one instead of stills camera. “Event cameras capture events as they occur for each pixel and may provide the capability to capture data about user movements significantly faster than many frame-based cameras,” it says.
There is a problem, though, in that an event camera that picks up everything happening in front of the Mac will detect so much that identifying gestures will be difficult.
“The need to analyze and interpret potentially vast amounts of event data from a physical environment may significantly reduce the ability of such event camera-based gesture recognition systems to quickly, efficiently, and accurately identify gestures,” says Apple.
Its proposed solution is to create what it calls a bounding box around where in view a user’s hands might be. It might do that in part by the frame-based camera being used to “remove the background from consideration in the event camera data analysis.”
That means identifying what’s the background and what is the user in the foreground. Apple’s illustrations show a bounding box encircling a user’s hand, so that also means recognizing hands.
Consequently, while the patent application describes gestures being used to control objects in VR, every bit of it points to it using at least a form of Face ID.
At present, the reason that there is no Face ID on Macs is thought to be because the lid of a MacBook Pro is simply too thin to contain all of the components.
However, Apple is expected to launch both an iPhone Fold and a slim iPhone 17 Air that would each have thinner chassis than previous models. Perhaps it’s wishful thinking, but the thin design plus this patent do make Face ID on the Mac seem hopeful.
