professorkamrankhan@gmail.com 
We won’t now see an Apple Car, but some of the technology invented for it may be coming to the Apple Vision Pro — image credit: Apple
Apple is researching how to make software for the Apple Vision Pro to help prevent motion sickness from perceived motion, as well as caused by the motion of a vehicle.
Probably all Apple Vision Pro users say that wearing your headset during a flight is more embarrassing — or at least prone to getting knocked in the overhead storage compartment. But some say that depending on what you’re watching, they can get motion sickness because of how the headset’s sensors are sometimes thrown by the movement of the aircraft.
It’s like an exaggerated version of the motion sickness that some users feel when, say, reading their iPhone in a moving bus. Apple addressed that specific issue with its Vehicle Motion Cues in iOS 18.
Originally announced as just one of a slew of new Accessibility options in May 2024, Vehicle Motion Cues present a quite mesmerising series of dots on the iPhone’s screen. The dots are small enough to not interfere with what you’re reading or watching, but large enough that you register when they move.
And the dots move in sync with the bus or car’s own movement. It’s using the iPhone’s accelerometer to convey vehicle movement so that a user’s eyes are getting the same information that their body is feeling.
A newly-granted patent called “Immersive virtual display” proposes a version of this that could come to the Apple Vision Pro, or perhaps to future versions of it.
Apple’s proposed technology involves implementing virtual reality methods “to address problems with vehicles in motion that may result in motion sickness for the passengers.” The issue, according to Apple’s patent, is that providing “immersive VR experiences to passengers in vehicles,” can entail “replacing the view of the real world with any of various types of virtual experiences and environments that a passenger may desire.”
Although clearly described as being for use in a vehicle, the proposal could readily be used anywhere — image credit: Apple
Consequently, there is a need to integrate real-world vehicle motions into these virtual experiences to help prevent motion sickness.
Some of this research clearly dates from when Apple was working on an Apple Car, because it assumes the headset will be worn in a totally controllable environment. For example, says Apple, the user could experience “rushing wind or heat through the HVAC system, surround sound and sound effects through the audio system, and acceleration or motion elects through the seat.”
If that sounds like it will just make things worse, Apple proposes that the Apple Vision Pro “may monitor a passenger for signs of discomfort or motion sickness,” and then do something about it. For instance, if the user is looking a bit queasy, the Apple Vision Pro could perhaps slow down the virtual visual movement.
There’s no detail concerning exactly how that might be achieved, and it would presumably depend on what virtual experiences the user was seeing. But Apple says it would be possible to achieve “a 1:1 mapping ratio” with the virtual motion and the actual vehicle movement.
As well as directly automated changes, though, the idea is that a user will be able to choose what they want — and the rate at which they get it. So rather than waiting for detection of motion sickness, a user who is prone to is could set the Apple Vision Pro to accommodate them from the start.
It would seem that the easier thing to do is to just not wear an Apple Vision Pro while travelling, but oddly Apple doesn’t seem to consider that.
Whereas Apple does very much consider how this same proposal could perhaps benefit people who do not have motion sickness. “[The] 3D virtual views may include views of the passenger’s environment, including the environment outside the vehicle, augmented with virtual content (e.g., virtual objects, virtual tags, etc.) or alternatively may provide immersive VR environments that may include visual cues of the environment outside the vehicle.”
So you could wear a headset and through it see both your real surroundings, and whatever AR additions you want.
As ever, Apple concentrates on how technology can work and be delivered, rather than proposing specific use cases. But, again showing its Apple Car origins, this patent does list a slew of options from connecting a MacBook Pro to the headset — to giving them control over braking in the vehicle.
A virtual environment could be supplemented with an indicator of the real-life horizon — image credit: Apple
We may never now see all of this as the Apple Car is dead, and so must this idea of a virtual backseat driver. But as Apple continues to develop further versions of the Apple Vision Pro, it looks like it is also continuing to consider associated accessibility needs.
The patent is credited to eight inventors, led by Mark B. Rober. In yet another sign of how well Apple hid its Apple Vision Pro patents in plain sight, back in 2022, Rober discussed how he had devised a virtual display for Apple Car that would have combated motion sickness.
