With the introduction of visionOS 27, Apple is opening the door for third parties to make their own high performance tracked controllers and accessories for Vision Pro.
The News
VisionOS 27, available to developers starting today, adds new support for actively-tracked accessories like motion controllers, which Apple is calling ‘spatial accessories’. Prior updates to visionOS allowed for passively-tracked objects which were tracked by their appearance but did not actively communicate with the headset. On the other hand, visionOS 27 allows Vision Pro to track accessories based on known IR LED patterns and positional data streamed from an accessory’s on-board IMU via Bluetooth. Spatial accessories can also send input data from things like buttons and thumbsticks, as detailed in a newly released developer session from Apple.
The combination of IR LEDs and an IMU is the defacto standard for tracking VR motion controllers. This approach enables greater precision and lower latency than trying to passively track an object based on its geometric shape alone. The IR LEDs are more robust to various lighting conditions while the IMUs can provide hundreds of updates per second, increasing accuracy by offering more recent updates about the motion of the controller. Apple says spatial accessories will be able to track up to the display rate of the headset (nominally 90Hz, but possibly up to 120Hz).
While this news doesn’t mean that you’ll be able to connect any old VR controller to Vision Pro, it does mean that it’s possible for the companies that make controllers (or other tracked accessories) to add official Vision Pro support for their current or upcoming products. Apple says that a spatial accessory’s tracking profile can be included in a visionOS app which “registers your accessory system-wide, so any app on Apple Vision Pro can use it.” However, third-party tracked accessories made in this way won’t be able to support the full VisionOS interface like the PSVR 2 controllers and Logitech stylus that Apple made official Vision Pro accessories last year.
Apple also revealed that it has been working with companies DFRobot and MIKROE to create off-the-shelf trackers, which can be used as “reference hardware and development kits” for building Vision Pro compatible tracked accessories.
The trackers will be available “later this year,” though other details like pricing were not shared in the session.
In addition to adding support for actively tracked spatial accessories, VisionOS 27 also improves upon existing support for passively tracked objects. Apple says a new high frame-rate tracking mode and extended training for tracking models bring lower latency and more reliable object tracking to the headset.
My Take
Apple has completed a full come about on motion controllers for Vision Pro. When the headset first launched, the user’s hands (and technically their eyes too) were the only tracked input device. That flew in the face of nearly every major AR and VR headset that launched before Vision Pro, and ensured that most of the best regarded VR games at the time would have no chance of running on the headset. This surely wasn’t an oversight by Apple, but rather an intentional choice.
It’s true that Vision Pro’s dismissal of motion controllers led Apple to the ‘look + pinch’ input system of Vision Pro, which I personally consider one of the headset’s most powerful innovations. But as great as that system is for quickly and accurately controlling flat content and interfaces, it’s absolutely no replacement for the kind of reliable, high performance motion controller tracking that makes most of VR’s best games possible.
As a company that often stands by difficult design choices, I wouldn’t have put it out of the realm of possibility that Apple would stick to its guns with the hope that VR game developers would figure out how to work within the confines of Vision Pro’s look + pinch input system. Luckily, it seems the company heard the feedback loud and clear.
Apple laid the foundation for motion controller support over the last year when it made PSVR 2 controllers and a Logitech stylus officially supported tracked accessories for Vision Pro. But until now, those were the only actively-tracked accessories that could work with the headset.
While it’s good news that Apple is opening the door to actively tracked third-party motion controllers on Vision Pro, it remains to be seen how much it will really matter to Vision Pro in its current form. Even with the addition of official PSVR 2 controller support, there hasn’t exactly been an influx of well-known VR games to the system. But considering the headset’s huge pricetag, it’s unlikely Apple thought there would be.
More likely, the company is answering a nearer-term need derived from niche use-cases of the headset in enterprise and other non-consumer settings like training and simulation. Granted, Apple generally adds features and developer capabilities with the intent to support them long-term. So while third-party motion controller support might not be a major boon for the current Vision Pro, it could have a much more meaningful impact on future VisionOS headsets, especially given that it would be available at launch rather than two and a half years after the headset first hit the market.
