LONG BEACH, CALIFORNIA – I am on the solar deck of the Queen Mary with Steve Lukas on the discharge day of his app to Meta’s storefront.
Lukas’ 15-year-old son Christian joined him for Augmented World Expo final week, and pressed the button a number of hours earlier on stage to launch Jigsaw Night time into early entry for patrons.
A mild breeze whips away the warmth of the solar from our pores and skin as we breathe within the recent ocean air and speak about what he is constructed during the last seven months. Or eight years, relying on how he seems to be at it. We’re piecing collectively a jigsaw puzzle and filling within the final gap within the image on the field with a satisfying click on. Every of us is carrying a Quest 3 headset.
“I do not know what that regarded prefer to you,” Lukas apologizes for the boundaries of his code at launch.
“Are you able to do me a favor and maintain [the puzzle] and stroll over right here?” I requested him, gesturing to the aspect of the ship.
He carries it over and hangs the completed puzzle within the air as we glance out over the water to the Aquarium of the Pacific.
“That is good,” he says.
“That is good,” I agree.
“Why did this take so lengthy and why does this really feel so good,” I ask him.
“I have been engaged on shared areas for slightly over eight years,” Lukas replies.
“Ever since I built-in ARKit, the Microsoft HoloLens and the Vive, and from that I used to be capable of get 6DOF controllers with an AR clear headset and anybody may see me utilizing ARKIt. And I stated, that is a magical second. But it surely takes all this tech to make that occur. And so from there, I have been all of the alternative ways to make that occur simpler. And the progress of expertise has introduced these instruments to do automated spatial anchors, automated detection, and I had encountered that there are people who stated, ‘you possibly can’t do it manually, it must be automated.’
“That is like saying, ‘we cannot make vehicles till we do away with the stick shift.’ After which you could have years that you do not be taught. As a result of when you get previous ‘this works’, then you definitely go onto different issues. Occupied with these issues hasn’t been accomplished by that many individuals. Some have been doing it very effectively, however the variety of individuals these issues is smaller than I anticipated it to be.”
After our puzzling session, which matches on longer than supposed as a result of we will’t cease speaking, he exhibits me the element record of his Unity venture. Among the many neatly sorted items in Unity:
- World Anchoring
- Meta Quest Platform
- Puzzle Technology
- Puzzle Monitoring
- Puzzle Database
- Music Field
- Voice Chat
- Blended Actuality Utility Package
- Wit.Ai
- LivCam
- FacebookSDK
- Leaderboards
- Store
After I noticed him earlier than the AWE present ground opened on Tuesday, he wasn’t sure Jigsaw Night time was going to launch. After I accomplished my single-player demo, and advised him I believed it was unbelievable work, he dedicated to the plan he executed together with his household on Thursday.
When you have a pair of Quest 2, 3, 3S, or Professional headsets that run Horizon OS, Jigsaw Night time is at present among the best experiences you should purchase from Meta for these headsets to have in the identical room with another person.
Jigsaw Night time is not good but, however it’s already higher than most apps as a result of that element record does some issues individuals have solely skilled in enterprise or analysis labs with entry to hundreds of {dollars} of apparatus.
As an example, Lukas makes use of the phrases “co-presence” and “co-location” to imply two various things, whereas different builders with out his understanding of the medium would possibly use them interchangeably. In Jigsaw Night time, you may be co-located in the identical bodily area as a number of different gamers additionally within the room with you. Or you possibly can have a few individuals distant in from one other location utilizing co-presence, sending their physique actions or voice over the web, and receiving again yours in return. From this, you all have a shared actuality — a playground sharing an structure for spending time collectively — and the thought is so necessary {that a} silly variety of Silicon Valley advertising and marketing leaders have tried to model this expertise in numerous methods.
Making a shared area in headsets of any form that enables for each co-presence and co-located interactions is exceedingly uncommon. It’s the stuff pursued by platform giants like Microsoft, Apple, Google, and Meta, in addition to by startups hoping for acquisition by a type of bigger corporations. In essence, the race to make a shared actuality for individuals is each aggressive and susceptible to quickly altering government priorities. This implies many builders have opted to keep away from utilizing these early instruments, serving to them to keep away from platform lock-in and the restricted market alternative related to it.
Not Lukas and never Jigsaw Night time. This isn’t only a puzzle sport. It’s a examine in each the blockers and alternatives in entrance of the whole XR business. Lukas’ work would possibly as effectively be a pattern venture demonstrating finest practices for blended actuality design on Horizon OS — the pattern venture Meta ought to have made for builders and headset patrons to allow them to perceive the potential of this blended actuality platform.
I’ve written about Lukas’ work twice earlier than. First when his son Christian, then a third grader, demonstrated this identical important idea to his classmates in 2017 on HoloLens. Then I coated Lukas’ work once more in 2018 as Magic Leap shipped its first model of restricted subject of view excessive price Augmented Actuality goggles as a bunch of builders confirmed off this basic concept. The identical concept that Lukas is now bringing on to VR and blended actuality through Quest headsets.
One significantly impactful second testing Jigsaw Night time got here after I activated the LIV digital digicam. The digicam is extra absolutely featured than some cell phone cameras, permitting for angles in first-person or third-person, adjustment for FOV and stabilization, in addition to for a selfie cam that may hold in midair or be handheld.
The magic is that the LIV digicam is technically able to solely filming what seems in your shared digital actuality area. The digicam can not seize your bodily setting, even if you are it, because the headset passes AR by means of VR. The function solely works as a result of Jigsaw Night time triggers a scan of your bodily room, giving the headset and apps a semantic understanding of its form in addition to the furnishings, home windows, and doorways inside. The function has been obtainable in Meta’s ecosystem since early 2024, and nonetheless right this moment only a few builders put it to use.
A four-minute video I captured with the LIV instrument exhibits the impact of the mix my first time testing it out. Contained in the headset, I can nonetheless see my lodge room aboard the completely docked ship in Lengthy Seaside utilizing passthrough blended actuality. On the LIV selfie cam I see solely VR, the identical because the viewer sees, utilizing Meta’s newest avatar tech and my room represented as a sparse illustration of what really surrounds me. I solely actually perceive what’s taking place on the finish of the video, after I see my very own reflection in a bodily mirror and finish the recording.
Most of the movies you see on UploadVR displaying gameplay are sometimes not our first instances trying to do what you see depicted. It usually takes observe. As an example, studying by means of trial and error how a developer implements the straightforward human motion of greedy an object.
The 4-minute video I embedded above is my first time making an attempt LIV integration in his app, and what you see is that it simply labored, demonstrating what quantities to a grasp class in interplay design. Lukas took the hassle to help each pinch and grasp gestures for holding issues with hand monitoring, and helps the identical with each the set off and grip buttons on controllers. He even helps the usage of one every — tracked controller for superior interactions in a single hand and simply your hand for the opposite.
That video is what led me to ask Lukas to the Queen Mary for a puzzling session collectively on the highest deck. I needed to check co-location away from the Augmented World Expo present ground, the place earlier within the week I noticed him involved as he struggled with the extreme wi-fi interference. The truth is, our first article on Tuesday was solely printed that day as a result of Lukas’ and I exited the occasion so he may Airdrop video to me illustrating the work.
I left my controller within the room, put my headset in a bag, and introduced Lukas as much as the highest of the ship. A short time later, Scott Stein from CNET joined us. Lukas placed on a headset and equipped Stein with one too. I logged onto the horrible lodge Wi-Fi and clicked “Get together Up”. That prompts an invisible handshake between the three headsets and creates a Meta Shared Spatial Anchor, which basically locks our digital content material to the identical anchoring level on the planet.
After which we had been chatting as we pieced collectively a double-sided puzzle. Stein and I reached for a similar piece from reverse sides, bumping palms after which laughing.
Lukas and I consider in about 20 years, youngsters will dwell in a world the place they can not conceive what it was like being unable to easily hand purely digital 3D content material to a good friend or member of the family. Proper now in 2025, doing that’s such a rarity an early entry app from a single developer confirmed up the whole market by integrating options which ought to be customary, however aren’t but.
Jigsaw Night time will get a lot proper out of the field that it turns into a extra tangible piece of our world, creating recollections in locations VR headsets don’t normally go — at the least not but.
“I really feel like the main target of the immersive expertise dialog has just lately moved away from co-present collaboration earlier than it’s even actually really begun,” Lukas wrote me over direct message. “This business is all the time trying to tomorrow’s expertise to resolve mainstream adoption, however actually it’s all attainable proper right here for us now. I simply hope there are sufficient of us on the market which are nonetheless targeted on it, and am grateful that we now have a framework – and perhaps even a brand new medium – to have that dialog.”
