I walked on Mars Wednesday.
I could look down, and see its rust-colored soil beneath my feet. If I turned around, I could see rocks and soil stretching to the horizon. If I physically walked forward, I could see new objects, objects that I could explore if I knelt down.
And that was after I blew a hole in a wall and exposed a virtual Minecraft cavern. And then wired up a live light switch, helped by an engineer who drew glowing arrows in the air to highlight a particular tool.
Meet HoloLens, part of what Microsoft called Microsoft Holographic during its Wednesday morning introduction of the Windows 10 consumer preview. It’s part of a project by Microsoft Research and Alex Kipman—the man who is credited with helping design the Microsoft Kinect—to rethink how users interact with machines.
“It’s the virtual world blended with the real world,” Kipman said during the keynote. He’s exactly right. And we got to try it out.
HoloLens was conceived in the bowels of the Microsoft visitor center, and that’s what a collection of reporters trooped downstairs to see. Unfortunately, Microsoft banned reporters from bringing audio or video recording equipment to document our adventure, so you’ll just have to depend on Microsoft’s concept shots and your own imagination.
This actually looks pretty close to what it’s like using HoloLens, but with a narrower field of augmented vision.
This isn’t Microsoft’s version of Google Glass
The HoloLens prototype that Microsoft demonstrated onstage bears little resemblance to the development hardware down below. I was able to examine the keynote prototype closely: With an eyeshade surrounding it, it looks like something the Daft Punk duo might wear. Inside, there’s a pair of lenses upon which images are projected, with a padded headband to secure it. Although Google’s Glass uses an overlay to project information about the world, HoloLens feels far more interactive—it’s a truer version of augmented reality.
The development version of HoloLens is much, much rougher. It’s a two-piece operation, with a special “holographic processor” (HPU) mounted in what’s essentially a large NUC, strapped to your chest. The goggles themselves are part of something that looks like a skullcap and is tightened to your head with straps. This hardware is far more about function than form. You’d probably call it ugly.
Image: Microsoft In this demo image, Microsoft’s HoloLens headset will let you view virtual 3D images within your dull, everyday real world.
What you “see” is a smart version of augmented reality, confined to a rectangle about the size of a large phablet, about six inches in front of your face. Within that the magic happens: Virtual images spring to life. But the HoloLens “knows” where they are, so that you can walk around them, kneel, or stand over them, and they remain in place. It’s uncanny. And a Kinect-like depth camera mounted in the HoloLens can “scan” the objects around you, too. More on that later.
There are several ways way of interacting with what you see. “Clicks” are triggered by holding your hand out a short distance from your face, and raising and lowering your index finger in a gesture. It then “clicks” where you’re looking, using a cursor that follows your gaze. HoloLens worked perfectly for Microsoft employees used to the gesture, but I wasn’t as proficient. You can also issue commands to the HoloLens, which can double as macros.
The best, first: A walk upon Mars
I saw the fruits of Microsoft’s partnership with NASA last, but it’s worth talking about first.
After the HoloLens was strapped to my head, I took a long look about Mars. Microsoft and NASA took composite images from the Martian landers and created a 3D mesh, so that you can literally walk around the surface of Mars. The idea, Microsoft executives explained, was that NASA scientists could use the map to plot out the next path for the rover, and “mark” objects that merited further exploration. Clicking a waypoint marked it with a little virtual flag. And you could even kneel and peer “under” rocks
Microsoft introduced another colleague into the simulation via a golden avatar that, in the demonstration, you could work with collaboratively, speaking to him via the HoloLens itself.
Sure, cynics may pooh-pooh it as just another version of Google Maps on Mars, with a little VR worked in. Nonsense. Any kid at heart—4 or 40—would love this.
Skype is still a 2D window, but it can be pinned in virtual space.
You want me to rewire a light switch? With Skype?
Yes, you can collaborate with Skype, but the extent of it is usually to share a screen, chat, or talk in real time. Microsoft asked me to rewire an actual light switch.
After the HoloLens was attached, I dialed “Alice” using an (unfortunately) 2D version of Skype that floated in my vision. Alice appeared, and explained my mission: Using some physical tools and her instructions, I would install a new light switch. Her version of Skype allowed her to see what I saw, and draw arrows and diagrams to help me. And to make it easier, I could “pin” her Skype window out of my field of vision. I did as I was told, an engineer turned on the power, and voila! I was a handyman.
About the only real disappointment I felt was that Alice was just a 2D video window. I was hoping she might be a 3D avatar that would hover at my elbow.
Holo Builder: So you want to play VR Minecraft?
For all the awe I felt at the Mars simulation, the Holo Builder virtual-world demo was undoubtedly the most fun. Because the HoloLens can scan the objects around you, Microsoft could turn a small room into a three-dimensional Minecraft level, complete with lava, caves, and zombies. In one task, I was asked to light a few barrels of TNT, blowing a virtual hole into a bench and plunging the zombies into a lava-filled pit as I peered down from above. A castle perched on a coffee table. And then yes, I blew a virtual hole through a physical hole with more dynamite and exposed another cave. I only wish I could have flicked a sheep off the side of the table and sent it bleating to its doom.
A surprisingly polished Holo Studio
You’re probably familiar with Project Spark, Microsoft’s collaborative venture that aims to turn gamers into game creators. I’m not really sold on the concept of Spark, but the similarly polished Holo Studio looks fun.
Mark Hachman A 3D-printed Holo Studio koala with jetpack that Microsoft turned into a USB drive.
Holo Studio posits that augmented reality is the gateway into 3D printing , and provides a collection of virtual objects to stick together, to form a koala or a truck or an X-Wing fighter. The tools were robust and showed a surprisingly cohesive design aesthetic—this looked like a product, not a demo. We didn’t have a chance to play with it, but Microsoft demo personnel created 3D objects on the fly, painting them and “magnetizing” them together, even using oral “copy” commands to reproduce several steps again and again.
Not for dorks
I asked one of our stylishly dressed guides: Would HoloLens be something she’d be seen in? No, another chimed in, because this wasn’t designed to be worn outside, but inside, “for work and play.” That’s a smart answer, as it avoids the whole “Glasshole” insult and makes HoloLens a product to be shared among friends and colleagues.
When will we see HoloLens? It will be out in the Windows 10 timeframe, according to Microsoft—whenever that is. (And it doesn’t necessarily mean at launch, either.)
The cynic in me says that HoloLens becomes the Kinect of the smartphone: an amazing technology that ends up with little real-world impact. I just slapped that cynic. Let’s see what you have in store for us, Microsoft.