You'll notice there aren't photos here, and that's because before we were even allowed into the labs where the HoloLens team tests out its user experiences, we had to deposit our cameras and phones into a locker. No recording equipment of any kind was allowed, not even audio. We entered the basement below Microsoft's visitor center laughing at the absurdity of it all — many reporters needed to get notepads from the company and weren't carrying pens, either.
But it was all worth it, because HoloLens is probably the most intriguing (and, in many ways, most infuriating) technology we've experienced since the Oculus Rift. And there are many parallels with the Rift to be had: both are immersive, but in different ways; both require you to strap a weird thing on your head; both leave you grinning like at absolute idiot at a scene only you can see; and, crucially, both need more work when it comes to thinking through exactly how to control and interact with virtual things.
Minecraft IRL
By far, the most impressive demo for my money was the Minecraft demo — though Microsoft called it something like "Building Blocks" or some such, presumably so as not to fully commit to releasing a full holograph version of Minecraft. But before we could enter this virtual world — actually, the virtual entered our world — we had to strap on the development unit for the HoloLens.
It's a contraption, to be sure. There's a small, heavy block you hang around your neck which contains all the computing power. It's comprised of lenses and tiny projectors and motion sensors and speakers (or something that makes sound, anyway), and god knows what else. And then there's a screen right there in your field of view.
But before you can apply your jaded "I've done VR before" attitude to this situation, you look down at the coffee table and there's a castle sitting right on the damn thing. It's not shimmery, but it's not quite real, either. It's just sitting there, perfectly flat on the table, reacting in space to your head movements. It's nearly as lifelike as the actual table, and there's no lag at all. The castle is there. It's simply magic.
You definitely have a big stupid grin on your face even though the contraption that's strapped to it is pressing your eyeglasses into the bridge of your nose in a painful way.
Then it's demo time. You can't touch anything, but you can look and point a little circle at objects on it by moving your head around. You learn how a "glance" is just you looking at things and pointing your reticle at them, and an "AirTap" is the equivalent of clicking your mouse. The demo involves digging Minecraft holes and blowing up Minecraft zombies with Minecraft TNT. It's basically incredible to see these digital things in real space.
You blow up a hole in the table and then you look through it to more digital objects on the floor. You blow up a hole in the wall, and tiny bats fly out. You see that behind your very normal wall is a virtual hellscape of lava and rock. You peer into the hole, around the corner, and see that dark realm extend far into space.
And then the demo's over.
Skype
Microsoft’s Skype demo was as equally impressive to me as playing around withMinecraft blocks in a living room. After a two-hour keynote, Microsoft wanted me to fix a light switch. It all started by sitting down and facing some tools and a socket with exposed wiring. A little dazed and confused, I looked up and scanned across the Skype interface which was suddenly appearing in front of me, and picked a face to call. The video call popped into a little window, and my journey to fix a light switch began.
On the other end of the call was a Microsoft engineer. I could see and hear her, but she could only hear me and see exactly what I was seeing in front of me. My eyes, or the headset on my head, was relaying everything over Skype. It was a support call of sorts — here she was to help me fix a light switch. We started by pinning her little window on top of a lamp. I could then look around the room and return to the lamp to see her face. She guided me where to go. It felt strangely natural, and I didn’t need to configure anything or learn gestures other than the same "Air Tap" you use to simulate a mouse click.
Holo Studio
Microsoft's next demo didn't have us using the HoloLens prototypes directly. Instead, we watched as "Nick" (nobody in Microsoft's blue-tinted demonstration basement has last names. I asked.) manipulate objects in digital space so he could build a Koala bear or a pickup truck. It was actually quite impressive, as cameras filmed him and screens showed both Alex and the virtual objects he was manipulating in the same space in real time.
The idea was to convince us that HoloLens would unleash a wave of creators who would be able to dream up 3D objects with little to no training. It's much easier to understand what a thing is in your living room than it is in AutoCad.
- Glance: you point your head at something.
- AirTap: you make a "Number 1" sign with your hand, then move your finger down like you're depressing a lever.
- Voice: you can issue commands, usually to switch what "tool" you're using.
- Mouse: So actually the neatest thing is that objects you use to interact with computers can be used to interact with holograms.
At one point in the demo, Alex needed to put a tire on his pickup. He had to twist his body and head around to get his pointer in just the right spot and get the tire arranged just right to fix on the axle. Then, AirTap! the tire is connected. But how much easier would it be if you could grab the tire in your actual hands?
HOW MUCH EASIER WOULD IT BE IF YOU COULD GRAB THE TIRE OR THE TRUCK IN YOUR ACTUAL HANDS?
But then Microsoft gave us 3D printed Koalas with a USB drive inside them, which was nice. And if this HoloLens thing takes off, you will be able to design your own and it will be way easier than learning current 3D design software. But not as easy as it would be if you just imagined building with holograms.
Walking on Mars
Microsoft has teamed up with NASA to let scientists explore what Curiosity sees on Mars. Instead of panoramic imagery on a computer screen, Microsoft’s demo lit up a room and turned it into Mars. I walked around the rocky terrain, bumped into the Curiosity rover, and generally just checked out a planet I will never visit in my lifetime. It’s a totally new perspective that felt like I was immersed in touring Mars, but not necessarily there. The field of view felt a little too limited to truly immerse myself and trick my brain into thinking I was really on another planet, but what impressed me most is what Microsoft has built into this experience.
I held a call with a NASA engineer, and he talked me through the terrain. I squatted to look more closely at rocks, took snapshots of various rock formations, and even planted flags for points of interest. My jaw dropped when I ventured over to a PC in the room and started to experiment with the mouse. I pulled the mouse pointer off the screen and suddenly it was on the floor next to me, allowing me to set markers in the virtual environment. It’s everything I’ve seen in demonstrations from Microsoft Research before, but here it was on my head and working.The collaboration part was the key here, allowing me to interact with this data in a unique way, but also alongside the NASA engineer who could drop flags on the Mars terrain and guide me to look at certain sections. While this isn’t traditional productivity with a mouse and keyboard, it’s certainly something new and intriguing. I could see this type of scenario working for big teams that need to communicate across time zones and on big sets of complex data.
Overall, HoloLens is Microsoft at its most ambitious. It’s a big bet on the future of computing, the future of Windows, and ultimately the future of Microsoft itself. While the company is struggling at mobile, it wants to catch the next wave of computing and lead. Is HoloLens the next wave? Developers and consumers will be the ultimate test of that, but if anything HoloLens is an incredibly brave and impressive project from Microsoft. It’s true innovation, which is something Microsoft has lacked during its obsession with protecting Windows. It’s also another example of an experience that takes the complex technology out of the way, leaving you to experience what really matters.