Dawn of the New Everything Read online

  Begin Reading

  Table of Contents

  About the Author

  Copyright Page

  Thank you for buying this

  Henry Holt and Company ebook.

  To receive special offers, bonus content,

  and info on new releases and other great reads,

  sign up for our newsletters.

  Or visit us online at

  us.macmillan.com/newslettersignup

  For email updates on the author, click here.

  The author and publisher have provided this e-book to you for your personal use only. You may not make this e-book publicly available in any way. Copyright infringement is against the law. If you believe the copy of this e-book you are reading infringes on the author’s copyright, please notify the publisher at: us.macmillanusa.com/piracy.

  To everyone mentioned in this book and the many more I wish I’d been able to mention: thank you for giving me my life.

  Acknowledgments

  Some passages of the present work are adapted from my contributions to John Brockman’s edge.org or my anthologies of science writing. Other passages are adapted from my works originally published in the Whole Earth Review or the New York Times.

  My wife, Lena, not only supported and put up with me while I wrote this book, but I am astonished at the strength and brilliance she displayed during a period when she was battling cancer. Thank you!

  Thanks to Maureen Dowd for correspondence that inspired some passages.

  Thanks to Satya Nadella, Peter Lee, Harry Shum, and everyone else at Microsoft Research for their camaraderie and support. Not a word here represents a Microsoft point of view, of course.

  Thanks to Mary and Steve Swig for the writing cottage at The Shadows.

  Thanks to my editors, Gillian Blake in the USA and Will Hammond in the UK, for their always graceful interventions, and especially for their patience during a year of delays and difficult circumstances; and to my agents, especially Jay Mandel. Thanks to Eleanor Embry at Holt for her tireless attention to manuscript details.

  Thanks for reading early drafts: Michael Angiulo, Tom Annau, Jeremy Bailenson, Steven Barclay, Maureen Dowd, George Dyson, Dave Eggers, Mar Gonzales Franco, Edward Frenkel, Alex Gibney, Ken Goldberg, Joseph Gordon-Levitt, Lena Lanier, Matthew McCauley, Chris Milk, Jane Rosenthal, Lee Smolin, Mary Swig, and Glen Weyl.

  Preface: Virtual Reality’s Moment

  It was the late 1980s, and a large envelope with a formidable DO NOT X-RAY sticker had just been dropped through the slot of the front door of a tech startup in Redwood City, California. The envelope contained a floppy disk that held the first digital model of a whole city. We had been waiting all morning. “Jaron, it’s here, get to the lab!” One of the engineers rushed to snatch the envelope before anyone else could get at it, sliced it open, ran to the lab, and slid the disk into a slot in a computer.

  It was time for me to enter a brand-new virtual world.

  I squinted up at my hand against a perfectly clear blue sky. My gargantuan hand, soaring above downtown Seattle. It might have been a thousand feet from wrist to fingertip.

  There was a bug, obviously. A hand should be about the right size to pick up an apple or a baseball, not bigger than a skyscraper. You shouldn’t have to measure a hand in feet, much less thousands of them.

  The city was abstract. This was in the early days of VR, so plasticine blocks stood in for most buildings, in a jumble of too-cheerful-for-Seattle colors. The fog was preternaturally uniform and milky.1

  My first thought was to stop and fix the bug, but instead I took a moment to experiment. I flew down and tried to nudge a ferry on the sparkling Puget Sound. It worked! I had control. Not what I expected. That meant that I could still inhabit my hand when it was preposterously huge.

  Once in a while, a bug in VR exposes a fresh way that people can connect to the world and each other. Those are the best moments. I always stop and linger when it happens, to hang on to the sensation.

  After a few experiences of VR bugs, you have to ask yourself, “Who is it who is suspended in nothing, experiencing these events?” It is you, but not exactly. What is left of you when you can change virtually everything about your body and the world?

  A bundle of cables connected my EyePhone, through a loop hanging from the ceiling, to a line of refrigerator-size computers that roared to keep cool. I wore a DataGlove on my hand; slick black mesh woven through with fiber optic sensors, and yet more thick cables from the wrist arcing up to ceiling rings. Blinking lights, flickering screens. The EyePhone’s rubber rings left moist red indentations around my eyes.

  I wondered at the strangeness of the world I found myself in, now back in the lab. Buildings in Silicon Valley used to have carpeted walls and cheap Space Age desks with fake wood grain. A faint smell of aluminum and dirty water.

  A gang of eccentric technical geniuses converged, impatient to try. Chuck in his wheelchair; a robust, bearded lumberjack. Tom acting all professional and analytical, even though only a few minutes earlier he’d been telling me about his crazy adventures exploring San Francisco overnight. Ann seemed to be wondering why she was yet again cast in the role of the only adult in the room.

  “Did it feel like being in Seattle?”

  “Kind of,” I said. “It’s, it’s … marvelous.” Everyone shoved toward the gear. Every little iteration of our project got better. “There’s a bug. The avatar hand is huge—by magnitudes.”

  I never got tired of the simple act of using my hand inside VR. When you could bring your body in there, you were not just an observer, but a native. But every tiny detail of functionality, of figuring out how a virtual hand could hold virtual things, turned out to be a struggle.

  Fix a problem with how virtual fingertips mistakenly penetrate objects they are trying to pick up, and you might accidentally make the hand gargantuan. Everything connects with everything. Every tweak of the rules of a new world is a potential setting for a startling, surrealistic bug.

  The author as he appeared in the late 1980s outside and inside VR.

  Bugs were the dreams within virtual reality. They transformed you.

  A moment with a giant hand changed not only how virtual reality felt to me, but how physical reality felt. My friends in the room now looked like pulsing beings, translucent. Their transparent eyes were filled with meaning. This was not hallucination, but improved perception.

  Physicality revealed in fresh light.

  Introduction

  What Is It?

  VR is those big headsets that make people look ridiculous from the outside; those who wear them radiate startled delight at what they’re experiencing from the inside. It’s one of the dominant clichés of science fiction. It’s where war veterans overcome PTSD. The very thought of VR is the fuel for millions of late night reveries about consciousness and reality. It’s one of the only ways, for the moment, to raise billions of dollars fleetly in Silicon Valley without necessarily promising to spy on everybody.

  VR is one of the scientific, philosophical, and technological frontiers of our era. It is a means for creating comprehensive illusions that you’re in a different place, perhaps a fantastical, alien environment, perhaps with a body that is far from human. And yet it’s also the farthest-reaching apparatus for researching what a human being is in the terms of cognition and perception.

  Never has a medium been so potent for beauty and so vulnerable to creepiness. Virtual reality will test us. It will amplify our character more than other media ever have.

  Virtual reality is all these things and more.

  My friends and I founded the first VR startup, VPL Research, Inc., in 1984. This book tells our story, and explores what VR might mean to the
human future.

  The first virtual reality system, according to the original definition, in which multiple people cohabited a virtual world at the same time. This was VPL’s RB2, or “Reality Built for Two.” In the screens behind each person, you can see how they see each other as avatars. This photo is from a trade show in the late 1980s.

  Recent VR enthusiasts might exclaim, “1984, no way!” But it’s true.

  You might have heard that VR failed for decades, but that was true only for the attempts to bring out a low-cost, blockbuster popular entertainment version. Just about every vehicle you’ve occupied in the last two decades, whether it rolls, floats, or flies, was prototyped in VR. VR for surgical training has become so widespread that concerns have been expressed that it’s overused. (No one would suggest that it shouldn’t be used at all; it’s been a success!)

  What Can a Book Do That VR Can’t, at Least as Yet?

  The romantic ideal of virtual reality thrives as ever. VR the ideal, as opposed to the real, technology weds the nerdy thing with the hippie mystic thing; it’s high-tech and like a dream or an elixir of unbounded experience all at the same time.

  I wish I could fully convey what it was like in the early days. There was a feeling of opening up a new plane of experience. Inhabiting the first immersive avatars, seeing others as avatars, experiencing one’s body for the first time as a nonrealistic avatar; these things transfixed us. Everything else in the tech world was dull in comparison.

  I cannot use VR to share what that experience was like with you, at least not yet. VR, for all it can do, is not yet a medium of internal states. There is less and less need for me to make this point as VR becomes more familiar, but it’s a clarification that I have been called upon to give many times.

  There’s occasional talk about VR as if it is on the verge of evolving into telepathic conjuring of arbitrary reality along with a conjoining of brains. It can be difficult to explain that VR is wonderful for what it is, precisely because it isn’t really everything.

  Eventually a new culture, a massive tradition of clichés and tricks of the VR trade, might arise, and that culture might allow me to convey to you how early VR felt, using VR-borne technique. I have spent many hours daydreaming about what a mature culture of expression would be like in VR. A cross between cinema, jazz, and programming, I used to say.

  * * *

  First VR Definition: A twenty-first-century art form that will weave together the three great twentieth-century arts: cinema, jazz, and programming.1

  * * *

  Even though no one knows how expressive VR might eventually become, there is always that little core of thrill in the idea of VR. Arbitrary experience, shared with other people, conversationally, under our control. An approach to a holistic form of expression. Shared lucid dreaming. A way out of the dull persistence of physicality. This thing we seek, it’s a way of being that isn’t tied just to our given circumstances in this world.

  If I tried to tell the story of VR dispassionately, I’d be lying. What makes VR worthwhile to me is that it’s about people. I can only tell you what VR means to me by telling my story.

  How to Read This Book

  Most of the chapters tell a story that begins in the midsixties, when I was a boy, and ends in 1992, when I left VPL.

  There are also chapters interspersed throughout that explain or comment on aspects of VR, such as a chapter on VR headsets. These “about” chapters include a dusting of basic introductory material, a hearty portion of sharp opinions, and more than a few out-of-sequence anecdotes. You have my permission to skip through them if you prefer storytelling to science or commentary. Or, if you don’t like storytelling and just want to read my thoughts on VR tech, then race right to those chapters.

  Some of my stories and observations are found in long footnotes. I bet you’ll be glad if you find the time to read them, but you can leave that for later. There are also three appendices that expand on my ideas from the period, but are ultimately more concerned with the future than the past. Read them if you want to know what it feels like to have an informed worldview that doesn’t include AI destroying humanity any minute.

  In keeping with the time period of the narrative, I’ll talk more about classical VR than mixed reality,2 even though that’s what I’ve worked on more lately. (Mixed reality means the real world is not hidden entirely by the virtual one; you see virtual stuff placed within the real world, as experienced lately in a HoloLens.)

  Meeting My Younger Self

  Never thought I’d see you again.

  What I always feared. You get old, then you milk your younger self. Like all the other writers.

  You are so wrong. It would be easier not to deal with you. I’ve been feeling more comfortable with myself than ever before. Dealing with you brings up crummy old patterns. I get insecure and depressed. You’re recidivism bait. I’m only doing this because I think it would be useful for other people to know about you.

  What’s going on with virtual reality? Is it even called VR?

  Yeah, most people call it VR now.

  You mean we won the terminology war?

  No one remembers or cares about that war. It’s just words.

  But is VR any good?

  Well, we’re about to find out. It looks like this book might come out at about the same time that VR gets commonplace.

  Oh crap, I hope they don’t screw it up.

  Yeah, who knows … You know how hard it is to do VR well.

  I hope VR isn’t still so—what’s the word?—pressured by all the psychedelic people.

  Oh, you’d miss them. You won’t believe it, but singularity freaks cross-bred with libertarians, and their fanatical offspring are the main drivers of tech culture these days.

  Wow, that sucks—worse than I imagined.

  I feel embarrassed that you were expecting a perfect world.

  I’m embarrassed that you think you’re noble or enlightened just because you learned to accept living with bullshit.

  Oh, c’mon, let’s not fight. There are plenty of people out there to fight with.

  Okay, so tell me about this cheap VR you say is shipping. Are people making up their own VR worlds?

  Well, usually not while they’re inside, but yeah, a lot of people will probably be able to make worlds.

  But if you can’t improvise the world from inside, what’s the point? Just more phenomena to clog the senses, and not even as good as in the natural world. Why does anyone care? You’ve got to do something to stop it before they bring out crap. What’s wrong with you?

  Hey man, I’m not the VR police. I don’t run the show.

  Why not? You were supposed to run the show!

  It’s actually great to watch the kids reinvent VR. There are all these cute VR startups and teams in the big companies. Some of them even remind me of you and VPL, though the fashion these days is a lot straighter.

  I’m insulted that you’d say someone reminds you of me if that person just thinks of VR as a spectacle. Don’t they know that’ll turn into a cliché pretty fast? What happened to the dream of improvising reality? Shared lucid dreaming? I mean, what’s the point of just making a flashier type of movie or video game?

  Look, you can’t devote yourself to serving people if you think you’re better than them. VR will be kind of crummy but also kind of great and it will evolve and hopefully get really great. You have to relax about it. Enjoy the process. Respect the people.

  What a load of crap. Are you at least screaming your head off about it?

  Well, yeah, I guess … this book …

  Okay, so who’s bringing out cheap VR? VPL?

  No, VPL is long gone. Microsoft brought out a self-contained mixed reality headset—doesn’t need a base station—goes anywhere. You’d be really impressed.

  Microsoft? Oh no …

  Um, my research post lately is in Microsoft’s labs.

  Are you institutionalized? Oh wait, you just said you are.

  Give it a re
st. Classic VR gear is also shipping; not unlike what we used to sell. One of the social media companies bought this little company called Oculus for two billion dollars.

  Waaaaiiit whaaaat? Two billion for a VR company that hadn’t shipped yet? Wow, the future sounds like paradise. And what’s a social media company?

  Oh, that’s a corporation people use to communicate with each other and keep personal remembrances, and there are algorithms that model the people so offers can be targeted; these companies can make people sadder or more likely to vote by tweaking the algorithms. They’re the center of a lot of people’s lives.

  But, but, combining that with VR would be like a Philip K. Dick novel. Oh my, the future sounds like hell.

  It’s both paradise and hell.

  But bright, rebellious young people wouldn’t want to be running their lives through a corporation’s computer …

  Weirdly, the new generation gap is—supposedly—that young people are more comfortable with corporations running digital society.

  You say that like it’s just another fact you can live with. I mean, wouldn’t they become like serfs? Do they just live with their parents more, or what? The world’s gone mad. Everything’s inverted.

  But that’s normal for the world. It’s what happens with time.

  I feel like I need to slap you.

  Maybe you do.

  1. 1960s: Terrors of Eden

  Border

  My parents fled the big city right after I was born. They roamed for a while; eventually alighting in what was, at the time, an obscure and harsh place. The westernmost corner of Texas, outside El Paso, at the juncture of New Mexico and Mexico proper, was an outback, barely part of America. It was impoverished, relatively lawless, and of unsurpassed irrelevance to the rest of the country.