Sunday, June 7, 2009

The Possibilities are Endless

OK, so yesterday I just kind of gushed over Microsoft's Wii-killer, Project Natal, which was unveiled at E3. Those who don't care much about video games might not be quite so impressed, and that's perfectly understandable. I want to take a few minutes to point out some things which I think that anyone could get excited about. I will be using video games and other tech-toys as take-off points, but as platforms upon which to build legitimately useful stuff.

Forget about Wii Fit - if Natal can recognize a drawing in your hand and recreate it in the game world, just use a play sword and you could have the most realistic swordfighting game ever. Or you could learn martial arts. One of the demos showed someone blocking balls like some kind of super-goalie - why not be the outfielder and catch the baseball (if you're into that)? With the voice recognition, you could create a magic system that relied on key words and rhyme schemes - with the biometrics, it could rely on hand gestures from the simple to the ornate. Or, if you're more practically-minded, you could learn sign-language. Or a foreign language (though these games already exist, I think that Natal would be a significant technological improvement to the general mechanism).

But this is child's play - peripherals are where it's at. I have a Wacom tablet, years old, which has a mouse and a pen. These are not powered peripherals, though: the tablet itself monitors whichever peripheral is being used at the time, and reacts to changes that I make. The pen and mouse themselves are mere props - the tablet is what does the work. Peripherals could, with very similar technology, be modular, instructive, and cheap! Learn anatomy and physiology from the comfort of your own home from a mannequin complete with internal organs. Learn how to operate complex machinery such as cranes or airplanes with control consoles. Hell, musical instrument controllers wouldn't have to be dumbed down any more - just get an accurate replica made from cheap plastic, follow the on-screen directions and the game will monitor your progress, and you could actually learn to play the guitar, or the flute, or the theremin, just like the current batch of games teach you how to actually sing and play drums (well, electronic drums, but you still can't dumb down the drums that much). Even better, the game could teach you scales, theory, anything you wanted to know about the instrument (not just entire songs, but really how to play the instrument). Once you've learned the song with the peripheral, get the real insturment and you can get silent on-screen instructions. If your friends do the same, you could form an actual rock band and practice like this.

Go shopping for clothes online and know that a size will fit you. It just needs to be measured at the other end, and the software can measure you if you just hold up an objective reference. You could see how entire outfits look together, and only lack the knowledge of how they would feel on your skin. Videoconferencing is a gimme, and looks to be easier and more intuitive than ever. What about graphic design? Or mechanical engineering? I could see this being very much like Tony Stark's computer in the Iron Man movie. The possibilities really are endless.

I'm going to take a moment to get back to the video-gamey stuff, but I promise that it will come back to real-world applications. In the first place, Milo's debut to the world - scripted though it might have been to a large degree - is a technological marvel nonetheless and represents another step towards genuine (Turing-complete) AI. Just take what we actually saw in the demo - the appearance of a natural conversation between a human being and a representation of a kid on a screen - and now think about procedurally generated content. The tools are all there: Milo's speech sounds easy and natural, the architecture for facial expressions is fairly robust, the voice recognition is superb and could be reversed (with phonetic- rather than text-based speech algorithms) to create on-the-fly speech complete with tone of voice and realistic facial expressions to match. The way Milo moves around the world model looks natural as well, and even if that was also pre-scripted, Left 4 Dead has some fairly spectacular animations for their garden-variety zombies running around under all sorts of conditions, and it all looks fantastic.

Left 4 Dead is actually the poster-child for procedurally generated content. I mean, Spore was amazing in terms of making a game of infinite content from a finite ruleset, but Left 4 Dead takes it several steps further: item pick-up locations, enemy spawn locations and times, zombie swarm sizes, player-character in-game dialogue, and each player's own musical score are all generated on-the-fly by in-game "directors." The AI director monitors player's stress levels in terms of how much damage they have taken over time, how many zombies they have killed, how often they've used healing items, and how much those things have changed over time, and uses this data to create an environment of continual tension: you're supposed to experience brief moments of rest between increasingly frantic periods of fighting for your life, barely making it to the next checkpoint. The music director also monitors this stuff for each individual player to create an improvised score that reflects what's been happening to you lately. Player-character dialogue also depends on the historical details of that specific play-through of that particular level (who has healed whom, who has protected whom, who has accidentally shot whom, what type of enemies you've been fighting, how long you've been in the same area, and so on and so forth), and each line has a "threshold" sort of value such that it won't be said more than once during an arbitrary interval of time. All of this is geared towards creating an experience with broad accessibility and infinite replayability, and they've done it: you could play the game every day for your whole life and never experience a level exactly the same way twice.

Take this kind of parallel improvisation from a broad set of elements, but make it geared toward personal interaction rather than simulating a zombie apocalypse. Now add an algorithm for learning new words - e.g. when an unknown word is encountered, ask, "What does that mean," then write it to long-term memory and practice using it - and I think you've got a recipe for a Turing-complete learning computer. Like any real person, Milo would need positive reinforcement when he did well, and correction when he erred - like any real person, the learning process would be dialectic. I could teach Milo the rules of D&D and have him help me level my character. I could teach Milo the rules of calculus and have him help me with my physics homework. I could teach Milo the rules of logic and bounce my philosophical ideas off of him. I could teach Milo about options trading and ask him for help optimizing my investment strategies. I could teach Milo about his own world and have him help me create mods for video games - or entire video games from scratch.

Milo could access the internet and be my research assistant. My secretary. My second player. He could use AIM, Skype, or Google Talk through my computer and text or talk to me on my cell phone while I'm away. At this point, I don't see what separates Milo from my real friends any more, aside from the fact that he can't give me a hug. This is a very significant difference, but it's the only kind of difference I can see. Yet how is this fundamentally different from the state of my relationship with my cousin in California, or my grandfather in Arizona?

It has long been a dream of mine to teach philosophy to an AI. I don't think that a fully-functional AI, as most people would think of it, could be truly robust right out of the box - I'm of the opinion that a learning algorithm constitutes AI (I think "intelligence" is "the capacity for learning"), but to pass the Turing test, it would need to be taught, just like a newborn child needs to be taught and isn't a fully fleshed-out person "right out of the box." Milo has broad consumer appeal for being a new thing presented in a familiar way, and to an extent, the ability

OK, time to get back down to Earth: this is all clearly over-optimistic and way far off, and though these obstacles could be overcome in principle, they still need to be overcome in practice before I should really be getting excited over any of it. But still - the future is looking amazing from where I stand. I can't wait to get there!

No comments: