dessalles

A couple months ago I read a story in the NYTimes about a device called the Boomerang developed by BBN Technologies. The Boomerang, which looks a bit like an over-sized cat toy, is attached to military vehicles to determine the direction of incoming enemy fire. It does this by recording the sound of the bullets with multiple microphones and then triangulating their source and trajectory based on the differences between the recordings - a process called acoustic triangulation.

In my post on Recording Reality, I mentioned two basic approaches to compiling a single recording of everything: things observe other things and things observe themselves. What’s fascinating about the Boomerang is it’s a hybrid of these two approaches - a sort of machine sousveillance. That is, by observing it’s environment (other things), it “observes” itself. Since it can track the trajectory of incoming bullets, it can effectively determine when the vehicle’s been hit by one. The Boomerang is a sensory prosthetic for Humvees.

But beyond the hybrid nature of what it observes is the hybrid nature of how it observes. The Boomerang records sound; loosely put, it produces a “photographic” recording. But as I said, things that “observe” themselves typically do so descriptively, using language. And the Boomerang uses the sound recording to produce a descriptive recording - a mathematical description of bullets relative to its own location, bullet vectors.

This all reminds me of independent, Japanese, game developer Kenta Chao, who needed a way to keep track of bullets in his video games. Sounds like a fairly trivial task until you learn that Chao designs “Bullet Hell” games, which typically involve hundreds, if not thousands, of bullets flying on the screen at once. And when the primary task a video game is dodging screen-fulls of bullets, one can imagine a major technical challenge is managing them all - hence BulletML.

I really like the idea that the military has something to learn from video games. BulletML is nothing new. Online multiplayer games typically share descriptions of entire battlefields in real-time. Halo 3, for example, records the location, trajectory and speed of every bullet, soldier, vehicle and explosion; and synchronizes the information on the consoles of up to 16 players across the world. The Boomerang just shares some directional information with the Humvee drivers. What happens if multiple Boomerangs are networked to share information and map the location of all enemy positions? Or what if the data generated by a Boomerang is fed into the head-up display of each soldier? From a user interface perspective, you end up with something pretty much identical to what gamers have been playing with for years. But we knew this already.

[Image: Slayer on Construct, 10/19/07 by Darkside1029]

So what’s this all about? I don’t mean to dwell on warfare at all. What I do want to think about is how the video game industry seems a step ahead when it comes to recording and distributing incredibly detailed descriptions of environments. Because beyond just overcoming the technical challenges of trying to record everything is the question of what on earth we’re going to do with the recording once we have it. As an example, Halo 3 archives the data from every single multiplayer match. In Theater Mode, a feature included to encourage machinima, players can playback the gameplay and relive every second of a match from any angle in the environment. Theater Mode includes the ability to take “photographs” of gameplay; and the captured images are automatically uploaded to Bungie’s community website and shared with other users. As a result, a hobby of Halo 3 photography has emerged. And more times than not, they’re photographing themselves. Gamers are recording recordings of virtual reality.