It's actually more complicated even than that. The notion of recreating in an audio medium games that are primarily visual is a tempting chimera that has, I feel, been one force behind the relative stagnation of the audio games world. The reasons for this make a lot of sense; people who grew up playing conventional video games are often more passionate than folk who never had that experience, which being so, makes the idea of recreating those games from the past seem more compelling than our actual experience shows that it is.
In the following discussion, I'm intentionally ignoring the problems of expense and size of development shops we've typically seen, as well as markets. These factors are relevant to the general problem of game development, but not to the discussion I'm having below. Yes, I know about them and understand their effects on what we've seen, but they obscure what is for me a more basic problem. Before I wander off on what is to me a more interesting tangent, let me just suggest that Castaways is probably a better or at least more comparable game than Revelation for Mine Craft. Revelation is more like the mobile game Alchemy. So why do I call the recreation of visual games in an audio form a chimera? Because we haven't yet cracked the problem of how to convey visual information in an audio context. Consider the following: When a sighted person processes visual information, they are processing shape, relative size, lighting, color, motion, perspective, relative position and many other data over a time scale that is on the order of tenths to hundreds of a second. All this processing is parallel, through many different information channels if you will. Any of these data may be relevant to figuring out a situation, route planning, tactics choice or strategic considerations. Now compare this with sound. First of all, sound processing is slower physiologically, allowing for less parallelism in how our brains deal with sound sources. Localization is less precise than a visual person gets from locating an object with her eyes. The problem of determining distance to objects is more complex; there are more visual cues to distinguish large objects far away vs. small objects close at hand. There is no audio equivalent of horizon, perspective or shading that provide the visual clues to perform this simple but necessary task. There are some analogs, pitch could be used to convey color fairly directly as, at least for simple primary hues, frequency could correlate directly and one could use volume to correlate with either saturation or brightness. (Saturation refers to the color's intensity, brightness measures how much light is reflected or produced for an energy input.) But how do you convey the color teal, which is a very particular hue, a mixture of blue and green? I suppose you could mix wave lengths from the "green" and "blue" part of the spectra, but the result would be a pitchless (though not white) noise that might be very difficult to distinguish from say the burnt orange hue that would be mixing a lot of yellow with a bit of red and darkening the brightness and lowering the saturation value. And all this assumes you have a meaningful concept of color to work with. I have had sufficient vision in the past to be able to visualize these colors. Does this discussion hold any meaning for someone who never saw them? (an honest question; I do not have that experience, so can't comment from my own life.) As a very simple example; I have spent the last year trying to wrap my brain around the idea of how best to convey in an audio format the information available to a sighted player of Angband, Moria, Nethack or any other roguelike in a meaningful way. I worked for a short while with one of the actual variant developers for these games, trying to design a system that would convey the information about position, relative positions of enemies, walls and floor features such as traps. Note that this doesn't even begin to touch the complexity available to a sighted game developer for a Call of Duty sort of game. I still haven't found a way to convey even this much reduced information load to a player in a way that doesn't take impractical amounts of time to play a game of any meaningful length. This doesn't mean it can't be done. It doesn't even mean that I won't solve this problem at some point. What it does mean is that I, a pretty smart cookie and good at algorithm development, as well as a pen-and-paper game designer who has created quite playable original designs and hacks of pre-existing work, have not yet solved this problem, indicating that it's not a trivial problem to solve. The solution probably involves information compression; first of all figuring out what information is actually necessary to the experience of playing a rogue-like; whether the physical exploration aspect that we saw in Entombed is actually important enough to justify its cost in information complexity, as opposed to the combat scenes that were the heart of that game. I haven't found a solution I would enjoy playing yet. Let's look at Swamp for another take on solving these problems, as well as showing the inherent limitations thereof. Aprone provided what was for us a big step forward in immersive gaming by freeing us from the keyboard for basic navigation/aiming tasks, and by making a real-time, multi-player experience that gave the illusion of simultaneous experience by multiple players of the same reality, that is until you looked a little closer beneath the gosh-wow factor. (And by the way, in the following, I'm sure that Aprone will realize that I'm not complaining; he has certainly read my praise often enough to know that I think he did something wonderful.) It was and is wonderful, especially compared to what had been available before that, but let's honestly compare it to the experience a sighted player has in a multi-player FPS. The sighted player would have richly detailed landscapes, where terrain has much more effect than the small effects on stealth that Swamp appears to provide. You would have line of sight effects that would be far more precise than Jeremy's use of sound processing to give the illusion of obscuration by terrain features. You'd have of course the detailed and likely rather graphic FX of the combat itself. The game would also move rather faster. These are just a few things I can think of off the top of my head. It seems to me that trying to directly recreate visual games in an audio format is a path that leads nowhere. Castaways extracts some of the core concepts of Minecraft, or, since if I recall correctly it precedes it, Castaways deal with some of the same issues. One could expand on that framework to produce a game that would play something like Minecraft; though I suspect that Aprone or another developer would have to start over, retaining only some of the concept, to accommodate a larger world and more direct MP interaction. Ambitious audio developers need to be focusing on creating tools that can enhance the audio experience, such as real-time signal processing to allow for more flexibility in the audio stream. We're approaching the time when we'll have sufficient computing power available to the general consumer to make this more possible than it would have been even five years ago. This would make audio gaming more immersive and give developers ways of manipulating the audio environment that would more closely approach the tools available to the visual developer. We also need to see some theoretical work done on how one might more closely approximate the inputs available to a visual perceiver using audio cues. This might involve moving away from representational sounds to more of a braille code approach; creating earcons to represent popular or common visual phenomena. Finally, it is perhaps time to reduce our dependence upon the trappings of the visual world in game conceptualization and development. If someone created a fighting game, where the main character is a highly trained, but blind kung fu master, than the use of audio design could be made to directly serve the game's story, rather than being an approximation of a story that would under normal circumstances be represented visually. This could provide the sort of immersive experience to all players, blind, impaired or fully sighted that has been absent from our market. All of a sudden, the reduction in the amount of information available becomes a feature that one has to learn to live with rather than an unfortunate side effect of physiology and culture. Chris Bartlett --- Gamers mailing list __ Gamers@audyssey.org If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org. You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/gamers@audyssey.org. If you have any questions or concerns regarding the management of the list, please send E-mail to gamers-ow...@audyssey.org.
