The Future Of Creating Accessible Games by Thomas Ward
April 27, 2008 Earlier this year, in February, I started a special developers list for new and old accessible game developers. A long with the mailing list I have been striving to put together a definitive wiki which contains articles, programming tutorials, and other resource materials for game developers. Not so surprisingly the number one question I am asked is what programming language is best for accessible games? Most of the time I offer up a generic answer like there is this programming language, that programming language, and here is the pros and cons of each. However, I wanted to know the answer to that question myself, and so I began an independent research study on what are the current best programming languages, sdks, and platforms for accessible games. After a good amount of research into such languages as C++, C#, Visual Basic, Java, Python, etc I am ready to share with you my results. In addition to what programming language to use I have been looking into the feasibility of cross platform games and cross platform sdks. Sad to say my results in this matter aren't all that encouraging for any accessible game developer. Truth is that most commercial game companies don't make Native Linux and native Mac OS games, and there are perfectly good reasons why they don't. The same reasons that apply to the mainstream market apply to us in the accessible games market. First, it might be important to state what may or may not be immediately obvious. At a programming level games are completely different from other software applications. The reason is based on the fact that the average game requires a tighter relationship between hardware and software than usual. That is why mainstream PC games keep requiring higher and higher specifications to run. The more advanced the game, the tighter the hardware relationship, the higher the specifications to run it. Here is a very simple example of what I am talking about. In your average everyday Windows application such as Word, notepad, Windows Mail, Internet Explorer, etc there is a very generic way that keyboard control is handled. Normally when you press a key or keys on your keyboard a standard Windows key down event is generated at the top of the Windows event cue, and over time it cycles down through various processes until an application like notepad catches the event and does something with it or discards it. Either way it is a rather generic way to handle keyboard input. Often times the process is handled on first come first served basis, and isn't made for rapid keyboard input. In 1995 Microsoft recognized the standard Windows Event mottle was impractical for Windows PC games so they invented DirectInput. What makes DirectInput special is that unlike with the generic Windows Event mottle DirectInput gives the game developer direct access and control of the user's keyboard. Instead of having to wait for a keyboard event to be generated and passed down layer by layer through the Windows Event cue the state of the keyboard is received almost instantly. IN addition where the standard Window Event cue can only handle a limited number of keys at a time DirectInput allows for the status of the entire keyboard to be polled at once allowing for new and complex keyboard states. Another example of where hardware control is important is sound output. With Windows 95 and later the Windows operating system has enjoyed background sound effects such as startup sounds, shut down sounds, default bells, dings, and so on. All of these sounds are played back using a generic library called Comaudio.dll. While Comaudio.dll is perfectly fine for playing back system sounds it doesn't have enough control over the sound hardware to mix and process game sounds in real time. The way to resolve this issue is to use Microsoft's DirectSound.dll or XAudio2.dll libraries. Both of these libraries are able to gain direct access to your sound card and render virtual 3D audio as well as mix and playback a very large number of sound effects in real time. Such audio support in games would not be possible without direct control of the sound hardware. Second, now that we know why games are different from all other types of software applications we need to know what development kits and tools are required to get the maximum performance out of our computer. At first glance there seams to be all kinds of game development kits such as DirectX, XNA, SDL, Pygame, etc. However, after all my testing there is really only one good development API for accessible games, and it is Microsoft's DirectX SDK. As part of my independent research I wanted to know if the open source SDL project was up to DirectX standards. For those who don't know what SDL is it is an open source game development sdk, similar to DirectX, for Mac, Linux, and Windows. However, after a couple of weeks of testing the sdk I was sadly disappointed in SDL. SDL has an input management library similar to DirectInput. However, I found it lacking. If all you are looking for is keyboard and mouse support it will do the job for the average accessible gamer. If you want joystick, game pad, and force feedback device support it is rather useless as an API. The SDL audio mixer, SDL-Mixer, was the real disappointment for me. SDL-Mixer is very far away from being anything close to DirectSound or XAudio2 quality. SDL-Mixer's audio support is rather generic. You can load play, mix, do some stereo panning, simple volume shifts, on sound effects, but that is basically it. For those of us who have been enjoying the quality of audio feedback from DirectSound's Virtual 3D support SDl-Mixer's audio rendering is rather lame in comparison. If all you want to make is a Jim's NFL type clone SDL might be alright, but if you want a game like Shades of Doom, Sarah, or Tank Commander SDL isn't the way to go. After my disappointing exploration of SDL I decided to give Java a hand. I looked at Java's awt keyboard and mouse interfaces, javax.sampled.sound, and javax.media.j3d, packages. While they were a bit more successful they still didn't meet all my expectations. The Java awt keyboard support was so so. I have trouble complaining about the Java keyboard support since it worked, is a cross platform library, but is somewhat slower than if you were using something like DirectInput. Again as with SDL I was disappointed in a lack of game controller support for racing wheels and that sort of thing. For that I had to check out JInput. After playing with awt a while I downloaded, installed, and tested JInput. As a cross platform game input manager it is pretty good. I can't complain about its keyboard and mouse support, but joysticks etc only seam to operate correctly on native Windows systems. If using game controller x under an operating system like Linux it is very much hit or miss, and usually is a miss from my experience. So I am beginning to conclude on non-Windows platforms joysticks, racing wheels, and game pads aren't exactly widely supported. After Java keyboard testing I turned my efforts over to researching Java based audio support. The first thing I tried was the java.applet.AudioClip class. Not surprisingly I found it was way too simple for my needs. If all you want to do is play, stop, loop, sounds etc the java.applet.AudioClip class is alright. However, I decided to look into the javax.sampled.sound.Clip class. Again I found the API was just so so. To be honest I didn't see much difference in quality between javax.sampled.sound.Clip and SDL-mixer. About the most you can do with javax.sampled.sound is play, loop, change panning, volume, etc on sounds but nothing near DirectSound quality. In addition to the lack of really quality virtual 3D audio support in javax.sampled.sound I have yet to find a working solution for acquiring the play status of a sound effect. Such information is very important when a developer wants to insure two sounds don't play at the same time when they shouldn't. For example, if you fire a weapon you might want the sound of the weapon to stop before the enemy screams, explodes, dies, etc. You might not want the enemies death scream to start at the same time as your weapon sound. So as my last ditch effort at trying to make Java an alternative I tried my hand at javax.media.j3d. To be honest I didn't spend as much time with the API as maybe I should have, but again I am rather disappointed. While it does provide for virtual 3D audio support it seams the entire API was designed from a completely sighted gamers perspective. The API is poorly documented, and from what I can tell the audio tends to follow the graphics state of the game. Wherever the player is facing in the 3D world the audio tries to match the state of the screen. It is really a funky way of going about handling audio. Especially, since in audio games there is little to no graphics in our games. So we need to fake it for j3d. It is definitely not as a clean a programming API as DirectSound 3D or XAudio2. Since my primary complaint with SDL and Java's javax.sampled.sound packages was lack of 3D audio support I decided to try my hand at OpenAL. OpenAL is currently being hosted by Creative Labs, the maker of the Soundblaster cards, and at one time seamed like a promising alternative to DirectSound. However, over the years community support for OpenAL has slowly decreased. While there are Python wrappers for OpenAL such as Pyopenal it is no longer being developed, supported, and is abandonware. The Java wrapper library package, JOAL, is also not widely being used by Java developers. I wanted to know why. After some research I discovered the main OpenAL sdk is not actively being worked on by the open source community and there hasn't been any new builds for Mac OS or Linux in ages. Only the Windows port is slightly newer than the cross platform builds. Conclusion it is a technology that isn't likely to have a bright future of upgrades. After all my research, testing, and my hours of compiling information the fact of the matter is Microsoft's DirectX SDK and API is the best there is. Microsoft's DirectX Direct3D graphics engine is the leader in 3D graphics technology and arguably is better than OpenGL. Microsoft's DirectInput and XInput libraries has very good support for keyboards, mice, and force feedback devices that is unmatched by anything else. Microsoft's DirectSound and new XAudio2 libraries are capable of realistic virtual 3D audio that no other audio API I have tried can match. DirectX is standardized, relatively simple to program, and is extremely well documented. Everything else out there is just playing second fiddle. Third, there is the matter of speech output for your accessible game. Under Windows 2000, XP, and Vista the Sapi 5 SDK is a fairly standard API for text to audio support. If you know how to program for Sapi then you can support just about any voice out there on the market for Windows. Some sound terrible while others are fairly human in nature. All depends on what you are willing to spend for a Sapi voice for Windows. Though, finding an alternative for Sapi is difficult. A lot of accessible game developers tend to mix and match pre-recorded wav files for speech output. While it can sound good if actual human voice recordings are used it has a lot of problems attached. For a start, the size of your game download will be much larger since your entire speech system may double or triple the size of the application. Second, it is a lot more complicated to use a lot of wav files for speech. It certainly will be a lot slower to program than directly sending a string of text to Sapi to be spoken. Third, it doesn't allow the end user to change voices, rate, pitch, volume, etc in most cases. If you planned to use a language like Java there is no native Sapi 5 support. The only text to speech support native to Java is via FreeTTS. While FreeTTS works it is very mechanical sounding, robotic, and lacks the more human quality of voices we have come to know and love on Windows and Mac systems. So Java with FreeTTS isn't in my opinion a really great solution. On non-Windows platforms such as Linux the state of text to speech software is abysmal. There is no centralized speech API like SAPI though the gnome-speech API is about as close as you can come to it. The gnome-speech API would be fine except for the way the software is licensed. Since the gnome-speech API is licensed under an open source license agreement only open source text to speech engines and drivers ship with it. Which restricts you to Espeak and Festival. While there are gnome-speech driver source code for Eloquence, Dectalk, Cepstral, do to the fact they are commercial products by default gnome-speech does not come with these drivers. Which leaves it up to you, the developer, to download the gnome-speech third party drivers, compile them, and ship them with your game if you want to support anything other than the default Espeak and Festival drivers. In the end I am forced to conclude that while Sapi 5 isn't a perfect solution it sure beats the alternatives that are out there for accessible game developers. I haven't yet figured out how to incorporate Mac's Apple voices into games, but it certainly has to be easier than Linux's lack of centralized support for voices based on license restrictions. Sapi 5 would definitely be my pick for a text to speech system for anyone thinking of creating accessible games for the PC. Fourth, at last, I think we need to talk about programming languages. Traditionally most of the accessible games out there for the blind have been written in Visual Basic 6. This is quite understandable since most accessible game developers are not professional software developers and found Visual Basic easy enough to learn and use. It is a basic language, lacking the advanced features of C++, lacking the complexity of languages like C++, and was fairly well documented when it was popular a few years ago. However, as a programming language it is growing quite old, and it is becoming less and less supported as the years go by. We are already seeing problems with it under Windows Vista in that dx8vb.dll, a required dll for Visual Basic 6 games, no longer ships with Vista and has to be installed independently. Not a major problem, but it does leave the question of what Visual Basic 6 support, if any, will be offered on the next major Windows OS after Vista. Then, there is the Java programming language. As a language I personally like it. However, after nearly two months of testing, tinkering with it, and running through its paces I found it really lacking in the kinds of games I could create. I was decidedly unsatisfied with its audio support, and it is because of that lack of good audio support that I'm not making it my first choice. Though, I was truly impressed by Java's networking API. In the end I can only give Java a 3 out of 5 stars rating. I looked at Python, and again I felt it was only so so. While Python has the possibility of supporting DirectX no one has at present created a really good wrapper for DirectX and the Pygame API, which is based on SDL, is not all that great. I did manage to write a simple wrapper for Sapi 5 which worked satisfactory, but it wasn't anything to write home about. I could have done the same thing in C++, Visual Basic, etc with a lot less work. The only thing about Python that remotely made it worth the effort is the ability to make cross platform games for Mac, Linux, and Windows. Again I can only rate this language with a 3 out of 5 stars. In addition to the languages above I have had extensive experience with games written in C# 2008 and Visual Basic 2008. From a simplicity standpoint both languages are excellent. They are easy to learn, Managed DirectX is much easier than doing it in C++, and working with Sapi 5 engines is simple. If not for the fact that Microsoft had decided to drop support for Managed DirectX 9.0C I would have rated them both a 5 out of 5 stars. However, on account of uncertainty of Managed DirectX I have to give them only 4 out of 5 stars. In the end there was only one programming language, only one, that met all my expectations. The only one that did everything I wanted it to do was C++. The C++ programming language, the game programming industry standard, is the native language for DirectX, SDL, Sapi, etc. While it has a much higher than usual learning curve I think in the end the learning curve will prove useful to any game developer who wants to write bleeding edge audio games. It has the hardware specific control required for games, is flexible, and unlike other programming languages C++ remains a rock solid programming standard in which the industry revolves around. As we are learning right now other language support such as Visual Basic, Java, C#, etc comes and goes but the C++ DirectX code you wrote 5 years ago is still compatible with DirectX March 2008 without a need to upgrade right away. Point being C++ code rarely changes unless there is a major industry shift as we are seeing with XInput and XAudio2 support in DirectX March 2008. The other advantage with C++ is that there is a Cider project for Mac. The thing I like about Cider is that it emulates the Windows platform closely so that only miner changes in a C++ app need to be made in order to make a Mac OS X Leopard port. The big game companies such as EA Games are using Cider to port some of their popular titles from the Windows PC to Mac OS. While not a totally native solution something like Cider, only available for C++ developers, does make the matter of how to make a Windows game work on a Mac much easier. Perhaps the same solution EA Games uses would apply for future accessible games. In conclusion, there is many many choices and options out there for writing accessible games. Much of it really comes down to what operating system you want to support and what your needs are. Do you need full virtual 3D audio support, or will simple stereo panning work? Are you looking for joystick support or will just keyboard and mouse support do? These are not questions I can answer, but as for me if you want it all DirectX and the Windows platform is where the gold is. Much attention has been brought up on the Audyssey gaming list to the idea of cross platform games. There is some notion a game developer can create a game and run it on a Mac, Linux, Windows, and a cell phone. The truth is that kind of development is total madness. There is no common APIs among all those operating systems and devices. The more complicated the game the more difficult it will be porting it from one operating system to another. I didn't even cover cell phones in this article, but needless to say it can be as much of a nightmare as what I have previously described. If you are writing text based games that is one thing. Those can easily be compiled and run on just about anything. Try and create a game like Tank Commander and run it on anything and everything and it isn't possible from a programming stand point. There is just too many differences to make it a practical solution. Then, there is the matter of the programming languages themselves. Java's build once run anywhere design is both its best and worst feature. If your app is intended to be Os independent the language will succeed in doing what you want, but because of lack of native operating support you can't easily get access to Sapi, DirectX, etc. Python has a similar design concept as Java and has the same basic strengths and weaknesses. Visual Basic 6 is out of date and shouldn't be considered for new games. The .Net games thing has come and looks like it may be leaving. C++ is really the only language that seams to have a stable future for gaming, supports everything we want, and has the majority of game documentation anyway. If I were to make an educated opinion here I'd say as a game community the best thing to do would begin to convert our accessible games to C++, DirectX, and Sapi 5. If we really want to have Mac OS support then those developers would do well to find out what changes are required to support Cider. At that point Mac OS X Leopard users would enjoy the same quality of games as Windows users without having to use Boot Camp and Windows installed over Mac OS. Beginning in May or June I plan to begin work on a C++ programming guide for accessible game developers. In the document I plan not only to discuss basic programming techniques as variables, classes, objects, etc but will begin covering such topics as Sapi 5, DirectX, etc. All of it from the newbies point of view. I feel strongly that the future of the accessible games community may depend on a future in C++ and DirectX 9. --- Gamers mailing list __ [email protected] If you want to leave the list, send E-mail to [EMAIL PROTECTED] You can make changes or update your subscription via the web, at http://audyssey.org/mailman/listinfo/gamers_audyssey.org. All messages are archived and can be searched and read at http://www.mail-archive.com/[EMAIL PROTECTED] If you have any questions or concerns regarding the management of the list, please send E-mail to [EMAIL PROTECTED]
