I think you pretty much covered all the major pros and cons of private
vs public testing so I really can't add too much to that part of the
discussion. However, what I may be able to add here is the benefit of my
experience with private verses public testing.
As far as public testing goes I've had mixed feelings about it. When I
originally started USA games back in 2004 I began with a free game
called Star Trek Final Conflict. During the testing of STFC the
experience was very positive, was fun for everyone, and I got lots of
helpful end user feedback and suggestions. However, that was for the
most part a low pressure project so the community was pretty cool about
it. However, the problem with public testing for me really began in 2006
soon after I agreed to take over some game projects James North had been
working on where money had already changed hands creating a very heated
and stressful situation.
Over the passed three years nagging has been my number one problem with
public testing. I often get requests like when is the next beta coming
out, why didn't I add this or that suggestion, why don't I stop fooling
around and get the game done, and so on. A few, not all, tend to think
because they paid in advance for this game that I must add the features
they suggest, and get the game done according to some time table they
themselves think would be reasonable. This, of course, has not been a
pleasant experience for me at all. However, i contribute most of this
to the fact money had already changed hands before I took over the
project, and now some people believe I owe them whatever they want when
they want it.
However, no matter how bad the negative feedback has been there have
been a lot of positives that have come out of public testing. I've found
bugs faster, tested it on a wider range of systems, and got a lot of
suggestions that could be added while the game was being designed from
scratch. I found out what people did and didn't like about my games
allowing me to tailer the game to what the majority of people wanted.
This undoubtedly has resulted in a much better product in the long run.
As far as private testing goes I've had some mixed results with that as
well. Some people just join in order hoping to get a free game out of it
which means they don't do a lot of testing and aren't really working for
your best interests. Some are honestly trying to help, but miss things
or don't encounter bugs that might show up on a different system. The
worst problem I've had is a couple of private testers that were doing a
good job testing, but they were bad mouthing and having personal issues
with the other testers. In the end I had to kick them off the test team
to restore peace and order to the list.
On the positive side private testing really helped me get the game
engine written, tested the early stages of MOTA, when the game and the
engine really wasn't ready for public consumption. There were several
bugs in the engine I know of, some popped up on some testers machines,
and not others. By using the test team first I managed to get through
allot of stability issues and bugs so by the time I released the first
public beta many of the bugs and serious problems had been resolved.
The other related advantage is how the test team views the project
verses the public. In a game's early development the developer may
change his/her mind many times and drastically change certain elements
of the game such as adding some and removing others here and there. A
test team understands this is an early test release so won't complain to
much if you add something new or yank something old out of the game. As
I've recently discovered with the public betas of MOTA sometimes the
public takes a completely different view and screams for that feature to
get immediately returned to the game even though the developer may have
reasons not to restore that feature to the game at that time. This
happened to me not to long ago when I decided to disable the save/load
game feature in the demo, and people screamed bloody murder over it. My
attempts to explain that the game was only a demo, not a full registered
game, fell on deaf ears. They had the feature in the previous release
and now they want it back. This wouldn't have happened if I would have
disabled that feature from the start, or was using a private test team.
Gamers mailing list __ Gamers@audyssey.org
If you want to leave the list, send E-mail to gamers-unsubscr...@audyssey.org.
You can make changes or update your subscription via the web, at
All messages are archived and can be searched and read at
If you have any questions or concerns regarding the management of the list,
please send E-mail to gamers-ow...@audyssey.org.