How gross would it be just to have a logical channel in TAP that could represent output to STDERR? That plus the exit status of the test script is pretty much all you have at the moment. Would that be so bad?
Perhaps, for non-Unix testing, perhaps not. Here is what I have. There are bots that are running on various systems. One asks the bot to do something, and it responds. My bots are being set up to do testing things. Can you ping the server, can you open a TCP connection to port 80, can you get a URL over HTTP, can you run this script, etc. The idea is that without a traditional debugger, there should be a command line user interface that does these things as well as an RPC method for automated tests. When I send an RPC response, it has the human readable reply and the response code. Sure, I could have written a regex to figure out if the response is correct of failed, and I allow that. It is easier to process the codes though, and that suffices for most tests. Sure all of this could have been done in SSH, if everyone had account on and the machines and etc. There are other ways, this is what I'm trying to do and I may fail. Then you all can learn from my 409, 416 or 417. Now, I know you are thinking about exit status on test scripts and I'm thinking individual tests (of which running another test script might be an instance), but in the distributed functional testing space, one really can't rely on independent test scripts to do much. I need to set up a tail with regex on a log file on one bot, hit it from another bot, and know that both were successful real time. Then keep going. Now if all TAP can handle is Perl, or even just code output, then those of us pushing the envelope will eventually migrate to some other test output format. Functional testing is years behind, especially testing distributed applications like network security. I'm trying to leverage what I can to get something useful working. Perl and Test::Harness seem to be good tools to try.
