Hi - I'm trying to find a web unit and load testing tool that will let me test URL-based APIs. This is basically a parameterized URL that will return JSON-encoded results. I would like to be able to easily set up 100+ test cases that are all test the same API, but with different URL parameter values and different expected results. I would to be able to flexibly configure these test cases, ideally as a text table or other simple-to-create-and-parse data structure, and generate a report of pass/fail cases for each test.
I would really like to use something like twill, but I'm getting stuck on 2 fronts: 1. How to configure a single test case. The key here is that to check for valid results I need to unserialize the json returned result. Twill obviously doesn't have this built in, so I assume I will have to call a twill script from python and then record its results for subsequent processing. Is that right? Should the calling code itself then be part of a unit testing framework? 2. How to configure the series of test cases. I would like to avoid writing a separate test class / function for each case, and instead somehow iterate over the param/result pairs. I can't figure out how to do this without having the entire suite abort after the first test failure. I would like to run all 100 tests, and have it report back which ones succeeded and which ones failed. Are there options or configurations available that allow the flexibility of a simple configuration file / data structure but that will still run through all the tests as individuals? Or is this a mismatch for twill's design? FWIW, I've looked at funkload for this as well, but it isn't clear how to handle the configuration issue above either, and unfortunately funkload's nice reporting functionality doesn't work for functional testing, only load testing, so I'm not actually gaining that much. I'm looking at other options right now, with twill and nose on the shortlist. Any pointers would be greatly appreciated! Thank you in advance for your time. Thanks, Ramon ----+---- This email message (and any attached document) contains information from Ingenuity Systems Inc. which may be considered confidential by Ingenuity, or which may be privileged or otherwise exempt from disclosure under law, and is for the sole use of the individual or entity to whom it is addressed. Any other dissemination, distribution or copying of this message is strictly prohibited. If you receive this message in error, please notify me and destroy the attached message (and all attached documents) immediately.
_______________________________________________ twill mailing list [email protected] http://lists.idyll.org/listinfo/twill
