I know py.test has distributed testing already but I have been using
Testswarm (a distributed javascript testing framework) for a while now
and thought the idea would fit nicely with py.test. This is just me
kind of brain storming the idea, so feel free to ignore me if you have
no interest in this idea.
Ok basic premise is:
Central server for anyone to schedule test runs on
Clients connect to the central server and run tests as they come
in for any and all apps they meet the requirements for
Goals:
Eliminate the need for everyone to establish their own distributed
testing network by establishing a remote distributed network where
clients connect and will run tests from any app that is registered on
the server.
Brainstorm of requirements:
Server - Needs a way for applications / libraries to register
Server - Apps need a way to set dependencies, and a way to specify
version for that dependency
Server - Apps need a way to set the environments to test on (os
version and python version)
Server - Apps need a way to schedule test runs and upload the code
to test against
Server - Need a UI to display a test metric like
http://testswarm.com/user/jquery/
Server - Needs a method of tracking available clients, and their
environments
Server - Needs a way to send tests to every available client that
meets the environment specs for each app.
Needs to keep track of the test run id for results
Server - Needs to accept results back
Needs to identify the run and clients environment id to make
sure we can mark an environment finished for the run
Server - Needs a fault catcher to reschedule a test run /
environment if a remote client disconnects before it finished
Server - Needs a way of removing code once all tests have been
executed for a particular run, or after X days
Server - Needs to ensure remote execution of apps is not possible
for security
Client - Needs a way to connect to the central server and keep it
notified that it is still alive
Client - Needs a way to receive Tests
Client - Needs a way to report results back
Brainstorm implementation details:
modified? py.execnet for the server test dispatcher
modified? py.execnet for the client test receiver
Client should be command line only so it can be run as a service
Web app for managing Apps and result display
REsT API for scheduling test runs and uploading the required code?
"py.test schedule" command?
_______________________________________________
py-dev mailing list
py-dev@codespeak.net
http://codespeak.net/mailman/listinfo/py-dev
