On 21.1.2011 09:08, Jan Pazdziora wrote:
Common test procedure, 1.
I don't think you should check if the Spacewalk server is setup in
a particular way. All you really care about is login and password
which your test suite will be given, and that will be used to log in
to the server. It might or might not be Spacewalk administrator. In
fact, I would very much like to see the test suite covering both admin
and regular user accessing the XML RPC API, where some calls obviously
have to fail for non-admin, and we really need to test if the call
fails properly in that case.
Well, I assumed that there will be some default state. But it would
probably be impossible to setup the default state before any tests.
I can definitely write tests which will assert just changes made by them
(and not in reference to some default state).
Common test procedure, 2.
Please do not waste time with these kinds of supplementary steps. They
are extremely hard to do in general case, and you are just better off
writing down the requirements and leaving it on users of your
testsuite to setup the environment properly.
But with such requirements, how would you run the whole suite?
Common test procedure, 5.
How exactly do you expect the cleanup to work? My gut feeling would be
-- ditch the cleanup step for the time being as well. People might
actully want to use your testsuite or parts of it to *setup*
environment for work or other tests, and any cleanups might get in
their way.
Clean the mess created in setup phase and during test itself. The
purpose of tests is not to initialize environment. They should assure
that the XML-RPC interface is running properly. Or not?
Underlying testing framework
Does the testing framework have to be in Python?
Did you look at beakerlib - https://fedorahosted.org/beakerlib/ ?
I mention beakerlib specifically because that's something used
heavily by the Spacewalk team, so if you used it, integration of your
tests would then be much easier.
Have not seen beaker before. But looks promising.
Jan Hutar has already asked me and I agreed.
Spacewalk has already unit tests written in unittest which are being
run by testoob. Since unittest with testoob meet all mentioned
requirements I will pick them.
Please note that the unit tests which expect testoob are for backend,
not frontend. And they are dead code -- we don't check their results
at all. So please do not get fixated on those and on testoob.
I have just seen them in repo therefore I thought that it is the
preferred way.
decorators of test cases
Could you be more specific about what decorators of test cases you
have in mind?
I got inspired by this blogpost:
http://kbyanc.blogspot.com/2007/06/pythons-unittest-module-aint-that-bad.html
But with beaker it is not an issue anymore.
Thanks for your input!
Jan
_______________________________________________
Spacewalk-devel mailing list
Spacewalk-devel@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-devel
