On Thu, Jan 20, 2011 at 04:07:19PM +0100, Jan Vlček wrote: > > I have started working on XML-RPC testsuite for Spacewalk as a part > of my bachelor thesis. > Few ideas are concentrated in following Google Docs document. > > https://docs.google.com/Doc?docid=0AfRvg1ST7uS2ZGMzOG5ycm5fMjZjZ3YzMzhmcw&hl=cs&authkey=CIWYod8F > > <https://docs.google.com/Doc?docid=0AfRvg1ST7uS2ZGMzOG5ycm5fMjZjZ3YzMzhmcw&hl=cs&authkey=CIWYod8F> > > If you have other ideas or possible improvements, please reply to > this message or contact me directly.
Heya Jan, thank you for you post. Here are my comments about your proposal. > Common test procedure, 1. I don't think you should check if the Spacewalk server is setup in a particular way. All you really care about is login and password which your test suite will be given, and that will be used to log in to the server. It might or might not be Spacewalk administrator. In fact, I would very much like to see the test suite covering both admin and regular user accessing the XML RPC API, where some calls obviously have to fail for non-admin, and we really need to test if the call fails properly in that case. > Common test procedure, 2. Please do not waste time with these kinds of supplementary steps. They are extremely hard to do in general case, and you are just better off writing down the requirements and leaving it on users of your testsuite to setup the environment properly. > Common test procedure, 5. How exactly do you expect the cleanup to work? My gut feeling would be -- ditch the cleanup step for the time being as well. People might actully want to use your testsuite or parts of it to *setup* environment for work or other tests, and any cleanups might get in their way. > Underlying testing framework Does the testing framework have to be in Python? Did you look at beakerlib - https://fedorahosted.org/beakerlib/ ? I mention beakerlib specifically because that's something used heavily by the Spacewalk team, so if you used it, integration of your tests would then be much easier. > Spacewalk has already unit tests written in unittest which are being > run by testoob. Since unittest with testoob meet all mentioned > requirements I will pick them. Please note that the unit tests which expect testoob are for backend, not frontend. And they are dead code -- we don't check their results at all. So please do not get fixated on those and on testoob. > decorators of test cases Could you be more specific about what decorators of test cases you have in mind? -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat _______________________________________________ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
