Hi all & Seb, > My question is the following: > > * one test can use mutliple library > * a test can pass or not (pass or fail) > * if a test passes, all libraries it uses can be mark as "successfully > tested" > * if a test fails,... what do I put ? > > Does all libraries it uses will be mark as "failed" ? > > I could do the following: > > * imagine testA uses lib1 and lib2. > * testB uses lib1 and lib3 > > testA fails. testB passes. I have: > > * lib1 : one fail, one pass > * lib2 : one fail > * lib3 : one pass > > which can give: > > * lib1 : successfully tested > * lib2 : failed > * lib3 : successfully tested > > that is, when a lib a least passes in one test, then it's a considered as a > success. > > What do you think ? > > Thanks for your help on this infinite topic, and sorry for my bullet list > abuse :) (maybe I've been eaten by a black hole, and now lost in the matrix, > yeah, the testing matrix...) > I think this a dangerous topic for me to discuss on a public list. I used to work in the testing bussiness and some collegues except me to do it the proper way which is thorough and with lots of details. And I once read that testers are as forgiving as bridge card game players ;)
But I don't think 'proper' in this sense is the way to go for this project. There are two issue's you do not address with the proposed approach: - what we call test is a file that actually contains multiple test cases, where a test case tests a specific situation. For a library to be tested properly, you need a decent coverage, meaning that most probable situations are covered. A lot of tests use the delay library, but most use is trivial (blinking the led on startup), which is a real low coverage. - if a testcase fails, it means that the library does not work as expected (assuming that the test is correct). But it does not say how bad that is. If an lcd lib does not print via the pseudo var, we have a major issue with this lib. format.jal on the other hand does not handle the number of chars like it should: the number of chars specified does not include the decimal point. But since the user specifies wether a decimal point is used or not, this is easy to work around. The first issue could be addressed by defining the libraries that are tested with a decent coverage with the lib and not rely on the include statements. The second issue can be dealt with in a procedural way: when it does not compile, it clearly fails. But when it does, someone always has to check proper working anyway (unlike you, I don't expect automated testing to become a real option for us in the near future. But you proved me wrong before). When it is not working properly, the testen must estimate if the failure is so significatant that the library fails. As an alternative we could add categories: pass, minor, major or fail. Joep --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "jallib" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/jallib?hl=en -~----------~----~----~----~------~----~------~--~---
