Hello Arnaud, Charles, regarding the nut_clock_* iface unit testing, I came to the following conclusion:
1/ It is IMO generally impossible to do deterministic unit test; the problem in question is inherently non-deterministic (I mean non-det. by nature). Justification: the main reason is that we can't perfectly predict the CPU time spent on computation, mainly because we have multi-tasking; there might be any number of context switches between almost any states, so if we measure time with us/ns precisions, we can't expect fitting times at that scale. 2/ This means that there will always be a possibility of false-positive unit test case failure; it is therefore inevitable to try to i) decrease the case possibility to minimum ii) think about whether we want to invalidate a build if such a UT fails ad i) The only thing I came with is usage of statistic techniques: we should randomise the unit test and/or run it multiple times. The meaningful deviation from expected result shall be the avg/mean deviation (with all the standard statistic tricks applied --- like extremes cuts etc). I also suggest that if there's a failure, we could re-run the very same test case another time to see whether the next run fails, again (thus eliminating possibility of a temporary influence etc). ad ii) I think the above should be quite enough to eliminate false positive UT failure reports. However, it's still not certain (and can never be). Therefore, we should decide whether we want to be pessimistic or optimistic by default, i.e. whether to invalidate the build or to just send a warning or something... 3/ The methods I'd use are actually quite simple; for the timestamps difference (i.e. typical usage of the monotonic clock) validity checking, I'd simply use select with ho descriptors and a timeout. I'd take a timestamp before, after and compare the diff. with the timeout. In pseudo-code: nut_clock_timestamp(&ts); select(0, NULL, NULL, NULL, &to); diff = nut_clock_sec_since(&ts); ut_pass = abs((to.tv_sec + to.tv_usec / 1000000.0) - diff) < sigma; (Note that this pseudo-code doesn't include application of the statistic methods discussed above.) I believe that cca 10 us precision is actually far the best we can hope (actually, I think that we could accept even worse precision regarding our practical needs). This doesn't apply to the case of using the time_t fallback, of course. As of the RTC unit testing, I think that we might be OK with unit-sec precision; if so, we can simply exec "date +%s" to make an "external authority" to give us seconds since the Epoch and compare this with what our iface provides. Note that there is generally the problem of "what if the UT is executed exactly at the time of clock change...?" Yes, that's a bugger (sorry for my Greek), but the repeated execution should filter such problems out (see 2/i). ... and that's it, actually. I'll be more than glad to know if you have better ideas; I know that's not much, but right now, it's all I've got (I mean all what can be simply and safely done automatically upon every build.) Note that the statistic methods make the UT quite time consuming; also note that it would actually be great if we could automate tests like "shifting the system clock and check that the diff. of 2 timestamps is still OK --- i.e. automatic UT of monotonic clock. However, I wouldn't want to actually mess with the buildslave system clock... It could be done on a virtual machine though; so we would need a VM/QEMU and do make test, there... That could be possible, but also quite complicated. And that's indeed all I have. As always, any comments, suggestions etc welcome, Regards, vasek -- Václav Krpec Network UPS Tools project Eaton Opensource Team ----------------------------- Eaton Elektrotechnika s.r.o. ~ S�dlo spolecnosti, jak je zaps�no v rejstr�ku: Kom�rovsk� 2406, Praha 9 - Horn� Pocernice, 193 00, Cesk� Republika ~ Jm�no, m�sto, kde byla spolecnost zaregistrov�na: Praha ~ Identifikacn� c�slo (ICO): 498 11 894 ----------------------------- _______________________________________________ Nut-upsdev mailing list [email protected] http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/nut-upsdev
