Hello. On 13/12/16 16:32, Gustavo Sverzut Barbieri wrote: > On Tue, Dec 13, 2016 at 11:54 AM, Stefan Schmidt <[email protected]> > wrote: >> Hello. >> >> On 13/12/16 14:09, Felipe Magno de Almeida wrote: >>> On Tue, Dec 13, 2016 at 8:28 AM, Stefan Schmidt <[email protected]> >>> wrote: >>>> Hello. >>> >>> [snip] >>> >>>> I fully agree (even have native IPv6 here) but we need to be defensive >>>> what we expect to be available in our test suite. >>> >>> Why? It is testing, I agree we should need to be defensive in the >>> implementation and interfaces where EFL might run in stranger >>> environments, >> >> If the implementation would handle this all gracefully the test would >> never had been a problem. :) > > no no... the code gracefully handles that, but the test is explicit > "resolve ::1 to ::1 and I expect it to work". The test itself should > be conditional and I'm willing to do that, HOWEVER our test bots > should have IPv6 so we check that code. > > > >> but a test that fails on a machine that doesn't >>> handle IPv6 seems fine to me as long as it is rare enough. >> >> We assume IPv6 now, we assume a working internet connection, we assume >> udev for eeze testing, we assume a working dbus setup, in some cases we >> assume a file system which supports extended attributes, etc... >> >> If it is complicated to run make check we will have even less people >> running it. It should be the other way around. I guess I could look at >> anybody here who contributed a few patches this year and see someone who >> broke make check. If it is to complicated or fails for some reason >> people will just stop using it. > > our test suite is unusable the way it is...
While I agree that the test suite needs improvements calling it unusable is over the top. really it lacks > granularity, it takes lots of time and most of the time it looks like > it failed since people forget to trap EINA_LOG to capture errors, then > you get tons of error messages (although the test reports success, > messages confuses users). In the normal run you see a green PASS or red FAILED. The eina_log errors are only in the log file itself. What is the way to trap these error messages that will come if we do test error cases? IIRC you did eina_log initially so you surely have a good solution here. :) > just the time it takes to run it all is one of the reasons I don't > run them that frequently. :-/ You know that you can run the suites individually. When doing ecore_con stuff for example you could ignore the suites like eina and ecore which take a lot time. On a bigger patchset I would still expect that you run a full make check before pushing due to all the possible side effects the changes could have. POretty sure the time costs of not running it, letting me find them, bisect them, report them back to you and then dealing with me over mail is higher compared to running them before pushing a patchset and fixing them. :P regards Stefan Schmidt ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ enlightenment-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
