On Thu, May 5, 2011 at 10:18 AM, Kristian Rietveld <[email protected]> wrote: > As we've already discussed on IRC some time ago, I would really like to see > all > GTK+ unit tests in one single place, instead of in several different places > in the > source code. We really need people to run the unit tests more often and thus > this needs to be made easy (like you also mention in your enumeration above), > I don't think putting different unit tests at different places makes this > easier. > I do agree with putting all things in one location. However, I do not agree with "people need to run tests more often". I think running tests is a job for machines.
> We can always distribute the unit tests as a separate tarball if that will > help, can't we? > Well yeah, but then it still requires compilation of something. The approach with "Here, open this in glade" and "Does it look like this screenshot?" is a lot better, because it's very easy. I guess having tests written in Python would come closest to this. > Another question: why was gtk-reftest put in gtk+/tests/reftests/gtk-reftest > instead of in gtk+/gtk/tests/gtk-reftest, with a subdirectory reftests > containing the glade files? Then on make check for the GTK+ unit tests, the > reftests would automatically be executed as well. Currently, you also need > to compile a GTK+ checkout to use reftests, right? > I never liked the idea of putting tests in the same directory tree as the actual source code, because in the good case it creates spam to stdout (about entering directories I don't care about) and in the worst case it actually compiles tests, so it not only spams stdout but it also takes lots of time relinking. If I am actively working on refactoring code I do not want to be annoyed by random test cases. Currently this is still bearable, but when there's 10.000 tests that each get relinked and that takes 5 minutes after every 1 line change in gtknotebook.c or so, I am going to be really annoyed. So I do think that tests, just like documentation should live outside of the normal source tree. Which is why I put it there. > These sounds like numbers I would expect. What in GTest would need > improvement to realize this? > GTest mainly needs usability improvements such as those you pointed out by those error messages. (I'm sorry if I offended you by taking a test written by you as the bad example, I just took a random file from gtk/tests as an example of why our current approach is bad.) The problem is that currently running tests (or a single test) manually is complicated and oftentimes ends up with unparsable error messages that often are no help in actually figuring out what got broken. Also, invocation of test runners should be a lot simpler (see below). > About organization, I think for one all GTK+ unit tests should be in one > place (and the GDK tests in another place). > Yes, I would very much suggest tests/ for this. I don't care a whole lot if we have tests/gdk and tests/gtk, because I think it's hard to test GDK without also testing GTK as GTK has all the niceties that you want to have when writing tests (like ui files) and it's very cumbersome to write to the GDK API. > Secondly, we also need to develop a consistent naming scheme for tests. Unit > tests currently have different ways of naming tests: > > /FilterModel/self/verify-test-suite: > /expander/click-expander: > /recent-manager/get-default: > /tests/column-new: (these are for icon view) > /Builder/Window: > I would attribute that to GTester. The path naming idea is kind of useless, because what you actually need to remember is the name of the test binary (so that you can run it), not the name of the actual test. There is no way to run "all GtkLabel tests" by saying gtester -p /tests/gtk/widgets/label and that way making it grab all tests in the testsuite and make it run those that match this path. Instead you run tests/gtk/widgets/label and that's it. You can also see that gtk-reftest basically ignores the test paths - or better: abuses them to store filenames in them so that you actually get useful output when running a test, because it tells you the actual file that failed. So for this case we'd actually first need to have a use for these test paths. Benjamin _______________________________________________ gtk-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/gtk-devel-list
