Hi Simon, > > 5) Possibly it makes also sense to allow GNULIB_TOOL_IMPL to be set to > > 'sh+py'. In this case the script will make a full copy of the destination > > dir, run the shell implementation and the Python implementation on the > > two destination dirs, separately, and compare the results (again, both > > in terms of effects on the file system, as well as standard output). > > And err out if they are different. > > Generally I'm happy to hear about speedupds of gnulib-tool! The plan > sounds fine. I think this step 5) is an important part to get > maintainers try the new implementation, and report failures that needs > to be looked into. If there was a small recipe I can follow to get a > diff that can be reported back, I would run it for a bunch of projects > that I contribute to.
Thanks for your feedback, and offer to use this mode. > While a self-test suite for gnulib-tool would be nice, some real > regression testing by attempting to build a bunch of real-world projects > that rely on gnulib-tool may be simpler to realize. If there is a CI/CD > that builds ~30 different real-world projects (perhaps at known-good > commits) and compares the output against an earlier known-good build, > for each modification to gnulib-tool in gnulib, that would give good > confidence to any change to gnulib-tool. I guess we are thinking about slightly different things: * (A) I am thinking about - for P in { coreutils, gettext, ... }, taking a frozen(!) checkout of P, removing irrelevant source files (esp. all *.h, *.c, documentation, etc.), - and a frozen(!) set of gnulib modules at a specific time point, - and merely invoke gnulib-tool and compare the generated files and stdout. * (B) You seem to be thinking about - for P in { coreutils, gettext, ... }, taking the current git of P (or latest release of P), - taking the current set of gnulib modules, - and invoke not only gnulib-tool, but also './configure' and make. I think that - With either approach, the confidence to any change in gnulib-tool will be the same. - With approach (A), when we make a change to gnulib-tool, we need to commit new expected test results, which is quite easy. No effort otherwise. - With approach (B), we will get failures for other reasons as well: when a gnulib module has changed in an incompatible way; when the git repository of P has moved; when package P itself is broken. Sounds like a continuous effort to hunt down (mostly) false positives. Bruno