Re: planning for beta-testing gnulib-tool.py
Collin Funk wrote: > I'm still finishing some things before we can recommend it for > testing. Right. It would not be productive if we had different package maintainers report the same bug(s) at the same time. > But Wget seems like a good project for testing. Do you have > any thoughts Bruno? Among the packages listed in users.txt, I can imagine that the following — whose maintainers have occasionally shown up on this mailing list — would respond to beta-testing requests in a friendly manner: autobuild https://git.savannah.nongnu.org/gitweb/?p=autobuild.git;a=tree bison https://git.savannah.gnu.org/gitweb/?p=bison.git;a=tree clisp https://gitlab.com/gnu-clisp/clisp coreutils https://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=tree diffutils https://git.savannah.gnu.org/gitweb/?p=diffutils.git;a=tree emacs https://git.savannah.gnu.org/gitweb/?p=emacs.git;a=tree findutils https://git.savannah.gnu.org/gitweb/?p=findutils.git;a=tree gettext https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=tree gnutls https://www.gnutls.org/ https://gitlab.com/gnutls/gnutls.git gsasl https://git.savannah.gnu.org/gitweb/?p=gsasl.git;a=tree grephttps://git.savannah.gnu.org/gitweb/?p=grep.git;a=tree gziphttps://git.savannah.gnu.org/gitweb/?p=gzip.git;a=tree libffcall https://git.savannah.gnu.org/gitweb/?p=libffcall.git;a=tree libiconvhttps://git.savannah.gnu.org/gitweb/?p=libiconv.git;a=tree libidn https://git.savannah.gnu.org/gitweb/?p=libidn.git;a=tree libidn2 https://gitlab.com/libidn/libidn2 libntlm https://git.savannah.nongnu.org/gitweb/?p=libntlm.git;a=tree libunistringhttps://git.savannah.gnu.org/gitweb/?p=libunistring.git;a=tree libvirt https://libvirt.org/ https://libvirt.org/git/?p=libvirt.git;a=tree m4 https://git.savannah.gnu.org/gitweb/?p=m4.git;a=tree pspphttps://git.savannah.gnu.org/gitweb/?p=pspp.git;a=tree wgethttps://git.savannah.gnu.org/gitweb/?p=wget.git;a=tree wget2 https://git.savannah.gnu.org/gitweb/?p=wget/wget2.git;a=tree Bruno
Re: planning for beta-testing gnulib-tool.py
Hi Darshit, On 3/14/24 12:12 PM, Darshit Shah wrote: > I'm following the development of gnulib-tool.py pretty closely. Wget and > Wget2 have tried to use the Python version multiple times in the past with > varying levels of success. I am enjoying working on it, so hopefully this time will be more successful. :) > I'll volunteer my time for using GNU Wget and Wget2 as initial beta testers > of the new Python based gnulib-tool. I am not too familiar with Wget but since I already have Wget2 cloned I attempted to use gnulib-tool.py to run the bootstrap script: $ env GNULIB_TOOL_IMPL=py ./bootstrap $ ./configure $ make all $ make check This builds successfully and passes the test cases. However, as expected the output of gnulib-tool.py does not match gnulib-tool.sh. I'm still finishing some things before we can recommend it for testing. But Wget seems like a good project for testing. Do you have any thoughts Bruno? Collin
Re: planning for beta-testing gnulib-tool.py
Hi Bruno and Colin, I'm following the development of gnulib-tool.py pretty closely. Wget and Wget2 have tried to use the Python version multiple times in the past with varying levels of success. I'll volunteer my time for using GNU Wget and Wget2 as initial beta testers of the new Python based gnulib-tool. On Thu, Mar 14, 2024, at 19:53, Collin Funk wrote: > Hi Bruno, > > On 3/10/24 8:59 AM, Bruno Haible wrote: >> 4) There is a list of packages that use gnulib, in the file users.txt. >>We can pick, say, the 10 most important or most active GNU packages among >>these, and repeat step 3) for each of them. > > I am still working on gnulib-tool.py.TODO and agree we shouldn't > recommend it until that is complete. However, gnulib-tool.py has > enough features implemented that it can be used to bootstrap many GNU > packages successfully. By successfully, I mean that I can run > > $ ./bootstrap > ./configure --prefix=$HOME/bin > make all && make check > > and get the expected results. Doing this helps me diff the files and > stdout emitted by gnulib-tool vs gnulib-tool.py. > > I like the plan of picking ~10 GNU packages and think it is a good > starting point that we can adjust if needed. Since the process is > useful for me anyways, I think it is best that go through the list of > packages in users.txt and try building them. > > I am thinking that I should note which packages build successfully, > which ones don't (hopefully none, though that is unlikely), and maybe > a note if I notice anything strange. I think this should help us pick > projects that will work best for testing. > > I can keep a file locally and just email it when we are ready for more > formal testing, or I can keep it in the git repository somewhere. Let > me know if you think this would be helpful. > > Collin
Re: planning for beta-testing gnulib-tool.py
Hi Bruno, On 3/10/24 8:59 AM, Bruno Haible wrote: > 4) There is a list of packages that use gnulib, in the file users.txt. >We can pick, say, the 10 most important or most active GNU packages among >these, and repeat step 3) for each of them. I am still working on gnulib-tool.py.TODO and agree we shouldn't recommend it until that is complete. However, gnulib-tool.py has enough features implemented that it can be used to bootstrap many GNU packages successfully. By successfully, I mean that I can run $ ./bootstrap ./configure --prefix=$HOME/bin make all && make check and get the expected results. Doing this helps me diff the files and stdout emitted by gnulib-tool vs gnulib-tool.py. I like the plan of picking ~10 GNU packages and think it is a good starting point that we can adjust if needed. Since the process is useful for me anyways, I think it is best that go through the list of packages in users.txt and try building them. I am thinking that I should note which packages build successfully, which ones don't (hopefully none, though that is unlikely), and maybe a note if I notice anything strange. I think this should help us pick projects that will work best for testing. I can keep a file locally and just email it when we are ready for more formal testing, or I can keep it in the git repository somewhere. Let me know if you think this would be helpful. Collin
Re: planning for beta-testing gnulib-tool.py
I like the plan to replace gnulib-tool with a faster implementation, and a two-year migration phase sounds reasonable to see if it will work in practice. Trying gnulib-tool.py on OATH Toolkit (which use a somewhat unorthodox gnulib usage style by adding code into git) results in error below. I thought it was a missing mkdir at some point, but I couldn't find a solution... ideas? git clone https://gitlab.com/oath-toolkit/oath-toolkit.git cd oath-toolkit echo $GNULIB_REFDIR /home/jas/src/gnulib # 3088ee223bb986ad51d7c71ca64aaf4b600bc06c $GNULIB_REFDIR/gnulib-tool.py --add-import Module list with included dependencies (indented): File list: lib/dummy.c m4/00gnulib.m4 m4/gnulib-common.m4 m4/zzgnulib.m4 /home/jas/src/gnulib/gnulib-tool.py: *** could not create file /home/jas/src/gnulib/lib/dummy.c /home/jas/src/gnulib/gnulib-tool.py: *** Stop. /Simon signature.asc Description: PGP signature
Re: planning for beta-testing gnulib-tool.py
Hi Simon and Bruno, On 3/10/24 7:00 PM, Bruno Haible wrote: > I think this would be a reasonable time after having made the Python > implementation > the default. I expect every package maintainer to run 'gnulib-tool' at least > once > in two years and report problems if they occur. Based on this assumption, we > could > drop gnulib-tool.sh two years after having made the Python implementation the > default. Two years seems like a good amount of time for projects to prepare and report issues. From my limited experience, it seems that gnulib-tool is pretty stable feature wise. Maintaining them both for two years would be extra work, but should hopefully result in a stress-free transition. > That's not the plan. Most features have been implemented for good reasons. > There is nothing that we can't easily implement on the Python side, other > than error behaviour. Just echoing Bruno's thoughts here, but the hardest part of the Python rewrite is my lack of experience with the regular gnulib-tool. If I had more knowledge of gnulib, the entire gnulib-tool.py.TODO file would be done by now. :) I am not a GNU maintainer or anything. I think even my first contribution was only even a month ago with some build issues in Coreutils on FreeBSD. I discovered gnulib-tool.py while messing around hacking. Since I have no interesting projects at the moment, and gnulib-tool.py seems like it will save maintainers a lot of time cumulatively, it was an easy decision for me to work on. I am continuing to work on it and hope to have it in a good place for testing sometime this month. You all seem to have good ideas for testing so far. I am happy to help where possible once there is a more concrete plan. Collin
Re: planning for beta-testing gnulib-tool.py
Hi Simon, > > - With approach (A), when we make a change to gnulib-tool, we need to > > commit > > new expected test results, which is quite easy. No effort otherwise. > > - With approach (B), we will get failures for other reasons as well: when > > a gnulib module has changed in an incompatible way; when the git > > repository > > of P has moved; when package P itself is broken. Sounds like a > > continuous > > effort to hunt down (mostly) false positives. > > Right, I agree! OK, then let's forget about (B). > I wonder when/if we could get rid of gnulib-tool.sh? I think this would be a reasonable time after having made the Python implementation the default. I expect every package maintainer to run 'gnulib-tool' at least once in two years and report problems if they occur. Based on this assumption, we could drop gnulib-tool.sh two years after having made the Python implementation the default. > Maybe we have to declare some features no longer supported if they > cannot be implemented easily in gnulib-tool.py... That's not the plan. Most features have been implemented for good reasons. There is nothing that we can't easily implement on the Python side, other than error behaviour. Bruno
Re: planning for beta-testing gnulib-tool.py
Bruno Haible writes:
> I guess we are thinking about slightly different things:
>
> * (A) I am thinking about
> - for P in { coreutils, gettext, ... }, taking a frozen(!) checkout of P,
> removing irrelevant source files (esp. all *.h, *.c, documentation,
> etc.),
> - and a frozen(!) set of gnulib modules at a specific time point,
> - and merely invoke gnulib-tool and compare the generated files and
> stdout.
>
> * (B) You seem to be thinking about
> - for P in { coreutils, gettext, ... }, taking the current git of P
> (or latest release of P),
> - taking the current set of gnulib modules,
> - and invoke not only gnulib-tool, but also './configure' and make.
>
> I think that
> - With either approach, the confidence to any change in gnulib-tool will be
> the same.
> - With approach (A), when we make a change to gnulib-tool, we need to commit
> new expected test results, which is quite easy. No effort otherwise.
> - With approach (B), we will get failures for other reasons as well: when
> a gnulib module has changed in an incompatible way; when the git
> repository
> of P has moved; when package P itself is broken. Sounds like a continuous
> effort to hunt down (mostly) false positives.
Right, I agree!
I wonder when/if we could get rid of gnulib-tool.sh? Maintaining both
would be time-consuming. Maybe we have to declare some features no
longer supported if they cannot be implemented easily in
gnulib-tool.py...
/Simon
signature.asc
Description: PGP signature
Re: planning for beta-testing gnulib-tool.py
Hi Simon,
> > 5) Possibly it makes also sense to allow GNULIB_TOOL_IMPL to be set to
> >'sh+py'. In this case the script will make a full copy of the destination
> >dir, run the shell implementation and the Python implementation on the
> >two destination dirs, separately, and compare the results (again, both
> >in terms of effects on the file system, as well as standard output).
> >And err out if they are different.
>
> Generally I'm happy to hear about speedupds of gnulib-tool! The plan
> sounds fine. I think this step 5) is an important part to get
> maintainers try the new implementation, and report failures that needs
> to be looked into. If there was a small recipe I can follow to get a
> diff that can be reported back, I would run it for a bunch of projects
> that I contribute to.
Thanks for your feedback, and offer to use this mode.
> While a self-test suite for gnulib-tool would be nice, some real
> regression testing by attempting to build a bunch of real-world projects
> that rely on gnulib-tool may be simpler to realize. If there is a CI/CD
> that builds ~30 different real-world projects (perhaps at known-good
> commits) and compares the output against an earlier known-good build,
> for each modification to gnulib-tool in gnulib, that would give good
> confidence to any change to gnulib-tool.
I guess we are thinking about slightly different things:
* (A) I am thinking about
- for P in { coreutils, gettext, ... }, taking a frozen(!) checkout of P,
removing irrelevant source files (esp. all *.h, *.c, documentation, etc.),
- and a frozen(!) set of gnulib modules at a specific time point,
- and merely invoke gnulib-tool and compare the generated files and stdout.
* (B) You seem to be thinking about
- for P in { coreutils, gettext, ... }, taking the current git of P
(or latest release of P),
- taking the current set of gnulib modules,
- and invoke not only gnulib-tool, but also './configure' and make.
I think that
- With either approach, the confidence to any change in gnulib-tool will be
the same.
- With approach (A), when we make a change to gnulib-tool, we need to commit
new expected test results, which is quite easy. No effort otherwise.
- With approach (B), we will get failures for other reasons as well: when
a gnulib module has changed in an incompatible way; when the git repository
of P has moved; when package P itself is broken. Sounds like a continuous
effort to hunt down (mostly) false positives.
Bruno
Re: planning for beta-testing gnulib-tool.py
Bruno Haible writes: > 5) Possibly it makes also sense to allow GNULIB_TOOL_IMPL to be set to >'sh+py'. In this case the script will make a full copy of the destination >dir, run the shell implementation and the Python implementation on the >two destination dirs, separately, and compare the results (again, both >in terms of effects on the file system, as well as standard output). >And err out if they are different. Generally I'm happy to hear about speedupds of gnulib-tool! The plan sounds fine. I think this step 5) is an important part to get maintainers try the new implementation, and report failures that needs to be looked into. If there was a small recipe I can follow to get a diff that can be reported back, I would run it for a bunch of projects that I contribute to. While a self-test suite for gnulib-tool would be nice, some real regression testing by attempting to build a bunch of real-world projects that rely on gnulib-tool may be simpler to realize. If there is a CI/CD that builds ~30 different real-world projects (perhaps at known-good commits) and compares the output against an earlier known-good build, for each modification to gnulib-tool in gnulib, that would give good confidence to any change to gnulib-tool. /Simon signature.asc Description: PGP signature
