Bug#797831: glibc: further problems with stage1
Hi Aurelien, On Tue, Sep 08, 2015 at 07:58:13PM +0200, Aurelien Jarno wrote: > Thanks for the patch and the detailed explanation. The changes make sense, > so I have applied the patch. Thank you. > That said looking as this part of the code as a whole, it ends up being a > bit complicated. Basically we define defaults value before the case, but > then we basically handle all cases. Then we use a for loop as a if, as > $templates contains either zero or one value. I concur with that observation. > The complexity comes from the fact this piece of code has been forked from > the non-staged one. I therefore wonder if we should try to either: > 1) Simplify it. > 2) make it as common as possible with the non-stage code. I believe it's > not a problem if we generate debhelper files that we don't use in > practice, as long as the stage ones are correct. Maybe we should try 3) merge it into the non-stage code? Having these two cases separate has the disadvantage that they will go out of sync as the non-staged variant is adapted to the current needs. We should strive for minimal differences in stage1 to minimize its maintenance cost. So what actually is the difference between these two implementations of the debhelper-common target? If I am not mistaken it is: * Generate fewer debhelper templates. As you observed there is no harm in just generating them anyway. * Interpolate more variables, in particular RTLD_SO. They are not used in the libc-dev templates. The computation of the shell variable rtld_so would probably result in garbage as debian/tmp-*/usr/bin/iconv will be missing, but maybe it still succeeds (from an exit code pov). * Remove lines containing LIBDIR in stage1. Unless I am missing something, this is the only relevant difference. So maybe one could have something along the lines. ifeq ($(filter stage1,$(DEB_BUILD_PROFILES)),) STAGE1_TEMPLATE_FILTER= else STAGE1_TEMPLATE_FILTER=sed -e "/LIBDIR.*\.a /d" $$t fi and then add "$(STAGE1_TEMPLATE_FILTER);" to the pile of sed invocations in the non-stage1 code path while eliminating the stage1 block altogether. Do you think this idea would be worth trying and preparing a proper patch? Do you have a better name for that strange make variable? Helmut
Bug#797831: glibc: further problems with stage1
On 2015-09-08 20:44, Helmut Grohne wrote: > Hi Aurelien, > > On Tue, Sep 08, 2015 at 07:58:13PM +0200, Aurelien Jarno wrote: > > Thanks for the patch and the detailed explanation. The changes make sense, > > so I have applied the patch. > > Thank you. > > > That said looking as this part of the code as a whole, it ends up being a > > bit complicated. Basically we define defaults value before the case, but > > then we basically handle all cases. Then we use a for loop as a if, as > > $templates contains either zero or one value. > > I concur with that observation. > > > The complexity comes from the fact this piece of code has been forked from > > the non-staged one. I therefore wonder if we should try to either: > > 1) Simplify it. > > 2) make it as common as possible with the non-stage code. I believe it's > > not a problem if we generate debhelper files that we don't use in > > practice, as long as the stage ones are correct. > > Maybe we should try 3) merge it into the non-stage code? Having these > two cases separate has the disadvantage that they will go out of sync as > the non-staged variant is adapted to the current needs. We should strive > for minimal differences in stage1 to minimize its maintenance cost. Your option 3), was actually the step I wanted to do after option 2) ;-) > So what actually is the difference between these two implementations of > the debhelper-common target? If I am not mistaken it is: > * Generate fewer debhelper templates. As you observed there is no harm >in just generating them anyway. > * Interpolate more variables, in particular RTLD_SO. They are not used >in the libc-dev templates. The computation of the shell variable >rtld_so would probably result in garbage as >debian/tmp-*/usr/bin/iconv will be missing, but maybe it still >succeeds (from an exit code pov). > * Remove lines containing LIBDIR in stage1. Unless I am missing >something, this is the only relevant difference. > > So maybe one could have something along the lines. > > ifeq ($(filter stage1,$(DEB_BUILD_PROFILES)),) > STAGE1_TEMPLATE_FILTER= > else > STAGE1_TEMPLATE_FILTER=sed -e "/LIBDIR.*\.a /d" $$t > fi You can even get rid of the call to sed there and just set directly STAGE1_TEMPLATE_FILTER=-e "/LIBDIR.*\.a /d" $$t so that it can be used in the same sed invocation. OTOH I don't think we really care about performance > and then add "$(STAGE1_TEMPLATE_FILTER);" to the pile of sed invocations > in the non-stage1 code path while eliminating the stage1 block > altogether. Do you think this idea would be worth trying and preparing a > proper patch? Yes I think it would be a good idea. > Do you have a better name for that strange make variable? I think the name is fine. Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://www.aurel32.net
Bug#797831: glibc: further problems with stage1
On 2015-09-02 23:19, Helmut Grohne wrote: > Source: glibc > Version: 2.21-0experimental1 > Tags: patch > User: helm...@debian.org > Usertags: rebootstrap > > Hi Aurelien, Hi, > thank you very much for applying #766877 and uploading to experimental. > This has moved us a big step closer to a working stage1. We are not > quite there yet. At this point I estimate a remaining patch stack for > the following problems: > > * stage1 fails to build for various reasons > * stage1 libc6-dev not installable due to dependency on libc6 > * wrong set of packages being built for stage1 > * dh_shlibdeps fails > * linux headers cannot be found > * various hurd things > > Even though I still carry patches for these, it is not clear that all of > these problems are still reproducible. The above list is meant as an > outlook, not a cumulative bug report. > > This particular bug shall address only the first of those problems > above, because I have a good understanding and can answer your > questions. I am attaching a patch and also explain the individual hunks > in what follows. All of them apply to stage1-specific code. > > - *:/lib32 | *:/lib64 | *:/libx32 | *:/lib/arm-linux-gnueabi*) \ > + *:/lib32 | *:/lib64 | *:/libo32 | *:/libx32 | > *:/lib/arm-linux-gnueabi*) \ > > The code fails to identify a certain mips architecture multilib build > and thus places the multilib build into the main package. > > + *:* ) \ > + templates="" \ > + ;; \ > > This extra case ensures that no templates are interpolated for optimized > builds (e.g. libc6-i686). These do not generate development packages > anyway, so dropping them (for stage1) is the right thing to do. Failing > to do so again overwrites the main package. > > - -e "/$$libdir.*.a /d" \ > + -e "/LIBDIR.*\.a /d" \ > > The immediate error resulting from this sed invocation is that the > command "u" is not understood by sed. $libdir becomes > "/usr/lib/triplet". Thus the resulting sed command starts with "//u" > which sed does not like. Fixing the escaping is not enough however, > since LIBDIR is not yet interpolated. That only happens a few lines > later. So instead, the match needs to target non-interpolated filenames > and thus match LIBDIR. > > I hope that the level of detail given is sufficient. If not, please ask. Thanks for the patch and the detailed explanation. The changes make sense, so I have applied the patch. That said looking as this part of the code as a whole, it ends up being a bit complicated. Basically we define defaults value before the case, but then we basically handle all cases. Then we use a for loop as a if, as $templates contains either zero or one value. The complexity comes from the fact this piece of code has been forked from the non-staged one. I therefore wonder if we should try to either: 1) Simplify it. 2) make it as common as possible with the non-stage code. I believe it's not a problem if we generate debhelper files that we don't use in practice, as long as the stage ones are correct. > Otherwise, please consider applying the patch. I would appreciate > another experimental upload that also includes the hurd changes staged > to SVN by Samuel Thibault already within a month from now. Thank you for > your support. For what I understood, Samuel has committed is change in both the unstable and experimental branches, so the changes should have been in the last upload to experimental. That said he has done a few more changes in the meantime. Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://www.aurel32.net
Bug#797831: glibc: further problems with stage1
Source: glibc Version: 2.21-0experimental1 Tags: patch User: helm...@debian.org Usertags: rebootstrap Hi Aurelien, thank you very much for applying #766877 and uploading to experimental. This has moved us a big step closer to a working stage1. We are not quite there yet. At this point I estimate a remaining patch stack for the following problems: * stage1 fails to build for various reasons * stage1 libc6-dev not installable due to dependency on libc6 * wrong set of packages being built for stage1 * dh_shlibdeps fails * linux headers cannot be found * various hurd things Even though I still carry patches for these, it is not clear that all of these problems are still reproducible. The above list is meant as an outlook, not a cumulative bug report. This particular bug shall address only the first of those problems above, because I have a good understanding and can answer your questions. I am attaching a patch and also explain the individual hunks in what follows. All of them apply to stage1-specific code. - *:/lib32 | *:/lib64 | *:/libx32 | *:/lib/arm-linux-gnueabi*) \ + *:/lib32 | *:/lib64 | *:/libo32 | *:/libx32 | *:/lib/arm-linux-gnueabi*) \ The code fails to identify a certain mips architecture multilib build and thus places the multilib build into the main package. + *:* ) \ + templates="" \ + ;; \ This extra case ensures that no templates are interpolated for optimized builds (e.g. libc6-i686). These do not generate development packages anyway, so dropping them (for stage1) is the right thing to do. Failing to do so again overwrites the main package. - -e "/$$libdir.*.a /d" \ + -e "/LIBDIR.*\.a /d" \ The immediate error resulting from this sed invocation is that the command "u" is not understood by sed. $libdir becomes "/usr/lib/triplet". Thus the resulting sed command starts with "//u" which sed does not like. Fixing the escaping is not enough however, since LIBDIR is not yet interpolated. That only happens a few lines later. So instead, the match needs to target non-interpolated filenames and thus match LIBDIR. I hope that the level of detail given is sufficient. If not, please ask. Otherwise, please consider applying the patch. I would appreciate another experimental upload that also includes the hurd changes staged to SVN by Samuel Thibault already within a month from now. Thank you for your support. Helmut diff -Nru glibc-2.19/debian/rules.d/debhelper.mk glibc-2.19/debian/rules.d/debhelper.mk --- glibc-2.19/debian/rules.d/debhelper.mk +++ glibc-2.19/debian/rules.d/debhelper.mk @@ -197,10 +197,13 @@ case "$$curpass:$$slibdir" in \ libc:*) \ ;; \ - *:/lib32 | *:/lib64 | *:/libx32 | *:/lib/arm-linux-gnueabi*) \ + *:/lib32 | *:/lib64 | *:/libo32 | *:/libx32 | *:/lib/arm-linux-gnueabi*) \ pass="-alt" \ suffix="-$(curpass)" \ ;; \ + *:* ) \ + templates="" \ + ;; \ esac ; \ for t in $$templates ; do \ for s in debian/$$t$$pass.* ; do \ @@ -219,7 +219,7 @@ cp $$s $$t ; \ fi ; \ sed -i \ - -e "/$$libdir.*.a /d" \ + -e "/LIBDIR.*\.a /d" \ -e "s#TMPDIR#debian/tmp-$$curpass#g" \ -e "s#RTLDDIR#$$rtlddir#g" \ -e "s#SLIBDIR#$$slibdir#g" \
