Bug#876191: most: configure cannot be built from source
Control: tags -1 + fixed-upstream * Helmut Grohne , 2017-09-19 14:27: I was trying to fix a bug in most that requires modifying configure. Thus I tried to regenerate it and ... failed. It seems that configure was generated with autoconf2.61. The latest upstream release (v5.2.0) uses Autoconf 2.69. -- Jakub Wilk
Bug#876191: most: configure cannot be built from source
severity 876191 minor retitle 876191 most: configure file cannot be regenerated automatically thanks Greetings! For the reasons discussed on this bug already, I'm retitling this bug and reducing its severity. I know Helmut doesn't agree. If we want to have a bigger conversation about policy on -legal or -policy or something, please keep me in the loop! Otherwise, we both agree that the first order of business should be closing this bug! I hope to getto that Real Soon but patches (even NMUs to fix the issue) are always, always, always welcome. Later, Mako -- Benjamin Mako Hill http://mako.cc/academic/ Creativity can be a social contribution, but only in so far as society is free to use the results. --GNU Manifesto signature.asc Description: PGP signature
Bug#876191: most: configure cannot be built from source
> On Sat, Nov 25, 2017 at 12:36:33PM -0800, Benj. Mako Hill wrote: > > I agree that a hand-modified binary a binary would not mean that you > > don't need to provide source for that binary. I think there's likely > > going to be consensus on that in Debian. > > That put's you in a difficult spot as to where to draw the line. Drawing lines on these sort of issues involves judgment calls—often difficult ones. We're not going to avoid that. > > To say that DFSG#3 is moot (i.e., that you /can not/ make > > modifications or derived versions) seems to me to be an > > exaggeration. It is more difficult than it might be if the software > > and build scripts were created in a different way. But that's not > > necessary a DFSG issue. Writing your software in a obscure programming > > language makes it harder to modify as well and that is obviously > > allowed. > > DFSG #3 is about enabling users. It doesn't really matter if users > refrain from modifying due to licensing or due to being > impractical. The end result is, that the purported freedom hasn't > transferred into reality. We agree on this. It should also be clear that we allow some amount of annoyance, difficulty, and practicality in modifying software in that we allow software in allow advertising clauses, esoteric programming languages, single character variable names, no comments, etc. We also, generally speaking, have allowed software that involves hand-modified versions of auto-generated build scripts and other code. > > Are you willing to say that source code can never contain code > > that was partially generated by another program that is provided? > > Even if it is generated by computers and then modified by hand? > > That's a much stronger position than I think is supportable by any > > reading of the DFSG than I've ever heard and it would eliminate > > many hundreds or even thousands of packages from Debian. > > That's a tough question indeed. Half a year ago, the strict answer > would have eliminated nothing less than perl (#762638). Even though > the Debian perl maintainers were heavily patching the generated > file, they are now generating it at build time. That's still just a bug about an autoconf-generated configure script. I was asking a much bigger question. Much of what we would both agree is the source code of many /programs/ (like the .c files or whatever) is at least partially generated by programs. How often to programmers create keyboard macros to autogenerate code? How often do they include them? > To evade answering it, I try to work from the impact. Whenever > fixing a bug is impeded by the inability to regenerate build > artifacts, I file a bug, because it clearly demonstrates that the > freedom to modify is limited here. I agree that this is the right approach. But that bug is still to have one severity or another and I'm still not convinced that this is a serious DFSG issue. > > This is distinctly different. The source code in the JS example is > > obviously not minified the Javascript if we use the GPL's "preferred > > form for modification" heuristic regardless of the change in > > question. If upstream wanted to make a change to the Javascript in > > their program, they would almost never use the minified version. If > > most's upstream wanted to make a change to their configure file, they > > would likely modify the pre-built version. Or they would go through > > rebuilding it again. > > Most commonly, Javascript is not copy-left, but something like MIT. > Downstream projects commonly embed minified, embedded copies and just > update them whenever convenient. So yes, some upstreams (e.g. a past > Doxygen) do prefer to deal with minified Javascript. The GPL defines source as preferred form for modification. Upstreams may prefer to just use/update their minified Javascript with a new one but there is no way athat the minified version is the version nearly anybody would turn to if they wanted to make a modification. It's nearly impossible to do so! Again, it's a judgment call. But if it came to a copyright case someone would have to make the argument in front of a judge that the minified version was the preferred form for modification. It would not be hard to find expert witnesses to convince any reasonable judge otherwise. > > I agree! But I think that having packages removed from testing (as > > has now happened to most) over this interpretation is an > > overreaction to what I think constitutes an annoyance. > > The removal happened due to not responding to the bug. You can defer > automatic removals indefinitely by continuously mailing the bug. I'm sure that neither one of us thinks that this is an actually solution. Either there's a serious bug, and it should be marked as much, or there's not. > Do you think DFGS #2 would be a better justification for the > severity? Yes. I think it's much more plausible but I still don't think it applies for reasons I've described here and that you've not talked me
Bug#876191: most: configure cannot be built from source
On Sat, Nov 25, 2017 at 12:36:33PM -0800, Benj. Mako Hill wrote: > I agree that a hand-modified binary a binary would not mean that you > don't need to provide source for that binary. I think there's likely > going to be consensus on that in Debian. That put's you in a difficult spot as to where to draw the line. > I also think this situation is different because I think that a > configure file is more like computer-generated, hand-modified, > /source/ than like a binary. What makes configure source? It doesn't come with indentation or things that make normal source code human-consumable. Even though shell scripting supports scoped variables, autoconf deliberately avoids using them. Thus they are much closer to named, global memory locations. Generated configure scripts are much closer to a disassembly of a binary. > I think one could also argue that there is an difference between the > program binary and a build script which is used to build the package > in question. So do you argue that you can use a pre-built binary during build, but you cannot ship in the binary package? > To say that DFSG#3 is moot (i.e., that you /can not/ make > modifications or derived versions) seems to me to be an > exaggeration. It is more difficult than it might be if the software > and build scripts were created in a different way. But that's not > necessary a DFSG issue. Writing your software in a obscure programming > language makes it harder to modify as well and that is obviously > allowed. DFSG #3 is about enabling users. It doesn't really matter if users refrain from modifying due to licensing or due to being impractical. The end result is, that the purported freedom hasn't transferred into reality. By your very same argument you could ship binaries, because you can simply disassemble them, edit them and reassemble. It might be a bit more difficult, but if the binary was liberally licensed would that prevent you from doing so? > Are you willing to say that source code can never contain code that > was partially generated by another program that is provided? Even if > it is generated by computers and then modified by hand? That's a much > stronger position than I think is supportable by any reading of the > DFSG than I've ever heard and it would eliminate many hundreds or even > thousands of packages from Debian. That's a tough question indeed. Half a year ago, the strict answer would have eliminated nothing less than perl (#762638). Even though the Debian perl maintainers were heavily patching the generated file, they are now generating it at build time. To evade answering it, I try to work from the impact. Whenever fixing a bug is impeded by the inability to regenerate build artifacts, I file a bug, because it clearly demonstrates that the freedom to modify is limited here. > This is distinctly different. The source code in the JS example is > obviously not minified the Javascript if we use the GPL's "preferred > form for modification" heuristic regardless of the change in > question. If upstream wanted to make a change to the Javascript in > their program, they would almost never use the minified version. If > most's upstream wanted to make a change to their configure file, they > would likely modify the pre-built version. Or they would go through > rebuilding it again. Most commonly, Javascript is not copy-left, but something like MIT. Downstream projects commonly embed minified, embedded copies and just update them whenever convenient. So yes, some upstreams (e.g. a past Doxygen) do prefer to deal with minified Javascript. > I agree! But I think that having packages removed from testing (as has > now happened to most) over this interpretation is an overreaction to > what I think constitutes an annoyance. The removal happened due to not responding to the bug. You can defer automatic removals indefinitely by continuously mailing the bug. Do you think DFGS #2 would be a better justification for the severity? The configure script in question was generated with autoconf 2.61. This version of autoconf has been removed from Debian. Now, most is missing the source for regenerating configure. Compare this to bumping gcc versions. When a package fails to build with a more recent gcc, it receives a FTBFS bug as well. You can temporarily avoid them by explicitly selecting a lower version, but when that version gets removed, such a package inevitably becomes RC buggy. The only difference here is that by not rebuilding from source at build time, you get the RC bug with a delay as has happened here. That's why debhelper 10 autoreconfs by default. I believe that the inability to fix bugs is a serious issue. Helmut
Bug#876191: most: configure cannot be built from source
Greetings Helmut! Thanks for engaging productively on this! > I deliberately avoided the FTBFS language, because it is not a FTBFS. > It's different. It's like shipping a binary that cannot be regenerated > and using that during build. The term FTBFS is well defined and does not > cover this case. Got it. We agree about that. I'm don't think we agree on what constitutes the source for this package. Until we do, I think a more descriptive title might be better. Something like: "configure file cannot be regenerated automatically." > Would you say the same if you compile a binary, use a hex editor to fix > the binary and then ship the binary in your source package? I mean you > preferred to edit the binary with a hex editor, so wouldn't it be > preferred form for modification? I agree that a hand-modified binary a binary would not mean that you don't need to provide source for that binary. I think there's likely going to be consensus on that in Debian. I also think this situation is different because I think that a configure file is more like computer-generated, hand-modified, /source/ than like a binary. I think one could also argue that there is an difference between the program binary and a build script which is used to build the package in question. > Also consider that autoconf projects tend to ship embedded copies of > .m4 files. A bug in those .m4 files is often fixed upstream. Fixing > it could be a simple matter of updating the embedded copy if one > could regenerate configure. > > In both of these examples, I very much do not prefer working with > the generated configure as it feels very much like editing a binary > with a hex editor. Thus I argue that configure is not the preferred > form for modification. Shipping only configure makes DFGS #3 > practically moot. To say that DFSG#3 is moot (i.e., that you /can not/ make modifications or derived versions) seems to me to be an exaggeration. It is more difficult than it might be if the software and build scripts were created in a different way. But that's not necessary a DFSG issue. Writing your software in a obscure programming language makes it harder to modify as well and that is obviously allowed. Are you willing to say that source code can never contain code that was partially generated by another program that is provided? Even if it is generated by computers and then modified by hand? That's a much stronger position than I think is supportable by any reading of the DFSG than I've ever heard and it would eliminate many hundreds or even thousands of packages from Debian. > > If this has been discussed elsewhere or if there is Debian policy > > on this I don't know about, I'd appreciate being pointed to this > > and I'm happy to defer to this. In the meantime, I'd appreciate > > help fixing this issue. I'm not likely to have bandwidth for a few > > more weeks. > > I guess we should document this issue class centrally. It is similar > to the javascript bundling issue (where people suppose that minified > or concatenated javascript files could be considered source). This is distinctly different. The source code in the JS example is obviously not minified the Javascript if we use the GPL's "preferred form for modification" heuristic regardless of the change in question. If upstream wanted to make a change to the Javascript in their program, they would almost never use the minified version. If most's upstream wanted to make a change to their configure file, they would likely modify the pre-built version. Or they would go through rebuilding it again. > This issue comes about every time Debian is bootstrapped for a new > architecture as configure tends to have bugs (e.g. ppc64el). I'm > seeing it now as I bootstrap Debian for something like a new > architecture (cross building). So this is the class of bugs to watch > out. I agree! But I think that having packages removed from testing (as has now happened to most) over this interpretation is an overreaction to what I think constitutes an annoyance. I think that the main goal should be focusing on trying to fix these bugs. I think a secondary goal should be discussing this in a systematic way to get some sort of consensus. Until then, my sense is to reduce the severity of this (likely to normal) and to retitle the bug as described above. As I said, I think this is a real bug. I just don't agree that it's a showstopper or a DFSG issue. Regards, Mako -- Benjamin Mako Hill http://mako.cc/ Creativity can be a social contribution, but only in so far as society is free to use the results. --GNU Manifesto signature.asc Description: PGP signature
Bug#876191: most: configure cannot be built from source
On Fri, Nov 24, 2017 at 06:22:05PM -0800, Benj. Mako Hill wrote: > I'll also say that I'm skeptical about both the severity you've chosen > here (serious), about describing this as a FTBFS issue, and about your > suggestion that this is a DFSG issue. I deliberately avoided the FTBFS language, because it is not a FTBFS. It's different. It's like shipping a binary that cannot be regenerated and using that during build. The term FTBFS is well defined and does not cover this case. > If there is a serious DFSG issue here, it's not a FTBFS issue but > rather a question of whether or not the source package contains the > full source in the first place. I'm open to being convinced otherwise > but I think it does. I do believe that this is a DFSG issue. > If I generate a configure file with autoconf and then modify it by > hand in order to create a working build script, I don't I've somehow > made it impossible to provide source for that package. I think I'm > still providing the preferred form of modification (as the GPL defines > source). I agree that that situation is brittle and bad and should be > avoided. But I don't think it's a DFSG issue. Would you say the same if you compile a binary, use a hex editor to fix the binary and then ship the binary in your source package? I mean you preferred to edit the binary with a hex editor, so wouldn't it be preferred form for modification? Now consider my case. Most commonly, I want to change AC_RUN_IFELSE into AC_TRY_COMPILE or AC_COMPUTE_INT. That's a very simple change on configure.ac that produces a big diff on the generated configure. Also consider that autoconf projects tend to ship embedded copies of .m4 files. A bug in those .m4 files is often fixed upstream. Fixing it could be a simple matter of updating the embedded copy if one could regenerate configure. In both of these examples, I very much do not prefer working with the generated configure as it feels very much like editing a binary with a hex editor. Thus I argue that configure is not the preferred form for modification. Shipping only configure makes DFGS #3 practically moot. > If this has been discussed elsewhere or if there is Debian policy on > this I don't know about, I'd appreciate being pointed to this and I'm > happy to defer to this. In the meantime, I'd appreciate help fixing > this issue. I'm not likely to have bandwidth for a few more weeks. I guess we should document this issue class centrally. It is similar to the javascript bundling issue (where people suppose that minified or concatenated javascript files could be considered source). A very similar bu is #874261. #882538 could be called related. This issue comes about every time Debian is bootstrapped for a new architecture as configure tends to have bugs (e.g. ppc64el). I'm seeing it now as I bootstrap Debian for something like a new architecture (cross building). So this is the class of bugs to watch out. Helmut
Bug#876191: most: configure cannot be built from source
Greetings Helmut! > I was trying to fix a bug in most that requires modifying configure. > Thus I tried to regenerate it and ... failed. I'll start by saying that this is a real bug and that I agree that it should be fixed. And thanks so much for notice and submitting it! And for trying to fix the other issue! I'll also say that I'm skeptical about both the severity you've chosen here (serious), about describing this as a FTBFS issue, and about your suggestion that this is a DFSG issue. After all, the package still builds from its source and I think we have everything that upstream has. If there is a serious DFSG issue here, it's not a FTBFS issue but rather a question of whether or not the source package contains the full source in the first place. I'm open to being convinced otherwise but I think it does. If I generate a configure file with autoconf and then modify it by hand in order to create a working build script, I don't I've somehow made it impossible to provide source for that package. I think I'm still providing the preferred form of modification (as the GPL defines source). I agree that that situation is brittle and bad and should be avoided. But I don't think it's a DFSG issue. If this has been discussed elsewhere or if there is Debian policy on this I don't know about, I'd appreciate being pointed to this and I'm happy to defer to this. In the meantime, I'd appreciate help fixing this issue. I'm not likely to have bandwidth for a few more weeks. Regards, Mako -- Benjamin Mako Hill http://mako.cc/ Creativity can be a social contribution, but only in so far as society is free to use the results. --GNU Manifesto signature.asc Description: PGP signature
Bug#876191: most: configure cannot be built from source
Source: most Version: 5.0.0a-4 Severity: serious User: helm...@debian.org Usertags: rebootstrap I was trying to fix a bug in most that requires modifying configure. Thus I tried to regenerate it and ... failed. It seems that configure was generated with autoconf2.61. The archive does have autoconf 2.13, 2.64 and 2.69, but not 2.61. So I tried autoconf 2.64 as that seemed closest. However autoheader2.64 failed: | configure.ac:4: warning: AC_REQUIRE: `AC_PROG_CC' was expanded before it was required | configure.ac:4: http://www.gnu.org/software/autoconf/manual/autoconf.html#Expanded-Before-Required | ../../lib/autoconf/c.m4:429: AC_LANG_COMPILER(C) is expanded from... | ../../lib/autoconf/lang.m4:329: AC_LANG_COMPILER_REQUIRE is expanded from... | ../../lib/autoconf/lang.m4:372: AC_LANG_PREPROC_REQUIRE is expanded from... | ../../lib/autoconf/general.m4:2549: AC_EGREP_CPP is expanded from... | ../../lib/m4sugar/m4sh.m4:639: AS_IF is expanded from... | ../../lib/autoconf/general.m4:2042: AC_CACHE_VAL is expanded from... | ../../lib/autoconf/general.m4:2063: AC_CACHE_CHECK is expanded from... | ../../lib/autoconf/c.m4:543: AC_PROG_GCC_TRADITIONAL is expanded from... | aclocal.m4:496: JD_ANSI_CC is expanded from... | configure.ac:4: the top level | configure.ac:4: warning: AC_REQUIRE: `AC_PROG_CPP' was expanded before it was required | configure.ac:4: http://www.gnu.org/software/autoconf/manual/autoconf.html#Expanded-Before-Required | ../../lib/autoconf/c.m4:336: AC_LANG_PREPROC(C) is expanded from... | ../../lib/autoconf/lang.m4:372: AC_LANG_PREPROC_REQUIRE is expanded from... | ../../lib/autoconf/general.m4:2549: AC_EGREP_CPP is expanded from... | ../../lib/m4sugar/m4sh.m4:639: AS_IF is expanded from... | ../../lib/autoconf/general.m4:2042: AC_CACHE_VAL is expanded from... | ../../lib/autoconf/general.m4:2063: AC_CACHE_CHECK is expanded from... | ../../lib/autoconf/c.m4:543: AC_PROG_GCC_TRADITIONAL is expanded from... | aclocal.m4:496: JD_ANSI_CC is expanded from... | configure.ac:4: the top level | configure.ac:4: warning: AC_REQUIRE: `AC_PROG_CC' was expanded before it was required | configure.ac:4: http://www.gnu.org/software/autoconf/manual/autoconf.html#Expanded-Before-Required | ../../lib/autoconf/c.m4:429: AC_LANG_COMPILER(C) is expanded from... | ../../lib/autoconf/lang.m4:329: AC_LANG_COMPILER_REQUIRE is expanded from... | ../../lib/autoconf/lang.m4:372: AC_LANG_PREPROC_REQUIRE is expanded from... | ../../lib/autoconf/general.m4:2549: AC_EGREP_CPP is expanded from... | ../../lib/m4sugar/m4sh.m4:639: AS_IF is expanded from... | ../../lib/autoconf/general.m4:2042: AC_CACHE_VAL is expanded from... | ../../lib/autoconf/general.m4:2063: AC_CACHE_CHECK is expanded from... | ../../lib/autoconf/c.m4:543: AC_PROG_GCC_TRADITIONAL is expanded from... | aclocal.m4:496: JD_ANSI_CC is expanded from... | configure.ac:4: the top level | configure.ac:4: warning: AC_REQUIRE: `AC_PROG_CPP' was expanded before it was required | configure.ac:4: http://www.gnu.org/software/autoconf/manual/autoconf.html#Expanded-Before-Required | ../../lib/autoconf/c.m4:336: AC_LANG_PREPROC(C) is expanded from... | ../../lib/autoconf/lang.m4:372: AC_LANG_PREPROC_REQUIRE is expanded from... | ../../lib/autoconf/general.m4:2549: AC_EGREP_CPP is expanded from... | ../../lib/m4sugar/m4sh.m4:639: AS_IF is expanded from... | ../../lib/autoconf/general.m4:2042: AC_CACHE_VAL is expanded from... | ../../lib/autoconf/general.m4:2063: AC_CACHE_CHECK is expanded from... | ../../lib/autoconf/c.m4:543: AC_PROG_GCC_TRADITIONAL is expanded from... | aclocal.m4:496: JD_ANSI_CC is expanded from... | configure.ac:4: the top level | configure.ac:4: warning: AC_REQUIRE: `AC_PROG_CC' was expanded before it was required | configure.ac:4: http://www.gnu.org/software/autoconf/manual/autoconf.html#Expanded-Before-Required | ../../lib/autoconf/c.m4:429: AC_LANG_COMPILER(C) is expanded from... | ../../lib/autoconf/lang.m4:329: AC_LANG_COMPILER_REQUIRE is expanded from... | ../../lib/autoconf/lang.m4:372: AC_LANG_PREPROC_REQUIRE is expanded from... | ../../lib/autoconf/general.m4:2549: AC_EGREP_CPP is expanded from... | ../../lib/m4sugar/m4sh.m4:639: AS_IF is expanded from... | ../../lib/autoconf/general.m4:2042: AC_CACHE_VAL is expanded from... | ../../lib/autoconf/general.m4:2063: AC_CACHE_CHECK is expanded from... | ../../lib/autoconf/c.m4:543: AC_PROG_GCC_TRADITIONAL is expanded from... | aclocal.m4:496: JD_ANSI_CC is expanded from... | configure.ac:4: the top level | configure.ac:4: warning: AC_REQUIRE: `AC_PROG_CPP' was expanded before it was required | configure.ac:4: http://www.gnu.org/software/autoconf/manual/autoconf.html#Expanded-Before-Required | ../../lib/autoconf/c.m4:336: AC_LANG_PREPROC(C) is expanded from... | ../../lib/autoconf/lang.m4:372: AC_LANG_PREPROC_REQUIRE is expanded from... | ../../lib/autoconf/general.m4:2549: AC_EGREP_CPP is expanded from... | ../../lib/m4sugar/m4sh.m4:639: AS_IF is expanded from... |
