Hi, I'm including Helmut, the submitter to the Debian bug report, as a CC, in case he can help provide additional context that I'm missing.
On Thu, Sep 25, 2014 at 09:07:59PM -0400, Andy Dougherty wrote: > On Thu, Sep 25, 2014 at 10:16:32PM +0100, Dominic Hargreaves wrote: > > Hi, > > > > The Debian project has recently received a bug report indicating that > > the perl source package we distribute does not comply the the DFSG[1] > > since the preferred form of modification of Configure is not provided. > > You can see the bug report at [2]. > > I'd quibble with that -- though I am no longer the primary Configure > maintainer, I actually did prefer most users to directly modify Configure. > It ended up being much easier than supporting users using metaconfig to > generate Configure. Users are of course welcome to fetch and install > metaconfig and use it, but I wouldn't say that was the "preferred" > form. I don't have the exact details, but Helmut suggests that this wasn't the case for him, as the changes would have had to be made in several places. In general I think a healthy free software project should aim to minimise cases where a casual hacker would be discouraged from doing things the same way a seasoned developer would. > Generating Configure requires both the 'dist' package (which contains > metaconfig) and the perl-specific metaconfig units. Historically, we have > often ended up using a patched 'dist', or a version that lagged behind > the main package. We definitely can't simply assume the version in the > Debian archive will work. Generating Configure also sometimes involves > patching or fixing the output of metaconfig in various ways. All the > information to do this should be in perl's metaconfig git repository. > > It is also sometimes difficult to check or merge the generated output. > The metaconfig units are assembled in order based on dependencies, > but those dependencies don't fully constrain the order. The result is > that the order of units can vary depending on who generates Configure. > Thus even a tiny patch to a single metaconfig unit can result in a > Configure patch with hundreds of lines that are just shuffling > parts around. > > That's another reason why modifying the metaconfig units is not > necessarily the "preferred" form of modification of Configure. > > Obviously, it's all a bit involved, and a fair amount of effort to > figure out. > > The way it's set up now, we encourage people to simply patch Configure. > If someone wants to go the metaconfig route instead, it's a lot of > extra work, but presumably the person deliberately chooses that route > knowing it's extra work. Again, there is an ideological point here. It *shouldn't* be a lot of extra work to do things in the way that upstream developers would. Clearly perl isn't going to be kicked out of Debian because of this, but a less important package might well be. > If we include all the units in the source tree > instead, I think end users will have an expectation of greater support. > They will expect it to work, and work easily, right out of the box. > Getting it to that point will require significant effort, and I really > don't see the benefit. Of course, if someone else wants to do it and > support it, that's fine with me. One benefit would be that we wouldn't this acknowledged bus factor issue with the metaconfig wrangling. > > If we have to, we can make a tarball of that repository and include > > it with the perl source package, but it seems like it would we should > > explore the possibility of fixing this discrepancy upstream, rather > > than working around it in Debian. In fact, it's likely we'd also have > > to supply the patches between the current metaconfig output and what's > > actually in the perl release tarball, since Configure is explicitly > > allowed to be patched even though it's generated. > > To be complete, you would have to include both perl's metaconfig > repository and an appropriately patched 'dist' source tree, which would > likely differ somewhat from Debian's existing 'dist' tree. If the > intent is to actually use it, there are probably a bunch of installation > issues to work out. If you want to check your results, you'll have to > confront the reshuffling problem. > Short summary: I think it would be a lot of work for very little > gain. On Fri, Sep 26, 2014 at 08:34:58AM +0200, H.Merijn Brand wrote: > The main problem with generating Configure from meta is not the > availability of the metaunits used for perl, but the way meta/dist has > to be installed to make them work the way it does. Do you mean the fact that dist is being patched, or some other reason? > As Andy already stated, it proves to be much easier to "backport" the > direct patches to Configure and friends to meta than it is to instruct > (new) users to patch metaunits the way it should. Even seasoned users > seem to have problems doing so. I had two sessions past year explaining > the ways and woes of how it works, sincerely hoping to get fresh blood > into the configuration process. The bus factor currently is about 1½ > > If I stop maintaining perl, only Andy understands the *complete* > process right now, and Jarkko would be able to (reluctantly) step in. > (Andy still understands some of the steps way better than I do). I did > try to improve the docs, and I also made the meta/dist *as I use it* > available on my CPAN, but it still is so much different a process from > the rest of perl5 core development that it does not attrackt (new) > people to join in: it is not a focus area that attracks youngsters, as > NO END USER will see what your work involves, while when you improve > the regular expression engine, the whole of the perl user community > will notice. As noted above, these sound like excellent reasons to make it so that the process is understood by the developer community at large rather than just a handful of individuals. I know it's easier to say that rather than to do it (particularly from someone who doesn't yet understand all the complexities). > As the current maint for this process, I communicate with the > maintainer of dist/meta on a rather regular basis. I give feedback of > what we changed, and I look at the changes he makes to see if those > will be usable for perl. > > As part of the metaunits is still used from dist, part is a set of > (slightly) modified units from dist, and part is the units written only > for perl, keeping those in sync is a hell of a job. On simply cannot > stay in sync 100% Okay, so clearly from a pragmatic view we would need to ship our own version of dist along with the rest of the stuff from the metaconfig repository. Depressingly this violates another part of Debian policy relating to embedding copies of code not intended to be embedded, but the freedom to modify the code using the preferred form clearly overrides that in my mind. > > It is also sometimes difficult to check or merge the generated output. > > The metaconfig units are assembled in order based on dependencies, > > but those dependencies don't fully constrain the order. The result is > > that the order of units can vary depending on who generates Configure. > > Even if you rsync from one system to another and use exactly the same > tools, the order can change based on locale setting :( > > > Thus even a tiny patch to a single metaconfig unit can result in a > > Configure patch with hundreds of lines that are just shuffling > > parts around. > > > > That's another reason why modifying the metaconfig units is not > > necessarily the "preferred" form of modification of Configure. > > If possible, modify the hint files instead. So, one data point is whether Helmut's changes could in fact have been achieved this way instead. > > Obviously, it's all a bit involved, and a fair amount of effort to > > figure out. > > > > The way it's set up now, we encourage people to simply patch Configure. > > If someone wants to go the metaconfig route instead, it's a lot of > > extra work, but presumably the person deliberately chooses that route > > knowing it's extra work. If we include all the units in the source tree > > instead, I think end users will have an expectation of greater support. > > One extra note here: the generation process decides what units to > include based on the references used in the perl source tree. This > means that not all units will be used. Including all units in a > distribution might lead to extra confusion. Could you expand on this a bit more? It's not obvious to me what this means. Aside from the 'promise of support' issue, this seems to be the main reason against moving the units into the main perl source, which would itself be a huge step forward. > > Short summary: I think it would be a lot of work for very little > > gain. > > 100% agree > > I'd rather put my free time in trying to get closer to dist-4.0 than we > are now (3.5+) than modifying the current process to enable > distribution of something that would work for all users that are > unlikely to use it anyway I'm genuinely confused here. Isn't making the perl build infrastructure less dependent on a small number of people a significant gain, even if you don't care about the ideological importance of the 'preferred form of modification' requirement from Debian? On Mon, Sep 29, 2014 at 01:33:41PM -0400, Andy Dougherty wrote: > > One final follow-up: Debian is certainly welcome to distribute the > files used to build Configure. The following two items would be needed: > > 1. The precise version of dist used. This can be obtained here: > > > http://www.cpan.org/modules/by-authors/id/H/HM/HMBRAND/perl-meta-3.5-20.tgz Incidentally, this should be mentioned in the comments at the top of Configure, rather than the misleading reference to the 'normal' version of dist. > 2. The custom metaconfig units used. These are available here: > > git://perl5.git.perl.org/metaconfig.git > > Rebuilding Configure would not be easy or automatic, but all the necessary > files would be available. Would that satisfy the Debian guidelines? I think it would satisfy the letter of the law, if not the full spirit. It's a starting point; I'd like it to not be the ending point, but I guess I would need to convince you and/or H.Merijn Brand of this in order to make significant progress. Having to pull in extra sources is a bit of a pain for the Debian package management, but it's doable. The main thing that seems to be missing is a way of knowing what version of that repository is to be used (or was used) for a given release of perl. That doesn't seem to be recorded in either the metaconfig repository or the perl repository. The other option open to us is to convince Debian that the metaconfig units genuinely are external bits of tooling that aren't part of the build process, and that Configure really is the preferred form of modification. Working through Helmut's use case might help there. Thank you both for taking the time to work through some of these issues with me/Debian (and I see that things are already better in that the version of dist used is now readily available). Cheers, Dominic. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
