Bug#1058678: blt: disable jpeg support
On Fri, Dec 15, 2023 at 01:43:12PM +0200, Adrian Bunk wrote: > The src:python3.x packages build idle-python3.x packages. > IDLE is an IDE for Python.[1] > > The idle-python3.x packages depend on python3-tk, which depends on blt. The idle-python3.x package is called python3.x-full. It happens to issue a runtime dependency on python3.x-tk which is provided from python3-tk which is built from python3-stdlib-extensions. Since idle is otherwise implemented in Python, one doesn't actually need tk to perform the install step for idle in the Python build. > There is a certain weirdness in Python shipping an IDE that is based > on Tcl/Tk, but that's an upstream issue. ... > The tk-dev/blt-dev build dependencies in src:python3.x: > - might be remnants from the times when python3-tk was built there, or This definitely is true in some sense. > - might be test-only, or I guess this is also true. > - might be workaroundable with a noidle build profile Given your pointer at pkg-config, I now performed another build comparison: * Add uuid-dev and pkg-config to Build-Depends. * Drop tk-dev and blt-dev from Build-Depends (in one of two builds). * Fix the build path. * Disable testing and profile guided optimization (DEB_BUILD_OPTIONS="nocheck nopgo"). The differences reduce significantly. I can no longer spot differences in the filenames. The sysconfigdata changes and around tkinter even though no tkinter module is installed into any package. LTO seems to still shuffle things around quite a bit. Diffing Python binary builds is hard. Helmut
Bug#1058678: blt: disable jpeg support
On Fri, Dec 15, 2023 at 11:20:09AM +0100, Helmut Grohne wrote: >... > The resulting artifacts > still differed a lot from an unmodified build and in particular, the > _uuid.so and _nis.so module went missing. The former probably is due to > something from tk or blt issuing a dependency on uuid-dev and with those > packages removed, uuid-dev was no longer pulled. I have not found a > probably cause for the nis module. >... Missing build dependency on pkg-config. > Helmut cu Adrian
Bug#1058678: blt: disable jpeg support
On Thu, Dec 14, 2023 at 06:22:45PM +0300, Sergei Golovan wrote: >... > Can someone investigate if blt is really necessary for Python now? > It's a very old > unmaintained piece of software, and I think that dependence of Python on it > should be a bug. >... The src:python3.x packages build idle-python3.x packages. IDLE is an IDE for Python.[1] The idle-python3.x packages depend on python3-tk, which depends on blt. There is a certain weirdness in Python shipping an IDE that is based on Tcl/Tk, but that's an upstream issue. The tk-dev/blt-dev build dependencies in src:python3.x: - might be remnants from the times when python3-tk was built there, or - might be test-only, or - might be workaroundable with a noidle build profile > Sergei Golovan cu Adrian [1] https://docs.python.org/3/library/idle.html
Bug#1058678: blt: disable jpeg support
Hi Sergei, On Thu, Dec 14, 2023 at 06:22:45PM +0300, Sergei Golovan wrote: > I'm sorry, I wasn't aware about these implications. Meanwhile, I'm > setting this bug > severity to serious and will fix it as soon as possible. Thank you. I agree that this is surprising at times. Hence, I'm looking after this. :) > > I suggest that you temporarily revert this change in unstable to restore > > the ability to bootstrap Debian from source. > > I'll do that. Thank you. > Can someone investigate if blt is really necessary for Python now? This is a very difficult question I fear. A very common approach for dropping dependencies is using reproducible builds as a QA tool: If the package builds and has the same output artifact with a dependency removed, it probably isn't used during build (but maybe used for testing). Please keep in mind that any build dependency annotated is irrelevant to bootstrapping problems, so this is quite powerful normally. Unfortunately, Python is not very reproducible. It embeds the build path into lots of places. It uses profile guided optimization and even if you build with DEB_BUILD_OPTIONS=nopgo, you still get lots of variations. The python$VER package used to contain tk, but since a while we have python3-stdlib-extensions. It feels fairly natural that python$VER should no longer need tk and blt. I went a head and tried reproducing this. On the good side, that build succeeded. The resulting artifacts still differed a lot from an unmodified build and in particular, the _uuid.so and _nis.so module went missing. The former probably is due to something from tk or blt issuing a dependency on uuid-dev and with those packages removed, uuid-dev was no longer pulled. I have not found a probably cause for the nis module. So from the outset it seems that breaking the cycle at Python is plausible, but it needs more validation work. Possibly, blt is still being used for testing the build and that's fine as long as it is annotated . (Do note that a failure to build from source with the nocheck profile enabled is considered to be an rc bug since trixie.) > Again, I'll revert the change shortly, but is there any way I can help > with untangling Python from blt? If you are up for doing that work, I definitely appreciate that. As detailed earlier, this probably involves changing python's Build-Depends only. Adding uuid-dev and probably other packages that currently are implied and then dropping or annotating blt-dev and probably also tk-dev. That validation work is what takes time. At a bare minimum, one needs to fix a build path and use DEB_BUILD_OPTIONS=nopgo, but I suspect that even that isn't enough to get variations down to a manageable level. I'm all for removing blt from the bootstrap set in general. Though keep in mind that you won't be removing it from python3-stdlib-extensions and python3-tk has 500+ build-rdeps, so your luck with being removed from bootstrap may vary. Helmut
Bug#1058678: blt: disable jpeg support
severity #1058678 serious thanks Hi Helmut, On Thu, Dec 14, 2023 at 3:18 PM Helmut Grohne wrote: > did you know that blt participates in architecture bootstrap? It's > required for building the Python interpreter for instance and thus far, > this has all worked out well, but now you added a build dependency on > libjpeg-dev. libjpeg-turbo in turn Build-Depends on default-jdk and > while it builds its -java package as an Arch:all package, the jdk > actually is being used as it stuffs jni symbols into the main library, > so really the Arch:any package already requires default-jdk. So now we > have a path from Python to Java. We also have a path from Java to Python > already, so your blt upload introduces a bootstrap dependency cycle and > this is bad. I'm sorry, I wasn't aware about these implications. Meanwhile, I'm setting this bug severity to serious and will fix it as soon as possible. > > I suggest that you temporarily revert this change in unstable to restore > the ability to bootstrap Debian from source. I'll do that. > > Then let's figure out the best way to break the cycle before re-enabling > jpeg support. This has mostly worked earlier, because libjpeg-turbo was > not part of the bootstrap set. Even adding libjpeg-turbo seems fine in > principle, but Java less so. As explained earlier, breaking the cycle > between libjpeg-turbo and Java is harder, because there are > Java-specific symbols in the main library. Making jpeg support optional > in blt seems similarly impractical to me. This is definitely more work > than just enabling jpeg support unfortunately. Do you have more ideas? Can someone investigate if blt is really necessary for Python now? It's a very old unmaintained piece of software, and I think that dependence of Python on it should be a bug. Again, I'll revert the change shortly, but is there any way I can help with untangling Python from blt? Cheers! -- Sergei Golovan
Bug#1058678: blt: disable jpeg support
Source: blt Version: 2.5.3+dfsg-6 Severity: important User: helm...@debian.org Usertags: rebootstrap Hi Sergei, did you know that blt participates in architecture bootstrap? It's required for building the Python interpreter for instance and thus far, this has all worked out well, but now you added a build dependency on libjpeg-dev. libjpeg-turbo in turn Build-Depends on default-jdk and while it builds its -java package as an Arch:all package, the jdk actually is being used as it stuffs jni symbols into the main library, so really the Arch:any package already requires default-jdk. So now we have a path from Python to Java. We also have a path from Java to Python already, so your blt upload introduces a bootstrap dependency cycle and this is bad. I suggest that you temporarily revert this change in unstable to restore the ability to bootstrap Debian from source. Then let's figure out the best way to break the cycle before re-enabling jpeg support. This has mostly worked earlier, because libjpeg-turbo was not part of the bootstrap set. Even adding libjpeg-turbo seems fine in principle, but Java less so. As explained earlier, breaking the cycle between libjpeg-turbo and Java is harder, because there are Java-specific symbols in the main library. Making jpeg support optional in blt seems similarly impractical to me. This is definitely more work than just enabling jpeg support unfortunately. Do you have more ideas? Helmut
