Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
Hi, I think that the core problem which led to this also was on GRPCs side. They should not tell users to compile their library but rather point them to install it from their distros' packet manager (https://grpc.io/docs/languages/cpp/quickstart/#install-grpc) if possible. I will try to do the switch tonight as I just use GRPC as library in a project and don't need to change anything in the GRPC source. Have a nice day, Jonathan Chung Am Mittwoch, dem 03.04.2024 um 08:35 +0200 schrieb Thomas Schmitt: > Hi, > > Chung Jonathan wrote: > > Yes, I think the local fix is the way to go. > > I wrote: > > > (You forgot to Cc: debian-user@lists.debian.org. > > > Consider to send your mail to the list address, too. I too would > > > then > > > resend my following reply to the list.) > > Since my "following reply" is quoted in Jonathan Chung's reply to the > list > i don't have to resend it. (I gave my opinion that the problem is not > a > bug in the context of Debian 12 or 13 and pointed to > https://wiki.debian.org/BuildingTutorial for a private fix of the > problem.) > > > Jeffrey Walton wrote: > > Your problem is one that plagues Linux. You compile and link > > against > > one version of a library, and then you runtime link against another > > version. > > This should not be a problem with a well maintained library which > cares > to stay ABI compatible with its older releases. > In the present case it was a bug in the loading program pigz which > prevented zlib from being usable. > > > > I consider it a > > security bug since essentially random libraries are being loaded at > > runtime. > > To fix the problem yourself, add an RPATH to your LDFLAGS when > > building your program: > > -Wl,-rpath=/path/to/expected/libz -Wl,--enable-new-dtags > > Well, this is nearly as unflexible as static compilation but does not > seem to prevent the use of a replaced library at the given path. > > Using .so files has its advantages and disadvantages. For a distro > the > advantage (without the pigz bug) is that customers of different > versions > of a library can be consolidated to using the newest available > version. > An advantage for the user is that bugs in a library can be fixed > without > the need for re-building all its customers. > > > Have a nice day :) > > Thomas >
Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
Hi, Chung Jonathan wrote: > Yes, I think the local fix is the way to go. I wrote: > > (You forgot to Cc: debian-user@lists.debian.org. > > Consider to send your mail to the list address, too. I too would then > > resend my following reply to the list.) Since my "following reply" is quoted in Jonathan Chung's reply to the list i don't have to resend it. (I gave my opinion that the problem is not a bug in the context of Debian 12 or 13 and pointed to https://wiki.debian.org/BuildingTutorial for a private fix of the problem.) Jeffrey Walton wrote: > Your problem is one that plagues Linux. You compile and link against > one version of a library, and then you runtime link against another > version. This should not be a problem with a well maintained library which cares to stay ABI compatible with its older releases. In the present case it was a bug in the loading program pigz which prevented zlib from being usable. > I consider it a > security bug since essentially random libraries are being loaded at > runtime. > To fix the problem yourself, add an RPATH to your LDFLAGS when > building your program: > -Wl,-rpath=/path/to/expected/libz -Wl,--enable-new-dtags Well, this is nearly as unflexible as static compilation but does not seem to prevent the use of a replaced library at the given path. Using .so files has its advantages and disadvantages. For a distro the advantage (without the pigz bug) is that customers of different versions of a library can be consolidated to using the newest available version. An advantage for the user is that bugs in a library can be fixed without the need for re-building all its customers. Have a nice day :) Thomas
Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
On Tue, Apr 02, 2024 at 06:34:43PM -0400, Jeffrey Walton wrote: > On Tue, Apr 2, 2024 at 6:24 PM Chung Jonathan wrote: > > > > Dear Franco Martelli, dear Thomas Schmitt, > > > > Sorry for the potential duplication. This mail should now also go to the > > list. > > > > I believe I found the problem which was on my side. I do have libz.so.1.3, > > since I manually compiled grpc on my machine and this also uses a newer > > version of zlib appearently. So this is not a Debian problem but rather > > specific to my setup. A clean install in a VM indeed works as expected. > > Do you still think a bug report is worth it? > > Your problem is one that plagues Linux. You compile and link against > one version of a library, and then you runtime link against another > version. This should have been fixed for users a long time ago, but > the folks responsible leave users to suffer it. I consider it a > security bug since essentially random libraries are being loaded at > runtime. > > To fix the problem yourself, add an RPATH to your LDFLAGS when > building your program: > > -Wl,-rpath=/path/to/expected/libz -Wl,--enable-new-dtags > > The loader will encounter the RPATH when loading your executable, and > load the correct library for your program. I've run into a variant of this issue myself. I had compiled an upstream version of Tcl, which builds a shared library named libtcl8.6.so in the lib/ subdirectory of the --prefix used for ./configure, where the default value of --prefix is /usr/local. This places the shared library in /usr/local/lib which happens to be in the default set of paths used by ldconfig and ld.so. Which in turn means that the locally built library gets used when running Debian's packaged version of Tcl, taking priority over the version installed in /usr/lib/x86_64-linux-gnu/ or whatver arch-specific directory is used. In my case, the solution was to build my local version of Tcl with a different --prefix. Then there's nothing dropped into /usr/local/lib and there's no conflict with the system's packages. Of course, this is a particularly subtle problem that most users are not going to be aware of. It's specific to packages that build part of themselves as a shared library, which immediately makes most programs immune to it. But when it *does* affect you, it can be tricky to figure out what's happening. It sounds like the OP's pigz issue was of a similar nature. If this is the case, then filing a Debian bug report isn't going to be helpful, unless it's done as a sort of documentation for anyone else who runs into the same problem in the future.
Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
On Tue, Apr 2, 2024 at 6:24 PM Chung Jonathan wrote: > > Dear Franco Martelli, dear Thomas Schmitt, > > Sorry for the potential duplication. This mail should now also go to the list. > > I believe I found the problem which was on my side. I do have libz.so.1.3, > since I manually compiled grpc on my machine and this also uses a newer > version of zlib appearently. So this is not a Debian problem but rather > specific to my setup. A clean install in a VM indeed works as expected. > Do you still think a bug report is worth it? Your problem is one that plagues Linux. You compile and link against one version of a library, and then you runtime link against another version. This should have been fixed for users a long time ago, but the folks responsible leave users to suffer it. I consider it a security bug since essentially random libraries are being loaded at runtime. To fix the problem yourself, add an RPATH to your LDFLAGS when building your program: -Wl,-rpath=/path/to/expected/libz -Wl,--enable-new-dtags The loader will encounter the RPATH when loading your executable, and load the correct library for your program. Jeff
Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
Dear Franco Martelli, dear Thomas Schmitt, Sorry for the potential duplication. This mail should now also go to the list. I believe I found the problem which was on my side. I do have libz.so.1.3, since I manually compiled grpc on my machine and this also uses a newer version of zlib appearently. So this is not a Debian problem but rather specific to my setup. A clean install in a VM indeed works as expected. Do you still think a bug report is worth it? Thanks for your prompt responses, Jonathan Chung Am 02.04.2024 um 21:37 schrieb Chung Jonathan : Dear Franco Martelli, dear Thomas Schmitt, Yes, /etc/debian_version reports 12.5. However, as I have the docker sources as external repository those might have introduced an update to zlib. I will try to narrow it down and then open a subsequent bug report - possibly on a clean VM too. The error first occurred as I installed a docker image. However, pigz --version also gave this error. Thank you for pointing me to the correct directions, Jonathan Chung Am 02.04.2024 um 21:07 schrieb Franco Martelli : On 02/04/24 at 10:27, Jonathan Chung wrote: Dear sir or madam, I'm new with Debian bug reporting and thus need some help with that. pigz 2.6-1 on Debian 12.5 fails to execute due to a fixed bug on upstream https://github.com/madler/pigz/issues/111 Installing the version from sid resolves the issue which is clearly not optimal. I think the fix should be backported. Can someone help me to file a bug report? To file a bug report use "reportbug pigz" and answer the questions… but I use regularly "pigz" in my backup scripts and I didn't notice nothing wrong ?_? My installed version is: ~$ dpkg -l | grep pigz ii pigz 2.6-1 amd64Parallel Implementation of GZip I just tested it and I've nothing to report: ~$ tar tf frankbck2.tgz --use-compress-program=pigz >/dev/null ~$ echo $? 0 Are you sure you are using Debian 12.5 ? Kind regards. -- Franco Martelli
Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
Dear Thomas Schmitt, Yes, I think the local fix is the way to go. Probably even getting rid of the source of this, my manual GRPC install and replacing it with libgrpc++-dev. But that’s outside of this issue. Yours, Jonathan Chung Am 02.04.2024 um 23:34 schrieb Thomas Schmitt : Hi, (You forgot to Cc: debian-user@lists.debian.org. Consider to send your mail to the list address, too. I too would then resend my following reply to the list.) Chung Jonathan wrote: I do have libz.so.1.3, since I manually compiled grpc on my machine and this also uses a newer version of zlib appearently. [...] Do you still think a bug report is worth it? Rather not. The original combination of pigz and zlib works in Debian 12. So it's not a bug in that context. The fact that it hampers upgrade to younger zlib is already addressed by the version of pigz in "testing". So in the context of Debian 13 it will be no bug either. This leaves only the possibility to really ask for a backport for the new feature that one can upgrade zlib before Debian offers it. But the desired use case is near to the topic of "Frankendebian": https://wiki.debian.org/DontBreakDebian If in urgent need for a fix, i expect that you would get more lucky by patching the "stable" pigz package for your local use according to https://wiki.debian.org/BuildingTutorial (patching is done in https://wiki.debian.org/BuildingTutorial#Edit_the_source_code and as usual in Debian, there are more ways than this one to do it ...) Have a nice day :) Thomas
Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
Dear Franco Martelli, dear Thomas Schmitt, Yes, /etc/debian_version reports 12.5. However, as I have the docker sources as external repository those might have introduced an update to zlib. I will try to narrow it down and then open a subsequent bug report - possibly on a clean VM too. The error first occurred as I installed a docker image. However, pigz --version also gave this error. Thank you for pointing me to the correct directions, Jonathan Chung > Am 02.04.2024 um 21:07 schrieb Franco Martelli : > On 02/04/24 at 10:27, Jonathan Chung wrote: >> Dear sir or madam, >> I'm new with Debian bug reporting and thus need some help with that. >> pigz 2.6-1 on Debian 12.5 fails to execute due to a fixed bug on >> upstream https://github.com/madler/pigz/issues/111 >> Installing the version from sid resolves the issue which is clearly not >> optimal. I think the fix should be backported. >> Can someone help me to file a bug report? > > To file a bug report use "reportbug pigz" and answer the questions… but I use > regularly "pigz" in my backup scripts and I didn't notice nothing wrong ?_? > My installed version is: > > ~$ dpkg -l | grep pigz > ii pigz 2.6-1 > amd64Parallel Implementation of GZip > > I just tested it and I've nothing to report: > > ~$ tar tf frankbck2.tgz --use-compress-program=pigz >/dev/null > ~$ echo $? > 0 > > Are you sure you are using Debian 12.5 ? > > Kind regards. > > -- > Franco Martelli
Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
On 02/04/24 at 10:27, Jonathan Chung wrote: Dear sir or madam, I'm new with Debian bug reporting and thus need some help with that. pigz 2.6-1 on Debian 12.5 fails to execute due to a fixed bug on upstream https://github.com/madler/pigz/issues/111 Installing the version from sid resolves the issue which is clearly not optimal. I think the fix should be backported. Can someone help me to file a bug report? To file a bug report use "reportbug pigz" and answer the questions… but I use regularly "pigz" in my backup scripts and I didn't notice nothing wrong ?_? My installed version is: ~$ dpkg -l | grep pigz ii pigz 2.6-1 amd64Parallel Implementation of GZip I just tested it and I've nothing to report: ~$ tar tf frankbck2.tgz --use-compress-program=pigz >/dev/null ~$ echo $? 0 Are you sure you are using Debian 12.5 ? Kind regards. -- Franco Martelli
Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
On Tue, 2 Apr 2024 10:27:33 +0200 Jonathan Chung wrote: > I'm new with Debian bug reporting and thus need some help with that. > pigz 2.6-1 on Debian 12.5 fails to execute due to a fixed bug on > upstream https://github.com/madler/pigz/issues/111 Are you sure this is a bug and not a problem with your system? I am also using pigz 2.6-1 on Debian 12.5, and I don't see it. charles@hawk:~$ pigz --version pigz 2.6 charles@hawk:~$ pre zlib pigz pigz2.6-1 amd64 zlib1g 1:1.2.13.dfsg-1 amd64 zlib1g-dev 1:1.2.13.dfsg-1 amd64 charles@hawk:~$ What version of zlib are you running? -- Does anybody read signatures any more? https://charlescurley.com https://charlescurley.com/blog/
Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
Hi,
Jonathan Chung wrote:
> > pigz 2.6-1 on Debian 12.5 fails to execute due to a fixed bug on
> > upstream https://github.com/madler/pigz/issues/111
> > Installing the version from sid resolves the issue which is clearly not
> > optimal. I think the fix should be backported.
> > Can someone help me to file a bug report?
Michael Kjörling wrote:
> https://www.debian.org/Bugs/Reporting
This covers the question "how", but not "what".
So my two cents:
"Backported" is probably the wrong wish.
https://backports.debian.org/Contribute/
says:
"Backports are about additional features that are only offered in a
new version, not a replacement for getting fixes into stable - use
stable-updates for that."
That would probably mean to patch the existing pigz package in Debian 12
by the upstream remedy
"Make pigz compatible with two-component zlib version numbers. "
https://github.com/madler/pigz/commit/907ca0763be4547a9b0cce8c105721748814974
pigz.c:
@@ -1333,7 +1333,7 @@ local long zlib_vernum(void) {
}
ver++;
} while (left);
- return left < 2 ? num << (left << 2) : -1;
+ return left < 3 ? num << (left << 2) : -1;
}
// -- check value combination routines for parallel calculation --
The new state can already be seen in
https://sources.debian.org/src/pigz/2.8-1/pigz.c/#L1339
The old state is in
https://sources.debian.org/src/pigz/2.6-1/pigz.c/#L1317
But there the "// -- check ..." comment is missing in favor of a line
#ifndef NOTHREAD
So a patch proposal would need some minor hand work.
There remains the question, though, why you run into zlib-1.3 on a
Debian "stable" system where the zlib version is "1:1.2.13.dfsg-1":
https://tracker.debian.org/pkg/zlib
Have a nice day :)
Thomas
Re: How to file a Debian bug report? was: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
On 2 Apr 2024 10:27 +0200, from jch...@student.ethz.ch (Jonathan Chung): > Can someone help me to file a bug report? https://www.debian.org/Bugs/Reporting -- Michael Kjörling https://michael.kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?”
Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 111)
Dear sir or madam, I'm new with Debian bug reporting and thus need some help with that. pigz 2.6-1 on Debian 12.5 fails to execute due to a fixed bug on upstream https://github.com/madler/pigz/issues/111 Installing the version from sid resolves the issue which is clearly not optimal. I think the fix should be backported. Can someone help me to file a bug report?
