Bug#1069322: diffoscope crashes when trying to compare unreproducible src:dasel build artifacts
Holger Levsen wrote: > I'm not sure how --debug output should survive, but you mean just > running diffoscope with an added --debug option? Ah, yeah. It won't survive from Jenkins' log perspective, huh? Hmm, the --debug output could perhaps to be directed straight to an on-disk file. Given that that should be flushed after every line, that should survive an OOM kill. If not, hmm, I'll have a think. Either way, apologies that I'm not more familiar with all the abstraction layers in our setup and thus which might survive an OOM. Regards, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org chris-lamb.co.uk `-
Bug#1069322: diffoscope crashes when trying to compare unreproducible src:dasel build artifacts
On Tue, May 14, 2024 at 11:43:29AM +0100, Chris Lamb wrote: > Ah, I was hoping that the systemd slice apparatus would be able to > contain any traceback, but now that I think of it, being OOM-killed is > not quite the same as CPython-level crash (and thus traceback). :) > > https://tests.reproducible-builds.org/debian/artifacts/r00t-me/trixie_i386_dasel_tmp-kqFaQ/ > > is maybe working as in crashing for you? > > Alas, this works for me and does not crash. I suppose the next thing > might be to try and run with --debug? That way, we might be able to > determine which file, comparator or external tool was being run when > diffoscope invoked the ire of the oom-killer. I'm not sure how --debug output should survive, but you mean just running diffoscope with an added --debug option? -- cheers, Holger ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org ⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C ⠈⠳⣄ The average US president has been charged with 2 felonies: #45 with 91 and the others with 0. signature.asc Description: PGP signature
Bug#1069322: diffoscope crashes when trying to compare unreproducible src:dasel build artifacts
Holger Levsen wrote: >> Hm, I can't seem to reproduce the crash with these files. In the first >> instance, can you paste a traceback or similar of the crash in >> question? Maybe it is fixable just from that without having to find >> and upload more files, etc. > > I don't have a traceback as the oom-kill also kills the surrounding > processes... Ah, I was hoping that the systemd slice apparatus would be able to contain any traceback, but now that I think of it, being OOM-killed is not quite the same as CPython-level crash (and thus traceback). > https://tests.reproducible-builds.org/debian/artifacts/r00t-me/trixie_i386_dasel_tmp-kqFaQ/ > is maybe working as in crashing for you? Alas, this works for me and does not crash. I suppose the next thing might be to try and run with --debug? That way, we might be able to determine which file, comparator or external tool was being run when diffoscope invoked the ire of the oom-killer. Regards, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org chris-lamb.co.uk `-
Bug#1069322: diffoscope crashes when trying to compare unreproducible src:dasel build artifacts
On Wed, May 08, 2024 at 03:07:00PM +0100, Chris Lamb wrote: > Hm, I can't seem to reproduce the crash with these files. In the first > instance, can you paste a traceback or similar of the crash in > question? Maybe it is fixable just from that without having to find > and upload more files, etc. I don't have a traceback as the oom-kill also kills the surrounding processes... May 13 20:45:41 jenkins kernel: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=/,mems_allowed=0,oom_memcg=/user.slice/user-103.slice/user@103.service/rb.slice,task_memcg=/user.slice/user-103.slice/user@103.service/rb.slice/rb-build.slice/rb-build-diffoscope.slice/rb-diffoscope-i386_1-26686.service,task=diffoscope,pid=4117661,uid=103 May 13 20:45:41 jenkins kernel: Memory cgroup out of memory: Killed process 4117661 (diffoscope) total-vm:218023820kB, anon-rss:86849224kB, file-rss:40448kB, shmem-rss:0kB, UID:103 pgtables:425080kB oom_score_adj:200 May 13 20:45:41 jenkins systemd[1]: user@103.service: A process of this unit has been killed by the OOM killer. May 13 20:45:41 jenkins systemd[1620]: rb-diffoscope-i386_1-26686.service: A process of this unit has been killed by the OOM killer. is basically all I see. https://tests.reproducible-builds.org/debian/artifacts/r00t-me/trixie_i386_dasel_tmp-kqFaQ/ is maybe working as in crashing for you? -- cheers, Holger ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org ⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C ⠈⠳⣄ https://showyourstripes.info signature.asc Description: PGP signature
Bug#1069322: diffoscope crashes when trying to compare unreproducible src:dasel build artifacts
Holger Levsen wrote: > I'm attaching the crashing artifacts now to this bug report, however minus > the orig.tar.gz files, though I suppose that the .deb files are enough to > crash diffoscope anyway... Hm, I can't seem to reproduce the crash with these files. In the first instance, can you paste a traceback or similar of the crash in question? Maybe it is fixable just from that without having to find and upload more files, etc. Best wishes, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org chris-lamb.co.uk `-
Bug#1069322: diffoscope crashes when trying to compare unreproducible src:dasel build artifacts
Package: diffoscope Version: 264 Severity: normal X-Debbugs-Cc: team+pkg...@tracker.debian.org Dear Maintainer, diffoscope crashes when comparing the build results of src:dasel. To make it more fun, src:dasel is only unreproducible on i386 (out of our four tested archs, amd64/i386/arm64/armhf) and only *sometimes*. vagrant added the following note to reproducible-notes.git, visible at https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/dasel.html ---begin-note--- timezone-dependent date in manpages triggered when building with reprotest but not reproducible builds test infrastructure. dasel itself is used to generate the manpage. https://sources.debian.org/src/dasel/2.7.0-1/internal/command/man.go/ . Something non-deterministic, possibly GO BUILDID only on i386. ---end-note--- several build artifacts at available at https://tests.reproducible-builds.org/debian/artifacts/r00t-me/ and only the i386 ones are sometimes unreproducible and then crashing diffoscope. (Please download them for investigations, they will vanish after 48h but I can easily and quickly recreate them anytime.) -- cheers, Holger ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org ⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C ⠈⠳⣄ I used to be scared for our grandchildren's future. Such optimism! signature.asc Description: PGP signature
