Re: Bug#790560: udev fails to start on sparc boot, breaking boot
This has been fixed upstream for a while now, thus closing. However, please note that ld.gold has still issues on sparc/sparc64 [1] which is why I have asked the binutils maintainer to disable it on sparc/sparc64 for the time being [2]. Cheers, Adrian > [1] https://sourceware.org/bugzilla/show_bug.cgi?id=19019 > [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=803474 -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
Reported a binutils/gold bug for it: https://sourceware.org/bugzilla/show_bug.cgi?id=18855 -- Regards, Artyom Tarasenko SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
On Tue, Aug 18, 2015 at 9:50 AM, Knut Petter Ølberg kpolb...@gmail.com wrote: On 17 August 2015 at 18:48, Artyom Tarasenko atar4q...@gmail.com wrote: Hi Knut Petter, On Mon, Aug 17, 2015 at 10:29 AM, Knut Petter Ølberg kpolb...@gmail.com wrote: On 9 August 2015 at 00:22, Sam Ravnborg s...@ravnborg.org wrote: Hi Artyom I think the kernel is going to need this patch: https://lists.debian.org/debian-sparc/2015/08/msg00014.html This patch does not fix any real bug - it is a cosmetic issue. I assume you wanted to point out the FPU fix: sparc64: Fix userspace FPU register corruptions https://lists.debian.org/debian-sparc/2015/08/msg00012.html Sam Hi, Sorry about my late reply. No worries here. I think the most people in this thread don't have much time for debian-sparc anyways. :-) I mistakenly said T5120 when it's actually is a T5140. Not sure if this makes any difference? No difference, it's a sun4v machine too and the FPU register corruption fix is still required. https://lists.debian.org/debian-sparc/2015/08/msg00012.html Anyways, just let me know what you need and I'll set it up. Since we identified that the udev (systemd) problems are caused by a gold linker, the next steps are build current binutils and check whether the situation is improved. And if it's not, patch a systemd package to use the bfd linker instead of gold. So until this is fixed, the ldom would be of no use as a build machine? Without the kernel fix the system just won't run stable. The bug in binutils is not that dramatic: the default linker is bfd, which works just fine. Only builds of the programs that are explicitly configured to use the gold linker are broken. And I think udev is the only one which is necessary for the minimal install. Artyom -- Regards, Artyom Tarasenko SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
On 17 August 2015 at 18:48, Artyom Tarasenko atar4q...@gmail.com wrote: Hi Knut Petter, On Mon, Aug 17, 2015 at 10:29 AM, Knut Petter Ølberg kpolb...@gmail.com wrote: On 9 August 2015 at 00:22, Sam Ravnborg s...@ravnborg.org wrote: Hi Artyom I think the kernel is going to need this patch: https://lists.debian.org/debian-sparc/2015/08/msg00014.html This patch does not fix any real bug - it is a cosmetic issue. I assume you wanted to point out the FPU fix: sparc64: Fix userspace FPU register corruptions https://lists.debian.org/debian-sparc/2015/08/msg00012.html Sam Hi, Sorry about my late reply. No worries here. I think the most people in this thread don't have much time for debian-sparc anyways. :-) I mistakenly said T5120 when it's actually is a T5140. Not sure if this makes any difference? No difference, it's a sun4v machine too and the FPU register corruption fix is still required. https://lists.debian.org/debian-sparc/2015/08/msg00012.html Anyways, just let me know what you need and I'll set it up. Since we identified that the udev (systemd) problems are caused by a gold linker, the next steps are build current binutils and check whether the situation is improved. And if it's not, patch a systemd package to use the bfd linker instead of gold. Artyom -- Regards, Artyom Tarasenko SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu So until this is fixed, the ldom would be of no use as a build machine? -kp
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
So until this is fixed, the ldom would be of no use as a build machine? Without the kernel fix the system just won't run stable. The bug in binutils is not that dramatic: the default linker is bfd, which works just fine. Only builds of the programs that are explicitly configured to use the gold linker are broken. And I think udev is the only one which is necessary for the minimal install. Artyom Sounds fair enough. In any case, I have a running installation which I can add more cores/memory to whenever it would be usable. So just let me know. -kp root@T1000-debian7:~# free -m total used free sharedbuffers cached Mem: 1993477 1516 0 94286 -/+ buffers/cache: 96 1897 Swap: 486 0486 root@T1000-debian7:~# uname -a Linux T1000-debian7 3.2.0-4-sparc64-smp #1 SMP Debian 3.2.68-1+deb7u2 sparc64 GNU/Linux root@T1000-debian7:~# cat /proc/cpuinfo cpu : UltraSparc T2 (Niagara2) fpu : UltraSparc T2 integrated FPU pmu : niagara2 prom : OBP 4.33.6.f 2014/07/10 10:24 type : sun4v ncpus probed : 8 ncpus active : 8 D$ parity tl1 : 0 I$ parity tl1 : 0 cpucaps : flush,stbar,swap,muldiv,v9,blkinit,n2,mul32,div32,v8plus,popc,vis,vis2,ASIBlkInit Cpu0ClkTck : 5458d748 Cpu1ClkTck : 5458d748 Cpu2ClkTck : 5458d748 Cpu3ClkTck : 5458d748 Cpu4ClkTck : 5458d748 Cpu5ClkTck : 5458d748 Cpu6ClkTck : 5458d748 Cpu7ClkTck : 5458d748 MMU Type : Hypervisor (sun4v) State: CPU0: online CPU1: online CPU2: online CPU3: online CPU4: online CPU5: online CPU6: online CPU7: online root@T1000-debian7:~# uptime 13:37:52 up 7 days, 4:57, 1 user, load average: 0.00, 0.01, 0.05
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
On Mon, Aug 10, 2015 at 9:33 AM, Frans van Berckel fberc...@xs4all.nl wrote: On Fri, 2015-08-07 at 20:48 +0200, Frans van Berckel wrote: On Fri, 2015-08-07 at 18:24 +0200, Artyom Tarasenko wrote: After 10 hours of building (my machine is probably not the fastest one), I can only confirm that the gold linker is still broken in binutils_2.25-11_sparc64.deb Am I right, there are some fails / test error's, building binutils_2.25-11_sparc64.deb on Sompek at 2015-07-31 ... snip http://buildd.debian-ports.org/status/logs.php?pkg=binutilsver=2.25 -11arch=sparc64 Seen the next, binutils_2.25.1-1_sparc64.deb is build on Raverin at 2015-08-08, with no fails / test error's anymore. http://buildd.debian-ports.org/status/logs.php?pkg=binutilsver=2.25.1 -1arch=sparc64 Checked with binutils_2.25.1-1_sparc64.deb . -Wl,-fuse-ld=gold still produces broken binaries. Tried manually compiling a couple of systemd binaries with -Wl,-fuse-ld=bfd. A smoke test shows they don't crash at startup. Shall I open another bug for binutils? The problem doesn't seem to be systemd specific. -- Regards, Artyom Tarasenko SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
On Mon, 2015-08-17 at 18:34 +0200, Artyom Tarasenko wrote: On Mon, Aug 10, 2015 at 9:33 AM, Frans van Berckel fberc...@xs4all.nl wrote: Checked with binutils_2.25.1-1_sparc64.deb . -Wl,-fuse-ld=gold still produces broken binaries. Tried manually compiling a couple of systemd binaries with -Wl,-fuse-ld=bfd. A smoke test shows they don't crash at startup. Shall I open another bug for binutils? The problem doesn't seem to be systemd specific. I did soms testing as well today. With a focus on binutils only. Building binutils_2.25.1-1_sparc64.deb from source in sparc64 chroot. Next I did simple go into the builddir-single directory. And started make check. What surprised me, it does goes true the Gold tests nice. But the ld testsuite does 4 of unexpected failures. Its looks like the same error as we have before. Having warnings, Couldn't find tool init file as well. Attaching the make-check.log file [plain txt]. Running /binutils-2.25.1/ld/testsuite/ld-elf/elf.exp FAIL: -Bsymbolic-functions Running /binutils-2.25.1/ld/testsuite/ld-elf/shared.exp FAIL: Run pr2404 with PIE Running /binutils-2.25.1/ld/testsuite/ld-sparc/sparc.exp FAIL: 32-bit: GOTDATA relocations FAIL: 64-bit: GOTDATA relocations I found out theres a, 130_gold_disable_testsuite_build.patch in debian / patches. So maybe Gold testing is still disabled? The question, how do we find out what goes really wrong with binutils? Thanks, Frans van Berckel make[1]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single' make[2]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/bfd' make check-recursive make[3]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/bfd' Making check in doc make[4]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/bfd/doc' make[4]: Nothing to be done for 'check'. make[4]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/bfd/doc' Making check in po make[4]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/bfd/po' make[4]: Nothing to be done for 'check'. make[4]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/bfd/po' make[4]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/bfd' make[4]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/bfd' make[3]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/bfd' make[2]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/bfd' make[2]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/opcodes' Making check in . make[3]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/opcodes' make[3]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/opcodes' Making check in po make[3]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/opcodes/po' make[3]: Nothing to be done for 'check'. make[3]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/opcodes/po' make[2]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/opcodes' make[2]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/binutils' make[3]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/binutils' make[3]: '../../binutils/rcparse.c' is up to date. make[3]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/binutils' make check-recursive make[3]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/binutils' Making check in doc make[4]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/binutils/doc' make[4]: Nothing to be done for 'check'. make[4]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/binutils/doc' Making check in po make[4]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/binutils/po' make[4]: Nothing to be done for 'check'. make[4]: Leaving directory '/usr/src/binutils/binutils-2.25.1/builddir-single/binutils/po' make[4]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/binutils' make check-DEJAGNU make[5]: Entering directory '/usr/src/binutils/binutils-2.25.1/builddir-single/binutils' srcdir=`cd ../../binutils pwd`; export srcdir; \ r=`pwd`; export r; \ LC_ALL=C; export LC_ALL; \ EXPECT=expect; export EXPECT; \ runtest=runtest; \ if /bin/bash -c $runtest --version /dev/null 21; then \ CC_FOR_TARGET=gcc CFLAGS_FOR_TARGET=-g -O2 \ $runtest --tool binutils --srcdir ${srcdir}/testsuite \ ; \ else echo WARNING: could not find \`runtest' 12; :;\ fi WARNING: Couldn't find tool init file Test Run By root on Mon Aug 17 09:26:51 2015 Native configuration is sparc64-unknown-linux-gnu === binutils tests === Schedule of variations: unix Running target unix Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target. Using /usr/share/dejagnu/config/unix.exp as generic interface file for target.
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
On 9 August 2015 at 00:22, Sam Ravnborg s...@ravnborg.org wrote: Hi Artyom I think the kernel is going to need this patch: https://lists.debian.org/debian-sparc/2015/08/msg00014.html This patch does not fix any real bug - it is a cosmetic issue. I assume you wanted to point out the FPU fix: sparc64: Fix userspace FPU register corruptions https://lists.debian.org/debian-sparc/2015/08/msg00012.html Sam Hi, Sorry about my late reply. I mistakenly said T5120 when it's actually is a T5140. Not sure if this makes any difference? Anyways, just let me know what you need and I'll set it up. -kp
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
On Fri, 2015-08-07 at 20:48 +0200, Frans van Berckel wrote: On Fri, 2015-08-07 at 18:24 +0200, Artyom Tarasenko wrote: After 10 hours of building (my machine is probably not the fastest one), I can only confirm that the gold linker is still broken in binutils_2.25-11_sparc64.deb Am I right, there are some fails / test error's, building binutils_2.25-11_sparc64.deb on Sompek at 2015-07-31 ... snip http://buildd.debian-ports.org/status/logs.php?pkg=binutilsver=2.25 -11arch=sparc64 Seen the next, binutils_2.25.1-1_sparc64.deb is build on Raverin at 2015-08-08, with no fails / test error's anymore. http://buildd.debian-ports.org/status/logs.php?pkg=binutilsver=2.25.1 -1arch=sparc64 Thanks, Frans van Berckel -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1439192021.3369.9.ca...@xs4all.nl
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
On Sat, Aug 8, 2015 at 12:21 AM, Knut Petter Ølberg kpolb...@gmail.com wrote: After 10 hours of building (my machine is probably not the fastest one), I can only confirm that the gold linker is still broken in binutils_2.25-11_sparc64.deb I'm no developer. But I do have access to a T5120, where I could assign an LDOM with 64 cores and 16gb of memory. If that would be of any help(I can install a basic wheezy installation)? Yes, I think this would help. My only concern is the stability of Wheezy on T5120. I think the kernel is going to need this patch: https://lists.debian.org/debian-sparc/2015/08/msg00014.html Artyom -- Regards, Artyom Tarasenko SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CACXAS8B+GCCnaR_Lv3DH9r0RLunCoRtK-LPVYxj=7v=_p2n...@mail.gmail.com
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
Hi Artyom I think the kernel is going to need this patch: https://lists.debian.org/debian-sparc/2015/08/msg00014.html This patch does not fix any real bug - it is a cosmetic issue. I assume you wanted to point out the FPU fix: sparc64: Fix userspace FPU register corruptions https://lists.debian.org/debian-sparc/2015/08/msg00012.html Sam -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/2015080805.ga5...@ravnborg.org
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
After 10 hours of building (my machine is probably not the fastest one), I can only confirm that the gold linker is still broken in binutils_2.25-11_sparc64.deb I'm no developer. But I do have access to a T5120, where I could assign an LDOM with 64 cores and 16gb of memory. If that would be of any help(I can install a basic wheezy installation)? -kp
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
After 10 hours of building (my machine is probably not the fastest one), I can only confirm that the gold linker is still broken in binutils_2.25-11_sparc64.deb Also I added ifneq ($(findstring $(DEB_BUILD_ARCH), sparc),) LD=ld endif to debian/rules (that's the only part of the patch referenced by Michael which seems to be relevant), but see no effect: I still observe the linker using -Wl,-fuse-ld=gold , and all the built systemd binaries are broken. Michael, I'm not familiar with the debian/rules. Would sparc64 also match the findsting above? Artyom -- Regards, Artyom Tarasenko SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CACXAS8DNxv7u=cPm0QJ9YPBLJ9R8LokckY6vq=94joxh6qj...@mail.gmail.com
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
On Fri, 2015-08-07 at 18:24 +0200, Artyom Tarasenko wrote: After 10 hours of building (my machine is probably not the fastest one), I can only confirm that the gold linker is still broken in binutils_2.25-11_sparc64.deb Am I right, there are some fails / test error's, building binutils_2.25 -11_sparc64.deb on Sompek at 2015-07-31 ... Running /«PKGBUILDDIR»/ld/testsuite/ld-elf/elf.exp ... FAIL: -Bsymbolic-functions Running /«PKGBUILDDIR»/ld/testsuite/ld-elf/shared.exp ... FAIL: Run pr2404 with PIE Running /«PKGBUILDDIR»/ld/testsuite/ld-sparc/sparc.exp ... FAIL: 32-bit: GOTDATA relocations FAIL: 64-bit: GOTDATA relocations === ld Summary === # of expected passes845 # of unexpected failures4 # of expected failures 26 # of untested testcases 1 # of unsupported tests 8 Searching the log file, for the word fail it counts 19, so further in the building there are even more hits ... http://buildd.debian-ports.org/status/logs.php?pkg=binutilsver=2.25 -11arch=sparc64 Thanks, Frans van Berckel -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1438973317.2246.24.ca...@xs4all.nl
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
That's what I see in the strace log: set_tid_address(0xf80100133790) = 9184 set_robust_list(0xf801001337a0, 24) = 0 rt_sigaction(SIGRTMIN, {0xf801006c6da0, [], SA_SIGINFO}, NULL, 0xf801006d2098, 8) = 0 rt_sigaction(SIGRT_1, {0xf801006c6c40, [], SA_RESTART|SA_SIGINFO}, NULL, 0xf801006d2098, 8) = 0 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0 statfs(/sys/fs/selinux, 0x7feffc49960) = -1 ENOENT (No such file or directory) statfs(/selinux, 0x7feffc49960) = -1 ENOENT (No such file or directory) brk(0) = 0x1084000 brk(0x10a6000) = 0x10a6000 open(/proc/filesystems, O_RDONLY) = 3 fstat64(3, {st_mode=0, st_size=0, ...}) = 0 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf8014000 read(3, nodev\tsysfs\nnodev\trootfs\nnodev\tb..., 1024) = 299 read(3, , 1024) = 0 close(3)= 0 munmap(0xf8014000, 8192)= 0 access(/proc/vz, F_OK)= -1 ENOENT (No such file or directory) ioctl(2, _IOC(_IOC_READ, 0x54, 0x08, 0x24), {B38400 opost isig icanon echo ...}) = 0 access(/proc/vz, F_OK)= -1 ENOENT (No such file or directory) writev(2, [{ine, ignoring: Invalid argument, 31}, {u, 1}], 2ine, ignoring: Invalid argumentu) = 32 getuid()= 0 sched_getaffinity(0, 128, {1}) = 8 chdir(/: %m) = -1 ENOENT (No such file or directory) writev(2, [{/dev, 4}, {u, 1}], 2/devu) = 5 exit_group(1) = ? +++ exited with 1 +++ Note the corruption in writev(2, [{ine, ignoring: Invalid argument, 31}, {u, 1}], 2ine, ignoring: Invalid argumentu). The things seem to go astray after failing to find /proc/vz and it's really not there: # ls /proc/v* /proc/version /proc/vmallocinfo /proc/vmstat # uname -a Linux debian 3.2.0-4-sparc64 #1 Debian 3.2.65-1 sparc64 GNU/Linux -- Regards, Artyom Tarasenko SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/cacxas8b6z-uubur0ew20hh5squik6fqtdnc8g3iet6slssm...@mail.gmail.com
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
It looks like stdout and/or stderr output is mixed up with the strace output but it looks udevd is failing because the chdir to / fails. Notes inline below. (Caution I'm comparing against current systemd git HEAD and not any specific version) http://cgit.freedesktop.org/systemd/systemd/tree/src/udev/udevd.c blob: 0661f7be00fc50c3d62643e572c48816342ba1e1 On 06/08/2015 10:47, Artyom Tarasenko wrote: That's what I see in the strace log: set_tid_address(0xf80100133790) = 9184 set_robust_list(0xf801001337a0, 24) = 0 rt_sigaction(SIGRTMIN, {0xf801006c6da0, [], SA_SIGINFO}, NULL, 0xf801006d2098, 8) = 0 rt_sigaction(SIGRT_1, {0xf801006c6c40, [], SA_RESTART|SA_SIGINFO}, NULL, 0xf801006d2098, 8) = 0 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0 statfs(/sys/fs/selinux, 0x7feffc49960) = -1 ENOENT (No such file or directory) statfs(/selinux, 0x7feffc49960) = -1 ENOENT (No such file or directory) brk(0) = 0x1084000 brk(0x10a6000) = 0x10a6000 open(/proc/filesystems, O_RDONLY) = 3 fstat64(3, {st_mode=0, st_size=0, ...}) = 0 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf8014000 read(3, nodev\tsysfs\nnodev\trootfs\nnodev\tb..., 1024) = 299 read(3, , 1024) = 0 close(3)= 0 munmap(0xf8014000, 8192)= 0 access(/proc/vz, F_OK)= -1 ENOENT (No such file or directory) ioctl(2, _IOC(_IOC_READ, 0x54, 0x08, 0x24), {B38400 opost isig icanon echo ...}) = 0 access(/proc/vz, F_OK)= -1 ENOENT (No such file or directory) Cannot see what got us this far but I would guess that it is checking whether the process is running in a virtualized environment. writev(2, [{ine, ignoring: Invalid argument, 31}, {u, 1}], 2ine, ignoring: Invalid argumentu) = 32 This seems to correspond to line 1660 log_warning_errno(r, failed to parse kernel command line, ignoring: %m); This failure to parse the kernel command line is not a fatal error but it might indicate something else is wrong. getuid()= 0 line 1667 - checking to make sure that udevd is running as root. sched_getaffinity(0, 128, {1}) = 8 Line 1677 set affinity. chdir(/: %m) = -1 ENOENT (No such file or directory) Line 1685 is where things fail. r = chdir(/); It is not clear whether the code/binary is wrong here i.e. it really is trying to chdir to /: %m or whether it is strace and stderr output getting mixed up. The error logging function on line 1687 has a %m r = log_error_errno(errno, could not change dir to /: %m); But the %m should have been expanded in the output. writev(2, [{/dev, 4}, {u, 1}], 2/devu) = 5 exit_group(1) = ? +++ exited with 1 +++ Note the corruption in writev(2, [{ine, ignoring: Invalid argument, 31}, {u, 1}], 2ine, ignoring: Invalid argumentu). The things seem to go astray after failing to find /proc/vz and it's really not there: # ls /proc/v* /proc/version /proc/vmallocinfo /proc/vmstat # uname -a Linux debian 3.2.0-4-sparc64 #1 Debian 3.2.65-1 sparc64 GNU/Linux -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55c34146.4040...@oldelvet.org.uk
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
Here is the correspondinf part of the gdb session with symbols from systemd-dbg_224-1_sparc64.deb: (gdb) run Starting program: /lib/systemd/systemd-udevd [Thread debugging using libthread_db enabled] Using host libthread_db library /lib/sparc64-linux-gnu/libthread_db.so.1. Breakpoint 3, main (argc=optimized out, argv=0x7fefc98) at ../src/udev/udevd.c:1662 1662r = parse_proc_cmdline(parse_proc_cmdline_item); (gdb) next 1663if (r 0) (gdb) 1664log_warning_errno(r, failed to parse kernel command line, ignoring: %m); (gdb) ine, ignoring: Invalid argumentu1666if (arg_debug) { (gdb) 1671if (getuid() != 0) { (gdb) 1676if (arg_children_max == 0) { (gdb) 1679arg_children_max = 8; (gdb) 1681if (sched_getaffinity(0, sizeof (cpu_set), cpu_set) == 0) { (gdb) 1682arg_children_max += CPU_COUNT(cpu_set) * 2; (gdb) 1685log_debug(set children_max to %u, arg_children_max); (gdb) 1690if (r 0) { (gdb) 1691r = log_error_errno(errno, could not change dir to /: %m); (gdb) /devu1760 mac_selinux_finish(); (gdb) 1761log_close(); (gdb) 1651_cleanup_free_ char *cgroup = NULL; (gdb) 1763} (gdb) -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CACXAS8BWK2cP+DW+5icrAm6Sqsh=h+eykqo-snujmeu-6+a...@mail.gmail.com
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
And here is an attempt to debug why parse_proc_cmdline fails: Breakpoint 3, main (argc=optimized out, argv=0x7fefc98) at ../src/udev/udevd.c:1662 1662r = parse_proc_cmdline(parse_proc_cmdline_item); (gdb) step parse_proc_cmdline (parse_item=0x1011180 parse_proc_cmdline_item.lto_priv.257) at ../src/basic/util.c:4832 4832assert(parse_item); (gdb) next 4834r = proc_cmdline(line); (gdb) step proc_cmdline (ret=0x7fef338) at ../src/basic/util.c:4819 4819assert(ret); (gdb) next 4821if (detect_container(NULL) 0) (gdb) 4824return read_one_line_file(/proc/cmdline, ret); (gdb) next read_one_line_file (fn=0x10552d8 root, line=0x7fef338) at ../src/basic/fileio.c:103 103 int read_one_line_file(const char *fn, char **line) { ^^ so read_one_line_file was called with a string /proc/cmdline, but for some reason it gets the string root. And this is how it looks at the assebler level: Breakpoint 1, proc_cmdline (ret=0x7fef338) at ../src/basic/util.c:4824 warning: Source file is more recent than executable. 4824return read_one_line_file(/proc/cmdline, ret); (gdb) disas $pc,+0x10 Dump of assembler code from 0x102f554 to 0x102f564: = 0x0102f554 proc_cmdline+52:xor %i0, -224, %i0 0x0102f558 proc_cmdline+56:add %l7, %i0, %i0 0x0102f55c proc_cmdline+60:b %xcc, 0x10339a0 read_one_line_file 0x0102f560 proc_cmdline+64:restore End of assembler dump. (gdb) info registers i0 l7 i0 0x2ac00 175104 l7 0x107ffb81099512151992 (gdb) nexti 0x0102f558 4824return read_one_line_file(/proc/cmdline, ret); (gdb) 0x0102f55c 4824return read_one_line_file(/proc/cmdline, ret); (gdb) x/s $i0 0x10552d8: root So it seems to be a compiler/linker bug. For some reason we are 0x2780 bytes off: (gdb) x/s 0x10552d8-0x2780 0x1052b58: /proc/cmdline (gdb) Can I do anything else to help debugging the issue? -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/cacxas8bapddf-4rktmjwjxq9q5o0o3x_2qkhfdym94my5qp...@mail.gmail.com
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
On 06/08/2015 12:48, Artyom Tarasenko wrote: Here is the correspondinf part of the gdb session with symbols from systemd-dbg_224-1_sparc64.deb: Many thanks. The log below pretty much does confirm that it is taking the suspected path through the code. Some steps are not visible to the debugger due to optimisation but otherwise are running. I wonder if the optimiser has somehow generated bad code and this is causing things to fail. Specifically it does look like it is only producing output of ine, ignoring: Invalid argument. The ine starts 32 characters into the string. Looking at a dump of the binary you can see this 050ce0 N : h V \0 \0 \0 \0 f a i l e d t 4e 3a 68 56 00 00 00 00 66 61 69 6c 65 64 20 74 050cf0 o p a r s e k e r n e l c 6f 20 70 61 72 73 65 20 6b 65 72 6e 65 6c 20 63 050d00 o m m a n d l i n e , i g n 6f 6d 6d 61 6e 64 20 6c 69 6e 65 2c 20 69 67 6e 050d10 o r i n g : % m \0 \0 \0 \0 \0 \0 \0 Later in the binary we see a similar thing the / for the chdir appears exactly 32 characters before the /: %m in the error message 050d50 t o % u \0 \0 / \0 \0 \0 \0 \0 \0 \0 20 74 6f 20 25 75 00 00 2f 00 00 00 00 00 00 00 050d60 c o u l d n o t c h a n g e 63 6f 75 6c 64 20 6e 6f 74 20 63 68 61 6e 67 65 050d70 d i r t o / : % m \0 \0 \0 20 64 69 72 20 74 6f 20 2f 3a 20 25 6d 00 00 00 So maybe the code is trying to use the wrong string as input to chdir and hence failing. I do not have easy access to a disassembler to attempt to confirm what the bad code is doing. I guess it is likely to be a compiler toolchain issue. Either bad code generation or maybe a linker script is getting alignments wrong on when compiled as a 64 bit binary as opposed to a 32 bit binary. (gdb) run Starting program: /lib/systemd/systemd-udevd [Thread debugging using libthread_db enabled] Using host libthread_db library /lib/sparc64-linux-gnu/libthread_db.so.1. Breakpoint 3, main (argc=optimized out, argv=0x7fefc98) at ../src/udev/udevd.c:1662 1662r = parse_proc_cmdline(parse_proc_cmdline_item); (gdb) next 1663if (r 0) (gdb) 1664log_warning_errno(r, failed to parse kernel command line, ignoring: %m); (gdb) ine, ignoring: Invalid argumentu1666if (arg_debug) { (gdb) 1671if (getuid() != 0) { (gdb) 1676if (arg_children_max == 0) { (gdb) 1679arg_children_max = 8; (gdb) 1681if (sched_getaffinity(0, sizeof (cpu_set), cpu_set) == 0) { (gdb) 1682arg_children_max += CPU_COUNT(cpu_set) * 2; (gdb) 1685log_debug(set children_max to %u, arg_children_max); (gdb) 1690if (r 0) { (gdb) 1691r = log_error_errno(errno, could not change dir to /: %m); (gdb) /devu1760 mac_selinux_finish(); (gdb) 1761log_close(); (gdb) 1651_cleanup_free_ char *cgroup = NULL; (gdb) 1763} (gdb) -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55c357c3.1020...@oldelvet.org.uk
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
Richard Mortimer ri...@oldelvet.org.uk writes: So maybe the code is trying to use the wrong string as input to chdir and hence failing. Is udev using the gold linker during build? I've been looking into a bug where in certain circumstances, when linking with gold, string literal function arguments are corrupted. This problem was also breaking qt, specifically moc. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=773590 -David Mattli -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/87io8sv72i@mattli.us
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
Am 06.08.2015 um 16:13 schrieb d...@mattli.us: Richard Mortimer ri...@oldelvet.org.uk writes: So maybe the code is trying to use the wrong string as input to chdir and hence failing. Is udev using the gold linker during build? It is, indeed. We had a hack in debian/rules for a while, to use ld.bfd on sparc due to build failures related to gtk-doc [1]. When the gtk-doc bits were removed from systemd, this hack was dropped again [2] I've been looking into a bug where in certain circumstances, when linking with gold, string literal function arguments are corrupted. This problem was also breaking qt, specifically moc. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=773590 Looks like this should be fixed in binutils for good instead of having individual packages work around that. Does any want to try rebuilding the package with ld.bfd to check if it's working properly then? Michael [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=760879 [2] https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?id=ff3e6f6fc82adeeb5341b3bbd9824b2591965af6 -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
On Thu, 2015-08-06 at 16:57 +0200, Michael Biebl wrote: Looks like this should be fixed in binutils for good instead of having individual packages work around that. They are active changing the sources in git the past months. Searching for Sparc. Question, Is the bug stil there? https://sourceware.org/git/gitweb.cgi?p=binutils -gdb.gita=searchh=HEADst=commits=sparc Frans van Berckel -- To UNSUBSCRIBE, email to debian-sparc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1438875665.1642.15.ca...@xs4all.nl
Re: Bug#790560: udev fails to start on sparc boot, breaking boot
Am 30.06.2015 um 10:04 schrieb Meelis Roos: Package: udev Version: 221-1 Severity: critical Justification: breaks the whole system udev 220-7 broke sparc boot with strange messages about different options of udevadm not supported (--cleanup-db un recognized, --action=add not recognized, --timeout=10 not recognized). Upgraded to 221-1 with init=/bin/bash and chroot, still the same: Loading, please wait... e or neveruudevadm: unrecognized option '--action=add' Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount .[ 63.869458] input: Sun Mouse as /devices/root/f005f9c0/f00601b4/f0061504/f0064df4/serio1/input/1 ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done. udevadm: unrecognized option '--timeout=10' Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. done. Gave up waitin[ 94.135656] usbcore: registered new interface driver usbfs g for [ 94.207229] usbcore: registered new interface driver hub root [ 94.276486] usbcore: registered new device driver usb device. Comm[ 94.350131] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver on prob[ 94.435984] ehci-pci: EHCI PCI platform driver lems: - Boot args (cat /proc/cmdline) - Check rootdelay= (did[ 94.558185] uhci_hcd: USB Universal Host Controller Interface driver the system wait long e[ 94.658718] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver nough?) - Check root= (did[ 94.763559] hidraw: raw HID events driver (C) Jiri Kosina the system [ 94.841188] usbcore: registered new interface driver usbhid wait [ 94.912277] usbhid: USB HID core driver for the right device?) - Missing modules (cat /proc/modules; ls /dev) ALERT! /dev/sda1 does not exist. Dropping to a shell! Kernel is 4.0.5-1 from debian (4.0.0-2-sparc64). -- Package-specific info: -- System Information: Debian Release: stretch/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'stable') Architecture: sparc (sparc64) CCing our debian-sparc porters mailing list. Would be great if the sparc porters can look into this issue. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature