btw. This has nothing to do with ssh, it is to do with the length of time the juju cli process is running for. Normally this is a very short period, but juju ssh forks ssh as a child then waits for it to finish. The crash is caused by a bug in the previous runtime (compiled statically into the binary) when the garbage collector had been running for long enough to call madvise(DONT_NEED, ...) with the wrong page size.
On Sat, Jul 26, 2014 at 8:22 AM, Matt Bruzek <[email protected]> wrote: > With sinzui's help I have compiled the latest version of Juju 1.20.2 > with the gccgo-4.9_4.9.1-1ubuntu3_ppc64el.deb. > > NOT able to reproduce the “juju ssh” panic/problem having several ssh > sessions open simultaneously and having output streaming for over 2 > hours. > > For clarity and review here are the details of how we build the deb > packages for Juju with the gccgo 4.9.1 compiler on the CI machine > stilson-09 (a P8 64k page size machine). > > ubuntu@stilson-09:~⟫ uname -a > Linux stilson-09 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:29:18 UTC 2014 > ppc64le ppc64le ppc64le GNU/Linux > ubuntu@stilson-09:~⟫ getconf PAGE_SIZE > 65536 > ubuntu@stilson-09:~⟫ more /proc/cpuinfo > processor : 0 > cpu : POWER8E (raw), altivec supported > clock : 4116.000000MHz > revision : 2.0 (pvr 004b 0200) > > processor : 1 > cpu : POWER8E (raw), altivec supported > clock : 4116.000000MHz > revision : 2.0 (pvr 004b 0200) > > timebase : 512000000 > platform : pSeries > model : IBM pSeries (emulated by qemu) > machine : CHRP IBM pSeries (emulated by qemu) > > Downloaded the following packages from: > https://launchpad.net/ubuntu/utopic/ppc64el/gccgo-4.9/4.9.1-1ubuntu3 > > binutils_2.24.51.20140709-1ubuntu1_ppc64el.deb > cpp-4.9_4.9.1-1ubuntu3_ppc64el.deb > gcc-4.9_4.9.1-1ubuntu3_ppc64el.deb > gcc-4.9-base_4.9.1-1ubuntu3_ppc64el.deb > gccgo-4.9_4.9.1-1ubuntu3_ppc64el.deb > libatomic1_4.9.1-1ubuntu3_ppc64el.deb > libgcc1_4.9.1-1ubuntu3_ppc64el.deb > libgcc-4.9-dev_4.9.1-1ubuntu3_ppc64el.deb > libgo5_4.9.1-1ubuntu3_ppc64el.deb > libgomp1_4.9.1-1ubuntu3_ppc64el.deb > libitm1_4.9.1-1ubuntu3_ppc64el.deb > > Installed the packages on stilson-09: > > ubuntu@stilson-09:~/gccgo⟫ sudo dpkg -i *.deb > > ubuntu@stilson-09:~/gccgo⟫ dpkg -l | grep gcc > ii gcc 4:4.8.2-1ubuntu6 > ppc64el GNU C compiler > ii gcc-4.8 4.8.3-3ubuntu0.2 > ppc64el GNU C compiler > ii gcc-4.8-base:ppc64el 4.8.3-3ubuntu0.2 > ppc64el GCC, the GNU Compiler Collection (base package) > ii gcc-4.9 4.9.1-1ubuntu3 > ppc64el GNU C compiler > ii gcc-4.9-base:ppc64el 4.9.1-1ubuntu3 > ppc64el GCC, the GNU Compiler Collection (base package) > ii gccgo 4:4.9-1ubuntu6 > ppc64el Go compiler, based on the GCC backend > ii gccgo-4.9 4.9.1-1ubuntu3 > ppc64el GNU Go compiler > ii gccgo-go 1.2.1-0ubuntu1 > ppc64el Go tool for use with gccgo > ii libgcc-4.8-dev:ppc64el 4.8.3-3ubuntu0.2 > ppc64el GCC support library (development files) > ii libgcc-4.9-dev:ppc64el 4.9.1-1ubuntu3 > ppc64el GCC support library (development files) > ii libgcc1:ppc64el 1:4.9.1-1ubuntu3 > ppc64el GCC support library > > Built Juju from source on stilson-09: > > <sinzui> ~/Work/juju-release-tools/make-source-packages.bash stable > juju-core_1.20.2.tar.gz 'Curtis Hovey <[email protected]>' 1340893 > 1345014 1344940 1345832 1307434 1347371 1337091 1342747 1343301 1341589 > <sinzui> ~/Work/juju-release-tools/build-package-with-dsc.bash > ubuntu@stilson-09 '' > /home/curtis/Downloads/tmp.WUZeXOfFz1/juju-core_1.20.2-0ubuntu1~14.04.1~juju1.dsc > > Resulting in: > > ubuntu@stilson-09:~⟫ ls -l juju-build > total 17332 > -rw-r--r-- 1 ubuntu ubuntu 6944 Jul 25 15:05 > juju_1.20.2-0ubuntu1~14.04.1~juju1_all.deb > drwxrwxr-x 7 ubuntu ubuntu 4096 Jul 25 15:04 juju-core-1.20.2 > -rw------- 1 ubuntu ubuntu 8356 Jul 25 15:04 > juju-core_1.20.2-0ubuntu1~14.04.1~juju1.debian.tar.gz > -rw------- 1 ubuntu ubuntu 1183 Jul 25 15:04 > juju-core_1.20.2-0ubuntu1~14.04.1~juju1.dsc > -rw-rw-r-- 1 ubuntu ubuntu 3133 Jul 25 15:05 > juju-core_1.20.2-0ubuntu1~14.04.1~juju1_ppc64el.changes > -rw-r--r-- 1 ubuntu ubuntu 14341816 Jul 25 15:05 > juju-core_1.20.2-0ubuntu1~14.04.1~juju1_ppc64el.deb > -rw-rw-r-- 1 ubuntu ubuntu 3354392 Jul 25 15:03 juju-core_1.20.2.orig.tar.gz > -rw-r--r-- 1 ubuntu ubuntu 7020 Jul 25 15:05 > juju-local_1.20.2-0ubuntu1~14.04.1~juju1_all.deb > -rw-r--r-- 1 ubuntu ubuntu 7022 Jul 25 15:05 > juju-local-kvm_1.20.2-0ubuntu1~14.04.1~juju1_all.deb > > I copied these deb packages to stilson-01: > ubuntu@stilson-01:~$ ls -l juju-build > total 14032 > -rw-r--r-- 1 ubuntu ubuntu 6944 Jul 25 15:21 > juju_1.20.2-0ubuntu1~14.04.1~juju1_all.deb > -rw-r--r-- 1 ubuntu ubuntu 14341816 Jul 25 15:22 > juju-core_1.20.2-0ubuntu1~14.04.1~juju1_ppc64el.deb > -rw-r--r-- 1 ubuntu ubuntu 7020 Jul 25 15:22 > juju-local_1.20.2-0ubuntu1~14.04.1~juju1_all.deb > -rw-r--r-- 1 ubuntu ubuntu 7022 Jul 25 15:22 > juju-local-kvm_1.20.2-0ubuntu1~14.04.1~juju1_all.deb > > Installed the deb packages on stilson-01: > > sudo dpkg -i *.deb > > ubuntu@stilson-01:~$ dpkg -l | grep juju > ii juju 1.20.2-0ubuntu1~14.04.1~juju1 > all next generation service orchestration system > ii juju-core 1.20.2-0ubuntu1~14.04.1~juju1 > ppc64el Juju is devops distilled - client > ii juju-deployer 0.3.6-0ubuntu2 > all Deploy complex stacks of services using Juju > ii juju-jitsu 0.20-1 > all external tools to enhance juju > ii juju-local 1.20.2-0ubuntu1~14.04.1~juju1 > all dependency package for the Juju local provider > ii juju-local-kvm 1.20.2-0ubuntu1~14.04.1~juju1 > all dependency package for the Juju local provider > ii juju-mongodb 2.4.9-0ubuntu3 > ppc64el MongoDB object/document-oriented database for Juju > ii juju-quickstart 1.4.1+bzr88+ppa25~ubuntu14.04.1 > all Easy configuration of Juju environments > ii python-jujuclient 0.17.5-0ubuntu2 > all Python API client for juju > > Examined the strings in juju and jujud: > > ubuntu@stilson-01:~$ strings $(which juju) | grep gcc > __gccgo_personality_v0 > libgcc_s.so.1 > -gccgoflags=-static-libgo > gccgo > ubuntu@stilson-01:~$ strings $(which jujud) | grep gcc > libgcc_s.so.1 > gccgo > -gccgoflags=-static-libgo > gccgo > xgcc (Ubuntu 4.9.1-1ubuntu3) 4.9.1 > > > Doing some testing on power that involves running the “juju test” command > which bootstraps an environment deploys waits for success/fail and then > destroys the environment. The test run will attempt to run 141 times and is > on 9 iterations without a juju failure. I am confident this gccgo fix is > good. > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1347322 > > Title: > juju ssh results in a panic: runtime error > > Status in juju-core: > Triaged > Status in “juju-core” package in Ubuntu: > Confirmed > > Bug description: > I am using Juju on Power 8 hardware and I get a panic when I use juju > to ssh to a system. The reproduction steps are > > juju init > juju bootstrap -e local > juju switch local > juju deploy local:trusty/ubuntu > juju ssh ubuntu/0 > > (use the session to the local machine and after some time the panic > happens and the screen is unreadable.) > > ubuntu@ubuntu-local-machine-1:~$ panic: runtime error: invalid memory > address or nil pointer dereference > [signal 0xb code=0x1 addr=0x8] > > goroutine 11 [running]: > > code.google.com_p_go.net_websocket.Send.N40_code. > google.com_p_go.net_websocket.Codec > > /build/buildd/juju-core-1.18.1/src/code.google.com/p/go.net/webso > cket/websocket.go:293 > > launchpad.net_juju_core_rpc_jsoncodec.Send.N48_launchpad.net_juju_core_rpc_jsonc > odec.wsJSONConn > > /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/rpc/jsoncodec/conn.go: > 21 > > launchpad.net_juju_core_rpc_jsoncodec.WriteMessage.pN43_launchpad.net_juju_core_rpc_jsoncodec.Codec > > /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/rpc/jsoncodec/codec.go:178 > > launchpad.ne > t_juju_core_rpc.send.pN32_launchpad.net_juju_core_rpc.Conn > > /build/buildd/juju-core-1.18.1/src/launch > pad.net/juju-core/rpc/client.go:72 > > launchpad.net_juju_core_rpc.Go.pN32_launchpad.net_juju_core_rpc.Con > n > > /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/rpc/client.go:174 > > launchpad.net_juju_co > re_rpc.Call.pN32_launchpad.net_juju_core_rpc.Conn > > /build/buildd/juju-core-1.18.1/src/launchpad.net/ > juju-core/rpc/client.go:148 > > launchpad.net_juju_core_state_api.Call.pN39_launchpad.net_juju_core_state_ > api.State > > /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/state/api/apiclient.go:168 > > launchpad.net_juju_core_state_api.Ping.pN39_launchpad.net_juju_core_state_api.State > > /build/buildd/juj > u-core-1.18.1/src/launchpad.net/juju-core/state/api/apiclient.go:158 > > launchpad.net_juju_core_state_api > .heartbeatMonitor.pN39_launchpad.net_juju_core_state_api.State > > /build/buildd/juju-core-1.18.1/sr > c/launchpad.net/juju-core/state/api/apiclient.go:149 > created by > launchpad.net_juju_core_state_api.Open > > /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/state/api/apiclient.go:143 > > > goroutine 1 > [syscall]: > goroutine in C code; stack unavailable > > goroutine 3 [syscall]: > > goroutine in C code; stac > k unavailable > > goroutine 10 [IO wait]: > > code.google.com_p_go.net_websocket.ReadByte.N57_code.google.com_p > _go.net_websocket.hybiFrameReaderFactory > > /build/buildd/juju-core-1.18.1/src/code.google.com/p/go.n > et/websocket/hybi.go:113 > > code.google.com_p_go.net_websocket.NewFrameReader.N57_code.google.com_p_go.ne > t_websocket.hybiFrameReaderFactory > > /build/buildd/juju-core-1.18.1/src/code.google.com/p/go.net/webso > cket/hybi.go:126 > > code.google.com_p_go.net_websocket.Receive.N40_code.google.com_p_go.net_websocket.Cod > ec > > /build/buildd/juju-core-1.18.1/src/code.google.com/p/go.net/websocket/websocket.go:314 > > launchpad.n > > et_juju_core_rpc_jsoncodec.Receive.N48_launchpad.net_juju_core_rpc_jsoncodec.wsJSONConn > > /build/bu > ildd/juju-core-1.18.1/src/launchpad.net/juju-core/rpc/jsoncodec/conn.go:25 > > launchpad.net_juju_core_rpc > _jsoncodec.ReadHeader.pN43_launchpad.net_juju_core_rpc_jsoncodec.Codec > > /build/buildd/juju-core-1 > .18.1/src/launchpad.net/juju-core/rpc/jsoncodec/codec.go:113 > > launchpad.net_juju_core_rpc.loop.pN32_lau > nchpad.net_juju_core_rpc.Conn > > /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/rpc/se > rver.go:344 > > launchpad.net_juju_core_rpc.input.pN32_launchpad.net_juju_core_rpc.Conn > > /build/buildd/juj > u-core-1.18.1/src/launchpad.net/juju-core/rpc/server.go:317 > created by > launchpad.net_juju_core_rpc.Sta > rt.pN32_launchpad.net_juju_core_rpc.Conn > > /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-cor > e/rpc/server.go:200 > > Here are the specifics on the host system. > > ubuntu@stilson-01:~$ uname -a > Linux stilson-01 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:50:31 UTC > 2014 ppc64le ppc64le ppc64le GN > U/Linux > ubuntu@stilson-01:~$ dpkg -l | grep juju > ii juju 1.18.1-0ubuntu1.1 > all next generation > service orchestration system > ii juju-core 1.18.1-0ubuntu1.1 > ppc64el Juju is devops > distilled - client > ii juju-deployer 0.3.6-0ubuntu2 > all Deploy complex > stacks of services using Juju > ii juju-jitsu 0.20-1 > all external tools > to enhance juju > ii juju-local 1.18.1-0ubuntu1.1 > all dependency pack > age for the Juju local provider > ii juju-mongodb 2.4.9-0ubuntu3 > ppc64el MongoDB object/ > document-oriented database for Juju > ii juju-quickstart 1.4.1+bzr88+ppa25~ubuntu14.04.1 > all Easy configurat > ion of Juju environments > ii python-jujuclient 0.17.5-0ubuntu2 > all Python API clie > nt for juju > ubuntu@stilson-01:~$ getconf PAGE_SIZE > 65536 > > I got this error when testing a fix for another bug: > https://bugs.launchpad.net/ubuntu/+source/gccgo-4.9/+bug/1304754 > > Since the dmesg looks different we believe this is a new problem. > > To manage notifications about this bug go to: > https://bugs.launchpad.net/juju-core/+bug/1347322/+subscriptions -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1347322 Title: juju ssh results in a panic: runtime error To manage notifications about this bug go to: https://bugs.launchpad.net/juju-core/+bug/1347322/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
