Hi Brian I have the file, can I get a manta url please ?
Thanks Len On Tue, 20 Sep 2016 at 20:32 Brian Bennett <[email protected]> wrote: > You can either put it somewhere I can download it, or I can give you a > Manta URL to upload to. > > > -- > Brian Bennett > Systems Engineer, Cloud Operations > Joyent, Inc. | www.joyent.com > > On Sep 20, 2016, at 11:22 AM, Len Weincier <[email protected]> wrote: > > Hi Brian > > How do we get the dump to you ? > > Thanks > Len > > > On Tue, 20 Sep 2016 at 20:21 Brian Bennett <[email protected]> > wrote: > > Len or Zak, would either of you be able to provide us with a crash dump? >> It should be in /var/crash/volatile on any CN that had this panic occur. >> >> >> -- >> Brian Bennett >> Systems Engineer, Cloud Operations >> Joyent, Inc. | www.joyent.com >> >> On Sep 19, 2016, at 5:09 AM, Len Weincier <[email protected]> wrote: >> >> Hi >> >> This has now happened to 3 hosts in the last 3 days. Any idea what we can >> look at ? >> >> It seems to happen under high load on those systems, all older E5 based >> hosts. >> >> We just had another reboot and this is in /var/adm/messages >> >> 2016-09-19T11:49:39.065057+00:00 c1a unix: [ID 836849 kern.notice] >> #012#015panic[cpu14]/thread=ffffff19e4687420: >> 2016-09-19T11:49:39.065068+00:00 c1a genunix: [ID 761616 kern.notice] >> turnstile_block(ffffff19ab6e9230): unowned mutex >> 2016-09-19T11:49:39.065074+00:00 c1a unix: [ID 100000 kern.notice] #012 >> 2016-09-19T11:49:39.065079+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689380 genunix:turnstile_block+78a () >> 2016-09-19T11:49:39.065084+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba6893f0 unix:mutex_vector_enter+3a3 () >> 2016-09-19T11:49:39.065089+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba6894c0 vnd:vnd_mac_input+12a () >> 2016-09-19T11:49:39.065094+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689580 dls:dls_rx_promisc+119 () >> 2016-09-19T11:49:39.065099+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba6895e0 mac:mac_promisc_dispatch_one+81 () >> 2016-09-19T11:49:39.065104+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689660 mac:mac_promisc_dispatch+b2 () >> 2016-09-19T11:49:39.065109+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689750 mac:mac_tx_send+33f () >> 2016-09-19T11:49:39.065114+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba6897f0 mac:mac_tx_single_ring_mode+6e () >> 2016-09-19T11:49:39.065132+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba6898a0 mac:mac_tx+da () >> 2016-09-19T11:49:39.065139+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689950 dld:str_mdata_raw_fastpath_put+85 () >> 2016-09-19T11:49:39.065144+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689990 vnd:vnd_squeue_tx_one+6a () >> 2016-09-19T11:49:39.065149+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689a20 vnd:vnd_squeue_tx_drain+112 () >> 2016-09-19T11:49:39.065164+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689ac0 vnd:vnd_squeue_tx_append+103 () >> 2016-09-19T11:49:39.065170+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689b50 ip:squeue_enter+41c () >> 2016-09-19T11:49:39.065175+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689ba0 gsqueue:gsqueue_enter_one+43 () >> 2016-09-19T11:49:39.065179+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689c40 vnd:vnd_frameio_write+10e () >> 2016-09-19T11:49:39.065184+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689cc0 vnd:vnd_ioctl+270 () >> 2016-09-19T11:49:39.065196+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689d00 genunix:cdev_ioctl+39 () >> 2016-09-19T11:49:39.065202+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689d50 specfs:spec_ioctl+60 () >> 2016-09-19T11:49:39.065208+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689de0 genunix:fop_ioctl+55 () >> 2016-09-19T11:49:39.065213+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689f00 genunix:ioctl+9b () >> 2016-09-19T11:49:39.065218+00:00 c1a genunix: [ID 655072 kern.notice] >> ffffff00ba689f10 unix:brand_sys_syscall+238 () >> 2016-09-19T11:49:39.065224+00:00 c1a unix: [ID 100000 kern.notice] >> >> >> Thanks >> Len >> >> On Mon, 19 Sep 2016 at 09:34 Zak McGregor <[email protected]> wrote: >> >>> Hi Brian >>> >>> Thanks, here's a full stack trace. >>> >>> Ciao >>> >>> Zak >>> >>> On 18 September 2016 at 21:58, Brian Bennett <[email protected]> >>> wrote: >>> > Zak, >>> > >>> > Considering that illumos #3917 is three years old, you've probably hit >>> a different bug involving mutexes. It would be best if you can give the >>> full stack trace, not just the top two frames. Having the full stack trace, >>> I may be able to identify the particular crash you encountered. >>> > >>> > Thanks. >>> > >>> > -- >>> > Brian Bennett >>> > Systems Engineer, Cloud Operations >>> > Joyent, Inc. | www.joyent.com >>> > >>> >> On Sep 16, 2016, at 4:04 AM, Zak McGregor <[email protected]> >>> wrote: >>> >> >>> >> Hi >>> >> >>> >> This issue here: >>> >> https://illumos.org/issues/3917 >>> >> >>> >> seems to have hit one of our production boxes today. I took a look at >>> >> the dump and it seems to tally precisely with this issue. >>> >> >>> >> Here's a snippet: >>> >> >>> >> mdb -k unix.0 vmcore.0 >>> >> Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc >>> >> pcplusmp scsi_vhci ufs ip hook neti sockfs arp usba uhci mm stmf_sbd >>> >> stmf zfs lofs idm crypto random cpc logindmux ptm kvm sd sppp nsmb >>> >> smbsrv nfs ipc ] >>> >>> ::status >>> >> debugging crash dump vmcore.0 (64-bit) from c7.jhb.cloudafrica.net >>> >> operating system: 5.11 joyent_20160330T234717Z (i86pc) >>> >> image uuid: (not set) >>> >> panic message: turnstile_block(ffffff3d8b4db508): unowned mutex >>> >> dump content: kernel pages only >>> >>> ::stack >>> >> vpanic() >>> >> turnstile_block+0x78a(0, 0, ffffff3e13c64d00, fffffffffbc07ac0, 0, 0) >>> >> mutex_vector_enter+0x3a3(ffffff3e13c64d00) >>> >> >>> >> If there is any further information you'd like please let me know. >>> >> >>> >> Thanks >>> >> >>> >> Cheers >>> >> >>> >> Zak >>> >> >>> > >>> > >>> >>> >> >> http://www.listbox.com >> > > ------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
