I had a different server reboot this morning. Had ZFS kernel errors. Do you want this crash dump too?
Any word on the 1 sent in on Friday? I assume not because of the weekend but I just thought I would ask. Im nervous to get into Monday :) Sent from Outlook Mobile<https://aka.ms/qtex0l> _____________________________ From: Robert Mustacchi <[email protected]<mailto:[email protected]>> Sent: Friday, January 8, 2016 5:13 PM Subject: Re: [smartos-discuss] Re: joyent_20150108T111855Z Kernel Panic To: <[email protected]<mailto:[email protected]>> Hi, I'm sorry that you're seeing these panics. The first thing that I'd do is go through and increase your dump device size. The older forms of the installers were a bit pessimal in how they sized them. I would recommend sizing it to either half or all of memory. That is specifically the ZFS volume zones/dump. You may need to remove the old dump device and then create and destroy the ZFS dataset and then enable the dump device. The second thing that's a bit worrying is the fact that we're seeing the kernel heap corruption. It may come at a large cost, but it may makes sense to turn on kmem debugging to get a better sense of what's going on. That can be done by setting kmem_flags to 0xf while booting via kmdb. If you need help on that, feel free to reach out on our IRC channel, #smartos on irc.freenode.net<http://irc.freenode.net>. Robert On 1/8/16 15:04 , John Barfield wrote: > It just happened again: > > > > 2016-01-08T22:45:48.616347+00:00 00-21-28-b2-4f-ea genunix: [ID 432124 > kern.notice] buffer freed to wrong cache > 2016-01-08T22:45:48.616355+00:00 00-21-28-b2-4f-ea genunix: [ID 815666 > kern.notice] buffer was allocated from zio_data_buf_4096, > 2016-01-08T22:45:48.616360+00:00 00-21-28-b2-4f-ea genunix: [ID 530907 > kern.notice] caller attempting free to zio_data_buf_8192. > 2016-01-08T22:45:48.616365+00:00 00-21-28-b2-4f-ea genunix: [ID 563406 > kern.notice] buffer=ffffff14b54a5000 bufctl=ffffff2cd356c020 cache: > zio_data_buf_8192 > 2016-01-08T22:45:48.616370+00:00 00-21-28-b2-4f-ea unix: [ID 836849 > kern.notice] #012#015panic[cpu6]/thread=ffffff0116017c40: > 2016-01-08T22:45:48.616375+00:00 00-21-28-b2-4f-ea genunix: [ID 812275 > kern.notice] kernel heap corruption detected > > 2016-01-08T22:45:48.616390+00:00 00-21-28-b2-4f-ea unix: [ID 100000 > kern.notice] #012 > 2016-01-08T22:45:48.616402+00:00 00-21-28-b2-4f-ea genunix: [ID 802836 > kern.notice] ffffff0116017a50 fffffffffba2c2f4 () > 2016-01-08T22:45:48.616412+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 > kern.notice] ffffff0116017ab0 genunix:kmem_slab_free+c1 () > 2016-01-08T22:45:48.616420+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 > kern.notice] ffffff0116017b20 genunix:kmem_magazine_destroy+6e () > 2016-01-08T22:45:48.616428+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 > kern.notice] ffffff0116017b60 genunix:kmem_depot_ws_reap+5d () > 2016-01-08T22:45:48.616435+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 > kern.notice] ffffff0116017c20 genunix:taskq_thread+2d0 () > 2016-01-08T22:45:48.616441+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 > kern.notice] ffffff0116017c30 unix:thread_start+8 () > 2016-01-08T22:45:48.616449+00:00 00-21-28-b2-4f-ea unix: [ID 100000 > kern.notice] > 2016-01-08T22:45:48.616456+00:00 00-21-28-b2-4f-ea genunix: [ID 672855 > kern.notice] syncing file systems... > 2016-01-08T22:45:48.616464+00:00 00-21-28-b2-4f-ea genunix: [ID 904073 > kern.notice] done > 2016-01-08T22:45:48.616472+00:00 00-21-28-b2-4f-ea genunix: [ID 111219 > kern.notice] dumping to /dev/zvol/dsk/zones/dump, offset 65536, content: > kernel > 2016-01-08T22:45:48.616480+00:00 00-21-28-b2-4f-ea genunix: [ID 100000 > kern.notice] > 2016-01-08T22:45:48.616488+00:00 00-21-28-b2-4f-ea genunix: [ID 665016 > kern.notice] #015 36% done: 2962335<tel:2962335> pages dumped, > 2016-01-08T22:45:48.616495+00:00 00-21-28-b2-4f-ea genunix: [ID 495082 > kern.notice] dump failed: error 28 > > > > > > John Barfield / Sr Principal Engineer > > +1 (214) 425-0783<tel:+1%20(214)%20425-0783>/ > [email protected]<mailto:[email protected]><mailto:[email protected]> > > BISS, Inc. Office: +1 (214) 506-8354<tel:+1%20(214)%20506-8354> > > 4925 Greenville Ave Suite 900<x-apple-data-detectors://31/1> > > Dallas, TX 75206 > > support.bissinc.com<http://support.bissinc.com><http://htmlsig.com/support.bissinc.com> > > This e-mail message may contain confidential or legally privileged > information and is intended only for the use of the intended recipient(s). > Any unauthorized disclosure, dissemination, distribution, copying or the > taking of any action in reliance on the information herein is prohibited. > E-mails are not secure and cannot be guaranteed to be error free as they can > be intercepted, amended, or contain viruses. Anyone who communicates with us > by e-mail is deemed to have accepted these risks. Company Name is not > responsible for errors or omissions in this message and denies any > responsibility for any damage arising from the use of e-mail. Any opinion and > other statement contained in this message and any attachment are solely those > of the author and do not necessarily represent those of the company. > > From: John Barfield > <[email protected]<mailto:[email protected]><mailto:[email protected]>> > Date: Friday, January 8, 2016 at 10:31 AM > To: > "[email protected]<mailto:[email protected]><mailto:[email protected]>" > > <[email protected]<mailto:[email protected]><mailto:[email protected]>> > Subject: joyent_20150108T111855Z Kernel Panic > > Greetings List, > > Brace yourselves: > > I’m writing because I had 3 servers give me kernel panics yesterday and > reboot themselves around the same time. (Effectively rebooting about 20 VM’s > as well). > > /var/adm/messages available here: > http://download.txtelecom.net/server-34-36-37.tar.gz for all 3 servers. > > If you have the time and patience, please let me know what you think :) > > I’m happy to provide more information if needed and potential partial dumps > if someone can tell me a little bit more about how to get them and If its of > any use. > > Also, let me know if I should post this to the illumos list as well or > instead. > > John Barfield > > ________________________________ > > > ### Rambling thoughts and info follow starting here: > > All 3 servers are using joyent_20150108T111855Z kernel on Sunfire X4170 M2’s > with 144GB of RAM each. They have dual hex core Intel’s at 2.66ghz with hyper > threading enabled. Zpool has L2Arc cache device and mirrored SLOG’s (All are > SAS drives, all have 1 raidz1 vdev) > > Let me say that I am not a kernel engineer at all, but I started following > Ben Rockwoods coredump guide trying to get a proper dump to analyze with mdb > and it keeps failing with the following: > > 'mdb: failed to read 'platform' string from kernel: No such file or directory > mdb: failed to initialize target: No such file or directory’ > > I’m not sure if I’m doing it wrong or not but besides the error, the logs > actually had a message that said the coredump was unsuccessful. > > Anyway, since I don’t have a viable core dump, I went reading through the > panic messages in the logs and just figured I would post them here for > someone else to see what they think from /var/adm/messages. > I could be way off but I think the kernel panic occurred when a VM wrote to a > zvol due to Qemu/ZIO errors on servers 1 & 3, but then I see a ZFS buffer > written to wrong cache on server 2. Could be a cascading event that started > between VM’s on srv2 in theory. > > Anyway, please, if you get a chance, someone tell me what you think happened, > and if an upgrade could resolve a bug (and which bug??). Unless someone > thinks its naughty memory or something, blah, I’m just rambling. > > My thoughts on Potential bugs: > https://www.illumos.org/issues/5504 > http://www.listbox.com/member/archive/182191/2015/01/sort/time_rev/page/1/entry/11:84/20150126102848:0193750E-A570-11E4-93E5-83F5155B1524/ > > ### End Rambling. > > > > Relevant bits in the logs: > > Server 1: > > 2016-01-08T02:07:55.180609+00:00 00-21-28-b2-24-6a unix: [ID 836849 > kern.notice] #012#015panic[cpu0]/thread=ffffff011765dc40: > 2016-01-08T02:07:55.180621+00:00 00-21-28-b2-24-6a genunix: [ID 683410 > kern.notice] BAD TRAP: type=e (#pf Page fault) rp=ffffff011765d890 > addr=ffffff0b4d92d000 > 2016-01-08T02:07:55.180627+00:00 00-21-28-b2-24-6a unix: [ID 100000 > kern.notice] #012 > 2016-01-08T02:07:55.180632+00:00 00-21-28-b2-24-6a unix: [ID 839527 > kern.notice] zpool-zones: > 2016-01-08T02:07:55.180637+00:00 00-21-28-b2-24-6a unix: [ID 753105 > kern.notice] #pf Page fault > 2016-01-08T02:07:55.180642+00:00 00-21-28-b2-24-6a unix: [ID 532287 > kern.notice] Bad kernel fault at addr=0xffffff0b4d92d000 > 2016-01-08T02:07:55.180658+00:00 00-21-28-b2-24-6a unix: [ID 243837 > kern.notice] pid=90, pc=0xfffffffffb85414a, sp=0xffffff011765d988, > eflags=0x10206 > 2016-01-08T02:07:55.180669+00:00 00-21-28-b2-24-6a unix: [ID 211416 > kern.notice] cr0: 8005003b<pg,wp,ne,et,ts,mp,pe> cr4: > 26f8<vmxe,xmme,fxsr,pge,mce,pae,pse,de> > 2016-01-08T02:07:55.180675+00:00 00-21-28-b2-24-6a unix: [ID 624947 > kern.notice] cr2: ffffff0b4d92d000 > 2016-01-08T02:07:55.180680+00:00 00-21-28-b2-24-6a unix: [ID 625075 > kern.notice] cr3: 1dc00000 > 2016-01-08T02:07:55.180685+00:00 00-21-28-b2-24-6a unix: [ID 625715 > kern.notice] cr8: c > 2016-01-08T02:07:55.180690+00:00 00-21-28-b2-24-6a unix: [ID 100000 > kern.notice] > 2016-01-08T02:07:55.180695+00:00 00-21-28-b2-24-6a unix: [ID 592667 > kern.notice] #011rdi: ffffff0b4d92d000 rsi: ffffff2b4973c000 rdx: 1000 > 2016-01-08T02:07:55.180701+00:00 00-21-28-b2-24-6a unix: [ID 592667 > kern.notice] #011rcx: 200 r8: 0 r9: ffffff26349d4000 > 2016-01-08T02:07:55.180706+00:00 00-21-28-b2-24-6a unix: [ID 592667 > kern.notice] #011rax: ffffff2b4d989b30 rbx: ffffff2b3e0f9498 rbp: > ffffff011765d9b0 > 2016-01-08T02:07:55.180711+00:00 00-21-28-b2-24-6a unix: [ID 592667 > kern.notice] #011r10: 6 r11: 0 r12: 0 > 2016-01-08T02:07:55.180716+00:00 00-21-28-b2-24-6a unix: [ID 592667 > kern.notice] #011r13: 0 r14: 0 r15: 0 > 2016-01-08T02:07:55.180721+00:00 00-21-28-b2-24-6a unix: [ID 592667 > kern.notice] #011fsb: fffffd7fff071240 gsb: fffffffffbc32600 ds: 38 > 2016-01-08T02:07:55.180726+00:00 00-21-28-b2-24-6a unix: [ID 592667 > kern.notice] #011 es: 38 fs: 0 gs: 0 > 2016-01-08T02:07:55.180731+00:00 00-21-28-b2-24-6a unix: [ID 592667 > kern.notice] #011trp: e err: 2 rip: fffffffffb85414a > 2016-01-08T02:07:55.180738+00:00 00-21-28-b2-24-6a unix: [ID 592667 > kern.notice] #011 cs: 30 rfl: 10206 rsp: ffffff011765d988 > 2016-01-08T02:07:55.180743+00:00 00-21-28-b2-24-6a unix: [ID 266532 > kern.notice] #011 ss: 38 > 2016-01-08T02:07:55.180750+00:00 00-21-28-b2-24-6a unix: [ID 100000 > kern.notice] > 2016-01-08T02:07:55.180756+00:00 00-21-28-b2-24-6a genunix: [ID 655072 > kern.notice] ffffff011765d770 unix:real_mode_stop_cpu_stage2_end+9fb3 () > 2016-01-08T02:07:55.180761+00:00 00-21-28-b2-24-6a genunix: [ID 655072 > kern.notice] ffffff011765d880 unix:trap+db3 () > 2016-01-08T02:07:55.180768+00:00 00-21-28-b2-24-6a genunix: [ID 655072 > kern.notice] ffffff011765d890 unix:cmntrap+e6 () > 2016-01-08T02:07:55.180791+00:00 00-21-28-b2-24-6a genunix: [ID 655072 > kern.notice] ffffff011765d9b0 unix:bcopy+55a () > 2016-01-08T02:07:55.180800+00:00 00-21-28-b2-24-6a genunix: [ID 655072 > kern.notice] ffffff011765da40 zfs:zio_done+342 () > 2016-01-08T02:07:55.180807+00:00 00-21-28-b2-24-6a genunix: [ID 655072 > kern.notice] ffffff011765da70 zfs:zio_execute+78 () > 2016-01-08T02:07:55.180814+00:00 00-21-28-b2-24-6a genunix: [ID 655072 > kern.notice] ffffff011765db30 genunix:taskq_thread+2d0 () > 2016-01-08T02:07:55.180819+00:00 00-21-28-b2-24-6a genunix: [ID 655072 > kern.notice] ffffff011765db40 unix:thread_start+8 () > 2016-01-08T02:07:55.180826+00:00 00-21-28-b2-24-6a unix: [ID 100000 > kern.notice] > 2016-01-08T02:07:55.180834+00:00 00-21-28-b2-24-6a genunix: [ID 672855 > kern.notice] syncing file systems... > 2016-01-08T02:07:55.180840+00:00 00-21-28-b2-24-6a genunix: [ID 733762 > kern.notice] 17 > 2016-01-08T02:07:55.180847+00:00 00-21-28-b2-24-6a genunix: [ID 904073 > kern.notice] done > 2016-01-08T02:07:55.180854+00:00 00-21-28-b2-24-6a genunix: [ID 111219 > kern.notice] dumping to /dev/zvol/dsk/zones/dump, offset 65536, content: > kernel > 2016-01-08T02:07:55.180860+00:00 00-21-28-b2-24-6a genunix: [ID 100000 > kern.notice] > 2016-01-08T02:07:55.180868+00:00 00-21-28-b2-24-6a genunix: [ID 665016 > kern.notice] #015 51% done: 3210871<tel:3210871> pages dumped, > 2016-01-08T02:07:55.180874+00:00 00-21-28-b2-24-6a genunix: [ID 495082 > kern.notice] dump failed: error 28 > > > Server2: > -01-08T03:12:39.904338+00:00 00-10-e0-0f-3f-a0 genunix: [ID 432124 > kern.notice] buffer freed to wrong cache > 2016-01-08T03:12:39.904347+00:00 00-10-e0-0f-3f-a0 genunix: [ID 815666 > kern.notice] buffer was allocated from zio_data_buf_4096, > 2016-01-08T03:12:39.904352+00:00 00-10-e0-0f-3f-a0 genunix: [ID 530907 > kern.notice] caller attempting free to zio_data_buf_8192. > 2016-01-08T03:12:39.904357+00:00 00-10-e0-0f-3f-a0 genunix: [ID 563406 > kern.notice] buffer=ffffff1397a14000 bufctl=ffffff2a46f33238 cache: > zio_data_buf_8192 > 2016-01-08T03:12:39.904362+00:00 00-10-e0-0f-3f-a0 unix: [ID 836849 > kern.notice] #012#015panic[cpu20]/thread=ffffff0116017c40: > 2016-01-08T03:12:39.904366+00:00 00-10-e0-0f-3f-a0 genunix: [ID 812275 > kern.notice] kernel heap corruption detected > 2016-01-08T03:12:39.904381+00:00 00-10-e0-0f-3f-a0 unix: [ID 100000 > kern.notice] #012 > 2016-01-08T03:12:39.904391+00:00 00-10-e0-0f-3f-a0 genunix: [ID 802836 > kern.notice] ffffff0116017a50 fffffffffba2c2f4 () > 2016-01-08T03:12:39.904397+00:00 00-10-e0-0f-3f-a0 genunix: [ID 655072 > kern.notice] ffffff0116017ab0 genunix:kmem_slab_free+c1 () > 2016-01-08T03:12:39.904402+00:00 00-10-e0-0f-3f-a0 genunix: [ID 655072 > kern.notice] ffffff0116017b20 genunix:kmem_magazine_destroy+6e () > 2016-01-08T03:12:39.904406+00:00 00-10-e0-0f-3f-a0 genunix: [ID 655072 > kern.notice] ffffff0116017b60 genunix:kmem_depot_ws_reap+5d () > 2016-01-08T03:12:39.904411+00:00 00-10-e0-0f-3f-a0 genunix: [ID 655072 > kern.notice] ffffff0116017c20 genunix:taskq_thread+2d0 () > 2016-01-08T03:12:39.904416+00:00 00-10-e0-0f-3f-a0 genunix: [ID 655072 > kern.notice] ffffff0116017c30 unix:thread_start+8 () > 2016-01-08T03:12:39.904420+00:00 00-10-e0-0f-3f-a0 unix: [ID 100000 > kern.notice] > > > Server3: > 2016-01-07T23:30:29.392494+00:00 00-21-28-b2-4f-ea unix: [ID 836849 > kern.notice] #012#015panic[cpu6]/thread=ffffff264340e880: > 2016-01-07T23:30:29.392507+00:00 00-21-28-b2-4f-ea genunix: [ID 683410 > kern.notice] BAD TRAP: type=e (#pf Page fault) rp=ffffff01171e0450 > addr=ffffff1529242000 2016-01-07T23:30:29.392512+00:00 00-21-28-b2-4f-ea > unix: [ID 100000 kern.notice] #012 2016-01-07T23:30:29.392518+00:00 > 00-21-28-b2-4f-ea unix: [ID 839527 kern.notice] qemu-system-x86_: > 2016-01-07T23:30:29.392523+00:00 00-21-28-b2-4f-ea unix: [ID 753105 > kern.notice] #pf Page fault 2016-01-07T23:30:29.392528+00:00 > 00-21-28-b2-4f-ea unix: [ID 532287 kern.notice] Bad kernel fault at > addr=0xffffff1529242000 2016-01-07T23:30:29.392545+00:00 00-21-28-b2-4f-ea > unix: [ID 243837 kern.notice] pid=4465, pc=0xfffffffff7dc5577, > sp=0xffffff01171e0540, eflags=0x10282 2016-01-07T23:30:29.392556+00:00 > 00-21-28-b2-4f-ea unix: [ID 211416 kern.notice] cr0: > 80050033<pg,wp,ne,et,mp,pe> cr4: 26f8<vmxe,xmme,fxsr,pge,mce,pae,ps e,de> 2016-01-07T23:30:29.392562+00:00 00-21-28-b2-4f-ea unix: [ID 624947 kern.notice] cr2: ffffff1529242000 2016-01-07T23:30:29.392568+00:00 00-21-28-b2-4f-ea unix: [ID 625075 kern.notice] cr3: 1213265000<tel:1213265000> 2016-01-07T23:30:29.392573+00:00 00-21-28-b2-4f-ea unix: [ID 625715 kern.notice] cr8: c 2016-01-07T23:30:29.392578+00:00 00-21-28-b2-4f-ea unix: [ID 100000 kern.notice] 2016-01-07T23:30:29.392583+00:00 00-21-28-b2-4f-ea unix: [ID 592667 kern.notice] #011rdi: ffffff39b2128c04 rsi: ffffff39b2128c0d rdx: ffffff1529243ff7 2016-01-07T23:30:29.392589+00:00 00-21-28-b2-4f-ea unix: [ID 592667 kern.notice] #011rcx: 200000098393bc0 r8: ffffff39b2128c61 r9: ffffff1529243ff8 2016-01-07T23:30:29.392594+00:00 00-21-28-b2-4f-ea unix: [ID 592667 kern.notice] #011rax: ffffff1529242000 rbx: ffffff1529242000 rbp: ffffff01171e0610 2016-01-07T23:30:29.392600+00:00 00-21-28-b2-4f-ea unix: [ID 592667 kern.notice] #011r10: ffffff1529244000 r11: ffffff39b2128c59 r12: ffffff152924200d 2016-01-07T23:30:29.392 604+00:00 00-21-28-b2-4f-ea unix: [ID 592667 kern.notice] #011r13: df r14: ffffff39b2128c12 r15: 0 2016-01-07T23:30:29.392610+00:00 00-21-28-b2-4f-ea unix: [ID 592667 kern.notice] #011fsb: fffffd7fff07e240 gsb: ffffff26016beac0 ds: 4b 2016-01-07T23:30:29.392615+00:00 00-21-28-b2-4f-ea unix: [ID 592667 kern.notice] #011 es: 4b fs: 0 gs: 0 2016-01-07T23:30:29.392620+00:00 00-21-28-b2-4f-ea unix: [ID 592667 kern.notice] #011trp: e err: 2 rip: fffffffff7dc5577 2016-01-07T23:30:29.392625+00:00 00-21-28-b2-4f-ea unix: [ID 592667 kern.notice] #011 cs: 30 rfl: 10282 rsp: ffffff01171e0540 2016-01-07T23:30:29.392630+00:00 00-21-28-b2-4f-ea unix: [ID 266532 kern.notice] #011 ss: 38 2016-01-07T23:30:29.392635+00:00 00-21-28-b2-4f-ea unix: [ID 100000 kern.notice] 2016-01-07T23:30:29.392640+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0330 unix:real_mode_stop_cpu_stage2_end+9fb3 () 2016-01-07T23:30:29.392645+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff0117 1e0440 unix:trap+db3 () 2016-01-07T23:30:29.392650+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0450 unix:cmntrap+e6 () 2016-01-07T23:30:29.392667+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0610 zfs:LZ4_uncompress_unknownOutputSize+147 () 2016-01-07T23:30:29.392673+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0650 zfs:lz4_decompress+44 () 2016-01-07T23:30:29.392678+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0690 zfs:zio_decompress_data+4b () 2016-01-07T23:30:29.392682+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e06d0 zfs:zio_decompress+56 () 2016-01-07T23:30:29.392687+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0700 zfs:zio_pop_transforms+7a () 2016-01-07T23:30:29.392692+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0790 zfs:zio_done+21a () 2016-01-07T23:30:29.392697+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern. notice] ffffff01171e07c0 zfs:zio_execute+78 () 2016-01-07T23:30:29.392702+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e07f0 zfs:zio_nowait+21 () 2016-01-07T23:30:29.392706+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e08e0 zfs:arc_read+4fe () 2016-01-07T23:30:29.392711+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0980 zfs:dbuf_read_impl+1a5 () 2016-01-07T23:30:29.392716+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e09f0 zfs:dbuf_read+de () 2016-01-07T23:30:29.392721+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0a50 zfs:dmu_tx_check_ioerr+76 () 2016-01-07T23:30:29.392729+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0b80 zfs:dmu_tx_count_write+3e7 () 2016-01-07T23:30:29.392737+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0bd0 zfs:dmu_tx_hold_write+5a () 2016-01-07T23:30:29.392744+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0c80 zfs:zvol_write+1bc () 2016-01-07T23:30:29.392755+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0cb0 genunix:cdev_write+2d () 2016-01-07T23:30:29.392764+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0d90 specfs:spec_write+4c1 () 2016-01-07T23:30:29.392770+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0e30 genunix:fop_write+8b () 2016-01-07T23:30:29.392777+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0f00 genunix:pwrite+194 () 2016-01-07T23:30:29.392800+00:00 00-21-28-b2-4f-ea genunix: [ID 655072 kern.notice] ffffff01171e0f10 unix:brand_sys_syscall+1f5 () 2016-01-07T23:30:29.392809+00:00 00-21-28-b2-4f-ea unix: [ID 100000 kern.notice] 2016-01-07T23:30:29.392814+00:00 00-21-28-b2-4f-ea genunix: [ID 672855 kern.notice] syncing file systems... 2016-01-07T23:30:29.392822+00:00 00-21-28-b2-4f-ea genunix: [ID 904073 kern.notice] done > > > John Barfield / Sr Principal Engineer > > +1 (214) 425-0783<tel:+1%20(214)%20425-0783>/ > [email protected]<mailto:[email protected]><mailto:[email protected]> > > BISS, Inc. Office: +1 (214) 506-8354<tel:+1%20(214)%20506-8354> > > 4925 Greenville Ave Suite 900<x-apple-data-detectors://151/1> > > Dallas, TX 75206 > > support.bissinc.com<http://support.bissinc.com><http://htmlsig.com/support.bissinc.com> > ------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
