[zfs-discuss] Re: NFS Performance and Tar
I was really hoping for some option other than ZIL_DISABLE, but finally gave up the fight. Some people suggested NFSv4 helping over NFSv3 but it didn't... at least not enough to matter. ZIL_DISABLE was the solution, sadly. I'm running B43/X86 and hoping to get up to 48 or so soonish (I BFU'd it straight to B48 last night and brick'ed it). Here are the times. This is an untar (gtar xfj) of SIDEkick (http://www.cuddletech.com/blog/pivot/entry.php?id=491) on NFSv4 on a 20TB RAIDZ2 ZFS Pool: ZIL Enabled: real1m26.941s ZIL Disabled: real0m5.789s I'll update this post again when I finally get B48 or newer on the system and try it. Thanks to everyone for their suggestions. benr. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: NFS Performance and Tar
On Tue, eric kustarz wrote: Ben Rockwood wrote: I was really hoping for some option other than ZIL_DISABLE, but finally gave up the fight. Some people suggested NFSv4 helping over NFSv3 but it didn't... at least not enough to matter. ZIL_DISABLE was the solution, sadly. I'm running B43/X86 and hoping to get up to 48 or so soonish (I BFU'd it straight to B48 last night and brick'ed it). Here are the times. This is an untar (gtar xfj) of SIDEkick (http://www.cuddletech.com/blog/pivot/entry.php?id=491) on NFSv4 on a 20TB RAIDZ2 ZFS Pool: ZIL Enabled: real1m26.941s ZIL Disabled: real0m5.789s I'll update this post again when I finally get B48 or newer on the system and try it. Thanks to everyone for their suggestions. I imagine what's happening is that tar is a single-threaded application and it's basically doing: open, asynchronous write, close. This will go really fast locally. But for NFS due to the way it does cache consistency, on CLOSE, it must make sure that the writes are on stable storage, so it does a COMMIT, which basically turns your asynchronous write into a synchronous write. Which means you basically have a single-threaded app doing synchronous writes- ~ 1/2 disk rotational latency per write. Check out 'mount_nfs(1M)' and the 'nocto' option. It might be ok for you to relax the cache consistency for client's mount as you untar the file(s). Then remount w/out the 'nocto' option once you're done. This will not correct the problem because tar is extracting and therefore creating files and directories; those creates will be synchronous at the NFS server and there is no method to change this behavior at the client. Spencer Another option is to run multiple untars together. I'm guessing that you've got I/O to spare from ZFS's point of view. eric ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] panic string assistance
Could someone offer insight into this panic, please? panic string: ZFS: I/O failure (write on unknown off 0: zio 6000c5fbc0 0 [L0 ZIL intent log] 1000L/1000P DVA[0]=1:249b68000:1000 zilog uncompre ssed BE contiguous birth=318892 fill=0 cksum=3b8f19730caa4327:9e102 panic kernel thread: 0x2a1015d7cc0 PID: 0 on CPU: 530 cmd: sched t_procp: 0x187c780(proc_sched) p_as: 0x187e4d0(kas) zone: global t_stk: 0x2a1015d7ad0 sp: 0x18aa901 t_stkbase: 0x2a1015d2000 t_pri: 99(SYS) pctcpu: 0.00 t_lwp: 0x0psrset: 0 last CPU: 530 idle: 0 ticks (0 seconds) start: Wed Sep 20 18:17:22 2006 age: 1788 seconds (29 minutes 48 seconds) tstate: TS_ONPROC - thread is being run on a processor tflg: T_TALLOCSTK - thread structure allocated from stk T_PANIC - thread initiated a system panic tpflg: none set tsched: TS_LOAD - thread is in memory TS_DONT_SWAP - thread/LWP should not be swapped TS_SIGNALLED - thread was awakened by cv_signal() pflag: SSYS - system resident process pc: 0x105f7f8 unix:panicsys+0x48: call unix:setjmp startpc: 0x119fa64 genunix:taskq_thread+0x0: save%sp, -0xd0, %sp unix:panicsys+0x48(0x7b6e53a0, 0x2a1015d77c8, 0x18ab2d0, 0x1, , , 0x4480001601, , , , , , , , 0x7b6e53a0, 0x2a1015d77c8) unix:vpanic_common+0x78(0x7b6e53a0, 0x2a1015d77c8, 0x7b6e3bf8, 0x7080bc30, 0x708 0bc70, 0x7080b840) unix:panic+0x1c(0x7b6e53a0, 0x7080bbf0, 0x7080bbc0, 0x7b6e4428, 0x0, 0x6000c5fbc 00, , 0x5) zfs:zio_done+0x284(0x6000c5fbc00) zfs:zio_next_stage(0x6000c5fbc00) - frame recycled zfs:zio_vdev_io_assess+0x178(0x6000c5fbc00, 0x6000c586da0, 0x7b6c79f0) genunix:taskq_thread+0x1a4(0x6000bc5ea38, 0x0) unix:thread_start+0x4() ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] panic string assistance
Hi, No, I can't offer insight, but I do have some questions that are not really on topic. What version of solaris are you running? Is this the console output at time of panic? When did the panic code (or mdb) learn about frame recycling? Or are you using scat to get this output? thanks, max On Tue, 2006-10-03 at 14:10 -0700, Frank Leers wrote: Could someone offer insight into this panic, please? panic string: ZFS: I/O failure (write on unknown off 0: zio 6000c5fbc0 0 [L0 ZIL intent log] 1000L/1000P DVA[0]=1:249b68000:1000 zilog uncompre ssed BE contiguous birth=318892 fill=0 cksum=3b8f19730caa4327:9e102 panic kernel thread: 0x2a1015d7cc0 PID: 0 on CPU: 530 cmd: sched t_procp: 0x187c780(proc_sched) p_as: 0x187e4d0(kas) zone: global t_stk: 0x2a1015d7ad0 sp: 0x18aa901 t_stkbase: 0x2a1015d2000 t_pri: 99(SYS) pctcpu: 0.00 t_lwp: 0x0psrset: 0 last CPU: 530 idle: 0 ticks (0 seconds) start: Wed Sep 20 18:17:22 2006 age: 1788 seconds (29 minutes 48 seconds) tstate: TS_ONPROC - thread is being run on a processor tflg: T_TALLOCSTK - thread structure allocated from stk T_PANIC - thread initiated a system panic tpflg: none set tsched: TS_LOAD - thread is in memory TS_DONT_SWAP - thread/LWP should not be swapped TS_SIGNALLED - thread was awakened by cv_signal() pflag: SSYS - system resident process pc: 0x105f7f8 unix:panicsys+0x48: call unix:setjmp startpc: 0x119fa64 genunix:taskq_thread+0x0: save%sp, -0xd0, %sp unix:panicsys+0x48(0x7b6e53a0, 0x2a1015d77c8, 0x18ab2d0, 0x1, , , 0x4480001601, , , , , , , , 0x7b6e53a0, 0x2a1015d77c8) unix:vpanic_common+0x78(0x7b6e53a0, 0x2a1015d77c8, 0x7b6e3bf8, 0x7080bc30, 0x708 0bc70, 0x7080b840) unix:panic+0x1c(0x7b6e53a0, 0x7080bbf0, 0x7080bbc0, 0x7b6e4428, 0x0, 0x6000c5fbc 00, , 0x5) zfs:zio_done+0x284(0x6000c5fbc00) zfs:zio_next_stage(0x6000c5fbc00) - frame recycled zfs:zio_vdev_io_assess+0x178(0x6000c5fbc00, 0x6000c586da0, 0x7b6c79f0) genunix:taskq_thread+0x1a4(0x6000bc5ea38, 0x0) unix:thread_start+0x4() ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Max Bruning [EMAIL PROTECTED] ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] panic string assistance
ZFS will currently panic on a write failure to a non replicated pool. In the case below the Intent Log (though it could have been any module) could not write an intent log block. Here's a previous response from Eric Schrock explaining how ZFS intends to handle this: Yes, there are three incremental fixes that we plan in this area: 6417772 need nicer message on write failure This just cleans up the failure mode so that we get a nice FMA failure message and can distinguish this from a random failed assert. 6417779 ZFS: I/O failure (write on ...) -- need to reallocate writes In a multi-vdev pool, this would take a failed write and attempt to do the write on another toplevel vdev. This would all but elminate the problem for multi-vdev pools. 6322646 ZFS should gracefully handle all devices failing (when writing) This is the real fix. Unfortunately, it's also really hard. Even if we manage to abort the current transaction group, dealing with the semantics of a filesystem which has lost an arbitrary amount of change and notifying the user in a meaningful way is difficult at best. Hope that helps. - Eric Frank Leers wrote On 10/03/06 15:10,: Could someone offer insight into this panic, please? panic string: ZFS: I/O failure (write on unknown off 0: zio 6000c5fbc0 0 [L0 ZIL intent log] 1000L/1000P DVA[0]=1:249b68000:1000 zilog uncompre ssed BE contiguous birth=318892 fill=0 cksum=3b8f19730caa4327:9e102 panic kernel thread: 0x2a1015d7cc0 PID: 0 on CPU: 530 cmd: sched t_procp: 0x187c780(proc_sched) p_as: 0x187e4d0(kas) zone: global t_stk: 0x2a1015d7ad0 sp: 0x18aa901 t_stkbase: 0x2a1015d2000 t_pri: 99(SYS) pctcpu: 0.00 t_lwp: 0x0psrset: 0 last CPU: 530 idle: 0 ticks (0 seconds) start: Wed Sep 20 18:17:22 2006 age: 1788 seconds (29 minutes 48 seconds) tstate: TS_ONPROC - thread is being run on a processor tflg: T_TALLOCSTK - thread structure allocated from stk T_PANIC - thread initiated a system panic tpflg: none set tsched: TS_LOAD - thread is in memory TS_DONT_SWAP - thread/LWP should not be swapped TS_SIGNALLED - thread was awakened by cv_signal() pflag: SSYS - system resident process pc: 0x105f7f8 unix:panicsys+0x48: call unix:setjmp startpc: 0x119fa64 genunix:taskq_thread+0x0: save%sp, -0xd0, %sp unix:panicsys+0x48(0x7b6e53a0, 0x2a1015d77c8, 0x18ab2d0, 0x1, , , 0x4480001601, , , , , , , , 0x7b6e53a0, 0x2a1015d77c8) unix:vpanic_common+0x78(0x7b6e53a0, 0x2a1015d77c8, 0x7b6e3bf8, 0x7080bc30, 0x708 0bc70, 0x7080b840) unix:panic+0x1c(0x7b6e53a0, 0x7080bbf0, 0x7080bbc0, 0x7b6e4428, 0x0, 0x6000c5fbc 00, , 0x5) zfs:zio_done+0x284(0x6000c5fbc00) zfs:zio_next_stage(0x6000c5fbc00) - frame recycled zfs:zio_vdev_io_assess+0x178(0x6000c5fbc00, 0x6000c586da0, 0x7b6c79f0) genunix:taskq_thread+0x1a4(0x6000bc5ea38, 0x0) unix:thread_start+0x4() ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to make an extended LUN size known to ZFS and Solaris
Michael Phua - PTS wrote: Hi, Our customer has an Sun Fire X4100 with Solaris 10 using ZFS and a HW RAID array (STK D280). He has extended a LUN on the storage array and want to make this new size known to ZFS and Solaris. Does anyone know if this can be done and how it can be done. Unfortunately, there's no good way to do this at the moment. When you give ZFS the whole disk, we put a EFI label on the disk and make one big slice for our use. However, when the LUN grows, that slice stays the same size. ZFS needs to write a new EFI label describing the new size, before it can use the new space. I've filed bug 6475340 to track this issue. As a workaround, it *should* be possible to manually relabel the device with format(1m), but unfortunately bug 4967547 (a problem with format) prevents this from working correctly. --matt ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
