[zfs-discuss] Re: Re: New zfs pr0n server :)))
It's not easy to retry now because the 4 disks attached to the sil3114 controller are half of the raidz2 mirror... but in the previous home server that I had, this controller had 3 * 320gb disks attached to it where I had a raidz1 pool. With dclarke's test it easily got 50MB/s in the first (mad :) writing test. So I think it's pretty fast for the price and for the fact that's sata 1.0 and pci33 (those little no-brand cards are the cheapest you can find online or at local computer fairs) This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Making 'zfs destroy' safer
Chris Gerhard wrote: You are not alone. My preference would be for an optional -t option to zfs destroy: zfs destroy -t snapshot tank/[EMAIL PROTECTED] or zfs destroy -t snapshot -r tank/fs would delete all the snapshots below tank/fs I agree since that would fit nicely with the existing CLI model. This is much nicer than the proposal of separate destroy verbs for each object type. It is is important to use this style because otherwise we restrict new object types and needlessly complicate the cli over time. On the other hand personally I just don't see the need for this since the @ char isn't special to the shell so I don't see where the original problem came from. One thing that might help here, when not running as root or as a user with ZFS File System Management RBAC profile, is user delegation. This will allow you to run the script as user that can only do certain operations to filesystems that they own or have been delegated specific access to operate on. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: Re: Making 'zfs destroy' safer
On the other hand personally I just don't see the
need for this since
the @ char isn't special to the shell so I don't see
where the original
problem came from.
It is the combination of the fear of doing something bad and the the
consequence of doing that something bad that make people worry about this.
Especially when writing scripts.
I would much prefer to do
for snap in $(zfs list -t snapshot -r foo/bar )
do
zfs destroy -t snapshot $snap
do
the not have the -t. Especially the further away the destroy is from the
generation of the list. The extra -t would be belt and braces but that is how
I like my data protected.
Yes you can add that in using a shell function:
zfs_destroy_snapshot
{
zfs destroy [EMAIL PROTECTED]@[EMAIL PROTECTED]
}
but that is more hassle than a nice admin model should require from the nervous
user.
On the same basis that we don't need the -t option to destroy we don't really
need a separate snapshot sub command. It would be implied by:
zfs create tank/[EMAIL PROTECTED]
Yet this is not allowed and we have to have a special command to create a
snapshot but a generic one to destroy it.
--chris
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: New zfs pr0n server :)))
XIU, I'm currently using that card with my modest three-disk raid-z home server and it works great! Solaris 10 had native support for it so no need to mess with drivers. On 5/20/07, XIU [EMAIL PROTECTED] wrote: About sata controllers, anyone tried http://www.promise.com/product/product_detail_eng.asp?segment=Non-RAID%20HBAsproduct_id=139# ? It sells here for like 90euro so would be a cheap way for me to add some extra harddrives for a personal zfs server. On 5/20/07, Diego Righi [EMAIL PROTECTED] wrote: The onboard one is the controller that's in the nVidia 430 south bridge: pci bus 0x cardnum 0x0e function 0x00: vendor 0x10de device 0x0266 nVidia Corporation MCP51 Serial ATA Controller pci bus 0x cardnum 0x0f function 0x00: vendor 0x10de device 0x0267 nVidia Corporation MCP51 Serial ATA Controller The other one is a no-brand 4 port sil3114 pci sata 1.0 controller that I bought at a local computer fair last september: pci bus 0x0001 cardnum 0x07 function 0x00: vendor 0x1095 device 0x3114 Silicon Image, Inc. SiI 3114 [SATALink/SATARaid] Serial ATA Controller Notice that I had to flash it to make it work like a regular plain ide controller, otherwise it always kept the disks for himself presenting to me only the logical raid created drive... This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Christopher Gibbs Email / LDAP Administrator Web Integration Programming Abilene Christian University ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: New zfs pr0n server :)))
Christopher Gibbs wrote: XIU, I'm currently using that card with my modest three-disk raid-z home server and it works great! Solaris 10 had native support for it so no need to mess with drivers. By native support, I assume you mean IDE/ATA driver support (ata in modinfo), not SATA driver support. But I'd love to be wrong ;-) On 5/20/07, XIU [EMAIL PROTECTED] wrote: About sata controllers, anyone tried http://www.promise.com/product/product_detail_eng.asp?segment=Non-RAID%20HBAsproduct_id=139# ? It sells here for like 90euro so would be a cheap way for me to add some extra harddrives for a personal zfs server. -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] While we're sharing server info...
My server used to use ODS/SDS/SVM/WhateverSunCallsItToday RAID 5. When my old motherboard decided to flake out on me, SVM refused to recognize the old RAID5 set. Fortunately, I resurrected my old parts long enough to copy off almost all my data on to a pair of 750GB disks. I'm now running on ZFS, and am much happier. The motherboard is Intel ICH7 based (so I eagerly await AHCI driver support - anyone know if it made U4?). My boot disks are SVM mirrored SATA disks off the motherboard. My data array is 8 Hitachi HDS72505 SATA disks off of 2 4-port SI3114 based PCI cards (I also eagerly await a supported PCI express internal SATA JBOD controller). The case is a Lian-Li PC-V100B, which has 12 internal 3.5 drive bays. I'm running Solaris x86 10 U3. carson:gandalf 0 $ zpool status pool: media state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM media ONLINE 0 0 0 raidz1ONLINE 0 0 0 c2d0ONLINE 0 0 0 c2d1ONLINE 0 0 0 c3d0ONLINE 0 0 0 c3d1ONLINE 0 0 0 c4d0ONLINE 0 0 0 c4d1ONLINE 0 0 0 c5d0ONLINE 0 0 0 c5d1ONLINE 0 0 0 errors: No known data errors carson:gandalf 0 $ zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT media 3.62T 1.70T 1.92T46% ONLINE - A basic sequential write test (compression on this file system is off): carson:gandalf 0 $ time dd if=/dev/zero of=/export/media/test bs=8192k count=1024 1024+0 records in 1024+0 records out real2m30.875s user0m0.010s sys 0m10.853s During the write: carson:gandalf 0 $ zpool iostat 5 capacity operationsbandwidth pool used avail read write read write -- - - - - - - media 1.71T 1.92T 0570 21.6K 63.8M media 1.71T 1.92T 0592 51.1K 63.7M media 1.71T 1.92T 0573613 64.9M media 1.71T 1.92T 0574 0 64.3M media 1.71T 1.92T 0573204 64.7M media 1.71T 1.92T 0563 0 63.6M media 1.71T 1.92T 0594613 67.5M media 1.71T 1.92T 0547 0 65.1M media 1.71T 1.92T 0558 0 59.2M carson:gandalf 130 $ iostat -l 11 -x 5 extended device statistics device r/sw/s kr/s kw/s wait actv svc_t %w %b cmdk01.80.0 11.20.0 0.0 0.04.8 0 1 cmdk10.4 200.67.3 9226.7 22.1 1.6 117.9 74 82 cmdk20.6 270.6 10.2 9339.4 26.4 1.8 104.0 85 94 cmdk30.6 198.4 10.3 9220.9 22.0 1.6 118.5 74 82 cmdk40.6 239.0 10.2 9287.0 26.1 1.8 116.4 84 92 cmdk50.8 200.6 23.1 9228.7 22.2 1.6 118.2 75 83 cmdk60.4 244.26.6 9271.8 26.4 1.8 115.1 84 92 cmdk70.6 197.8 10.3 9256.1 21.7 1.6 117.4 73 81 cmdk80.4 267.26.6 9288.6 26.1 1.8 104.3 84 92 cmdk90.00.00.00.0 0.0 0.00.0 0 0 cmdk12 1.60.0 11.00.0 0.0 0.07.3 0 1 extended device statistics device r/sw/s kr/s kw/s wait actv svc_t %w %b cmdk02.20.6 12.00.3 0.0 0.03.3 0 1 cmdk10.0 176.40.0 9350.9 22.0 1.5 133.7 74 80 cmdk20.0 246.20.0 9386.9 26.4 1.8 114.6 85 92 cmdk30.0 177.80.0 9346.9 22.2 1.5 133.4 74 80 cmdk40.0 223.00.0 9374.8 26.3 1.7 125.9 84 90 cmdk50.0 177.00.0 9317.6 22.3 1.6 134.5 75 80 cmdk60.0 228.80.0 9414.3 26.5 1.8 123.4 85 90 cmdk70.0 176.60.0 9322.8 22.1 1.5 133.7 74 80 cmdk80.0 247.20.0 9399.7 26.4 1.7 113.8 84 90 cmdk90.00.00.00.0 0.0 0.00.0 0 0 cmdk12 2.40.2 15.20.1 0.0 0.04.6 0 1 extended device statistics device r/sw/s kr/s kw/s wait actv svc_t %w %b cmdk00.01.80.01.0 0.0 0.00.3 0 0 cmdk10.0 183.40.0 9425.3 22.8 1.6 132.8 75 80 cmdk20.0 249.00.0 9477.2 27.2 1.8 116.4 86 92 cmdk30.0 183.80.0 9421.0 22.9 1.6 133.3 75 81 cmdk40.0 228.60.0 9473.5 27.2 1.8 126.8 85 91 cmdk50.0 187.40.0 9449.0 22.9 1.6 130.5 75 81 cmdk60.0 233.20.0 9471.1 27.3 1.8 124.6 86 91 cmdk70.0 183.60.0 9425.9 22.7 1.6 132.1 75 80 cmdk80.0 248.80.0 9480.9 27.0 1.8 115.7 85 91 cmdk90.00.00.00.0 0.0 0.00.0 0 0 cmdk12 0.21.00.80.6 0.0 0.00.3 0 0 carson:gandalf 0 $ vmstat 5 kthr
Re: [zfs-discuss] Re: New zfs pr0n server :)))
Sorry, I'm fairly new to Solaris... I'm not sure if it's using the ata driver or sata driver. Here are my current disks (0,1, and 2 are the sata disks): AVAILABLE DISK SELECTIONS: 0. c0d0 Maxtor 6-L59LQ0W-0001-233.76GB /[EMAIL PROTECTED],0/[EMAIL PROTECTED]/[EMAIL PROTECTED]/[EMAIL PROTECTED],0 1. c0d1 Maxtor 6-L59MD11-0001-233.76GB /[EMAIL PROTECTED],0/[EMAIL PROTECTED]/[EMAIL PROTECTED]/[EMAIL PROTECTED],0 2. c1d0 Maxtor 6-L59M4DQ-0001-233.76GB /[EMAIL PROTECTED],0/[EMAIL PROTECTED]/[EMAIL PROTECTED]/[EMAIL PROTECTED],0 3. c2d0 DEFAULT cyl 2488 alt 2 hd 255 sec 63 /[EMAIL PROTECTED],0/[EMAIL PROTECTED],1/[EMAIL PROTECTED]/[EMAIL PROTECTED],0 Specify disk (enter its number): Since they are labeled as pci-ide is it safe to assume they are using the ide/ata driver? If so, is there a performance gain when using a sata driver? On 5/21/07, Carson Gaspar [EMAIL PROTECTED] wrote: Christopher Gibbs wrote: XIU, I'm currently using that card with my modest three-disk raid-z home server and it works great! Solaris 10 had native support for it so no need to mess with drivers. By native support, I assume you mean IDE/ATA driver support (ata in modinfo), not SATA driver support. But I'd love to be wrong ;-) On 5/20/07, XIU [EMAIL PROTECTED] wrote: About sata controllers, anyone tried http://www.promise.com/product/product_detail_eng.asp?segment=Non-RAID%20HBAsproduct_id=139# ? It sells here for like 90euro so would be a cheap way for me to add some extra harddrives for a personal zfs server. -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Christopher Gibbs Email / LDAP Administrator Web Integration Programming Abilene Christian University ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Making 'zfs destroy' safer
On the other hand personally I just don't see the need for this since the @ char isn't special to the shell so I don't see where the original problem came from. I never actually *had* a problem, I am just nervous about it. And yes, @ is not special for classical shells, but it's still more special than alphanumerics or '/', and probably more likely to *be* special in some languages/alternate shells. Then there is the seemingly trivial issue of the physical keyboard layout. The most common layout will tend to make you use the right shift key in order to type the @, in a particular sequence such that a slight slip of the finger there and *kaboom*, you have lost your (and/or everybody else's) data by accidentally hitting enter instead of shift. One can of course protect against this by writing commands backwards and such, which is what I do for cases like this along with SQL DELETE statements, but to me it just feels unnecessarily dangerous. One thing that might help here, when not running as root or as a user with ZFS File System Management RBAC profile, is user delegation. This will allow you to run the script as user that can only do certain operations to filesystems that they own or have been delegated specific access to operate on. On the other than a very minor modification to the command line tool gets you a pretty significant payoff without complicating things. It will affect the safety of the out-of-the-box tool, regardless of the local policy for privilege delegation. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller [EMAIL PROTECTED]' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org signature.asc Description: OpenPGP digital signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: Making 'zfs destroy' safer
I would much prefer to do for snap in $(zfs list -t snapshot -r foo/bar ) do zfs destroy -t snapshot $snap do the not have the -t. Especially the further away the destroy is from the generation of the list. The extra -t would be belt and braces but that is how I like my data protected. Especially if we imagine that someone further down the line decides to slightly modify the format of the zfs list -t snapshot output - such as by having it give a hierarchal view with the roots being the filesystem path... This is of course a normal problem with shell scripting (unless the zfs command is documented to guarantee backward compatible output?), but in cases like this it really really becomes critical. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller [EMAIL PROTECTED]' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org signature.asc Description: OpenPGP digital signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Making 'zfs destroy' safer
Actually, if your zfs filesystem has snapshots zfs will complain that the fs can't be destroyed (or that you have to use the -f switch to force it). So the first thing I do when making a new filesystem is create a snapshot to protect me from destroying a filesystem :) On 5/21/07, Peter Schuller [EMAIL PROTECTED] wrote: On the other hand personally I just don't see the need for this since the @ char isn't special to the shell so I don't see where the original problem came from. I never actually *had* a problem, I am just nervous about it. And yes, @ is not special for classical shells, but it's still more special than alphanumerics or '/', and probably more likely to *be* special in some languages/alternate shells. Then there is the seemingly trivial issue of the physical keyboard layout. The most common layout will tend to make you use the right shift key in order to type the @, in a particular sequence such that a slight slip of the finger there and *kaboom*, you have lost your (and/or everybody else's) data by accidentally hitting enter instead of shift. One can of course protect against this by writing commands backwards and such, which is what I do for cases like this along with SQL DELETE statements, but to me it just feels unnecessarily dangerous. One thing that might help here, when not running as root or as a user with ZFS File System Management RBAC profile, is user delegation. This will allow you to run the script as user that can only do certain operations to filesystems that they own or have been delegated specific access to operate on. On the other than a very minor modification to the command line tool gets you a pretty significant payoff without complicating things. It will affect the safety of the out-of-the-box tool, regardless of the local policy for privilege delegation. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller [EMAIL PROTECTED]' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Rsync update to ZFS server over SSH faster than over NFS?
We're testing an X4100M2, 4GB RAM, with a 2-port 4GB Fibre Channel QLogic connected to a 2GB Fibre Channel 6140 array. The X4100M2 is running OpenSolaris b63. We have 8 drives in the Sun 6140 configured as individual RAID-0 arrays and have a ZFS RAID-Z2 array comprising 7 of the drives (for testing, we're treating the 6140 as JBOD for now). The RAID-0 stripe size is 128k. We're testing updates to the X4100M2 using rsync across the network with ssh and using NFS: 1. [copy 400MB of gcc-3.4.3 via rsync/NFS] # mount file-server:/opt/test /mnt # rsync -vaHR --delete --stats gcc343 /mnt ... sent 409516941 bytes received 80590 bytes 5025736.58 bytes/sec 2. [copy 400MB of gcc-3.4.3 via rsync/ssh] # rsync -vaHR -e 'ssh' --delete --stats gcc343 file-server:/opt/test ... sent 409516945 bytes received 80590 bytes 9637589.06 bytes/sec The network is 100MB. /etc/system on the file server is: set maxphys = 0x80 set ssd:ssd_max_throttle = 64 set zfs:zfs_nocacheflush = 1 Why can't the NFS performance match that of SSH? -- albert chin ([EMAIL PROTECTED]) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Rsync update to ZFS server over SSH faster than over NFS?
[EMAIL PROTECTED] said: Why can't the NFS performance match that of SSH? Hi Albert, My first guess is the NFS vs array cache-flush issue. Have you configured the 6140 to ignore SYNCHRONIZE_CACHE requests? That'll make a huge difference for NFS clients of ZFS file servers. Also, you might make your ssh connection go faster if you change the rsync arg from -e 'ssh' to -e 'ssh -c blowfish'. Depends, of course, on how fast both client and server CPU's are. Regards, Marion ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: ZFS - Use h/w raid or not? Thoughts. Considerations.
Thanks for the input. So, I'm trying to meld the two replies and come up with a direction for my case and maybe a rule of thumb that I can use in the future (i.e., near future until new features come out in zfs) when I have external storage arrays that have built in RAID. At the moment, I'm hearing that using h/w raid under my zfs may be better for some workloads and the h/w hot spare would be nice to have across multiple raid groups, but the checksum capabilities in zfs are basically nullified with single/multiple h/w lun's resulting in reduced protection. Therefore, it sounds like I should be strongly leaning towards not using the hardware raid in external disk arrays and use them like a JBOD. When will Sun have global hot spare capability? This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Rsync update to ZFS server over SSH faster than over NFS?
Albert Chin wrote: Why can't the NFS performance match that of SSH? One big reason is that the sending CPU has to do all the comparisons to compute the list of files to be sent - it has to fetch the attributes from both local and remote and compare timestamps. With ssh, local processes at each end do lstat() calls in parallel and chatter about the timestamps, and the lstat() calls are much cheaper. I would wonder how long the attr-chatter takes in your two cases before bulk data starts to be sent - deducting that should reduce the imbalance you're seeing. If rsync were more multi-threaded and could manage multiple lstat() calls in parallel NFS would be closer. Rob T ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Rsync update to ZFS server over SSH faster than over NFS?
On Mon, May 21, 2007 at 02:55:18PM -0600, Robert Thurlow wrote: Albert Chin wrote: Why can't the NFS performance match that of SSH? One big reason is that the sending CPU has to do all the comparisons to compute the list of files to be sent - it has to fetch the attributes from both local and remote and compare timestamps. With ssh, local processes at each end do lstat() calls in parallel and chatter about the timestamps, and the lstat() calls are much cheaper. I would wonder how long the attr-chatter takes in your two cases before bulk data starts to be sent - deducting that should reduce the imbalance you're seeing. If rsync were more multi-threaded and could manage multiple lstat() calls in parallel NFS would be closer. Well, there is no data on the file server as this is an initial copy, so there is very little for rsync to do. To compare the rsync overhead, I conducted some more tests, using tar: 1. [copy 400MB of gcc-3.4.3 via tar/NFS to ZFS file system] # mount file-server:/opt/test /mnt # time tar cf - gcc343 | (cd /mnt; tar xpf - ) ... 419721216 bytes in 1:08.65 = 6113928.86 bytes/sec 2. [copy 400MB of gcc-3.4.3 via tar/ssh to ZFS file system] # time tar cf - gcc343 | ssh -oForwardX11=no file-server \ 'cd /opt/test; tar xpf -' ... 419721216 bytes in 35:82 = 11717510.21 bytes/sec 3. [copy 400MB of gcc-3.4.3 via tar/NFS to Fibre-attached file system] # mount file-server:/opt/fibre-disk /mnt # time tar cf - gcc343 | (cd /mnt; tar xpf - ) ... 419721216 bytes in 56:87 = 7380362.51 bytes/sec 4. [copy 400MB of gcc-3.4.3 via tar/ssh to Fibre-attached file system] # time tar cf - gcc343 | ssh -oForwardX11=no file-server \ 'cd /opt/fibre-disk; tar xpf -' ... 419721216 bytes in 35:89 = 11694656.34 bytes/sec So, it would seem using #1 and #2, NFS performance can stand some improvement. And, I'd have thought that since #2/#4 were similar, #1/#3 should be as well. Maybe some NFS/ZFS issues would answer the discrepancy. I think the bigger problem is the NFS performance penalty so we'll go lurk somewhere else to find out what the problem is. -- albert chin ([EMAIL PROTECTED]) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Rsync update to ZFS server over SSH faster than over NFS?
Albert Chin wrote: Well, there is no data on the file server as this is an initial copy, Sorry Albert, I should have noticed that from your e-mail :-( I think the bigger problem is the NFS performance penalty so we'll go lurk somewhere else to find out what the problem is. Is this with Solaris 10 or OpenSolaris on the client as well? I guess this goes back to some of the why is tar slow over NFS discussions we've had, some here and some on nfs-discuss. A more multi-threaded workload would help; so will planned work to focus on performance of NFS and ZFS together, which can sometimes be slower than expected. Rob T ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: Re: New zfs pr0n server :)))
I wanted to confirm the drivers I was using for the hard drives in my PC, and here is the method I used. Maybe you can try something similar, and see what you get. I used the 'prtconf' command, with the device path from the 'format' command. (Use bash as the shell, and use the tab key to expand the path) My sata drive is using the 'ahci' driver, connecting to the ICH7 chipset on the motherboard. And I have a scsi drive on a Adaptec card, plugged into a PCI slot. Thanks Nigel Smith # format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t0d0 DEFAULT cyl 2229 alt 2 hd 255 sec 63 /[EMAIL PROTECTED],0/pci8086,[EMAIL PROTECTED]/pci9005,[EMAIL PROTECTED]/[EMAIL PROTECTED],0 1. c2t0d0 DEFAULT cyl 9723 alt 2 hd 255 sec 63 /[EMAIL PROTECTED],0/pci1028,[EMAIL PROTECTED],2/[EMAIL PROTECTED],0 Specify disk (enter its number): ^D # prtconf -acD /devices/[EMAIL PROTECTED],0/pci1028\,[EMAIL PROTECTED],2 i86pc (driver name: rootnex) pci, instance #0 (driver name: npe) pci1028,1a8, instance #0 (driver name: ahci) disk, instance #0 (driver name: sd) # prtconf -acD /devices/[EMAIL PROTECTED],0/pci8086\,[EMAIL PROTECTED]/pci9005\,[EMAIL PROTECTED]/[EMAIL PROTECTED],0 i86pc (driver name: rootnex) pci, instance #0 (driver name: npe) pci8086,244e, instance #0 (driver name: pci_pci) pci9005,62a0, instance #0 (driver name: cadp160) sd, instance #2 (driver name: sd) # modinfo | egrep -i 'root|npe|ahci|sd |pci_pci|cadp160' 20 fbb309c8 4768 1 1 rootnex (i86pc root nexus 1.141) 30 fbb970e0 6d20 183 1 npe (Host to PCIe nexus driver 1.7) 34 fbba4aa0 77a8 58 1 ahci (ahci driver 1.1) 36 fbbb9f70 25fb8 31 1 sd (SCSI Disk Driver 1.548) 146 f7c73000 1a58 84 1 pci_pci (PCI to PCI bridge nexus driver ) 147 f7d2 275a0 32 1 cadp160 (Adaptec Ultra160 HBA d1.21) This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Rsync update to ZFS server over SSH faster than over NFS?
On Mon, May 21, 2007 at 06:09:46PM -0500, Albert Chin wrote: But still, how is tar/SSH any more multi-threaded than tar/NFS? It's not that it is, but that NFS sync semantics and ZFS sync semantics conspire against single-threaded performance. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Rsync update to ZFS server over SSH faster than over NFS?
On Mon, May 21, 2007 at 06:11:36PM -0500, Nicolas Williams wrote: On Mon, May 21, 2007 at 06:09:46PM -0500, Albert Chin wrote: But still, how is tar/SSH any more multi-threaded than tar/NFS? It's not that it is, but that NFS sync semantics and ZFS sync semantics conspire against single-threaded performance. What's why we have set zfs:zfs_nocacheflush = 1 in /etc/system. But, that's only helps ZFS. Is there something similar for NFS? -- albert chin ([EMAIL PROTECTED]) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Rsync update to ZFS server over SSH faster than over NFS?
On Mon, May 21, 2007 at 04:55:35PM -0600, Robert Thurlow wrote: Albert Chin wrote: I think the bigger problem is the NFS performance penalty so we'll go lurk somewhere else to find out what the problem is. Is this with Solaris 10 or OpenSolaris on the client as well? Client is RHEL 4/x86_64. But, we just ran a concurrent tar/SSH across Solaris 10, HP-UX 11.23/PA, 11.23/IA, AIX 5.2, 5.3, RHEL 4/x86, 4/x86_64 and the average was ~4562187 bytes/sec. But, the gcc343 copy on each of these machines isn't the same size. It's certainly less than 400MBx7 though. While performance on one system is fine, things degrade when you add clients. I guess this goes back to some of the why is tar slow over NFS discussions we've had, some here and some on nfs-discuss. A more multi-threaded workload would help; so will planned work to focus on performance of NFS and ZFS together, which can sometimes be slower than expected. But still, how is tar/SSH any more multi-threaded than tar/NFS? I've posted to nfs-discuss so maybe someone knows something. -- albert chin ([EMAIL PROTECTED]) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Rsync update to ZFS server over SSH faster than over NFS?
On Mon, May 21, 2007 at 06:21:40PM -0500, Albert Chin wrote: On Mon, May 21, 2007 at 06:11:36PM -0500, Nicolas Williams wrote: On Mon, May 21, 2007 at 06:09:46PM -0500, Albert Chin wrote: But still, how is tar/SSH any more multi-threaded than tar/NFS? It's not that it is, but that NFS sync semantics and ZFS sync semantics conspire against single-threaded performance. What's why we have set zfs:zfs_nocacheflush = 1 in /etc/system. But, that's only helps ZFS. Is there something similar for NFS? NFS's semantics for open() and friends is that they are synchronous, whereas POSIX's semantics are that they are not. You're paying for a sync() after every open. Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Rsync update to ZFS server over SSH faster than over NFS?
On May 21, 2007 6:30:42 PM -0500 Nicolas Williams [EMAIL PROTECTED] wrote: On Mon, May 21, 2007 at 06:21:40PM -0500, Albert Chin wrote: On Mon, May 21, 2007 at 06:11:36PM -0500, Nicolas Williams wrote: On Mon, May 21, 2007 at 06:09:46PM -0500, Albert Chin wrote: But still, how is tar/SSH any more multi-threaded than tar/NFS? It's not that it is, but that NFS sync semantics and ZFS sync semantics conspire against single-threaded performance. What's why we have set zfs:zfs_nocacheflush = 1 in /etc/system. But, that's only helps ZFS. Is there something similar for NFS? NFS's semantics for open() and friends is that they are synchronous, whereas POSIX's semantics are that they are not. You're paying for a sync() after every open. nocto? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: ZFS over a layered driver interface
hey swetha,
i don't think there is any easy answer for you here.
i'd recommend watching all device operations (open, read, write, ioctl,
strategy, prop_op, etc) that happen to the ramdisk device when you don't
use your layered driver, and then again when you do. then you could
compare the two to see what the differences are. or optionally, you
could do some more analysis into that ioctl failure to see if it's
an important ioctl that should be succeeding and instead is causing
a cascading failure. to investigate both of these possiblilties i'd
recommend using dtrace fbt probes, the answer won't be easy to find,
but dtrace will make finding it easier.
lastly, you could start analyzing to see what zio_wait() is waiting for.
for this you'd want to use mdb -k to look at the current kernel state
and compare that to the source to see what zfs is trying to do and what
it's blocked on. (you might actually consider forcing a crash dump
and analyzing it offline, since then the state is not changing and you
can always come back to it.)
ed
On Tue, May 15, 2007 at 01:12:06PM -0700, Shweta Krishnan wrote:
With what Edward suggested, I got rid of the ldi_get_size() error by defining
the prop_op entry point appropriately.
However, the zpool create still fails - with zio_wait() returning 22.
bash-3.00# dtrace -n 'fbt::ldi_get_size:entry{self-t=1;}
fbt::ldi_get_size:entry/self-t/{}
fbt::ldi_get_size:return/self-t/{trace((int)arg1);}
fbt::ldi_get_size:return{self-t=0;}' -c 'zpool create adsl-pool
/dev/layerzfsminor1'
dtrace: description 'fbt::ldi_get_size:entry' matched 4 probes
cannot create 'adsl-pool': invalid argument for this pool operation
dtrace: pid 2487 has exited
CPU ID FUNCTION:NAME
0 21606 ldi_get_size:entry
0 21607 ldi_get_size:return 0
bash-3.00# dtrace -n 'fbt:zfs:zfs_ioc_pool_create:entry{self-t=1;}
fbt:zfs::return/self-t arg1 == 22/{stack(); exit(0);}
fbt:zfs:zfs_ioc_pool_create:return{self-t=0;}'
dtrace: description 'fbt:zfs:zfs_ioc_pool_create:entry' matched 1317 probes
CPU ID FUNCTION:NAME
0 63848 zio_wait:return
zfs`vdev_label_init+0x4ed
zfs`vdev_label_init+0x4e
zfs`vdev_create+0x4b
zfs`spa_create+0x233
zfs`zfs_ioc_pool_create+0x4a
zfs`zfsdev_ioctl+0x119
genunix`cdev_ioctl+0x48
specfs`spec_ioctl+0x86
genunix`fop_ioctl+0x37
genunix`ioctl+0x16b
unix`sys_syscall32+0x101
I see the strategy routine of my layered driver being invoked, and reads and
writes are being done. (in the more detailed dtrace dump, I see
zio_vdev_io_start and other zio functions being invoked).
Is there a way to figure out where exactly this is breaking?
Could it be due to an ioctl failure, since the kernel log shows a failure for
the ioctl to the real device?
Thanks,
Swetha.
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: ZFS - Use h/w raid or not? Thoughts. Considerations.
There isn't a global hot spare, but you can add a hot spare to multiple pools. Paul This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: ZFS - Use h/w raid or not? Thoughts. Considerations.
Personally I would go with ZFS entirely in most cases. That's the rule of thumb :) If you have a fast enough CPU and enough RAM, do everything with ZFS. This sounds koolaid-induced, but you'll need nothing else because ZFS does it all. My second personal rule of thumb concerns RAIDZ performance. Benchmarks that were posted here in the past showed that RAIDZ worked best with no more than 4 or 5 disks per array. After that, certain types of performance dropped off pretty hard. So if top performance matters and you can handle doing 4-5 disk arrays, that is a smart path to take. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Rsync update to ZFS server over SSH faster than over
On Mon, May 21, 2007 at 08:26:37PM -0700, Paul Armstrong wrote: GIven you're not using compression for rsync, the only thing I can think if would be that the stream compression of SSH is helping here. SSH compresses by default? I thought you had to specify -oCompression and/or -oCompressionLevel? -- albert chin ([EMAIL PROTECTED]) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: ZFS - Use h/w raid or not? Thoughts. Considerations.
More redundancy below...
Torrey McMahon wrote:
Phillip Fiedler wrote:
Thanks for the input. So, I'm trying to meld the two replies and come
up with a direction for my case and maybe a rule of thumb that I can
use in the future (i.e., near future until new features come out in
zfs) when I have external storage arrays that have built in RAID.
At the moment, I'm hearing that using h/w raid under my zfs may be
better for some workloads and the h/w hot spare would be nice to have
across multiple raid groups, but the checksum capabilities in zfs are
basically nullified with single/multiple h/w lun's resulting in
reduced protection. Therefore, it sounds like I should be strongly
leaning towards not using the hardware raid in external disk arrays
and use them like a JBOD.
The bit ...
the checksum capabilities in zfs are basically nullified with
single/multiple h/w lun's resulting in reduced protection.
is not accurate. With one large LUN, then yes, you can only detect
errors. With multiple LUNs in a mirror or RAIDZ{2} then you can correct
errors.
You can also add redundancy with ZFS filesystem copies parameter. This is
similar to, but not the same as mirroring.
The big reasons for continuing to use hw raid is speed, in some cases,
and heterogeneous environments where you can't farm out non-raid
protected LUNs and raid protected LUNs from the same storage array. In
some cases the array will require a raid protection setting, like the
99x0, before you can even start farming out storage.
Yes. ZFS data protection builds on top of this. You always gain a benefit
when the data protection is done as close to the application as possible -- as
opposed to implementing the data protection as close to the storage as possible.
-- richard
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
