Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
You have not described your requirements (low-power ??, low-cost ??). But I'll contribute some pointers anyway! :) Well for a home NAS I'm looking at noise as a big factor. Also for a 24x7 box, power consumption, that's why the northbridge is putting me off slightly. So far the other solutions mentioned here, and what I've found on various blogs are going to be a lot more noisy and power hungry. Although the cost also makes this option very attractive, I'd be happy to pay a bit more for something with more grunt so long as it's quiet and energy efficient. Although I don't really mind if the box is slow, with ZFS what I'm looking for is secure data storage. But with only two SATA ports I don't see how this board will quite acheive that. Unless I boot from USB and use 2 SATA and 2 IDE drives. btw I'm seeing mixed reports if the D945GCLF2 is seen as 64 bit by solaris. Anyway thanks all for the great feedback! -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
On Sun, Oct 26, 2008 at 12:47 AM, Peter Bridge [EMAIL PROTECTED] wrote: Well for a home NAS I'm looking at noise as a big factor. Also for a 24x7 box, power consumption, that's why the northbridge is putting me off slightly. That's why I built a full-sized tower using a Lian-Li case with noise dampening on the sides, robber grommets and such in the drive bays, etc.. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
Same case and same idea :) I have 2 dampered drives already installed from a previous project. Another 2 I pulled out to install into a qnap 209. Ideally I'd return the disks and replace the qnap with this new single ZFS NAS, although I'm quite fond of the qnap bt client, and can't use all four SATA with this board anyway... -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Verify files' checksums
1) When I notice an error in a file that I've copied from a ZFS disk I want to know whether that error is also in the original file on my ZFS disk or if it's only in the copy. This was already addressed but let me do so slightly differently: One of the major points of ZFS checksumming is that, in the abscence of software bugs or hardware memory corruption issues when the file is read on the host, successfully reading a file is supposed to mean that you got the correct version of the file (either from physical disk or from cache, having previously been read from physical disk). A scrub is still required if you want to make sure the file is okay *ON DISK*, unless you can satisfy yourself that no relevant data is cached somewhere (or unless someone can inform me of a way to nuke a particular file and related resources from cache). Up to now I've been storing md5sums for all files, but keeping the files and their md5sums synchronized is a burden I could do without. FWIW I wanted to mention here that if you care a lot about this, I'd recommend something like par2[1] instead. It uses forward error correction[2], allowing you to not only detect corruption, but also correct it. You can choose your desired level of redundancy expressed as a percentage of the file size. [1] http://en.wikipedia.org/wiki/Parchive [2] http://en.wikipedia.org/wiki/Forward_error_correction -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller [EMAIL PROTECTED]' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org pgpTBfuHa8mpF.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Verify files' checksums
A slight nit. Using cat(1) to read the file to /dev/null will not actually cause the data to be read thanks to the magic that is mmap(). If you use dd(1) to read the file then yes you will either get the data and thus know it's blocks match their checksums or dd will give you an error if you have no redundancy. --chris -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] recommendations on adding vdev to raidz zpool
On Sun, Oct 26, 2008 at 5:31 AM, Peter Baumgartner [EMAIL PROTECTED] wrote: I have a 7x150GB drive (+1 spare) raidz pool that I need to expand. There are 6 open drive bays, so I bought 6 300GB drives and went to add them as a raidz vdev to the existing zpool, but I didn't realize the raidz vdevs needed to have the same number of drives. (why is that?) They do not have to have the same number of drivers, you can even mix raidz and plain disks. That is more a recommendation. Add -f to the command. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Boot from mirror
Is it or isn't it possible to boot off two mirrored ZFS disks and if yes, can this be done in the upcoming solaris 10 10/08 too? -- Dick Hoogendijk -- PGP/GnuPG key: 01D2433D ++ http://nagual.nl/ + SunOS sxce snv99 ++ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Verify files' checksums
A nit on the nit... cat does not use mmap for files = 32K in size. For those files it's a simple read() into a buffer and write() it out. Jim --- Chris Gerhard wrote: A slight nit. Using cat(1) to read the file to /dev/null will not actually cause the data to be read thanks to the magic that is mmap(). If you use dd(1) to read the file then yes you will either get the data and thus know it's blocks match their checksums or dd will give you an error if you have no redundancy. --chris -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] recommendations on adding vdev to raidz zpool
On Sun, Oct 26, 2008 at 4:02 AM, Mattias Pantzare [EMAIL PROTECTED] wrote: On Sun, Oct 26, 2008 at 5:31 AM, Peter Baumgartner [EMAIL PROTECTED] wrote: I have a 7x150GB drive (+1 spare) raidz pool that I need to expand. There are 6 open drive bays, so I bought 6 300GB drives and went to add them as a raidz vdev to the existing zpool, but I didn't realize the raidz vdevs needed to have the same number of drives. (why is that?) They do not have to have the same number of drivers, you can even mix raidz and plain disks. That is more a recommendation. Add -f to the command. What is the risk of creating a pool consisting of two raidz vdevs that don't have the same number of disks? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
On Sat, Oct 25, 2008 at 06:50:46PM -0700, Nigel Smith wrote:
Hi Matt
What chipset is your PCI network card?
(obviously, it not Intel, but what is it?)
Do you know which driver the card is using?
I believe it's some sort of Realtek (8139 probably). It's coming up as rtls0
You say '..The system was fine for a couple of weeks..'.
At that point did you change any software - do any updates or upgrades?
For instance, did you upgrade to a new build of OpenSolaris?
No, since the original problem with the onboard NICs it hasn't been upgraded
or anything.
If not, then I would guess it's some sort of hardware problem.
Can you try different cables and a different switch - anything
in the path between client server is suspect.
Have tried different cables and switch ports, I will try a different switch
as soon as I can get some space on one of the others.
A mismatch of Ethernet duplex settings can cause problems - are
you sure this is Ok.
Not 100% sure, but I will check as best I can.
To get an idea of how the network is running try this:
On the Solaris box, do an Ethernet capture with 'snoop' to a file.
http://docs.sun.com/app/docs/doc/819-2240/snoop-1m?a=view
# snoop -d {device} -o {filename}
.. then while capturing, try to play your video file through the network.
Control-C to stop the capture.
You can then use Ethereal or WireShark to analyze the capture file.
On the 'Analyze' menu, select 'Expert Info'.
This will look through all the packets and will report
any warning or errors it sees.
It's coming up with a huge number of TCP Bad Checksum errors, a few
Previous Segment Lost and a few Fast retransmission.
Thanks
Matt
pgpodoJIT4jdS.pgp
Description: PGP signature
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] recommendations on adding vdev to raidz zpool
On Sun, Oct 26, 2008 at 3:00 PM, Peter Baumgartner [EMAIL PROTECTED] wrote: On Sun, Oct 26, 2008 at 4:02 AM, Mattias Pantzare [EMAIL PROTECTED] wrote: On Sun, Oct 26, 2008 at 5:31 AM, Peter Baumgartner [EMAIL PROTECTED] wrote: I have a 7x150GB drive (+1 spare) raidz pool that I need to expand. There are 6 open drive bays, so I bought 6 300GB drives and went to add them as a raidz vdev to the existing zpool, but I didn't realize the raidz vdevs needed to have the same number of drives. (why is that?) They do not have to have the same number of drivers, you can even mix raidz and plain disks. That is more a recommendation. Add -f to the command. What is the risk of creating a pool consisting of two raidz vdevs that don't have the same number of disks? Slightly different reliability and performance on different parts of the pool. Nothing to worry about in your case. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] recommendations on adding vdev to raidz zpool
On Sun, Oct 26, 2008 at 2:00 PM, Peter Baumgartner [EMAIL PROTECTED] wrote: They do not have to have the same number of drivers, you can even mix raidz and plain disks. That is more a recommendation. Add -f to the command. What is the risk of creating a pool consisting of two raidz vdevs that don't have the same number of disks? One risk is that you mistyped the command, when you actually meant to specify a balanced configuration. -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Cannot remove slog device from zpool
Hello, I've looked quickly through the archives and haven't found mention of this issue. I'm running SXCE (snv_99), which I believe uses zfs version 13. I had an existing zpool: ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Cannot remove slog device from zpool
Sorry for the first incomplete send, stupid Ctrl-Enter. :-) Hello, I've looked quickly through the archives and haven't found mention of this issue. I'm running SXCE (snv_99), which uses zfs version 13. I had an existing zpool: -- [EMAIL PROTECTED] ~]$ zpool status -v data pool: data state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 mirror ONLINE 0 0 0 c4t1d0p0 ONLINE 0 0 0 c4t9d0p0 ONLINE 0 0 0 ... cache c4t15d0p0ONLINE 0 0 0 errors: No known data errors -- The cache device (c4t15d0p0) is an Intel SSD. To test zil, I removed the cache device, and added it as a log device: -- [EMAIL PROTECTED] ~]$ pfexec zpool remove data c4t15d0p0 [EMAIL PROTECTED] ~]$ pfexec zpool add data log c4t15d0p0 [EMAIL PROTECTED] ~]$ zpool status -v data pool: data state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 mirror ONLINE 0 0 0 c4t1d0p0 ONLINE 0 0 0 c4t9d0p0 ONLINE 0 0 0 ... logs ONLINE 0 0 0 c4t15d0p0ONLINE 0 0 0 errors: No known data errors -- The device is working fine. I then said, that was fun, time to remove and add as cache device. But that doesn't seem possible: -- [EMAIL PROTECTED] ~]$ pfexec zpool remove data c4t15d0p0 cannot remove c4t15d0p0: only inactive hot spares or cache devices can be removed -- I've also tried using detach, offline, each failing in other more obvious ways. The manpage does say that those devices should be removable/replaceable. At this point the only way to reclaim my SSD device is to destroy the zpool. Just in-case you are wondering about versions: -- [EMAIL PROTECTED] ~]$ zpool upgrade data This system is currently running ZFS pool version 13. Pool 'data' is already formatted using the current version. [EMAIL PROTECTED] ~]$ uname -a SunOS opensolaris 5.11 snv_99 i86pc i386 i86pc -- Any ideas? Thanks, Ethan ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
I've done some more research, but would still greatly appreciate someone helping me understand this. It seems that writes to only the home directory of the person logged in to the console suffers from degraded performance. If I write to a subdirectory beneath my home, or to any other directory on the system, performance is great. But if I have a session on the console, no matter where else I test from (Gnome or a remote shell), writes ONLY to my home suffer. If I log out of the console and then SSH in from another system, writes to the home directory no longer suffer from degraded performance. This has proven true on every OpenSolaris system I've tried--all of which are using ZFS. So what is it about logging into the console that slows write performance to ONLY the top level home directory of the username on the same console? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
William Bauer wrote: I've done some more research, but would still greatly appreciate someone helping me understand this. It seems that writes to only the home directory of the person logged in to the console suffers from degraded performance. If I write to a subdirectory beneath my home, or to any other directory on the system, performance is great. But if I have a session on the console, no matter where else I test from (Gnome or a remote shell), writes ONLY to my home suffer. If I log out of the console and then SSH in from another system, writes to the home directory no longer suffer from degraded performance. This has proven true on every OpenSolaris system I've tried--all of which are using ZFS. So what is it about logging into the console that slows write performance to ONLY the top level home directory of the username on the same console Maybe something to do with autofs? What happens if you move your home dir? Ethan ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
I just built a homeserver that pulls 62 watts from the plug at idle for ~$700. I had some of the parts lying around but even if you bought everything at frys you should be able to set yourself up for under 1K for the next 3-5 years. Seasonic 80 plus 300 watt power supply Intel DP35DP motherboard (onboard nic is an intel, has 6 sata from ich9) E5200 Wolfdale processor (5 watts at idle, 30 watts at load) 4 x 1TB Samsung EcoGreen 5400 rpm harddrives 4GB ram 80GB laptop harddrive I had lying around for the system disk Ati PCI 9200 video card I had lying around (pulls 2-3 watts) Cheapest mid tower case off newegg with free shipping The samsung drives are great, they don't ever spindown. I had problems with some seagates a year or so back where the drive firmware would spin them down after 90 seconds of inactivity. The samsungs just chug along at 4-5 watts each. I use raidz on the 4 1TB drives and set copies=2 on the system drive. The real power savings is from the cpu. Any dual core 45nm intel chip idles at unbelivably low wattage. Add the 5400 rpm drives and you've got a nice cool running, quiet, low power system. If I only needed 1 TB I'd probably use the atom board with the two ports and a usb jump drive for the system disk, but I needed more space. Daniel On Sun, Oct 26, 2008 at 2:51 AM, Peter Bridge [EMAIL PROTECTED]wrote: Same case and same idea :) I have 2 dampered drives already installed from a previous project. Another 2 I pulled out to install into a qnap 209. Ideally I'd return the disks and replace the qnap with this new single ZFS NAS, although I'm quite fond of the qnap bt client, and can't use all four SATA with this board anyway... -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
If that were the case, why would it matter if I was logged into the console, and why would subdirectories of my home exhibit better write performance than the top level home directory? A write to /export/home/username is slower than to /export/home/username/blah, but ONLY if that user is logged into the console. I'm not saying it's not an odd autofs interaction, because I've seen things just as strange. It just seems like the console login is the tripwire for this performance issue, and autofs would come to play at every login and not just a login to the console. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] recommendations on adding vdev to raidz zpool
On Sun, 26 Oct 2008, Peter Baumgartner wrote: What is the risk of creating a pool consisting of two raidz vdevs that don't have the same number of disks? Virtually no risk. The only matter of concern would be if the vdevs have substantially different I/O performance and latencies since ZFS's load share mechanism will put more data on the vdev which is more responsive when the vdevs are under heavy write load. However, if the faster vdev is built with drives of newer design and with higher capacity, this behavior may be a good thing. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Boot from mirror
dick hoogendijk wrote: Is it or isn't it possible to boot off two mirrored ZFS disks and if yes, can this be done in the upcoming solaris 10 10/08 too? Yes. Yes. For details, please consult the ZFS Administration Guide. http://www.opensolaris.org/os/community/zfs/docs/zfsadmin.pdf -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Cannot remove slog device from zpool
CR 6574286 removing a slog doesn't work http://bugs.opensolaris.org/view_bug.do?bug_id=6574286 -- richard Ethan Erchinger wrote: Sorry for the first incomplete send, stupid Ctrl-Enter. :-) Hello, I've looked quickly through the archives and haven't found mention of this issue. I'm running SXCE (snv_99), which uses zfs version 13. I had an existing zpool: -- [EMAIL PROTECTED] ~]$ zpool status -v data pool: data state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 mirror ONLINE 0 0 0 c4t1d0p0 ONLINE 0 0 0 c4t9d0p0 ONLINE 0 0 0 ... cache c4t15d0p0ONLINE 0 0 0 errors: No known data errors -- The cache device (c4t15d0p0) is an Intel SSD. To test zil, I removed the cache device, and added it as a log device: -- [EMAIL PROTECTED] ~]$ pfexec zpool remove data c4t15d0p0 [EMAIL PROTECTED] ~]$ pfexec zpool add data log c4t15d0p0 [EMAIL PROTECTED] ~]$ zpool status -v data pool: data state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 mirror ONLINE 0 0 0 c4t1d0p0 ONLINE 0 0 0 c4t9d0p0 ONLINE 0 0 0 ... logs ONLINE 0 0 0 c4t15d0p0ONLINE 0 0 0 errors: No known data errors -- The device is working fine. I then said, that was fun, time to remove and add as cache device. But that doesn't seem possible: -- [EMAIL PROTECTED] ~]$ pfexec zpool remove data c4t15d0p0 cannot remove c4t15d0p0: only inactive hot spares or cache devices can be removed -- I've also tried using detach, offline, each failing in other more obvious ways. The manpage does say that those devices should be removable/replaceable. At this point the only way to reclaim my SSD device is to destroy the zpool. Just in-case you are wondering about versions: -- [EMAIL PROTECTED] ~]$ zpool upgrade data This system is currently running ZFS pool version 13. Pool 'data' is already formatted using the current version. [EMAIL PROTECTED] ~]$ uname -a SunOS opensolaris 5.11 snv_99 i86pc i386 i86pc -- Any ideas? Thanks, Ethan ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
For those of you who wants to build a NAS, this is mandatory reading I think. Read all comments too. http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ I use a P45 mobo, Intel Q9450, ATI4850 and 4 GB RAM. AOC SATA card with 8 Sata slots. 4 Samsung 500GB drives. Works excellent in a P182 antec. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
On Sun, 26 Oct 2008, William Bauer wrote: This has proven true on every OpenSolaris system I've tried--all of which are using ZFS. So what is it about logging into the console that slows write performance to ONLY the top level home directory of the username on the same console? Recently OpenSolaris included a 'tracker' daemon and supportive framework which responds to any filesystem change in the user's home directory and opens the changed file so that it can be indexed. There were probably already Gnome features which depend on this file change detection (e.g. so that an icon may be generated). Perhaps this explains the problem? Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
If that were the case, why would it matter if I was logged into the console, and why would subdirectories of my home exhibit better write performance than the top level home directory? A write to /export/home/username is slower than to /export/home/username/blah, but ONLY if that user is logged into the console. This smells of name resolution delays somewhere. Do you have a shell prompt that gets some host name or user name from name services? Is your /home directory owned by a non-existing user or group? Do you accidentally have something enabled in /etc/nsswitch.conf that does not exist (ldap, nis, nis+)? Maybe the first miss gets cached and all other misses get resolved from the cache? Or your nscd is disabled/confused/broken disabled altogether? You have an interesting problem there... Good luck -- Volker -- Volker A. Brandt Consulting and Support for Sun Solaris Brandt Brandt Computer GmbH WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim Email: [EMAIL PROTECTED] Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgröße: 45 Geschäftsführer: Rainer J. H. Brandt und Volker A. Brandt ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zpool import problem
Folks, I have a zpool with a raidz2 configuration which I've been switching between two machines - an old one with a hardware problem and a new one, which doesn't have hardware issues, but has a different configuration . I've been trying to import the pool on the new machine, so I can back up the data, because the old (broken) machine resets (I don't think it's panicking, because there are no logged messages) every time I try to tar off the data from the ZFS. Unfortunately, the first time I tried to import the pool on the new machine, I didn't have the right five drives in it, so it didn't work. After I figured out that I was confused about which was the boot drive, I did get the five drives into the new machine and asked it to import the pool. It said that the pool could not be imported due to damaged devices or data. Which is slightly odd, since it had been mounting the pool fine on the broken machine before. I then moved the drives back into the old machine, figuring I'd at least copy some small stuff onto a USB stick (it only dies reading large files, apparently), but now the old machine can't mount the pool either, and asking it to import gives the same message. It shows all five drives online, but says the pool is UNAVAIL due to insufficient replicas, and the raidz2 is UNAVAIL due to corrupted data. Must I resign myself to having lost this pool due to the hardware problems I've had, and restore such backups as I have on the new machine, or is there something that can be done to get the pool back online at least in degraded mode? Thanks in advance, --Terry. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
Ok on the answers to all my questions. There's nothing that really stands out as being obviously wrong. Just out of interest, what build of OpenSolaris are you using? One thing you could try on the Ethernet capture file, is to set the WireShark 'Time' column like this: View Time Display Format Seconds Since Previous Displayed Packet Then look down the time column for any unusual high time delays between packets. Any unusually high delays during a data transfer phase, may indicate a problem. Another thing you could try is measuring network performance with a utility called 'iperf'. It's not part of Solaris, so you would need to compile it. Download the source from here: http://sourceforge.net/projects/iperf/ I've just compiled the latest version 2.0.4 on snv_93 without problem, using the normal configure, make, make install. If you want to run 'iperf' on a windows box, you can download a '.exe' of an older version here: http://www.noc.ucf.edu/Tools/Iperf/ You can find tutorials on how to use it at these links: http://www.openmaniak.com/iperf.php http://www.enterprisenetworkingplanet.com/netos/article.php/3657236 I've just tried 'iperf' between my OpenSolaris pc an old Windows pc, both with low-cost realtek gigabit cards and linked via a low-cost NetGear switch. I measured a TCP bandwidth of 196 Mbit/sec in one direction and 145 Mbit/sec in the opposite direction. (On OpenSolaris, Iperf was not able to increase the default TCP window size of 48K bytes.) Regards Nigel Smith -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] HELP! zfs with iscsitadm (on poweredge1900) and VMWare!
I asked Tano to use the 'snoop' command to capture the Ethernet
packets to a file, while he attempted VMware's 'VMotion'.
# snoop -d {device} -o {filename} tcp port 3260
This file was made available to me on Tano's web server.
The file size was nearly 85 Mbytes, capturing over 100,000 packets.
I have downloaded the capture file, and been looking at it with
Ethereal and WireShark.
I do not have a corresponding 'iscsisnoop.d' file, but from the pattern
of activity that I see, I can well imagine that it would show the same
pattern of that we saw from Eugene, which I reported on here:
http://mail.opensolaris.org/pipermail/storage-discuss/2008-October/006444.html
(So here I'm looking at what's happening at the lower TCP level,
rather than at the iScsi level.)
In the Ethernet capture file, I can see the pattern of bursts of
writes from the initiator. The Target can accept so many of these,
and then needs to slow things down by reducing the TCP window size.
Eventually the target says the TCP Window size is zero, effectively
asking the initiator to stop.
Now to start with, the target only leaves the 'TCP ZeroWindow', in
place for a fraction of a second. Then it opens things up again
by sending a 'TCP Window Update', restoring the window to 65160 bytes,
and transfer resumes. This is normal and expected.
But eventually we get to a stage where the target sets the TCP 'ZeroWindow'
and leaves it there for an extended period of time. I talking about seconds
here.
The initiator starts to send 'TCP ZeroWindowProbe' packets every 5 seconds.
The target promptly responds with a 'TCP ZeroWindowProbeAck' packet.
(Presumably, this is the initiator just confirming that the target is still
alive.)
This cycle of Probes Ack's repeats for 50 seconds.
During this period the target shows no sign of wanting to accept any more data.
Then the initiator seems to decide it has had enough, and just cannot
be bothered to wait any longer, and it [RST,ACK]'s the TCP session, and
then starts a fresh iscsi login.
(And then we go around the whole cycle of the pattern again.)
The question is why has the target refused to accept any more data for over 50
seconds!
The obvious conclusion would be that the OpenSolaris box is so busy that
it does not have any time left to empty the network stack buffers.
But this then just leads you to another question - why?
So the mystery deepens, and I am running out of ideas!
Tano, maybe you could check the network performance, with the 'iperf'
programs, as mentioned here:
http://mail.opensolaris.org/pipermail/zfs-discuss/2008-October/052136.html
Does the OpenSolaris box give any indication of being busy with other things?
Try running 'prstat' to see if it gives any clues.
Presumably you are using ZFS as the backing store for iScsi, in
which case, maybe try with a UFS formatted disk to see if that is a factor.
Regards
Nigel Smith
--
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
Nigel Smith wrote: Ok on the answers to all my questions. There's nothing that really stands out as being obviously wrong. Just out of interest, what build of OpenSolaris are you using? One thing you could try on the Ethernet capture file, is to set the WireShark 'Time' column like this: View Time Display Format Seconds Since Previous Displayed Packet Then look down the time column for any unusual high time delays between packets. Any unusually high delays during a data transfer phase, may indicate a problem. Along with the errors that I noted previously, some of the packets to seem to be taking a rather long time (0.5s). I've taken a cap file from wireshark in the hope it clears up some information. The capture is less than a minute of playing a video over the cifs share. It's a little too large to send in a mail so I've posted it at http://distfiles.genestate.com/_1_20081027010354.zip Another thing you could try is measuring network performance with a utility called 'iperf'. Thanks for pointing this program out, I've just run it to the gentoo firewall we've got, and it's reporting good speeds for the network. Thanks Matt ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
Nigel Smith wrote: Ok on the answers to all my questions. There's nothing that really stands out as being obviously wrong. Just out of interest, what build of OpenSolaris are you using? Damn forgot to add that, I'm running SXCE snv_97. Thanks Matt ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
This smells of name resolution delays somewhere. Do you have a shell prompt that gets some host name or user name from name services? Is your /home directory owned by a non-existing user or group? Do you accidentally have something enabled in /etc/nsswitch.conf that does not exist (ldap, nis, nis+)? Maybe the first miss gets cached and all other misses get resolved from the cache? Or your nscd is disabled/confused/broken disabled altogether? I think you've misunderstood something here, perhaps in the way I've tried to explain it. I'm having trouble figuring out what that might be. Perhaps your response was to the wrong discussion? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
This sounds plausible I suppose Being unfamiliar with this tracker daemon, I can blindly accept it as a maybe! -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
For clarity, here's how you can reproduce what I'm asking about: This is for local file systems on build 86 and not about NFS or any remote mounts. You can repeat these 100 times and always get the same result, whether you reboot between trials or leave the system running. 1. Log into the console of your OpenSolaris system. 2. Open a terminal in your home directory. 3. Create a file as in the command below and note the write speed it returns. I know this method is not a true test of write performance, but I'm interested in relative and not absolute numbers. This method seems quite consistent in the results it returns. dd if=/dev/zero of=outfile bs=1024 count=100 4. Enter a subdirectory of your home and do the same as above. 5. Log out of the console and perform the same tests via a remote login using SSH or your favorite method. See that now there is no difference in performance. I've found the difference between steps 3 and 4 to be typically 25%. This happens even with the most vanilla new install. On VirtualBox, the differences can be more like 5x (i.e. 500%!). Note that if you do step 5 when still logged into the console, you get the same results as in step 3-4. That's it. Maybe not hugely important, but I'm trying to understand why this happens. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool import problem
Terry Heatlie wrote: Folks, I have a zpool with a raidz2 configuration which I've been switching between two machines - an old one with a hardware problem and a new one, which doesn't have hardware issues, but has a different configuration . I've been trying to import the pool on the new machine, so I can back up the data, because the old (broken) machine resets (I don't think it's panicking, because there are no logged messages) every time I try to tar off the data from the ZFS. Unfortunately, the first time I tried to import the pool on the new machine, I didn't have the right five drives in it, so it didn't work. After I figured out that I was confused about which was the boot drive, I did get the five drives into the new machine and asked it to import the pool. It said that the pool could not be imported due to damaged devices or data. Which is slightly odd, since it had been mounting the pool fine on the broken machine before. I then moved the drives back into the old machine, figuring I'd at least copy some small stuff onto a USB stick (it only dies reading large files, apparently), but now the old machine can't mount the pool either, and asking it to import gives the same message. It shows all five drives online, but says the pool is UNAVAIL due to insufficient replicas, and the raidz2 is UNAVAIL due to corrupted data. Must I resign myself to having lost this pool due to the hardware problems I've had, and restore such backups as I have on the new machine, or is there something that can be done to get the pool back online at least in degraded mode? Note: we're also working on a troubleshooting wiki... need more days in the hour... You should try to read the labels from each device. zdb -l /dev/rdsk/... You should see 4 labels for each proper device. Here is my hypothesis: If you see a device which has only label 0 and 1, then it may be the case that the label has overlapping partitions. Why does this matter? Because under normal circumstances, the actual devices used for creating or importing the pool are stored in the /etc/zfs/zpool.cache file. When the system boots, it looks there first and will import the pools listed therein. When you export the pool, the zpool.cache entries for the pool are removed. If the pool is not in zpool.cache, then zpool import scans all of the devices found in /dev/dsk for valid pools. If you have overlapping partitions or slices, then a partially exposed vdev may be found. But since it won't be complete, due to perhaps not being able to see the end of the device which is where labels 2 3 are located, then it will be marked as bad. The solution would be to reconcile the partions/slices using format. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
I cannot recreate this on b101. There is no significant difference between the two on my system. -- richard William Bauer wrote: For clarity, here's how you can reproduce what I'm asking about: This is for local file systems on build 86 and not about NFS or any remote mounts. You can repeat these 100 times and always get the same result, whether you reboot between trials or leave the system running. 1. Log into the console of your OpenSolaris system. 2. Open a terminal in your home directory. 3. Create a file as in the command below and note the write speed it returns. I know this method is not a true test of write performance, but I'm interested in relative and not absolute numbers. This method seems quite consistent in the results it returns. dd if=/dev/zero of=outfile bs=1024 count=100 4. Enter a subdirectory of your home and do the same as above. 5. Log out of the console and perform the same tests via a remote login using SSH or your favorite method. See that now there is no difference in performance. I've found the difference between steps 3 and 4 to be typically 25%. This happens even with the most vanilla new install. On VirtualBox, the differences can be more like 5x (i.e. 500%!). Note that if you do step 5 when still logged into the console, you get the same results as in step 3-4. That's it. Maybe not hugely important, but I'm trying to understand why this happens. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
I cannot recreate this on b101. There is no significant difference between the two on my system. That's encouraging...unless no one can reproduce it on 86, then I'm forgetting something. I've done this a dozen times on several systems, so maybe ZFS performance has been improved. What numbers did you get (out of curiosity)? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
Bingo! I just updated a system from 86 to 99 and the problem is gone. Even better, it was a VB guest, and the ZFS performance on the guest increased 5x in this test, as I mentioned earlier. Granted, a VB guest may not be the best test and it only applies to top level home directories, but it was consistent and now returns the same higher numbers no matter what. Thank you for testing b101. So how close ARE we to the official November release? Early, mid, or late in the month...! -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write performance on boot disk
After a zpool upgrade, this simple test's write speed jumped up yet another 20%. Looks like ZFS is getting better. As one would hope expect. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
