Re: [zfs-discuss] Faster than 1G Ether... ESX to ZFS
From: Saxon, Will [mailto:will.sa...@sage.com] What I am wondering is whether this is really worth it. Are you planning to share the storage out to other VM hosts, or are all the VMs running on the host using the 'local' storage? I know we like ZFS vs. traditional RAID and volume management, and I get that being able to boot any ZFS-capable OS is good for disaster recovery, but what I don't get is how this ends up working better than a larger dedicated ZFS system and a storage network. Is it cheaper over several hosts? Are you getting better performance through e.g. the vmxnet3 adapter and NFS than you would just using the disks directly? I also don't know enough details of how this works out. In particular: If your goal is high performance storage, snapshots, backups, and data integrity for Linux or some other OS (AKA, ZFS on Linux or Windows) then you should be able to win with this method of Linux ZFS server both in VM's of a single physical server, utilizing a vmnet switch and either NFS or iSCSI or CIFS. But until some benchmarks are done, to show that vmware isn't adding undue overhead, I must consider it still unproven. As compared to one big ZFS server being used as the backend SAN for a bunch of vmware hosts... If your goal is high performance for distributed computing, then you always need to use local disk attached independently to each of the compute nodes. There's simply no way you can scale any central server large enough to handle a bunch of hosts without any performance loss. Assuming the ZFS server is able to max out its local disks... If there exists a bus which is fast enough for a remote server to max out those disks... Then the remote server should have the storage attached locally, because the physical disks are the performance bottleneck. If your goal is just use ZFS datastore for all your vmware hosts, ... AKA you're mostly interested in checksumming and snapshots, you're not terribly concerned with performance as long as it's fast enough, then most likely you'll be fine with using 1Gb ether because it's so cheap. Maybe you upgrade to a faster or different type of bus (fc or ib). ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Faster than 1G Ether... ESX to ZFS
Suppose if you wanted to boot from an iscsi target, just to get vmware a ZFS server up. And then you could pass-thru the entire local storage bus(es) to the ZFS server, and you could create other VM's whose storage is backed by the ZFS server on local disk. One way you could do this is to buy FC or IB adapter, and I presume these have some BIOS support to configure bootable iscsi targets. But is there any such thing for 1Gb ether? For this purpose, I think it would be fine for vmware ZFS server OSes to be physically remote via iscsi 1Gb. OTOH ... maybe PXE is the way to go. I've never done anything with PXE, but I certainly know it's pretty darn universal. I guess I'll have to look into it. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS - Sudden decrease in write performance
arc-discuss doesn't have anything specifically to do with ZFS; in particular, it has nothing to do with the ZFS ARC. Just an unfortunate overlap of acronyms. Cross-posted to zfs-discuss, where this probably belongs. Hey all1 Recently I've decided to implement OpenSolaris as a target for BackupExec. The server I've converted into a Storage Appliance is an IBM x3650 M2 w/ ~4TB of on board storage via ~10 local SATA drives and I'm using OpenSolaris svn_134. I'm using a QLogic 4Gb FC HBA w/ the QLT driver and presented an 8TB sparse volume to the host due to dedup and compression being turned on for the zpool. When writes begin, I see anywhere from 4.5GB/Min to 5.5GB/Min and then it drops of quickly (I mean down to 1GB/Min or less). I've already swapped out the card, cable, and port with no results. I have since ensured that every piece of equipment on the box had it's firmware updated. While doing so, I installed Windows Server 2008 to flash all the firmware (IBM doesn't have a Solaris installer). While in Server 2008, I decided to just attempt a backup via share on the 1Gbs copper connection. I saw speeds of up to 5.5GB/Min consistently and they were sustained throughout 3 days of testing. Today I decided to move back to OpenSolaris with confidence. All writes began at 5.5GB/Min and quickly dropped off. In my troubleshooting efforts, I have also dropped the fiber connection and made it an iSCSI target with no performance gains. I have let the on board RAID controller do the RAID portion instead of creating a zpool of multiple disks with no performance gains. And, I have created the target LUN using both rdsk and dsk paths. I did notice today though, that there is a direct correlation between the ARC memory usage and speed. Using arcstat.pl, as soon as arcsz hits 1G (half of c column [commit?]), my throughput hits the floor (i.e. 600MB/Min or less). I can't figure it out. I tried every configuration possible. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Running on Dell hardware?
From: Edward Ned Harvey [mailto:sh...@nedharvey.com] I have a Dell R710 which has been flaky for some time. It crashes about once per week. I have literally replaced every piece of hardware in it, and reinstalled Sol 10u9 fresh and clean. It has been over 3 weeks now, with no crashes, and me doing everything I can to get it to crash again. So I'm going to call this one resolved... Tentatively acknowledging the remote possibility that the problem could still come back. All I did was disable the built-in Broadcom network cards, and buy an add-on Intel network card (EXPI9400PT). It is worth noting, that the built-in bcom card cannot be completely disabled if you want to use ipmi... It's disabled for OS only, but the iDRAC ipmi traffic still goes across the bcom interface. So now I have two network cables running to the machine, one of which is only used for ipmi. No big deal. I had ports to spare on my switch, and the system is stable. It's the fault of the Broadcom card. Rumor has it (from dell support technician) that the bcom cards have been problematic in other OSes too... It's not isolated to solaris. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zpool lockup and dedupratio meanings
I've previously posted about some lockups I've experienced with ZFS. There were two suspected causes at the time: one was deduplication, and one was the 2009.06 code we were running. I upgraded to OpenIndiana 147 (initially without upgrading the zpool and zfs disk versions). The lockups reduced in frequency but still occurred. I've since upgraded the zpool and zfs versions and we'll see what happens. Dedup was the more likely causes and so we turned it off and recreated all the iscsi LUNS that were being exported so as to eliminate the deduplicated data. That almost entirely eliminated the lockups. Having said all that- I have two questions: When I query for the dedupratio, I still a value of 2.37x -- r...@nas:~# zpool get dedupratio pool0 NAME PROPERTYVALUE SOURCE pool0 dedupratio 2.37x - -- Considering the fact that all of the iscsi LUNS that were created when dedup was on were subsequently deleted and recreated with dedup disabled- I don't understand why the value is still 2.37x. It should be near zero (There are probably a couple of small luns that were not removed but they are rarely used). Am I misinterpreting the meaning of this number? Second question: The most recent pool lockup was caused when a zpool scrub was kicked off. Initially we see 0 values for the write bandwidth in a zpool iostat and average numbers for the read. After a few minutes we see the read numbers jump to several hundred megs/second and the write performance fluctuate between 0 and a few kilobytes/second. Anyone else see this behavior and can provide some insight? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
