Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread gm_sjo
2008/9/30 Jean Dion [EMAIL PROTECTED]: iSCSI requires dedicated network and not a shared network or even VLAN. Backup cause large I/O that fill your network quickly. Like ans SAN today. Could you clarify why it is not suitable to use VLANs for iSCSI?

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread MC
The good news is that even though the answer to your question is no, it doesn't matter because it sounds like what you are doing is a piece of cake :) Given how cheap hardware is, and how modest your requirements sound, I expect you could build multiple custom systems for the cost of an EMC

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
Thanks for all the answers .. Please find more questions below :) - Good to know EMC filers do not have end2end checksums! What about netapp ? - Any other limitations of the big two NAS vendors as compared to zfs ? - I still don't have my original question answered, I want to somehow assess the

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread David Magda
On Sep 30, 2008, at 06:58, Ahmed Kamal wrote: - I still don't have my original question answered, I want to somehow assess the reliability of that zfs storage stack. If there's no hard data on that, then if any storage expert who works with lots of systems can give his impression of

[zfs-discuss] ZSF Solaris

2008-09-30 Thread Ram Sharma
Hi, can anyone please tell me what is the maximum number of files that can be there in 1 folder in Solaris with ZSF file system. I am working on an application in which I have to support 1mn users. In my application I am using MySql MyISAM and in MyISAM there is 3 files created for 1 table. I

Re: [zfs-discuss] ZSF Solaris

2008-09-30 Thread Mark J Musante
On Tue, 30 Sep 2008, Ram Sharma wrote: Hi, can anyone please tell me what is the maximum number of files that can be there in 1 folder in Solaris with ZSF file system. By folder, I assume you mean directory and not, say, pool. In any case, the 'limit' is 2^48, but that's effectively no

Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread Jean Dion
Simple. You cannot go faster than the slowest link. Any VLAN share the bandwidth workload and do not provide a dedicated bandwidth for each of them. That means if you have multiple VLAN coming out of the same wire of your server you do not have "n" time the bandwidth but only a fraction of

Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread Gary Mills
On Mon, Sep 29, 2008 at 06:01:18PM -0700, Jean Dion wrote: Do you have dedicated iSCSI ports from your server to your NetApp? Yes, it's a dedicated redundant gigabit network. iSCSI requires dedicated network and not a shared network or even VLAN. Backup cause large I/O that fill your

Re: [zfs-discuss] ZSF Solaris

2008-09-30 Thread Marcelo Leal
ZFS has not limit for snapshots and filesystems too, but try to create a lot snapshots and filesytems and you will have to wait a lot for your pool to import too... ;-) I think you should not think about the limits, but performance. Any filesytem with *too many entries by directory will

Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread Jean Dion
For Solaris internal debugging tools look here http://opensolaris.org/os/community/advocacy/events/techdays/seattle/OS_SEA_POD_JMAURO.pdf;jsessionid=9B3E275EEB6F1A0E0BC191D8DEC0F965 ZFS specifics is available here http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide Jean

[zfs-discuss] c1t0d0 to c3t1d0

2008-09-30 Thread dick hoogendijk
I have a ZFS disk (c1t0d0) in a eSATA/USB2 enclosure. If I would build this drive in the machine (internal SATA) it would become c3t1do. When I did it (for testing) zpool status did not see it. What do I have to do to be able to switch this drive? -- Dick Hoogendijk -- PGP/GnuPG key: 01D2433D

Re: [zfs-discuss] c1t0d0 to c3t1d0

2008-09-30 Thread Will Murnane
On Tue, Sep 30, 2008 at 14:00, dick hoogendijk [EMAIL PROTECTED] wrote: What do I have to do to be able to switch this drive? I'd suggest running zpool import. If that doesn't show the pool, put it back in the external enclosure, run zpool export mypool and then see if it shows up in zpool

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Richard Elling
Ahmed Kamal wrote: Thanks for all the answers .. Please find more questions below :) - Good to know EMC filers do not have end2end checksums! What about netapp ? If they are not at the end, they can't do end-to-end data validation. Ideally, application writers would do this, but it is a lot

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Tue, 30 Sep 2008, Ahmed Kamal wrote: - I still don't have my original question answered, I want to somehow assess the reliability of that zfs storage stack. If there's no hard data on that, then if any storage expert who works with lots of systems can give his impression of the reliability

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
I guess I am mostly interested in MTDL for a zfs system on whitebox hardware (like pogo), vs dataonTap on netapp hardware. Any numbers ? On Tue, Sep 30, 2008 at 4:36 PM, Bob Friesenhahn [EMAIL PROTECTED] wrote: On Tue, 30 Sep 2008, Ahmed Kamal wrote: - I still don't have my original

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Tue, 30 Sep 2008, Ahmed Kamal wrote: I guess I am mostly interested in MTDL for a zfs system on whitebox hardware (like pogo), vs dataonTap on netapp hardware. Any numbers ? Barring kernel bugs or memory errors, Richard Elling's blog entry seems to be the best place use as a guide:

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Richard Elling
Ahmed Kamal wrote: I guess I am mostly interested in MTDL for a zfs system on whitebox hardware (like pogo), vs dataonTap on netapp hardware. Any numbers ? It depends to a large degree on the disks chosen. NetApp uses enterprise class disks and you can expect better reliability from such

Re: [zfs-discuss] zpool import of bootable root pool renders it

2008-09-30 Thread Juergen Nickelsen
Stephen Quintero [EMAIL PROTECTED] writes: I am running OpenSolaris 2008.05 as a PV guest under Xen. If you import the bootable root pool of a VM into another Solaris VM, the root pool is no longer bootable. I had a similar problem: After installing and booting Opensolaris 2008.05, I succeded

Re: [zfs-discuss] zpool error: must be a block device or regular file

2008-09-30 Thread William D. Hathaway
The zfs kernel modules handle the caching/flushing of data across all the devices in the zpools. It uses a different method for this than the standard virtual memory system used by traditional file systems like UFS. Try defining your NVRAM card with ZFS as a log device using the /dev/dsk/xyz

Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread William D. Hathaway
Gary - Besides the network questions... What does your zpool status look like? Are you using compression on the file systems? (Was single-threaded and fixed in s10u4 or equiv patches) -- This message posted from opensolaris.org ___

Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread gm_sjo
2008/9/30 Jean Dion [EMAIL PROTECTED]: Simple. You cannot go faster than the slowest link. That is indeed correct, but what is the slowest link when using a Layer 2 VLAN? You made a broad statement that iSCSI 'requires' a dedicated, standalone network. I do not believe this is the case. Any

Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread Jean Dion
Normal iSCSI setup split network traffic at physical layer and not logical layer. That mean physical ports and often physical PCI bridge chip if you can. That will be fine for small traffic but we are talking backup performance issues. IP network and number of small files are very often the

Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread Gary Mills
On Tue, Sep 30, 2008 at 10:32:50AM -0700, William D. Hathaway wrote: Gary - Besides the network questions... Yes, I suppose I should see if traffic on the Iscsi network is hitting a limit of some sort. What does your zpool status look like? Pretty simple: $ zpool status pool:

Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread Gary Mills
On Mon, Sep 29, 2008 at 06:01:18PM -0700, Jean Dion wrote: Legato client and server contains tuning parameters to avoid such small file problems. Check your Legato buffer parameters. These buffer will use your server memory as disk cache. Our backup person tells me that there are no

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Miles Nordin
ak == Ahmed Kamal [EMAIL PROTECTED] writes: ak I need to answer and weigh against the cost. I suggest translating the reliability problems into a cost for mitigating them: price the ZFS alternative as two systems, and keep the second system offline except for nightly backup. Since you care

Re: [zfs-discuss] zfs allow interaction with file system privileges

2008-09-30 Thread Paul B. Henson
On Tue, 23 Sep 2008, Darren J Moffat wrote: Run the service with the file_chown privilege. See privileges(5), rbac(5) and if it runs as an SMF service smf_method(5). Thanks for the pointer. After reviewing this documentation, it seems that file_chown_self is the best privilege to delegate, as

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
Thanks guys, it seems the problem is even more difficult than I thought, and it seems there is no real measure for the software quality of the zfs stack vs others, neutralizing the hardware used under both. I will be using ECC RAM, since you mentioned it, and I will shift to using enterprise disks

Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread gm_sjo
2008/9/30 Jean Dion [EMAIL PROTECTED]: If you want performance you do not put all your I/O across the same physical wire. Once again you cannot go faster than the physical wire can support (CAT5E, CAT6, fibre). No matter if it is layer 2 or not. Using VLAN on single port you share the

[zfs-discuss] Oracle DB sequential dump questions

2008-09-30 Thread Louwtjie Burger
Server: T5120 on 10 U5 Storage: Internal 8 drives on SAS HW RAID (R5) Oracle: ZFS fs, recordsize=8K and atime=off Tape: LTO-4 (half height) on SAS interface. Dumping a large file from memory using tar to LTO yields 44 MB/s ... I suspect the CPU cannot push more since it's a single thread doing

Re: [zfs-discuss] c1t0d0 to c3t1d0

2008-09-30 Thread andrew
Inserting the drive does not automatically mount the ZFS filesystem on it. You need to use the zpool import command which lists any pools available to import, then zpool import -f {name of pool} to force the import (to force the import if you haven't exported the pool first). Cheers Andrew.

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 2:10 PM, Ahmed Kamal [EMAIL PROTECTED] wrote: * Now that I'm using ECC RAM, and enterprisey disks, Does this put this solution in par with low end netapp 2020 for example ? *sort of*. What are you going to be using it for? Half the beauty of NetApp are all the

Re: [zfs-discuss] ZFS performance degradation when backups are running

2008-09-30 Thread Richard Elling
gm_sjo wrote: 2008/9/30 Jean Dion [EMAIL PROTECTED]: If you want performance you do not put all your I/O across the same physical wire. Once again you cannot go faster than the physical wire can support (CAT5E, CAT6, fibre). No matter if it is layer 2 or not. Using VLAN on single port

Re: [zfs-discuss] Oracle DB sequential dump questions

2008-09-30 Thread Carson Gaspar
Louwtjie Burger wrote: Dumping a large file from memory using tar to LTO yields 44 MB/s ... I suspect the CPU cannot push more since it's a single thread doing all the work. Dumping oracle db files from filesystem yields ~ 25 MB/s. The interesting bit (apart from it being a rather slow

Re: [zfs-discuss] ZFS poor performance on Areca 1231ML

2008-09-30 Thread Tim
No apology necessary and I'm glad you figured it out - I was just reading this thread and thinking I'm missing something here - this can't be right. If you have the budget to run a few more experiments, try this SuperMicro card: http://www.springsource.com/repository/app/faq that others

Re: [zfs-discuss] Segmentation fault / core dump with recursive send/recv

2008-09-30 Thread BJ Quinn
Is there more information that I need to post in order to help diagnose this problem? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] ZFS poor performance on Areca 1231ML

2008-09-30 Thread Al Hopper
On Mon, Sep 29, 2008 at 12:57 PM, Ross Becker [EMAIL PROTECTED] wrote: I have to come back and face the shame; this was a total newbie mistake by myself. I followed the ZFS shortcuts for noobs guide off bigadmin; http://wikis.sun.com/display/BigAdmin/ZFS+Shortcuts+for+Noobs What that had

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread MC
Just to confuse you more, I mean, give you another point of view: - CPU: 1 Xeon Quad Core E5410 2.33GHz 12MB Cache 1333MHz The reason the Xeon line is good is because it allows you to squeeze maximum performance out of a given processor technology from Intel, possibly getting the highest

Re: [zfs-discuss] Segmentation fault / core dump with recursive send/recv

2008-09-30 Thread Richard Elling
BJ Quinn wrote: Is there more information that I need to post in order to help diagnose this problem? Segmentation faults should be correctly handled by the software. Please file a bug and attach the core. http://bugs.opensolaris.org -- richard

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Toby Thain
On 30-Sep-08, at 6:58 AM, Ahmed Kamal wrote: Thanks for all the answers .. Please find more questions below :) - Good to know EMC filers do not have end2end checksums! What about netapp ? Blunty - no remote storage can have it by definition. The checksum needs to be computed as close as

Re: [zfs-discuss] zpool import of bootable root pool renders it

2008-09-30 Thread Robert Milkowski
Hello Juergen, Tuesday, September 30, 2008, 5:43:56 PM, you wrote: JN Stephen Quintero [EMAIL PROTECTED] writes: I am running OpenSolaris 2008.05 as a PV guest under Xen. If you import the bootable root pool of a VM into another Solaris VM, the root pool is no longer bootable. JN I had a

Re: [zfs-discuss] ZSF Solaris

2008-09-30 Thread Toby Thain
On 30-Sep-08, at 7:50 AM, Ram Sharma wrote: Hi, can anyone please tell me what is the maximum number of files that can be there in 1 folder in Solaris with ZSF file system. I am working on an application in which I have to support 1mn users. In my application I am using MySql MyISAM

Re: [zfs-discuss] ZFS poor performance on Areca 1231ML

2008-09-30 Thread Al Hopper
On Tue, Sep 30, 2008 at 3:51 PM, Tim [EMAIL PROTECTED] wrote: No apology necessary and I'm glad you figured it out - I was just reading this thread and thinking I'm missing something here - this can't be right. If you have the budget to run a few more experiments, try this SuperMicro

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Erik Trimble
Toby Thain wrote: On 30-Sep-08, at 6:58 AM, Ahmed Kamal wrote: Thanks for all the answers .. Please find more questions below :) - Good to know EMC filers do not have end2end checksums! What about netapp ? Blunty - no remote storage can have it by definition. The checksum

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 4:26 PM, Toby Thain [EMAIL PROTECTED]wrote: On 30-Sep-08, at 6:58 AM, Ahmed Kamal wrote: Thanks for all the answers .. Please find more questions below :) - Good to know EMC filers do not have end2end checksums! What about netapp ? Blunty - no remote storage

Re: [zfs-discuss] zpool import of bootable root pool renders it

2008-09-30 Thread David Finberg
On Tue, 30 Sep 2008, Robert Milkowski wrote: Hello Juergen, Tuesday, September 30, 2008, 5:43:56 PM, you wrote: JN Stephen Quintero [EMAIL PROTECTED] writes: I am running OpenSolaris 2008.05 as a PV guest under Xen. If you import the bootable root pool of a VM into another Solaris VM, the

Re: [zfs-discuss] Segmentation fault / core dump with recursive send/recv

2008-09-30 Thread BJ Quinn
Please forgive my ignorance. I'm fairly new to Solaris (Linux convert), and although I recognize that Linux has the same concept of Segmentation faults / core dumps, I believe my typical response to a Segmentation Fault was to upgrade the kernel and that always fixed the problem (i.e. somebody

Re: [zfs-discuss] ZFS poor performance on Areca 1231ML

2008-09-30 Thread Ross Becker
At this point, ZFS is performing admirably with the Areca card. Also, that card is only 8-port, and the Areca controllers I have are 12-port. My chassis has 24 SATA bays, so being able to cover all the drives with 2 controllers is preferable. Also, the driver for the Areca controllers is

Re: [zfs-discuss] Segmentation fault / core dump with recursive send/recv

2008-09-30 Thread Richard Elling
BJ Quinn wrote: Please forgive my ignorance. I'm fairly new to Solaris (Linux convert), and although I recognize that Linux has the same concept of Segmentation faults / core dumps, I believe my typical response to a Segmentation Fault was to upgrade the kernel and that always fixed the

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Erik Trimble
Will Murnane wrote: On Tue, Sep 30, 2008 at 21:48, Tim [EMAIL PROTECTED] wrote: why ZFS can do this and hardware solutions can't (being several unreliable subsystems away from the data). So how is a Server running Solaris with a QLogic HBA connected to an FC JBOD any different

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread A Darren Dunham
On Tue, Sep 30, 2008 at 03:19:40PM -0700, Erik Trimble wrote: To make Will's argument more succinct (wink), with a NetApp, undetectable (by the NetApp) errors can be introduced at the HBA and transport layer (FC Switch, slightly damage cable) levels. ZFS will detect such errors, and fix

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 5:19 PM, Erik Trimble [EMAIL PROTECTED] wrote: To make Will's argument more succinct (wink), with a NetApp, undetectable (by the NetApp) errors can be introduced at the HBA and transport layer (FC Switch, slightly damage cable) levels. ZFS will detect such errors,

Re: [zfs-discuss] ZFS poor performance on Areca 1231ML

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 5:04 PM, Ross Becker [EMAIL PROTECTED]wrote: At this point, ZFS is performing admirably with the Areca card. Also, that card is only 8-port, and the Areca controllers I have are 12-port. My chassis has 24 SATA bays, so being able to cover all the drives with 2

Re: [zfs-discuss] zpool import of bootable root pool renders it unbootable

2008-09-30 Thread William Schumann
I have not tried importing bootable root pools onto other VMs, but there have been recent ZFS bug fixes in the area of importing and exporting bootable root pools - the panic might not occur on Solaris Nevada releases after approximately 97. There are still issues with renaming of bootable

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
Intel mainstream (and indeed many tech companies') stuff is purposely stratified from the enterprise stuff by cutting out features like ECC and higher memory capacity and using different interface form factors. Well I guess I am getting a Xeon anyway There is nothing magical about SAS

Re: [zfs-discuss] Segmentation fault / core dump with recursive send/recv

2008-09-30 Thread BJ Quinn
True, but a search for zfs segmentation fault returns 500 bugs. It's possible one of those is related to my issue, but it would take all day to find out. If it's not flaky or unstable, I'd like to try upgrading to the newest kernel first, unless my Linux mindset is truly out of place here, or

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 6:03 PM, Ahmed Kamal [EMAIL PROTECTED] wrote: Hmm ... well, there is a considerable price difference, so unless someone says I'm horribly mistaken, I now want to go back to Barracuda ES 1TB 7200 drives. By the way, how many of those would saturate a single (non

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Robert Thurlow
Ahmed Kamal wrote: BTW, for everyone saying zfs is more reliable because it's closer to the application than a netapp, well at least in my case it isn't. The solaris box will be NFS sharing and the apps will be running on remote Linux boxes. So, I guess this makes them equal. How about a

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Tue, 30 Sep 2008, Miles Nordin wrote: I think you need two zpools, or zpool + LVM2/XFS, some kind of two-filesystem setup, because of the ZFS corruption and panic/freeze-on-import problems. Having two zpools helps with other If ZFS provides such a terrible experience for you can I be

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, Ahmed Kamal wrote: So, I guess this makes them equal. How about a new reliable NFS protocol, that computes the hashes on the client side, sends it over the wire to be written remotely on the zfs storage node ?! Modern NFS runs over a TCP connection, which includes its own

Re: [zfs-discuss] ZSF Solaris

2008-09-30 Thread Nathan Kroenert
Actually, the one that'll hurt most is ironically the most closely related to bad database schema design... With a zillion files in the one directory, if someone does an 'ls' in that directory, it'll not only take ages, but steal a whole heap of memory and compute power... Provided the only

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Nicolas Williams
On Tue, Sep 30, 2008 at 06:09:30PM -0500, Tim wrote: On Tue, Sep 30, 2008 at 6:03 PM, Ahmed Kamal [EMAIL PROTECTED] wrote: BTW, for everyone saying zfs is more reliable because it's closer to the application than a netapp, well at least in my case it isn't. The solaris box will be NFS

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Miles Nordin
rt == Robert Thurlow [EMAIL PROTECTED] writes: rt introduces a separate protection domain for the NFS link. There are checksums in the ethernet FCS, checksums in IP headers, checksums in UDP headers (which are sometimes ignored), and checksums in TCP (which are not ignored). There might be

Re: [zfs-discuss] Segmentation fault / core dump with recursive send/recv

2008-09-30 Thread Richard Elling
BJ Quinn wrote: True, but a search for zfs segmentation fault returns 500 bugs. It's possible one of those is related to my issue, but it would take all day to find out. If it's not flaky or unstable, I'd like to try upgrading to the newest kernel first, unless my Linux mindset is truly

Re: [zfs-discuss] Segmentation fault / core dump with recursive send/recv

2008-09-30 Thread Bob Friesenhahn
On Tue, 30 Sep 2008, BJ Quinn wrote: True, but a search for zfs segmentation fault returns 500 bugs. It's possible one of those is related to my issue, but it would take all day to find out. If it's not flaky or unstable, I'd like to try upgrading to the newest kernel first, unless my

Re: [zfs-discuss] ZSF Solaris

2008-09-30 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, Nathan Kroenert wrote: zillion I/O's you need to deal with each time you list the entire directory. an ls -1rt on a directory with about 1.2 million files with names like afile1202899 takes minutes to complete on my box, and we see 'ls' get to in excess of 700MB rss...

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread David Magda
On Sep 30, 2008, at 19:44, Miles Nordin wrote: There are checksums in the ethernet FCS, checksums in IP headers, checksums in UDP headers (which are sometimes ignored), and checksums in TCP (which are not ignored). There might be an RPC layer checksum, too, not sure. Not of which helped

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread David Magda
On Sep 30, 2008, at 19:09, Tim wrote: SAS has far greater performance, and if your workload is extremely random, will have a longer MTBF. SATA drives suffer badly on random workloads. Well, if you can probably afford more SATA drives for the purchase price, you can put them in a

[zfs-discuss] ZFS Pool Question

2008-09-30 Thread Josh Hardman
Hello, I'm looking for info on adding a disk to my current zfs pool. I am running OpenSoarlis snv_98. I have upgraded my pool since my image-update. When I installed OpenSolaris it was a machine with 2 hard disks (regular IDE). Is it possible to add the second hard disk to the pool to

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 7:15 PM, David Magda [EMAIL PROTECTED] wrote: On Sep 30, 2008, at 19:09, Tim wrote: SAS has far greater performance, and if your workload is extremely random, will have a longer MTBF. SATA drives suffer badly on random workloads. Well, if you can probably afford

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
Well, if you can probably afford more SATA drives for the purchase price, you can put them in a striped-mirror set up, and that may help things. If your disks are cheap you can afford to buy more of them (space, heat, and power not withstanding). Hmm, that's actually cool ! If I configure

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Robert Thurlow
Bob Friesenhahn wrote: On Wed, 1 Oct 2008, Ahmed Kamal wrote: So, I guess this makes them equal. How about a new reliable NFS protocol, that computes the hashes on the client side, sends it over the wire to be written remotely on the zfs storage node ?! Modern NFS runs over a TCP

[zfs-discuss] Fwd: Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 7:30 PM, Ahmed Kamal [EMAIL PROTECTED] wrote: Well, if you can probably afford more SATA drives for the purchase price, you can put them in a striped-mirror set up, and that may help things. If your disks are cheap you can afford to buy more of them (space, heat,

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Robert Thurlow
Miles Nordin wrote: There are checksums in the ethernet FCS, checksums in IP headers, checksums in UDP headers (which are sometimes ignored), and checksums in TCP (which are not ignored). There might be an RPC layer checksum, too, not sure. Different arguments can be made against each, I

Re: [zfs-discuss] ZSF Solaris

2008-09-30 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, Nathan Kroenert wrote: That being said, there is a large delta in your results and mine... If I get a chance, I'll look into it... I suspect it's a cached versus I/O issue... The first time I posted was the first time the directory has been read in well over a month so

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Richard Elling
Tim wrote: On Tue, Sep 30, 2008 at 7:15 PM, David Magda [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: On Sep 30, 2008, at 19:09, Tim wrote: SAS has far greater performance, and if your workload is extremely random, will have a longer MTBF. SATA drives

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
I observe that there are no disk vendors supplying SATA disks with speed 7,200 rpm. It is no wonder that a 10k rpm disk outperforms a 7,200 rpm disk for random workloads. I'll attribute this to intentional market segmentation by the industry rather than a deficiency in the transfer

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Richard Elling
Ahmed Kamal wrote: I observe that there are no disk vendors supplying SATA disks with speed 7,200 rpm. It is no wonder that a 10k rpm disk outperforms a 7,200 rpm disk for random workloads. I'll attribute this to intentional market segmentation by the industry rather than

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
So, performance aside, does SAS have other benefits ? Data integrity ? How would a 8 raid1 sata compare vs another 8 smaller SAS disks in raidz(2) ? Like apples and pomegranates. Both should be able to saturate a GbE link. You're the expert, but isn't the 100M/s for streaming not random

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, Ahmed Kamal wrote: Always assuming 2 spare disks, and Using the sata disks, I would configure them in raid1 mirror (raid6 for the 400G), Besides being cheaper, I would get more useable space (4TB vs 2.4TB), Better performance of raid1 (right?), and better data reliability

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 8:13 PM, Ahmed Kamal [EMAIL PROTECTED] wrote: So, performance aside, does SAS have other benefits ? Data integrity ? How would a 8 raid1 sata compare vs another 8 smaller SAS disks in raidz(2) ? Like apples and pomegranates. Both should be able to saturate a GbE

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Tue, 30 Sep 2008, Robert Thurlow wrote: Modern NFS runs over a TCP connection, which includes its own data validation. This surely helps. Less than we'd sometimes like :-) The TCP checksum isn't very strong, and we've seen corruption tied to a broken router, where the Ethernet

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
Hm, richard's excellent Graphs here http://blogs.sun.com/relling/tags/mttdl as well as his words say he prefers mirroring over raidz/raidz2 almost always. It's better for performance and MTTDL. Since 8 sata raid1 is cheaper and probably more reliable than 8 raidz2 sas (and I dont need extra sas

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Toby Thain
On 30-Sep-08, at 6:31 PM, Tim wrote: On Tue, Sep 30, 2008 at 5:19 PM, Erik Trimble [EMAIL PROTECTED] wrote: To make Will's argument more succinct (wink), with a NetApp, undetectable (by the NetApp) errors can be introduced at the HBA and transport layer (FC Switch, slightly damage

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Miles Nordin
rt == Robert Thurlow [EMAIL PROTECTED] writes: dm == David Magda [EMAIL PROTECTED] writes: dm Not of which helped Amazon when their S3 service went down due dm to a flipped bit: ok, I get that S3 went down due to corruption, and that the network checksums I mentioned failed to prevent

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 8:50 PM, Toby Thain [EMAIL PROTECTED]wrote: * NetApp's block-appended checksum approach appears similar but is in fact much stronger. Like many arrays, NetApp formats its drives with 520-byte sectors. It then groups them into 8-sector blocks: 4K of data (the WAFL

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Richard Elling
Ahmed Kamal wrote: So, performance aside, does SAS have other benefits ? Data integrity ? How would a 8 raid1 sata compare vs another 8 smaller SAS disks in raidz(2) ? Like apples and pomegranates. Both should be able to saturate a GbE link. You're the expert, but

[zfs-discuss] ZFS, NFS and Auto Mounting

2008-09-30 Thread Douglas R. Jones
I am in the process of beefing up our development environment. In essence I am really going simply replicate what we have spread across here and there (that what happens when you keep running out of disk space). Unfortunately, I inherited all of this and the guy who dreamed up the conflagration

Re: [zfs-discuss] ZSF Solaris

2008-09-30 Thread Al Hopper
On Tue, Sep 30, 2008 at 6:30 PM, Nathan Kroenert [EMAIL PROTECTED] wrote: Actually, the one that'll hurt most is ironically the most closely related to bad database schema design... With a zillion files in the one directory, if someone does an 'ls' in that directory, it'll not only take ages,

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Toby Thain
On 30-Sep-08, at 9:54 PM, Tim wrote: On Tue, Sep 30, 2008 at 8:50 PM, Toby Thain [EMAIL PROTECTED] wrote: NetApp's block-appended checksum approach appears similar but is in fact much stronger. Like many arrays, NetApp formats its drives with 520-byte sectors. It then groups them

Re: [zfs-discuss] ZSF Solaris

2008-09-30 Thread Ian Collins
Bob Friesenhahn wrote: On Wed, 1 Oct 2008, Nathan Kroenert wrote: zillion I/O's you need to deal with each time you list the entire directory. an ls -1rt on a directory with about 1.2 million files with names like afile1202899 takes minutes to complete on my box, and we see 'ls' get to

Re: [zfs-discuss] ZFS Pool Question

2008-09-30 Thread Richard Elling
Josh Hardman wrote: Hello, I'm looking for info on adding a disk to my current zfs pool. I am running OpenSoarlis snv_98. I have upgraded my pool since my image-update. When I installed OpenSolaris it was a machine with 2 hard disks (regular IDE). Is it possible to add the second hard

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Nicolas Williams
On Tue, Sep 30, 2008 at 08:54:50PM -0500, Tim wrote: As it does in ANY fileserver scenario, INCLUDING zfs. He is building a FILESERVER. This is not an APPLICATION server. You seem to be stuck on this idea that everyone is using ZFS on the server they're running the application. That does a

Re: [zfs-discuss] ZSF Solaris

2008-09-30 Thread Jens Elkner
On Tue, Sep 30, 2008 at 09:44:21PM -0500, Al Hopper wrote: This behavior is common to tmpfs, UFS and I tested it on early ZFS releases. I have no idea why - I have not made the time to figure it out. What I have observed is that all operations on your (victim) test directory will max out

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ian Collins
Tim wrote: As it does in ANY fileserver scenario, INCLUDING zfs. He is building a FILESERVER. This is not an APPLICATION server. You seem to be stuck on this idea that everyone is using ZFS on the server they're running the application. That does a GREAT job of creating disparate storage

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 10:44 PM, Toby Thain [EMAIL PROTECTED]wrote: ZFS allows the architectural option of separate storage without losing end to end protection, so the distinction is still important. Of course this means ZFS itself runs on the application server, but so what? --Toby So

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Wed, Oct 1, 2008 at 12:24 AM, Ian Collins [EMAIL PROTECTED] wrote: Tim wrote: As it does in ANY fileserver scenario, INCLUDING zfs. He is building a FILESERVER. This is not an APPLICATION server. You seem to be stuck on this idea that everyone is using ZFS on the server they're

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 11:58 PM, Nicolas Williams [EMAIL PROTECTED] wrote: On Tue, Sep 30, 2008 at 08:54:50PM -0500, Tim wrote: As it does in ANY fileserver scenario, INCLUDING zfs. He is building a FILESERVER. This is not an APPLICATION server. You seem to be stuck on this idea that

Re: [zfs-discuss] ZSF Solaris

2008-09-30 Thread Ram Sharma
Hi Guys, Thanks for so many good comments. Perhaps I got even more than what I asked for! I am targeting 1 million users for my application.My DB will be on solaris machine.And the reason I am making one table per user is that it will be a simple design as compared to keeping all the data in