Re: [zfs-discuss] file concatenation with ZFS copy-on-write

2010-06-15 Thread Jim Klimov
Hi all I wonder if there has been any new development on this matter over the past 6 months. Today i pondered an idea of zfs-aware mv, capable of doing zero read/write of file data when moving files between datasets of one pool. This seems like a (z)cp idea proposed in this thread and seems

Re: [zfs-discuss] file concatenation with ZFS copy-on-write

2010-06-15 Thread Jim Klimov
Hi all I wonder if there has been any new development on this matter over the past 6 months. Today i pondered an idea of zfs-aware mv, capable of doing zero read/write of file data when moving files between datasets of one pool. This seems like a (z)cp idea proposed in this thread and seems

Re: [zfs-discuss] Native ZFS for Linux

2010-06-15 Thread Joerg Schilling
Peter Jeremy peter.jer...@alcatel-lucent.com wrote: On 2010-Jun-11 17:41:38 +0800, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: PP.S.: Did you know that FreeBSD _includes_ the GPLd Reiserfs in the FreeBSD kernel since a while and that nobody did complain about this, see e.g.:

[zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Arve Paalsrud
Hi, We are currently building a storage box based on OpenSolaris/Nexenta using ZFS. Our hardware specifications are as follows: Quad AMD G34 12-core 2.3 GHz (~110 GHz) 10 Crucial RealSSD (6Gb/s) 42 WD RAID Ed. 4 2TB disks + 6Gb/s SAS expanders LSI2008SAS (two 4x ports) Mellanox InfiniBand 40

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Erik Trimble
On 6/15/2010 4:42 AM, Arve Paalsrud wrote: Hi, We are currently building a storage box based on OpenSolaris/Nexenta using ZFS. Our hardware specifications are as follows: Quad AMD G34 12-core 2.3 GHz (~110 GHz) 10 Crucial RealSSD (6Gb/s) 42 WD RAID Ed. 4 2TB disks + 6Gb/s SAS expanders

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Darren J Moffat
On 15/06/2010 14:09, Erik Trimble wrote: I'm going to say something sacrilegious here: 128GB of RAM may be overkill. You have the SSDs for L2ARC - much of which will be the DDT, The point of L2ARC is that you start adding L2ARC when you can no longer physically put in (or afford) to add any

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Erik Trimble
On 6/15/2010 6:17 AM, Darren J Moffat wrote: On 15/06/2010 14:09, Erik Trimble wrote: I'm going to say something sacrilegious here: 128GB of RAM may be overkill. You have the SSDs for L2ARC - much of which will be the DDT, The point of L2ARC is that you start adding L2ARC when you can no

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Roy Sigurd Karlsbakk
I'm going to say something sacrilegious here: 128GB of RAM may be overkill. You have the SSDs for L2ARC - much of which will be the DDT, but, if I'm reading this correctly, even if you switch to the 160GB Intel X25-M, that give you 8 x 160GB = 1280GB of L2ARC, of which only half is in-use by

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Erik Trimble
On 6/15/2010 6:40 AM, Roy Sigurd Karlsbakk wrote: I'm going to say something sacrilegious here: 128GB of RAM may be overkill. You have the SSDs for L2ARC - much of which will be the DDT, but, if I'm reading this correctly, even if you switch to the 160GB Intel X25-M, that give you 8 x 160GB =

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Richard Elling
On Jun 15, 2010, at 6:40 AM, Roy Sigurd Karlsbakk wrote: I'm going to say something sacrilegious here: 128GB of RAM may be overkill. You have the SSDs for L2ARC - much of which will be the DDT, but, if I'm reading this correctly, even if you switch to the 160GB Intel X25-M, that give you 8 x

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Richard Elling
On Jun 15, 2010, at 4:42 AM, Arve Paalsrud wrote: Hi, We are currently building a storage box based on OpenSolaris/Nexenta using ZFS. Our hardware specifications are as follows: Quad AMD G34 12-core 2.3 GHz (~110 GHz) 10 Crucial RealSSD (6Gb/s) 42 WD RAID Ed. 4 2TB disks + 6Gb/s SAS

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Garrett D'Amore
On Tue, 2010-06-15 at 04:42 -0700, Arve Paalsrud wrote: Hi, We are currently building a storage box based on OpenSolaris/Nexenta using ZFS. Our hardware specifications are as follows: Quad AMD G34 12-core 2.3 GHz (~110 GHz) 10 Crucial RealSSD (6Gb/s) 42 WD RAID Ed. 4 2TB disks + 6Gb/s

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Garrett D'Amore
On Tue, 2010-06-15 at 07:36 -0700, Richard Elling wrote: What I want to achieve is 2 GB/s+ NFS traffic against our ESX clusters (also InfiniBand-based), with both dedupe and compression enabled in ZFS. In general, both dedup and compression gain space by trading off performance. You

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Arve Paalsrud
On Tue, Jun 15, 2010 at 3:09 PM, Erik Trimble erik.trim...@oracle.comwrote: On 6/15/2010 4:42 AM, Arve Paalsrud wrote: Hi, We are currently building a storage box based on OpenSolaris/Nexenta using ZFS. Our hardware specifications are as follows: Quad AMD G34 12-core 2.3 GHz (~110 GHz)

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Arve Paalsrud
On Tue, Jun 15, 2010 at 4:20 PM, Erik Trimble erik.trim...@oracle.comwrote: On 6/15/2010 6:57 AM, Arve Paalsrud wrote: On Tue, Jun 15, 2010 at 3:09 PM, Erik Trimble erik.trim...@oracle.comwrote: I'd go with 2 Intel X25-E 32GB models for ZIL. Mirror them - striping isn't really going to buy

[zfs-discuss] Dedup... still in beta status

2010-06-15 Thread Fco Javier Garcia
Data: 90% of current computers has less than 9 GB of RAM, less than 5% has SSD systems. Let use a computer storage standard, with a capacity of 4 TB ... dedupe on, dataset with blocks of 32 kb ..., 2 TB of data in use ... need 16 GB of memory just only for DTT ... but this will not see it until

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Arve Paalsrud
On Tue, Jun 15, 2010 at 3:33 PM, Erik Trimble erik.trim...@oracle.comwrote: On 6/15/2010 6:17 AM, Darren J Moffat wrote: On 15/06/2010 14:09, Erik Trimble wrote: I'm going to say something sacrilegious here: 128GB of RAM may be overkill. You have the SSDs for L2ARC - much of which will be

[zfs-discuss] OCZ Devena line of enterprise SSD

2010-06-15 Thread Roger Hernandez
OCZ has a new line of enterprise SSDs, based on the SandForce 1500 controller. These three have a supercapacitor: OCZ Deneva Reliability 2.5 MLC SSDhttp://www.oczenterprise.com/products/details/ocz-deneva-reliability-2-5-mlc-ssd.html OCZ Deneva Reliability 2.5 SLC

Re: [zfs-discuss] COMSTAR dropouts with dedup enabled

2010-06-15 Thread Matthew Anderson
Thanks Brandon, This system has 24GB of RAM and currently no L2ARC. The total de duplicated data was about 250GB so I wouldn't have thought I would be out of RAM, I've removed the LUN for the time being so I can't get the DDR size at the moment. I have some X25-E's to go in as L2ARC and SLOG

Re: [zfs-discuss] ZFS Data Loss on system crash/upgrade

2010-06-15 Thread Cindy Swearingen
Hi Austin, No much help, as it turns out. I don't see any evidence that a recovery mechanism, where you might lose a few seconds of data transactions, was triggered. It almost sounds like your file system was rolled back to a previous snapshot because the data is lost as of a certain date. I

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Arve Paalsrud
-Original Message- From: Garrett D'Amore [mailto:garr...@nexenta.com] Sent: 15. juni 2010 17:43 To: Arve Paalsrud Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] High-Performance ZFS (2000MB/s+) On Tue, 2010-06-15 at 04:42 -0700, Arve Paalsrud wrote: Hi, We are

[zfs-discuss] zpool export / import discrepancy

2010-06-15 Thread Scott Squires
Hello All, I've migrated a JBOD of 16 drives from one server to another. I did a zpool export from the old system and a zpool import to the new system. One thing I did notice is since the drives are on a different controller card, the naming is different (as expected) but the order is also

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Garrett D'Amore
On Tue, 2010-06-15 at 18:33 +0200, Arve Paalsrud wrote: What about the ZIL bandwidth in this case? I mean, could I stripe across multiple devices to be able to handle higher throughput? Otherwise I would still be limited to the performance of the unit itself (155 MB/s). I think so.

[zfs-discuss] NexentaStor Community edition 3.0.3 released

2010-06-15 Thread Anil Gulecha
Hi All, On behalf of NexentaStor team, I'm happy to announce the release of NexentaStor Community Edition 3.0.3. This release is the result of the community efforts of Nexenta Partners and users. Changes over 3.0.2 include * Many fixes to ON/ZFS backported to b134. * Multiple bug fixes in the

Re: [zfs-discuss] OCZ Devena line of enterprise SSD

2010-06-15 Thread Brandon High
On Mon, Jun 14, 2010 at 2:07 PM, Roger Hernandez rhvar...@gmail.com wrote: OCZ has a new line of enterprise SSDs, based on the SandForce 1500 controller. The SLC based drive should be great as a ZIL, and the MLC drives should be a close second. Neither is cost effective as a L2ARC, since the

Re: [zfs-discuss] Dedup... still in beta status

2010-06-15 Thread Erik Trimble
On 6/15/2010 9:03 AM, Fco Javier Garcia wrote: Data: 90% of current computers has less than 9 GB of RAM, less than 5% has SSD systems. Let use a computer storage standard, with a capacity of 4 TB ... dedupe on, dataset with blocks of 32 kb ..., 2 TB of data in use ... need 16 GB of memory just

[zfs-discuss] SSDs adequate ZIL devices?

2010-06-15 Thread Arne Jansen
There has been many threads in the past asking about ZIL devices. Most of them end up in recommending Intel X-25 as an adequate device. Nevertheless there is always the warning about them not heeding cache flushes. But what use is a ZIL that ignores cache flushes? If I'm willing to tolerate that

Re: [zfs-discuss] Dedup... still in beta status

2010-06-15 Thread Fco Javier Garcia
Realistically, I think people are overtly-enamored with dedup as a feature - I would generally only consider it worth-while in cases where you get significant savings. And by significant, I'm talking an order of magnitude space savings. A 2x savings isn't really enough to counteract

Re: [zfs-discuss] Dedup... still in beta status

2010-06-15 Thread Erik Trimble
On 6/15/2010 10:52 AM, Erik Trimble wrote: Frankly, dedup isn't practical for anything but enterprise-class machines. It's certainly not practical for desktops or anything remotely low-end. This isn't just a ZFS issue - all implementations I've seen so far require enterprise-class

Re: [zfs-discuss] Dedup... still in beta status

2010-06-15 Thread Geoff Nordli
From: Fco Javier Garcia Sent: Tuesday, June 15, 2010 11:21 AM Realistically, I think people are overtly-enamored with dedup as a feature - I would generally only consider it worth-while in cases where you get significant savings. And by significant, I'm talking an order of magnitude space

Re: [zfs-discuss] Dedup... still in beta status

2010-06-15 Thread Fco Javier Garcia
or as a member of the ZFS team (which I'm not). Then you have to be brutally good with Java -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA ___ zfs-discuss mailing list

Re: [zfs-discuss] Dedup... still in beta status

2010-06-15 Thread Erik Trimble
On 6/15/2010 11:49 AM, Geoff Nordli wrote: From: Fco Javier Garcia Sent: Tuesday, June 15, 2010 11:21 AM Realistically, I think people are overtly-enamored with dedup as a feature - I would generally only consider it worth-while in cases where you get significant savings. And by

Re: [zfs-discuss] Dedup... still in beta status

2010-06-15 Thread Erik Trimble
On 6/15/2010 11:53 AM, Fco Javier Garcia wrote: or as a member of the ZFS team (which I'm not). Then you have to be brutally good with Java Thanks, but I do get it wrong every so often (hopefully, rarely). More importantly, I don't know anything about the internal goings-on

[zfs-discuss] Complete Linux Noob

2010-06-15 Thread CarlPalmer
I have been researching different types of raids, and I happened across raidz, and I am blown away. I have been trying to find resources to answer some of my questions, but many of them are either over my head in terms of details, or foreign to me as I am a linux noob, and I have to admit I

Re: [zfs-discuss] Complete Linux Noob

2010-06-15 Thread Roy Sigurd Karlsbakk
snip/ How do I share these types of raid pools across the network. Or more specifically, how do I access them from Windows based systems? Is there any special trick? Most of your questions are answered here http://hub.opensolaris.org/bin/download/Community+Group+zfs/docs/zfslast.pdf

Re: [zfs-discuss] Native ZFS for Linux

2010-06-15 Thread Bob Friesenhahn
On Tue, 15 Jun 2010, Joerg Schilling wrote: Sorry but your reply is completely misleading as the people who claim that there is a legal problem with having ZFS in the Linux kernel would of course also claim that Reiserfs cannot be in the FreeBSD kernel. It seems that it is a license violation

Re: [zfs-discuss] OCZ Devena line of enterprise SSD

2010-06-15 Thread Scott Meilicke
Price? I cannot find it. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Native ZFS for Linux

2010-06-15 Thread Joerg Schilling
Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Tue, 15 Jun 2010, Joerg Schilling wrote: Sorry but your reply is completely misleading as the people who claim that there is a legal problem with having ZFS in the Linux kernel would of course also claim that Reiserfs cannot be in

Re: [zfs-discuss] Complete Linux Noob

2010-06-15 Thread David Dyer-Bennet
On Tue, June 15, 2010 14:13, CarlPalmer wrote: I have been researching different types of raids, and I happened across raidz, and I am blown away. I have been trying to find resources to answer some of my questions, but many of them are either over my head in terms of details, or foreign to

Re: [zfs-discuss] zpool export / import discrepancy

2010-06-15 Thread Giovanni Tirloni
On Tue, Jun 15, 2010 at 1:56 PM, Scott Squires ssqui...@gmail.com wrote: Is ZFS dependent on the order of the drives?  Will this cause any issue down the road?  Thank you all; No. In your case the logical names changed but ZFS managed to order the disks correctly as they were before. --

Re: [zfs-discuss] SSDs adequate ZIL devices?

2010-06-15 Thread Bob Friesenhahn
On Tue, 15 Jun 2010, Arne Jansen wrote: In case of a power failure I will likely lose about as many writes as I do with SSDs, a few milliseconds. I agree with your concerns, but the data loss may span as much as 30 seconds rather than just a few milliseconds. Using an SSD as the ZIL allows

Re: [zfs-discuss] zfs-discuss Digest, Vol 56, Issue 78

2010-06-15 Thread Nikos George
the storage (files, shares, block devices, etc). -- Freddie Cash fjwc...@gmail.com -- next part -- An HTML attachment was scrubbed... URL: http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100615/767d6c7d/attachment-0001.html

Re: [zfs-discuss] ZFS Data Loss on system crash/upgrade

2010-06-15 Thread Austin Rotondo
Cindy, I've attached the results of the fmdump -eV command. They don't tell me anything, but someone more knowledgeable might be able to decipher it. As for snapshots, the earliest one I have is from after the new hardware. It doesn't appear that there are any snapshots from before the

Re: [zfs-discuss] SSDs adequate ZIL devices?

2010-06-15 Thread Christopher George
So why buy SSD for ZIL at all? For the record, not all SSDs ignore cache flushes. There are at least two SSDs sold today that guarantee synchronous write semantics; the Sun/Oracle LogZilla and the DDRdrive X1. Also, I believe it is more accurate to describe the root cause as not power

Re: [zfs-discuss] High-Performance ZFS (2000MB/s+)

2010-06-15 Thread Przemyslaw Ceglowski
On 15/06/2010 12:42, Arve Paalsrud arve.paals...@gmail.com wrote: Hi, We are currently building a storage box based on OpenSolaris/Nexenta using ZFS. Our hardware specifications are as follows: Quad AMD G34 12-core 2.3 GHz (~110 GHz) 10 Crucial RealSSD (6Gb/s) 42 WD RAID Ed. 4 2TB

Re: [zfs-discuss] SSDs adequate ZIL devices?

2010-06-15 Thread Przemyslaw Ceglowski
On 15/06/2010 23:46, Christopher George cgeo...@ddrdrive.com wrote: So why buy SSD for ZIL at all? For the record, not all SSDs ignore cache flushes. There are at least two SSDs sold today that guarantee synchronous write semantics; the Sun/Oracle LogZilla and the DDRdrive X1. Also, I

Re: [zfs-discuss] zpool export / import discrepancy

2010-06-15 Thread Frank Contrepois
http://blogs.sun.com/constantin/entry/csi_munich_how_to_save 2010/6/15 Scott Squires ssqui...@gmail.com Hello All, I've migrated a JBOD of 16 drives from one server to another. I did a zpool export from the old system and a zpool import to the new system. One thing I did notice is since

Re: [zfs-discuss] Dedup... still in beta status

2010-06-15 Thread Bill Sommerfeld
On 06/15/10 10:52, Erik Trimble wrote: Frankly, dedup isn't practical for anything but enterprise-class machines. It's certainly not practical for desktops or anything remotely low-end. We're certainly learning a lot about how zfs dedup behaves in practice. I've enabled dedup on two desktops

Re: [zfs-discuss] Dedup... still in beta status

2010-06-15 Thread David Magda
On Jun 15, 2010, at 14:20, Fco Javier Garcia wrote: I think dedup may have its greatest appeal in VDI environments (think about a environment with 85% if the data that the virtual machine needs is into ARC or L2ARC... is like a dream...almost instantaneous response... and you can boot a

Re: [zfs-discuss] Dedup... still in beta status

2010-06-15 Thread Mike Gerdts
On Tue, Jun 15, 2010 at 7:28 PM, David Magda dma...@ee.ryerson.ca wrote: On Jun 15, 2010, at 14:20, Fco Javier Garcia wrote: I think dedup may have its greatest appeal in VDI environments (think about a environment with 85% if the data that the virtual machine needs is into ARC or L2ARC... is

Re: [zfs-discuss] size of slog device

2010-06-15 Thread Edward Ned Harvey
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Bob Friesenhahn It is good to keep in mind that only small writes go to the dedicated slog. Large writes to to main store. A succession of that many small writes (to fill RAM/2) is highly

Re: [zfs-discuss] size of slog device

2010-06-15 Thread Richard Elling
On Jun 15, 2010, at 8:13 PM, Edward Ned Harvey wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Bob Friesenhahn It is good to keep in mind that only small writes go to the dedicated slog. Large writes to to main store. A

Re: [zfs-discuss] SSDs adequate ZIL devices?

2010-06-15 Thread Arne Jansen
Bob Friesenhahn wrote: On Tue, 15 Jun 2010, Arne Jansen wrote: In case of a power failure I will likely lose about as many writes as I do with SSDs, a few milliseconds. I agree with your concerns, but the data loss may span as much as 30 seconds rather than just a few milliseconds. Wait,

Re: [zfs-discuss] size of slog device

2010-06-15 Thread Richard Elling
On Jun 15, 2010, at 8:51 PM, Richard Elling wrote I thought, if you didn't explicitly tune these, all sync writes go to ZIL before the main store. Can't seem to find any way to verify this. Cake. All sync writes go to the ZIL. The ZIL may be in the pool or in the separate log device :-)