Re: [zfs-discuss] TLER and ZFS
As a home user, here are my thoughts. WD = ignore (TLER issues, parking issues, etc) I recently built up a server on Osol running Samsung 1.5TB drives. They are green, but don't seem to have the irritating features found on the WD green drives. They are 5400RPM, but seem to transfer data plenty fast for a home setup. Current setup is 2x6-disk raidz2. Seek times obviously hurt, and ZIL caused so many issues that I turned it off. Yes, I know I might lose some data doing that, yes, I'm OK with the tradeoff. The ZFS devs say I won't lose filesystem consistency, just that the write cache could be lost, about 30sec of data in most cases. As it's on a UPS and the rest of the network isn't, or is on small UPSes, it will be the last box online, so any clients will probably have their data saved before the server goes down. The next upgrade is a UPS that can tell the server power is out so it can shut down gracefully. I'll probably get an SSD for slog/l2arc at some point and re-enable ZIL, but for now, this does the job as SSDs that don't have similar issues when used as slog devices are rare and expensive. If the X25e won't do... This setup with the 5400 RPM drives is significantly faster than the same box with 7200RPM Seagate 400G drives was. Of course, those 400G drives are a few years old now, but I was pleasantly surprised by the speed I get out of the Samsungs. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
Changing the sector size (if it's possible at all) would require a reformat of the drive. The WD drives only support a 4K sector but they pretend to have 512byte sectors. I don't think they need to format the drive when changing to 4K sectors. A non-aligned write requires a read-modify-write operation and that makes the file slower. Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
This would require a low-level re-format and would significantly reduce the available space if it was possible at all. I don't think it is possible. WD has a jumper, but is there explicitly to work with WindowsXP, and is not a real way to dumb down the drive to 512. All it does is offset the sector numbers by 1 so that sector 63 becomes physical sector 64 (a multiple of 4KB). Is that all? And this forces 4K alignment? I would presume that any vendor that is shipping 4K sector size drives now, with a jumper to make it 'real' 512, would be supporting that over the long run? I would be very surprised if any vendor shipped a drive that could be jumpered to real 512 bytes. The best you are going to get is jumpered to logical 512 bytes and maybe a 1-sector offset (needed for WindozeXP only). These jumpers will probably last as long as the 8GB jumpers that were needed by old BIOS code. (Eg BIOS boots using simulated 512-byte sectors and then the OS tells the drive to switch to native mode). I would assume that such a jumper would change the drive from 4K native to pretend to be have 512 byte sectors/ It's unfortunate that Sun didn't bite the bullet several decades ago and provide support for block sizes other than 512-bytes instead of getting custom firmware for their CD drives to make them provide 512-byte logical blocks for 2KB CD-ROMs. Since Solaris x86 works fine with standard CD/DVD drives, that is no longer an issue. Solaris does support larger sectors. It's even more idiotic of WD to sell a drive with 4KB sectors but not provide any way for an OS to identify those drives and perform 4KB aligned I/O. I'm not sure that that is correct; the drive works on naive clients but I believe it can reveal its true colors. Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
Can you give us release numbers that confirm that this is 'automatic'. It is my understanding that the last available public release of OpenSolaris does not do this. On Oct 5, 2010, at 8:52 PM, Richard Elling wrote: ZFS already aligns the beginning of data areas to 4KB offsets from the label. For modern OpenSolaris and Solaris implementations, the default starting block for partitions is also aligned to 4KB. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
If you're spending upwards of $30,000 on a storage system, you probably shouldn't skimp on the most important component. You might as well be complaining that ECC ram costs more. Don't be ridiculous. For one, this is a disk backup system, not a fileserver, and TLER is far from as critic al as ECC. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
On Tue, Oct 5, 2010 at 11:49 PM, casper@sun.com wrote: I'm not sure that that is correct; the drive works on naive clients but I believe it can reveal its true colors. The drive reports 512 byte sectors to all hosts. AFAIK there's no way to make it report 4k sectors. -B -- Brandon High : bh...@freaks.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
On Tue, Oct 5, 2010 at 11:49 PM, casper@sun.com wrote: I'm not sure that that is correct; the drive works on naive clients but I believe it can reveal its true colors. The drive reports 512 byte sectors to all hosts. AFAIK there's no way to make it report 4k sectors. Too bad because it makes it less useful (specifically because the label mentions sectors and if you can use bigger sectors, you can address a larger drive). They still have all sizes w/o Advanced Format (non EARS/AARS models) Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
casper@sun.com wrote: On Tue, Oct 5, 2010 at 11:49 PM, casper@sun.com wrote: I'm not sure that that is correct; the drive works on naive clients but I believe it can reveal its true colors. The drive reports 512 byte sectors to all hosts. AFAIK there's no way to make it report 4k sectors. Too bad because it makes it less useful (specifically because the label mentions sectors and if you can use bigger sectors, you can address a larger drive). Having now read a number of forums about these, there's a strong feeling WD screwed up by not providing a switch to disable pseudo 512b access so you can use the 4k native. The industry as a whole will transition to 4k sectorsize over next few years, but these first 4k sectorsize HDs are rather less useful with 4k sectorsize-aware OS's. Let's hope other manufacturers get this right in their first 4k products. -- Andrew Gabriel ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
www.solarisinternals.com has always been a community. It never was hosted by Sun, and it's not hosted by Oracle. True, many of the contributors were Sun employees, but not so many remain at Oracle. If it's out if date, I suspect that's because the original contributors are too busy doing other fun things. However, it is a wiki, so YOU can apply for a login and edit it if you have something useful to share :) On 6 Oct 2010, at 02:36, Michael DeMan sola...@deman.com wrote: Hi upfront, and thanks for the valuable information. On Oct 5, 2010, at 4:12 PM, Peter Jeremy wrote: Another annoying thing with the whole 4K sector size, is what happens when you need to replace drives next year, or the year after? About the only mitigation needed is to ensure that any partitioning is based on multiples of 4KB. I agree, but to be quite honest, I have no clue how to do this with ZFS. It seems that it should be something under the regular tuning documenation. http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide Is it going to be the case that basic information like about how to deal with common scenarios like this is no longer going to be publicly available, and Oracle will simply keep it 'close to the vest', with the relevant information simply available for those who choose to research it themselves, or only available to those with certain levels of support contracts from Oracle? To put it another way - does the community that uses ZFS need to fork 'ZFS Best Practices' and 'ZFZ Evil Tuning' to ensure that it is reasonably up to date? Sorry for the somewhat hostile in the above, but the changes w/ the merger have demoralized a lot of folks I think. - Mike ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
On Tue, October 5, 2010 17:20, Richard Elling wrote: On Oct 5, 2010, at 2:06 PM, Michael DeMan wrote: On Oct 5, 2010, at 1:47 PM, Roy Sigurd Karlsbakk wrote: Well, here it's about 60% up and for 150 drives, that makes a wee difference... Understood on 1.6 times cost, especially for quantity 150 drives. One service outage will consume far more in person-hours and downtime than this little bit of money. Penny-wise == Pound-foolish? That looks to be true, yes (going back to the actual prices, 150 drives would cost $6000 extra for the enterprise versions). It's still quite annoying to be jerked around by people charging 60% extra for changing a timeout in the firmware, and carefully making it NOT user-alterable. Also, the non-TLER versions are a constant threat to anybody running home systems, who might quite reasonably think they could put those in a home server. (Yeah, I know the enterprise versions have other differences. I'm not nearly so sure I CARE about the other differences, in the size servers I'm working with.) -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
On Tue, October 5, 2010 16:47, casper@sun.com wrote: My immediate reaction to this is time to avoid WD drives for a while; until things shake out and we know what's what reliably. But, um, what do we know about say the Seagate Barracuda 7200.12 ($70), the SAMSUNG Spinpoint F3 1TB ($75), or the HITACHI Deskstar 1TB 3.5 ($70)? I've seen several important features when selecting a drive for a mirror: TLER (the ability of the drive to timeout a command) I went and got what detailed documentation I could on a couple of the Seagate drives last night, and I couldn't find anything on how they behaved in that sort of error cases. (I believe TLER is a WD-specific term, but I didn't just search, I read them through.) So that's inconvenient. How do we find out about that sort of thing? sector size (native vs virtual) Richard Elling said ZFS handles the 4k real 512byte fake drives okay now in default setups; but somebody immediately asked for version info, so I'm still watching this one. power use (specifically at home) Hadn't thought about that. But when I'm upgrading drives, I figure I'm always going to come out better on power than when I started. performance (mostly for work) I can't bring myself to buy below 7200RPM, but it's probably foolish (except that other obnoxious features tend to come in the green drives). price Yeah, well. I'm cheap. I've heard scary stories about a mismatch of the native sector size and unaligned Solaris partitions (4K sectors, unaligned cylinder). So have I. Sounds like you get read-modify-write actions for non-aligned accesses. I hope the next generation of drives admit to being 4k sectors, and that ZFS will be prepared to use them sensibly. But I'm not sure I'm willing to wait for that; the oldest drives in my box are now 4 years old, and I'm about ready for the next capacity upgrade. I was pretty happen with the WD drives (except for the one with a seriously broken cache) but I see the reasons to not to pick WD drives over the 1TB range. And the big ones are what pretty much everybody is using at home. Capacity and price are vastly more important than performance for most of us. -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
TLER (the ability of the drive to timeout a command) I went and got what detailed documentation I could on a couple of the Seagate drives last night, and I couldn't find anything on how they behaved in that sort of error cases. (I believe TLER is a WD-specific term, but I didn't just search, I read them through.) So that's inconvenient. How do we find out about that sort of thing? From http://en.wikipedia.org/wiki/TLER Similar technologies are called Error Recovery Control (ERC), used by competitor Seagate, and Command Completion Time Limit (CCTL), used by Samsung and Hitachi. I haven't checked which drives have those abilities, though... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
- Original Message - On Tue, October 5, 2010 17:20, Richard Elling wrote: On Oct 5, 2010, at 2:06 PM, Michael DeMan wrote: On Oct 5, 2010, at 1:47 PM, Roy Sigurd Karlsbakk wrote: Well, here it's about 60% up and for 150 drives, that makes a wee difference... Understood on 1.6 times cost, especially for quantity 150 drives. One service outage will consume far more in person-hours and downtime than this little bit of money. Penny-wise == Pound-foolish? That looks to be true, yes (going back to the actual prices, 150 drives would cost $6000 extra for the enterprise versions). I somehow doubt a service outage will consume that lot. The drives will be carefully distributed in smallish RAIDz2 VDEVs on two separate large systems and one small one, and all of them are dedicated for backup targets (Bacula using their drives for storing backup). We already have a 50TB setup on mostly Green drives, and although I now know that's a terrible idea, it's been running stably for about a year with quite constant load. So really, I beleive the chance for non-TLER drives to mess this up badly is a minor one (and perhaps more importantly, so does my boss). Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
Hi all I just discovered WD Black drives are rumored not to be set to allow TLER. Yep: http://opensolaris.org/jive/message.jspa?messageID=501159#501159 Enterprise drives will cost about 60% more, and on a large install, that means a lot of money... True, sometimes more than twice the price. If these are for a business, personally I would invest in TLER-capable drives like the WD REx models (RAID Edition). These allow for fast fails on read/write errors so that the data can be remapped. This prevents the possibility of the drive being kicked from the array. If these are for home and you don't have, or are not willing to spend a lot more on TLER-capable drives then go for something reliable. Forget WD Green drives (see links below). After WD removed TLER-setting on their non-enterprise drives, I have switched to Samsung HD203WI drives and so far these have been flawless. I believe it's a 4-platter model. Samsung have very recently (last month?) brought out a HD204UI model which is a 3-platter (667GB per platter) model, which should be even better -- check the newegg ratings for good/bad news etc. http://opensolaris.org/jive/thread.jspa?threadID=121871tstart=0 http://breden.org.uk/2009/05/01/home-fileserver-a-year-in-zfs/#drives http://jmlittle.blogspot.com/2010/03/wd-caviar-green-drives-and-zfs.html Cheers, Simon -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
ag == Andrew Gabriel andrew.gabr...@oracle.com writes: ag Having now read a number of forums about these, there's a ag strong feeling WD screwed up by not providing a switch to ag disable pseudo 512b access so you can use the 4k native. this reporting lie is no different from SSD's which have 2 - 8 kB sectors on the inside and benefit from alignment. I think probably everything will report 512 byte sectors forever. If a device had a 4224-byte sector, it would make sense to report that, but I don't see a big downside to reporting 512 when it's really 4096. NAND flash often does have sectors with odd sizes like 4224, and (some of) Linux's NAND-friendly filesystems (ubifs, yaffs, nilfs) use this OOB area for filesystem structures, which are intermixed with the ECC. but in that case it's not a SCSI interface to the odd-sized sector---it's an ``mtd'' interface that supports operations like ``erase page'', ``suspend erasing'', ``erase some more''. that said I am in the ``ignore WD for now'' camp. but this isn't why. Ignore them (among other, better reasons) because they have 4k sectors at all which don't yet work well until we can teach ZFS to never write smaller than 4kB. but failure to report 4k as SCSI 4kB sector is not a problem, to my view. You can just align your partitions. pgp6jwIDoUJ9i.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
dd == David Dyer-Bennet d...@dd-b.net writes: dd Richard Elling said ZFS handles the 4k real 512byte fake dd drives okay now in default setups There are two steps to handling it well. one is to align the start of partitions to 4kB, and apparently on Solaris (thanks to all the cumbersome partitioning tools) that is done. On Linux you often have to really pay attention to make this happen, depending on the partitioning tool that happens to be built into your ``distro'' or whatever. The second step is to never write anything smaller than 4kB. ex., if you want to write 0.5kB, pad it with 3.5kB of zeroes to avoid the read-modify-write penalty. AIUI that is not done yet, and zfs does sometimes want to write 0.5kB. When it's writing 128kB of course there is no penalty. For this, I think XFS and NTFS are actually better and tend not to write the small blocks, but I could be wrong. pgpn3kSSlfThy.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] TLER and ZFS
Hi all I just discovered WD Black drives are rumored not to be set to allow TLER. Does anyone know how much performance impact the lack of TLER might have on a large pool? Choosing Enterprise drives will cost about 60% more, and on a large install, that means a lot of money... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
I'm not sure on the TLER issues by themselves, but after the nightmares I have gone through dealing with the 'green drives', which have both the TLER issue and the IntelliPower head parking issues, I would just stay away from it all entirely and pay extra for the 'RAID Editiion' drives. Just out of curiosity, I took a peek a newegg. Western Digital RE3 WD1002FBYS 1TB 7200 RPM SATA 3.0Gb/s 3.5 Internal Hard Drive -Bare Drive are only $129. vs. $89 for the 'regular' black drives. 45% higher price, but it is my understanding that the 'RAID Edition' ones also are physically constructed for longer life, lower vibration levels, etc. On Oct 5, 2010, at 1:30 PM, Roy Sigurd Karlsbakk wrote: Hi all I just discovered WD Black drives are rumored not to be set to allow TLER. Does anyone know how much performance impact the lack of TLER might have on a large pool? Choosing Enterprise drives will cost about 60% more, and on a large install, that means a lot of money... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
Western Digital RE3 WD1002FBYS 1TB 7200 RPM SATA 3.0Gb/s 3.5 Internal Hard Drive -Bare Drive are only $129. vs. $89 for the 'regular' black drives. 45% higher price, but it is my understanding that the 'RAID Edition' ones also are physically constructed for longer life, lower vibration levels, etc. Well, here it's about 60% up and for 150 drives, that makes a wee difference... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
On Oct 5, 2010, at 1:47 PM, Roy Sigurd Karlsbakk wrote: Western Digital RE3 WD1002FBYS 1TB 7200 RPM SATA 3.0Gb/s 3.5 Internal Hard Drive -Bare Drive are only $129. vs. $89 for the 'regular' black drives. 45% higher price, but it is my understanding that the 'RAID Edition' ones also are physically constructed for longer life, lower vibration levels, etc. Well, here it's about 60% up and for 150 drives, that makes a wee difference... Vennlige hilsener / Best regards roy Understood on 1.6 times cost, especially for quantity 150 drives. I think (and if I am wrong, somebody else correct me) - that if you are using commodity controllers, which seems to generally fine for ZFS, then if a drive times out trying to constantly re-read a bad sector, it could stall out the read on the entire pool overall. On the other hand, if the drives are exported as JBOD from a RAID controller, I would think the RAID controller itself would just mark the drive as bad and offline it quickly based on its own internal algorithms. The above would also be relevant to the anticipated usage. For instance, if it is some sort of backup machine and delays due to some reads stalling on out TLER then perhaps it is not a big deal. If it is for more of an up-front production use, that could be intolerable. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
On Tue, October 5, 2010 15:30, Roy Sigurd Karlsbakk wrote: I just discovered WD Black drives are rumored not to be set to allow TLER. Does anyone know how much performance impact the lack of TLER might have on a large pool? Choosing Enterprise drives will cost about 60% more, and on a large install, that means a lot of money... My immediate reaction to this is time to avoid WD drives for a while; until things shake out and we know what's what reliably. But, um, what do we know about say the Seagate Barracuda 7200.12 ($70), the SAMSUNG Spinpoint F3 1TB ($75), or the HITACHI Deskstar 1TB 3.5 ($70)? This is not a completely theoretical question to me; it's getting on towards time to at least consider replacing my oldest mirrored pair; those are 400GB Seagate, I think, dating from 2006. I'd want something at least twice as big (to make the space upgrade worthwhile), and I'm expecting to buy three of them rather than just two because I think it's time to add a hot spare to the system (currently 3 pair of data disks, and I've got two more bays; I think a hot spare is a better use for them than a fourth pair; safety of the data is very important, performance is adequate, and I need a modest capacity upgrade, but the whole pool is currently 1.2TB usable, not large). On the third hand, there's the Barracuda 7200.11 1.5TB for only $75, which is a really small price increment for a big space increment. The WD RE3 1TB is $130 (all these prices are from Newegg just now). That's very close to TWICE the price of the competing 1TB drives. -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
On Tue, Oct 5, 2010 at 3:47 PM, Roy Sigurd Karlsbakk r...@karlsbakk.netwrote: Western Digital RE3 WD1002FBYS 1TB 7200 RPM SATA 3.0Gb/s 3.5 Internal Hard Drive -Bare Drive are only $129. vs. $89 for the 'regular' black drives. 45% higher price, but it is my understanding that the 'RAID Edition' ones also are physically constructed for longer life, lower vibration levels, etc. Well, here it's about 60% up and for 150 drives, that makes a wee difference... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ If you're spending upwards of $30,000 on a storage system, you probably shouldn't skimp on the most important component. You might as well be complaining that ECC ram costs more. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
My immediate reaction to this is time to avoid WD drives for a while; until things shake out and we know what's what reliably. But, um, what do we know about say the Seagate Barracuda 7200.12 ($70), the SAMSUNG Spinpoint F3 1TB ($75), or the HITACHI Deskstar 1TB 3.5 ($70)? I've seen several important features when selecting a drive for a mirror: TLER (the ability of the drive to timeout a command) sector size (native vs virtual) power use (specifically at home) performance (mostly for work) price I've heard scary stories about a mismatch of the native sector size and unaligned Solaris partitions (4K sectors, unaligned cylinder). I was pretty happen with the WD drives (except for the one with a seriously broken cache) but I see the reasons to not to pick WD drives over the 1TB range. Are people now using 4K native sectors and formating them with 4K sectors in (Open)Solaris? Performance sucks when you use unaligned accesses but is performance good when the performance is aligned? Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
On Oct 5, 2010, at 2:47 PM, casper@sun.com wrote: I've seen several important features when selecting a drive for a mirror: TLER (the ability of the drive to timeout a command) sector size (native vs virtual) power use (specifically at home) performance (mostly for work) price I've heard scary stories about a mismatch of the native sector size and unaligned Solaris partitions (4K sectors, unaligned cylinder). Yes, avoiding the 4K sector sizes is a huge issue right now too - another item I forgot on the reasons to absolutely avoid those WD 'green' drives. Three good reasons to avoid WD 'green' drives for ZFS... - TLER issues - IntelliPower head park issues - 4K sector size issues ...they are an absolutely nightmare. The WD 1TB 'enterprise' drives are still 512 sector size and safe to use, who knows though, maybe they just started shipping with 4K sector size as I write this e-mail? Another annoying thing with the whole 4K sector size, is what happens when you need to replace drives next year, or the year after? That part has me worried on this whole 4K sector migration thing more than what to buy today. Given the choice, I would prefer to buy 4K sector size now, but operating system support is still limited. Does anybody know if there any vendors that are shipping 4K sector drives that have a jumper option to make them 512 size? WD has a jumper, but is there explicitly to work with WindowsXP, and is not a real way to dumb down the drive to 512. I would presume that any vendor that is shipping 4K sector size drives now, with a jumper to make it 'real' 512, would be supporting that over the long run? I would be interested, and probably others would too, on what the original poster finally decides on this? - Mike ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
On Oct 5, 2010, at 2:06 PM, Michael DeMan wrote: On Oct 5, 2010, at 1:47 PM, Roy Sigurd Karlsbakk wrote: Western Digital RE3 WD1002FBYS 1TB 7200 RPM SATA 3.0Gb/s 3.5 Internal Hard Drive -Bare Drive are only $129. vs. $89 for the 'regular' black drives. 45% higher price, but it is my understanding that the 'RAID Edition' ones also are physically constructed for longer life, lower vibration levels, etc. Well, here it's about 60% up and for 150 drives, that makes a wee difference... Vennlige hilsener / Best regards roy Understood on 1.6 times cost, especially for quantity 150 drives. One service outage will consume far more in person-hours and downtime than this little bit of money. Penny-wise == Pound-foolish? -- richard -- OpenStorage Summit, October 25-27, Palo Alto, CA http://nexenta-summit2010.eventbrite.com ZFS and performance consulting http://www.RichardElling.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
Michael DeMan wrote: The WD 1TB 'enterprise' drives are still 512 sector size and safe to use, who knows though, maybe they just started shipping with 4K sector size as I write this e-mail? Another annoying thing with the whole 4K sector size, is what happens when you need to replace drives next year, or the year after? That part has me worried on this whole 4K sector migration thing more than what to buy today. Given the choice, I would prefer to buy 4K sector size now, but operating system support is still limited. Does anybody know if there any vendors that are shipping 4K sector drives that have a jumper option to make them 512 size? WD has a jumper, but is there explicitly to work with WindowsXP, and is not a real way to dumb down the drive to 512. I would presume that any vendor that is shipping 4K sector size drives now, with a jumper to make it 'real' 512, would be supporting that over the long run? Changing the sector size (if it's possible at all) would require a reformat of the drive. On SCSI disks which support it, you do it by changing the sector size on the relevant mode select page, and then sending a format-unit command to make the drive relayout all the sectors. I've no idea if these 4K sata drives have any such mechanism, but I would expect they would. BTW, I've been using a pair of 1TB Hitachi Ultrastar for something like 18 months without any problems at all. Of course, a 1 year old disk model is no longer available now. I'm going to have to swap out for bigger disks in the not too distant future. -- Andrew Gabriel ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
On 2010-Oct-06 05:59:06 +0800, Michael DeMan sola...@deman.com wrote: Another annoying thing with the whole 4K sector size, is what happens when you need to replace drives next year, or the year after? About the only mitigation needed is to ensure that any partitioning is based on multiples of 4KB. Does anybody know if there any vendors that are shipping 4K sector drives that have a jumper option to make them 512 size? This would require a low-level re-format and would significantly reduce the available space if it was possible at all. WD has a jumper, but is there explicitly to work with WindowsXP, and is not a real way to dumb down the drive to 512. All it does is offset the sector numbers by 1 so that sector 63 becomes physical sector 64 (a multiple of 4KB). I would presume that any vendor that is shipping 4K sector size drives now, with a jumper to make it 'real' 512, would be supporting that over the long run? I would be very surprised if any vendor shipped a drive that could be jumpered to real 512 bytes. The best you are going to get is jumpered to logical 512 bytes and maybe a 1-sector offset (needed for WindozeXP only). These jumpers will probably last as long as the 8GB jumpers that were needed by old BIOS code. (Eg BIOS boots using simulated 512-byte sectors and then the OS tells the drive to switch to native mode). It's unfortunate that Sun didn't bite the bullet several decades ago and provide support for block sizes other than 512-bytes instead of getting custom firmware for their CD drives to make them provide 512-byte logical blocks for 2KB CD-ROMs. It's even more idiotic of WD to sell a drive with 4KB sectors but not provide any way for an OS to identify those drives and perform 4KB aligned I/O. -- Peter Jeremy ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
Hi upfront, and thanks for the valuable information. On Oct 5, 2010, at 4:12 PM, Peter Jeremy wrote: Another annoying thing with the whole 4K sector size, is what happens when you need to replace drives next year, or the year after? About the only mitigation needed is to ensure that any partitioning is based on multiples of 4KB. I agree, but to be quite honest, I have no clue how to do this with ZFS. It seems that it should be something under the regular tuning documenation. http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide Is it going to be the case that basic information like about how to deal with common scenarios like this is no longer going to be publicly available, and Oracle will simply keep it 'close to the vest', with the relevant information simply available for those who choose to research it themselves, or only available to those with certain levels of support contracts from Oracle? To put it another way - does the community that uses ZFS need to fork 'ZFS Best Practices' and 'ZFZ Evil Tuning' to ensure that it is reasonably up to date? Sorry for the somewhat hostile in the above, but the changes w/ the merger have demoralized a lot of folks I think. - Mike ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] TLER and ZFS
ZFS already aligns the beginning of data areas to 4KB offsets from the label. For modern OpenSolaris and Solaris implementations, the default starting block for partitions is also aligned to 4KB. On Oct 5, 2010, at 6:36 PM, Michael DeMan wrote: Hi upfront, and thanks for the valuable information. On Oct 5, 2010, at 4:12 PM, Peter Jeremy wrote: Another annoying thing with the whole 4K sector size, is what happens when you need to replace drives next year, or the year after? About the only mitigation needed is to ensure that any partitioning is based on multiples of 4KB. I agree, but to be quite honest, I have no clue how to do this with ZFS. It seems that it should be something under the regular tuning documenation. Disagree. Starting alignment is not a problem OOB. You have to go out of your way to make the starting alignments not be 4KB aligned. http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide Is it going to be the case that basic information like about how to deal with common scenarios like this is no longer going to be publicly available, and Oracle will simply keep it 'close to the vest', with the relevant information simply available for those who choose to research it themselves, or only available to those with certain levels of support contracts from Oracle? To put it another way - does the community that uses ZFS need to fork 'ZFS Best Practices' and 'ZFZ Evil Tuning' to ensure that it is reasonably up to date? ZFS Best Practices and Evil Tuning Guide are not hosted by Oracle. They are hosted at the SolarisInternals.com site. -- richard -- OpenStorage Summit, October 25-27, Palo Alto, CA http://nexenta-summit2010.eventbrite.com ZFS and performance consulting http://www.RichardElling.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
