Re: [zfs-discuss] Recommendation for home NAS external JBOD
2012-06-20 0:32, Timothy Coalson wrote: when it has 4k sectors, then in theory if First sector is a multiple of 8, it is aligned, but it will probably issue writes of size 512 which will degrade performance anyway). I think this is dependent on the firmware (vendor), and queued writes into the same HW sector (4Kb) which came in as a series of 512 byte blocks should be recombined in HDD cache and written in one stroke. Thus it should not hurt performance - if write-caching is enabled, at least, and/or NCQ/TCQ support. That has its caveats - starting with bug-less-ness of TCQ/NCQ/RMW and caching, and on into power-failure support regarding caches (i.e. have an UPS or capacitors big enough to flush the disk's caches safely before parking the heads). Also by default if you don't give the whole drive to ZFS, its cache may be disabled upon pool import and you may have to reenable it manually (if you only actively use this disk for one or more ZFS pools - which play with caching nicely). HTH, //Jim Klimov ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On Jun 20, 2012, at 4:08 AM, Jim Klimov wrote: Also by default if you don't give the whole drive to ZFS, its cache may be disabled upon pool import and you may have to reenable it manually (if you only actively use this disk for one or more ZFS pools - which play with caching nicely). This is not correct. The behaviour is to attempt to enable the disk's write cache if ZFS has the whole disk. Relevant code: http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/vdev_disk.c#319 Please help us to stop propagating the misinformation that ZFS disables write caches. -- richard -- ZFS and performance consulting http://www.RichardElling.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
2012-06-21 1:58, Richard Elling wrote: On Jun 20, 2012, at 4:08 AM, Jim Klimov wrote: Also by default if you don't give the whole drive to ZFS, its cache may be disabled upon pool import and you may have to reenable it The behaviour is to attempt to enable the disk's write cache if ZFS has the whole disk. Relevant code: http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/vdev_disk.c#319 Please help us to stop propagating the misinformation that ZFS disables write caches. -- richard I see, sorry. So, the possible states are: 1) Before pool import, disk cache was disabled; then pool is imported: 1a) If ZFS has whole disk (how is that defined BTW, since partitions and slices are really used? Is the presence of a slice#7 which is 16384 sector long the trigger?) - then cache is enabled; 1b) ZFS does not have whole disk - cache is neither enabled nor disabled; 2) Before import disk cache was enabled; after import: no change regardless of whole-diskness. Is this correct? How does a disk become cache disabled then - only manually? Or due to UFS usage? Or does it inherit HW setting? Or somehow else? I think the cache is enabled in the OS by default... Thanks, //Jim Klimov ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On Jun 20, 2012, at 5:08 PM, Jim Klimov wrote: 2012-06-21 1:58, Richard Elling wrote: On Jun 20, 2012, at 4:08 AM, Jim Klimov wrote: Also by default if you don't give the whole drive to ZFS, its cache may be disabled upon pool import and you may have to reenable it The behaviour is to attempt to enable the disk's write cache if ZFS has the whole disk. Relevant code: http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/vdev_disk.c#319 Please help us to stop propagating the misinformation that ZFS disables write caches. -- richard I see, sorry. So, the possible states are: 1) Before pool import, disk cache was disabled; then pool is imported: 1a) If ZFS has whole disk (how is that defined BTW, since partitions and slices are really used? Is the presence of a slice#7 which is 16384 sector long the trigger?) - then cache is enabled; by the command use: zpool create c0t0d0 == whole disk zpool create c0t0d0s0 == not whole disk 1b) ZFS does not have whole disk - cache is neither enabled nor disabled; 2) Before import disk cache was enabled; after import: no change regardless of whole-diskness. correct Is this correct? How does a disk become cache disabled then - only manually? Or due to UFS usage? Or does it inherit HW setting? Or somehow else? For Sun, it was done by setting the disk firmware. I think the cache is enabled in the OS by default… In general, illumos does not touch the cache. I don't know of a way to set the cache policy in most BIOSes. In some cases, you can set it using format(1m), but whether it remains set after power-off depends on the drive manufacturer. Bottom line: don't worry about it. -- richard -- ZFS and performance consulting http://www.RichardElling.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
- Will I be able to buy a replacement in 3-3 years that reports the disk in such a way, that resilvering will work? According to the Advanced Format threat this seems to be a problem. I was hopimg to get arond this with these disks and have a more future proof solution I think that if you are running an illumos kernel, you can use /kernel/drv/sd.conf and tell it that the physical sectors for a disk model are 4k, despite what the disk says (and whether they really are). So, if you want an ashift=12 pool on disks that report 512 sectors, you should be able to do it now without a patched version of zpool. - Disk alignment: I am currently using whole disks AFAIK. But I do not remember. Did I use slicing etc? Is my alignment correct (btw. How do check?) So I thought: If I start over with a new pool O might get ot right and this seemed easier with those disks... Whole disk method is generally recommended, and should align if it gets the sector size right, the only time I have manually sliced was to overprovision an SSD. I think prtvtoc is all you need to determine if it is aligned, if the bytes/sector value under Dimensions is the true (physical) sector size, it should be aligned (if it reports 512 when it has 4k sectors, then in theory if First sector is a multiple of 8, it is aligned, but it will probably issue writes of size 512 which will degrade performance anyway). Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
Hi Timothy, I think that if you are running an illumos kernel, you can use /kernel/drv/sd.conf and tell it that the physical sectors for a disk model are 4k, despite what the disk says (and whether they really are). So, if you want an ashift=12 pool on disks that report 512 sectors, you should be able to do it now without a patched version of zpool. That refers to creating a new pool and is good to know. However I was more afraid about the comments in the Advanced Format threat stating that if you have an ashift=9 512b based pool and need to replace a drive, resilver might fail if you put in a 4K disk. Assuming that in 2-3 years you might not be able to get 512b disks in the size you need them anymore, this could be a serious problem. I think prtvtoc is all you need to determine if it is aligned, if the bytes/sector value under Dimensions is the true (physical) sector size, it should be aligned (if it reports 512 when it has 4k sectors, then in theory if First sector is a multiple of 8, it is aligned, but it will probably issue writes of size 512 which will degrade performance anyway). Thanks. I will note that down and check once the drives arrive. Kind regards, JP smime.p7s Description: S/MIME cryptographic signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
I think that if you are running an illumos kernel, you can use /kernel/drv/sd.conf That refers to creating a new pool and is good to know. Two things: one, it looks like you should also be able to trick it into using 512 sectors on a 4k disk, allowing you to do exactly such a replacement (incurring a similar write penalty to undetected 512 emulated sectors, but at least your pool won't be degraded, though regular scrubs may be more important), and two, a caveat: the current OpenIndiana oi_151a4 release doesn't seem to have a new enough version of illumos to support this yet, at least the man page for sd doesn't mention the needed tunable. Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
Hi Carson, I have 2 Sans Digital TR8X JBOD enclosures, and they work very well. They also make a 4-bay TR4X. http://www.sansdigital.com/towerraid/tr4xb.html http://www.sansdigital.com/towerraid/tr8xb.html looks nice! The only thing coming to mind is that according to the specifications the enclosure is 3Gbits only. If I choose to put in a SSD with 6Gbits this would be not optimal. I looked at their site but failed to find 6GBit enclosures. But I will keep looking since sooner or later they will provide it. I think I will go for the option of replacing the four drives for now with the Hitachi 3TB drives. This will give me 9TB net with RAID-Z1 level. I will calculate how expensive a 8bay enclosure with a LSI 8port external controller will be. Just in case the 9TB are not sufficient, I need a backup place or I decide to go for RAID-Z2. :-) They cost a bit more than the one you linked to, but the drives are hot swap. They also make similar cases with port multipliers, RAID, etc., but I've only used the JBOD. I will bookmark them. Then enclosures do look nice. Kind regards, JP smime.p7s Description: S/MIME cryptographic signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On Mon, Jun 18, 2012 at 2:19 PM, Koopmann, Jan-Peter jan-pe...@koopmann.eu wrote: Hi Carson, I have 2 Sans Digital TR8X JBOD enclosures, and they work very well. They also make a 4-bay TR4X. http://www.sansdigital.com/towerraid/tr4xb.html http://www.sansdigital.com/towerraid/tr8xb.html looks nice! The only thing coming to mind is that according to the specifications the enclosure is 3Gbits only. You mean http://www.sansdigital.com/towerraid-plus/index.php ? -- Fajar ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On Mon, 18 Jun 2012, Koopmann, Jan-Peter wrote: looks nice! The only thing coming to mind is that according to the specifications the enclosure is 3Gbits only. If I choose to put in a SSD with 6Gbits this would be not optimal. I looked at their site but failed to find 6GBit enclosures. But I will keep looking since sooner or later they will provide it. I browsed the site and saw many 6GBit enclosures. I also saw one with Nexenta (Solaris/zfs appliance) inside. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
Hi Bob, On Mon, 18 Jun 2012, Koopmann, Jan-Peter wrote: looks nice! The only thing coming to mind is that according to the specifications the enclosure is 3Gbits only. If I choose to put in a SSD with 6Gbits this would be not optimal. I looked at their site but failed to find 6GBit enclosures. But I will keep looking since sooner or later they will provide it. I browsed the site and saw many 6GBit enclosures. I also saw one with Nexenta (Solaris/zfs appliance) inside. I found several high end enclosures. Or ones with bundled RAID cards. But the equivalent of the one originally suggested I was not able to find. However after looking at tons of sites for hours I might simply have missed it. If you found one, can you please forward a link? Kind regards, JP smime.p7s Description: S/MIME cryptographic signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On Mon, 18 Jun 2012, Koopmann, Jan-Peter wrote: I browsed the site and saw many 6GBit enclosures. I also saw one with Nexenta (Solaris/zfs appliance) inside. I found several high end enclosures. Or ones with bundled RAID cards. But the equivalent of the one originally suggested I was not able to find. However after looking at tons of sites for hours I might simply have missed it. If you found one, can you please forward a link? So you want high-end performance at a low-end price? It seems unlikely that you will notice the difference between 3Gbit or 6Gbit for a home application. FLASH-based SSDs seem to burn-out pretty quickly if you don't use them carefully. The situation is getting worse rather than better over time as FLASH geometries get smaller and they try to store more bits in one cell. What was described as a bright new future is starting to look more like an end of the road to me. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On 6/18/12 12:19 AM, Koopmann, Jan-Peter wrote: Hi Carson, I have 2 Sans Digital TR8X JBOD enclosures, and they work very well. They also make a 4-bay TR4X. http://www.sansdigital.com/towerraid/tr4xb.html http://www.sansdigital.com/towerraid/tr8xb.html looks nice! The only thing coming to mind is that according to the specifications the enclosure is 3Gbits only. If I choose to put in a SSD with 6Gbits this would be not optimal. I looked at their site but failed to find 6GBit enclosures. But I will keep looking since sooner or later they will provide it. The JBOD enclosures are completely passive. I can't imagine any reason they wouldn't support 6Gbit SATA/SAS - there are no electronics in them, just wire routing. -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
Thanks. Just noticed that the Hitachi 3TB drives are not available. The 4TB ones are but with 512b emulated only. However I can get Barracudas 7200.14 with supposedly real 4k quite cheap. Anyone any experience with those? I might be getting one or two more and go for z2 instead of z1. I even found affordable passive enclosures available in Germany for very little money... The overall plan then would really be to switch to external JBODs and use the existing drives for backup only.. Kind regards, JP Am 19.06.2012 um 01:02 schrieb Carson Gaspar car...@taltos.org: On 6/18/12 12:19 AM, Koopmann, Jan-Peter wrote: Hi Carson, I have 2 Sans Digital TR8X JBOD enclosures, and they work very well. They also make a 4-bay TR4X. http://www.sansdigital.com/towerraid/tr4xb.html http://www.sansdigital.com/towerraid/tr8xb.html looks nice! The only thing coming to mind is that according to the specifications the enclosure is 3Gbits only. If I choose to put in a SSD with 6Gbits this would be not optimal. I looked at their site but failed to find 6GBit enclosures. But I will keep looking since sooner or later they will provide it. The JBOD enclosures are completely passive. I can't imagine any reason they wouldn't support 6Gbit SATA/SAS - there are no electronics in them, just wire routing. -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss smime.p7s Description: S/MIME cryptographic signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On 6/18/12 4:07 PM, Koopmann, Jan-Peter wrote: Thanks. Just noticed that the Hitachi 3TB drives are not available. The 4TB ones are but with 512b emulated only. However I can get Barracudas 7200.14 with supposedly real 4k quite cheap. Anyone any experience with those? I might be getting one or two more and go for z2 instead of z1. What makes you think the Barracuda 7200.14 drives report 4k sectors? I gave up looking for 4kn drives, as everything I could find was 512e. I would _love_ to be wrong, as I have 8 4TB Hitachis on backorder that I would gladly replace with 4kn drives, even if I had to drop to 3TB density. From page 11 of http://www.seagate.com/staticfiles/support/docs/manual/desktop/Barracuda%207200.14/100686584c.pdf Formatted capacity (512 bytes/sector)** Bytes per sector (4K physical emulated at 512-byte sectors) From the smartmontools list, http://www.mail-archive.com/smartmontools-database@lists.sourceforge.net/msg00537.html Sector Sizes: 512 bytes logical, 4096 bytes physical -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
What makes you think the Barracuda 7200.14 drives report 4k sectors? I gave up looking for 4kn drives, as everything I could find was 512e. I would _love_ to be wrong, as I have 8 4TB Hitachis on backorder that I would gladly replace with 4kn drives, even if I had to drop to 3TB density. I have a western digital drive with 4k physical sectors that causes OpenIndiana to use ashift=12 (and refuse to use it as a replacement in an ashift=9 vdev), though it appears to report 512 logical sectors, so somehow OpenIndiana noticed the real sector size. It is model WD30EURS (a green AV-GP drive). Is it close enough to what you want for solaris to know that it is really 4k, even if it is a 512e drive? Some smartctl info: Device Model: WDC WD30EURS-63R8UY0 ... Firmware Version: 80.00A80 ... Sector Sizes: 512 bytes logical, 4096 bytes physical hdparm -I (on linux) also figures out that the drive has 4k physical sectors. Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On Mon, Jun 18, 2012 at 5:26 PM, Carson Gaspar car...@taltos.org wrote: What makes you think the Barracuda 7200.14 drives report 4k sectors? I gave up looking for 4kn drives, as everything I could find was 512e. I would _love_ to be wrong, as I have 8 4TB Hitachis on backorder that I would gladly replace with 4kn drives, even if I had to drop to 3TB density. From page 11 of http://www.seagate.com/staticfiles/support/docs/manual/desktop/Barracuda%207200.14/100686584c.pdf Formatted capacity (512 bytes/sector)** Bytes per sector (4K physical emulated at 512-byte sectors) From the smartmontools list, http://www.mail-archive.com/smartmontools-database@lists.sourceforge.net/msg00537.html Sector Sizes: 512 bytes logical, 4096 bytes physical Hmm, interesting. Like Timothy noted. My $work is also running into the same problem the Seagate 7200.14 where an ashift=9 pool would not accept the 4k drive. If you check the archives for a Advanced Format HDD's - are we there yet? [1] thread from the end of May we discuss this same topic. [1] http://mail.opensolaris.org/pipermail/zfs-discuss/2012-May/051559.html ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On Mon, 18 Jun 2012, Carson Gaspar wrote: What makes you think the Barracuda 7200.14 drives report 4k sectors? I gave up looking for 4kn drives, as everything I could find was 512e. I would _love_ to be wrong, as I have 8 4TB Hitachis on backorder that I would gladly replace with 4kn drives, even if I had to drop to 3TB density. Why would you want native 4k drives right now? Not much would work with such drives. Maybe in a dedicated chassis (e.g. the JBOD) they could be of some use. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
What makes you think the Barracuda 7200.14 drives report 4k sectors? http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg48912.html Nigel stated this here a few days ago. I did not check for myself. Maybe Nigel can comment on this? As for the question why do you want 4k drives: My thinking - I will buy 4-6 disks now. - I must assume that during the next 3-4 years one of them might fail - Will I be able to buy a replacement in 3-3 years that reports the disk in such a way, that resilvering will work? According to the Advanced Format threat this seems to be a problem. I was hopimg to get arond this with these disks and have a more future proof solution Moreover: - If I buy new disks and a new JBOD etc. I might as well get a performant solution. In other threats ashift 9 vs 12 is presented as a problem. - Disk alignment: I am currently using whole disks AFAIK. But I do not remember. Did I use slicing etc? Is my alignment correct (btw. How do check?) So I thought: If I start over with a new pool O might get ot right and this seemed easier with those disks... Might be totally wrong withmmy assumptions and if so: Hey that's the reason for asking you, knowing I am not the expert myself. :-) Kind regards, JP smime.p7s Description: S/MIME cryptographic signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
So I can either exchange the disks one by one with autoexpand, use 2-4 TB disks and be happy. This was my original approach. However I am totally unclear about the 512b vs 4Kb issue. What sata disk could I use that is big enough and still uses 512b? I know about the discussion about the upgrade from a 512b based pool to a 4 KB pool but I fail to see a conclusion. Will the autoexpand mechanism upgrade ashift? And what disks do not lie? Is the performance impact significant? Replacing devices will not change the ashift, it is set permanently when a vdev is created, and zpool will refuse to replace a device in an ashift=9 vdev with a device that it would use ashift=12 on. Large Western Digital disks tend to say they have 4k sectors, and hence cannot be used to replace your current disks, while hitachi and seagate offer 512 emulated disks, which should allow you to replace your current disks without needing to copy the contents of the pool to a new one. If you don't have serious performance requirements, you may not notice the impact of emulated 512 sectors (especially since zfs buffers async writes into transaction groups). I did some rudimentary testing on a large pool of hitachi 3TB 512 emulated disks with ashift=9 vs ashift=12 with bonnie, and it didn't seem to matter a whole lot (though its possibly relevant tests were large writes, which have little penalty, and character at a time, which was bottlenecked by the cpu since the test was single threaded, so it didn't test the worst case). The worst case for 512 emulated sectors on zfs is probably small (4KB or so) synchronous writes (which if they mattered to you, you would probably have a separate log device, in which case the data disk write penalty may not matter). So I started to think about option 2. That would be using an external JOBD chassis (4-8 disks) and eSATA. But I would either need a JBOD with 4-8 eSATA connectors (which I am yet to find) or use a JBOD with a good expander. I see several cheap sata to esata jbod chassis making use of port multiplier. Is this referring to a expander backplane and will work with oi, LSI and mpt or mpt_sas? I'm wondering, based on the comment about routing 4 eSATA cables, what kind of options your NAS case has, if your LSI controller has SFF-8087 connectors (or possibly even if it doesn't), you might be able to use an adapter to the SFF-8088 external 4 lane SAS connector, which may increase your options. It seems that support for SATA port multiplier is not mandatory in a controller, so you will want to check with LSI before trying it (I would hope they support it on SAS controllers, since I think it is a vastly simplified version of SAS expanders). Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
2012-06-17 19:11, Koopmann, Jan-Peter пишет: Hi, my oi151 based home NAS is approaching a frightening drive space level. Right now the data volume is a 4*1TB Raid-Z1, 3 1/2 local disks individually connected to an 8 port LSI 6Gbit controller. So I can either exchange the disks one by one with autoexpand, use 2-4 TB disks and be happy. This was my original approach. However I am totally unclear about the 512b vs 4Kb issue. What sata disk could I use that is big enough and still uses 512b? I know about the discussion about the upgrade from a 512b based pool to a 4 KB pool but I fail to see a conclusion. Will the autoexpand mechanism upgrade ashift? And what disks do not lie? Is the performance impact significant? AFAIK the Hitachi Desk/Ultra-Star (5K3000, 7K3000) should be 512b native, maybe the only ones at this size. Larger 4TB Hitachi models are 4KB native, 512e emulated - according to datasheets on site. HTH, //Jim Klimov ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On Sun, Jun 17, 2012 at 03:19:18PM -0500, Timothy Coalson wrote: Replacing devices will not change the ashift, it is set permanently when a vdev is created, and zpool will refuse to replace a device in an ashift=9 vdev with a device that it would use ashift=12 on. Yep. [..] while hitachi and seagate offer 512 emulated disks I did some rudimentary testing on a large pool of hitachi 3TB 512 emulated disks with ashift=9 vs ashift=12 with bonnie, and it didn't seem to matter a whole lot Hitachi are native 512-byte sectors. At least, the 5k3000 and 7k3000 are, in the 2T and 3T sizes. I haven't noticed if they have a newer model which is 4k native. How long that continues to remain the case, and how long these models continue to remain available (e.g. for replacements) is entirely another matter. The replacement applies even to under-warranty cases; I know someone who recently had a 4k-only drive supplied as a warranty replacement for a 512 native drive (not, in this case, from Hitachi). As for performance, at least in my experience with WD disks emulating 512-byte sectors, you *will* notice the difference; heavy metadata updates being the most obvious impact. The conclusion is that unless your environment is well controlled, the time has probably come where new general-purpose pools should be made at ashift=12, to allow future flexibility. I'm wondering, based on the comment about routing 4 eSATA cables, what kind of options your NAS case has, if your LSI controller has SFF-8087 connectors (or possibly even if it doesn't), you might be able to use an adapter to the SFF-8088 external 4 lane SAS connector, which may increase your options. It seems that support for SATA port multiplier is not mandatory in a controller, so you will want to check with LSI before trying it (I would hope they support it on SAS controllers, since I think it is a vastly simplified version of SAS expanders). SATA port-multipliers and SAS expanders are not related in any sense of common driver support; they're similar only in general concept. Do not conflate them. -- Dan. pgpzXJn70kX7n.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
Hi Tim, thanks to you and the others for answering. worst case). The worst case for 512 emulated sectors on zfs is probably small (4KB or so) synchronous writes (which if they mattered to you, you would probably have a separate log device, in which case the data disk write penalty may not matter). Good to know. This really opens up the possibility of buying 3 or 4TB Hitachi drives. At least the 4TB Hitachi drives are 4k (512b emulated) drives according to the latest news. I'm wondering, based on the comment about routing 4 eSATA cables, what kind of options your NAS case has, if your LSI controller has SFF-8087 connectors (or possibly even if it doesn't), It has actually. you might be able to use an adapter to the SFF-8088 external 4 lane SAS connector, which may increase your options. So what you are saying is that something like this will do the trick? http://www.pc-pitstop.com/sata_enclosures/scsat44xb.asp If I interpret this correctly I get a SFF-8087 to SFF-8088 bracket, connect the 4 port LSI SFF-8077 to that bracket, then get a cable for this JBOD and throw in 4 drives? This would leave me with four additional HDDs without any SAS expander hassle. I had not come across these JBODs. Thanks a million for the hint. Do we agree that for a home NAS box a Hitachi Deskstar (not explicitly being a server SATA drive) will suffice despite potential TLER problems? I was thinking about Hitachi Deskstar 5k3000 drives. The 4TB seemingly came out but are rather expensive in comparison Kind regards, JP smime.p7s Description: S/MIME cryptographic signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On 6/17/12 3:21 PM, Koopmann, Jan-Peter wrote: Hi Tim, you might be able to use an adapter to the SFF-8088 external 4 lane SAS connector, which may increase your options. So what you are saying is that something like this will do the trick? http://www.pc-pitstop.com/sata_enclosures/scsat44xb.asp If I interpret this correctly I get a SFF-8087 to SFF-8088 bracket, connect the 4 port LSI SFF-8077 to that bracket, then get a cable for this JBOD and throw in 4 drives? This would leave me with four additional HDDs without any SAS expander hassle. I had not come across these JBODs. Thanks a million for the hint. I have 2 Sans Digital TR8X JBOD enclosures, and they work very well. They also make a 4-bay TR4X. http://www.sansdigital.com/towerraid/tr4xb.html http://www.sansdigital.com/towerraid/tr8xb.html They cost a bit more than the one you linked to, but the drives are hot swap. They also make similar cases with port multipliers, RAID, etc., but I've only used the JBOD. -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
worst case). The worst case for 512 emulated sectors on zfs is probably small (4KB or so) synchronous writes (which if they mattered to you, you would probably have a separate log device, in which case the data disk write penalty may not matter). Good to know. This really opens up the possibility of buying 3 or 4TB Hitachi drives. At least the 4TB Hitachi drives are 4k (512b emulated) drives according to the latest news. It appears from the specs listed on the hitachi site that the drives I have may actually be 512 native, in which case my testing was moot. This does explain some other things I saw testing the drives in question, so I will assume they are 512 native, and that my testing was meaningless. If you copy folders containing thousands of small files frequently, the performance impact may be relevant, if you go for the 512 emulated drives. So what you are saying is that something like this will do the trick? http://www.pc-pitstop.com/sata_enclosures/scsat44xb.asp If I interpret this correctly I get a SFF-8087 to SFF-8088 bracket, connect the 4 port LSI SFF-8077 to that bracket, then get a cable for this JBOD and throw in 4 drives? This would leave me with four additional HDDs without any SAS expander hassle. I had not come across these JBODs. Thanks a million for the hint. No problem, and yes, I think that should work. One thing to keep in mind, though, is that if the internals of the enclosure simply split the multilane SAS cable into 4 connectors without an expander, and you use SATA drives, the controller will use SATA mode, which as I understand it runs at a lower signalling voltage, and won't work over long cables, so get a short cable (1 meter, shorter if you can find one). It looks like all of the ones mentioned so far use this method, though it would be good to know if Carson populated his with SATA drives. Do we agree that for a home NAS box a Hitachi Deskstar (not explicitly being a server SATA drive) will suffice despite potential TLER problems? I was thinking about Hitachi Deskstar 5k3000 drives. The 4TB seemingly came out but are rather expensive in comparison… I'm not sure what ZFS's timeout for dropping an unresponsive disk is, or what it does when it responds again, so I don't know if TLER would help. I have not had any serious problems with my pool of hitachi 3TB 5400 drives. Two different drives had a checksum error, once each, but stayed online in the pool. Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation for home NAS external JBOD
On 6/17/12 6:36 PM, Timothy Coalson wrote: No problem, and yes, I think that should work. One thing to keep in mind, though, is that if the internals of the enclosure simply split the multilane SAS cable into 4 connectors without an expander, and you use SATA drives, the controller will use SATA mode, which as I understand it runs at a lower signalling voltage, and won't work over long cables, so get a short cable (1 meter, shorter if you can find one). It looks like all of the ones mentioned so far use this method, though it would be good to know if Carson populated his with SATA drives. SATA drives using 1m cables from an LSI SAS9201-16e -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
