Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Sun, 29 Apr 2012, Enrico wrote: thank you Andy for this info let us know your experience when you'll setup-up the disks I didn't have any problems at all when I did this. I added the three 4 TB disks to the system alongside the 250 GB disk it was supplied with and booted OI 151a off a USB DVD drive after first editing the grub boot stanza to replace the $ISADIR variable with 'amd64', as OI doesn't auto-detect the fact that this microserver is fitted with a 64-bit CPU. At the start of the live image installation, OI correctly detects all 4 disks but reports that only 2 TB of the 4 TB disks is usable if they are used as boot disks. So I installed to the 250 GB disk and then created the pool with: zpool create lapback1 raidz c2t1d0 c2t2d0 c2t3d0 zfs create lapback1/backups It checks out OK: andy@lapback1:~# zfs list NAME USED AVAIL REFER MOUNTPOINT lapback1 890K 7.13T 202K /lapback1 lapback1/backups 192K 7.13T 192K /lapback1/backups rpool 4.90G 223G 46.5K /rpool rpool/ROOT 2.93G 223G31K legacy rpool/ROOT/openindiana 2.93G 223G 2.91G / rpool/dump 959M 223G 959M - rpool/export34.2M 223G32K /export rpool/export/home 34.2M 223G32K /export/home rpool/export/home/andy 34.2M 223G 34.2M /export/home/andy rpool/swap 1020M 224G 137M - and: andy@lapback1:~# iostat -en errors --- s/w h/w trn tot device 0 0 0 0 c2t0d0 0 0 0 0 c2t1d0 0 0 0 0 c2t2d0 0 0 0 0 c2t3d0 I will be using these servers at a university for automated backups of user laptops (Windows, Macs, Linux and the odd FreeBSD user) so that 5 minutes after a laptop connects to the network, it either backs up incrementally or syncs the laptop with the server using rsync. (The existing service runs on a Sun E450 running Open Solaris 134 SPARC and fitted with 19 x 300 GB disks in three ZFS pools but we now need more storage space and 300 GB is the limit for conventional SCSI disks). I will post any problems I have with this in 'production' here. Andy On 04/29/2012 09:24 AM, andy thomas wrote: On Sat, 28 Apr 2012, Enrico wrote: did a little research, just because i'm curious about this, my storage needs are much lower :) they say (http://forums.overclockers.com.au/showpost.php?p=13963218postcount=4727) that SB820M south-bridge on n40l has a limit of 3.2tb for hdd capacity. but there are no such details on the official specs: support.amd.com/us/Embedded_TechDocs/47283.pdf i hope this helps a bit A comment from a poster called 'AFireInside' on this same forum says AMD confirms that 4 TB disks will work in the N40L Microserver: http://forums.overclockers.com.au/showthread.php?s=a97e85a928a1e6997245557220f4b729t=1009531page=2 Also, HP's own specifications confirm that this unit is suitable for 4 TB disks: http://h18000.www1.hp.com/products/quickspecs/13716_div/13716_div.html A lot of the comments saying otherwise seem to relate to the earlier N36L model, earlier BIOS versions and booting Windows from disks larger than 3.2 TB. As for using Advanced Format disks with 4K sectors in ZFS pools, the worst case scenario would be that these disks can be used in a backwards-compatible legacy 512 byte sector format but with slower r/w performance. Also, Hitachi provide a Windows utility to re-align 4k sectors on their drives although I'm not sure whether this would suit OI/ZFS. Finally, it looks like OI has some support for 4k sector disks: http://wiki.openindiana.org/pages/viewpage.action?pageId=4883847 and ZFS pools can be created with the 'zpool create block-size 4096 ...' option. I'll take the plunge and order a N40L Microserver plus 3 x 4 TB disks and I'll post my experiences here. cheers, Andy On 04/28/2012 08:39 PM, Enrico wrote: Hi roy, yes, right, it was just to tell Andy what I've tested with this hp system and OI. i think there are many people who have written in detail their experience about supported hdd sizes in forums easily reachable just by googling n40l. let me also correct what i've written before: the adaptec 1045/1405 are luckily supported by the aac driver, despite what is written on the man page. the si3124 driver refers to a sata controller i've tested successfully on the pciex1 slot. Best regards, Enrico On 04/28/2012 06:13 PM, Roy Sigurd Karlsbakk wrote: It probably won't help much. Any SATA controller should recognize the 4TB drives, and the problem with the drives lying about sector sizes won't be any better whatever controller you use. roy - Opprinnelig melding - maybe this might help: http://www.adaptec.com/en-us/products/controllers/hardware/sas/entry/asc-1045/ it is
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
There are two issues. The first is correct partition alignment, the second ashift value. In theory, I haven't tested this yet, manually creating the slices with a start position to sector 64 and using slices instead of whole disks for the zpool devices, and creating with an ashift of 12 may produce the desired outcome. I have used 4k disks (wd20ears) in 3 and 4 disk raidz pools, but they are used for archiving, so just have data dumped to them. Few issues on Sata ports, but dodgy on SAS. Mark On 1/05/2012 8:23 a.m., Peter Wood wrote: I'm building a storage server with Dell MD1000 DAS and I just bought 30 drives with 4K sectors. One of the reasons I selected the new 4K sector is so I can easily find replacement drives 2-3 years from now when they start failing. Looks like this was a huge mistake. I'm fine if the drives report 512B sectors and work in slower legacy mode as long as they work reliable but seems that this may not be the case. On top of that my internal drives that make the rpool have 512B sectors so I'm not sure how workarounds will effect this pool. Is it fair to say that if one uses 4K drives he will run into alignment issue sooner or later? I'm really puzzled what to do here. Should I try to replace the drives with 512B ones before the storage goes life? Any thoughts? Thank you Peter On Mon, Apr 30, 2012 at 7:54 AM, Richard Elling richard.ell...@richardelling.com wrote: On Apr 29, 2012, at 7:38 PM, Gordon Ross wrote: On Sun, Apr 29, 2012 at 8:46 PM, Richard Elling richard.ell...@richardelling.com wrote: On Apr 29, 2012, at 11:45 AM, George Wilson wrote: [...] Speaking of 4K sectors, I've taken a slightly different approach that fixes this outside of ZFS. The idea is to allow sd to override the physical-block-size which ZFS will pick up. The way this works is you can specify the Vendor/Product id in sd.conf. Here's an example: sd-config-list = NETAPP LUN, physical-block-size:4096; This is the preferred solution and there are several implementations running around in various stages of test/release/acceptance. I look forward to getting this upstream :-) -- richard Providing a work-around in sd is great. We should do that, at least. But is it sufficient? What happens if I replace a mirrored drive with 512 byte sectors with one having 4k sectors? What if I want to plan ahead for that? Maybe in only some of my ZFS pools but not all? It would seem that a pool-level override for ashift might also be useful. ashift is set for the top-level vdev at creation time. So you have to override prior to creation of the mirror. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On May 2, 2012, at 12:25 AM, Mark wrote: There are two issues. The first is correct partition alignment, the second ashift value. In theory, I haven't tested this yet, manually creating the slices with a start position to sector 64 and using slices instead of whole disks for the zpool devices, and creating with an ashift of 12 may produce the desired outcome. Default starting offset for slice 0 is sector 256 for 512-byte sector disks. You shouldn't have to manually touch these unless you are running an ancient (circa 2006-2007) version of Solaris where the default offset was 34. I have used 4k disks (wd20ears) in 3 and 4 disk raidz pools, but they are used for archiving, so just have data dumped to them. Few issues on Sata ports, but dodgy on SAS. That is not encouraging. OTOH, I've never had good experiences with WD drives :-( Unfortunately, for those who purchase from Dell or HP, you are often stuck with WD. At least HP does a decent job of putting up firmware updates. If you want a fright, look at HPs fixes for WD firmware :-( -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Wed, May 2, 2012 at 2:47 PM, Richard Elling richard.ell...@richardelling.com wrote: On May 2, 2012, at 12:25 AM, Mark wrote: There are two issues. The first is correct partition alignment, the second ashift value. In theory, I haven't tested this yet, manually creating the slices with a start position to sector 64 and using slices instead of whole disks for the zpool devices, and creating with an ashift of 12 may produce the desired outcome. Default starting offset for slice 0 is sector 256 for 512-byte sector disks. You shouldn't have to manually touch these unless you are running an ancient (circa 2006-2007) version of Solaris where the default offset was 34. Yes, but be warned: the format command will try to make fdisk partitions that are aligned on cyninder boundaries, which are usually odd numbers of sectors. I have a fix for that here: http://yalms.org/cr/zfs-blksize/ (the first three files. ignore the zfs part for now) Gordon ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On 3/05/2012 7:07 a.m., Gordon Ross wrote: On Wed, May 2, 2012 at 2:47 PM, Richard Elling richard.ell...@richardelling.com wrote: On May 2, 2012, at 12:25 AM, Mark wrote: There are two issues. The first is correct partition alignment, the second ashift value. In theory, I haven't tested this yet, manually creating the slices with a start position to sector 64 and using slices instead of whole disks for the zpool devices, and creating with an ashift of 12 may produce the desired outcome. Default starting offset for slice 0 is sector 256 for 512-byte sector disks. You shouldn't have to manually touch these unless you are running an ancient (circa 2006-2007) version of Solaris where the default offset was 34. Yes, but be warned: the format command will try to make fdisk partitions that are aligned on cyninder boundaries, which are usually odd numbers of sectors. I have a fix for that here: http://yalms.org/cr/zfs-blksize/ (the first three files. ignore the zfs part for now) Gordon One day this low level disk stuff will get into this millenium. I see my CDC Wren IV 94171-327 museum exhibit still rates a mention :) Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Apr 29, 2012, at 7:38 PM, Gordon Ross wrote: On Sun, Apr 29, 2012 at 8:46 PM, Richard Elling richard.ell...@richardelling.com wrote: On Apr 29, 2012, at 11:45 AM, George Wilson wrote: [...] Speaking of 4K sectors, I've taken a slightly different approach that fixes this outside of ZFS. The idea is to allow sd to override the physical-block-size which ZFS will pick up. The way this works is you can specify the Vendor/Product id in sd.conf. Here's an example: sd-config-list = NETAPP LUN, physical-block-size:4096; This is the preferred solution and there are several implementations running around in various stages of test/release/acceptance. I look forward to getting this upstream :-) -- richard Providing a work-around in sd is great. We should do that, at least. But is it sufficient? What happens if I replace a mirrored drive with 512 byte sectors with one having 4k sectors? What if I want to plan ahead for that? Maybe in only some of my ZFS pools but not all? It would seem that a pool-level override for ashift might also be useful. ashift is set for the top-level vdev at creation time. So you have to override prior to creation of the mirror. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
I'm building a storage server with Dell MD1000 DAS and I just bought 30 drives with 4K sectors. One of the reasons I selected the new 4K sector is so I can easily find replacement drives 2-3 years from now when they start failing. Looks like this was a huge mistake. I'm fine if the drives report 512B sectors and work in slower legacy mode as long as they work reliable but seems that this may not be the case. On top of that my internal drives that make the rpool have 512B sectors so I'm not sure how workarounds will effect this pool. Is it fair to say that if one uses 4K drives he will run into alignment issue sooner or later? I'm really puzzled what to do here. Should I try to replace the drives with 512B ones before the storage goes life? Any thoughts? Thank you Peter On Mon, Apr 30, 2012 at 7:54 AM, Richard Elling richard.ell...@richardelling.com wrote: On Apr 29, 2012, at 7:38 PM, Gordon Ross wrote: On Sun, Apr 29, 2012 at 8:46 PM, Richard Elling richard.ell...@richardelling.com wrote: On Apr 29, 2012, at 11:45 AM, George Wilson wrote: [...] Speaking of 4K sectors, I've taken a slightly different approach that fixes this outside of ZFS. The idea is to allow sd to override the physical-block-size which ZFS will pick up. The way this works is you can specify the Vendor/Product id in sd.conf. Here's an example: sd-config-list = NETAPP LUN, physical-block-size:4096; This is the preferred solution and there are several implementations running around in various stages of test/release/acceptance. I look forward to getting this upstream :-) -- richard Providing a work-around in sd is great. We should do that, at least. But is it sufficient? What happens if I replace a mirrored drive with 512 byte sectors with one having 4k sectors? What if I want to plan ahead for that? Maybe in only some of my ZFS pools but not all? It would seem that a pool-level override for ashift might also be useful. ashift is set for the top-level vdev at creation time. So you have to override prior to creation of the mirror. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Sat, 28 Apr 2012, Enrico wrote: did a little research, just because i'm curious about this, my storage needs are much lower :) they say (http://forums.overclockers.com.au/showpost.php?p=13963218postcount=4727) that SB820M south-bridge on n40l has a limit of 3.2tb for hdd capacity. but there are no such details on the official specs: support.amd.com/us/Embedded_TechDocs/47283.pdf i hope this helps a bit A comment from a poster called 'AFireInside' on this same forum says AMD confirms that 4 TB disks will work in the N40L Microserver: http://forums.overclockers.com.au/showthread.php?s=a97e85a928a1e6997245557220f4b729t=1009531page=2 Also, HP's own specifications confirm that this unit is suitable for 4 TB disks: http://h18000.www1.hp.com/products/quickspecs/13716_div/13716_div.html A lot of the comments saying otherwise seem to relate to the earlier N36L model, earlier BIOS versions and booting Windows from disks larger than 3.2 TB. As for using Advanced Format disks with 4K sectors in ZFS pools, the worst case scenario would be that these disks can be used in a backwards-compatible legacy 512 byte sector format but with slower r/w performance. Also, Hitachi provide a Windows utility to re-align 4k sectors on their drives although I'm not sure whether this would suit OI/ZFS. Finally, it looks like OI has some support for 4k sector disks: http://wiki.openindiana.org/pages/viewpage.action?pageId=4883847 and ZFS pools can be created with the 'zpool create block-size 4096 ...' option. I'll take the plunge and order a N40L Microserver plus 3 x 4 TB disks and I'll post my experiences here. cheers, Andy On 04/28/2012 08:39 PM, Enrico wrote: Hi roy, yes, right, it was just to tell Andy what I've tested with this hp system and OI. i think there are many people who have written in detail their experience about supported hdd sizes in forums easily reachable just by googling n40l. let me also correct what i've written before: the adaptec 1045/1405 are luckily supported by the aac driver, despite what is written on the man page. the si3124 driver refers to a sata controller i've tested successfully on the pciex1 slot. Best regards, Enrico On 04/28/2012 06:13 PM, Roy Sigurd Karlsbakk wrote: It probably won't help much. Any SATA controller should recognize the 4TB drives, and the problem with the drives lying about sector sizes won't be any better whatever controller you use. roy - Opprinnelig melding - maybe this might help: http://www.adaptec.com/en-us/products/controllers/hardware/sas/entry/asc-1045/ it is recognized on a n40l and it is supported by the si3124 driver On 04/28/2012 09:54 AM, andy thomas wrote: For some time now I've successfully been using ZFS RAIDz pools made up of 3 x 1 TB or 3 x 2 TB Western Digital Caviar Black disks in a HP MicroServer N36L running OI 148 or 151. I'm now thinking of using the newer HP N40L microserver with 3 x 4 TB disks (Hitachi DeskStart 7K4000) which have 4096 byte sectors. Can anyone foresee a problem with this? I plan to set up several of these storage servers in mirrored pairs so I want to make sure this will work before ordering anything. Thanks in advance for any comments or advice, Andy - Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss - Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
thank you Andy for this info let us know your experience when you'll setup-up the disks On 04/29/2012 09:24 AM, andy thomas wrote: On Sat, 28 Apr 2012, Enrico wrote: did a little research, just because i'm curious about this, my storage needs are much lower :) they say (http://forums.overclockers.com.au/showpost.php?p=13963218postcount=4727) that SB820M south-bridge on n40l has a limit of 3.2tb for hdd capacity. but there are no such details on the official specs: support.amd.com/us/Embedded_TechDocs/47283.pdf i hope this helps a bit A comment from a poster called 'AFireInside' on this same forum says AMD confirms that 4 TB disks will work in the N40L Microserver: http://forums.overclockers.com.au/showthread.php?s=a97e85a928a1e6997245557220f4b729t=1009531page=2 Also, HP's own specifications confirm that this unit is suitable for 4 TB disks: http://h18000.www1.hp.com/products/quickspecs/13716_div/13716_div.html A lot of the comments saying otherwise seem to relate to the earlier N36L model, earlier BIOS versions and booting Windows from disks larger than 3.2 TB. As for using Advanced Format disks with 4K sectors in ZFS pools, the worst case scenario would be that these disks can be used in a backwards-compatible legacy 512 byte sector format but with slower r/w performance. Also, Hitachi provide a Windows utility to re-align 4k sectors on their drives although I'm not sure whether this would suit OI/ZFS. Finally, it looks like OI has some support for 4k sector disks: http://wiki.openindiana.org/pages/viewpage.action?pageId=4883847 and ZFS pools can be created with the 'zpool create block-size 4096 ...' option. I'll take the plunge and order a N40L Microserver plus 3 x 4 TB disks and I'll post my experiences here. cheers, Andy On 04/28/2012 08:39 PM, Enrico wrote: Hi roy, yes, right, it was just to tell Andy what I've tested with this hp system and OI. i think there are many people who have written in detail their experience about supported hdd sizes in forums easily reachable just by googling n40l. let me also correct what i've written before: the adaptec 1045/1405 are luckily supported by the aac driver, despite what is written on the man page. the si3124 driver refers to a sata controller i've tested successfully on the pciex1 slot. Best regards, Enrico On 04/28/2012 06:13 PM, Roy Sigurd Karlsbakk wrote: It probably won't help much. Any SATA controller should recognize the 4TB drives, and the problem with the drives lying about sector sizes won't be any better whatever controller you use. roy - Opprinnelig melding - maybe this might help: http://www.adaptec.com/en-us/products/controllers/hardware/sas/entry/asc-1045/ it is recognized on a n40l and it is supported by the si3124 driver On 04/28/2012 09:54 AM, andy thomas wrote: For some time now I've successfully been using ZFS RAIDz pools made up of 3 x 1 TB or 3 x 2 TB Western Digital Caviar Black disks in a HP MicroServer N36L running OI 148 or 151. I'm now thinking of using the newer HP N40L microserver with 3 x 4 TB disks (Hitachi DeskStart 7K4000) which have 4096 byte sectors. Can anyone foresee a problem with this? I plan to set up several of these storage servers in mirrored pairs so I want to make sure this will work before ordering anything. Thanks in advance for any comments or advice, Andy - Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss - Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
As for using Advanced Format disks with 4K sectors in ZFS pools, the worst case scenario would be that these disks can be used in a backwards-compatible legacy 512 byte sector format but with slower r/w performance. Also, Hitachi provide a Windows utility to re-align 4k sectors on their drives although I'm not sure whether this would suit OI/ZFS. The Windows tool certainly does not work on OI. Finally, it looks like OI has some support for 4k sector disks: http://wiki.openindiana.org/pages/viewpage.action?pageId=4883847 and ZFS pools can be created with the 'zpool create block-size 4096 ...' option. The patch allowing this option was, for various reasons, rejected, and no such option currently exists. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Sun, Apr 29, 2012 at 9:29 AM, Roy Sigurd Karlsbakk r...@karlsbakk.net wrote: Finally, it looks like OI has some support for 4k sector disks: http://wiki.openindiana.org/pages/viewpage.action?pageId=4883847 and ZFS pools can be created with the 'zpool create block-size 4096 ...' option. The patch allowing this option was, for various reasons, rejected, and no such option currently exists. Are you able to forward a link to the reasons? I'm asking because were they technical (i.e. something breaks), or merely political (and there isn't a reason why OI couldn't ship zpool with the patch)? Jan ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
Finally, it looks like OI has some support for 4k sector disks: http://wiki.openindiana.org/pages/viewpage.action?pageId=4883847 and ZFS pools can be created with the 'zpool create block-size 4096 ...' option. The patch allowing this option was, for various reasons, rejected, and no such option currently exists. Are you able to forward a link to the reasons? I'm asking because were they technical (i.e. something breaks), or merely political (and there isn't a reason why OI couldn't ship zpool with the patch)? IIRC the reason was that the patch breaks standard syntax, and that certain people thought it bad to fix what appears to be a hardware issue with drives reporting wrong sector size. I don't have any disk with advanced format, so I can't test - is there a way to query the disk somehow to find if it's using 4k sectors? Model lookup seems an ugly way to do this, and although the patch added an option to force it, it's not an optimal solution. More discussions and a drop-in replacement for zpool can be found here http://wiki.openindiana.org/pages/viewpage.action?pageId=4883847 Also, I posted a bug report for it here https://www.illumos.org/issues/2663 Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Sun, Apr 29, 2012 at 10:33 AM, Roy Sigurd Karlsbakk r...@karlsbakk.net wrote: The patch allowing this option was, for various reasons, rejected, and no such option currently exists. Are you able to forward a link to the reasons? I'm asking because were they technical (i.e. something breaks), or merely political (and there isn't a reason why OI couldn't ship zpool with the patch)? IIRC the reason was that the patch breaks standard syntax, and that certain people thought it bad to fix what appears to be a hardware issue with drives reporting wrong sector size. I don't have any disk with advanced format, so I can't test - is there a way to query the disk somehow to find if it's using 4k sectors? Model lookup seems an ugly way to do this, and although the patch added an option to force it, it's not an optimal solution. More discussions and a drop-in replacement for zpool can be found here http://wiki.openindiana.org/pages/viewpage.action?pageId=4883847 Most hard drives manufactured this year will be advanced format. It may still be several years before we get hard drives that don't lie. I haven't found any way to query the drive, other than looking up the model number on the manufacturer's website (which isn't always easy, and sometimes finds conflicting information because you get a logical size of 512 and a physical size of 4k). Also, I posted a bug report for it here https://www.illumos.org/issues/2663 Thanks :-). We can now track the progress of the OI-specific discussion about this issue. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
Also, I posted a bug report for it here https://www.illumos.org/issues/2663 Thanks :-). We can now track the progress of the OI-specific discussion about this issue. Seems the old post about the initial patch is here https://www.illumos.org/issues/453 After som discussion on #openindiana (irc.freenode.net), it seems the best would perhaps be to port the ashift flag available in Linux. Also, some SSDs might need ashift=13 for optimal performance… Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Apr 29, 2012, at 1:28 PM, Roy Sigurd Karlsbakk wrote: Also, I posted a bug report for it here https://www.illumos.org/issues/2663 Thanks :-). We can now track the progress of the OI-specific discussion about this issue. Seems the old post about the initial patch is here https://www.illumos.org/issues/453 After som discussion on #openindiana (irc.freenode.net), it seems the best would perhaps be to port the ashift flag available in Linux. Also, some SSDs might need ashift=13 for optimal performance… Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss Speaking of 4K sectors, I've taken a slightly different approach that fixes this outside of ZFS. The idea is to allow sd to override the physical-block-size which ZFS will pick up. The way this works is you can specify the Vendor/Product id in sd.conf. Here's an example: sd-config-list = NETAPP LUN, physical-block-size:4096; I'll be sending this out for illumos review soon. - George ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Sun, Apr 29, 2012 at 02:45:05PM -0400, George Wilson wrote: On Apr 29, 2012, at 1:28 PM, Roy Sigurd Karlsbakk wrote: Also, I posted a bug report for it here https://www.illumos.org/issues/2663 Seems the old post about the initial patch is here https://www.illumos.org/issues/453 Speaking of 4K sectors, I've taken a slightly different approach that fixes this outside of ZFS. The idea is to allow sd to override the physical-block-size which ZFS will pick up. The way this works is you can specify the Vendor/Product id in sd.conf. Here's an example: sd-config-list = NETAPP LUN, physical-block-size:4096; Yes, since this is a hardware issue, the best course of action would be to list the offending disk drives in a system file. Does the Netapp lie about the block size of its LUNs? -- -Gary Mills--refurb--Winnipeg, Manitoba, Canada- ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Apr 29, 2012, at 2:53 PM, Gary Mills wrote: On Sun, Apr 29, 2012 at 02:45:05PM -0400, George Wilson wrote: On Apr 29, 2012, at 1:28 PM, Roy Sigurd Karlsbakk wrote: Also, I posted a bug report for it here https://www.illumos.org/issues/2663 Seems the old post about the initial patch is here https://www.illumos.org/issues/453 Speaking of 4K sectors, I've taken a slightly different approach that fixes this outside of ZFS. The idea is to allow sd to override the physical-block-size which ZFS will pick up. The way this works is you can specify the Vendor/Product id in sd.conf. Here's an example: sd-config-list = NETAPP LUN, physical-block-size:4096; Yes, since this is a hardware issue, the best course of action would be to list the offending disk drives in a system file. Does the Netapp lie about the block size of its LUNs? Yeah, this apparently is a common problem when using ZFS and Netapp. - George -- -Gary Mills- -refurb--Winnipeg, Manitoba, Canada- ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Apr 29, 2012, at 11:45 AM, George Wilson wrote: On Apr 29, 2012, at 1:28 PM, Roy Sigurd Karlsbakk wrote: Also, I posted a bug report for it here https://www.illumos.org/issues/2663 Thanks :-). We can now track the progress of the OI-specific discussion about this issue. Seems the old post about the initial patch is here https://www.illumos.org/issues/453 After som discussion on #openindiana (irc.freenode.net), it seems the best would perhaps be to port the ashift flag available in Linux. Also, some SSDs might need ashift=13 for optimal performance… Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss Speaking of 4K sectors, I've taken a slightly different approach that fixes this outside of ZFS. The idea is to allow sd to override the physical-block-size which ZFS will pick up. The way this works is you can specify the Vendor/Product id in sd.conf. Here's an example: sd-config-list = NETAPP LUN, physical-block-size:4096; This is the preferred solution and there are several implementations running around in various stages of test/release/acceptance. I look forward to getting this upstream :-) -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On Sun, Apr 29, 2012 at 8:46 PM, Richard Elling richard.ell...@richardelling.com wrote: On Apr 29, 2012, at 11:45 AM, George Wilson wrote: [...] Speaking of 4K sectors, I've taken a slightly different approach that fixes this outside of ZFS. The idea is to allow sd to override the physical-block-size which ZFS will pick up. The way this works is you can specify the Vendor/Product id in sd.conf. Here's an example: sd-config-list = NETAPP LUN, physical-block-size:4096; This is the preferred solution and there are several implementations running around in various stages of test/release/acceptance. I look forward to getting this upstream :-) -- richard Providing a work-around in sd is great. We should do that, at least. But is it sufficient? What happens if I replace a mirrored drive with 512 byte sectors with one having 4k sectors? What if I want to plan ahead for that? Maybe in only some of my ZFS pools but not all? It would seem that a pool-level override for ashift might also be useful. Gordon ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
For some time now I've successfully been using ZFS RAIDz pools made up of 3 x 1 TB or 3 x 2 TB Western Digital Caviar Black disks in a HP MicroServer N36L running OI 148 or 151. I'm now thinking of using the newer HP N40L microserver with 3 x 4 TB disks (Hitachi DeskStart 7K4000) which have 4096 byte sectors. Can anyone foresee a problem with this? I plan to set up several of these storage servers in mirrored pairs so I want to make sure this will work before ordering anything. Thanks in advance for any comments or advice, Andy - Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
i have been using oi-151a1 on hp n40L for 6 months without problems. my pool is smaller than yours and i do not have 4k disks. if you look at the quick specs of the n40l: Maximum internal SATA storage capacity of up to 8.0TB (4 x 2TB 3.5 SATA drives) 2TB 3G SATA 7.2K, Up to 4, SATA controller with Embedded RAID on Motherboard maybe 2 4tb disk would work, but for sure not 3 of them On 04/28/2012 09:54 AM, andy thomas wrote: For some time now I've successfully been using ZFS RAIDz pools made up of 3 x 1 TB or 3 x 2 TB Western Digital Caviar Black disks in a HP MicroServer N36L running OI 148 or 151. I'm now thinking of using the newer HP N40L microserver with 3 x 4 TB disks (Hitachi DeskStart 7K4000) which have 4096 byte sectors. Can anyone foresee a problem with this? ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
For some time now I've successfully been using ZFS RAIDz pools made up of 3 x 1 TB or 3 x 2 TB Western Digital Caviar Black disks in a HP MicroServer N36L running OI 148 or 151. I'm now thinking of using the newer HP N40L microserver with 3 x 4 TB disks (Hitachi DeskStart 7K4000) which have 4096 byte sectors. Can anyone foresee a problem with this? i have been using oi-151a1 on hp n40L for 6 months without problems. my pool is smaller than yours and i do not have 4k disks. if you look at the quick specs of the n40l: Maximum internal SATA storage capacity of up to 8.0TB (4 x 2TB 3.5 SATA drives) 2TB 3G SATA 7.2K, Up to 4, SATA controller with Embedded RAID on Motherboard maybe 2 4tb disk would work, but for sure not 3 of them If one works, 10 or 100 will. The only known issue with 4k sector drives, is alignment issues. Since the drives lie about the sector size, reporting 512B logical sectors, while really having 4kiB sectors, the OS cannot foresee real alignment, and may align partitions/filesystems to the reported 512B size, so in case of overlap, multiple I/O operations may be issued to the same physical sector. Patches for zpool has been submitted to fix this, but was, for various reasons, rejected. I don't know the current status for these. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
maybe this might help: http://www.adaptec.com/en-us/products/controllers/hardware/sas/entry/asc-1045/ it is recognized on a n40l and it is supported by the si3124 driver On 04/28/2012 09:54 AM, andy thomas wrote: For some time now I've successfully been using ZFS RAIDz pools made up of 3 x 1 TB or 3 x 2 TB Western Digital Caviar Black disks in a HP MicroServer N36L running OI 148 or 151. I'm now thinking of using the newer HP N40L microserver with 3 x 4 TB disks (Hitachi DeskStart 7K4000) which have 4096 byte sectors. Can anyone foresee a problem with this? I plan to set up several of these storage servers in mirrored pairs so I want to make sure this will work before ordering anything. Thanks in advance for any comments or advice, Andy - Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
It probably won't help much. Any SATA controller should recognize the 4TB drives, and the problem with the drives lying about sector sizes won't be any better whatever controller you use. roy - Opprinnelig melding - maybe this might help: http://www.adaptec.com/en-us/products/controllers/hardware/sas/entry/asc-1045/ it is recognized on a n40l and it is supported by the si3124 driver On 04/28/2012 09:54 AM, andy thomas wrote: For some time now I've successfully been using ZFS RAIDz pools made up of 3 x 1 TB or 3 x 2 TB Western Digital Caviar Black disks in a HP MicroServer N36L running OI 148 or 151. I'm now thinking of using the newer HP N40L microserver with 3 x 4 TB disks (Hitachi DeskStart 7K4000) which have 4096 byte sectors. Can anyone foresee a problem with this? I plan to set up several of these storage servers in mirrored pairs so I want to make sure this will work before ordering anything. Thanks in advance for any comments or advice, Andy - Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
Hi roy, yes, right, it was just to tell Andy what I've tested with this hp system and OI. i think there are many people who have written in detail their experience about supported hdd sizes in forums easily reachable just by googling n40l. let me also correct what i've written before: the adaptec 1045/1405 are luckily supported by the aac driver, despite what is written on the man page. the si3124 driver refers to a sata controller i've tested successfully on the pciex1 slot. Best regards, Enrico On 04/28/2012 06:13 PM, Roy Sigurd Karlsbakk wrote: It probably won't help much. Any SATA controller should recognize the 4TB drives, and the problem with the drives lying about sector sizes won't be any better whatever controller you use. roy - Opprinnelig melding - maybe this might help: http://www.adaptec.com/en-us/products/controllers/hardware/sas/entry/asc-1045/ it is recognized on a n40l and it is supported by the si3124 driver On 04/28/2012 09:54 AM, andy thomas wrote: For some time now I've successfully been using ZFS RAIDz pools made up of 3 x 1 TB or 3 x 2 TB Western Digital Caviar Black disks in a HP MicroServer N36L running OI 148 or 151. I'm now thinking of using the newer HP N40L microserver with 3 x 4 TB disks (Hitachi DeskStart 7K4000) which have 4096 byte sectors. Can anyone foresee a problem with this? I plan to set up several of these storage servers in mirrored pairs so I want to make sure this will work before ordering anything. Thanks in advance for any comments or advice, Andy - Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
did a little research, just because i'm curious about this, my storage needs are much lower :) they say (http://forums.overclockers.com.au/showpost.php?p=13963218postcount=4727) that SB820M south-bridge on n40l has a limit of 3.2tb for hdd capacity. but there are no such details on the official specs: support.amd.com/us/Embedded_TechDocs/47283.pdf i hope this helps a bit Best Regards, Enrico On 04/28/2012 08:39 PM, Enrico wrote: Hi roy, yes, right, it was just to tell Andy what I've tested with this hp system and OI. i think there are many people who have written in detail their experience about supported hdd sizes in forums easily reachable just by googling n40l. let me also correct what i've written before: the adaptec 1045/1405 are luckily supported by the aac driver, despite what is written on the man page. the si3124 driver refers to a sata controller i've tested successfully on the pciex1 slot. Best regards, Enrico On 04/28/2012 06:13 PM, Roy Sigurd Karlsbakk wrote: It probably won't help much. Any SATA controller should recognize the 4TB drives, and the problem with the drives lying about sector sizes won't be any better whatever controller you use. roy - Opprinnelig melding - maybe this might help: http://www.adaptec.com/en-us/products/controllers/hardware/sas/entry/asc-1045/ it is recognized on a n40l and it is supported by the si3124 driver On 04/28/2012 09:54 AM, andy thomas wrote: For some time now I've successfully been using ZFS RAIDz pools made up of 3 x 1 TB or 3 x 2 TB Western Digital Caviar Black disks in a HP MicroServer N36L running OI 148 or 151. I'm now thinking of using the newer HP N40L microserver with 3 x 4 TB disks (Hitachi DeskStart 7K4000) which have 4096 byte sectors. Can anyone foresee a problem with this? I plan to set up several of these storage servers in mirrored pairs so I want to make sure this will work before ordering anything. Thanks in advance for any comments or advice, Andy - Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
