Re: [zfs-discuss] Slow zfs writes
> root@host:~# fmadm faulty > --- -- > - > TIME EVENT-ID MSG-ID SEVERITY > --- -- > - > Jan 05 08:21:09 7af1ab3c-83c2-602d-d4b9-f9040db6944a ZFS-8000-HC Major > Host : host > Platform : PowerEdge-R810 > Product_sn : > Fault class : fault.fs.zfs.io_failure_wait > Affects : zfs://pool=test > faulted but still in service > Problem in : zfs://pool=test > faulted but still in service > Description : The ZFS pool has experienced currently unrecoverable I/O > failures. Refer to http://illumos.org/msg/ZFS-8000-HC for > more information. > Response : No automated response will be taken. > Impact : Read and write I/Os cannot be serviced. > Action : Make sure the affected devices are connected, then run > 'zpool clear'. > -- The pool looks healthy to me, but it it isn't very well balanced. Have you been adding new VDEVs on your way to grow it? Check if of the VDEVs are fuller than others. I don't have an OI/IllumOS system available ATM, but IIRC this can be done with iostat -v. Older versions of ZFS striped to all VDEVs regardless to fill, which slowed down the write speeds rather horribly if some VDEVs were full (>90%). This shouldn't be the case with OmniOS, but it *may* be the case with an old zpool version. I don't know. I'd check fill rate of the VDEVs first, then perhaps try to upgrade the zpool unless you have to be able to mount it on an older version of zpool (on S10 or similar). Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZIL devices and fragmentation
- Opprinnelig melding - > On Mon, Jul 30, 2012 at 9:38 AM, Roy Sigurd Karlsbakk > wrote: > >> > Also keep in mind that if you have an SLOG (ZIL on a separate > >> > device), and then lose this SLOG (disk crash etc), you will > >> > probably > >> > lose the pool. So if you want/need SLOG, you probably want two of > >> > them in a mirror… > >> > >> That's only true on older versions of ZFS. ZFSv19 (or 20?) includes > >> the ability to import a pool with a failed/missing log device. You > >> lose any data that is in the log and not in the pool, but the pool > >> is > >> importable. > > > > Are you sure? I booted this v28 pool a couple of months back, and > > found it didn't recognize its pool, apparently because of a missing > > SLOG. It turned out the cache shelf was disconnected, after > > re-connecting it, things worked as planned. I didn't try to force a > > new import, though, but it didn't boot up normally, and told me it > > couldn't import its pool due to lack of SLOG devices. > > Positive. :) I tested it with ZFSv28 on FreeBSD 9-STABLE a month or > two ago. See the updated man page for zpool, especially the bit about > "import -m". :) On 151a2, man page just says 'use this or that mountpoint' with import -m, but the fact was zpool refused to import the pool at boot when 2 SLOG devices (mirrored) and 10 L2ARC devices were offline. Should OI/Illumos be able to boot cleanly without manual action with the SLOG devices gone? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZIL devices and fragmentation
> > Also keep in mind that if you have an SLOG (ZIL on a separate > > device), and then lose this SLOG (disk crash etc), you will probably > > lose the pool. So if you want/need SLOG, you probably want two of > > them in a mirror… > > That's only true on older versions of ZFS. ZFSv19 (or 20?) includes > the ability to import a pool with a failed/missing log device. You > lose any data that is in the log and not in the pool, but the pool is > importable. Are you sure? I booted this v28 pool a couple of months back, and found it didn't recognize its pool, apparently because of a missing SLOG. It turned out the cache shelf was disconnected, after re-connecting it, things worked as planned. I didn't try to force a new import, though, but it didn't boot up normally, and told me it couldn't import its pool due to lack of SLOG devices. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZIL devices and fragmentation
> > For several times now I've seen statements on this list implying > > that a dedicated ZIL/SLOG device catching sync writes for the log, > > also allows for more streamlined writes to the pool during normal > > healthy TXG syncs, than is the case with the default ZIL located > > within the pool. > > After reading what some others have posted, I should remind that zfs > always has a ZIL (unless it is specifically disabled for testing). > If it does not have a dedicated ZIL, then it uses the disks in the > main pool to construct the ZIL. Dedicating a device to the ZIL should > not improve the pool storage layout because the pool already had a > ZIL. Also keep in mind that if you have an SLOG (ZIL on a separate device), and then lose this SLOG (disk crash etc), you will probably lose the pool. So if you want/need SLOG, you probably want two of them in a mirror… Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Can the ZFS "copies" attribute substitute HW disk redundancy?
"copies" won't help much if the pool is unavailable. It may, however, help if, say, you have a RAIDz2, and two drives die, and htere are errors on a third drive, but not sufficiently bad for zfs to reject the pool roy - Opprinnelig melding - > Hello all, > > Over the past few years there have been many posts suggesting > that for modern HDDs (several TB size, around 100-200MB/s best > speed) the rebuild times grow exponentially, so to build a well > protected pool with these disks one has to plan for about three > disk's worth of redundancy - that is, three- or four-way mirrors, > or raidz3 - just to allow systems to survive a disk outage (with > accpetably high probability of success) while one is resilvering. > > There were many posts on this matter from esteemed members of > the list, including (but certainly not limited to) these articles: > * https://blogs.oracle.com/ahl/entry/triple_parity_raid_z > * https://blogs.oracle.com/ahl/entry/acm_triple_parity_raid > * http://queue.acm.org/detail.cfm?id=1670144 > * > http://blog.richardelling.com/2010/02/zfs-data-protection-comparison.html > > Now, this brings me to such a question: when people build a > home-NAS box, they are quite constrained in terms of the number > of directly attached disks (about 4-6 bays), or even if they > use external JBODs - to the number of disks in them (up to 8, > which does allow a 5+3 raidz3 set in a single box, which still > seems like a large overhead to some buyers - a 4*2 mirror would > give about as much space and higher performance, but may have > unacceptably less redundancy). If I want to have considerable > storage, with proper reliability, and just a handful of drives, > what are my best options? > > I wondered if the "copies" attribute can be considered sort > of equivalent to the number of physical disks - limited to seek > times though. Namely, for the same amount of storage on a 4-HDD > box I could use raidz1 and 4*1tb@copies=1 or 4*2tb@copies=2 or > even 4*3tb@copies=3, for example. > > To simplify the matters, let's assume that this is a small > box (under 10GB RAM) not using dedup, though it would likely > use compression :) > > Question to theorists and practicians: is any of these options > better or worse than the others, in terms of reliability and > access/rebuild/scrub speeds, for either a single-sector error > or for a full-disk replacement? > > Would extra copies on larger disks actually provide the extra > reliability, or only add overheads and complicate/degrade the > situation? > > Would the use of several copies cripple the write speeds? > > Can the extra copies be used by zio scheduler to optimize and > speed up reads, like extra mirror sides would? > > Thanks, > //Jim Klimov > > ___________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Persistent errors?
> It seems as though every time I scrub my mirror I get a few megabytes > of checksum errors on one disk (luckily corrected by the other). Is > there some way of tracking down a problem which might be persistent? Check iostat -en or iostat -En . If the latter shows media errors, the drive is dying, and should be replaced. > I wonder if it's anything to do with these messages which are > constantly appearing on the console: > > Jun 17 12:06:18 sunny scsi: [ID 107833 kern.warning] WARNING: > /pci@0,0/pci1000,8000@16/sd@0,0 (sd2): > Jun 17 12:06:18 sunny SYNCHRONIZE CACHE command failed (5) > > I've no idea what they are about (this is on Solaris 11 btw). No idea, sorry… Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Compatibility of Hitachi Deskstar 7K3000 HDS723030ALA640 with ZFS
We're using Hitachi HDS723030ALA640 on this rather busy server, and they've been stable for about a year - I don't even think we've lost any yet (out of 22) roy - Opprinnelig melding - Greetings, Quick question: I am about to acquire some disks for use with ZFS (currently using zfs-fuse v0.7.0). I'm aware of some 4k alignment issues with Western Digital advanced format disks. As far as I can tell, the Hitachi Deskstar 7K3000 (HDS723030ALA640) uses 512B sectors and so I presume does not suffer from such issues (because it doesn't lie about the physical layout of sectors on-platter) Can someone confirm this or point out any other known issues with the disks? I will be using the disks raw, unpartitioned. Many thanks, Luis. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Deskstars and CCTL (aka TLER)
Hi all Reading the docs for the Hitachi drives, it seems CCTL (aka TLER) is settable for Deskstar drives. See page 97 in http://goo.gl/ER0WD Does anyone know if this is possible from OI/Solaris, or if this needs to be done on driver level? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] BAD WD drives - defective by design?
> The common use for desktop drives is having a single disk without > redundancy.. If a sector is feeling bad, it's better if it tries a bit > harder to recover it than just say "blah, there was a bit of dirt in > the corner.. I don't feel like looking at it, so I'll just say your data > is screwed instead".. In a raid setup, that data is sitting safe(?) on > some other disk as well, so it might as well give up early. Still, there's a wee difference between shaving and cutting your head off. A drive retrying a single sector for two whole minutes is nonsense, even on a desktop or laptop, at least when it does so without logging the error to SMART or summing up the issues so to flag the disk unusable. And, beleive it or not, a drive spending 2 minutes trying to fetch 512 bytes from a dead sector is quite unusable when the the number of bad sectors start climbing. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] BAD WD drives - defective by design?
> http://wdc.custhelp.com/app/answers/detail/a_id/1397/~/difference-between-desktop-edition-and-raid-%28enterprise%29-edition-drives "When an error is found on a desktop edition hard drive, the drive will enter into a deep recovery cycle to attempt to repair the error, recover the data from the problematic area, and then reallocate a dedicated area to replace the problematic area. This process can take up to 2 minutes depending on the severity of the issue" Or in other words: "When an error occurs on a desktop drive, the drive will refuse to realize the sector is bad, and retry forever and ever without even increasing SMART counters, so that even if Western Digital Data LifeGuard Diagnostics need to spend 36 hours to test a drive (as opposed to the normal 5-ish hours for a 2TB 7k2 drive), WD will refuse return of the drive because IT WORKS". Or in yet other words: "Desktop drives aren't meant to be used for anything productive or important." Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [OpenIndiana-discuss] BAD WD drives - defective by design?
And, yes, they're connected to an LSI SAS expander from Super Micro. Works well with Seagate and Hitachi, but not with WD - Original Message - > The drives are attached to a backplane? > > Try using 4k sector sizes and see if that improves it - I've seen and > been part of a number of discussions which involved this - and you, I > think, actually. > > - Rich > > On Mon, Aug 29, 2011 at 5:07 PM, Roy Sigurd Karlsbakk > wrote: > > Hi all > > > > It seems recent WD drives that aren't "Raid edition" can cause > > rather a lot of problems on RAID systems. We have a few machines > > with LSI controllers (6801/6081/9201) and we're seeing massive > > errors occuring. The usual pattern is a drive failing or even a > > resilver/scrub starting and then, suddenly, most drives on the whole > > backplane report errors. These are usually No Device (as reported by > > iostat -En), but the result is that we may see data corruption at > > the end. We also have a system setup with Hitachi Deskstars, which > > has been running for almost a year without issues. One system with a > > mixture of WD Blacks and greens showed the same errors as described, > > but has been working well after the WD drives were replaced by > > deskstars. > > > > Now, it seems WD has changed their firmware to inhibit people from > > using them for other things than toys (read: PCs etc). Since we've > > seen this issue on different controllers and different drives, and > > can't reproduce it with Hitachi Deskstars, I would guess the > > firmware "upgrade" from WD is the issue. > > > > Would it be possible to fix this in ZFS somehow? The drives seem to > > work well except for those "No Device" errors > > > > Vennlige hilsener / Best regards > > > > roy > > -- > > Roy Sigurd Karlsbakk > > (+47) 97542685 > > r...@karlsbakk.net > > http://blogg.karlsbakk.net/ > > -- > > I all pedagogikk er det essensielt at pensum presenteres > > intelligibelt. Det er et elementært imperativ for alle pedagoger å > > unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de > > fleste tilfeller eksisterer adekvate og relevante synonymer på > > norsk. > > > > ___ > > OpenIndiana-discuss mailing list > > openindiana-disc...@openindiana.org > > http://openindiana.org/mailman/listinfo/openindiana-discuss > > > > ___ > OpenIndiana-discuss mailing list > openindiana-disc...@openindiana.org > http://openindiana.org/mailman/listinfo/openindiana-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [OpenIndiana-discuss] BAD WD drives - defective by design?
All drives have 512b sector sizes, WD FASS (blacks) and WD EADS (greens) both use plain old 512 sectors. - Original Message - > The drives are attached to a backplane? > > Try using 4k sector sizes and see if that improves it - I've seen and > been part of a number of discussions which involved this - and you, I > think, actually. > > - Rich > > On Mon, Aug 29, 2011 at 5:07 PM, Roy Sigurd Karlsbakk > wrote: > > Hi all > > > > It seems recent WD drives that aren't "Raid edition" can cause > > rather a lot of problems on RAID systems. We have a few machines > > with LSI controllers (6801/6081/9201) and we're seeing massive > > errors occuring. The usual pattern is a drive failing or even a > > resilver/scrub starting and then, suddenly, most drives on the whole > > backplane report errors. These are usually No Device (as reported by > > iostat -En), but the result is that we may see data corruption at > > the end. We also have a system setup with Hitachi Deskstars, which > > has been running for almost a year without issues. One system with a > > mixture of WD Blacks and greens showed the same errors as described, > > but has been working well after the WD drives were replaced by > > deskstars. > > > > Now, it seems WD has changed their firmware to inhibit people from > > using them for other things than toys (read: PCs etc). Since we've > > seen this issue on different controllers and different drives, and > > can't reproduce it with Hitachi Deskstars, I would guess the > > firmware "upgrade" from WD is the issue. > > > > Would it be possible to fix this in ZFS somehow? The drives seem to > > work well except for those "No Device" errors > > > > Vennlige hilsener / Best regards > > > > roy > > -- > > Roy Sigurd Karlsbakk > > (+47) 97542685 > > r...@karlsbakk.net > > http://blogg.karlsbakk.net/ > > -- > > I all pedagogikk er det essensielt at pensum presenteres > > intelligibelt. Det er et elementært imperativ for alle pedagoger å > > unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de > > fleste tilfeller eksisterer adekvate og relevante synonymer på > > norsk. > > > > ___ > > OpenIndiana-discuss mailing list > > openindiana-disc...@openindiana.org > > http://openindiana.org/mailman/listinfo/openindiana-discuss > > > > ___ > OpenIndiana-discuss mailing list > openindiana-disc...@openindiana.org > http://openindiana.org/mailman/listinfo/openindiana-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] BAD WD drives - defective by design?
Hi all It seems recent WD drives that aren't "Raid edition" can cause rather a lot of problems on RAID systems. We have a few machines with LSI controllers (6801/6081/9201) and we're seeing massive errors occuring. The usual pattern is a drive failing or even a resilver/scrub starting and then, suddenly, most drives on the whole backplane report errors. These are usually No Device (as reported by iostat -En), but the result is that we may see data corruption at the end. We also have a system setup with Hitachi Deskstars, which has been running for almost a year without issues. One system with a mixture of WD Blacks and greens showed the same errors as described, but has been working well after the WD drives were replaced by deskstars. Now, it seems WD has changed their firmware to inhibit people from using them for other things than toys (read: PCs etc). Since we've seen this issue on different controllers and different drives, and can't reproduce it with Hitachi Deskstars, I would guess the firmware "upgrade" from WD is the issue. Would it be possible to fix this in ZFS somehow? The drives seem to work well except for those "No Device" errors.... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [OpenIndiana-discuss] Question about ZFS/CIFS
> > We've migrated from an old samba installation to a new box with > > openindiana, and it works well, but... It seems Windows now honours > > the executable bit, so that .exe files for installing packages, are > > no longer directly executable. While it is positive that windows > > honours this bit, it breaks things when we have a software > > repository on this server. > > > > Does anyone know a way to counter this without chmod -R o+x? > > Does setting the aclinherit=passthrough-x zfs property on the > filesystem help? > > I'm not sure, but you may still need to do a chmod -R on each > filesystem to set the ACLs on each existing directory. Setting aclinherit didn't help much. It seems +x isn't inherited to files, only dirs. I found new files are created with the correct permissions, so I just chmod +x the lot... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Possible ZFS problem
> GNU tar does not follow the standard when creating archives, so Sun > tar may be unable to unpack the archive correctly. > > But GNU tar makes strange things when unpacking symlinks. > > I recommend to use star, it understands GNU tar archives. Even if you used some wierd tar program, the I/O pattern would be the same. Blaming userspace for kernel issues is nonesense -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [OpenIndiana-discuss] Question about ZFS/CIFS
> Does setting the aclinherit=passthrough-x zfs property on the > filesystem help? I'll try that once I'm back at the office - thanks! > I'm not sure, but you may still need to do a chmod -R on each > filesystem to set the ACLs on each existing directory. I'll try the one above first... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Question about ZFS/CIFS
Hi all We've migrated from an old samba installation to a new box with openindiana, and it works well, but... It seems Windows now honours the executable bit, so that .exe files for installing packages, are no longer directly executable. While it is positive that windows honours this bit, it breaks things when we have a software repository on this server. Does anyone know a way to counter this without chmod -R o+x? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] unable to mount zfs file system..pl help
> # uname -a > > Linux testbox 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT 2010 > x86_64 x86_64 x86_64 GNU/Linux > > # rpm -qa|grep zfs > zfs-test-0.5.2-1 > zfs-modules-0.5.2-1_2.6.18_194.el5 > zfs-0.5.2-1 > zfs-modules-devel-0.5.2-1_2.6.18_194.el5 > zfs-devel-0.5.2-1 This smells like an old redhat/centos/fedora installation. Use something newer if you want ZFS to work well Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS read errors
What sort of controller/backplane/etc are you using? I've seen similar iostat output with western drives on a supermicro SAS expander roy - Original Message - > Also should be getting Illegal Request errors? (no hard or soft > errors) > > Some more info: (I am doing a Scrub hence the high blocking levels) > > var/log$ iostat -Ex > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 1.1 16.9 30.6 463.0 0.2 0.0 10.6 1 1 > sd1 1.0 16.9 30.3 463.0 0.2 0.0 13.5 2 2 > sd2 208.7 4.8 14493.8 16.0 3.0 0.5 16.5 45 48 > sd3 212.4 4.8 14493.4 16.0 2.6 0.4 13.8 41 44 > sd4 221.9 4.8 14491.9 16.0 0.0 1.8 8.1 0 46 > sd5 212.3 4.8 14493.5 16.0 2.5 0.4 13.4 41 44 > sd6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd8 231.7 4.8 14692.7 16.3 0.0 1.8 7.5 0 42 > sd9 239.9 4.7 14691.7 16.3 0.0 1.2 5.1 0 36 > sd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 > Vendor: ATA Product: WDC WD2500SD-01K Revision: 2D08 Serial No: > Size: 250.06GB <250059350016 bytes> > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 > Illegal Request: 3 Predictive Failure Analysis: 0 > sd1 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 > Vendor: ATA Product: WDC WD2500JD-75G Revision: 5D02 Serial No: > Size: 250.00GB <2500 bytes> > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 > Illegal Request: 4 Predictive Failure Analysis: 0 > sd2 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 > Vendor: ATA Product: WDC WD3200SD-01K Revision: 5J08 Serial No: > Size: 320.07GB <320072933376 bytes> > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 > Illegal Request: 2 Predictive Failure Analysis: 0 > sd3 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 > Vendor: ATA Product: WDC WD2500SD-01K Revision: 2D08 Serial No: > Size: 250.06GB <250059350016 bytes> > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 > Illegal Request: 2 Predictive Failure Analysis: 0 > sd4 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 > Vendor: ATA Product: WDC WD5000YS-01M Revision: 2E07 Serial No: > Size: 500.11GB <500107862016 bytes> > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 > Illegal Request: 2 Predictive Failure Analysis: 0 > sd5 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 > Vendor: ATA Product: WDC WD2500SD-01K Revision: 2D08 Serial No: > Size: 250.06GB <250059350016 bytes> > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 > Illegal Request: 2 Predictive Failure Analysis: 0 > sd6 Soft Errors: 0 Hard Errors: 5 Transport Errors: 0 > Vendor: CREATIVE Product: DVD-ROM DVD1243E Revision: IC01 Serial No: > Size: 0.00GB <0 bytes> > Media Error: 0 Device Not Ready: 5 No Device: 0 Recoverable: 0 > Illegal Request: 0 Predictive Failure Analysis: 0 > sd8 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 > Vendor: ATA Product: WDC WD5000AACS-0 Revision: 1B01 Serial No: > Size: 500.11GB <500106780160 bytes> > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 > Illegal Request: 7 Predictive Failure Analysis: 0 > sd9 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 > Vendor: ATA Product: ST31000340AS Revision: SD15 Serial No: > Size: 1000.20GB <1000204886016 bytes> > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 > Illegal Request: 7 Predictive Failure Analysis: 0 > /var/log$ > -- > This message posted from opensolaris.org > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Issues with supermicro
then create a ZVOL and share it over iSCSI and from the initiator host, run
some benchmarks. You'll never get good results from local tests. For that sort
of load, I'd guess a stripe of mirrors should be good. RAIDzN will probably be
rather bad
roy
- Original Message -
> This system is for serving VM images through iSCSI to roughly 30
> xenserver hosts. I would like to know what type of performance I can
> expect in the coming months as we grow this system out. We currently
> have 2 intel ssds mirrored for the zil and 2 intel ssds for the l2arc
> in a stripe. I am interested more in max throughput of the local
> storage at this point and time.
>
> On Wed, Aug 10, 2011 at 12:01 PM, Roy Sigurd Karlsbakk
> wrote:
> > What sort of load will this server be serving? sync or async writes?
> > what sort of reads? random i/o or sequential? if sequential, how
> > many streams/concurrent users? those are factors you need to
> > evaluate before running a test. A local test will usually be using
> > async i/o and a dd with only 4k blocksize is bound to be slow,
> > probably because of cpu overhead.
> >
> > roy
> >
> > - Original Message -
> >> Hello All,
> >> Sorry for the lack of information. Here is some answers to some
> >> questions:
> >> 1) createPool.sh:
> >> essentially can take 2 params, one is number of disks in pool, the
> >> second is either blank or mirrored, blank means number of disks in
> >> the
> >> pool i.e. raid 0, mirrored makes 2 disk mirrors.
> >>
> >> #!/bin/sh
> >> disks=( `cat diskList | grep Hitachi | awk '{print $2}' | tr '\n' '
> >> '`
> >> )
> >> #echo ${disks[1]}
> >> #$useDisks=" "
> >> for (( i = 0; i < $1; i++ ))
> >> do
> >> #echo "Thus far: "$useDisks
> >> if [ "$2" = "mirrored" ]
> >> then
> >> if [ $(($i % 2)) -eq 0 ]
> >> then
> >> useDisks="$useDisks mirror ${disks[i]}"
> >> else
> >> useDisks=$useDisks" "${disks[i]}
> >> fi
> >> else
> >> useDisks=$useDisks" "${disks[i]}
> >> fi
> >>
> >> if [ $(($i - $1)) -le 2 ]
> >> then
> >> echo "spares are: ${disks[i]}"
> >> fi
> >> done
> >>
> >> #echo $useDisks
> >> zpool create -f fooPool0 $useDisks
> >>
> >>
> >>
> >> 2) hardware:
> >> Each server attached to each storage array is a dell r710 with 32
> >> GB
> >> memory each. To test for issues with another platform the below
> >> info,
> >> is from a dell 1950 server with 8GB memory. However, I see similar
> >> results from the r710s as well.
> >>
> >>
> >> 3) In order to deal with caching, I am writing larger amounts of
> >> data
> >> to the disk then I have memory for.
> >>
> >> 4) I have tested with bonnie++ as well and here are the results, i
> >> have read that it is best to test with 4x the amount of memory:
> >> /usr/local/sbin/bonnie++ -s 32000 -d /fooPool0/test -u gdurham
> >> Using uid:101, gid:10.
> >> Writing with putc()...done
> >> Writing intelligently...done
> >> Rewriting...done
> >> Reading with getc()...done
> >> Reading intelligently...done
> >> start 'em...done...done...done...
> >> Create files in sequential order...done.
> >> Stat files in sequential order...done.
> >> Delete files in sequential order...done.
> >> Create files in random order...done.
> >> Stat files in random order...done.
> >> Delete files in random order...done.
> >> Version 1.03d --Sequential Output-- --Sequential Input-
> >> --Random-
> >> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
> >> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
> >> %CP
> >> cm-srfe03 32000M 230482 97 477644 76 223687 44 209868 91 541182
> >> 41 1900 5
> >> --Sequential Create-- Random Create
> >> -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> >> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
> >> 16 29126 100 + +++ + +++ 24761 100 + +++ + +++
> >> cm-srfe03,32000M,230482,97,477644,76,223687,44,209868,91,541182,41,1899.7,5,16,29126,100,+,+++,+,+++,24761,100,+,+++,+,+++
> >>
> >>
> >> I will run these with th
Re: [zfs-discuss] Issues with supermicro
in the front, 0 in the back. > >> Created a stripe of different numbers of disks. After each test, I > >> destroy the underlying storage volume and create a new one. As you > >> can > >> see by the results, adding more disks, makes no difference to the > >> performance. This should make a large difference from 4 disks to 8 > >> disks, however no difference is shown. > >> > >> Any help would be greatly appreciated! > >> > >> This is the result: > >> > >> root@cm-srfe03:/home/gdurham~# time dd if=/dev/zero > >> of=/fooPool0/86gb.tst bs=4096 count=20971520 > >> ^C3503681+0 records in > >> 3503681+0 records out > >> 14351077376 bytes (14 GB) copied, 39.3747 s, 364 MB/s > > > > So, the problem here is that you're not testing the storage at all. > > You're basically measuring dd. > > > > To get meaningful results, you need to do two things: > > > > First, run it for long enough so you eliminate any write cache > > effects. Writes go to memory and only get sent to disk in the > > background. > > > > Second, use a proper benchmark suite, and one that isn't itself > > a bottleneck. Something like vdbench, although there are others. > > > > -- > > -Peter Tribble > > http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ > > ___ > > zfs-discuss mailing list > > zfs-discuss@opensolaris.org > > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > > > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Large scale performance query
> The hardware ist SM-Board, Xeon, 16 GB reg RAM, LSI 9211-8i HBA, > 6 x Hitachi 2TB Deskstar 5K3000 HDS5C3020ALA632. > Server is standing in the basement by 32°C > The HDs are filled to 80% and the workload ist only most reading. > > Whats the best? Scrubbing every week, every second week once a month? Generally, you can't scrub too often. If you have a set of striped mirrors, the scrub shouldn't take too long. The extra stress on the drives during scrub shouldn't matter much, drives are made to be used. By the way, 32˚C is a bit high for most servers. Could you check the drive temperature with smartctl or ipmi tools? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [vserver] hybrid zfs pools as iSCSI targets for vserver
> > 1) is this a good idea? > > > > 2) any of you are running vserver guests on iSCSI targets? Happy > > with it? > > > Yes, we have been using iSCSI to hold vserver guests for a couple of > years now and are generally unhappy with it. Besides our general > distress at Nexenta, there is the constraint of the Linux file system. > > Someone please correct me if I'm wrong because this is a big problem > for > us. As far as I know, Linux file system block size cannot exceed the > maximum memory page size and is limited to no more than 4KB. iSCSI > appears to acknowledge every individual block that is sent. That means > the most data one can stream without an ACK is 4KB. That means the > throughput is limited by the latency of the network rather than the > bandwidth. Even if Linux filesystems generally stick to a block size of 4kB, that doesn't mean all transfers are maximum 4kB. If that would have been the case, Linux would be quite useless for a server. I/O operations are queued and if, for instance, a read() call requests 8MB, that's done in a single operation. > Nexenta is built on OpenSolaris and has a significantly higher > internal > network latency than Linux. It is not unusual for us to see round trip > times from host to Nexenta well upwards of 100us (micro-seconds). > Let's > say it was even as good as 100us. One could send up to 10,000 packets > per second * 4KB = 40MBps maximum throughput for any one iSCSI > conversation. That's pretty lousy disk throughput. That's why, back in 1992, the sliding window protocol was created (http://tools.ietf.org/html/rfc1323), so that a peer won't wait for a TCP ACK before resuming operation. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Question about WD drives with Super Micro systems
> I'm using 4 x WD RE3 1TB drives with a Supermicro X7SB3 mobo with a > builtin LSI 1068E controller and a CSE-SAS-833TQ SAS backplane. > > Have run ZFS with both Solaris and FreeBSD without a problem for a > couple years now. Had one drive go bad, but it was caught early by > running periodic scrubs. We have some 180 drives on these systems, with 1068-based chipsets, and it seems problems arouse when traffic is high. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Large scale performance query
> How much time needs the thread opener with his config? > > Technical Specs: > > 216x 3TB 7k3000 HDDs > > 24x 9 drive RAIDZ3 > > I suggest resilver need weeks and the chance that a second or > third HD crashs in that time is high. Murphy’s Law With a full pool, perhaps a couple of weeks, but unless the pool is full (something that's strictly discouraged), a few days should do. I'm currently replacing WD drives on a server with 4 9-drive RAIDz2 VDEVs, and it takes about two days. A single drive replacement takes about 24 hours. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [OpenIndiana-discuss] Question about WD drives with Super Micro systems
> Might this be the SATA drives taking too long to reallocate bad > sectors? This is a common problem "desktop" drives have, they will > stop and basically focus on reallocating the bad sector as long as it > takes, which causes the raid setup to time out the operation and flag > the drive as failed. The "enterprise" sata drives, typically the same > as the high performing desktop drive, only they have a short timeout > on how long they are allowed to try and reallocate a bad sector so > they don't hit the failed drive timeout. Some drive firmwares, such as > older WD blacks if memory serves, had the ability to be forced to > behave like the enterprise drive, but WD updated the firmware so this > is longer possible. > > This is why you see SATA drives that typically have almost identical > specs, but one will be $69 and the other $139 - the former is a > "desktop" model while the latter is an "enterprise" or "raid" specific > model. I believe it's called different things by different brands: > TLER, ERC, and CCTL (?). I doubt this is about the lack of TLER et al. Some, or most, of the drives ditched by ZFS have shown to be quite good indeed. I guess this is a WD vs Intel SAS expanders issue Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Question about WD drives with Super Micro systems
> In my experience, SATA drives behind SAS expanders just don't work. > They "fail" in the manner you > describe, sooner or later. Use SAS and be happy. Funny thing is Hitachi and Seagate drives work stably, whereas WD drives tend to fail rather quickly Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Question about WD drives with Super Micro systems
Hi all We have a few servers with WD Black (and some green) drives on Super Micro systems. We've seen both drives work well with direct attach, but with LSI controllers and Super Micro's SAS expanders, well, that's another story. With those SAS expanders, we've seen numerous drives being kicked out and flagged as bad during high load (typically scrub/resilver). We have not seen this on the units we have with Hitachi or Seagate drives. After a drive is kicked out, we run a test on it, using WDs tool, and in many (or most) cases, we find the drive being error free. We've seen these issues on several machines, so hardware failure seem not to be the case. Have anyone here used WD drives with LSI controllers (3801/3081/9211) with Super Micro machines? Any success stories? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Wrong rpool used after reinstall!
> I am having a problem after a new install of Solaris 10. The installed > rpool > works fine when I have only those disks connected. When I connect > disks from > an rpool I created during a previous installation, my newly installed > rpool > is ignored even though the BIOS (x86) is set to boot only from the new > rpool > drives. When the system starts it uses the old rpool! How can I get > around > this? I want to use my new install and then import the old rpool under > a new > name, and clean things up. Boot on the install CD, import the old rpool (leave the new rpool drives unplugged) and export it again. Once you have rebooted on the new rpool, you can reimport the old rpool with a new name (rpool import someid newname). Keep in mind that reimporting an rpool may render it useless for booting, at least that's not optimal. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Large scale performance query
> Even with a controller per JBOD, you'll be limited by the SAS > connection. The 7k3000 has throughput from 115 - 150 MB/s, meaning > each of your JBODs will be capable of 5.2 GB/sec - 6.8 GB/sec, roughly > 10 times the bandwidth of a single SAS 6g connection. Use multipathing > if you can to increase the bandwidth to each JBOD. With (something like) LSI 9211 and those supermicro babies I guess he's planning on using, you'll have one quad-port SAS2 cable to each backplane/SAS expander, one in front and one in the back, meaning theroretical 24Gbps (or 2,4GBps) to each backplane. With a maximum of 24 drives per back, this should probably suffice, since you'll never get 150MB/s sustained from all drives. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Large scale performance query
> Workloads: > > Mainly streaming compressed data. That is, pulling compressed data in > a sequential manner however could have multiple streams happening at > once making it somewhat random. We are hoping to have 5 clients pull > 500Mbit sustained. That shouldn't be much of a problem with that amount of drives. I have a couple of smaller setups with 11x7-drive raidz2, about 100TiB each, and even they can handle 2,5Gbps load. > Considerations: > > The main reason RAIDZ3 was chosen was so we can distribute the parity > across the JBOD enclosures. With this method even if an entire JBOD > enclosure is taken offline the data is still accessible. Sounds like a good idea to me. > How to manage the physical locations of such a vast number of drives? > I have read this ( > http://blogs.oracle.com/eschrock/entry/external_storage_enclosures_in_solaris > ) and am hoping some can shed some light if the SES2 enclosure > identification has worked for them? (enclosures are SES2) Which enclosures will you be using? From the data you've posted, it looks like SuperMicro, and AFAIK, the ones we have, don't support SES2. > What kind of performance would you expect from this setup? I know we > can multiple the base IOPS by 24 but what about max sequential > read/write? Parallell read/write from several clients will look like random I/O on the server. If bandwidth is crucial, use RAID1+0. Also, it looks to me you're planning to fill up all external bays with data drives - where do you plan to put the root? If you're looking at the SuperMicro SC847 line, there's indeed room for a couple of 2,5" drives inside, but the chassis is screwed tightly together and doesn't allow for opening during runtime. Also, those drives are placed in a rather awkward slot. If planning to use RAIDz, a couple of SSDs for the SLOG will help write performance a lot especially during scrub/resilver. For streaming, L2ARC won't be of much use, though. Finally, a few spares won't hurt even with redundancy levels as high as RAIDz3. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] recover raidz from fried server ??
Could you try to just boot up fbsd or linux on the box to see if zfs (native or fuse-based, respecively) can see the drives? roy - Original Message - > root@san:~# zdb -l /dev/dsk/c7t6d0s0 > cannot open '/dev/rdsk/c7t6d0s0': I/O error > root@san:~# zdb -l /dev/dsk/c7t6d0p1 > > LABEL 0 > > failed to unpack label 0 > > LABEL 1 > > failed to unpack label 1 > > LABEL 2 > > failed to unpack label 2 > > LABEL 3 > > failed to unpack label 3 > root@san:~# > -- > This message posted from opensolaris.org > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] recover raidz from fried server ??
> Hey experts: does OpenIndiana dev-151 support ZFSv31 too, in a manner > compatible with sol11x? OpenIndiana b151/Illumos would support zpool v31 if Oracle would release that code. Since that doesn't seem to happen, or haven't happend yet, the chances are rather low for b151 to get it. The current build is still at zpool v28, but with a bunch of fixes as compared to the old one from osol Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zil on multiple usb keys
If using two mirrors, you'll end up with a badly balanced pool. As it was, and possibly is, this will lead to a performance penalty when one VDEV is full (the 300GB VDEV). This write performance has reportedly been fixed in Illumos, but I don't know about S11ex. For OpenIndiana/Nexenta, the fix probably isn't in yet. roy - Original Message - > Ok, so, taking 2 300Gb disks, and 2 500Gb disks, and creating an 800Gb > mirrored striped thing is sounding like a bad idea... what about just > creating a pool of all disks, without using mirrors? I seen something > called "copies", which if i am reading correctly, will make sure a > number of copies of a file exist... Am i reading that correctly? If > this does work the way i think it works, then taking all 4 disks, and > making one large 1.6Tb pool, setting copies to 2, should, in theory, > create a poor mans pool with striping, right? > --Tiernan > On Mon, Jul 18, 2011 at 7:20 AM, Brandon High < bh...@freaks.com > > wrote: > > On Sun, Jul 17, 2011 at 12:13 PM, Edward Ned Harvey > > < opensolarisisdeadlongliveopensola...@nedharvey.com > wrote: > > > Actually, you can't do that. You can't make a vdev from other > > > vdev's, and when it comes to striping and mirroring your only > > > choice > > > is to do it the right way. > > > > > > If you were REALLY trying to go out of your way to do it wrong > > > somehow, I suppose you could probably make a zvol from a stripe, > > > and > > > then export it to yourself via iscsi, repeat with another zvol, > > > and > > > then mirror the two iscsi targets. ;-) You might even be able to > > > do > > > the same crazy thing with simply zvol's and no iscsi... But either > > > way you'd really be going out of your way to create a problem. ;-) > > The right way to do it, um, incorrectly is to create a striped > > device > > using SVM, and use that as a vdev for your pool. > > So yes, you could create two 800GB stripes, and use them to create a > > ZFS mirror. But it would be a really bad idea. > > -B > > -- > > Brandon High : bh...@freaks.com > > ___ > > zfs-discuss mailing list > > zfs-discuss@opensolaris.org > > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > -- > Tiernan O'Toole > blog.lotas-smartman.net > www.tiernanotoolephotography.com > www.the-hairy-one.com > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Summary: Dedup memory and performance (again, again)
> > From: Roy Sigurd Karlsbakk [mailto:r...@karlsbakk.net] > > Sent: Saturday, July 09, 2011 2:33 PM > > > > Could you test with some SSD SLOGs and see how well or bad the > > system > > performs? > > These are all async writes, so slog won't be used. Async writes that > have a single fflush() and fsync() at the end to ensure system > buffering is not skewing the results. Sorry, my bad, I meant L2ARC to help buffer the DDT Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Replacement disks for Sun X4500
> Oh - and as a final point - if you are planning to run Solaris on this > box, make sure they are not the 4KB sector disks, as at least in my > experience, their performance with ZFS is profoundly bad. Particularly > with all the metadata update stuff... Hitachi deskstar uses 512byte sectors Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Summary: Dedup memory and performance (again, again)
> When it's not cached, of course the read time was equal to the > original > write time. When it's cached, it goes 4x faster. Perhaps this is only > because I'm testing on a machine that has super fast storage... 11 > striped > SAS disks yielding 8Gbit/sec as compared to all-RAM which yielded > 31.2Gbit/sec. It seems in this case, RAM is only 4x faster than the > storage > itself... But I would have expected a couple orders of magnitude... So > perhaps my expectations are off, or the ARC itself simply incurs > overhead. > Either way, dedup is not to blame for obtaining merely 2x or 4x > performance > gain over the non-dedup equivalent. Could you test with some SSD SLOGs and see how well or bad the system performs? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Pure SSD Pool
> I have a dual xeon 64GB 1U server with two free 3.5" drive slots. I > also have a free PCI-E slot. > > I'm going to run a postgress database with a business intelligence > application. > > The database size is not really set. It will be between 250-500GB > running on Solaris 10 or b134. Running business critical stuff on b134 isn't what I'd recommend - no updates anymore - either use S10, or S11ex or perhaps openindiana. > My storage choices are > > 1. OCZ Z-Drive R2 which fits in a 1U PCI slot. The docs say it has a > raid controller built in. I don't know if that can be disabled. > 2. Mirrored OCZ Talos C Series 3.5" SAS drives. > 3. Mirrored OCZ SATA II 3.5" drives. > > I'm looking for comments on the above drives or recommendations on > other affordable drives running in a pure SSD pool. > > Also what drives do you run as a pure SSD pool? Most drives should work well for a pure SSD pool. I have a postgresql database on a linux box on a mirrored set of C300s. AFAIK ZFS doesn't yet support TRIM, so that can be an issue. Apart from that, it should work well. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] 700GB gone? "zfs list" and "df" differs!
try 'zfs list -t all' roy - Original Message - > PS. I do not have any snapshots: > > root@frasse:~# zfs list > NAME USED AVAIL REFER MOUNTPOINT > TempStorage 916G 45,1G 37,3G /mnt/TempStorage > TempStorage/Backup 799G 45,1G 177G /mnt/TempStorage/Backup > TempStorage/EmmasFolder 78,6G 45,1G 78,6G /mnt/TempStorage/EmmasFolder > TempStorage/Stuff 1,08G 45,1G 1,08G /mnt/TempStorage/Stuff > > > > root@frasse:/mnt/TempStorage# df -h > Filesystem Size Used Avail Use% Mounted on > TempStorage 83G 38G 46G 46% /mnt/TempStorage > TempStorage/Backup 223G 178G 46G 80% /mnt/TempStorage/Backup > TempStorage/EmmasFolder > 124G 79G 46G 64% /mnt/TempStorage/EmmasFolder > TempStorage/Stuff 47G 1,1G 46G 3% /mnt/TempStorage/Stuff > /dev/dsk/c8t5d0p3 98G 97G 1,6G 99% /mnt/Temporary > -- > This message posted from opensolaris.org > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Problems installing grub
Hi all One of the rpool drives on this server died the other day, so I got a replacement that was 1 cylinder larger (60798 vs 60797). Still, I tried prtvtoc /dev/rdsk/c7d0s2 | fmthard -s - /dev/rdsk/c6d0s2 zpool replace worked and the pool resilvered within a few minutes. Now, installing grub fails. root@prv-backup:~# installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c6d0s0 Partition 0 of the disk has an incorrect offset Unable to gather device information for /dev/rdsk/c6d0s0 Can thie be because of a bad partition table? Any idae how to fix this properly? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] 700GB gone? "zfs list" and "df" differs!
> NAME USED AVAIL REFER MOUNTPOINT > TempStorage 916G 45,1G 37,3G /mnt/TempStorage > TempStorage/Backup 799G 45,1G 177G /mnt/TempStorage/Backup < OBS! > 800GB! You probably have snapshots eating the remaining 620GB... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] 700GB gone?
> On Thu, Jun 30, 2011 at 11:40:53PM +0100, Andrew Gabriel wrote: > > On 06/30/11 08:50 PM, Orvar Korvar wrote: > >> I have a 1.5TB disk that has several partitions. One of them is > >> 900GB. Now I can only see 300GB. Where is the rest? Is there a > >> command I can do to reach the rest of the data? Will scrub help? > > > > Not much to go on - no one can answer this. > > > > How did you go about partitioning the disk? > > What does the fdisk partitioning look like (if its x86)? > > What does the VToC slice layout look like? > > What are you using each partition and slice for? > > What tells you that you can only see 300GB? > > Are you using 32-bit or 64-bit solaris? IIRC Solaris x86-32 can't even address 1,5TB, so it shouldn't show up at all, or as a 1TB drive. If he sees 300GB, there might be some slight overhead and the fact that 1TB as reported by drive producers equals 0.9TiB as reported by the OS (1TB = 10^12 bytes, 1TiB = 2^40 bytes). The 1.5TB drive will be shown as a 1.35 drive or so. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] replace zil drive
I'd guess removing the SLOG altogether might be the safest, so for this configuration logs mirror-7 ONLINE 0 0 0 c2t22d0 ONLINE 0 0 0 c2t23d0 ONLINE 0 0 0 just `zpool remove zwimming mirror-7`, run `cfgadm -a` to find the full device path, do `cfgadm -c unconfigure devpath`, replace the drive, run devfsadm for good measure, check if it's connected with cfgadm -a, and add the SLOG again. roy - Original Message - > I think that the least disruptive way would be to detach the HDD from > the ZIL > mirror and offline it, remove and replace with an SSD, and then attach > the > SSD to the ZIL to make it a mirror again. > Note that this would create a window of possible ZIL failure (and you > had > such a window already when the first SSD died), but the system > *should* > survive that (fall back to on-pool ZIL after a short timeout of the > dedicated > device) unless the power dies during this time. > - Исходное сообщение - > От: Carsten John > Дата: Monday, June 27, 2011 13:21 > Тема: [zfs-discuss] replace zil drive > Кому (To): zfs-discuss@opensolaris.org > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 > > > > Hello everybody, > > > > some time ago a SSD within a ZIL mirror died. As I had no SSD > > availableto replace it, I dropped in a normal SAS harddisk to > > rebuild the mirror. > > > > In the meantime I got the warranty replacement SSD. > > > > Now I'm wondering about the best option to replace the HDD with > > the SSD: > > > > 1. Remove the log mirror, put the new disk in place, add log mirror > > > > 2. Pull the HDD, forcing the mirror to fail, replace the HDD > > with the SSD > > > > Unfortunately I have no free slot in the JBOD available (want to > > keep > > the ZIL in the same JBAD as the rest of the pool): > > > > 3. Put additional temporary SAS HDD in free slot of different JBOD, > > replace the HDD in the ZIL mirror with temporary HDD, pull now > > unused > > HDD, use free slot for SSD, replace temporary HDD with SSD. > > > > > > > > Any suggestions? > > > > > > thx > > > > > > > > Carsten > > > > > > > > > > > > - -- > > Max Planck Institut fuer marine Mikrobiologie > > - - Network Administration - > > Celsiustr. 1 > > D-28359 Bremen > > Tel.: +49 421 2028568 > > Fax.: +49 421 2028565 > > PGP public key:http://www.mpi-bremen.de/Carsten_John.html > > -BEGIN PGP SIGNATURE- > > Version: GnuPG v1.4.10 (GNU/Linux) > > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > > > iEYEARECAAYFAk4ISy8ACgkQsRCwZeehufs9MQCfetuYQwjbqH2Rb7qyY8G4vxaQ > > TvUAoNcHPnHED1Ykat8VHF8EJIRiPmct > > =jwZQ > > -END PGP SIGNATURE- > > ___ > > zfs-discuss mailing list > > zfs-discuss@opensolaris.org > > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > -- > ++ > | | > | Климов Евгений, Jim Klimov | > | технический директор CTO | > | ЗАО "ЦОС и ВТ" JSC COS&HT | > | | > | +7-903-7705859 (cellular) mailto:jimkli...@cos.ru | > | CC:ad...@cos.ru,jimkli...@gmail.com | > ++ > | () ascii ribbon campaign - against html mail | > | /\ - against microsoft attachments | > ++ > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] recovering from "zfs destroy -r"
- Original Message - > Hi, > Is there a simple way of rolling back to a specific TXG of a volume to > recover from such a situation? You can't undo a zfs destroy - restore from backup... -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Cannot format 2.5TB ext disk (EFI)
> I cannot run format -e to change it since it will crash my sys or > the server I am trying to attach the disk to. Did you try to do as Jim Dunham said? zpool create test_pool c5t0d0p0 zpool destroy test_pool format -e c5t0d0p0 partition print D -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Question about drive LEDs
Hi all I have a few machines setup with OI 148, and I can't make the LEDs on the drives work when something goes bad. The chassies are supermicro ones, and work well, normally. Any idea how to make drive LEDs wirk with this setup? -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] # disks per vdev
> I have decided to bite the bullet and change to 2TB disks now rather > than go through all the effort using 1TB disks and then maybe changing > in 6-12 months time or whatever. The price difference between 1TB and > 2TB disks is marginal and I can always re-sell my 6x 1TB disks. > > I think I have also narrowed down the raid config to these 4; > > 2x 7 disk raid-z2 with 1 hot spare - 20TB usable > 3x 5 disk raid-z2 with 0 hot spare - 18TB usable > 2x 6 disk raid-z2 with 2 hot spares - 16TB usable > > with option 1 probably being preferred at the moment. I would choose option 1. I have similar configurations in production. A hot spare can be very good when a drive dies while you're not watching. > I am aware that bad batches of disks do exist so I tend to either a) > buy them in sets from different suppliers or b) use different > manufacturers. How sensitive to different disks is ZFS, in terms of > disk features (NCQ, RPM speed, firmware/software versions, cache etc). For a home server, it shouldn't make much difference - the network is likely to be the bottleneck anyway. If you choose drives with different spin rate in a pool/vdev, the lower ones will probably pull down performance, so if you're considering "green" drives, you should use that for all the drives. Mixing Seagate, Samsung and Western drives should work well for this. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] OpenIndiana | ZFS | scrub | network | awful slow
> System1 (inhouse) > > SuperMicro-enclosure 2U SC825TQ > Mainboard: X8DTH-IF > 1 x 1 x Quad-Core Intel Xeon E5620 processor, 2.4GHz, 12MB L3 Cache > 24GB of RAM (3 x 8GB) > 1 x LSISAS9211-8I (for the internal 8 drive-carriers) > 1 x LSISAS9200-8E (for the JBOD) > > attached is a > 1 x SC847E16-RJBOD1 with two backplanes (each backplane is connected > to one port of the LSI-controller). You mention further down that you had dedup turned on for some time. Could you do a "zpool list" and report the dedup amount? Dedup is known to require a LOT of memory and/or L2ARC, and 24GB isn't really much with 34TBs of data. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] # disks per vdev
> 3x 5 disk raid-z. 3 disk failures in the right scenario, 12TB storage > 2x 7 disk raid-z + hot spare. 2 disk failures in the right scenario, > 12TB storage > 1x 15 disk raid-z2. 2 disk failures, 13TB storage > 2x 7 disk raid-z2 + hot spare. 4 disk failures in the right scenario, > 10TB storage If paranoid, use two RAIDz2 VDEVs and a spare. If not, use a single RAIDz2 or RAIDz3 VDEV with 14-15 drives and 1-2 spares. If you choose two VDEVs, replacing the drives in one of them with bigger ones as the pool grows will be more flexible, but may lead to badly balanced pools (although I just saw that fixed in Illumos/openindiana - dunno about s11ex, fbsd or other platforms). Personally, I'm a bit paranoid, and prefer to use smaller VDEVs. With 7 drives per VDEV in RAIDz2, and a spare, you may still have sufficient space for some time. If this isn't backed up somewhere else, I'd be a wee bit paranoid indeed :) Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS for Linux?
> Are there estimates on how performant and stable would > it be to run VirtualBox with a Solaris-derived NAS with > dedicated hardware disks, and use that from the same > desktop? I did actually suggest this as a considered > variant as well ;) > > I am going to try and build such a VirtualBox for my ailing > HomeNAS as well - so it would import that iSCSI "dcpool" > and try to process its defer-free blocks. At least if the > hardware box doesn't stall so that a human has to be > around to go and push reset, this would be a more > viable solution for my repair-reboot cycles... If you want good performance and ZFS, I'd suggest using something like OpenIndiana or Solaris 11EX or perhaps FreeBSD for the host and VirtualBox for a linux guest if that's needed. Doing so, you'll get good I/O performance, and you can use the operating system or distro you like for the rest of the services. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Disk replacement need to scan full pool ?
> Hi, > > I want to replace some slow consumer drives with new edc re4 ones but > when I do a replace it needs to scan the full pool and not only that > disk set (or just the old drive) > > Is this normal ? (the speed is always slow in the start so thats not > what I am wondering about, but that it needs to scan all of my 18.7T > to replace one drive) I had a pool where some three VDEVs got full and replaced the 2TB (green) members with new and faster 3TB drives. To do this, I replaced one drive in each VDEV at a time. This worked well for me. Just make sure you have a backup in case the shit hits the fan Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Hard link space savings
> If anyone has any ideas be it ZFS based or any useful scripts that
> could help here, I am all ears.
Something like this one-liner will show what would be allocated by everything
if hardlinks weren't used:
# size=0; for i in `find . -type f -exec du {} \; | awk '{ print $1 }'`; do
size=$(( $size + $i )); done; echo $size
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 97542685
r...@karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er
et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av
idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og
relevante synonymer på norsk.
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [OpenIndiana-discuss] Another zfs issue
> > Yeah, this is a known problem. The DTL on the toplevel shows an > > outage, and is preventing the removal of the spare even though > > removing the spare won't make the outage worse. > > > > Unfortunately, for opensolaris anyway, there is no workaround. > > > > You could try doing a full scrub, replacing any disks that show > > errors, and waiting for the resilver to complete. That may clean up > > the DTL enough to detach the spare. > > I'm not familiar with the DTL term, but does this mean, in English, > that if data errors are found and are not correctable, they will stay? > If so, is there another workaround than upgrading to something? Is > this bug still in s11ex? Can someone please fill me in on how this problem can be fixed? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Another zfs issue
> Yeah, this is a known problem. The DTL on the toplevel shows an > outage, and is preventing the removal of the spare even though > removing the spare won't make the outage worse. > > Unfortunately, for opensolaris anyway, there is no workaround. > > You could try doing a full scrub, replacing any disks that show > errors, and waiting for the resilver to complete. That may clean up > the DTL enough to detach the spare. I'm not familiar with the DTL term, but does this mean, in English, that if data errors are found and are not correctable, they will stay? If so, is there another workaround than upgrading to something? Is this bug still in s11ex? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] changing vdev types
> Since you can't mix vdev types in a single pool, you'll have to create > a new pool. But you can use zfs send/recv to move the datasets, so > your mountpoints and other properties will be preserved. Last I checked, mixing different VDEV types in a pool was possible, but not really recommended. Try zpool add -f somedev to a raidz3 pool, and it will work, but break redundancy Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Another zfs issue
Hi all I have this pool that has been suffering from some bad backplanes etc. Currently it's showing up ok, but after a resilver, a spare is stuck. raidz2-5 ONLINE 0 0 4 c4t1d0 ONLINE 0 0 0 c4t2d0 ONLINE 1 0 0 c4t3d0 ONLINE 0 0 0 c4t4d0 ONLINE 0 0 0 spare-4ONLINE 0 0 0 c4t5d0 ONLINE 0 0 0 c4t44d0 ONLINE 0 0 0 c4t6d0 ONLINE 0 0 0 c4t7d0 ONLINE 0 0 0 So, the VDEV seems ok, the pool reports two data errors, which is sad, but not a showstopper, however, trying to detach the spare from that vdev doesn's seem to easy roy@dmz-backup:~$ sudo zpool detach dbpool c4t44d0 cannot detach c4t44d0: no valid replicas iostat -en shows some issues with drives in that pool, but none on the two in the spare mirror 0 0 0 0 c4t1d0 0 82 131 213 c4t2d0 0 0 0 0 c4t3d0 0 0 0 0 c4t4d0 0 0 0 0 c4t5d0 0 0 0 0 c4t6d0 0 0 0 0 c4t7d0 0 0 0 0 c4t44d0 Is there a good explaination why I can't detach this mirror from the VDEV? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Is another drive worth anything?
> Hi. I have a development system on Intel commodity hardware with a > 500G ZFS > root mirror. I have another 500G drive same as the other two. Is there > any > way to use this disk to good advantage in this box? I don't think I > need any > more redundancy, I would like to increase performance if possible. I > have > only one SATA port left so I can only use 3 drives total unless I buy > a PCI > card. Would you please advise me. Many thanks. A third drive in the mirror (aka three-way mirror) will increase read performance from the pool, as ZFS reads from all drives in a mirror. Theoretically, you'll get a 50% read increase, but I doubt it'll be that high in practice. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] optimal layout for 8x 1 TByte SATA (consumer)
> > And if the ZFS is supposedly smart enough to use request coalescing > > as to minimize mechanical seek times, then it might actually be > > possible that your disks would get "stuck" averagely serving > > requests > > from different parts of the platter, i.e. middle-inside and > > middle-outside > > and this might even be averagely more than 2x faster than a single > > drive (due to non-zero track-to-track seek times). > > In practice I've just found out I'm completely CPU-bound. > Load goes to >11 during scrub, dd a large file causes ssh > to crap out, etc. Completely unusable, in other words. That's I/O-bound, not CPU-bound. The CPU load is likely to be rather low during scrub, but the I/O load will be high. IIRC this is tunable, at least in Solaris and friends. Also, adding SLOG/L2ARC will help a lot to make the system more resposible during scrub/resilver, again, at least on Solaris and friends. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] optimal layout for 8x 1 TByte SATA (consumer)
> How bad would raidz2 do on mostly sequential writes and reads > (Athlon64 single-core, 4 GByte RAM, FreeBSD 8.2)? > > The best way is to go is striping mirrored pools, right? > I'm worried about losing the two "wrong" drives out of 8. > These are all 7200.11 Seagates, refurbished. I'd scrub > once a week, that'd probably suck on raidz2, too? I see no problems with that. I've had rather large pools in production with 'commodity' drives without much issues. Every now and then a drive fails, but then, no pool failures yet. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS issues and the choice of platform
Hi all I have a few servers running openindiana 148, and it's been running rather well for some time. Lately, however, we've seen some hichups that may be related to the platform, rather than the hardware. The actual errors have been variable. Some issues were due to some supermicro backplanes that tended to fail, causing drives to report massive i/o errors. But then, the really bad ones are issues where zfs reports bad drives even though iostat report them as good. So far, we haven't lost a pool, it has been sorted out, but I still wonder what happens if I'm gone for a few weeks and something like that happens. The systems where we have had issues, are two 100TB boxes, with some 160TB "raw" storage each, so licensing this with nexentastor will be rather expensive. What would you suggest? Will a solaris express install give us good support when the shit hits the fan? -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [OpenIndiana-discuss] zpool lost, and no pools available?
> > I just attended this HTC conference and had a chat with a guy from > > UiO (university of oslo) about ZFS. He claimed Solaris/OI will die > > silently if a single pool fails. I have seen similar earlier, then > > due to a bug in ZFS (two drives lost in a RAIDz2, spares taking > > over, resilvering and then a third drive lost), and the system is > > hanging. Not even the rpool seems to be available. > > > > Can someone please confirm this or tell me if there is a workaround > > available? > > man zpool /failmode You may want to RTFM yourself befor replying. The docs say standard procedure is to put the pool into wait, which is ok, but the problem is that not only the pool in question is put into wait, but all pools. Please refrain from RTFMing people before digging into the material Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Bad pool...
> > The system became non-responsible after two drives was lost, and > > replaced with spares, in that VDEV. That bug has been filed and > > acknowleged. Take a RAIDz2 with two spares and remove a drive from > > the pool, let it resilver to a spare, remove another, wait until it > > resilvers again, and remove the third. The system will become rather > > dead - even the rpool will be unavailable, even if both the data > > pool and the rpool are bothe theoretically healthy > > Can't say I've ever run into that situation. I'd suggest looking into > the pool failmode setting but that still wouldn't make a lot of sense. > Any idea why you are getting so many failures? CC:ing this to the appropriate lists As a first, the default is to let go of failed devices. I haven't tweaked that part, nor any part of the pool. If a drive failes, it should be replaced by a spare, and when a drive is replaced by a new one, the old "ghost" should disappear. Neither of this happens at times. It seems sometimes the zpool "forgets" a dead drive and let ihang. This may trigger the bug which turns a pool and indeed the system unusable (if two drives in a raidz2 are lost, but resilvererd, losing the third will hang the system). The remedy seemed to be to zpool detach the drives. Still, the bug(s) exist(s) to allow a system to be rendered unusable just with a few drives lost, long before the pool is lost. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Bad pool...
> > Shouldn't ZFS detach these automatically? It has done so earlier... > It is not supposed to- at least that I recall. Earlier replaces have gone well. One thing is spares, which I can understand somewhat, but dead drives should definetely be tossed off when replaced Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zpool lost, and no pools available?
Hi all I just attended this HTC conference and had a chat with a guy from UiO (university of oslo) about ZFS. He claimed Solaris/OI will die silently if a single pool fails. I have seen similar earlier, then due to a bug in ZFS (two drives lost in a RAIDz2, spares taking over, resilvering and then a third drive lost), and the system is hanging. Not even the rpool seems to be available. Can someone please confirm this or tell me if there is a workaround available? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Bad pool...
Hi all I have a rather large pool that has been a bit troublesome. We've lost some drives (WD Black), and though that should work out well, I now have a pool that doesn't look too healthy. http://paste.ubuntu.com/611973/ Two drives have been resilvered, but the old drives still stick. The drive that has died still hasn't been taken over by a spare, although the two spares show up as AVAIL. Anyone that know how I can fix this? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Extremely slow zpool scrub performance
> Running a zpool scrub on our production pool is showing a scrub rate > of about 400K/s. (When this pool was first set up we saw rates in the > MB/s range during a scrub). Usually, something like this is caused by a bad drive. Can you post iostat -en output? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ls reports incorrect file size
> > Hi. While doing a scan of disk usage, I noticed the following > > oddity. > > I have a directory of files (named file.dat for this example) that > > all > > appear as ~1.5GB when using 'ls -l', but that (correctly) appear as > > ~250KB > > files when using 'ls -s' or du commands: > > These are probably just sparse files. Nothing to be alarmed about. or very tightly compressed files (like those from a dd if=/dev/zero). du and ls -ls will show how much is stored - ls -l will show the theoretical file size. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Dedup and L2ARC memory requirements (again)
> And one of these: > Assertion failed: space_map_load(&msp->ms_map, &zdb_space_map_ops, > 0x0, > &msp->ms_smo, spa->spa_meta_objset) == 0, file ../zdb.c, line 1439, > function > zdb_leak_init > Abort (core dumped) > > I saved the core and ran again. This time it spewed "leaked space" > messages > for an hour, and completed. But the final result was physically > impossible > (it counted up 744k total blocks, which means something like 3Megs per > block > in my 2.39T used pool. I checked compressratio is 1.00x and I have no > compression.) > > I ran again. > > Still spewing messages. This can't be a good sign. > > Anyone know what it means, or what to do about it? IIRC it runs out of memory, not space. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Dedup and L2ARC memory requirements (again)
> Controls whether deduplication is in effect for a > dataset. The default value is off. The default checksum > used for deduplication is sha256 (subject to change). > > This is from b159. This was fletcher4 earlier, and still is in opensolaris/openindiana. Given a combination with verify (which I would use anyway, since there are always tiny chances of collisions), why would sha256 be a better choice? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Dedup and L2ARC memory requirements (again)
- Original Message - > On 04/25/11 11:55, Erik Trimble wrote: > > On 4/25/2011 8:20 AM, Edward Ned Harvey wrote: > > > And one more comment: Based on what's below, it seems that the DDT > > > gets stored on the cache device and also in RAM. Is that correct? > > > What > > > if you didn't have a cache device? Shouldn't it *always* be in > > > ram? > > > And doesn't the cache device get wiped every time you reboot? It > > > seems > > > to me like putting the DDT on the cache device would be harmful... > > > Is > > > that really how it is? > > Nope. The DDT is stored only in one place: cache device if present, > > /or/ RAM otherwise (technically, ARC, but that's in RAM). If a cache > > device is present, the DDT is stored there, BUT RAM also must store > > a > > basic lookup table for the DDT (yea, I know, a lookup table for a > > lookup table). > No, that's not true. The DDT is just like any other ZFS metadata and > can be split over the ARC, > cache device (L2ARC) and the main pool devices. An infrequently > referenced DDT block will get > evicted from the ARC to the L2ARC then evicted from the L2ARC. and with the default size of a zfs configuration's metadata being (ram size - 1GB) / 4, without tuning, and with 128kB blocks all over, you'll need some 5-6GB+ per terabyte stored. -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Dedup and L2ARC memory requirements (again)
> After modifications that I hope are corrections, I think the post > should look like this: > > The rule-of-thumb is 270 bytes/DDT entry, and 200 bytes of ARC for > every L2ARC entry. > > DDT doesn't count for this ARC space usage > > E.g.: I have 1TB of 4k blocks that are to be deduped, and it turns out > that I have about a 5:1 dedup ratio. I'd also like to see how much ARC > usage I eat up with a 160GB L2ARC. > > (1) How many entries are there in the DDT: > > 1TB of 4k blocks means there are 268million blocks. However, at a 5:1 > dedup ratio, I'm only actually storing 20% of that, so I have about 54 > million blocks. Thus, I need a DDT of about 270bytes * 54 million =~ > 14GB in size > > (2) My L2ARC is 160GB in size, but I'm using 14GB for the DDT. Thus, I > have 146GB free for use as a data cache. 146GB / 4k =~ 38 million > blocks can be stored in the > remaining L2ARC space. However, 38 million files takes up: 200bytes * > 38 million =~ 7GB of space in ARC. > > Thus, I better spec my system with (whatever base RAM for basic OS and > cache and application requirements) + 14G because of dedup + 7G > because of L2ARC. Thanks, but one more ting: Add some tuning parameters, such as "set zfs:zfs_arc_meta_limit = somevalue in /etc/system" to help zfs use more memory for its metadata (like the DDT), as it won't use more than (RAM-1GB)/4 by default Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool scrub on b123
> I'm going to replace c9t15d0 with a new drive. > > I find it odd that zfs needed to resilver the drive after the reboot. > Shouldn't the resilvered information be kept across reboots? the iostat data, as returned from iostat -en, are not kept over a reboot. I don't know if it's possible to keep them in zfs or otherwise. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool scrub on b123
> I'm going to wait until the scrub is complete before diving in some > more. > > I'm wondering if replacing the LSI SAS 3801E with an LSI SAS 9200-8e > might help too. I've seen similar errors with 3801 - seems to be SAS timeouts. Reboot the box and it'll probably work well again for a while. I replaced the 3801 with a 9200 and the problem was gone. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] trouble replacing spare disk
Hi I was a little short earlier today, but then... First of all, using 40+ drives in a single VDEV on RAIDz1 is a little like BASE jumping with an old, round, parachute without a reserve, under nasty weather conditions; not what I'd recommend. What you see below is 2 spares in use plus a spare that has been flagged in use, and then failed. With RAIDz1, you can lose a single drive, and as far as I can see, you are now down on two dead ones, meaning you've probably lost the pool. If you find a way to recover it, make a good backup. If you manage to back it up, or already have a backup, recreate the pool in smaller VDEVs, and preferably with RAIDz2. The ZFS Best Practices document at http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide is contains good reading about this and other subjects. roy - Original Message - Hi, I have a SunFire X4540 with 19TB in a RAID-Z configuration; here's my zpool status: pool: raid state: UNAVAIL status: One or more devices are faulted in response to IO failures. action: Make sure the affected devices are connected, then run 'zpool clear'. see: http://www.sun.com/msg/ZFS-8000-HC scrub: resilver in progress for 84h11m, 99.47% done, 0h27m to go config: NAME STATE READ WRITE CKSUM raid UNAVAIL 0 0 451 insufficient replicas raidz1 UNAVAIL 0 0 902 insufficient replicas c0t3d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 c2t3d0 ONLINE 0 0 0 c3t3d0 ONLINE 0 0 0 c4t3d0 UNAVAIL 472 94 0 cannot open c5t3d0 ONLINE 0 0 0 c0t7d0 ONLINE 0 0 0 c1t7d0 ONLINE 0 0 0 c2t7d0 ONLINE 0 0 0 c3t7d0 ONLINE 0 0 0 c0t2d0 ONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c2t2d0 ONLINE 0 0 0 c3t2d0 ONLINE 0 0 0 c4t2d0 ONLINE 0 0 0 c4t6d0 ONLINE 0 0 0 spare DEGRADED 7 0 66.8M c5t2d0 FAULTED 11 2 0 too many errors replacing DEGRADED 0 0 0 c5t7d0 FAULTED 13 0 0 too many errors c5t6d0 ONLINE 0 0 0 202G resilvered c0t6d0 ONLINE 0 0 0 c1t6d0 ONLINE 0 0 0 c2t6d0 ONLINE 0 0 0 c3t6d0 ONLINE 0 0 0 spare DEGRADED 0 0 0 c0t1d0 FAULTED 0 0 0 too many errors c4t7d0 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 c2t1d0 ONLINE 0 0 0 c3t1d0 ONLINE 0 0 0 c4t1d0 ONLINE 0 0 0 c4t5d0 ONLINE 0 0 0 c0t5d0 ONLINE 0 0 0 c1t5d0 ONLINE 0 0 0 c2t5d0 ONLINE 0 0 0 c3t5d0 ONLINE 0 0 0 c5t1d0 ONLINE 0 0 0 c5t5d0 ONLINE 0 0 0 c0t4d0 ONLINE 0 0 0 c2t0d0 ONLINE 0 0 0 c3t0d0 ONLINE 0 0 0 c4t0d0 ONLINE 0 0 0 c5t0d0 ONLINE 0 0 0 c1t4d0 ONLINE 0 0 0 c2t4d0 ONLINE 0 0 0 c3t4d0 ONLINE 0 0 0 c4t4d0 ONLINE 0 0 0 spares c4t7d0 INUSE currently in use c5t7d0 INUSE currently in use c5t6d0 INUSE currently in use c5t4d0 AVAIL errors: 911 data errors, use '-v' for a list It looks like the resilver has got stuck; Oracle have sent out a replacement disk today and are asking me to replace c5t7d0. If I am understanding the documentation correctly, I believe I need to do the following: zpool offline raid c5t7d0 cfgadm -c unconfigure c5::dsk/c5t7d0 before physically replacing the disk. However, I get the following messages when trying to do this: # zpool offline raid c5t7d0 cannot offline c5t7d0: device is reserved as a hot spare # cfgadm -c unconfigure c5::dsk/c5t7d0 cfgadm: Hardware specific failure: failed to unconfigure SCSI device: Device busy I also tried a detach: # zpool detach raid c5t7d0 cannot detach c5t7d0: pool I/O is currently suspended And I also tried using the last available spare to try and free up the disk I need to replace: # zpool replace raid c5t2d0 c5t4d0 Cannot replace c5t2d0 with c5t4d0: device has already been replaced with a spare I am new to ZFS, how would I go about safely removing the affected drive in the software, before physically replacing it? I'm also not sure at exactly which juncture to do a 'zpool clear' and 'zpool scrub'? I'd appreciate any guidance - thanks in advance, Mark Mark Mahabir Systems Manager, X-Ray and Observational Astronomy Dept. of Physics & Astronomy, University of Leicester, LE1 7RH Tel: +44(0)116 252 5652 email: mark.maha...@leicester.ac.uk Elite Without Being Elitist Times Higher Awards Winner 2007, 2008, 2009, 2010 Follow us on Twitter http://twitter.com/uniofleicsnews ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] trouble replacing spare disk
Sorry, but what exactly were you thinking of when putting 40+ drives in a single RAIDz1 VDEV? roy - Original Message - Hi, I have a SunFire X4540 with 19TB in a RAID-Z configuration; here's my zpool status: pool: raid state: UNAVAIL status: One or more devices are faulted in response to IO failures. action: Make sure the affected devices are connected, then run 'zpool clear'. see: http://www.sun.com/msg/ZFS-8000-HC scrub: resilver in progress for 84h11m, 99.47% done, 0h27m to go config: NAME STATE READ WRITE CKSUM raid UNAVAIL 0 0 451 insufficient replicas raidz1 UNAVAIL 0 0 902 insufficient replicas c0t3d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 c2t3d0 ONLINE 0 0 0 c3t3d0 ONLINE 0 0 0 c4t3d0 UNAVAIL 472 94 0 cannot open c5t3d0 ONLINE 0 0 0 c0t7d0 ONLINE 0 0 0 c1t7d0 ONLINE 0 0 0 c2t7d0 ONLINE 0 0 0 c3t7d0 ONLINE 0 0 0 c0t2d0 ONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c2t2d0 ONLINE 0 0 0 c3t2d0 ONLINE 0 0 0 c4t2d0 ONLINE 0 0 0 c4t6d0 ONLINE 0 0 0 spare DEGRADED 7 0 66.8M c5t2d0 FAULTED 11 2 0 too many errors replacing DEGRADED 0 0 0 c5t7d0 FAULTED 13 0 0 too many errors c5t6d0 ONLINE 0 0 0 202G resilvered c0t6d0 ONLINE 0 0 0 c1t6d0 ONLINE 0 0 0 c2t6d0 ONLINE 0 0 0 c3t6d0 ONLINE 0 0 0 spare DEGRADED 0 0 0 c0t1d0 FAULTED 0 0 0 too many errors c4t7d0 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 c2t1d0 ONLINE 0 0 0 c3t1d0 ONLINE 0 0 0 c4t1d0 ONLINE 0 0 0 c4t5d0 ONLINE 0 0 0 c0t5d0 ONLINE 0 0 0 c1t5d0 ONLINE 0 0 0 c2t5d0 ONLINE 0 0 0 c3t5d0 ONLINE 0 0 0 c5t1d0 ONLINE 0 0 0 c5t5d0 ONLINE 0 0 0 c0t4d0 ONLINE 0 0 0 c2t0d0 ONLINE 0 0 0 c3t0d0 ONLINE 0 0 0 c4t0d0 ONLINE 0 0 0 c5t0d0 ONLINE 0 0 0 c1t4d0 ONLINE 0 0 0 c2t4d0 ONLINE 0 0 0 c3t4d0 ONLINE 0 0 0 c4t4d0 ONLINE 0 0 0 spares c4t7d0 INUSE currently in use c5t7d0 INUSE currently in use c5t6d0 INUSE currently in use c5t4d0 AVAIL errors: 911 data errors, use '-v' for a list It looks like the resilver has got stuck; Oracle have sent out a replacement disk today and are asking me to replace c5t7d0. If I am understanding the documentation correctly, I believe I need to do the following: zpool offline raid c5t7d0 cfgadm -c unconfigure c5::dsk/c5t7d0 before physically replacing the disk. However, I get the following messages when trying to do this: # zpool offline raid c5t7d0 cannot offline c5t7d0: device is reserved as a hot spare # cfgadm -c unconfigure c5::dsk/c5t7d0 cfgadm: Hardware specific failure: failed to unconfigure SCSI device: Device busy I also tried a detach: # zpool detach raid c5t7d0 cannot detach c5t7d0: pool I/O is currently suspended And I also tried using the last available spare to try and free up the disk I need to replace: # zpool replace raid c5t2d0 c5t4d0 Cannot replace c5t2d0 with c5t4d0: device has already been replaced with a spare I am new to ZFS, how would I go about safely removing the affected drive in the software, before physically replacing it? I'm also not sure at exactly which juncture to do a 'zpool clear' and 'zpool scrub'? I'd appreciate any guidance - thanks in advance, Mark Mark Mahabir Systems Manager, X-Ray and Observational Astronomy Dept. of Physics & Astronomy, University of Leicester, LE1 7RH Tel: +44(0)116 252 5652 email: mark.maha...@leicester.ac.uk Elite Without Being Elitist Times Higher Awards Winner 2007, 2008, 2009, 2010 Follow us on Twitter http://twitter.com/uniofleicsnews ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zpool resize
> > IIRC if you pass a DISK to "zpool create", it would create > > partition/slice on it, either with SMI (the default for rpool) or > > EFI > > (the default for other pool). When the disk size changes (like when > > you change LUN size on storage node side), you PROBABLY need to > > resize > > the partition/slice as well. > > > > When I test with openindiana b148, simply setting zpool set > > autoexpand=on is enough (I tested with Xen, and openinidiana reboot > > is > > required). Again, you might need to set both "autoexpand=on" and > > resize partition slice. > > > > As a first step, try choosing c2t1d0 in "format", and see what the > > size of this first slice is. > > > Hi, > > I choosed format and change type to the auto-configure and now I see > new > value if I choosed partition -> print, but when I exit from format and > reboot the old value is stay. How I can write new settings? Normally no change should be needed. Can you please paste the output from 'zpool status'? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zpool resize
> LUN is connected to solaris 10u9 from NETAP FAS2020a with ISCSI. I'm > changing LUN size on netapp and solaris format see new value but zpool > still have old value. > I tryed zpool export and zpool import but it didn't resolve my > problem. > > bash-3.00# format > Searching for disks...done > > > AVAILABLE DISK SELECTIONS: > 0. c0d1 > /pci@0,0/pci-ide@1,1/ide@0/cmdk@1,0 > 1. c2t1d0 > /iscsi/d...@iqn.1992-08.com.netapp%3Asn.13510595203E9,0 > Specify disk (enter its number): ^C > bash-3.00# zpool list > NAME SIZE ALLOC FREE CAP HEALTH ALTROOT > TEST 9,94G 93K 9,94G 0% ONLINE - > > What can I do that zpool show new value? Enable autoexpand. If you created a partition or slice on the drive and put the zpool onto that partition or slice, you'll need to change that partition's size. If you just used the whole device, autoexpand should work automatically once enabled (or after a few minutes - see my earlier post). If you can paste the output of 'zpool status', that should show if you're using the whole device or a slice. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zpool resize
> > I tried your suggestion, but no effect. > > Did you modify the partition table? > > IIRC if you pass a DISK to "zpool create", it would create > partition/slice on it, either with SMI (the default for rpool) or EFI > (the default for other pool). When the disk size changes (like when > you change LUN size on storage node side), you PROBABLY need to resize > the partition/slice as well. zpool create won't create a partition or slice, it'll just use the whole drive unless you give it a partition or slice. Last I expanded a pool, replacing 21 2TB drives with 3TB ones, it took some 2-3 minutes before the pool was being expanded, and then one VDEV at a time, so simply turning autoexpand on and then off may be a bit too quick. Leave it on for a while... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Cannot remove zil device
> You can add and remove mirrored or non-mirrored log devices. > > Jordan is probably running into CR 7000154: > > cannot remove log device I tried to google that, but couldn't find anything but this thread. Got a link to that bug? Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Cannot remove zil device
> http://pastebin.com/nD2r2qmh > > > Here is zpool status and zpool version The only thing I wonder about here, is why you have two striped log devices. I didn't even know that was supported. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Cannot remove zil device
> zpool remove storage c4t11d0 Can you pastebin 'zpool status' as well? PS: Get a decent email client - your quoting is severely broken :P Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Cannot remove zil device
> I have a pool that is experiencing the same problem (currently at > version 26). I've been able to remove zil devices from it before but > at some point they become "stuck". The zpool remove command completes > with no error, but the device is never really removed and remains in > the pool. > zpool iostat will show no activity to the device however, and the > alloc column shows 16.0E. It is possible to do a zpool replace on the > device, but the replacing device will end up in the same state > (unusable and unable to be removed). I had a similar problem that turned out to be a PEBKAC - I tried to remove devices in a mirror, not the mirror with the devices... Can you paste the command(s) you're using for this? Btw, log device removal is in from zpool v19, so the one you're using should work. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] A resilver record?
> > In my case we do not have the bandwidth to do a FULL send/recv, it > > would take weeks. > > I'm not 100% sure how the data would end up (whether de-fragmentation > would be achieved), but could you do a rend/receive to a new > filesystem on the same host? You could, but doing so on the same pool won't lead to defragmentation. With a new pool, though, it'll do the exact same as sending over the network. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs incremental send?
- Original Message - > On 2011-Mar-29 02:19:30 +0800, Roy Sigurd Karlsbakk > wrote: > >Is it (or will it) be possible to do a partial/resumable zfs > >send/receive? If having 30TB of data and only a gigabit link, such > >transfers takes a while, and if interrupted, will require a > >re-transmit of all the data. > > zfs send/receive works on snapshots: The smallest chunk of data that > can be sent/received is the delta between two snapshots. There's no > way to do a partial delta - defining the endpoint of a partial > transfer or the starting point for resumption is effectively a > snapshot. I know that's how it works, I'm merely pointing out that changing this to something resumable would be rather nice, since an initial transfer or 30 or 300 terabytes may easily be interrupted. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zfs incremental send?
Hi all I have a few boxes with rather large amounts of data on ZFS, and so far, it's been working quite well. Now, one little problem is mentioned every now and then. Is it (or will it) be possible to do a partial/resumable zfs send/receive? If having 30TB of data and only a gigabit link, such transfers takes a while, and if interrupted, will require a re-transmit of all the data. What could make a partial transfer work? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Any use for extra drives?
> Right, put some small (30GB or something trivial) disks in for root > and > then make a nice fast multi-spindle pool for your data. If your 320s > are around the same performance as your 500s, you could stripe and > mirror them all into a big pool. ZFS will waste the extra 180 on the > bigger disks but that's fine because your pool will bigger and faster > anyway. Actually, it won't. With two 500GB drives, it'll create a 500GB mirror even if you add that to a pool with 320GB mirrors. That way you will end up with a somewhat badly balanced pool, so once the 320GB mirrors are filled up, only the 500GB mirrors will be used. You can work around this by using partitions, but generally it's preferred to use drives same size and not partitions at all (IIRC ZFS turns off write cache by default when talking to a partition device, but then, don't shoot me if I'm wrong). Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Good benchmarking software?
> depending on the data you have you could use gnuplot to visualize. > Normally an X for time and Y for the data show enough. > I did this once with CPU and Memory usage. > RRD is also a nice Tool to visualize (most OpenSource Tools use it) > but for me gnuplot was the easier way to do it. > If you like you could send some example Reports. That would be brilliant :) Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Good benchmarking software?
Hi all I've been doing some testing on a test box to try to generate good performance reports. I've been trying bonnie++ and iozone, and while both give me lots and lots of statistics and numbers, I can't find a good way to visualize them. I see there are some spreadsheets avaliable for iozone, but I'm still rather confused. I'm not a statistics/math guy, I just want to show, clear and once and for all, the differences between a set of striped mirrors and a raidz2. Also, I want to visualize the performance gain by using L2ARC/SLOG on normal operation or during a resilver/scrub. So - question is quite simple - where can I find a good way to do this without too much hassle? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] A resilver record?
> Our main backups storage server has 3x 8-drive raidz2 vdevs. Was > replacing the 500 GB drives in one vdev with 1 TB drives. The last 2 > drives took just under 300 hours each. :( The first couple drives > took approx 150 hours each, and then it just started taking longer and > longer for each drive. That's strange indeed. I just replaced 21 drives (seven 2TB drives in three raidz2 VDEVs) drives with 3TB ones, and resilver times were quite stable, until the last replace, which was a bit faster. Have you checked 'iostat -en'? If one (or more) of the drives are having i/o errors, that may slow down the whole pool. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] A resilver record?
> The 30+ second latency I see on this system during a resilver renders > it pretty useless as a staging server (lots of small snapshots). I've seen similar numbers on a system during resilver, without L2ARC/SLOG. Adding L2ARC/SLOG made the system work quite well during resilver/scrub, but without them, it wasn't very useful. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] A resilver record?
> I think maybe the "number of drives in the vdev" perhaps come into > play because that when people have a lot of disks, they often put them > into RAIDZ[123] configurations. So it's just a matter of confusing the > (IOps limiting) configuration with the fact that one may have many > disks. My answer was not meant to be a generic one, but based on the original question, which was about a raidz2 VDEV, but then, thanks for the info Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] A resilver record?
> > It all depends on the number of drives in the VDEV(s), traffic > > patterns during resilver, speed VDEV fill, of drives etc. Still, > > close to 6 days is a lot. Can you detail your configuration? > > How many times do we have to rehash this? The speed of resilver is > dependent on the amount of data, the distribution of data on the > resilvering > device, speed of the resilvering device, and the throttle. It is NOT > dependent > on the number of drives in the vdev. Thanks for clearing this up - I've been told large VDEVs lead to long resilver times, but then, I guess that was wrong. Btw after replacing some 2TB drives with 3TB ones in three VDEVs that were 95% full at the time, resilver times dropped by 30%, so I guess very full VDEVs aren't much fun even on the resilver side. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] A resilver record?
> Has anyone seen a resilver longer than this for a 500G drive in a > riadz2 vdev? > > scrub: resilver completed after 169h25m with 0 errors on Sun Mar 20 > 19:57:37 2011 > c0t0d0 ONLINE 0 0 0 769G resilvered > > and I told the client it would take 3 to 4 days! It all depends on the number of drives in the VDEV(s), traffic patterns during resilver, speed VDEV fill, of drives etc. Still, close to 6 days is a lot. Can you detail your configuration? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] best migration path from Solaris 10
> I think we all feel the same pain with Oracle's purchase of Sun. > > FreeBSD that has commercial support for ZFS maybe? Fbsd currently has a very old zpool version, not suitable for running with SLOGs, since if you lose it, you may lose the pool, which isn't very amusing... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] GNU 'cp -p' can't work well with ZFS-based-NFS
/usr/bin/cp will work - the path on OpenIndiana has gnu in first, which is a bad thing imho - Original Message - Always show info like ‘operation not supported’. Any workaround? Thanks. Fred ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] cannot replace c10t0d0 with c10t0d0: device is too small
> You cannot. That's why I suggested two years ago that they chop off 1% > from the end of the disk at install time to equalize drive sizes. That > way you you wouldn't run into this problem trying to replace disks > from a different vendor or different batch. The response was that Sun > makes sure all drives are exactly the same size (although I do recall > someone on this forum having this issue with Sun OEM disks as well). We had a disk crash some months back on a Sun pizzabox - disks were mirrored (linux software raid), so no problem, except the new drive was a bit smaller than the original. This was a replacement part from Sun^WOracle... Since the machine was a compute node, I ended up reinstalling it instead of waiting for another replacement. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] cannot replace c10t0d0 with c10t0d0: device is too small
> > One comment: The IDEMA LBA01 spec size of a 160GB device is > > 312,581,808 sectors. > > > > Instead of those WD models, where neither the old nor new drives > > follow the IDEMA recommendation, consider buying a drive that > > reports > > that many sectors. Almost all models these days should be following > > the IDEMA recommendations due to all the troubles people have had. > > > > --eric > > > > -- > > Eric D. Mudama > > edmud...@bounceswoosh.org > > > > > Thats encouraging, if I have to I would rather buy one new disk then > 4. Get one that's a bit larger. It won't cost you a fortune. If the reseller is nice, you may even return the old one.. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [illumos-Developer] ZFS spare disk usage issue
So should I post a bug, or is there one there already? Btw, I can't reach http://bugs.illumos.org/ - it times out roy - Original Message - > We've talked about this, and I will be putting together a fix for this > incorrect state handling. :-) > > - Garrett > > On Fri, 2011-03-04 at 11:50 -0500, Eric Schrock wrote: > > This looks like a pretty simple bug. The issue is that the state of > > the SPARE vdev is being reported as REMOVED instead of DEGRADED. If > > it were the latter (as it should be), then everything would work > > just > > fine. Please file a bug at bugs.illumos.org. > > > > > > On a side note, this continues to expose the overly simplistic vdev > > state model used by ZFS (one which I can take a bulk of the > > responsibility for). Back before the days of ditto blocks and > > SPA3.0, > > it was sufficient to model state as a fairly binary proposition. But > > this now has ramifications that don't necessarily make sense. For > > example, one may be able open a pool even if a toplevel vdev is > > faulted. And even when a spare has finished resilvering, it's left > > in > > the DEGRADED state, which has implications for allocation policies > > (though I remember discussions around changing this). But the pool > > state is derived directly from the toplevel vdev state, so if you > > switch spares to be ONLINE, then 'zpool status' would think your > > pool > > is perfectly healthy. In this case it's true from a data protection > > standpoint, but not necessarily from a "all is well in the world" > > standpoint, as you are down one spare, and that spare may not have > > the > > same RAS properties as other devices in your RAID-Z stripe (it may > > put > > 3 disks on the same controller in one stripe, for example). > > > > > > - Eric > > > > On Fri, Mar 4, 2011 at 7:06 AM, Roy Sigurd Karlsbakk > > wrote: > > Hi all > > > > I just did a small test on RAIDz2 to check whether my > > suspicion was right about ZFS not treating spares as > > replicas/copies of drives, and I think I've found it true. > > The > > short story: If two spares replaces two drives in raidz2, > > losing a third drive, even with the spares active, makes the > > pool unavailable. See full report on > > > > ODT: http://karlsbakk.net/ZFS/ZFS%20Spare%20disk%20usage.odt > > PDF: http://karlsbakk.net/ZFS/ZFS%20Spare%20disk%20usage.pdf > > > > Vennlige hilsener / Best regards > > > > roy > > -- > > Roy Sigurd Karlsbakk > > (+47) 97542685 > > r...@karlsbakk.net > > http://blogg.karlsbakk.net/ > > -- > > I all pedagogikk er det essensielt at pensum presenteres > > intelligibelt. Det er et elementært imperativ for alle > > pedagoger å unngå eksessiv anvendelse av idiomer med fremmed > > opprinnelse. I de fleste tilfeller eksisterer adekvate og > > relevante synonymer på norsk. > > > > ___ > > Developer mailing list > > develo...@lists.illumos.org > > http://lists.illumos.org/m/listinfo/developer > > > > > > > > -- > > Eric Schrock > > Delphix > > > > > > 275 Middlefield Road, Suite 50 > > Menlo Park, CA 94025 > > http://www.delphix.com > > > > > > ___ > > Developer mailing list > > develo...@lists.illumos.org > > http://lists.illumos.org/m/listinfo/developer -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [illumos-Developer] ZFS spare disk usage issue
I understand that some of it may be a simple bug, but should it hang _all_ the pools? That's what happens when the third drive is removed... roy - Original Message - This looks like a pretty simple bug. The issue is that the state of the SPARE vdev is being reported as REMOVED instead of DEGRADED. If it were the latter (as it should be), then everything would work just fine. Please file a bug at bugs.illumos.org . On a side note, this continues to expose the overly simplistic vdev state model used by ZFS (one which I can take a bulk of the responsibility for). Back before the days of ditto blocks and SPA3.0, it was sufficient to model state as a fairly binary proposition. But this now has ramifications that don't necessarily make sense. For example, one may be able open a pool even if a toplevel vdev is faulted. And even when a spare has finished resilvering, it's left in the DEGRADED state, which has implications for allocation policies (though I remember discussions around changing this). But the pool state is derived directly from the toplevel vdev state, so if you switch spares to be ONLINE, then 'zpool status' would think your pool is perfectly healthy. In this case it's true from a data protection standpoint, but not necessarily from a "all is well in the world" standpoint, as you are down one spare, and that spare may not have the same RAS properties as other devices in your RAID-Z stripe (it may put 3 disks on the same controller in one stripe, for example). - Eric On Fri, Mar 4, 2011 at 7:06 AM, Roy Sigurd Karlsbakk < r...@karlsbakk.net > wrote: Hi all I just did a small test on RAIDz2 to check whether my suspicion was right about ZFS not treating spares as replicas/copies of drives, and I think I've found it true. The short story: If two spares replaces two drives in raidz2, losing a third drive, even with the spares active, makes the pool unavailable. See full report on ODT: http://karlsbakk.net/ZFS/ZFS%20Spare%20disk%20usage.odt PDF: http://karlsbakk.net/ZFS/ZFS%20Spare%20disk%20usage.pdf Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ Developer mailing list develo...@lists.illumos.org http://lists.illumos.org/m/listinfo/developer -- Eric Schrock Delphix 275 Middlefield Road, Suite 50 Menlo Park, CA 94025 http://www.delphix.com -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [illumos-Developer] ZFS spare disk usage issue
- Original Message - > Hi all > > I just did a small test on RAIDz2 to check whether my suspicion was > right about ZFS not treating spares as replicas/copies of drives, and > I think I've found it true. The short story: If two spares replaces > two drives in raidz2, losing a third drive, even with the spares > active, makes the pool unavailable. See full report on Update 2010-03-04 14:15 CET I just tested on another system. This one, not in production yet, has a mirrored rpool and a 14-drive RAID10 pool named tos-data. I started a copy from a Windows machine into this CIFS share just to generate some traffic. Then I did a zfs detach of one side of each of the mirrors for tos-data and created a new 5-drive raidz2 pool name jalla with two dedicated spares. I started a dd to fill it up and plugged one drive, waited for it to resilver and plugged another, again waited for the resilver to finish and plugged the third. The server now hangs on all pools. I've also tested removing drives from mirrors and waiting for them to resilver to spares. This seems to work as expected, although I doubt booting from one will work without grub being installed. > ODT: http://karlsbakk.net/ZFS/ZFS%20Spare%20disk%20usage.odt > PDF: http://karlsbakk.net/ZFS/ZFS%20Spare%20disk%20usage.pdf These are mow updated as well Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
