Re: [OmniOS-discuss] big zfs storage?

Thanks everyone who answered, very insightful. What scares me the most is hearing about the panics and FMA not having time to react at all, and also stories of sub-optimal multi hot-spare kicking into action like described by Chip. Recipe for disaster. I guess this is an area where Nexenta has

Re: [OmniOS-discuss] big zfs storage?

So... how does Nexenta copes with hot spares and all kinds of disk failures ? Adding hot spares is part of their administration manuals so can we assume things are almost always handled smoothly ? I'd like to hear from tangible experiences in production. thanks On Mon, Jul 13, 2015 at 7:58 AM,

Re: [OmniOS-discuss] big zfs storage?

> On Oct 7, 2015, at 1:59 PM, Mick Burns wrote: > > So... how does Nexenta copes with hot spares and all kinds of disk failures ? > Adding hot spares is part of their administration manuals so can we > assume things are almost always handled smoothly ? I'd like to hear >

Re: [OmniOS-discuss] big zfs storage?

> I completely concur with Richard on this. Let me give an a real example > that emphases this point as it's a critical design decision. [...] > Now I only run one hot spare per pool. Most of my pools are raidz2 or > raidz3. This way any event like this can not take out more than one > disk and

Re: [OmniOS-discuss] big zfs storage?

Liam, This report is encouraging. Please share some details of your configuration. What disk failure parameters are have you set? Which JBODs and disks are you running? I have mostly DataON JBODs and a some Supermicro. DataON has PMC SAS expanders and Supermicro has LSI, both setups have

Re: [OmniOS-discuss] big zfs storage?

This really saddens me. To me my favorite part of solaris was FMA. But happy about the 3008. thanks linda On 7/10/15 12:31 PM, Schweiss, Chip wrote: Unfortunately for the past couple years panics on disk failure has been the norm. All my production systems are HA with RSF-1, so at least

Re: [OmniOS-discuss] big zfs storage?

FWIW I have not had panics on disk failure. The pool will hang, and often not recover until a reboot -- depending on the disk failure mode. I blame sd for this rather than ZFS or FMA, but at the end of the day it is still suboptimal behavior. -- bdha On Jul 11, 2015, at 16:11, Linda Kateley

Re: [OmniOS-discuss] big zfs storage?

I have two 800T ZFS systems on OmniOS and a bunch of smaller 50T systems. Things generally work very well. We loose a disk here and there but its never resulted in downtime. They're all on Dell hardware with LSI or Dell PERC controllers. Putting in smaller disk failure parameters, so disks fail

Re: [OmniOS-discuss] big zfs storage?

Unfortunately for the past couple years panics on disk failure has been the norm. All my production systems are HA with RSF-1, so at least things come back online relatively quick. There are quite a few open tickets in the Illumos bug tracker related to mpt_sas related panics. Most of the work

Re: [OmniOS-discuss] big zfs storage?

Linda, I have 3.5 PB running under OmniOS. All my systems have LSI 2108 HBAs which is considered the best choice for HBAs. Illumos leaves a bit to be desired with handling faults from disks or SAS problems, but things under OmniOS have been improving, much thanks to Dan McDonald and OmniTI.

[OmniOS-discuss] big zfs storage?

Hey is there anyone out there running big zfs on omni? I have been doing mostly zol and freebsd for the last year but have to build a 300+TB box and i want to come back home to roots(solaris). Feeling kind of hesitant :) Also, if you had to do over, is there anything you would do different.

Re: [OmniOS-discuss] big zfs storage?

Hi Linda, I have two dual-headed OmniOS system attached to two 60 bay JBODs w/ 3TB drives. Using raidz2 and hot spares, each JBOD has about 106TB usable, and i use a single filesystem per JBOD. We may or may not add another JBOD, but we will be expanding into the PB realm eventually. I use