NFS writes (especially for lots of small files) to omniOS *really*
benefit from having ZIL on those SSD.
You could remove the cache from the pool, carve of an 8GB chunk for ZIL
on each and the rest for L2arc if you want that.
then add a mirrored zil using the 8GB chunks and the other partition for
l2arc
An SSD zil helps with metadata update absorption and small files writes
that are synchronous over NFS. A lot. (that's my experience)
On 8/23/2018 12:38 PM, Lee Damon wrote:
These are 12TB SAS drives (Seagate ST12000NM0027) for data & hot
spare. ZIL & L2ARC are 480GB INTEL SSDSC2KG48 SSDs. Everything is left
at default for sector size, etc. They were basically prepared for into
the pool with a simple fdisk -B /dev/rdsk/drive.
Ping never shows loss of connectivity. I ran this for about 5 minutes
during a test:
303 packets transmitted, 303 received, 0% packet loss, time 302021ms
rtt min/avg/max/mdev = 0.109/0.281/2.881/0.227 ms
CIFS, scp, and rsync do not exhibit the problem. I forgot to mention
that local copies on the file server are also as fast as I would
expect (~2 min).
pool: pool0
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
pool0 ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
c0t5000C500A612DA93d0 ONLINE 0 0 0
c0t5000C500957D4A93d0 ONLINE 0 0 0
c0t5000C500957D4C1Bd0 ONLINE 0 0 0
c0t5000C500957D25B3d0 ONLINE 0 0 0
c0t5000C500957D27F3d0 ONLINE 0 0 0
c0t5000C500957D2553d0 ONLINE 0 0 0
logs
mirror-1 ONLINE 0 0 0
c0t55CD2E414EC0FF43d0s0 ONLINE 0 0 0
c3t0d0s0 ONLINE 0 0 0
cache
c0t55CD2E414EC0FF43d0s1 ONLINE 0 0 0
c3t0d0s1 ONLINE 0 0 0
spares
c0t5000C50095722E27d0 AVAIL
iostat:
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
15.8 57.5 1302.5 5372.3 823.0 0.6 11233.2 8.4 9 9 pool0
0.1 19.9 1.7 163.0 0.0 0.0 1.0 0.1 0 0 rpool
0.1 10.2 0.9 81.5 0.0 0.0 0.0 0.0 0 0 c1t4d0
0.0 10.1 0.8 81.5 0.0 0.0 0.0 0.1 0 0 c1t5d0
2.7 19.5 337.8 1359.4 0.4 0.0 16.1 0.5 8 1 c3t0d0
1.9 6.0 114.5 442.2 0.0 0.0 0.0 5.0 0 1
c0t5000C500957D27F3d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0 0
c0t5000C50095722E27d0
1.5 6.0 86.2 442.9 0.0 0.0 0.0 4.7 0 1
c0t5000C500957D25B3d0
1.5 6.0 78.2 442.5 0.0 0.0 0.0 4.2 0 1
c0t5000C500957D4C1Bd0
1.7 6.1 102.4 442.8 0.0 0.0 0.0 4.7 0 1
c0t5000C500A612DA93d0
1.6 6.0 86.6 442.5 0.0 0.0 0.0 4.3 0 1
c0t5000C500957D2553d0
2.0 5.9 122.2 442.2 0.0 0.0 0.0 5.0 0 1
c0t5000C500957D4A93d0
2.9 19.5 374.7 1357.9 0.0 0.0 0.0 1.2 0 1
c0t55CD2E414EC0FF43d0
c1t4d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: INTEL SSDSC2KB24 Revision: 0121 Serial No:
BTYS817407RE240
Size: 240.06GB <240057409536 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c1t5d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: INTEL SSDSC2KB24 Revision: 0121 Serial No:
BTYS817409YS240
Size: 240.06GB <240057409536 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c3t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: INTEL SSDSC2KG48 Revision: 0121 Serial No:
BTYM7405027L480
Size: 480.10GB <480103981056 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c0t5000C500957D27F3d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
ZJV0VFGX0000J74
Size: 12000.14GB <12000138625024 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c0t5000C50095722E27d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
ZJV0S42H0000J75
Size: 12000.14GB <12000138625024 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c0t5000C500957D25B3d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
ZJV0VFJV0000J74
Size: 12000.14GB <12000138625024 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c0t5000C500957D4C1Bd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
ZJV0P6050000J83
Size: 12000.14GB <12000138625024 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c0t5000C500A612DA93d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
ZJV0WCCN0000J80
Size: 12000.14GB <12000138625024 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c0t5000C500957D2553d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
ZJV0VFK30000J74
Size: 12000.14GB <12000138625024 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c0t5000C500957D4A93d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
ZJV0VBQ80000R81
Size: 12000.14GB <12000138625024 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c0t55CD2E414EC0FF43d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: INTEL SSDSC2KG48 Revision: 0121 Serial No:
BTYM740600ZT480
Size: 480.10GB <480103981056 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
nomad
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
