[OmniOS-discuss] ZFS Slog - force all writes to go to Slog
hi all, I found an entry about zil_slog_limit here: http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWritesAndZILII it basically explains how writes larger than 1MB per default hits the main pool rather than my Slog device - I could not find much further information nor the equivalent setting in OmniOS. I also read http://nex7.blogspot.ca/2013/04/zfs-intent-log.html but it didn't truly help me understand just how I can force every written byte to my ZFS box to go the ZIL regardless of size, I never ever want anything to go directly to my man pool ever. I have sync=always and disabled write back cache on my volume based LU's. Testing with zfs_txg_timeout set to 30 or 60 seconds seems to make no difference if I write large files to my LU's - I don't seem the write speed being consistent with the performance of the Slog devices. It looks as if it goes straight to disk and hence the performance is less than great to say the least. How do I ensure 100% that all writes always goes to my Slog devices - no exceptions. br, Rune ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
On Wed, Feb 18, 2015 at 7:41 AM, Andy omn...@citrus-it.net wrote: I'd prefer to run Supermicro but might have some problems convincing those with the purse strings. Is anyone running, or can anyone recommend, a Supermicro server roughly equivalent to the Del R730xd, or give me an idea of what chipsets/HBAs etc. to choose or avoid for OmniOS? I've been quite happy with the 6028U-TR4+. It was the first 2U Ultra server Supermicro was shipping. Any of the 2U Ultra's would be a good choice. What I like about these in particular is lots of PCIe slots for HBAs and 10GB NICs. If your running RJ45 10GB they have versions with 10GB built in. If it were available at the time I would have went with the 2028U, since it has all 2 1/2 drives. The only drives I've put in the SATA side has been SSDs for boot and L2ARC. With an internal HBA you could load it up with 24 drives. I think with the on board SATA you are limited to 10 drives. -Chip Any help appreciated, Thanks Andy -- Citrus IT Limited | +44 (0)870 199 8000 | enquir...@citrus-it.co.uk Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ Registered in England and Wales | Company number 4899123 ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
In customers I've dealt with, if there isn't a binding data center policy in place for one or the other, it's usually a trade-off between TCO, and the differing field service offerings with respect to mean time to repair (which can be very location-specific). This can also be influenced by the ability of your own data center staff to diagnose/repair hardware faults and your own spares holding, and of course the availability requirements of your systems. Andy wrote: Hi, We have a number of Dell PE1950 servers running OmniOS which are in need of replacement. Management here have always used Dell and have suggested the R730xd platform (as we want more disk slots in the new servers). I've read some useful posts on this mailing list about the Dell R730xd - specifically that if we ask Dell to configure it for Nexenta then it will come with an LSI HBA and Intel NICs but I haven't seen any further posts saying whether anyone has had any success getting one of these up and running. I got the impression from the posts that it wasn't a great idea and Supermicro is a better option. I'd prefer to run Supermicro but might have some problems convincing those with the purse strings. Is anyone running, or can anyone recommend, a Supermicro server roughly equivalent to the Del R730xd, or give me an idea of what chipsets/HBAs etc. to choose or avoid for OmniOS? Any help appreciated, Thanks Andy ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
Just some anecdotal info... sun had a very nice partnership with dell.. back in the day, if you searched for dell in the hardware compat list you would show thousand of entries.. Dell now has a guy who sits on nexenta's board, this fact might have some impact in what they recommend. On the other hand, most if not all of the companies selling openzfs appliances are running supermicro or similar gear...Except for compellent which definitely runs dell :) I agree with dan though, the hba is the most important, as is nics. All this being said, I would personally prefer that dell r730 if i had a choice... lk On 2/18/15 9:34 AM, Dan McDonald wrote: On Feb 18, 2015, at 8:41 AM, Andy omn...@citrus-it.net wrote: I've read some useful posts on this mailing list about the Dell R730xd - specifically that if we ask Dell to configure it for Nexenta then it will come with an LSI HBA and Intel NICs but I haven't seen any further posts saying whether anyone has had any success getting one of these up and running. I got the impression from the posts that it wasn't a great idea and Supermicro is a better option. The key is to get an HBA illumos likes. The *default* Dell HBAs aren't all that great, and stick you with HW-RAID. I've not heard success stories with Nexenta Configured + OmniOS, but that's because every Nexenta configuration was used to... well... run NexentaStor. :) I know lots of illumos shops run SuperMicro with success. Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Dell vs. Supermicro and any recommendations..
On Feb 18, 2015, at 8:41 AM, Andy omn...@citrus-it.net wrote: I've read some useful posts on this mailing list about the Dell R730xd - specifically that if we ask Dell to configure it for Nexenta then it will come with an LSI HBA and Intel NICs but I haven't seen any further posts saying whether anyone has had any success getting one of these up and running. I got the impression from the posts that it wasn't a great idea and Supermicro is a better option. The key is to get an HBA illumos likes. The *default* Dell HBAs aren't all that great, and stick you with HW-RAID. I've not heard success stories with Nexenta Configured + OmniOS, but that's because every Nexenta configuration was used to... well... run NexentaStor. :) I know lots of illumos shops run SuperMicro with success. Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
[OmniOS-discuss] Dell vs. Supermicro and any recommendations..
Hi, We have a number of Dell PE1950 servers running OmniOS which are in need of replacement. Management here have always used Dell and have suggested the R730xd platform (as we want more disk slots in the new servers). I've read some useful posts on this mailing list about the Dell R730xd - specifically that if we ask Dell to configure it for Nexenta then it will come with an LSI HBA and Intel NICs but I haven't seen any further posts saying whether anyone has had any success getting one of these up and running. I got the impression from the posts that it wasn't a great idea and Supermicro is a better option. I'd prefer to run Supermicro but might have some problems convincing those with the purse strings. Is anyone running, or can anyone recommend, a Supermicro server roughly equivalent to the Del R730xd, or give me an idea of what chipsets/HBAs etc. to choose or avoid for OmniOS? Any help appreciated, Thanks Andy -- Citrus IT Limited | +44 (0)870 199 8000 | enquir...@citrus-it.co.uk Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ Registered in England and Wales | Company number 4899123 ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] ZFS Slog - force all writes to go to Slog
From: Richard Elling richard.ell...@richardelling.com Sent: Thursday, February 19, 2015 1:27 AM To: Rune Tipsmark Cc: omnios-discuss@lists.omniti.com Subject: Re: [OmniOS-discuss] ZFS Slog - force all writes to go to Slog On Feb 18, 2015, at 12:04 PM, Rune Tipsmark r...@steait.netmailto:r...@steait.net wrote: hi all, I found an entry about zil_slog_limit here: http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWritesAndZILII it basically explains how writes larger than 1MB per default hits the main pool rather than my Slog device - I could not find much further information nor the equivalent setting in OmniOS. I also read http://nex7.blogspot.ca/2013/04/zfs-intent-log.html but it didn't truly help me understand just how I can force every written byte to my ZFS box to go the ZIL regardless of size, I never ever want anything to go directly to my man pool ever. never ever want anything to go to main pool is not feasible. The ZIL is a ZFS Intent Log http://en.wikipedia.org/wiki/Intent_log and, unless you overwrite prior to txg commit, everything ends up in the main pool. yeah I know, I meant everything needs to go thru the ZIL before hitting the main pool... I have sync=always and disabled write back cache on my volume based LU's. Testing with zfs_txg_timeout set to 30 or 60 seconds seems to make no difference if I write large files to my LU's - I don't seem the write speed being consistent with the performance of the Slog devices. It looks as if it goes straight to disk and hence the performance is less than great to say the least. Ultimately, the pool must be able to sustain the workload, or it will have to throttle. it should be OK to take in some hundreds MB/sec (11 SAS mirrors each can do ~150MB/Sec sequential) The comment for zil_slog_limit is concise: /* * Use the slog as long as the logbias is 'latency' and the current commit size * doesn't exceed the limit or the total list size doesn't exceed its limit. * Limit checking is disabled by setting zil_slog_limit to UINT64_MAX. */ uint64_t zil_slog_limit = (1024 * 1024); uint64_t zil_slog_list_limit = (1024 * 1024 * 200); and you can change this on the fly using mdb to experiment. how do I do this? I could not find any property matching How do I ensure 100% that all writes always goes to my Slog devices - no exceptions. The question really isn't how the question is why? Now that you know what an Intent Log is, and how the performance of the pool is your ultimate limit, perhaps you can explain what you are really trying to accomplish? consistent fast write speeds at all times rather than yoyo write speeds... I get fine benchmarks but rather less fine file copy performance... and I never ever see disks being particular busy during file copy tests -- richard ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
[OmniOS-discuss] OmniOS Bloody update for Feb 18
There will be only 1-3 more updates prior to the next OmniOS stable (and for this time, this stable is also Long-Term Support) release. If you notice anything weird about the bloody bits, please let me know ASAP. I've heard little/no complaints about the updates to pkg(5), so either you're very happy, or not using it. :) This update will only be reaching the repo. * omnios-build master branch, revision f3d6d48 * Git to 2.3.0. * UnZIP fixes. * OpenJDK7 up to update 76, build 31. * Microtasking libraries (/lib/libmtsk*) are back as a distinct package (system/library/mtsk) now that they are not part of the (now open-source) Math libraries. * illumos-omnios master branch, revision cbf73e4 (last illumos-gate merge 336069c) * Bugfixes in PF_PACKET, SMB/CIFS, header files, NFS, and man pages. Thanks, Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] ZFS Slog - force all writes to go to Slog
On Feb 18, 2015, at 12:04 PM, Rune Tipsmark r...@steait.net wrote: hi all, I found an entry about zil_slog_limit here: http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWritesAndZILII http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWritesAndZILII it basically explains how writes larger than 1MB per default hits the main pool rather than my Slog device - I could not find much further information nor the equivalent setting in OmniOS. I also read http://nex7.blogspot.ca/2013/04/zfs-intent-log.html http://nex7.blogspot.ca/2013/04/zfs-intent-log.html but it didn't truly help me understand just how I can force every written byte to my ZFS box to go the ZIL regardless of size, I never ever want anything to go directly to my man pool ever. never ever want anything to go to main pool is not feasible. The ZIL is a ZFS Intent Log http://en.wikipedia.org/wiki/Intent_log http://en.wikipedia.org/wiki/Intent_log and, unless you overwrite prior to txg commit, everything ends up in the main pool. I have sync=always and disabled write back cache on my volume based LU's. Testing with zfs_txg_timeout set to 30 or 60 seconds seems to make no difference if I write large files to my LU's - I don't seem the write speed being consistent with the performance of the Slog devices. It looks as if it goes straight to disk and hence the performance is less than great to say the least. Ultimately, the pool must be able to sustain the workload, or it will have to throttle. The comment for zil_slog_limit is concise: /* * Use the slog as long as the logbias is 'latency' and the current commit size * doesn't exceed the limit or the total list size doesn't exceed its limit. * Limit checking is disabled by setting zil_slog_limit to UINT64_MAX. */ uint64_t zil_slog_limit = (1024 * 1024); uint64_t zil_slog_list_limit = (1024 * 1024 * 200); and you can change this on the fly using mdb to experiment. How do I ensure 100% that all writes always goes to my Slog devices - no exceptions. The question really isn't how the question is why? Now that you know what an Intent Log is, and how the performance of the pool is your ultimate limit, perhaps you can explain what you are really trying to accomplish? -- richard ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
