Re: [smartos-discuss] Remove ZIL from zones pool?

2018-01-24 Thread Richard Elling
bonnie++ should not be considered until after you read https://blogs.oracle.com/roch/decoding-bonnie -- richard > On Jan 24, 2018, at 5:33 PM, Sam Nicholson wrote: > > Well, I'm wrong. > > I got to thinking about this and decided to test it. See >

Re: [smartos-discuss] zpool detach mistake

2017-09-27 Thread Richard Elling
> On Sep 27, 2017, at 3:10 AM, Antoine Jacoutot wrote: > > Hello. > > So I made a mistake today replacing a faulty drive in a RAID10 setup. > i.e. 4 mirrors in a pool > I stupidly detached the drive (c7) which means I lost one mirror. > Surprisingly, > it _seems_

Re: [smartos-discuss] hardware selection question

2017-08-06 Thread Richard Elling
> On Aug 6, 2017, at 4:01 PM, Steve wrote: > > I saw a Supermicro with 60 drives in a 4U chassis! They max out at 90 I > believe. We use 60 SAS 2.5" drives in 2u chassis with dual, 2-socket + 2 12G SAS HBA controllers. Next week we'll have them on the exhibit floor at

Re: [smartos-discuss] znapzend (or similar) in a pull configuration

2017-05-23 Thread Richard Elling
> On May 23, 2017, at 6:53 AM, Chris Ferebee wrote: > > Hi all, > > znapzend is wonderful for snapshots and backups. > > However, for backups of internet-facing zones, I would prefer a "pull" rather > than "push" configuration, such that the backup host initiates the

Re: [smartos-discuss] sysinfo script modification question

2017-04-04 Thread Richard Elling
> On Apr 3, 2017, at 11:37 PM, Dale Ghent wrote: > > > I have SMCI servers that have mangled or all-zero UUIDs as well. very common with supermicro gear. You'll also see an occasional bogus 00010002-0003-0004-0005-000600070008. The sysinfo code in kernel recognizes some

Re: [smartos-discuss] Expanded pool only seeing 2TB of 3TB disks

2017-02-02 Thread Richard Elling
> On Feb 2, 2017, at 7:31 AM, Gareth Howell wrote: > > Hi > I’ve just upgraded the two mirrored disks in my zones pool from 1TB disks to > 3TB disks by replacing each in turn and then doing zpool replace. > > I then did > 'pool scrub' > and > 'zpool set autoexpand=on'

Re: [smartos-discuss] smartos ssd disk question

2017-01-23 Thread Richard Elling
> On Jan 23, 2017, at 9:54 AM, Gernot Straßer > wrote: > > Hi Robert, > > so let me try the other way around then: Do you know a device where the > manufacturer explicitly says so? STEC did. They also said synchronize_cache is a nop. I’ve been working a lot with

Re: [smartos-discuss] Can not override physical sector disks

2016-09-20 Thread Richard Elling
> On Sep 20, 2016, at 6:48 AM, Humberto Ramirez wrote: > > Is there a definitive approach / guide / manual / wiki as to how to properly > work / replace 4k - 512 - 512e disks? This has been asked before, here and in > some other lists and obviously continues to be a source

Re: [smartos-discuss] zpool replace with a disk which has different physical blocksize

2016-07-19 Thread Richard Elling
> On Jul 19, 2016, at 1:19 PM, smartos-discuss@lists.smartos.org wrote: > > Hi, > > the new model is a 2 TB Seagate NAS HDD ST2000VN000 and as to the report it > has 4K as physical blocksize: > dkmp_ext.dki_lbsize = 512 > dkmp_ext.dki_pbsize = 4096 Friendly advice: you can force this 512e

Re: [smartos-discuss] low network bandwidth with 40 vms

2016-05-04 Thread Richard Elling
> On May 4, 2016, at 8:37 AM, Stefan wrote: > > Dear List, > > I would like to provide you with an update on the issue. We now use an > intel 10GbE card (ixgbe) which reduces the %tim in intrstat but there is > no increase in throughput. Next steps are as Robert suggests,

Re: [smartos-discuss] low network bandwidth with 40 vms

2016-05-03 Thread Richard Elling
> On May 3, 2016, at 10:16 AM, Stefan wrote: > > Dear List, > > on our machines with about 40 vservers we observe low network > throughput. It seems to scale inversely with the number of vms on the > respective node: > > # vms bw (Mbps) > - - > 37

Re: [smartos-discuss] preallocated zVOL / improving SSD performance

2016-04-16 Thread Richard Elling
> On Apr 14, 2016, at 1:56 AM, Dirk Steinberg <d...@steinbergnet.net> wrote: > >> >> Am 14.04.2016 um 02:39 schrieb Richard Elling >> <richard.ell...@richardelling.com <mailto:richard.ell...@richardelling.com>>: >> >> >>

Re: [smartos-discuss] preallocated zVOL / improving SSD performance

2016-04-12 Thread Richard Elling
> On Apr 12, 2016, at 3:21 PM, Dirk Steinberg wrote: > > Hi, > > in order to improve long-term performance on consumer-grade SSDs, > I would like to reserve a certain range of LBA addresses on a freshly > TRIMmed SSD to never be written to. That can be done by slicing

Re: [smartos-discuss] (U)EFI boot for SmartOS?

2016-04-12 Thread Richard Elling
> On Apr 12, 2016, at 2:41 PM, Dirk Steinberg <d...@steinbergnet.net> wrote: > >> >> Am 12.04.2016 um 23:30 schrieb Dirk Steinberg <d...@steinbergnet.net >> <mailto:d...@steinbergnet.net>>: >> >> >>> Am 12.04.2016 um 23:0

Re: [smartos-discuss] (U)EFI boot for SmartOS?

2016-04-12 Thread Richard Elling
> On Apr 12, 2016, at 1:39 PM, Dirk Steinberg wrote: > > The root file system actually resides on a ram disk, which cannot be used for > booting. The RAM disk contains a UFS file system. > > If I do not boot from USB or PXE, I like to put the boot files > (kernel and

Re: [smartos-discuss] Can not install Build20160317T000621Z HPE Proliant DL380 Gen9, 2 CPU (E5-2650 v3)

2016-03-24 Thread Richard Elling
> On Mar 23, 2016, at 8:29 AM, Robert Mustacchi <r...@joyent.com> wrote: > >> On 3/23/16 8:22 , Richard Elling wrote: >> >>> On Mar 23, 2016, at 6:59 AM, Robert Mustacchi <r...@joyent.com> wrote: >>> >>> On 3/23/16 2:00 , Benny Kjel

Re: [smartos-discuss] Can not install Build20160317T000621Z HPE Proliant DL380 Gen9, 2 CPU (E5-2650 v3)

2016-03-23 Thread Richard Elling
> On Mar 23, 2016, at 6:59 AM, Robert Mustacchi wrote: > > On 3/23/16 2:00 , Benny Kjellgren wrote: >> Thank you >> >> "-B pci-reprog=off" solved my problem with SmartOS not booting up. >> >> And have changed from raidz1 with pair of mirrored disks LV >> to raidz2 with single

Re: [smartos-discuss] Can not install Build20160317T000621Z HPE Proliant DL380 Gen9, 2 CPU (E5-2650 v3)

2016-03-22 Thread Richard Elling
> On Mar 22, 2016, at 8:10 PM, 许若辰 wrote: > > I found a interesting material benny > http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04633840=en=en_US > > > I don’t

Re: [smartos-discuss] Can not install Build20160317T000621Z HPE Proliant DL380 Gen9, 2 CPU (E5-2650 v3)

2016-03-22 Thread Richard Elling
we find that bge2 is toxic. more later today... -- richard > On Mar 22, 2016, at 6:36 AM, Benny Kjellgren > wrote: > > Update : > I trace the reboot to /smartdc/lib/smartos_prompt_config.sh > for iface in $(dladm show-phys -pmo link); do > ifconfig

Re: [smartos-discuss] Can not install Build20160317T000621Z HPE Proliant DL380 Gen9, 2 CPU (E5-2650 v3)

2016-03-21 Thread Richard Elling
> On Mar 21, 2016, at 10:49 AM, Benny Kjellgren > wrote: > > Hi Richard > > I can verify that disabling the x2APIC made the NOTICE lines to disappear. > Also notice if setting the SmartArray in HBA mode then "format" command > display no disks. yes. It seems

Re: [smartos-discuss] Can not install Build20160317T000621Z HPE Proliant DL380 Gen9, 2 CPU (E5-2650 v3)

2016-03-21 Thread Richard Elling
> On Mar 21, 2016, at 7:50 AM, Benny Kjellgren > wrote: > > Hi, > > I get this on the console : > > NOTICE: System detected 256 cpus, but only 1 cpu(s) were enabled during > boot. > NOTICE: Use "boot-ncpus" parameter to enable more CPU(s). See eeprom(1M). >

Re: [smartos-discuss] zfs_arc_max setting for SmartOS

2016-03-19 Thread Richard Elling
> On Mar 16, 2016, at 8:21 AM, Jon Dison wrote: > > I’m reasonably certain that the VMs are using mostly swap, at least that’s > what top says. Reserving swap is not the same thing as using swap. It is quite common to see swap reservations, especially for KVM zones. How

Re: [zfs] [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built

2016-03-06 Thread Richard Elling
> On Mar 6, 2016, at 9:06 PM, Fred Liu <fred.f...@gmail.com> wrote: > > > > 2016-03-06 22:49 GMT+08:00 Richard Elling <richard.ell...@richardelling.com > <mailto:richard.ell...@richardelling.com>>: > >> On Mar 3, 2016, at 8:35 PM, Fred Liu <

Re: [smartos-discuss] Update gone bad, the Aftermath

2016-02-01 Thread Richard Elling
> On Jan 31, 2016, at 2:59 PM, Ian Collins <i...@ianshome.com> wrote: > > Richard Elling wrote: >> >>> On Jan 28, 2016, at 1:09 AM, Ian Collins <i...@ianshome.com >>> <mailto:i...@ianshome.com>> wrote: >>> >>> It isn't usual

Re: [smartos-discuss] Update gone bad, the Aftermath

2016-01-31 Thread Richard Elling
> On Jan 28, 2016, at 1:09 AM, Ian Collins wrote: > > the outsider wrote: >> >> Dear all, >> >> Many thanks to all that helped yesterday and tonight for restoring my server. >> >> Everything works fine now, but I have some unanswered questions. Maybe >> someone can shed

Re: [developer] [smartos-discuss] Is zfs dead man timer tunable?

2016-01-26 Thread Richard Elling
> On Jan 25, 2016, at 11:22 PM, Fred Liu wrote: > > > Yes. I cleaned the whole folder > > Fred savecore serves two functions: 1. copy the dump from the dump device to a filesystem: vmdump.# (done at boot by the dumpadm SMF service) 2. extract the vmcore.# and

Re: [smartos-discuss] Is zfs deadm man timer tunable?

2016-01-21 Thread Richard Elling
> On Jan 21, 2016, at 4:25 AM, Fred Liu wrote: zfs deadman timer is tunable. But if you hit it, you've got problems that tuning the deadman won't help. The tunable is zfs_deadman_synctime_ms, which is milliseconds. For example, on a test machine here: [root@elvis

Re: [smartos-discuss] Is zfs dead man timer tunable?

2016-01-21 Thread Richard Elling
answer far below... > On Jan 21, 2016, at 8:44 PM, Fred Liu <fred_...@issi.com> wrote: > > > >> -Original Message- >> From: Richard Elling [mailto:richard.ell...@richardelling.com] >> Sent: 星期五, 一月 22, 2016 12:02 >> To: smartos-discuss@lists.sm

Re: [smartos-discuss] Deleting large files blocks all IO

2014-09-26 Thread Richard Elling via smartos-discuss
On Sep 26, 2014, at 9:00 AM, Rajesh Poddar via smartos-discuss smartos-discuss@lists.smartos.org wrote: When I delete large files (2-3TB) on my smartos system it often blocks all IO operations. The only info I found online pointed to dedup as a possible culprit but I have never enabled

Re: [smartos-discuss] Strange overheat problem with Supermicro 4U chassis

2014-08-08 Thread Richard Elling via smartos-discuss
On May 24, 2014, at 1:54 AM, Ian Collins via smartos-discuss smartos-discuss@lists.smartos.org wrote: Ian Collins via smartos-discuss wrote: Ian Collins via smartos-discuss wrote: Richard Elling via smartos-discuss wrote: Firmware rev 0001, 0002, 0004, and a004 work. Firmware rev 0003

Re: [smartos-discuss] 10GbE throughput limited to 1Gb/s, why?

2014-07-20 Thread Richard Elling via smartos-discuss
On Jul 19, 2014, at 6:42 AM, Chris Ferebee via smartos-discuss smartos-discuss@lists.smartos.org wrote: I'm trying to debug a network performance issue. I have two servers running SmartOS (20140613T024634Z and 20140501T225642Z), one is a Supermicro dual Xeon E5649 (64 GB RAM) and the

Re: [smartos-discuss] ghc ./configure generating bad config.status:

2014-07-12 Thread Richard Elling via smartos-discuss
On Jul 12, 2014, at 2:21 PM, Alain O'Dea via smartos-discuss smartos-discuss@lists.smartos.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri 11 Jul 2014 12:33:09 PM UTC, Jonathan Perkin via smartos-discuss wrote: * On 2014-07-11 at 13:28 BST, Nahum Shalman via

Re: [smartos-discuss] Joyents experience with HGST drives and performance

2014-07-07 Thread Richard Elling via smartos-discuss
On Jul 7, 2014, at 11:28 AM, Keith Wesolowski via smartos-discuss smartos-discuss@lists.smartos.org wrote: On Mon, Jul 07, 2014 at 08:20:43PM +0200, Ibrahim Tachijian wrote: Sounds like a plan Ketih. And the additional need for a SLOG (like ZeusRam) is def. not required because of my

Re: [smartos-discuss] DogeOS/SmartOS smf dependencies visualization tool!

2014-05-29 Thread Richard Elling via smartos-discuss
On May 29, 2014, at 10:06 PM, Yu Li via smartos-discuss smartos-discuss@lists.smartos.org wrote: Hi, all, just let you know that I did some work to visualize the SmartOS smf dependencies. http://www.dogeos.net/smfgraph/ well done! d3 rules! You can adjust the charge and linkDistance

Re: [smartos-discuss] ZIL recommendations

2014-05-10 Thread Richard Elling via smartos-discuss
On May 9, 2014, at 10:32 AM, Len Weincier via smartos-discuss smartos-discuss@lists.smartos.org wrote: I see from the manufacturing docs that you have a single 50GB device (270-022 usually). From reading around it seems to be recommend that the slog be mirrored as it is taking the

Re: [smartos-discuss] Working backplane/enclosure for SES in mass non-Solaris/fishwork-ZFS deployment

2014-05-10 Thread Richard Elling via smartos-discuss
Speaking of lighting LEDs... On May 9, 2014, at 7:11 PM, Robert Mustacchi via smartos-discuss smartos-discuss@lists.smartos.org wrote: On 5/9/14 18:01 , Fred Liu via smartos-discuss wrote: I post here for I don't know where the largest non-Solaris/fishwork-ZFS deployment in the world is.

Re: [smartos-discuss] ZIL recommendations

2014-05-08 Thread Richard Elling via smartos-discuss
On May 8, 2014, at 7:51 AM, Ibrahim Tachijian via smartos-discuss smartos-discuss@lists.smartos.org wrote: Thanks for that list Keith (http://eng.joyent.com/manufacturing). For everyone who hasn't read it yet I do recommend you to do so. I'm guessing joyent has gone through a huge amount

Re: [smartos-discuss] ZFS continually report data written for MTBF calculations

2014-04-07 Thread Richard Elling
On Apr 7, 2014, at 10:20 AM, Jakob Borg ja...@nym.se wrote: 2014-04-07 18:13 GMT+02:00 Evan Rowley rowley.e...@gmail.com: I'd like to measure the total amount of data written over time to the physical devices that make up zpool zils, l2arcs, and vdevs. Each one of these physical

Re: [smartos-discuss] Network setup

2014-03-08 Thread Richard Elling
On Mar 6, 2014, at 2:49 PM, tono t...@ownmail.net wrote: Hello, I'm looking for advice about redundant NIC setup, specifically avoiding single point of failure at the switch. I did see the wiki page on link aggregations. We use non-stacking switches, which means that all members would