On Aug 18, 2017, at 11:29 AM, Robert Mustacchi <[email protected]> wrote:
On 8/17/17 18:31 , Rob Seastrom wrote:
Hi folks,
I'm scratching my head over SmartOS on a DL360g6 which I've been trying to piece together for deployment in a remote datacenter (DR and DNS service), so it's smaller / less capable than the machines that I usually run.
Some time ago I tried running SmartOS on these machines with an HP P410i RAID controller. The disk performance at the time was generally unsatisfactory, but there's a new driver for that controller family effective late last year ( https://smartos.org/bugview/OS-5564 ) so I figured I'd give it a try.
I'm running 20170608T172228Z on all the other machines around here so I figured I'd give it a go on this one as well. Booted fine from the USB, zpool was a ZFS mirror of two drives on the HP controller configured as singles. Everything seemed to come up OK, disk performace was adequate but nothing to write home about... but creating a VM timed out waiting for the VM to become ready... vmadm list showed the VM in provisioning status... and it eventually went to "running".
Odd. Well, maybe 6g was not enough ram for it to be truly happy. I upped the memory to 24g. No dice same deal.
OK, must be the disk subsystem right? Picked up some HP H220s (SAS2308 aka 9207s) and reflashed to IT mode. System boots but throws an odd error in the middle of booting: "warning: KCS error: ff" - Google tells me little except that maybe bmc is weird about the card. Disabled the built-in SmartArray just in case.
That's an error we've seen on some BMCs. In general, I wouldn't worry
about it.
OK, cool.
Disk array faster for copy across the network (getting 110-112 MByte/sec on gigabit ethernet vs. 75-80 before). zpool scrub operates as expected - "37.6G scanned out of 102G at 134M/s, 0h8m to go". And still, vmadm create runs out the clock:
[root@00-23-7d-e8-af-38 /zones/rs]# time vmadm create -f test29.json
first of 1 error: timed out waiting for zone to transition to running
real 0m57.031s
user 0m2.322s
sys 0m1.847s
[root@00-23-7d-e8-af-38 /zones/rs]#
In this case, there are a few different things I'd check before we even
worry about the disks. First, there's the zone_bh log in /var/log.
That'll have some information about all the transitions that the brand
went through. I'd first use that to see if the zone transitioned to
running or not. Let's figure that out, as that'll tell us where to look
next.
OK, so for reference on another platform (coincidentally same CPU but two chips rather than one, different motherboard, also running 20170608T172228Z) we have:
[root@f4-ce-46-b0-39-7a /zones/rs/json-templates-woodburn]# time vmadm create -f test29.json Successfully created VM 6810208d-a13a-ea34-c319-dcd4fda082e6
real 0m36.720s user 0m1.940s sys 0m2.578s [root@f4-ce-46-b0-39-7a /zones/rs/json-templates-woodburn]#
We got this:
[root@00-23-7d-e8-af-38 /zones/rs]# date ; time vmadm create -f test29.json August 18, 2017 at 03:39:15 PM UTC first of 1 error: timed out waiting for zone to transition to running
real 0m55.425s user 0m2.323s sys 0m1.862s [root@00-23-7d-e8-af-38 /zones/rs]#
while in another window
[root@00-23-7d-e8-af-38 ~]# date ; vmadm list August 18, 2017 at 03:40:35 PM UTC UUID TYPE RAM STATE ALIAS 41c3bca2-c8ae-6b74-daab-f4f559372a47 OS 2048 provisioning test29 [root@00-23-7d-e8-af-38 ~]# date ; vmadm list August 18, 2017 at 03:40:38 PM UTC UUID TYPE RAM STATE ALIAS 41c3bca2-c8ae-6b74-daab-f4f559372a47 OS 2048 running test29 [root@00-23-7d-e8-af-38 ~]#
So the zone transitioned to running after 1:23 of wall clock time, after vmadm create timed out at about 55 seconds.
It looks like /var/log/zone_bh.41c3bca2-c8ae-6b74-daab-f4f559372a47 was written in one shot when vmadm create exited; no further entries afterwards, and nothing in the leadup to it timing out. I've attached it for your review (hopefully the list doesn't stomp on it).
Thoughts?
-r
| 
zone_bh.41c3bca2-c8ae-6b74-daab-f4f559372a47