Blake, Thanks for your 15 October 2014 response to my 15 October 2014 message.
(I delayed sending this present email until now, when I have more encouraging progress to report, from my trial of SmartOS on an Elastic Hosts VM. Those results, I reported 25 October 2014, in a separate email to [smartos-discuss] titled "SmartOS -- Elastic Hosts VM: (1) "trap: unknown trap type 8 in user mode" - fixed, (2) Using VNC For Console - No SSH Yet - Network Problem?;".) In your email, you make very good arguments in favor of e.g. OmniOS, as an alternative to running SmartOS under a KVM virtualization layer. As promised in my 25 October email, I comment below on the specific points you raise, and provide explanation of my thinking, in favour of keeping to the SmartOS source code tree, for my choice of production operating system, for hosting a new website on a QEMU-KVM VM. * * * * * * [BI] SmartOS is designed to be the ‘hypervisor’ OS that controls other VMs. Running it inside of another virtualization layer is possible, but you’ll be complicating things without need in this case. [SP] I've already discovered one SmartOS "complication" (not directly related to the bare metal / VM issue) -- that the standard SmartOS distribution doesn't seem to get along very happily with an AMD64 CPU :) I'm just glad that I ran into this little glitch on an inexpensive AMD64 VM, and not on a pricey dedicated bare metal AMD64 server. I assure you, I did start out with the intention of using SmartOS exactly the way you describe it is designed to be used -- as a bare metal hypervisor that controls other VMs. But after I researched pricing for a dedicated server for running SmartOS -- I came away discouraged. The business model behind the new website I will be hosting -- requires me to keep costs ultra-low. This means that I cannot justify the expense of a dedicated server for SmartOS. I must turn to the commodity VM hosting market, where competition works in my favour. Initially I was disturbed, by dire warnings I was reading, found via Google searches, that SmartOS can be very particular about the hardware it runs on. My conspiracy theorist gene is not very active, but I must admit that cynicism got the better of me, and I wondered if these dire warnings were in part a smokescreen raised by SmartOS "insiders" to keep penny-pinching "proletarians" like me, from besmirching the "purity" of SmartOS, by running it under KVM ?? :) Thanks to the humbling AMD64 CPU trap I have already stumbled over -- I'm a little bruised, but less cynical and a bit wiser ... * * * * * * [BI] If you want to run your app on SmartOS in ‘the cloud’, you would probably be better off just renting a VM from Joyent. [SP] Yes -- I did try to go down this route, intending to sign up as a Joyent SmartOS VM customer. The cruel reality is that I am such a small fish -- and Joyent seems so very busy raking in profits from much bigger fish, that I decided I had to find a different hosting provider, one that seems more enthusiastic about having me as a customer. (I do admit that like many small-fish customers, I can be a very demanding and picky little fish:) I have a firm business principle -- avoid if at all possible, dependency on a sole supplier source for a key resource. The Joyent company has an enviable reputation, and are to be praised highly for their contribution of the excellent SmartOS to the open source world. However, the additional facts that Joyent seems: (1) to build its hardware according to very detailed specs (not necessarily a bad thing in itself), and (2) to show signs of having pricing power to offer its VMs according to their scarcity and value, both also got me looking for a more competitive multi-source hosting provider market -- and inevitably the market in virtualized servers. * * * * * * [BI] If you want to run some illumos-derived system in ‘the cloud’ (SmartOS is one illumos distribution), then something like OmniOS might be better for you, as it can be run as a VM guest without extra overhead. [SP] Yes I do want to run an illumos-derived system in the cloud -- but I particularly want to run the SmartOS distribution of illumos. Not to in any way downrate the good work of the good folks who put their all into e.g. OmniOS, my conservative business mind finds big comfort in keeping to what appears to me to be, the most commercially established illumos variant -- namely SmartOS. It is my understanding that, at present it is only the ZFS that comes with SmartOS, that offers encryption at the (ZFS) file system level. This feature alone, is worth to me a great deal, as it means no messing about with encryption at multiple points in the application layer. Even though it might make me a pariah to some SmartOS devotees -- I can't make any apology for my desire to "eat" the unsurpassed benefits of 3/4 of the SmartOS cake (DTrace, ZFS, Zones) and "have" my SmartOS cake at the same time too, by running SmartOS under a virtualization layer that offers huge cost-effectiveness, where I make no use of SmartOS hypervision and KVM supervision functionalities. There will undoubtedly be work involved, in slimming down SmartOS to run as well as possible under a KVM virtualization layer. But I've already waded in to the SmartOS customization arena -- by having to strip out the "intel_nhm*" drivers, to get SmartOS up and running on an Elastic Hosts AMD64 VM. * * * * * * It seems to me that, SmartOS could only benefit from widespread adoption as a guest OS in the wide-open virtualized hosting market. My hope is that, with help from [smartos-discuss] members, I will be able to give back, by continuing to report in detail on my hands-on experience with SmartOS, running under an Elastic Hosts VM. Steve * * * Steve Petrie, P.Eng. ITS-ETO Consortium Oakville, Ontario, Canada (905) 847-3253 [email protected] ----- Original Message ----- From: Blake Irvin To: [email protected] ; Steve Petrie, P.Eng. Sent: Wednesday, October 15, 2014 7:59 AM Subject: Re: [smartos-discuss] Fw: SmartOS setup & boot Errors -- Testing SmartOS On An Elastic Hostd VM; Steve, one thing to keep in mind here is that SmartOS is designed to be the ‘hypervisor’ OS that controls other VMs. Running it inside of another virtualization layer is possible, but you’ll be complicating things without need in this case. If you want to run your app on SmartOS in ‘the cloud’, you would probably be better off just renting a VM from Joyent. If you want to run some illumos-derived system in ‘the cloud’ (SmartOS is one illumos distribution), then something like OmniOS might be better for you, as it can be run as a VM guest without extra overhead. Blake On 15 Oct 2014, at 12:41, Steve Petrie, P.Eng. via smartos-discuss <[email protected]> wrote: Greetings to smartos-discuss, Here is a forward of an email that I sent to [email protected] on 9 October 2014, when I was not subscribed to the list. I don't see my message in the archives, so I have subscribed to the list, and am trying again to get this message on to the list. Regards, Steve * * * Steve Petrie, P.Eng. ITS-ETO Consortium Oakville, Ontario, Canada (905) 847-3253 [email protected] ----- Original Message ----- From: Steve Petrie, P.Eng. To: [email protected] Cc: Elastic Hosts - Support Sent: Thursday, October 09, 2014 2:50 PM Subject: SmartOS setup & boot Errors -- Testing SmartOS On An Elastic Hostd VM; Greetings To smartos-discuss, I am working to get SmartOS operational under a VM provisioned on the Elastic Hosts (EH) cloud infrastucture www.elastichosts.com Presently, on a trial EH VM, a SmartOS setup / SmartOS boot, running from from the iso CD image on an EH cloud disk, is giving four messages that seem particularly of note: a.. trap: unknown trap type 8 in user mode b.. Notice: MPO disabled because memory is disabled c.. WARNING: KVM no hardware support d.. WARNING rtls0: failure resetting PHY I would appreciate very much, any comments smart-os list members would care to make. The KVM warning is not a big issue for my particular application. EH support advise me, that the "trap" message could be of particular concern. They also advise trying to disable ACPI. * * * * * * At the http://wiki.smartos.org/display/DOC/Home web page, a Google custom search for the string: a.. smartos setup message "trap: unknown trap type 8 in user mode" ? yields a single result, being the illumos IRC log of [August 18 - 2011], a copy of which is attached to this email, in edited form: a.. <SmartOS -- Elastic Hosts VM - _Unknown trap type 8 in user mode_ - boot message - edited search results - 20141409.txt> * * * * * * As a help to smart-os list participants, here are snippets (emphasis added) selected from the attached edited IRC log: a.. [01:51:17] <grey> I'm getting "Unknown trap type 8 in user mode" right after it starts trying to load the Kernel ... [01:57:25] <richlowe> I've seen the unknown usermode trap before b.. [02:06:34] <richlowe> could be ACPI, I suppose. [02:06:48] <grey> It's possible I didn't give nexenta enough time, in hindsight, that message may not have been fatal? c.. [02:06:49] <lewellyn> grey: does OI boot? ... [02:07:11] <richlowe> grey: it is. [02:07:13] <grey> Nexenta booted up this time [02:07:20] <richlowe> drat. d.. [02:08:00] <richlowe> last I saw it, we took the "bogus interrupt" early in boot, took it to be a trapped, and tried to core a process that didn't exist. ... [02:09:30] <richlowe> lewellyn: if it involved counter underflow, that was different. ... [02:09:55] <lewellyn> i don't remember details. that's why i have computers. but in any case, it was something i didn't want to even get near. [02:09:59] <richlowe> if you booted with kmdb loaded, and you can get there, ::interrupts might be useful. e.. [02:16:13] <richlowe> got it. ... [02:16:20] <richlowe> this is the !@#$% intel_nhmex pain. ... [02:16:25] <richlowe> grey: can you modify your boot image? [02:16:33] <grey> can I? you tell me ;) [02:16:45] <richlowe> intel_nhmex (and intel_nhm, actually) do actual things in _init, presumably mistaking them for _attach f.. [02:17:00] <richlowe> when they do this on certain CPUs, it traps and the whole world blows up [02:17:08] <richlowe> if you're looking, you just get a warning from ACPI ... [02:17:40] <lewellyn> i get messages at boot regarding acpi and then things go wonky later [02:17:43] <richlowe> I went through hell on this older athlon64 chasing that down. [02:18:33] <richlowe> it turns out that this used to crash Xen too, and rather than fixing it, they just made the module not load under i86xpv g.. [02:19:03] <richlowe> grey: if you can delete /kernel/**/intel_nhm* you may come up. [02:19:11] <richlowe> I don't think -Bdisable-intel_nhmex works h.. [02:19:43] <grey> richlowe: ok, I'll take a crack at that (I don't suppose you know off the top of your head the easiest way to modify an ISO image in that fashion?, I'll google around for it) [02:20:06] <richlowe> I think on the iso, you'd have to unpack it, delete the modules, and then rebuild its boot_archive. i.. [02:22:49] <grey> richlowe: that's the kernel/**/intel_nhm* under platform/i86pc/ ? [02:23:15] <richlowe> under /kernel/drv, I think the other is the cpu mod [02:24:46] <grey> this is on the nexenta distro? I don't see any intel_nhm* files anywhere in the entire tree ... [02:25:43] <grey> http://pastebin.com/Q9Nzb30D There's the directory structure [02:26:28] <grey> oh, is it inside the usr.img.zlib? [02:27:11] <richlowe> If that's OI media, I have no idea where it'd be on the install media [02:27:33] <grey> that's from nexenta-core-platform_3.0.1-b134_x86.iso j.. [02:33:21] <richlowe> sort of interesting would be, boot with -kvd ( hit 'e' in grub, add -kvd to the kernel$ line), and hit 'b' to boot. it'll stop saying [0]>. Type "::bp pcplusmp`apic_error_intr -c $C" then :c [02:34:07] <richlowe> hopefully it'll hit the breakpoint and dump the stack, rather than exploding. ... [02:34:20] <richlowe> if it doesn't, I think you found a new way to get a random interrupt. [02:34:42] <richlowe> if it happened to log APIC errors, even when it worked, that'd help to place the blame on nhmex again, too. * * * * * * Would smart-os list members have any knowlledge -- was there a resolution of this "trap" issue? In the interest of getting SmartOS to work on the EH cloud platform, if this requires building a special distribution that simply omits some SmartOS functionality, whose absence is not fatal, I would be happy to undertake that. I'm also happy to try to make any diagnostic efforts, that smart-os list participants might recommend. Please be aware that, although I am a software engineer with 30 years experience, I have little Unix and less Linux expeience. I do have C++ experience and some hardware-level comprehension. And a keen desire to enter the SmartOS world. So I would need advice from smart-os list members and from EH sipport. * * * * * * My motivation to climb a SmartOS learning curve, is because I really like the idea of hosting in production under SmartOS, a new website I have developed. Please see the attached PDF rendition of the home page, for the new website (not yet online anywhere): a.. <eto_home - 20140812.pdf> (Unfortunately, the PDF does not provide animation of the horizontal graphic of the expressway section. If smartos-discuss list members would like to see the animation, just let me know, and I will supply the necessary files.) * * * * * * Hopefully Clicking "Send" Now :) Steve * * * Steve Petrie, P.Eng. ITS-ETO Consortium Oakville, Ontario, Canada (905) 847-3253 [email protected] smartos-discuss | Archives | Modify Your Subscription <eto_home - 20140812.pdf> <SmartOS -- Elastic Hosts VM - _Unknown trap type 8 in user mode_ - boot message - edited search results - 20141409.txt> ------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
