Re: [smartos-discuss] bhyve: 'old' virtual machine images, network, vnc
August 15, 2018 3:43 PM, "Stefan" wrote: > Dear list, > > we started experimenting with bhyve recently (joyent_20180802T002654Z) > and ran into some issues and open questions: > > 1. Are the 'old' SmartOS virtual machines images, e.g. > > 47f66e34-2c6d-11e8-bef9-4780fac9ac03 centos-7 > 20180320 linux zvol 2018-03-20 > > supposed to be usable under bhyve? I have tested installing the centos-7 > image using b6-json Sort of, most of those will have there network configured for DHCP. The KVM brand had a built in DHCP server to hand out the addresses configured via vmadm. Bhyve does not and relies on cloud-init to do the configuration. You should be able to use the old images, if you configure the network manually. At least for linux images. > > https://gist.github.com/mgerdts/2fa6f05d3c6edd6bb7f7a9842af0579e#file-b6-json > > with the image uuid replaced by that of the centos-7 image. This VM > comes up, vmadm console works, but it does not get its network settings. They not getting network settings is expected, see above. > > 2. Is there a migration path from TYPE=KVM to TYPE=BHYV? I modified the > brand attribute in /etc/zones/...xml file of an a earlier installed > instance (image uuid 6aac0370-56b8-11e7-a29b-df1de8b42795 = centos-6 > 20170621) and startet the VM. It comes up, network is up, I can ssh > into the VM. But > Some attribute are not supported or ignored by bhyve (e.g. model other than virtio for nics) ... bhyve also wants a few extra ones to configure the serial console. The easiest way I have found to migrated an existing KVM is: - configure the network statically while it is still booted via KVM - halt the KVM zone - (remove the nics - to avoid conflict of IPs, optionally remove the hostname value) - create a bhyve zone with same disk/network config - zfs destroy zones/NEW_ZONE_UUID/disk0 - zfs rename zones/OLD_ZONE_UUID-disk0 zones/NEW_ZONE_UUID/disk0 - boot the bhyve zone and see if everything works. I believe eventually joyent wants to add an auto convert of sorts. > - there is no output on vmadm console > - mdata-get sdc:net hangs and repeats "plat_recv timeout" > - a start script hangs: > > |-rc,757 /etc/rc.d/rc 3 > | `-bash,1485 /etc/rc3.d/S99local start > | `-mdata-get,1495 dummykey If you just swapped the type from KVM -> bhyve using zonecfg/editing the xml, that is to be expected as the serial consoles are set up differently. > > - vncviewer connects to the vnc port but does not ask > for the password and shows only a black screen. There is no password support or VNC yet (I think it is planned in the future though, but don't quote me on that). I never had any success with vnc + uefi-csm-rom.bin (what is shipped currently), the soon to come PI will have both uefi-rom.bin and uefi-csm-rom.bin shipped, you can select the uefi only one by setting bootrom=uefi, that way vnc should work. > > 3. I added vnc_port and vnc_password to the b6-json and installed the > 462d1d03-8457-e134-a408-cf9ea2b9be96 centos-bhyve-7 image. This starts > fine, has network and vmadm console but the vnc is not working as noted > above. You will still hit rough spots here and there so now might not be the time to switch over all your VMs. I myself hit a nasty issue with bhyve and netwerk traffic to zones on the same CN which can hang the bhyve guest. For now I have a CN dedicated for testing with bhyve, none of my guests are based on pre made images though. > > Any suggestion is appreciated. > > Kind Regards, > Stefan Regards and good luck! Jorge > --- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125 Powered by Listbox: https://www.listbox.com
Re: [smartos-discuss] bhyve: 'old' virtual machine images, network, vnc
Hi Stefan, Thanks for trying out bhyve. I've included notes below that may help you until we get our official bhyve+kvm hybrid images published. On Wed, Aug 15, 2018 at 8:43 AM, Stefan wrote: > Dear list, > > we started experimenting with bhyve recently (joyent_20180802T002654Z) > and ran into some issues and open questions: > > > 1. Are the 'old' SmartOS virtual machines images, e.g. > >47f66e34-2c6d-11e8-bef9-4780fac9ac03 centos-7 > 20180320linuxzvol 2018-03-20 > > supposed to be usable under bhyve? I have tested installing the centos-7 > image using b6-json > >https://gist.github.com/mgerdts/2fa6f05d3c6edd6bb7f7a9842af > 0579e#file-b6-json > > with the image uuid replaced by that of the centos-7 image. This VM > comes up, vmadm console works, but it does not get its network settings. > That is expected. With KVM, QEMU provides a built-in dhcp server. This allows the guest OS's DHCP client and QEMU to cooperate to get the network configuration that is found in the zone configuration. This does not exist for bhyve. Instead, we are shifting over to using cloud-init in all of our linux images. You can find a version of cloud-init that should work with centos 7 at https://us-east.manta.joyent.com/mgerdts/public/cloud-init/centos-7.4/cloud-init-18.3+2.g2923073e-1.el7.centos.noarch.rpm https://us-east.manta.joyent.com/mgerdts/public/cloud-init/centos-7.4/cloud-init-18.3+2.g2923073e-1.el7.centos.src.rpm To install: yum install -y pyserial rpm -Uvh cloud-init-18.3+2.g2923073e-1.el7.centos.noarch.rpm echo 'datasource_list: [ SmartOS ]' > /etc/cloud/cloud.cfg.d/90_smartos.cfg With cloud-init in place, the old rc.local is no longer used. Keeping it is considered harmful. You will need to do something like: cd /etc/rc.d rm -f rc.local mv rc.local.saved rc.local > 2. Is there a migration path from TYPE=KVM to TYPE=BHYV? There is not a supported conversion procedure, but it can usually be done. What I say below (and above) is largely from memory and a smattering of notes generated as we were working on official images. > I modified the > brand attribute in /etc/zones/...xml file of an a earlier installed > instance (image uuid 6aac0370-56b8-11e7-a29b-df1de8b42795 = centos-6 > 20170621) and startet the VM. It comes up, network is up, I can ssh > into the VM. But > >- there is no output on vmadm console > You will need to update /etc/default/grub, adding this about the first title line: serial --unit=0 --speed 115220 terminal --timeout=2 serial console Update the kernel line so it has console=ttyS0 and does not have console=tty0 Then run grub2-mkconfig -o /boot/grub2/grub.cfg If your VM has /etc/init/ttyS0.conf, remove it. For cloud-init, you will want these three packages: https://us-east.manta.joyent.com/mgerdts/public/cloud-init/centos-6.9/cloud-init-0.7.5-10.el6.2.joyent.4.x86_64.rpm https://us-east.manta.joyent.com/mgerdts/public/cloud-init/centos-6.9/ cloud-init-joyent-0.7.5-10.el6.2.joyent.4.x86_64.rpm https://us-east.manta.joyent.com/mgerdts/public/cloud-init/centos-6.9/ pyserial-3.4-1.noarch.rpm The corresponding source RPMs are: https://us-east.manta.joyent.com/mgerdts/public/cloud-init/centos-6.9/cloud-init-0.7.5-10.el6.2.joyent.4.src.rpm https://us-east.manta.joyent.com/mgerdts/public/cloud-init/centos-6.9/ cloud-init-joyent-0.7.5-10.el6.2.joyent.4.src.rpm https://us-east.manta.joyent.com/mgerdts/public/cloud-init/centos-6.9/ pyserial-3.4-1.src.rpm To install: rpm -Uvh cloud-init-0.7.5-10.el6.2.joyent.4.x86_64.rpm cloud-init-joyent-0.7.5-10.el6.2.joyent.4.x86_64.rpm pyserial-3.4-1.noarch.rpm >- mdata-get sdc:net hangs and repeats "plat_recv timeout" >- a start script hangs: > > |-rc,757 /etc/rc.d/rc 3 > | `-bash,1485 /etc/rc3.d/S99local start > | `-mdata-get,1495 dummykey > I'm not sure what is happening here. If it persists after the setup above, please let me know. >- vncviewer connects to the vnc port but does not ask > for the password and shows only a black screen. > bhyve and CSM (BIOS compatibility support module) do not work with VNC. Rather, the text console should be configured and you should use `vmadm console` or `zlogin -C`. Unlike kvm, `vmadm console` requires a "." after ^] to terminate the console session. That is, bhyve works like all the other zone brands - kvm is the oddball. 3. I added vnc_port and vnc_password to the b6-json and installed the > 462d1d03-8457-e134-a408-cf9ea2b9be96 centos-bhyve-7 image. This starts > fine, has network and vmadm console but the vnc is not working as noted > above. > As I said above, VNC does not work with guests that require BIOS support. As we are adding support for Windows, we are adding the ability to use UEFI instead of BIOS. This combination allows VNC to work, but does not support VNC authentication. Regards, Mike > > Any suggestion is appreciated. > > Kind Regards, > Stefan > ---
[smartos-discuss] bhyve: 'old' virtual machine images, network, vnc
Dear list, we started experimenting with bhyve recently (joyent_20180802T002654Z) and ran into some issues and open questions: 1. Are the 'old' SmartOS virtual machines images, e.g. 47f66e34-2c6d-11e8-bef9-4780fac9ac03 centos-7 20180320linuxzvol 2018-03-20 supposed to be usable under bhyve? I have tested installing the centos-7 image using b6-json https://gist.github.com/mgerdts/2fa6f05d3c6edd6bb7f7a9842af0579e#file-b6-json with the image uuid replaced by that of the centos-7 image. This VM comes up, vmadm console works, but it does not get its network settings. 2. Is there a migration path from TYPE=KVM to TYPE=BHYV? I modified the brand attribute in /etc/zones/...xml file of an a earlier installed instance (image uuid 6aac0370-56b8-11e7-a29b-df1de8b42795 = centos-6 20170621) and startet the VM. It comes up, network is up, I can ssh into the VM. But - there is no output on vmadm console - mdata-get sdc:net hangs and repeats "plat_recv timeout" - a start script hangs: |-rc,757 /etc/rc.d/rc 3 | `-bash,1485 /etc/rc3.d/S99local start | `-mdata-get,1495 dummykey - vncviewer connects to the vnc port but does not ask for the password and shows only a black screen. 3. I added vnc_port and vnc_password to the b6-json and installed the 462d1d03-8457-e134-a408-cf9ea2b9be96 centos-bhyve-7 image. This starts fine, has network and vmadm console but the vnc is not working as noted above. Any suggestion is appreciated. Kind Regards, Stefan --- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125 Powered by Listbox: https://www.listbox.com
