Re: OpenBSD and Compute Engine
Thought I'd share a quick update here, since a few people have expressed interest: I've committed the dhclient patch and vioscsi driver. I expect the next set of OpenBSD snapshots that include these commits to work out of box on Compute Engine. There are docs online for how to use Cloud Storage and Compute Engine already, so I'm going to just focus on creating a image for OpenBSD. Also, I've only tested these steps on Ubuntu; they might work on OpenBSD. 1. Create a disk image. It needs to be named disk.raw and should be a raw disk image: qemu-img create -f raw disk.raw 10G 2. Install OpenBSD using install54.iso: qemu-system-x86_64 -drive if=virtio,file=disk.raw -cdrom install54.iso -boot order=cd (I was using a remote machine and X11 forwarding was too slow for me, so I actually first used echo 'set tty com0' boot.conf and growisofs -M install54.iso -l -R -graft-points /etc/boot.conf=boot.conf so that cdboot(8) would use the serial port instead of VGA, and then I used qemu's -nographic option. YMMV.) 3. Tar and gzip the disk image. File name doesn't matter, but -S flag makes the .tar.gz file smaller. OpenBSD tar doesn't support -S, but I haven't checked if gtar from ports does. tar -Szcf my-openbsd-image.tar.gz disk.raw 4. Now you should have an OpenBSD disk image usable with Compute Engine. Follow the instructions online for how to upload it to Cloud Storage, register it as a disk image with Compute Engine, and then create instances based on it: https://developers.google.com/compute/docs/building-image#publishingimage I've only tested the n1-standard-1 instance type so far. Feel free to contact me via the mailing list or privately if you have any feedback or questions.
Re: OpenBSD and Compute Engine
On Tue, Dec 03, 2013 at 01:44:45PM -0800, Matthew Dempsky wrote: On Tue, Dec 3, 2013 at 1:35 PM, Jiri B ji...@devio.us wrote: Aha, big brother grows up :-) I will try to test virtio-scsi on RHEVM which also exposes this to VMs. Cool, let me know if you run into any issues. I'm sure there's still some bugs in the driver so I wouldn't put it into production yet, but it should at least let you mount a virtio-scsi disk. I have problem to compile it :/ cc -Werror -Wall -Wstrict-prototypes -Wmissing-prototypes -Wno-main -Wno-uninitialized -Wno-format -Wstack-larger-than-2047 -mcmodel=kernel -mno-red-zone -mno-sse2 -mno-sse -mno-3dnow -mno-mmx -msoft-float -fno-omit-frame-pointer -fno-builtin-printf -fno-builtin-snprintf -fno-builtin-vsnprintf -fno-builtin-log -fno-builtin-log2 -fno-builtin-malloc -fno-pie -O2 -pipe -nostdinc -I../../../.. -I. -I../../../../arch -DDDB -DDIAGNOSTIC -DKTRACE -DACCOUNTING -DKMEMSTATS -DPTRACE -DPOOL_DEBUG -DCRYPTO -DSYSVMSG -DSYSVSEM -DSYSVSHM -DUVM_SWAP_ENCRYPT -DCOMPAT_43 -DLKM -DFFS -DFFS2 -DFFS_SOFTUPDATES -DUFS_DIRHASH -DQUOTA -DEXT2FS -DMFS -DNFSCLIENT -DNFSSERVER -DCD9660 -DUDF -DMSDOSFS -DFIFO -DFUSE -DSOCKET_SPLICE -DTCP_SACK -DTCP_ECN -DTCP_SIGNATURE -DINET -DALTQ -DINET6 -DIPSEC -DPPP_BSDCOMP -DPPP_DEFLATE -DPIPEX -DMROUTING -DMPLS -DBOOT_CONFIG -DUSER_PCICONF -DAPERTURE -DMTRR -DNTFS -DHIBERNATE -DPCIVERBOSE -DUSBVERBOSE -DWSDISPLAY_COMPAT_USL -DWSDISPLAY_COMPAT_RAWKBD -DWSDIS! PLAY_DEFAULTSCREENS=6 -DWSDISPLAY_COMPAT_PCVT -DX86EMU -DONEWIREVERBOSE -DMAXUSERS=80 -D_KERNEL -MD -MP -c ../../../../dev/pci/vioscsi.c In file included from ../../../../dev/pci/vioscsi.c:25: ../../../../dev/pci/vioscsireg.h:125: error: redefinition of 'struct virtio_scsi_req_hdr' ../../../../dev/pci/vioscsireg.h:136: error: redefinition of 'struct virtio_scsi_res_hdr' ../../../../dev/pci/vioscsireg.h:197: error: redefinition of 'struct virtio_scsi_req_hdr' ../../../../dev/pci/vioscsireg.h:208: error: redefinition of 'struct virtio_scsi_res_hdr' In file included from ../../../../dev/pci/vioscsi.c:457: ../../../../dev/pci/vioscsireg.h:53: error: redefinition of 'struct virtio_scsi_req_hdr' ../../../../dev/pci/vioscsireg.h:64: error: redefinition of 'struct virtio_scsi_res_hdr' ../../../../dev/pci/vioscsireg.h:125: error: redefinition of 'struct virtio_scsi_req_hdr' ../../../../dev/pci/vioscsireg.h:136: error: redefinition of 'struct virtio_scsi_res_hdr' ../../../../dev/pci/vioscsireg.h:197: error: redefinition of 'struct virtio_scsi_req_hdr' ../../../../dev/pci/vioscsireg.h:208: error: redefinition of 'struct virtio_scsi_res_hdr' ../../../../dev/pci/vioscsi.c:463: error: redeclaration of enumerator 'debug' ../../../../dev/pci/vioscsi.c:31: error: previous definition of 'debug' was here ../../../../dev/pci/vioscsi.c:465: error: redefinition of 'struct vioscsi_req' ../../../../dev/pci/vioscsi.c:473: error: redefinition of 'struct vioscsi_softc' ../../../../dev/pci/vioscsi.c:499: error: redefinition of 'vioscsi_ca' ../../../../dev/pci/vioscsi.c:67: error: previous definition of 'vioscsi_ca' was here ../../../../dev/pci/vioscsi.c:505: error: redefinition of 'vioscsi_cd' ../../../../dev/pci/vioscsi.c:73: error: previous definition of 'vioscsi_cd' was here ../../../../dev/pci/vioscsi.c:511: error: redefinition of 'vioscsi_switch' ../../../../dev/pci/vioscsi.c:79: error: previous definition of 'vioscsi_switch' was here ../../../../dev/pci/vioscsi.c:516: error: redefinition of 'vioscsi_vq_names' ../../../../dev/pci/vioscsi.c:84: error: previous definition of 'vioscsi_vq_names' was here ../../../../dev/pci/vioscsi.c:524: error: redefinition of 'vioscsi_match' ../../../../dev/pci/vioscsi.c:92: error: previous definition of 'vioscsi_match' was here ../../../../dev/pci/vioscsi.c:534: error: redefinition of 'vioscsi_attach' ../../../../dev/pci/vioscsi.c:102: error: previous definition of 'vioscsi_attach' was here ../../../../dev/pci/vioscsi.c:598: error: redefinition of 'vioscsi_scsi_cmd' ../../../../dev/pci/vioscsi.c:166: error: previous definition of 'vioscsi_scsi_cmd' was here ../../../../dev/pci/vioscsi.c:706: error: redefinition of 'vioscsi_req_done' ../../../../dev/pci/vioscsi.c:274: error: previous definition of 'vioscsi_req_done' was here ../../../../dev/pci/vioscsi.c:745: error: redefinition of 'vioscsi_vq_done' ../../../../dev/pci/vioscsi.c:313: error: previous definition of 'vioscsi_vq_done' was here ../../../../dev/pci/vioscsi.c:771: error: redefinition of 'vioscsi_req_get' ../../../../dev/pci/vioscsi.c:339: error: previous definition of 'vioscsi_req_get' was here ../../../../dev/pci/vioscsi.c:821: error: redefinition of 'vioscsi_req_put' ../../../../dev/pci/vioscsi.c:389: error: previous definition of 'vioscsi_req_put' was here ../../../../dev/pci/vioscsi.c:841: error: redefinition of 'vioscsi_alloc_reqs' ../../../../dev/pci/vioscsi.c:409: error: previous definition of 'vioscsi_alloc_reqs' was here In file included from
Re: OpenBSD and Compute Engine
On Wed, Dec 4, 2013 at 9:19 AM, Jiri B ji...@devio.us wrote: I have problem to compile it :/ Are you sure you applied the patch cleanly? I'd recommend running cd /usr/src/sys ftp https://codereview.appspot.com/download/issue33540044_80001.diff patch issue33540044_80001.diff In file included from ../../../../dev/pci/vioscsi.c:889: vioscsi.c should only be 432 lines long, so I suspect you accidentally applied the patch three times?
Re: OpenBSD and Compute Engine
On Wed, Dec 04, 2013 at 09:33:18AM -0800, Matthew Dempsky wrote: On Wed, Dec 4, 2013 at 9:19 AM, Jiri B ji...@devio.us wrote: I have problem to compile it :/ Are you sure you applied the patch cleanly? I'd recommend running cd /usr/src/sys ftp https://codereview.appspot.com/download/issue33540044_80001.diff patch issue33540044_80001.diff In file included from ../../../../dev/pci/vioscsi.c:889: vioscsi.c should only be 432 lines long, so I suspect you accidentally applied the patch three times? You were right, PEBKAC issue. This is iscsi lun path-through via virtio-scsi on RHEVM 3.3: -%- virtio1 at pci0 dev 4 function 0 vendor Qumranet, unknown product 0x1004 rev 0x00: Virtio SCSI host Device vioscsi0 at virtio1: qsize 128 scsibus1 at vioscsi0: 255 targets sd0 at scsibus1 targ 0 lun 0: RH-str02, VIRTUAL-DISK, 0001 SCSI3 0/direct fixed t10.jbelka_ sd0: 112640MB, 512 bytes/sector, 230686720 sectors virtio1: apic 0 int 11 -%- virtio1 should be a HBA? http://www.ovirt.org/Features/Virtio-SCSI qemu-kvm cmd: -%- # ps axww | grep 'qemu.*obsd' 25739 ?Sl 6:18 /usr/libexec/qemu-kvm -name test-obsd -S -M rhel6.5.0 -cpu SandyBridge -enable-kvm -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 4571ad62-1505-48f4-a897-7888734b81f2 -smbios type=1,manufacturer=Red Hat,product=RHEV Hypervisor,version=6.5-20131127.0.el6,serial=4C4C4544-0058-3410-8058-C3C04F38354A,uuid=4571ad62-1505-48f4-a897-7888734b81f2 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/test-obsd.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2013-12-04T19:39:38,driftfix=slew -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw,serial= -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/rhev/data-center/mnt/10.34.63.199:_jbelka_jb33/3a2b98c! 2-74a0-45fa-8112-331099642625/images/9f1673a3-23a6-4b78-98e8-6a810bb570b4/9679f3dd-9dc7-413d-a0b2-926acb4128c2,if=none,id=drive-virtio-disk0,format=raw,serial=9f1673a3-23a6-4b78-98e8-6a810bb570b4,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/dev/mapper/1jbelka,if=none,id=drive-scsi0-0-0-0,format=raw,serial=,cache=none,werror=stop,rerror=stop,aio=native -device scsi-block,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 -netdev tap,fd=31,id=hostnet0,vhost=on,vhostfd=32 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:e3:3f:9f,bus=pci.0,addr=0x3 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/4571ad62-1505-48f4-a897-7888734b81f2.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1! ,path=/var/lib/libvirt/qemu/channels/4571ad62-1505-48f4-a897-7888734b81f2.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -spice port=5902,tls-port=5903,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -k en-us -vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=33554432 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -%- full dmesg: -%- OpenBSD 5.4-current (GENERIC) #0: Wed Dec 4 20:32:28 CET 2013 r...@test-obsd.rhev.lab.eng.brq.redhat.com:/usr/src/sys/arch/amd64/compile/GENERIC real mem = 1056952320 (1007MB) avail mem = 1020768256 (973MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.4 @ 0x3e90 (10 entries) bios0: vendor Seabios version 0.5.1 date 01/01/2007 bios0: Red Hat RHEV Hypervisor acpi0 at bios0: rev 0 acpi0: sleep states S5 acpi0: tables DSDT FACP SSDT APIC SSDT acpi0: wakeup devices acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat acpiprt0 at acpi0: bus 0 (PCI0) acpicpu0 at acpi0 mpbios0 at bios0: Intel MP Specification 1.4 cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel Xeon E312xx (Sandy Bridge), 3292.78 MHz cpu0: FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,CX16,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,LONG,LAHF cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line 16-way L2 cache cpu0: ITLB 255 4KB entries direct-mapped, 255
Re: OpenBSD and Compute Engine
On Wed, Dec 4, 2013 at 12:04 PM, Jiri B ji...@devio.us wrote: This is iscsi lun path-through via virtio-scsi on RHEVM 3.3: -%- virtio1 at pci0 dev 4 function 0 vendor Qumranet, unknown product 0x1004 rev 0x00: Virtio SCSI host Device vioscsi0 at virtio1: qsize 128 scsibus1 at vioscsi0: 255 targets sd0 at scsibus1 targ 0 lun 0: RH-str02, VIRTUAL-DISK, 0001 SCSI3 0/direct fixed t10.jbelka_ sd0: 112640MB, 512 bytes/sector, 230686720 sectors virtio1: apic 0 int 11 -%- virtio1 should be a HBA? http://www.ovirt.org/Features/Virtio-SCSI Yep, the above looks right to me. But something is odd... -%- # dmesg | tail ASC/ASCQ: Illegal Field in CDB sd0(vioscsi0:0:0): Check Condition (error 0x70) on opcode 0x28 SENSE KEY: Illegal Request ASC/ASCQ: Illegal Field in CDB sd0(vioscsi0:0:0): Check Condition (error 0x70) on opcode 0x28 SENSE KEY: Illegal Request ASC/ASCQ: Illegal Field in CDB sd0(vioscsi0:0:0): Check Condition (error 0x70) on opcode 0x28 SENSE KEY: Illegal Request ASC/ASCQ: Illegal Field in CDB -%- That is pretty odd. I wonder if we're not properly zeroing out the CDB somewhere. I'll take a look.
Re: OpenBSD and Compute Engine
On Tue, Dec 03, 2013 at 12:12:09PM -0800, Matthew Dempsky wrote: Compute Engine now supports custom OSes [1], so I've been working on getting OpenBSD working on it. I thought I'd share a status update. [1] http://googlecloudplatform.blogspot.com/2013/12/google-compute-engine-is-now-generally-available.html ** virtio-scsi Compute Engine exposes disks as virtio-scsi devices, whereas OpenBSD currently only supports virtio-blk disks. I've got a mostly working virtio-scsi driver (WIP: https://codereview.appspot.com/33540044), but I still need to polish it a bit. Aha, big brother grows up :-) I will try to test virtio-scsi on RHEVM which also exposes this to VMs. Have you tried OpenBSD on SmartOS? Some people say it would need some love (some storage issue). jirib
Re: OpenBSD and Compute Engine
+-- | On 2013-12-03 16:35:45, Jiri B wrote: | | Have you tried OpenBSD on SmartOS? Some people say it would need | some love (some storage issue). I used OpenBSD on SmartOS a couple years ago. There was a bug in KVM that would kill the host, but that's been fixed for a while... Storage works fine if you use IDE. :-) -- bdha cyberpunk is dead. long live cyberpunk.
Re: OpenBSD and Compute Engine
On Tue, Dec 3, 2013 at 1:35 PM, Jiri B ji...@devio.us wrote: Aha, big brother grows up :-) I will try to test virtio-scsi on RHEVM which also exposes this to VMs. Cool, let me know if you run into any issues. I'm sure there's still some bugs in the driver so I wouldn't put it into production yet, but it should at least let you mount a virtio-scsi disk. Have you tried OpenBSD on SmartOS? Some people say it would need some love (some storage issue). Nope, haven't tried that, and probably won't to be honest. I don't really have a need for running OpenBSD on Compute Engine even, just thought it would be a fun weekend project when I heard about it. :)
Re: OpenBSD and Compute Engine
On Tue, Dec 03, 2013 at 04:45:28PM -0500, Bryan Horstmann-Allen wrote: I used OpenBSD on SmartOS a couple years ago. There was a bug in KVM that would kill the host, but that's been fixed for a while... I saw when a VM with virtio devices killed RHEL host too. Funny :) Baf, it was secured with SELinux, haha :) jirib
