Re: Illumos boot

2015-10-16 Thread Andriy Gapon 
On 15/10/2015 18:56, Peter Grehan wrote:
> Hi Matt,
> 
>>> -s 0,hostbridge
> 
>  Things should work if you leave out the hostbridge. The PCIe capability
> that is tacked on to this will make Illumos use MSI/MSIx for the virtio
> adapter which apparently hits a bug in the driver. Without it, the
> virtio driver will fall back to legacy interrupts. This also means that
> the virtio adapter will be confined to slots 3/4/5/6.
> 
> [root@smartos ~]# echo ::interrupts | mdb -k
> IRQ  Vect IPL BusTrg Type   CPU Share APIC/INT# ISR(s)
> 10x40 5   ISAEdg Fixed  0   1 0x0/0x1   i8042_intr
> 30xb1 12  ISAEdg Fixed  0   1 0x0/0x3   asyintr
> 40xb0 12  ISAEdg Fixed  0   1 0x0/0x4   asyintr
> 90x81 9   PCILvl Fixed  1   1 0x0/0x9   acpi_wrapper_isr
> 12   0x41 5   ISAEdg Fixed  1   1 0x0/0xc   i8042_intr
> 16   0x42 5   PCILvl Fixed  1   1 0x0/0x10  ahci_intr
> 17   0x43 5   PCILvl Fixed  0   1 0x0/0x11  ahci_intr
> 18   0x60 6   PCILvl Fixed  1   1 0x0/0x12  virtio_intx_dispatch
> 160  0xa0 0  Edg IPIall 0 - poke_cpu
> 208  0xd0 14 Edg IPIall 1 - kcpc_hw_overflow_intr
> 209  0xd1 14 Edg IPIall 1 - cbe_fire
> 210  0xd3 14 Edg IPIall 1 - cbe_fire
> 240  0xe0 15 Edg IPIall 1 - xc_serv
> 241  0xe1 15 Edg IPIall 1 - apic_error_intr
> 
> 
>  (I believe Andriy Gapon (cc'd) has a fix for this in Illumos)

Yes, it's here https://www.illumos.org/rb/r/86/
I plan to RTI that change soon-ish.


-- 
Andriy Gapon
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

RE: Illumos boot

2015-10-15 Thread Matt Churchyard via freebsd-virtualization 
> Hi,

>> ..
> /pci@0,0/pci8086,2821@4/disk@0,0 (sd1) online
> NOTICE: vioif0: Got MAC address from host: e4:94:1:0:ff:ff
> pseudo-device: tun0
> tun0 is /pseudo/tun@0
> pseudo-device: lx_systrace0
> lx_systrace0 is /pseudo/lx_systrace@0
> 
> panic[cpu0]/thread=ff0002566c40: BAD TRAP: type=d (#gp General 
> protection) rp=ff00025664c0 addr=20
> 
> sched: #gp General protection
> addr=0x20
> pid=0, pc=0xf80d375a, sp=0xff00025665b0, eflags=0x10282
> cr0: 8005003b cr4: 
> 406b8
> cr2: fed3b5accr3: 1dc0cr8: c
> 
>rdi: 7f1a90ff00c3 rsi: ff00c33321f8 rdx: ff00c37f5828
>rcx: ff00c3868603  r8: ff00ca7aa600  r9: 2ba6
>rax:0 rbx: ff00c30d0ef0 rbp: ff00025665c0
>r10: fb8554c4 r11:1 r12:   1f
>r13: ff00c319f880 r14:   10 r15:   20
>fsb:0 gsb: fbc326a0  ds:   4b
> es:   4b  fs:0  gs:  1c3
>trp:d err:0 rip: f80d375a
> cs:   30 rfl:10282 rsp: ff00025665b0
> ss:   38
> 
> This is the log of the bhyve options used (apart from 1 cpu, 1G ram)
> 
> Oct 13 14:22:43:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 
> 4:0,ahci-hd,/data/vm/smartos/disk0.img -s 7:0,virtio-net,tap1] Oct 13 
> 14:22:43:  [bhyve console: -l com1,/dev/nmdm1A -l com2,/dev/nmdm2A] 
> Oct 13 14:22:43:  [bhyve options: -Hw -l 
> bootrom,/data/vm/.config/BHYVE_UEFI_CSM.fd]
> Oct 13 14:22:43:  [bhyve iso device: -s 
> 3:0,ahci-cd,/data/vm/.iso/smartos-latest.iso]

> Ouch, even with the additional verbosity, the output isn't very insightful.  
> All I can glean is that you are reasonably far along in the boot > process.  
> You could try to run with KMDB (-k boot option) and do some disassembly 
> around that program counter to see if any specific module is > implicated.

> Alternately, I'd omit the network device and see how far that gets you.

It does appear that I can get as far as a login console without the virtio-net 
device

Matt
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Illumos boot

2015-10-15 Thread Peter Grehan 

Hi Matt,


-s 0,hostbridge


 Things should work if you leave out the hostbridge. The PCIe 
capability that is tacked on to this will make Illumos use MSI/MSIx for 
the virtio adapter which apparently hits a bug in the driver. Without 
it, the virtio driver will fall back to legacy interrupts. This also 
means that the virtio adapter will be confined to slots 3/4/5/6.


[root@smartos ~]# echo ::interrupts | mdb -k
IRQ  Vect IPL BusTrg Type   CPU Share APIC/INT# ISR(s)
10x40 5   ISAEdg Fixed  0   1 0x0/0x1   i8042_intr
30xb1 12  ISAEdg Fixed  0   1 0x0/0x3   asyintr
40xb0 12  ISAEdg Fixed  0   1 0x0/0x4   asyintr
90x81 9   PCILvl Fixed  1   1 0x0/0x9   acpi_wrapper_isr
12   0x41 5   ISAEdg Fixed  1   1 0x0/0xc   i8042_intr
16   0x42 5   PCILvl Fixed  1   1 0x0/0x10  ahci_intr
17   0x43 5   PCILvl Fixed  0   1 0x0/0x11  ahci_intr
18   0x60 6   PCILvl Fixed  1   1 0x0/0x12  virtio_intx_dispatch
160  0xa0 0  Edg IPIall 0 - poke_cpu
208  0xd0 14 Edg IPIall 1 - kcpc_hw_overflow_intr
209  0xd1 14 Edg IPIall 1 - cbe_fire
210  0xd3 14 Edg IPIall 1 - cbe_fire
240  0xe0 15 Edg IPIall 1 - xc_serv
241  0xe1 15 Edg IPIall 1 - apic_error_intr


 (I believe Andriy Gapon (cc'd) has a fix for this in Illumos)

later,

Peter.

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

RE: Illumos boot

2015-10-15 Thread Matt Churchyard via freebsd-virtualization 


>> -s 0,hostbridge

> Things should work if you leave out the hostbridge. The PCIe
> capability that is tacked on to this will make Illumos use MSI/MSIx for
> the virtio adapter which apparently hits a bug in the driver. Without
> it, the virtio driver will fall back to legacy interrupts. This also
> means that the virtio adapter will be confined to slots 3/4/5/6.

> [root@smartos ~]# echo ::interrupts | mdb -k
> IRQ  Vect IPL BusTrg Type   CPU Share APIC/INT# ISR(s)
> 10x40 5   ISAEdg Fixed  0   1 0x0/0x1   i8042_intr
> 30xb1 12  ISAEdg Fixed  0   1 0x0/0x3   asyintr
> 40xb0 12  ISAEdg Fixed  0   1 0x0/0x4   asyintr
> 90x81 9   PCILvl Fixed  1   1 0x0/0x9   acpi_wrapper_isr
> 12   0x41 5   ISAEdg Fixed  1   1 0x0/0xc   i8042_intr
> 16   0x42 5   PCILvl Fixed  1   1 0x0/0x10  ahci_intr
> 17   0x43 5   PCILvl Fixed  0   1 0x0/0x11  ahci_intr
> 18   0x60 6   PCILvl Fixed  1   1 0x0/0x12  virtio_intx_dispatch
> 160  0xa0 0  Edg IPIall 0 - poke_cpu
> 208  0xd0 14 Edg IPIall 1 - kcpc_hw_overflow_intr
> 209  0xd1 14 Edg IPIall 1 - cbe_fire
> 210  0xd3 14 Edg IPIall 1 - cbe_fire
> 240  0xe0 15 Edg IPIall 1 - xc_serv
> 241  0xe1 15 Edg IPIall 1 - apic_error_intr


>   (I believe Andriy Gapon (cc'd) has a fix for this in Illumos)

> later,

> Peter.

Thanks Peter. I can't believe how many times I checked you Illumos instructions 
making sure there were no obvious differences, and didn't notice the hostbridge 
was missing. I think I just had it in my mind that the hostbridge was a basic 
required device.

Regards
Matt
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

RE: Illumos boot

2015-10-13 Thread Matt Churchyard via freebsd-virtualization 
>Hi,

>Please see inline.

>On Oct 13, 2015, at 7:17 AM, Matt Churchyard via freebsd-virtualization 
> wrote:

> In my quest to continue expanding guest support in my vm-bhyve utility (See 
> https://github.com/churchers/vm-bhyve :) ), I've found the Windows support 
> pretty solid once I got clear on the slot requirements. I'm now trying an OS 
> that requires CSM (Illumos) but unfortunately I'm currently struggling to get 
> it to boot up correctly.
> 
> Here's an example of the command I'm generating at the moment (This is 
> running on an Intel Core-i3):
> 
> bhyve -c 2 -m 2G -s 0,hostbridge -s 31,lpc \
>  -s 3,ahci-cd,/data/vm/.iso/smartos-latest.iso \
>  -s 4:0,ahci-hd,/data/vm/smartos/disk0.img \
>  -s 5:0,virtio-net,tap0 \
>  -l com1,stdio -l com2,/dev/nmdm2A \
>  -H -l bootrom,/data/vm/.config/BHYVE_UEFI_CSM.fd \
>  smartos
> 
> I have com1 set to stdio so I can easily watch the output as it runs.
> It tends to get as far as "Legacy INT19 Boot...", then fall over.
> Depending on whether I put the network interface directly in the slot after 
> the HDD, I seem to get different errors -
> 
> slot 3 - cd
> slot 4 - hdd
> slot 5 - virtio-net
> 
> panic[cpu0]/thread=ff01457cdb40: BAD TRAP: type=e (#pf Page fault) 
> rp=ff0004a69a60 addr=40 occurred in module "genunix" due to a NULL 
> pointer dereference
> 
> slot 3 - cd
> slot 4 - hdd
> slot 7 - virtio-net
> 
> panic[cpu1]/thread=ff0004002c40: BAD TRAP: type=d (#gp General 
> protection) rp=ff0004002740 addr=0
> 
> On com2 I see the boot menu, then one and a half lines of dots. The second 
> line of dots stops about 2/3 of the way across.

>Have you tried booting illumos in verbose mode - edit the grub command line 
>and provide '-v'.  That may give you a better backtrace than a >program 
>counter.

This is what I get from a verbose boot:

Bhyve-HandleProtocol: Copying DevPath: 
PciRoot(0x0)/Pci(0x3,0x0)/Sata(0x0,0x0,0x0) [32]
Legacy INT19 Boot...
cpu0: x86 (chipid 0x0 GenuineIntel 206A7 family 6 model 42 step 7 clock 3109 
MHz)
cpu0: Intel(r) Core(tm) i3-2100 CPU @ 3.10GHz
pseudo-device: stmf_sbd0
stmf_sbd0 is /pseudo/stmf_sbd@0
pseudo-device: lofi0
lofi0 is /pseudo/lofi@0
pseudo-device: devinfo0
devinfo0 is /pseudo/devinfo@0
acpinex0 at root
acpinex0 is /fw
iscsi0 at root
iscsi0 is /iscsi
xsvc0 at root: space 0 offset 0
xsvc0 is /xsvc@0,0
acpinex: sb@0, acpinex1
acpinex1 is /fw/sb@0
pseudo-device: pseudo1
pseudo1 is /pseudo/zconsnex@1
pseudo-device: pseudo2
pseudo2 is /pseudo/zfdnex@2
/pci@0,0/pci8086,2821@3 :
SATA CD/DVD (ATAPI) device at port 0
model BHYVE SATA DVD ROM
firmware 001
serial number BHYVE-EA14-A68A-54FA
supported features:
 DMA
SATA Gen3 signaling speed (6.0Gbps)
pseudo-device: llc10
llc10 is /pseudo/llc1@0
pseudo-device: power0
power0 is /pseudo/power@0
pseudo-device: ramdisk1024
ramdisk1024 is /pseudo/ramdisk@1024
pseudo-device: ucode0
ucode0 is /pseudo/ucode@0
pseudo-device: zfs0
zfs0 is /pseudo/zfs@0
pseudo-device: srn0
srn0 is /pseudo/srn@0
pseudo-device: dtrace0
dtrace0 is /pseudo/dtrace@0
pseudo-device: dcpc0
dcpc0 is /pseudo/dcpc@0
pseudo-device: fasttrap0
fasttrap0 is /pseudo/fasttrap@0
pseudo-device: fbt0
fbt0 is /pseudo/fbt@0
pseudo-device: profile0
profile0 is /pseudo/profile@0
pseudo-device: lockstat0
lockstat0 is /pseudo/lockstat@0
pseudo-device: sdt0
sdt0 is /pseudo/sdt@0
pseudo-device: systrace0
systrace0 is /pseudo/systrace@0
pseudo-device: ipd0
ipd0 is /pseudo/ipd@0
pseudo-device: stmf0
stmf0 is /pseudo/stmf@0
sd0 at ahci0: target 0 lun 0
sd0 is /pci@0,0/pci8086,2821@3/cdrom@0,0
pseudo-device: fssnap0
fssnap0 is /pseudo/fssnap@0
/pci@0,0/pci8086,2821@3/cdrom@0,0 (sd0) online
/pci@0,0/pci8086,2821@4 :
SATA disk device at port 0
model BHYVE SATA DISK
firmware 001
serial number BHYVE-3083-1AF1-1754
supported features:
 48-bit LBA, DMA, Native Command Queueing
SATA Gen3 signaling speed (6.0Gbps)
Supported queue depth 32
capacity = 62914560 sectors
WARNING: kvm: no hardware support

pseudo-device: pool0
pool0 is /pseudo/pool@0
pseudo-device: bpf0
bpf0 is /pseudo/bpf@0
sd1 at ahci1: target 0 lun 0
sd1 is /pci@0,0/pci8086,2821@4/disk@0,0
pseudo-device: pm0
pm0 is /pseudo/pm@0
pseudo-device: nsmb0
nsmb0 is /pseudo/nsmb@0
pseudo-device: tap0
tap0 is /pseudo/tap@0
/pci@0,0/pci8086,2821@4/disk@0,0 (sd1) online
NOTICE: vioif0: Got MAC address from host: e4:94:1:0:ff:ff
pseudo-device: tun0
tun0 is /pseudo/tun@0
pseudo-device: lx_systrace0
lx_systrace0 is /pseudo/lx_systrace@0

panic[cpu0]/thread=ff0002566c40: BAD TRAP: type=d (#gp General protection) 
rp=ff00025664c0 addr=20

sched: #gp General protection
addr=0x20
pid=0, pc=0xf80d375a, sp=0xff00025665b0, eflags=0x10282
cr0: 8005003b cr4: 406b8
cr2: fed3b5accr3: 1dc0cr8: c

rdi:

Re: Illumos boot

2015-10-13 Thread Tycho Nightingale via freebsd-virtualization 

Hi,

Please see inline.

On Oct 13, 2015, at 7:17 AM, Matt Churchyard via freebsd-virtualization 
 wrote:

> In my quest to continue expanding guest support in my vm-bhyve utility (See 
> https://github.com/churchers/vm-bhyve :) ), I've found the Windows support 
> pretty solid once I got clear on the slot requirements. I'm now trying an OS 
> that requires CSM (Illumos) but unfortunately I'm currently struggling to get 
> it to boot up correctly.
> 
> Here's an example of the command I'm generating at the moment (This is 
> running on an Intel Core-i3):
> 
> bhyve -c 2 -m 2G -s 0,hostbridge -s 31,lpc \
>  -s 3,ahci-cd,/data/vm/.iso/smartos-latest.iso \
>  -s 4:0,ahci-hd,/data/vm/smartos/disk0.img \
>  -s 5:0,virtio-net,tap0 \
>  -l com1,stdio -l com2,/dev/nmdm2A \
>  -H -l bootrom,/data/vm/.config/BHYVE_UEFI_CSM.fd \
>  smartos
> 
> I have com1 set to stdio so I can easily watch the output as it runs.
> It tends to get as far as "Legacy INT19 Boot...", then fall over.
> Depending on whether I put the network interface directly in the slot after 
> the HDD, I seem to get different errors -
> 
> slot 3 - cd
> slot 4 - hdd
> slot 5 - virtio-net
> 
> panic[cpu0]/thread=ff01457cdb40: BAD TRAP: type=e (#pf Page fault) 
> rp=ff0004a69a60 addr=40 occurred in module "genunix" due to a NULL 
> pointer dereference
> 
> slot 3 - cd
> slot 4 - hdd
> slot 7 - virtio-net
> 
> panic[cpu1]/thread=ff0004002c40: BAD TRAP: type=d (#gp General 
> protection) rp=ff0004002740 addr=0
> 
> On com2 I see the boot menu, then one and a half lines of dots. The second 
> line of dots stops about 2/3 of the way across.

Have you tried booting illumos in verbose mode — edit the grub command line and 
provide ‘-v’.  That may give you a better backtrace than a program counter.

> Interestingly, my code normally puts the CD after the HDD, which Windows 
> seems happy with as long as the slots are consecutive.
> In SmartOS this gives me a different error:
> 
> slot 3 - hdd
> slot 4 - cd
> slot 5 - virtio-net
> 
> PlatformBdsBootFail
> Boot Failed. Harddisk 1
>  Find PE image 
> /home/grehan/proj/stock_edk2/Build/BhyveX64/DEBUG_GCC48/X64/UefiCpuPkg/CpuDxe/CpuDxe/DEBUG/CpuDxe.dll
>  (ImageBase=7F8DC000, EntryPoint=7F8DC2AF) 

This error is from the UEFI code.  It implies that the CSM boot failed or was 
never invoked.  If the HDD isn’t bootable, yet the CD is, that is the most 
likely source as the CSM assumes the first block device it encounters is the 
desired boot source.

Tycho
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"