Updated and recompiled. Here's what it's doing now. No idea what the Lock
Order Reversal noise is between the two VM starts, but I've seen a lot of
it on this laptop lately, even without using vmm. Probably unrelated.
"vmctl start 3" refers to "OBSDSnap64.vm" in the vm.conf up thread.
[axon@transient ~]$ doas vmctl start foo -c -b /bsd.rd -m 512m -i 1 -n local
vmctl: starting without disks
Connected to /dev/ttyp1 (speed 115200)
[EOT]
[axon@transient ~]$ vmctl start 3 -c
Connected to /dev/ttyp2 (speed 115200)
[EOT]
[axon@transient ~]$ dmesg
OpenBSD 6.3-current (GENERIC.MP) #2: Fri Jun 29 08:17:14 CDT 2018
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8227655680 (7846MB)
avail mem = 7902158848 (7536MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xe9d90 (47 entries)
bios0: vendor Acer version "V1.07" date 11/07/2011
bios0: Acer Aspire 5733Z
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP ASF! HPET APIC MCFG SLIC BOOT ASPT DMAR SSDT SSDT
SSDT
acpi0: wakeup devices AZAL(S3) P0P1(S4) EHC1(S3) USB1(S3) USB2(S3) USB3(S3)
USB4(S3) USB5(S3) USB6(S3) USB7(S3) RP01(S4) PEG5(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz, 2527.87 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 133MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz, 2527.44 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz, 2527.44 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 5 (application processor)
cpu3: Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz, 2527.44 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 1, core 2, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
, remapped to apid 2
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (P0P2)
acpiprt2 at acpi0: bus 3 (P0P1)
acpiprt3 at acpi0: bus 1 (RP01)
acpiprt4 at acpi0: bus 2 (RP02)
acpiprt5 at acpi0: bus -1 (RP03)
acpiprt6 at acpi0: bus -1 (RP04)
acpiprt7 at acpi0: bus -1 (RP05)
acpiprt8 at acpi0: bus -1 (RP06)
acpiprt9 at acpi0: bus -1 (RP07)
acpiprt10 at acpi0: bus -1 (RP08)
acpiprt11 at acpi0: bus -1 (PEG5)
acpiec0 at acpi0
acpicpu0 at acpi0: C3(350@245 mwait.3@0x20), C1(1000@3 mwait.1), PSS
acpicpu1 at acpi0: C3(350@245 mwait.3@0x20), C1(1000@3 mwait.1), PSS
acpicpu2 at acpi0: C3(350@245 mwait.3@0x20), C1(1000@3 mwait.1), PSS
acpicpu3 at acpi0: C3(350@245 mwait.3@0x20), C1(1000@3 mwait.1), PSS
"pnp0c14" at acpi0 not configured
acpibat0 at acpi0: BAT0 model "Li_Ion_4000mA " serial 78da type Lion oem
"SANYO "
acpiac0 at acpi0: AC unit online
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: LID0
acpibtn2 at acpi0: SLPB
acpicmos0 at acpi0
"ETD0500" at acpi0 not configured
"PNP0C14" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD03
acpivideo1 at acpi0: VGA_
cpu0: Enhanced SpeedStep 2527 MHz: speeds: 2533, 2399, 2266, 2133, 1999,
1866, 1733, 1599, 1466, 1333, 1199 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Core Host" rev 0x02
inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics" rev 0x02
drm0 at inteldrm0
intagp0 at inteldrm0
agp0 at intagp0: aperture at 0xc0000000, size 0x10000000
inteldrm0: msi
inteldrm0: 1366x768, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
"Intel 3400 MEI" rev 0x06 at pci0 dev 22 function 0 not configured
ehci0 at pci0 dev 26 function 0 "Intel 3400 USB" rev 0x05: apic 2 int 16
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
azalia0 at pci0 dev 27 function 0 "Intel 3400 HD Audio" rev 0x05: msi
azalia0: codecs: Realtek ALC272
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 3400 PCIE" rev 0x05: msi
pci1 at ppb0 bus 1
bge0 at pci1 dev 0 function 0 "Broadcom BCM57780" rev 0x01, BCM57780 A1
(0x57780001): msi, address dc:0e:a1:91:7a:b9
brgphy0 at bge0 phy 1: BCM57780 10/100/1000baseT PHY, rev. 1
ppb1 at pci0 dev 28 function 1 "Intel 3400 PCIE" rev 0x05: msi
pci2 at ppb1 bus 2
athn0 at pci2 dev 0 function 0 "Atheros AR9285" rev 0x01: apic 2 int 17
athn0: AR9285 rev 2 (1T1R), ROM rev 13, address 00:25:d3:f4:f8:4b
ehci1 at pci0 dev 29 function 0 "Intel 3400 USB" rev 0x05: apic 2 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb2 at pci0 dev 30 function 0 "Intel 82801BAM Hub-to-PCI" rev 0xa5
pci3 at ppb2 bus 3
pcib0 at pci0 dev 31 function 0 "Intel HM55 LPC" rev 0x05
ahci0 at pci0 dev 31 function 2 "Intel 3400 AHCI" rev 0x05: msi, AHCI 1.3
ahci0: port 0: 3.0Gb/s
ahci0: port 1: 1.5Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, SanDisk SDSSDA12, Z320> SCSI3 0/direct
fixed naa.5001b444a668bf8d
sd0: 114473MB, 512 bytes/sector, 234441648 sectors, thin
cd0 at scsibus1 targ 1 lun 0: <HL-DT-ST, CDRW/DVD GCCT20N, A108> ATAPI
5/cdrom removable
ichiic0 at pci0 dev 31 function 3 "Intel 3400 SMBus" rev 0x05: apic 2 int 19
iic0 at ichiic0
spdmem0 at iic0 addr 0x50: 4GB DDR3 SDRAM PC3-10600 SO-DIMM
spdmem1 at iic0 addr 0x52: 4GB DDR3 SDRAM PC3-10600 SO-DIMM
itherm0 at pci0 dev 31 function 6 "Intel 3400 Thermal" rev 0x05
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
pms0: Elantech Touchpad, version 2, firmware 0x40213
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
pci4 at mainbus0 bus 255
pchb1 at pci4 dev 0 function 0 "Intel QuickPath" rev 0x02
pchb2 at pci4 dev 0 function 1 "Intel QuickPath" rev 0x02
pchb3 at pci4 dev 2 function 0 "Intel QPI Link" rev 0x02
pchb4 at pci4 dev 2 function 1 "Intel QPI Physical" rev 0x02
pchb5 at pci4 dev 2 function 2 "Intel Reserved" rev 0x02
pchb6 at pci4 dev 2 function 3 "Intel Reserved" rev 0x02
vmm0 at mainbus0: VMX/EPT
uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching Hub"
rev 2.00/0.00 addr 2
uvideo0 at uhub2 port 1 configuration 1 interface 0 "SuYin 1.3M HD WebCam"
rev 2.00/1.02 addr 3
video0 at uvideo0
umass0 at uhub2 port 2 configuration 1 interface 0 "Generic USB2.0-CRW" rev
2.00/38.82 addr 4
umass0: using SCSI over Bulk-Only
scsibus2 at umass0: 2 targets, initiator 0
sd1 at scsibus2 targ 1 lun 0: <Generic-, Multi-Card, 1.00> SCSI0 0/direct
removable serial.0bda0138516388200000
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub"
rev 2.00/0.00 addr 2
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on sd0a (7c4cada893c18112.a) swap on sd0b dump on sd0b
vm_impl_init_vmx: created vm_map @ 0xffff800000d10c00
vm_resetcpu: resetting vm 1 vcpu 0 to power on defaults
Guest EPTP = 0x23908301e
vmm_alloc_vpid: allocated VPID/ASID 1
vmm_fpurestore: guest attempted to set invalid bits in xcr0 (guest
%xcr0=0x1, host mask=0xffffffffffffffff)
vmm_free_vpid: freed VPID/ASID 1
lock order reversal:
1st 0xffffff01f98a8450 vmmaplk (&map->lock) @
/mnt/src/sys/uvm/uvm_fault.c:1441
2nd 0xffff80000015c138 drmdevlk (&dev->struct_mutex) @
/mnt/src/sys/dev/pci/drm/i915/i915_gem.c:1801
lock order "&dev->struct_mutex"(rwlock) -> "&map->lock"(rwlock) first seen
at:
#0 witness_checkorder+0x4c0
#1 _rw_enter+0x68
#2 vm_map_lock_ln+0xbc
#3 uvm_map+0x1a1
#4 km_alloc+0x16a
#5 bus_space_map+0x159
#6 i965_alloc_ifp+0xc1
#7 intel_gtt_chipset_setup+0x1b1
#8 intel_enable_gtt+0x26
#9 i915_gem_init_hw+0x43
#10 i915_gem_init+0x24e
#11 i915_driver_load+0xfc1
#12 inteldrm_attach+0x37f
#13 config_attach+0x20e
#14 pci_probe_device+0x501
#15 pci_enumerate_bus+0xb7
#16 config_attach+0x20e
#17 mainbus_attach+0x24a
#18 config_attach+0x20e
lock order "&map->lock"(rwlock) -> "&dev->struct_mutex"(rwlock) first seen
at:
#0 witness_checkorder+0x4c0
#1 _rw_enter_write+0x53
#2 i915_gem_object_wait_rendering__nonblocking+0x1fa
#3 i915_gem_fault+0x144
#4 drm_fault+0x18a
#5 uvm_fault+0x743
#6 pageflttrap+0x14c
#7 trap+0x2b6
#8 recall_trap+0x8
vm_impl_init_vmx: created vm_map @ 0xffff800000d5f000
vm_resetcpu: resetting vm 2 vcpu 0 to power on defaults
Guest EPTP = 0x1f7df601e
vmm_alloc_vpid: allocated VPID/ASID 1
vmm_fpurestore: guest attempted to set invalid bits in xcr0 (guest
%xcr0=0x1, host mask=0xffffffffffffffff)
vmm_free_vpid: freed VPID/ASID 1
[axon@transient ~]$
On Thu, Jun 28, 2018 at 11:50 PM, Mike Larkin <[email protected]> wrote:
> On Wed, Jun 27, 2018 at 05:01:04AM -0500, Ax0n wrote:
> > Trying to spin up a diskless vmm with the latest bsd.rd:
> > [axon@transient vmm]$ doas vmctl start foo -c -b /bsd.rd -m 512m -i 1 -n
> > local
> > vmctl: starting without disks
> > Connected to /dev/ttyp3 (speed 115200)
> >
> > [EOT]
> > [axon@transient vmm]$ dmesg | tail
> > Guest EPTP = 0x112d3001e
> > vmm_alloc_vpid: allocated VPID/ASID 1
> > vmm_fpurestore: guest attempted to set invalid bits in xcr0
> > vmm_free_vpid: freed VPID/ASID 1
> > vm_impl_init_vmx: created vm_map @ 0xffff800000730600
> > vm_resetcpu: resetting vm 4 vcpu 0 to power on defaults
> > Guest EPTP = 0x112d3001e
> > vmm_alloc_vpid: allocated VPID/ASID 1
> > vmm_fpurestore: guest attempted to set invalid bits in xcr0
>
> This is the problem, although we don't know why it would be happening. Can
> you apply the following diff and give me the new dmesg line? This will tell
> me what the guest is trying to do.
>
> I'll also apply this to -current, so you could also just update your tree
> and
> you'll get the diff that way also.
>
> -ml
>
> Index: vmm.c
> ===================================================================
> RCS file: /cvs/src/sys/arch/amd64/amd64/vmm.c,v
> retrieving revision 1.202
> diff -u -p -a -u -r1.202 vmm.c
> --- vmm.c 22 Jun 2018 05:21:45 -0000 1.202
> +++ vmm.c 29 Jun 2018 04:47:32 -0000
> @@ -3829,8 +3829,9 @@ vmm_fpurestore(struct vcpu *vcpu)
> if (vcpu->vc_fpuinited) {
> /* Restore guest XCR0 and FPU context */
> if (vcpu->vc_gueststate.vg_xcr0 & ~xsave_mask) {
> - DPRINTF("%s: guest attempted to set invalid %s\n",
> - __func__, "bits in xcr0");
> + DPRINTF("%s: guest attempted to set invalid bits
> in "
> + "xcr0 (guest %%xcr0=0x%llx, host
> mask=0x%llx)\n",
> + __func__, vcpu->vc_gueststate.vg_xcr0,
> ~xsave_mask);
> return EINVAL;
> }
>
>
