On 24-10-2016 13:02, Roger Pau Monné wrote:
> On Sat, Oct 22, 2016 at 04:14:33PM -0400, n...@bigfoot.com wrote:
>> Running a FreeBSD 11.0-RELEASE HVM guest, using ufs, on Xen 4.4 (debian
>> jessie) on LVM with drbd 8.4. The FreeBSD guest boots fine from LVM
>> using 'disk' /dev// in domU config.
>>
>> I have a number of Linux guests running on drbd (8.4). Booting FreeBSD
>> from the drbd device associated with drbd (with 'disk' /dev/drbdx)
>> fails. I know I can't use drbd: with HVM guest.
>>
>> Trying to mount root from ufs:/dev/xbd0p2 [rw]...
>> mountroot: waiting for device /dev/xbd0p2...
>> Mounting from ufs:/dev/xbd0p2 failed with error 19.
>
> Ops, hit send to soon. Can you also paste the output of `xenstore-ls -fp`
> executed from Dom0 when the guest is in this state?
>
> Thanks, Roger.
>
Thanks for replying.
Sorry for the long post.
There is one Linux PV guest on this host, not on drbd, but there are
configured drbd devices on this host that belong to linux PV guests on
another host also running debian jessie. They boot from drbd without
problems.
To collect more data I created a fresh FreeBSD HVM guest on the same
host. Interestingly, this guest does boot from drbd. You'll find it as
domain 16 (vps13) in the attached output of xenstore-ls.
The HVM guest that fails to mount root is domain 17 (vps10).
The error message is:
Trying to mount root from ufs:/dev/xbd0p2 [rw]...
GEOM: xbd0: corrupt or invalid GPT detected.
GEOM: xbd0: GPT rejected -- may not be recoverable.
I should mention that there seems to be a conflict with the second GEOM
GPT table in FreeBSD and the drbd metadata both writing to the end of
the partition (we are using internal metadata). After creating metadata
(drbdadm create-md) the kernel complains about the secondary GPT table,
but that does not prevent it from mounting the fs.
Inside the domu this can be resolved by issuing 'gpart recover', but
this in turn seems to destroy the drbd metadata, which is apparent by
the need to re-create md and re-sync at the next boot.
Perhaps we should use external metadata in this case.
As an aside, for your information, we tried to run the FreeBSD 11.0
guest in PVH mode as well, having followed the instructions on
https://wiki.xen.org/wiki/FreeBSD_PVH.
The guest failed to start with xl reporting en error:
xc: error: elf_xen_addr_calc_check: ERROR: ELF start or entries are out
of bounds.: Invalid kernel
Output of readelf -l /xenkernels/freebsd/kernel-11.0-RELEASE:
Elf file type is EXEC (Executable file)
Entry point 0x802ff000
There are 6 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSizMemSiz Flags Align
PHDR 0x0040 0x80200040 0x00200040
0x0150 0x0150 R E8
INTERP 0x0190 0x80200190 0x00200190
0x000d 0x000d R 1
[Requesting program interpreter: /red/herring]
LOAD 0x 0x8020 0x0020
0x014ed860 0x014ed860 R E20
LOAD 0x014ee000 0x818ee000 0x018ee000
0x00132538 0x005ecfa0 RW 20
DYNAMIC0x014ee000 0x818ee000 0x018ee000
0x00d0 0x00d0 RW 8
GNU_STACK 0x 0x 0x
0x 0x RWE8
Section to Segment mapping:
Segment Sections...
00
01 .interp
02 .interp .hash .dynsym .dynstr .text .rodata set_sysctl_set
set_sysinit_set set_sysuninit_set set_modmetadata_set set_ah_chips
set_ah_rfs set_kbddriver_set set_sdt_providers_set set_sdt_probes_set
set_sdt_argtypes_set set_cons_set usb_host_id set_vt_drv_set
set_kdb_dbbe_set set_ratectl_set set_crypto_set
set_ieee80211_ioctl_getset set_ieee80211_ioctl_setset set_scanner_set
set_videodriver_set set_scterm_set set_scrndr_set set_vga_set kern_conf
.eh_frame
03 .dynamic .got.plt .data set_pcpu .bss
04 .dynamic
05
Finally, here is the output of xenstore-ls -fp when the FreeBSD HVM
guest is hanging at mount root:
/tool = "" (n0)
/tool/xenstored = "" (n0)
/local = "" (n0)
/local/domain = "" (n0)
/local/domain/0 = "" (n0)
/local/domain/0/name = "Domain-0" (n0)
/local/domain/0/domid = "0" (n0)
/local/domain/0/memory = "" (n0)
/local/domain/0/memory/target = "10730008" (n0)
/local/domain/0/memory/static-max = "4294967292" (n0)
/local/domain/0/memory/freemem-slack = "410368" (n0)
/local/domain/0/libxl = "" (n0)
/local/domain/0/libxl/disable_udev = "1" (n0)
/local/domain/0/backend = "" (n0)
/local/domain/0/backend/vbd = "" (n0)
/local/domain/0/backend/vbd/1 = "" (n0)
