Hi Bertrand,
Thanks for your support!
As suggested, I enabled BLK_DEV_LOOP in the kernel & now I can see loop
devices in the "/dev" directory.
I could see xen-guest-image-minimal loading but at the end it's throwing
error *INIT: cannot execute "/sbin/getty-wrapper"*
I found start_getty in the image but getty-wrapper is not there. while
exploring source could see getty-wrapper in meta-virtualization sysvinit,
but start_getty is in poky sysvinit.
Any suggestions on this issue? Adding full logs below for reference:
*root@raspberrypi4-64:~# xl create -c guest1.cfg Parsing config from
guest1.cfgBooting Linux on physical CPU 0x0000000000 [0x410fd083]Linux
version 5.15.54-yocto-standard (oe-user@oe-host) (aarch64-poky-linux-gcc
(GCC) 11.3.0, GNU ld (GNU Binutils) 2.38.20220623) #1 SMP PREEMPT Thu Jul
14 18:52:13 UTC 2022random: crng init doneMachine model: XENVM-4.16Xen 4.16
support foundefi: UEFI not found.Zone ranges: DMA [mem
0x0000000040000000-0x000000007fffffff] DMA32 empty Normal
emptyMovable zone start for each nodeEarly memory node ranges node 0:
[mem 0x0000000040000000-0x000000007fffffff]Initmem setup node 0 [mem
0x0000000040000000-0x000000007fffffff]cma: Reserved 16 MiB at
0x000000007dc00000psci: probing for conduit method from DT.psci: PSCIv1.1
detected in firmware.psci: Using standard PSCI v0.2 function IDspsci:
Trusted OS migration not requiredpsci: SMC Calling Convention v1.1percpu:
Embedded 27 pages/cpu s70360 r8192 d32040 u110592Detected PIPT I-cache on
CPU0CPU features: detected: Spectre-v2CPU features: detected: Spectre-v4CPU
features: detected: Spectre-BHBCPU features: detected: ARM errata 1165522,
1319367, or 1530923Built 1 zonelists, mobility grouping on. Total pages:
258048Kernel command line: console=hvc0 earlyprintk=xen sync_console
root=/dev/xvdaUnknown kernel command line parameters "sync_console
earlyprintk=xen", will be passed to user space.Dentry cache hash table
entries: 131072 (order: 8, 1048576 bytes, linear)Inode-cache hash table
entries: 65536 (order: 7, 524288 bytes, linear)mem auto-init: stack:off,
heap alloc:off, heap free:offMemory: 985220K/1048576K available (13376K
kernel code, 2060K rwdata, 4372K rodata, 4160K init, 714K bss, 46972K
reserved, 16384K cma-reserved)SLUB: HWalign=64, Order=0-3, MinObjects=0,
CPUs=1, Nodes=1ftrace: allocating 41866 entries in 164 pagesftrace:
allocated 164 pages with 3 groupstrace event string verifier disabledrcu:
Preemptible hierarchical RCU implementation.rcu: RCU event tracing is
enabled.rcu: RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=1.
Trampoline variant of Tasks RCU enabled. Rude variant of Tasks RCU enabled.
Tracing variant of Tasks RCU enabled.rcu: RCU calculated value of
scheduler-enlistment delay is 25 jiffies.rcu: Adjusting geometry for
rcu_fanout_leaf=16, nr_cpu_ids=1NR_IRQS: 64, nr_irqs: 64, preallocated
irqs: 0Root IRQ handler: gic_handle_irqkfence: initialized - using 2097152
bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)arch_timer:
cp15 timer(s) running at 54.00MHz (virt).clocksource: arch_sys_counter:
mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123
nssched_clock: 56 bits at 54MHz, resolution 18ns, wraps every
4398046511102nsConsole: colour dummy device 80x25printk: console [hvc0]
enabledCalibrating delay loop (skipped), value calculated using timer
frequency.. 108.00 BogoMIPS (lpj=216000)pid_max: default: 32768 minimum:
301LSM: Security Framework initializinglandlock: Up and running.Mount-cache
hash table entries: 2048 (order: 2, 16384 bytes, linear)Mountpoint-cache
hash table entries: 2048 (order: 2, 16384 bytes, linear)xen:grant_table:
Grant tables using version 1 layoutGrant table initializedxen:events: Using
FIFO-based ABIXen: initializing cpu0rcu: Hierarchical SRCU
implementation.EFI services will not be available.smp: Bringing up
secondary CPUs ...smp: Brought up 1 node, 1 CPUSMP: Total of 1 processors
activated.CPU features: detected: 32-bit EL0 SupportCPU features: detected:
CRC32 instructionsCPU: All CPU(s) started at EL1alternatives: patching
kernel codedevtmpfs: initializedclocksource: jiffies: mask: 0xffffffff
max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 nsfutex hash table
entries: 256 (order: 2, 16384 bytes, linear)pinctrl core: initialized
pinctrl subsystemDMI not present or invalid.NET: Registered
PF_NETLINK/PF_ROUTE protocol familyDMA: preallocated 128 KiB GFP_KERNEL
pool for atomic allocationsDMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA
pool for atomic allocationsDMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32
pool for atomic allocationsthermal_sys: Registered thermal governor
'step_wise'hw-breakpoint: found 6 breakpoint and 4 watchpoint
registers.ASID allocator initialised with 65536 entriesSerial: AMBA PL011
UART driverraid6: neonx8 gen() 1585 MB/sraid6: neonx8 xor() 1177
MB/sraid6: neonx4 gen() 1701 MB/sraid6: neonx4 xor() 1279 MB/sraid6:
neonx2 gen() 1473 MB/sraid6: neonx2 xor() 1161 MB/sraid6: neonx1
gen() 1131 MB/sraid6: neonx1 xor() 929 MB/sraid6: int64x8 gen() 946
MB/sraid6: int64x8 xor() 555 MB/sraid6: int64x4 gen() 985 MB/sraid6:
int64x4 xor() 570 MB/sraid6: int64x2 gen() 914 MB/sraid6: int64x2
xor() 506 MB/sraid6: int64x1 gen() 714 MB/sraid6: int64x1 xor()
377 MB/sraid6: using algorithm neonx4 gen() 1701 MB/sraid6: .... xor() 1279
MB/s, rmw enabledraid6: using neon recovery algorithmxen:balloon:
Initialising balloon driveriommu: Default domain type: Translated iommu:
DMA domain TLB invalidation policy: strict mode vgaarb: loadedSCSI
subsystem initializedusbcore: registered new interface driver usbfsusbcore:
registered new interface driver hubusbcore: registered new device driver
usbpps_core: LinuxPPS API ver. 1 registeredpps_core: Software ver. 5.3.6 -
Copyright 2005-2007 Rodolfo Giometti <giome...@linux.it
<giome...@linux.it>>PTP clock support registeredBluetooth: Core ver
2.22NET: Registered PF_BLUETOOTH protocol familyBluetooth: HCI device and
connection manager initializedBluetooth: HCI socket layer
initializedBluetooth: L2CAP socket layer initializedBluetooth: SCO socket
layer initializedclocksource: Switched to clocksource arch_sys_counterNET:
Registered PF_INET protocol familyIP idents hash table entries: 16384
(order: 5, 131072 bytes, linear)tcp_listen_portaddr_hash hash table
entries: 512 (order: 1, 8192 bytes, linear)Table-perturb hash table
entries: 65536 (order: 6, 262144 bytes, linear)TCP established hash table
entries: 8192 (order: 4, 65536 bytes, linear)TCP bind hash table entries:
8192 (order: 5, 131072 bytes, linear)TCP: Hash tables configured
(established 8192 bind 8192)UDP hash table entries: 512 (order: 2, 16384
bytes, linear)UDP-Lite hash table entries: 512 (order: 2, 16384 bytes,
linear)NET: Registered PF_UNIX/PF_LOCAL protocol familyRPC: Registered
named UNIX socket transport module.RPC: Registered udp transport
module.RPC: Registered tcp transport module.RPC: Registered tcp NFSv4.1
backchannel transport module.PCI: CLS 0 bytes, default 64Initialise system
trusted keyringsworkingset: timestamp_bits=46 max_order=18
bucket_order=0NFS: Registering the id_resolver key typeKey type id_resolver
registeredKey type id_legacy registeredKey type cifs.idmap registeredxor:
measuring software checksum speed 8regs : 2522 MB/sec 32regs
: 2865 MB/sec arm64_neon : 2319 MB/secxor: using
function: 32regs (2865 MB/sec)async_tx: api initialized (async)Key type
asymmetric registeredAsymmetric key parser 'x509' registeredBlock layer
SCSI generic (bsg) driver version 0.4 loaded (major 249)io scheduler
mq-deadline registeredio scheduler kyber registeredxen:xen_evtchn:
Event-channel device installedSerial: 8250/16550 driver, 1 ports, IRQ
sharing enabledcacheinfo: Unable to detect cache hierarchy for CPU 0brd:
module loadedloop: module loadedInvalid max_queues (4), will use default
max: 1.xen_netfront: Initialising Xen virtual ethernet driverusbcore:
registered new interface driver lan78xxusbcore: registered new interface
driver asixusbcore: registered new interface driver ax88179_178ausbcore:
registered new interface driver cdc_etherusbcore: registered new interface
driver smsc95xxusbcore: registered new interface driver net1080usbcore:
registered new interface driver cdc_subsetusbcore: registered new interface
driver zaurususbcore: registered new interface driver cdc_ncmusbcore:
registered new interface driver aqc111usbcore: registered new interface
driver usb-storagedevice-mapper: ioctl: 4.45.0-ioctl (2021-03-22)
initialised: dm-de...@redhat.com <dm-de...@redhat.com>sdhci: Secure Digital
Host Controller Interface driversdhci: Copyright(c) Pierre
Ossmansdhci-pltfm: SDHCI platform and OF driver helperusbcore: registered
new interface driver usbhidusbhid: USB HID core driveru32 classifier
input device check on Actions configuredNET: Registered PF_INET6
protocol familySegment Routing with IPv6In-situ OAM (IOAM) with IPv6sit:
IPv6, IPv4 and MPLS over IPv4 tunneling driverNET: Registered PF_PACKET
protocol familyBridge firewalling registeredlib80211: common routines for
IEEE802.11 driversKey type dns_resolver registeredLoading compiled-in X.509
certificatesKey type ._fscrypt registeredKey type .fscrypt registeredKey
type fscrypt-provisioning registeredBtrfs loaded, crc32c=crc32c-generic,
zoned=no, fsverity=noKey type encrypted registeredxen_netfront: backend
supports XDP headroomblkfront: xvda: flush diskcache: enabled; persistent
grants: enabled; indirect descriptors: enabled; bounce buffer:
enabledprintk: console [netcon0] enablednetconsole: network logging
startedmd: Waiting for all devices to be available before autodetectmd: If
you don't use raid, use raid=noautodetectmd: Autodetecting RAID arrays.md:
autorun ...md: ... autorun DONE.EXT4-fs (xvda): mounting ext3 file system
using the ext4 subsystemEXT4-fs (xvda): INFO: recovery required on readonly
filesystemEXT4-fs (xvda): write access will be enabled during
recoveryEXT4-fs (xvda): recovery completeEXT4-fs (xvda): mounted filesystem
with ordered data mode. Opts: (null). Quota mode: disabled.VFS: Mounted
root (ext3 filesystem) readonly on device 202:0.devtmpfs: mountedFreeing
unused kernel memory: 4160KRun /sbin/init as init processINIT: version 3.01
bootingStarting udevudevd[128]: starting version 3.2.10udevd[129]: starting
eudev-3.2.10EXT4-fs (xvda): re-mounted. Opts: (null). Quota mode:
disabled.INIT: Entering runlevel: 5Configuring network interfaces...
udhcpc: started, v1.35.0udhcpc: broadcasting discoverIPv6: eth0: IPv6
duplicate address fe80::e65f:1ff:fecd:7bdb used by e4:5f:01:cd:7b:db
detected!udhcpc: broadcasting discoverudhcpc: broadcasting discoverudhcpc:
no lease, failingifup: failed to bring up eth0Starting syslogd/klogd:
doneStarting domain watchdog daemon: [ OK ]INIT: cannot execute
"/sbin/getty-wrapper"INIT: cannot execute "/sbin/getty-wrapper"INIT: cannot
execute "/sbin/getty-wrapper"INIT: cannot execute
"/sbin/getty-wrapper"INIT: cannot execute "/sbin/getty-wrapper"INIT: cannot
execute "/sbin/getty-wrapper"INIT: cannot execute
"/sbin/getty-wrapper"INIT: cannot execute "/sbin/getty-wrapper"INIT: cannot
execute "/sbin/getty-wrapper"INIT: cannot execute
"/sbin/getty-wrapper"INIT: Id "X0" respawning too fast: disabled for 5
minutes*
Regards,
Vipul Kumar
On Thu, Aug 25, 2022 at 2:58 PM Bertrand Marquis <bertrand.marq...@arm.com>
wrote:
> Hi Vipul,
>
> > On 25 Aug 2022, at 09:56, Vipul Suneja <vsunej...@gmail.com> wrote:
> >
> > Hi Bertrand,
> >
> > Thanks!
> >
> > No, I couldn't see /dev/loop0. Can you please guide me to create it?
>
> First thing to try is “modprobe loop"
>
> It that does not work (ie module not found) you should check in your linux
> config if BLK_DEV_LOOP is enabled.
>
> >
> > I didn't change dom0 configurations, it's default generated by yocto.
> >
> > I will append this "IMAGE_FSTYPES:append = " wic.gz”" in local.conf &
> will update you.
> >
>
> Cheers
> Bertrand
>
> > Regards,
> > Vipul Kumar
> >
> > On Thu, Aug 25, 2022 at 1:25 PM Bertrand Marquis <
> bertrand.marq...@arm.com> wrote:
> > Hi Vipul,
> >
> > > On 25 Aug 2022, at 08:31, Vipul Suneja <vsunej...@gmail.com> wrote:
> > >
> > > Hi Stefano,
> > >
> > > Thanks!
> > >
> > > As suggested, I changed the guest1.cfg file. Below are the contents of
> config file
> > >
> > > kernel = "/home/root/Image"
> > > cmdline = "console=hvc0 earlyprintk=xen sync_console root=/dev/xvda"
> > > memory = "1024"
> > > name = "guest1"
> > > vcpus = 1
> > > serial="pty"
> > > disk = [
> 'file:/home/root/xen-guest-image-minimal-raspberrypi4-64.ext3,xvda,w' ]
> > > vif=[ 'mac=00:11:22:66:88:22,bridge=xenbr0,type=netfront', ]
> > >
> > > Its failing with below logs:
> > >
> > > root@raspberrypi4-64:~# xl create -c guest1.cfg
> > > Parsing config from guest1.cfg
> > > Invalid parameter `type'.
> > > libxl: error: libxl_exec.c:117:libxl_report_child_exitstatus:
> /etc/xen/scripts/block add [742] exited with error status 1
> > > libxl: error: libxl_device.c:1265:device_hotplug_child_death_cb:
> script: losetup /dev/loop0
> /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3 failed
> > > libxl: error: libxl_create.c:1643:domcreate_launch_dm: Domain 1:unable
> to add disk devices
> > > libxl: error: libxl_exec.c:117:libxl_report_child_exitstatus:
> /etc/xen/scripts/block remove [793] exited with error status 1
> > > libxl: error: libxl_device.c:1265:device_hotplug_child_death_cb:
> script: /etc/xen/scripts/block failed; error detected.
> > > libxl: error: libxl_domain.c:1183:libxl__destroy_domid: Domain
> 1:Non-existant domain
> > > libxl: error: libxl_domain.c:1137:domain_destroy_callback: Domain
> 1:Unable to destroy guest
> > > libxl: error: libxl_domain.c:1064:domain_destroy_cb: Domain
> 1:Destruction of domain failed
> >
> > I think you have a loop issue.
> >
> > Could you check if /dev/loop0 exists ?
> >
> > Did you change something on the dom0 linux configuration generated by
> Yocto ?
> >
> > We are using Yocto on RPI4 here without any issue like that, only
> difference with
> > your setup is that we generate a wic image to have a real disk image
> instead of
> > using the ext3/ext4 one.
> >
> > Should be possible to do the same on your side by adding the following
> in local.conf:
> > IMAGE_FSTYPES:append = " wic.gz”
> >
> > >
> > > Even after removing 'type=netfront' from vif it's failing.
> >
> > This option is only for hvm on x86, so you can remove it from your
> configuration.
> >
> > > One more doubt here, could this mac address be a dummy or actual here?
> >
> > This is a dummy one you set for the guest network interface and this is
> the Mac
> > address other devices on your network will see so it must be fully
> valid (and
> > not conflicting with other devices on your network).
> >
> > Cheers
> > Bertrand
> >
> > >
> > > Regards,
> > > Vipul Kumar
> > >
> > > On Thu, Aug 25, 2022 at 2:36 AM Stefano Stabellini <
> sstabell...@kernel.org> wrote:
> > > On Wed, 24 Aug 2022, Vipul Suneja wrote:
> > > > Hi Bertrand,
> > > > Thanks for your response!
> > > >
> > > > I builded the guest image on yocto kirkstone source which has FSTYPE
> ext3. Guest image generated is
> > > > xen-guest-image-minimal-raspberrypi4-64.ext3.
> > > > Below is the content of guest.cfg file
> > > >
> > > > kernel = "/home/root/Image"
> > > > cmdline = "console=hvc0 earlyprintk=xen sync_console
> root=/dev/xvda"
> > > > memory = "256"
> > > > name = "guest1"
> > > > vcpus = 1
> > > > serial="pty"
> > > > disk = [ 'phy:/dev/loop0,xvda,w' ]
> > > > vif=[ 'mac=00:11:22:66:88:22,bridge=xenbr0,type=netfront', ]
> > > >
> > > > I am trying to mount xen-guest-image-minimal-raspberrypi4-64.ext3 to
> a virtual device & then will run the guest VM by command "xl create -c
> > > > guest.cfg". But facing issue while trying to mount.
> > >
> > > You don't actually need to mount
> > > xen-guest-image-minimal-raspberrypi4-64.ext3 anywhere to use it to run
> > > your guest VM with "xl create".
> > >
> > > It is enough to do this instead, as Bertrand suggested:
> > >
> > >
> disk=["file:/path/to/file/xen-guest-image-minimal-raspberrypi4-64.ext3,xvda,w"]
> > >
> > > No need to call losetup or mount. Just xl create -c.
> > >
> > > More answers below.
> > >
> > >
> > > > Regards,
> > > > Vipul Kumar
> > > >
> > > > On Wed, Aug 24, 2022 at 8:06 PM Bertrand Marquis <
> bertrand.marq...@arm.com> wrote:
> > > > Hi Vipul,
> > > >
> > > > > On 24 Aug 2022, at 15:16, Vipul Suneja <vsunej...@gmail.com>
> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I am porting xen hypervisor on rpi4 with yocto kirkstone
> sources. Followed the basic steps to build xen-image-minimal &
> > > > xen-guest-image-minimal. I could flash sd card with xen
> minimal image & could see dom0 up. I copied "Image",
> > > > "xen-guest-image-minimal" .ext3 file & guest.cfg to
> "/home/root". After that created a bridge with below step:
> > > > >
> > > > > killall -SIGUSR2 udhcpc
> > > > > brctl addbr xenbr0
> > > > > brctl addif xenbr0 eth0
> > > > > killall udhcpc
> > > > > udhcpc -R -b -p /var/run/udhcpc.xenbr0.pid -i xenbr0
> > > > >
> > > > > Could see the xenbr0 interface up.
> > > > > After that while mounting the guest file system it shows no
> such file or directory but the file is already there.
> > > > >
> > > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# ls -l
> > > > > [23:40:15] <Guest9046> -rw-r--r-- 1 root root
> 24652288 Mar 9 12:36 Image
> > > > > [23:40:15] <Guest9046> -rw-r--r-- 1 root root
> 247 Mar 9 12:37 guest1.cfg
> > > > > [23:40:15] <Guest9046> -rw-r--r-- 1 root root
> 868220928 Mar 9 12:39 xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# chmod 0777
> xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# ls -l
> > > > > [23:40:15] <Guest9046> -rw-r--r-- 1 root root
> 24652288 Mar 9 12:36 Image
> > > > > [23:40:15] <Guest9046> -rw-r--r-- 1 root root
> 247 Mar 9 12:37 guest1.cfg
> > > > > [23:40:15] <Guest9046> -rwxrwxrwx 1 root root
> 868220928 Mar 9 12:39 xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# losetup
> /dev/loop0 xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > > [23:40:15] <Guest9046> losetup:
> xen-guest-image-minimal-raspberrypi4-64.ext3: No such file or directory
> > > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# losetup
> /dev/loop0 /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > > [23:40:15] <Guest9046> losetup:
> /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3: No such file or
> directory
> > > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~#
> > > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~#
> > > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~#
> > > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# losetup
> /dev/loop0 /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > > [23:40:15] <Guest9046> losetup:
> /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3: No such file or
> directory
> > >
> > > It looks like either
> > > /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3 doesn't exist
> or
> > > /dev/loop0 doesn't exist
> > >
> > >
> > > > Why do you want to mount the file system ?
> > > >
> > > > Anyway this is not related to Xen, I guess you could start
> without xen and still not manage to mount the file like that (linux
> > > > configuration issue ?)
> > > >
> > > > What is the content of you guest.cfg
> > > > How do you want to pass the guest root file system ?
> > > >
> > > > Yocto should actually generate an img file and you could use
> it by having something like this in your guest.cfg:
> > > > disk=["file:/home/root/guest1.img,xvda,w”]
> > > >
> > > > Cheers
> > > > Bertrand
> > > >
> > > > >
> > > > > Any input on this issue will be really helpful, expecting
> your response.
> > > > >
> > > > > Thanks & Regards,
> > > > > Vipul Kumar
> > > >
> > > >
> > > >
> >
>
>
