Updated the wiki as well. Feel free to look at it to verify if anything is off.
I noticed that using '--qemu-path' to specify the location of qemu executable is not enough for virtiofsd as it is often located in another directory. So one may be better of explicitly adding both to the PATH or maybe we can do something in run.py to automate it. On Thursday, April 1, 2021 at 10:49:14 AM UTC-4 Waldek Kozaczuk wrote: > The intel version of DAX works fine. The ARM not, I get this using this > qemu (which seems like a year old) - > https://gitlab.com/virtio-fs/qemu/-/tree/virtio-fs-dev: > > sudo > PATH=/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build/tools/virtiofsd:/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build:$PATH > > ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export > --virtio-fs-dax=1024M --api -V --arch=aarch64 -c 1 > OSv v0.55.0-239-g47601e6e > getauxval() stubbed > PSCI: version 0.2 detected. > 1 CPUs detected > getauxval() stubbed > Firmware vendor: Unknown > bsd: initializing - done > VFS: mounting ramfs at / > VFS: mounting devfs at /dev > net: initializing - done > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > eth0: ethernet address: 52:54:00:12:34:56 > virtio-blk: Add blk device instances 0 as vblk0, devsize=18869248 > random: virtio-rng registered as a source. > virtio-fs: Detected device with tag: [] and num_queues: 0 > virtio-fs: Detected DAX window with length 1073741824 > random: <Software, Yarrow> initialized > VFS: unmounting /dev > VFS: mounting virtiofs at /virtiofs > [virtiofs] Error opening device! > failed to mount /virtiofs, error = No such device or address > Could not mount virtiofs root filesystem. > > Even the regular - non-DAX - version does not work: > > sudo > PATH=/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build/tools/virtiofsd:/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build:$PATH > > ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export > --api -V --arch=aarch64 -c 1 > OSv v0.55.0-239-g47601e6e > getauxval() stubbed > PSCI: version 0.2 detected. > 1 CPUs detected > getauxval() stubbed > Firmware vendor: Unknown > bsd: initializing - done > VFS: mounting ramfs at / > VFS: mounting devfs at /dev > net: initializing - done > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > eth0: ethernet address: 52:54:00:12:34:56 > virtio-blk: Add blk device instances 0 as vblk0, devsize=18869248 > random: virtio-rng registered as a source. > qemu-system-aarch64: wrong value for queue_enable 0 > qemu-system-aarch64: wrong value for queue_enable 0 > virtio-fs: Detected device with tag: [] and num_queues: -1 > Assertion failed: sched::preemptable() (arch/aarch64/mmu.cc: page_fault: > 46) > > [backtrace] > 0x00000000400e8c10 <__assert_fail+32> > 0x000000004020bb10 <page_fault+220> > 0x000000004020b828 <???+1075886120> > 0x0000000040208c64 <virtio::fs::req_done()+100> > 0x0000000040208e5c <???+1075875420> > 0x00000000402e72b0 <thread_main_c+32> > 0x000000004020b784 <thread_main+0> > > So it looks the version of QEMU from > https://gitlab.com/virtio-fs/qemu/-/tree/virtio-fs-dev is somewhat > outdated as far as ARM support goes. The non-DAX virtiofs work on ARM using > the regular released version of qemu - 5.1.0, as I stated in other emails. > > BTW you can build qemu from sources using a script we have in place: > > ./scripts/download_and_build_qemu.sh virtio-fs-dev > https://gitlab.com/virtio-fs/qemu.git > > On Wednesday, March 31, 2021 at 11:14:32 AM UTC-4 Waldek Kozaczuk wrote: > >> Sorry. I missed it. Let me build that specific version of QEMU with DAX >> support and test. I will also try it on ARM to see if it works. >> >> On Wednesday, March 31, 2021 at 10:56:27 AM UTC-4 Fotis Xenakis wrote: >> >>> On Saturday, March 13, 2021 at 8:58:57 PM UTC+2 Fotis Xenakis wrote: >>> >>>> On Saturday, March 13, 2021 at 6:08:29 PM UTC+2 [email protected] >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> The other 2 patches work fine but this one fails when I test it. What >>>>> am I doing wrong? >>>>> >>>>> >>>>> sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH >>>>> ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export >>>>> --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64 >>>>> --virtio-fs-dax=1024M >>>>> qemu-system-x86_64: -device >>>>> vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M: >>>>> >>>>> Property 'vhost-user-fs-pci.cache-size' not found >>>>> qemu failed. >>>>> >>>>> sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH >>>>> ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export >>>>> --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64 >>>>> --virtio-fs-dax=1024M --dry-run >>>>> /home/wkozaczuk/projects/osv-master/scripts/../scripts/imgedit.py >>>>> setargs /home/wkozaczuk/projects/osv-master/build/last/usr.img >>>>> "--rootfs=virtiofs /cli" >>>>> virtiofsd \ >>>>> --socket-path=/tmp/vhostqemu \ >>>>> -o source=/home/wkozaczuk/projects/osv-master/build/export >>>>> build/downloaded_packages/qemu/build/qemu-system-x86_64 \ >>>>> -m 2G \ >>>>> -smp 4 \ >>>>> -vnc :1 \ >>>>> -gdb tcp::1234,server,nowait \ >>>>> -device virtio-blk-pci,id=blk0,drive=hd0,scsi=off,bootindex=0 \ >>>>> -drive >>>>> file=/home/wkozaczuk/projects/osv-master/build/last/usr.img,if=none,id=hd0,cache=none,aio=native >>>>> >>>>> \ >>>>> -chardev socket,id=char0,path=/tmp/vhostqemu \ >>>>> -device >>>>> vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M >>>>> \ >>>>> -object memory-backend-file,id=mem,size=2G,mem-path=/dev/shm,share=on \ >>>>> -numa node,memdev=mem \ >>>>> -netdev user,id=un0,net=192.168.122.0/24,host=192.168.122.1 \ >>>>> -device virtio-net-pci,netdev=un0 \ >>>>> -device virtio-rng-pci \ >>>>> -enable-kvm \ >>>>> -cpu host,+x2apic \ >>>>> -chardev stdio,mux=on,id=stdio,signal=off \ >>>>> -mon chardev=stdio,mode=readline \ >>>>> -device isa-serial,chardev=stdio >>>>> >>>>> I am using QEMU 5.2 built from sources. >>>>> >>>> Unfortunately DAX support has yet to be merged to upstream QEMU (I was >>>> under the impression that it had been staged for QEMU 5.2.0, but alas it >>>> is >>>> not part of it). >>>> So, to use it, one still needs to use the virtio-fs downstream version >>>> which includes the necessary patches [1]. >>>> The virtio-fs team are actively pursuing its upstreaming, but I 'm >>>> afraid I don't have a clear view as to how long that might still take. >>>> >>>> Regarding the patch, do you think it would be better to hold it back >>>> until there's proper support upstream? >>>> >>> Bumping on this: I guess we should wait for QEMU here? >>> >>>> >>>> [1] https://gitlab.com/virtio-fs/qemu/-/tree/virtio-fs-dev >>>> >>>> >>>>> >>>>> Waldek >>>>> On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote: >>>>> >>>>>> Signed-off-by: Fotis Xenakis <[email protected]> >>>>>> --- >>>>>> scripts/run.py | 5 ++++- >>>>>> 1 file changed, 4 insertions(+), 1 deletion(-) >>>>>> >>>>>> diff --git a/scripts/run.py b/scripts/run.py >>>>>> index 71c6cdd7..6a81e98b 100755 >>>>>> --- a/scripts/run.py >>>>>> +++ b/scripts/run.py >>>>>> @@ -178,9 +178,10 @@ def start_osv_qemu(options): >>>>>> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)] >>>>>> >>>>>> if options.virtio_fs_tag: >>>>>> + dax = (",cache-size=%s" % options.virtio_fs_dax) if >>>>>> options.virtio_fs_dax else "" >>>>>> args += [ >>>>>> "-chardev", "socket,id=char0,path=/tmp/vhostqemu", >>>>>> - "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" >>>>>> % options.virtio_fs_tag, >>>>>> + "-device", >>>>>> "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s" % >>>>>> (options.virtio_fs_tag, dax), >>>>>> "-object", >>>>>> "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" % >>>>>> options.memsize, >>>>>> "-numa", "node,memdev=mem"] >>>>>> >>>>>> @@ -602,6 +603,8 @@ if __name__ == "__main__": >>>>>> help="virtio-fs device tag") >>>>>> parser.add_argument("--virtio-fs-dir", action="store", >>>>>> help="path to the directory exposed via virtio-fs mount") >>>>>> + parser.add_argument("--virtio-fs-dax", action="store", >>>>>> + help="DAX window size for virtio-fs device (disabled if not >>>>>> specified)") >>>>>> parser.add_argument("--mount-fs", default=[], action="append", >>>>>> help="extra mounts (forwarded to respective kernel command line >>>>>> option)") >>>>>> parser.add_argument("--ip", default=[], action="append", >>>>>> -- >>>>>> 2.30.1 >>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/d1c7caa0-0f04-4a2d-80ff-aa40fc1ceaf9n%40googlegroups.com.
