Re: [osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
Hello Matias,
Thank you for chiming in,
libkrun indeed seems very promising and, very importantly, pretty valuable
from an end-user perspective if it becomes practical!
To my eyes, virtio-fs makes most sense in such setups: it's kind of the
equivalent of bind mounts in Docker terminology.
On a side note, it's started by some very prominent people in the
lightweight virtualization community.
P.S. It's great to have such, more general, discussions or even sharing
resources and perspectives. I wouldn't want to hijack this thread,
but feel free to start a fresh one if you 'd like to continue this
discussion.
On Wednesday, March 31, 2021 at 9:39:29 PM UTC+3 matia...@gmail.com wrote:
> Hello,
>
> I think it may be interesting to have a look at libkrun (
> https://github.com/containers/libkrun). As far I know, it supports
> virtio-fs.
>
> Matias
>
> On Wed, 31 Mar 2021 at 16:55, Fotis Xenakis wrote:
>
>> On Tuesday, March 30, 2021 at 9:23:24 PM UTC+3 jwkoz...@gmail.com wrote:
>>
>>> Fotis,
>>>
>>> Thanks a lot!
>>>
>>> I have updated this page to include the information about ARM -
>>> virtio-fs works on this arch as well which is great!
>>>
>> It is awesome to hear that! What would be mildly surprising is if DAX
>> also worked on ARM (on the host/QEMU side as well, given it's not yet fully
>> stable).
>>
>>>
>>> The virtio-fs support is probably the single most exciting new feature
>>> added to OSv in the last couple of years! I am not aware of any other
>>> unikernel supporting it.
>>>
>> It sure has a great value to dev effort ratio! The only other unikernel
>> supporting it is toro AFAIK: https://github.com/torokernel/torokernel
>>
>>>
>>> Thanks a lot,
>>> Waldek
>>>
>>> PS. It would be nice if some other hypervisors (thinking of firecracker)
>>> supported virtio-fs as well.
>>>
>> Indeed, it seems a great fit for the lightweight VMMs. Firecracker was
>> close to adding it, but eventually did not:
>> https://github.com/firecracker-microvm/firecracker/pull/1351
>> Yet, cloud-hypervisor does support it, notably with the contributions of
>> upstream virtio-fs devs:
>> https://github.com/cloud-hypervisor/cloud-hypervisor/blob/v0.14.0/docs/fs.md
>>
>>>
>>> On Friday, March 26, 2021 at 1:44:30 PM UTC-4 Fotis Xenakis wrote:
>>>
Update: the wiki page is here:
https://github.com/cloudius-systems/osv/wiki/virtio-fs
All questions/feedback and of course contributions welcome!
On Saturday, March 13, 2021 at 9:04:03 PM UTC+2 Fotis Xenakis wrote:
> You are absolutely right, some proper guide and documentation is
> necessary, thank you for the suggestion!
> I shall take care of it upon first chance.
>
> On Saturday, March 13, 2021 at 6:29:28 PM UTC+2 jwkoz...@gmail.com
> wrote:
>
>> Hey,
>>
>> I have applied 3 of your patches.
>>
>> As I was doing and testing it I realized we do not have any Wiki page
>> that describes how to use virtio-fs with OSv. Would you have a bit of
>> time
>> to add a page that does so - a simple step-by-step cookbook - how to
>> build
>> with export and fs options, and then run with various new options (DAX,
>> no
>> dax, etc), need to use sudo, which version of QEMU, etc -
>> https://github.com/cloudius-systems/osv/wiki? Maybe also link to
>> your QEMU article?
>>
>> Right now we have some info scattered acros some commit messages but
>> nothing in one place.
>>
>> I would really appreciate it,
>> Waldek
>>
>> On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
>>
>>> Signed-off-by: Fotis Xenakis
>>> ---
>>> scripts/run.py | 5 -
>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/run.py b/scripts/run.py
>>> index 71c6cdd7..6a81e98b 100755
>>> --- a/scripts/run.py
>>> +++ b/scripts/run.py
>>> @@ -178,9 +178,10 @@ def start_osv_qemu(options):
>>> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
>>>
>>> if options.virtio_fs_tag:
>>> + dax = (",cache-size=%s" % options.virtio_fs_dax) if
>>> options.virtio_fs_dax else ""
>>> args += [
>>> "-chardev", "socket,id=char0,path=/tmp/vhostqemu",
>>> - "-device",
>>> "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" %
>>> options.virtio_fs_tag,
>>> + "-device",
>>> "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s" %
>>> (options.virtio_fs_tag, dax),
>>> "-object",
>>> "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" %
>>> options.memsize,
>>> "-numa", "node,memdev=mem"]
>>>
>>> @@ -602,6 +603,8 @@ if __name__ == "__main__":
>>> help="virtio-fs device tag")
>>> parser.add_argument("--virtio-fs-dir", action="store",
>>> help="path to the directory exposed via virtio-fs mount")
>>> + parser.add_argument("--virtio-fs-dax",
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
Thank you for taking the time to try it out Waldek! It would be somewhat surprising if DAX worked on ARM, at least given the devs are not focused on it. Not really sure what the current difference to stable QEMU is, apart from DAX, which would also justify the non-DAX version not working. BTW, this is not based on such an old QEMU version: the latest non-virtio-fs commits are from Jan 6, 2021, so after QEMU 5.2.0. Also, thanks for the heads up for the new script, I had totally missed that up to now. Finally, thank you for the prompt update to the wiki page, it seems spot on to me! There was I think a mention of having to add virtiofsd to the PATH, but it's good to stress it out as much as possible: it's probably the easiest point to miss. On Thursday, April 1, 2021 at 5:49:14 PM UTC+3 jwkoz...@gmail.com wrote: > The intel version of DAX works fine. The ARM not, I get this using this > qemu (which seems like a year old) - > https://gitlab.com/virtio-fs/qemu/-/tree/virtio-fs-dev: > > sudo > PATH=/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build/tools/virtiofsd:/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build:$PATH > > ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export > --virtio-fs-dax=1024M --api -V --arch=aarch64 -c 1 > OSv v0.55.0-239-g47601e6e > getauxval() stubbed > PSCI: version 0.2 detected. > 1 CPUs detected > getauxval() stubbed > Firmware vendor: Unknown > bsd: initializing - done > VFS: mounting ramfs at / > VFS: mounting devfs at /dev > net: initializing - done > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > eth0: ethernet address: 52:54:00:12:34:56 > virtio-blk: Add blk device instances 0 as vblk0, devsize=18869248 > random: virtio-rng registered as a source. > virtio-fs: Detected device with tag: [] and num_queues: 0 > virtio-fs: Detected DAX window with length 1073741824 > random: initialized > VFS: unmounting /dev > VFS: mounting virtiofs at /virtiofs > [virtiofs] Error opening device! > failed to mount /virtiofs, error = No such device or address > Could not mount virtiofs root filesystem. > > Even the regular - non-DAX - version does not work: > > sudo > PATH=/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build/tools/virtiofsd:/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build:$PATH > > ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export > --api -V --arch=aarch64 -c 1 > OSv v0.55.0-239-g47601e6e > getauxval() stubbed > PSCI: version 0.2 detected. > 1 CPUs detected > getauxval() stubbed > Firmware vendor: Unknown > bsd: initializing - done > VFS: mounting ramfs at / > VFS: mounting devfs at /dev > net: initializing - done > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > Version 1 not supported! > eth0: ethernet address: 52:54:00:12:34:56 > virtio-blk: Add blk device instances 0 as vblk0, devsize=18869248 > random: virtio-rng registered as a source. > qemu-system-aarch64: wrong value for queue_enable 0 > qemu-system-aarch64: wrong value for queue_enable 0 > virtio-fs: Detected device with tag: [] and num_queues: -1 > Assertion failed: sched::preemptable() (arch/aarch64/mmu.cc: page_fault: > 46) > > [backtrace] > 0x400e8c10 <__assert_fail+32> > 0x4020bb10 > 0x4020b828 > 0x40208c64 > 0x40208e5c > 0x402e72b0 > 0x4020b784 > > So it looks the version of QEMU from > https://gitlab.com/virtio-fs/qemu/-/tree/virtio-fs-dev is somewhat > outdated as far as ARM support goes. The non-DAX virtiofs work on ARM using > the regular released version of qemu - 5.1.0, as I stated in other emails. > > BTW you can build qemu from sources using a script we have in place: > > ./scripts/download_and_build_qemu.sh virtio-fs-dev > https://gitlab.com/virtio-fs/qemu.git > > On Wednesday, March 31, 2021 at 11:14:32 AM UTC-4 Waldek Kozaczuk wrote: > >> Sorry. I missed it. Let me build that specific version of QEMU with DAX >> support and test. I will also try it on ARM to see if it works. >> >> On Wednesday, March 31, 2021 at 10:56:27 AM UTC-4 Fotis Xenakis wrote: >> >>> On Saturday, March 13, 2021 at 8:58:57 PM UTC+2 Fotis Xenakis wrote: >>> On Saturday, March 13, 2021 at 6:08:29 PM UTC+2 jwkoz...@gmail.com wrote: > Hi, > > The other 2 patches work fine but this one fails when I test it. What > am I doing wrong? > > > sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH > ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export > --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64 > --virtio-fs-dax=1024M >
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
The intel version of DAX works fine. The ARM not, I get this using this qemu (which seems like a year old) - https://gitlab.com/virtio-fs/qemu/-/tree/virtio-fs-dev: sudo PATH=/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build/tools/virtiofsd:/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build:$PATH ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export --virtio-fs-dax=1024M --api -V --arch=aarch64 -c 1 OSv v0.55.0-239-g47601e6e getauxval() stubbed PSCI: version 0.2 detected. 1 CPUs detected getauxval() stubbed Firmware vendor: Unknown bsd: initializing - done VFS: mounting ramfs at / VFS: mounting devfs at /dev net: initializing - done Version 1 not supported! Version 1 not supported! Version 1 not supported! Version 1 not supported! Version 1 not supported! Version 1 not supported! Version 1 not supported! Version 1 not supported! eth0: ethernet address: 52:54:00:12:34:56 virtio-blk: Add blk device instances 0 as vblk0, devsize=18869248 random: virtio-rng registered as a source. virtio-fs: Detected device with tag: [] and num_queues: 0 virtio-fs: Detected DAX window with length 1073741824 random: initialized VFS: unmounting /dev VFS: mounting virtiofs at /virtiofs [virtiofs] Error opening device! failed to mount /virtiofs, error = No such device or address Could not mount virtiofs root filesystem. Even the regular - non-DAX - version does not work: sudo PATH=/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build/tools/virtiofsd:/home/wkozaczuk/projects/osv/build/downloaded_packages/qemu/build:$PATH ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export --api -V --arch=aarch64 -c 1 OSv v0.55.0-239-g47601e6e getauxval() stubbed PSCI: version 0.2 detected. 1 CPUs detected getauxval() stubbed Firmware vendor: Unknown bsd: initializing - done VFS: mounting ramfs at / VFS: mounting devfs at /dev net: initializing - done Version 1 not supported! Version 1 not supported! Version 1 not supported! Version 1 not supported! Version 1 not supported! Version 1 not supported! Version 1 not supported! Version 1 not supported! eth0: ethernet address: 52:54:00:12:34:56 virtio-blk: Add blk device instances 0 as vblk0, devsize=18869248 random: virtio-rng registered as a source. qemu-system-aarch64: wrong value for queue_enable 0 qemu-system-aarch64: wrong value for queue_enable 0 virtio-fs: Detected device with tag: [] and num_queues: -1 Assertion failed: sched::preemptable() (arch/aarch64/mmu.cc: page_fault: 46) [backtrace] 0x400e8c10 <__assert_fail+32> 0x4020bb10 0x4020b828 0x40208c64 0x40208e5c 0x402e72b0 0x4020b784 So it looks the version of QEMU from https://gitlab.com/virtio-fs/qemu/-/tree/virtio-fs-dev is somewhat outdated as far as ARM support goes. The non-DAX virtiofs work on ARM using the regular released version of qemu - 5.1.0, as I stated in other emails. BTW you can build qemu from sources using a script we have in place: ./scripts/download_and_build_qemu.sh virtio-fs-dev https://gitlab.com/virtio-fs/qemu.git On Wednesday, March 31, 2021 at 11:14:32 AM UTC-4 Waldek Kozaczuk wrote: > Sorry. I missed it. Let me build that specific version of QEMU with DAX > support and test. I will also try it on ARM to see if it works. > > On Wednesday, March 31, 2021 at 10:56:27 AM UTC-4 Fotis Xenakis wrote: > >> On Saturday, March 13, 2021 at 8:58:57 PM UTC+2 Fotis Xenakis wrote: >> >>> On Saturday, March 13, 2021 at 6:08:29 PM UTC+2 jwkoz...@gmail.com >>> wrote: >>> Hi, The other 2 patches work fine but this one fails when I test it. What am I doing wrong? sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64 --virtio-fs-dax=1024M qemu-system-x86_64: -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M: Property 'vhost-user-fs-pci.cache-size' not found qemu failed. sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64 --virtio-fs-dax=1024M --dry-run /home/wkozaczuk/projects/osv-master/scripts/../scripts/imgedit.py setargs /home/wkozaczuk/projects/osv-master/build/last/usr.img "--rootfs=virtiofs /cli" virtiofsd \ --socket-path=/tmp/vhostqemu \ -o source=/home/wkozaczuk/projects/osv-master/build/export build/downloaded_packages/qemu/build/qemu-system-x86_64 \ -m 2G \ -smp 4 \ -vnc :1 \ -gdb tcp::1234,server,nowait \ -device virtio-blk-pci,id=blk0,drive=hd0,scsi=off,bootindex=0 \ -drive
Re: [osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
Hello,
I think it may be interesting to have a look at libkrun (
https://github.com/containers/libkrun). As far I know, it supports
virtio-fs.
Matias
On Wed, 31 Mar 2021 at 16:55, Fotis Xenakis wrote:
> On Tuesday, March 30, 2021 at 9:23:24 PM UTC+3 jwkoz...@gmail.com wrote:
>
>> Fotis,
>>
>> Thanks a lot!
>>
>> I have updated this page to include the information about ARM - virtio-fs
>> works on this arch as well which is great!
>>
> It is awesome to hear that! What would be mildly surprising is if DAX also
> worked on ARM (on the host/QEMU side as well, given it's not yet fully
> stable).
>
>>
>> The virtio-fs support is probably the single most exciting new feature
>> added to OSv in the last couple of years! I am not aware of any other
>> unikernel supporting it.
>>
> It sure has a great value to dev effort ratio! The only other unikernel
> supporting it is toro AFAIK: https://github.com/torokernel/torokernel
>
>>
>> Thanks a lot,
>> Waldek
>>
>> PS. It would be nice if some other hypervisors (thinking of firecracker)
>> supported virtio-fs as well.
>>
> Indeed, it seems a great fit for the lightweight VMMs. Firecracker was
> close to adding it, but eventually did not:
> https://github.com/firecracker-microvm/firecracker/pull/1351
> Yet, cloud-hypervisor does support it, notably with the contributions of
> upstream virtio-fs devs:
> https://github.com/cloud-hypervisor/cloud-hypervisor/blob/v0.14.0/docs/fs.md
>
>>
>> On Friday, March 26, 2021 at 1:44:30 PM UTC-4 Fotis Xenakis wrote:
>>
>>> Update: the wiki page is here:
>>> https://github.com/cloudius-systems/osv/wiki/virtio-fs
>>> All questions/feedback and of course contributions welcome!
>>>
>>> On Saturday, March 13, 2021 at 9:04:03 PM UTC+2 Fotis Xenakis wrote:
>>>
You are absolutely right, some proper guide and documentation is
necessary, thank you for the suggestion!
I shall take care of it upon first chance.
On Saturday, March 13, 2021 at 6:29:28 PM UTC+2 jwkoz...@gmail.com
wrote:
> Hey,
>
> I have applied 3 of your patches.
>
> As I was doing and testing it I realized we do not have any Wiki page
> that describes how to use virtio-fs with OSv. Would you have a bit of time
> to add a page that does so - a simple step-by-step cookbook - how to build
> with export and fs options, and then run with various new options (DAX, no
> dax, etc), need to use sudo, which version of QEMU, etc -
> https://github.com/cloudius-systems/osv/wiki? Maybe also link to
> your QEMU article?
>
> Right now we have some info scattered acros some commit messages but
> nothing in one place.
>
> I would really appreciate it,
> Waldek
>
> On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
>
>> Signed-off-by: Fotis Xenakis
>> ---
>> scripts/run.py | 5 -
>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/scripts/run.py b/scripts/run.py
>> index 71c6cdd7..6a81e98b 100755
>> --- a/scripts/run.py
>> +++ b/scripts/run.py
>> @@ -178,9 +178,10 @@ def start_osv_qemu(options):
>> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
>>
>> if options.virtio_fs_tag:
>> + dax = (",cache-size=%s" % options.virtio_fs_dax) if
>> options.virtio_fs_dax else ""
>> args += [
>> "-chardev", "socket,id=char0,path=/tmp/vhostqemu",
>> - "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s"
>> % options.virtio_fs_tag,
>> + "-device",
>> "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s" %
>> (options.virtio_fs_tag, dax),
>> "-object",
>> "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" %
>> options.memsize,
>> "-numa", "node,memdev=mem"]
>>
>> @@ -602,6 +603,8 @@ if __name__ == "__main__":
>> help="virtio-fs device tag")
>> parser.add_argument("--virtio-fs-dir", action="store",
>> help="path to the directory exposed via virtio-fs mount")
>> + parser.add_argument("--virtio-fs-dax", action="store",
>> + help="DAX window size for virtio-fs device (disabled if not
>> specified)")
>> parser.add_argument("--mount-fs", default=[], action="append",
>> help="extra mounts (forwarded to respective kernel command line
>> option)")
>> parser.add_argument("--ip", default=[], action="append",
>> --
>> 2.30.1
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/538fe958-fb89-4286-8cb5-a0dc4664ae72n%40googlegroups.com
>
> .
>
--
You received this
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
Sorry. I missed it. Let me build that specific version of QEMU with DAX
support and test. I will also try it on ARM to see if it works.
On Wednesday, March 31, 2021 at 10:56:27 AM UTC-4 Fotis Xenakis wrote:
> On Saturday, March 13, 2021 at 8:58:57 PM UTC+2 Fotis Xenakis wrote:
>
>> On Saturday, March 13, 2021 at 6:08:29 PM UTC+2 jwkoz...@gmail.com wrote:
>>
>>> Hi,
>>>
>>> The other 2 patches work fine but this one fails when I test it. What am
>>> I doing wrong?
>>>
>>>
>>> sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH
>>> ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export
>>> --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64
>>> --virtio-fs-dax=1024M
>>> qemu-system-x86_64: -device
>>> vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M:
>>> Property 'vhost-user-fs-pci.cache-size' not found
>>> qemu failed.
>>>
>>> sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH
>>> ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export
>>> --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64
>>> --virtio-fs-dax=1024M --dry-run
>>> /home/wkozaczuk/projects/osv-master/scripts/../scripts/imgedit.py
>>> setargs /home/wkozaczuk/projects/osv-master/build/last/usr.img
>>> "--rootfs=virtiofs /cli"
>>> virtiofsd \
>>> --socket-path=/tmp/vhostqemu \
>>> -o source=/home/wkozaczuk/projects/osv-master/build/export
>>> build/downloaded_packages/qemu/build/qemu-system-x86_64 \
>>> -m 2G \
>>> -smp 4 \
>>> -vnc :1 \
>>> -gdb tcp::1234,server,nowait \
>>> -device virtio-blk-pci,id=blk0,drive=hd0,scsi=off,bootindex=0 \
>>> -drive
>>> file=/home/wkozaczuk/projects/osv-master/build/last/usr.img,if=none,id=hd0,cache=none,aio=native
>>>
>>> \
>>> -chardev socket,id=char0,path=/tmp/vhostqemu \
>>> -device
>>> vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M \
>>> -object memory-backend-file,id=mem,size=2G,mem-path=/dev/shm,share=on \
>>> -numa node,memdev=mem \
>>> -netdev user,id=un0,net=192.168.122.0/24,host=192.168.122.1 \
>>> -device virtio-net-pci,netdev=un0 \
>>> -device virtio-rng-pci \
>>> -enable-kvm \
>>> -cpu host,+x2apic \
>>> -chardev stdio,mux=on,id=stdio,signal=off \
>>> -mon chardev=stdio,mode=readline \
>>> -device isa-serial,chardev=stdio
>>>
>>> I am using QEMU 5.2 built from sources.
>>>
>> Unfortunately DAX support has yet to be merged to upstream QEMU (I was
>> under the impression that it had been staged for QEMU 5.2.0, but alas it is
>> not part of it).
>> So, to use it, one still needs to use the virtio-fs downstream version
>> which includes the necessary patches [1].
>> The virtio-fs team are actively pursuing its upstreaming, but I 'm afraid
>> I don't have a clear view as to how long that might still take.
>>
>> Regarding the patch, do you think it would be better to hold it back
>> until there's proper support upstream?
>>
> Bumping on this: I guess we should wait for QEMU here?
>
>>
>> [1] https://gitlab.com/virtio-fs/qemu/-/tree/virtio-fs-dev
>>
>>
>>>
>>> Waldek
>>> On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
>>>
Signed-off-by: Fotis Xenakis
---
scripts/run.py | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/scripts/run.py b/scripts/run.py
index 71c6cdd7..6a81e98b 100755
--- a/scripts/run.py
+++ b/scripts/run.py
@@ -178,9 +178,10 @@ def start_osv_qemu(options):
"-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
if options.virtio_fs_tag:
+ dax = (",cache-size=%s" % options.virtio_fs_dax) if
options.virtio_fs_dax else ""
args += [
"-chardev", "socket,id=char0,path=/tmp/vhostqemu",
- "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" %
options.virtio_fs_tag,
+ "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s"
% (options.virtio_fs_tag, dax),
"-object",
"memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" %
options.memsize,
"-numa", "node,memdev=mem"]
@@ -602,6 +603,8 @@ if __name__ == "__main__":
help="virtio-fs device tag")
parser.add_argument("--virtio-fs-dir", action="store",
help="path to the directory exposed via virtio-fs mount")
+ parser.add_argument("--virtio-fs-dax", action="store",
+ help="DAX window size for virtio-fs device (disabled if not
specified)")
parser.add_argument("--mount-fs", default=[], action="append",
help="extra mounts (forwarded to respective kernel command line
option)")
parser.add_argument("--ip", default=[], action="append",
--
2.30.1
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
On Saturday, March 13, 2021 at 8:58:57 PM UTC+2 Fotis Xenakis wrote:
> On Saturday, March 13, 2021 at 6:08:29 PM UTC+2 jwkoz...@gmail.com wrote:
>
>> Hi,
>>
>> The other 2 patches work fine but this one fails when I test it. What am
>> I doing wrong?
>>
>>
>> sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH
>> ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export
>> --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64
>> --virtio-fs-dax=1024M
>> qemu-system-x86_64: -device
>> vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M:
>> Property 'vhost-user-fs-pci.cache-size' not found
>> qemu failed.
>>
>> sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH
>> ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export
>> --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64
>> --virtio-fs-dax=1024M --dry-run
>> /home/wkozaczuk/projects/osv-master/scripts/../scripts/imgedit.py setargs
>> /home/wkozaczuk/projects/osv-master/build/last/usr.img "--rootfs=virtiofs
>> /cli"
>> virtiofsd \
>> --socket-path=/tmp/vhostqemu \
>> -o source=/home/wkozaczuk/projects/osv-master/build/export
>> build/downloaded_packages/qemu/build/qemu-system-x86_64 \
>> -m 2G \
>> -smp 4 \
>> -vnc :1 \
>> -gdb tcp::1234,server,nowait \
>> -device virtio-blk-pci,id=blk0,drive=hd0,scsi=off,bootindex=0 \
>> -drive
>> file=/home/wkozaczuk/projects/osv-master/build/last/usr.img,if=none,id=hd0,cache=none,aio=native
>>
>> \
>> -chardev socket,id=char0,path=/tmp/vhostqemu \
>> -device
>> vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M \
>> -object memory-backend-file,id=mem,size=2G,mem-path=/dev/shm,share=on \
>> -numa node,memdev=mem \
>> -netdev user,id=un0,net=192.168.122.0/24,host=192.168.122.1 \
>> -device virtio-net-pci,netdev=un0 \
>> -device virtio-rng-pci \
>> -enable-kvm \
>> -cpu host,+x2apic \
>> -chardev stdio,mux=on,id=stdio,signal=off \
>> -mon chardev=stdio,mode=readline \
>> -device isa-serial,chardev=stdio
>>
>> I am using QEMU 5.2 built from sources.
>>
> Unfortunately DAX support has yet to be merged to upstream QEMU (I was
> under the impression that it had been staged for QEMU 5.2.0, but alas it is
> not part of it).
> So, to use it, one still needs to use the virtio-fs downstream version
> which includes the necessary patches [1].
> The virtio-fs team are actively pursuing its upstreaming, but I 'm afraid
> I don't have a clear view as to how long that might still take.
>
> Regarding the patch, do you think it would be better to hold it back until
> there's proper support upstream?
>
Bumping on this: I guess we should wait for QEMU here?
>
> [1] https://gitlab.com/virtio-fs/qemu/-/tree/virtio-fs-dev
>
>
>>
>> Waldek
>> On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
>>
>>> Signed-off-by: Fotis Xenakis
>>> ---
>>> scripts/run.py | 5 -
>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/run.py b/scripts/run.py
>>> index 71c6cdd7..6a81e98b 100755
>>> --- a/scripts/run.py
>>> +++ b/scripts/run.py
>>> @@ -178,9 +178,10 @@ def start_osv_qemu(options):
>>> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
>>>
>>> if options.virtio_fs_tag:
>>> + dax = (",cache-size=%s" % options.virtio_fs_dax) if
>>> options.virtio_fs_dax else ""
>>> args += [
>>> "-chardev", "socket,id=char0,path=/tmp/vhostqemu",
>>> - "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" %
>>> options.virtio_fs_tag,
>>> + "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s"
>>> % (options.virtio_fs_tag, dax),
>>> "-object",
>>> "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" %
>>> options.memsize,
>>> "-numa", "node,memdev=mem"]
>>>
>>> @@ -602,6 +603,8 @@ if __name__ == "__main__":
>>> help="virtio-fs device tag")
>>> parser.add_argument("--virtio-fs-dir", action="store",
>>> help="path to the directory exposed via virtio-fs mount")
>>> + parser.add_argument("--virtio-fs-dax", action="store",
>>> + help="DAX window size for virtio-fs device (disabled if not
>>> specified)")
>>> parser.add_argument("--mount-fs", default=[], action="append",
>>> help="extra mounts (forwarded to respective kernel command line
>>> option)")
>>> parser.add_argument("--ip", default=[], action="append",
>>> --
>>> 2.30.1
>>>
>>>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/fd7dc8d8-30c1-481c-997c-e7bed43a27fcn%40googlegroups.com.
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
On Tuesday, March 30, 2021 at 9:23:24 PM UTC+3 jwkoz...@gmail.com wrote:
> Fotis,
>
> Thanks a lot!
>
> I have updated this page to include the information about ARM - virtio-fs
> works on this arch as well which is great!
>
It is awesome to hear that! What would be mildly surprising is if DAX also
worked on ARM (on the host/QEMU side as well, given it's not yet fully
stable).
>
> The virtio-fs support is probably the single most exciting new feature
> added to OSv in the last couple of years! I am not aware of any other
> unikernel supporting it.
>
It sure has a great value to dev effort ratio! The only other unikernel
supporting it is toro AFAIK: https://github.com/torokernel/torokernel
>
> Thanks a lot,
> Waldek
>
> PS. It would be nice if some other hypervisors (thinking of firecracker)
> supported virtio-fs as well.
>
Indeed, it seems a great fit for the lightweight VMMs. Firecracker was
close to adding it, but eventually did not:
https://github.com/firecracker-microvm/firecracker/pull/1351
Yet, cloud-hypervisor does support it, notably with the contributions of
upstream virtio-fs devs:
https://github.com/cloud-hypervisor/cloud-hypervisor/blob/v0.14.0/docs/fs.md
>
> On Friday, March 26, 2021 at 1:44:30 PM UTC-4 Fotis Xenakis wrote:
>
>> Update: the wiki page is here:
>> https://github.com/cloudius-systems/osv/wiki/virtio-fs
>> All questions/feedback and of course contributions welcome!
>>
>> On Saturday, March 13, 2021 at 9:04:03 PM UTC+2 Fotis Xenakis wrote:
>>
>>> You are absolutely right, some proper guide and documentation is
>>> necessary, thank you for the suggestion!
>>> I shall take care of it upon first chance.
>>>
>>> On Saturday, March 13, 2021 at 6:29:28 PM UTC+2 jwkoz...@gmail.com
>>> wrote:
>>>
Hey,
I have applied 3 of your patches.
As I was doing and testing it I realized we do not have any Wiki page
that describes how to use virtio-fs with OSv. Would you have a bit of time
to add a page that does so - a simple step-by-step cookbook - how to build
with export and fs options, and then run with various new options (DAX, no
dax, etc), need to use sudo, which version of QEMU, etc -
https://github.com/cloudius-systems/osv/wiki? Maybe also link to your
QEMU article?
Right now we have some info scattered acros some commit messages but
nothing in one place.
I would really appreciate it,
Waldek
On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
> Signed-off-by: Fotis Xenakis
> ---
> scripts/run.py | 5 -
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/run.py b/scripts/run.py
> index 71c6cdd7..6a81e98b 100755
> --- a/scripts/run.py
> +++ b/scripts/run.py
> @@ -178,9 +178,10 @@ def start_osv_qemu(options):
> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
>
> if options.virtio_fs_tag:
> + dax = (",cache-size=%s" % options.virtio_fs_dax) if
> options.virtio_fs_dax else ""
> args += [
> "-chardev", "socket,id=char0,path=/tmp/vhostqemu",
> - "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s"
> % options.virtio_fs_tag,
> + "-device",
> "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s" %
> (options.virtio_fs_tag, dax),
> "-object",
> "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" %
> options.memsize,
> "-numa", "node,memdev=mem"]
>
> @@ -602,6 +603,8 @@ if __name__ == "__main__":
> help="virtio-fs device tag")
> parser.add_argument("--virtio-fs-dir", action="store",
> help="path to the directory exposed via virtio-fs mount")
> + parser.add_argument("--virtio-fs-dax", action="store",
> + help="DAX window size for virtio-fs device (disabled if not
> specified)")
> parser.add_argument("--mount-fs", default=[], action="append",
> help="extra mounts (forwarded to respective kernel command line
> option)")
> parser.add_argument("--ip", default=[], action="append",
> --
> 2.30.1
>
>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/538fe958-fb89-4286-8cb5-a0dc4664ae72n%40googlegroups.com.
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
Fotis,
Thanks a lot!
I have updated this page to include the information about ARM - virtio-fs
works on this arch as well which is great!
The virtio-fs support is probably the single most exciting new feature
added to OSv in the last couple of years! I am not aware of any other
unikernel supporting it.
Thanks a lot,
Waldek
PS. It would be nice if some other hypervisors (thinking of firecracker)
supported virtio-fs as well.
On Friday, March 26, 2021 at 1:44:30 PM UTC-4 Fotis Xenakis wrote:
> Update: the wiki page is here:
> https://github.com/cloudius-systems/osv/wiki/virtio-fs
> All questions/feedback and of course contributions welcome!
>
> On Saturday, March 13, 2021 at 9:04:03 PM UTC+2 Fotis Xenakis wrote:
>
>> You are absolutely right, some proper guide and documentation is
>> necessary, thank you for the suggestion!
>> I shall take care of it upon first chance.
>>
>> On Saturday, March 13, 2021 at 6:29:28 PM UTC+2 jwkoz...@gmail.com wrote:
>>
>>> Hey,
>>>
>>> I have applied 3 of your patches.
>>>
>>> As I was doing and testing it I realized we do not have any Wiki page
>>> that describes how to use virtio-fs with OSv. Would you have a bit of time
>>> to add a page that does so - a simple step-by-step cookbook - how to build
>>> with export and fs options, and then run with various new options (DAX, no
>>> dax, etc), need to use sudo, which version of QEMU, etc -
>>> https://github.com/cloudius-systems/osv/wiki? Maybe also link to your
>>> QEMU article?
>>>
>>> Right now we have some info scattered acros some commit messages but
>>> nothing in one place.
>>>
>>> I would really appreciate it,
>>> Waldek
>>>
>>> On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
>>>
Signed-off-by: Fotis Xenakis
---
scripts/run.py | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/scripts/run.py b/scripts/run.py
index 71c6cdd7..6a81e98b 100755
--- a/scripts/run.py
+++ b/scripts/run.py
@@ -178,9 +178,10 @@ def start_osv_qemu(options):
"-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
if options.virtio_fs_tag:
+ dax = (",cache-size=%s" % options.virtio_fs_dax) if
options.virtio_fs_dax else ""
args += [
"-chardev", "socket,id=char0,path=/tmp/vhostqemu",
- "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" %
options.virtio_fs_tag,
+ "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s"
% (options.virtio_fs_tag, dax),
"-object",
"memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" %
options.memsize,
"-numa", "node,memdev=mem"]
@@ -602,6 +603,8 @@ if __name__ == "__main__":
help="virtio-fs device tag")
parser.add_argument("--virtio-fs-dir", action="store",
help="path to the directory exposed via virtio-fs mount")
+ parser.add_argument("--virtio-fs-dax", action="store",
+ help="DAX window size for virtio-fs device (disabled if not
specified)")
parser.add_argument("--mount-fs", default=[], action="append",
help="extra mounts (forwarded to respective kernel command line
option)")
parser.add_argument("--ip", default=[], action="append",
--
2.30.1
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/ba06bfd1-6345-48ca-9de6-d7d79cba8decn%40googlegroups.com.
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
Update: the wiki page is here:
https://github.com/cloudius-systems/osv/wiki/virtio-fs
All questions/feedback and of course contributions welcome!
On Saturday, March 13, 2021 at 9:04:03 PM UTC+2 Fotis Xenakis wrote:
> You are absolutely right, some proper guide and documentation is
> necessary, thank you for the suggestion!
> I shall take care of it upon first chance.
>
> On Saturday, March 13, 2021 at 6:29:28 PM UTC+2 jwkoz...@gmail.com wrote:
>
>> Hey,
>>
>> I have applied 3 of your patches.
>>
>> As I was doing and testing it I realized we do not have any Wiki page
>> that describes how to use virtio-fs with OSv. Would you have a bit of time
>> to add a page that does so - a simple step-by-step cookbook - how to build
>> with export and fs options, and then run with various new options (DAX, no
>> dax, etc), need to use sudo, which version of QEMU, etc -
>> https://github.com/cloudius-systems/osv/wiki? Maybe also link to your
>> QEMU article?
>>
>> Right now we have some info scattered acros some commit messages but
>> nothing in one place.
>>
>> I would really appreciate it,
>> Waldek
>>
>> On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
>>
>>> Signed-off-by: Fotis Xenakis
>>> ---
>>> scripts/run.py | 5 -
>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/run.py b/scripts/run.py
>>> index 71c6cdd7..6a81e98b 100755
>>> --- a/scripts/run.py
>>> +++ b/scripts/run.py
>>> @@ -178,9 +178,10 @@ def start_osv_qemu(options):
>>> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
>>>
>>> if options.virtio_fs_tag:
>>> + dax = (",cache-size=%s" % options.virtio_fs_dax) if
>>> options.virtio_fs_dax else ""
>>> args += [
>>> "-chardev", "socket,id=char0,path=/tmp/vhostqemu",
>>> - "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" %
>>> options.virtio_fs_tag,
>>> + "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s"
>>> % (options.virtio_fs_tag, dax),
>>> "-object",
>>> "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" %
>>> options.memsize,
>>> "-numa", "node,memdev=mem"]
>>>
>>> @@ -602,6 +603,8 @@ if __name__ == "__main__":
>>> help="virtio-fs device tag")
>>> parser.add_argument("--virtio-fs-dir", action="store",
>>> help="path to the directory exposed via virtio-fs mount")
>>> + parser.add_argument("--virtio-fs-dax", action="store",
>>> + help="DAX window size for virtio-fs device (disabled if not
>>> specified)")
>>> parser.add_argument("--mount-fs", default=[], action="append",
>>> help="extra mounts (forwarded to respective kernel command line
>>> option)")
>>> parser.add_argument("--ip", default=[], action="append",
>>> --
>>> 2.30.1
>>>
>>>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/678edccc-859f-4c8d-8109-7227479e8341n%40googlegroups.com.
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
On Saturday, March 13, 2021 at 6:53:28 PM UTC+2 jwkoz...@gmail.com wrote:
> BTW is DAX enabled by default or you have to explicitly enable it?
>
If the --virtio-fs-dax option to run.py is not specified, the DAX window
is disabled, so you have to explicitly enable it (trying to be backwards
compatible and not intrusive).
>
> On Saturday, March 13, 2021 at 11:29:28 AM UTC-5 Waldek Kozaczuk wrote:
>
>> Hey,
>>
>> I have applied 3 of your patches.
>>
>> As I was doing and testing it I realized we do not have any Wiki page
>> that describes how to use virtio-fs with OSv. Would you have a bit of time
>> to add a page that does so - a simple step-by-step cookbook - how to build
>> with export and fs options, and then run with various new options (DAX, no
>> dax, etc), need to use sudo, which version of QEMU, etc -
>> https://github.com/cloudius-systems/osv/wiki? Maybe also link to your
>> QEMU article?
>>
>> Right now we have some info scattered acros some commit messages but
>> nothing in one place.
>>
>> I would really appreciate it,
>> Waldek
>>
>> On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
>>
>>> Signed-off-by: Fotis Xenakis
>>> ---
>>> scripts/run.py | 5 -
>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/run.py b/scripts/run.py
>>> index 71c6cdd7..6a81e98b 100755
>>> --- a/scripts/run.py
>>> +++ b/scripts/run.py
>>> @@ -178,9 +178,10 @@ def start_osv_qemu(options):
>>> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
>>>
>>> if options.virtio_fs_tag:
>>> + dax = (",cache-size=%s" % options.virtio_fs_dax) if
>>> options.virtio_fs_dax else ""
>>> args += [
>>> "-chardev", "socket,id=char0,path=/tmp/vhostqemu",
>>> - "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" %
>>> options.virtio_fs_tag,
>>> + "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s"
>>> % (options.virtio_fs_tag, dax),
>>> "-object",
>>> "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" %
>>> options.memsize,
>>> "-numa", "node,memdev=mem"]
>>>
>>> @@ -602,6 +603,8 @@ if __name__ == "__main__":
>>> help="virtio-fs device tag")
>>> parser.add_argument("--virtio-fs-dir", action="store",
>>> help="path to the directory exposed via virtio-fs mount")
>>> + parser.add_argument("--virtio-fs-dax", action="store",
>>> + help="DAX window size for virtio-fs device (disabled if not
>>> specified)")
>>> parser.add_argument("--mount-fs", default=[], action="append",
>>> help="extra mounts (forwarded to respective kernel command line
>>> option)")
>>> parser.add_argument("--ip", default=[], action="append",
>>> --
>>> 2.30.1
>>>
>>>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/b13c3a51-51ca-4298-b5af-0e364e7086acn%40googlegroups.com.
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
On Saturday, March 13, 2021 at 6:08:29 PM UTC+2 jwkoz...@gmail.com wrote:
> Hi,
>
> The other 2 patches work fine but this one fails when I test it. What am I
> doing wrong?
>
>
> sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH
> ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export
> --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64
> --virtio-fs-dax=1024M
> qemu-system-x86_64: -device
> vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M:
> Property 'vhost-user-fs-pci.cache-size' not found
> qemu failed.
>
> sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH
> ./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export
> --qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64
> --virtio-fs-dax=1024M --dry-run
> /home/wkozaczuk/projects/osv-master/scripts/../scripts/imgedit.py setargs
> /home/wkozaczuk/projects/osv-master/build/last/usr.img "--rootfs=virtiofs
> /cli"
> virtiofsd \
> --socket-path=/tmp/vhostqemu \
> -o source=/home/wkozaczuk/projects/osv-master/build/export
> build/downloaded_packages/qemu/build/qemu-system-x86_64 \
> -m 2G \
> -smp 4 \
> -vnc :1 \
> -gdb tcp::1234,server,nowait \
> -device virtio-blk-pci,id=blk0,drive=hd0,scsi=off,bootindex=0 \
> -drive
> file=/home/wkozaczuk/projects/osv-master/build/last/usr.img,if=none,id=hd0,cache=none,aio=native
>
> \
> -chardev socket,id=char0,path=/tmp/vhostqemu \
> -device
> vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M \
> -object memory-backend-file,id=mem,size=2G,mem-path=/dev/shm,share=on \
> -numa node,memdev=mem \
> -netdev user,id=un0,net=192.168.122.0/24,host=192.168.122.1 \
> -device virtio-net-pci,netdev=un0 \
> -device virtio-rng-pci \
> -enable-kvm \
> -cpu host,+x2apic \
> -chardev stdio,mux=on,id=stdio,signal=off \
> -mon chardev=stdio,mode=readline \
> -device isa-serial,chardev=stdio
>
> I am using QEMU 5.2 built from sources.
>
Unfortunately DAX support has yet to be merged to upstream QEMU (I was
under the impression that it had been staged for QEMU 5.2.0, but alas it is
not part of it).
So, to use it, one still needs to use the virtio-fs downstream version
which includes the necessary patches [1].
The virtio-fs team are actively pursuing its upstreaming, but I 'm afraid I
don't have a clear view as to how long that might still take.
Regarding the patch, do you think it would be better to hold it back until
there's proper support upstream?
[1] https://gitlab.com/virtio-fs/qemu/-/tree/virtio-fs-dev
>
> Waldek
> On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
>
>> Signed-off-by: Fotis Xenakis
>> ---
>> scripts/run.py | 5 -
>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/scripts/run.py b/scripts/run.py
>> index 71c6cdd7..6a81e98b 100755
>> --- a/scripts/run.py
>> +++ b/scripts/run.py
>> @@ -178,9 +178,10 @@ def start_osv_qemu(options):
>> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
>>
>> if options.virtio_fs_tag:
>> + dax = (",cache-size=%s" % options.virtio_fs_dax) if
>> options.virtio_fs_dax else ""
>> args += [
>> "-chardev", "socket,id=char0,path=/tmp/vhostqemu",
>> - "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" %
>> options.virtio_fs_tag,
>> + "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s" %
>> (options.virtio_fs_tag, dax),
>> "-object",
>> "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" %
>> options.memsize,
>> "-numa", "node,memdev=mem"]
>>
>> @@ -602,6 +603,8 @@ if __name__ == "__main__":
>> help="virtio-fs device tag")
>> parser.add_argument("--virtio-fs-dir", action="store",
>> help="path to the directory exposed via virtio-fs mount")
>> + parser.add_argument("--virtio-fs-dax", action="store",
>> + help="DAX window size for virtio-fs device (disabled if not
>> specified)")
>> parser.add_argument("--mount-fs", default=[], action="append",
>> help="extra mounts (forwarded to respective kernel command line option)")
>> parser.add_argument("--ip", default=[], action="append",
>> --
>> 2.30.1
>>
>>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/4d4a5a13-64de-4b6b-b9a0-bbee60d384aen%40googlegroups.com.
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
BTW is DAX enabled by default or you have to explicitly enable it?
On Saturday, March 13, 2021 at 11:29:28 AM UTC-5 Waldek Kozaczuk wrote:
> Hey,
>
> I have applied 3 of your patches.
>
> As I was doing and testing it I realized we do not have any Wiki page that
> describes how to use virtio-fs with OSv. Would you have a bit of time to
> add a page that does so - a simple step-by-step cookbook - how to build
> with export and fs options, and then run with various new options (DAX, no
> dax, etc), need to use sudo, which version of QEMU, etc -
> https://github.com/cloudius-systems/osv/wiki? Maybe also link to your
> QEMU article?
>
> Right now we have some info scattered acros some commit messages but
> nothing in one place.
>
> I would really appreciate it,
> Waldek
>
> On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
>
>> Signed-off-by: Fotis Xenakis
>> ---
>> scripts/run.py | 5 -
>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/scripts/run.py b/scripts/run.py
>> index 71c6cdd7..6a81e98b 100755
>> --- a/scripts/run.py
>> +++ b/scripts/run.py
>> @@ -178,9 +178,10 @@ def start_osv_qemu(options):
>> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
>>
>> if options.virtio_fs_tag:
>> + dax = (",cache-size=%s" % options.virtio_fs_dax) if
>> options.virtio_fs_dax else ""
>> args += [
>> "-chardev", "socket,id=char0,path=/tmp/vhostqemu",
>> - "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" %
>> options.virtio_fs_tag,
>> + "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s" %
>> (options.virtio_fs_tag, dax),
>> "-object",
>> "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on" %
>> options.memsize,
>> "-numa", "node,memdev=mem"]
>>
>> @@ -602,6 +603,8 @@ if __name__ == "__main__":
>> help="virtio-fs device tag")
>> parser.add_argument("--virtio-fs-dir", action="store",
>> help="path to the directory exposed via virtio-fs mount")
>> + parser.add_argument("--virtio-fs-dax", action="store",
>> + help="DAX window size for virtio-fs device (disabled if not
>> specified)")
>> parser.add_argument("--mount-fs", default=[], action="append",
>> help="extra mounts (forwarded to respective kernel command line option)")
>> parser.add_argument("--ip", default=[], action="append",
>> --
>> 2.30.1
>>
>>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/9d394a4c-f6bd-46be-9045-86d9d7a21fb7n%40googlegroups.com.
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
Hey,
I have applied 3 of your patches.
As I was doing and testing it I realized we do not have any Wiki page that
describes how to use virtio-fs with OSv. Would you have a bit of time to
add a page that does so - a simple step-by-step cookbook - how to build
with export and fs options, and then run with various new options (DAX, no
dax, etc), need to use sudo, which version of QEMU, etc -
https://github.com/cloudius-systems/osv/wiki? Maybe also link to your QEMU
article?
Right now we have some info scattered acros some commit messages but
nothing in one place.
I would really appreciate it,
Waldek
On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
> Signed-off-by: Fotis Xenakis
> ---
> scripts/run.py | 5 -
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/run.py b/scripts/run.py
> index 71c6cdd7..6a81e98b 100755
> --- a/scripts/run.py
> +++ b/scripts/run.py
> @@ -178,9 +178,10 @@ def start_osv_qemu(options):
> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
>
> if options.virtio_fs_tag:
> + dax = (",cache-size=%s" % options.virtio_fs_dax) if
> options.virtio_fs_dax else ""
> args += [
> "-chardev", "socket,id=char0,path=/tmp/vhostqemu",
> - "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" %
> options.virtio_fs_tag,
> + "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s" %
> (options.virtio_fs_tag, dax),
> "-object", "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on"
> % options.memsize,
> "-numa", "node,memdev=mem"]
>
> @@ -602,6 +603,8 @@ if __name__ == "__main__":
> help="virtio-fs device tag")
> parser.add_argument("--virtio-fs-dir", action="store",
> help="path to the directory exposed via virtio-fs mount")
> + parser.add_argument("--virtio-fs-dax", action="store",
> + help="DAX window size for virtio-fs device (disabled if not specified)")
> parser.add_argument("--mount-fs", default=[], action="append",
> help="extra mounts (forwarded to respective kernel command line option)")
> parser.add_argument("--ip", default=[], action="append",
> --
> 2.30.1
>
>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/7a9bd19e-0174-4399-af04-4df2d697240bn%40googlegroups.com.
[osv-dev] Re: [PATCH 3/3] scripts: add virtio-fs DAX option to run.py
Hi,
The other 2 patches work fine but this one fails when I test it. What am I
doing wrong?
sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH
./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export
--qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64
--virtio-fs-dax=1024M
qemu-system-x86_64: -device
vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M:
Property 'vhost-user-fs-pci.cache-size' not found
qemu failed.
sudo PATH=build/downloaded_packages/qemu/build/tools/virtiofsd:$PATH
./scripts/run.py --virtio-fs-tag=myfs --virtio-fs-dir=$(pwd)/build/export
--qemu-path build/downloaded_packages/qemu/build/qemu-system-x86_64
--virtio-fs-dax=1024M --dry-run
/home/wkozaczuk/projects/osv-master/scripts/../scripts/imgedit.py setargs
/home/wkozaczuk/projects/osv-master/build/last/usr.img "--rootfs=virtiofs
/cli"
virtiofsd \
--socket-path=/tmp/vhostqemu \
-o source=/home/wkozaczuk/projects/osv-master/build/export
build/downloaded_packages/qemu/build/qemu-system-x86_64 \
-m 2G \
-smp 4 \
-vnc :1 \
-gdb tcp::1234,server,nowait \
-device virtio-blk-pci,id=blk0,drive=hd0,scsi=off,bootindex=0 \
-drive
file=/home/wkozaczuk/projects/osv-master/build/last/usr.img,if=none,id=hd0,cache=none,aio=native
\
-chardev socket,id=char0,path=/tmp/vhostqemu \
-device
vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=1024M \
-object memory-backend-file,id=mem,size=2G,mem-path=/dev/shm,share=on \
-numa node,memdev=mem \
-netdev user,id=un0,net=192.168.122.0/24,host=192.168.122.1 \
-device virtio-net-pci,netdev=un0 \
-device virtio-rng-pci \
-enable-kvm \
-cpu host,+x2apic \
-chardev stdio,mux=on,id=stdio,signal=off \
-mon chardev=stdio,mode=readline \
-device isa-serial,chardev=stdio
I am using QEMU 5.2 built from sources.
Waldek
On Saturday, March 6, 2021 at 4:34:31 PM UTC-5 Fotis Xenakis wrote:
> Signed-off-by: Fotis Xenakis
> ---
> scripts/run.py | 5 -
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/run.py b/scripts/run.py
> index 71c6cdd7..6a81e98b 100755
> --- a/scripts/run.py
> +++ b/scripts/run.py
> @@ -178,9 +178,10 @@ def start_osv_qemu(options):
> "-drive", "file=%s,if=none,id=hd1" % (options.cloud_init_image)]
>
> if options.virtio_fs_tag:
> + dax = (",cache-size=%s" % options.virtio_fs_dax) if
> options.virtio_fs_dax else ""
> args += [
> "-chardev", "socket,id=char0,path=/tmp/vhostqemu",
> - "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s" %
> options.virtio_fs_tag,
> + "-device", "vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=%s%s" %
> (options.virtio_fs_tag, dax),
> "-object", "memory-backend-file,id=mem,size=%s,mem-path=/dev/shm,share=on"
> % options.memsize,
> "-numa", "node,memdev=mem"]
>
> @@ -602,6 +603,8 @@ if __name__ == "__main__":
> help="virtio-fs device tag")
> parser.add_argument("--virtio-fs-dir", action="store",
> help="path to the directory exposed via virtio-fs mount")
> + parser.add_argument("--virtio-fs-dax", action="store",
> + help="DAX window size for virtio-fs device (disabled if not specified)")
> parser.add_argument("--mount-fs", default=[], action="append",
> help="extra mounts (forwarded to respective kernel command line option)")
> parser.add_argument("--ip", default=[], action="append",
> --
> 2.30.1
>
>
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/70ccaf02-eb39-4480-b1d4-db111295d18an%40googlegroups.com.
