Re: Intention to work on GSoC project
Hi,
On Monday, May 13, 2024 7:53:40 PM GMT+5:30 Eugenio Perez Martin wrote:
> [...]
> > I have started working on implementing packed virtqueue support in
> > vhost-shadow-virtqueue.c. The changes I have made so far are very
> > minimal. I have one confusion as well.
> >
> > In "vhost_svq_add()" [1], a structure of type "VhostShadowVirtqueue"
> > is being used. My initial idea was to create a whole new structure (eg:
> > VhostShadowVirtqueuePacked). But I realized that "VhostShadowVirtqueue"
> > is being used in a lot of other places such as in "struct vhost_vdpa" [2]
> > (in "vhost-vdpa.h"). So maybe this isn't a good idea.
> >
> > The problem is that "VhostShadowVirtqueue" has a member of type "struct
> > vring" [3] which represents a split virtqueue [4]. My idea now is to
> > instead wrap this member in a union so that the struct would look
> > something like this.
> >
> > struct VhostShadowVirtqueue {
> > union {
> > struct vring vring;
> > struct packed_vring vring;
> > }
> > ...
> > }
> >
> > I am not entirely sure if this is a good idea. It is similar to what's
> > been done in linux's "drivers/virtio/virtio_ring.c" ("struct
> > vring_virtqueue" [5]).
> >
> > I thought I would ask this first before continuing further.
>
> That's right, this second option makes perfect sense.
>
> VhostShadowVirtqueue should abstract both split and packed. You'll see
> that some members are reused, while others are only used in one
> version so they are placed after a union. They should follow the same
> pattern, although it is not a problem if we need to divert a little
> bit from the kernel's code.
>
Understood, thank you for the reply.
Thanks,
Sahil
Re: Intention to work on GSoC project
Hi,
On Wednesday, May 8, 2024 8:53:12 AM GMT+5:30 Sahil wrote:
> Hi,
>
> On Tuesday, May 7, 2024 12:44:33 PM IST Eugenio Perez Martin wrote:
> > [...]
> >
> > > Shall I start by implementing a mechanism to check if the feature bit
> > > "VIRTIO_F_RING_PACKED" is set (using "virtio_vdev_has_feature")? And
> > > if it's supported, "vhost_svq_add" should call "vhost_svq_add_packed".
> > > Following this, I can then start implementing "vhost_svq_add_packed"
> > > and progress from there.
> > >
> > > What are your thoughts on this?
> >
> > Yes, that's totally right.
> >
> > I recommend you to also disable _F_EVENT_IDX to start, so the first
> > version is easier.
> >
> > Also, you can send as many incomplete RFCs as you want. For example,
> > you can send a first version that only implements reading of the guest
> > avail ring, so we know we're aligned on that. Then, we can send
> > subsequents RFCs adding features on top.
>
I have started working on implementing packed virtqueue support in
vhost-shadow-virtqueue.c. The changes I have made so far are very
minimal. I have one confusion as well.
In "vhost_svq_add()" [1], a structure of type "VhostShadowVirtqueue"
is being used. My initial idea was to create a whole new structure (eg:
VhostShadowVirtqueuePacked). But I realized that "VhostShadowVirtqueue"
is being used in a lot of other places such as in "struct vhost_vdpa" [2]
(in "vhost-vdpa.h"). So maybe this isn't a good idea.
The problem is that "VhostShadowVirtqueue" has a member of type "struct
vring" [3] which represents a split virtqueue [4]. My idea now is to instead
wrap this member in a union so that the struct would look something like
this.
struct VhostShadowVirtqueue {
union {
struct vring vring;
struct packed_vring vring;
}
...
}
I am not entirely sure if this is a good idea. It is similar to what's been done
in linux's "drivers/virtio/virtio_ring.c" ("struct vring_virtqueue" [5]).
I thought I would ask this first before continuing further.
Thanks,
Sahil
[1]
https://gitlab.com/qemu-project/qemu/-/blob/master/hw/virtio/vhost-shadow-virtqueue.c#L249
[2]
https://gitlab.com/qemu-project/qemu/-/blob/master/include/hw/virtio/vhost-vdpa.h#L69
[3]
https://gitlab.com/qemu-project/qemu/-/blob/master/hw/virtio/vhost-shadow-virtqueue.h#L52
[4]
https://gitlab.com/qemu-project/qemu/-/blob/master/include/standard-headers/linux/virtio_ring.h#L156
[5]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/virtio/virtio_ring.c#n199
Re: Intention to work on GSoC project
Hi, On Tuesday, May 7, 2024 12:44:33 PM IST Eugenio Perez Martin wrote: > [...] > > Shall I start by implementing a mechanism to check if the feature bit > > "VIRTIO_F_RING_PACKED" is set (using "virtio_vdev_has_feature")? And > > if it's supported, "vhost_svq_add" should call "vhost_svq_add_packed". > > Following this, I can then start implementing "vhost_svq_add_packed" > > and progress from there. > > > > What are your thoughts on this? > > Yes, that's totally right. > > I recommend you to also disable _F_EVENT_IDX to start, so the first > version is easier. > > Also, you can send as many incomplete RFCs as you want. For example, > you can send a first version that only implements reading of the guest > avail ring, so we know we're aligned on that. Then, we can send > subsequents RFCs adding features on top. > > Does that make sense to you? > > Thanks! > Thank you for your reply. This is perfect. I'll start with this and will send the first RFC as soon as possible. Thanks, Sahil
Re: Intention to work on GSoC project
Hi, It's been a while since I last gave an update. Sorry about that. I am ready to get my hands dirty and start with the implementation. I have gone through the source of linux's drivers/virtio/virtio_ring.c [1], and QEMU's hw/virtio/virtio.c [2] and hw/virtio/vhost-shadow-virtqueue.c [3]. Before actually starting I would like to make sure I am on the right track. In vhost-shadow-virtqueue.c, there's a function "vhost_svq_add" which in turn calls "vhost_svq_add_split". Shall I start by implementing a mechanism to check if the feature bit "VIRTIO_F_RING_PACKED" is set (using "virtio_vdev_has_feature")? And if it's supported, "vhost_svq_add" should call "vhost_svq_add_packed". Following this, I can then start implementing "vhost_svq_add_packed" and progress from there. What are your thoughts on this? Thanks, Sahil [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/virtio/virtio.c [2] https://gitlab.com/qemu-project/qemu/-/blob/master/hw/virtio/virtio.c [3] https://gitlab.com/qemu-project/qemu/-/blob/master/hw/virtio/vhost-shadow-virtqueue.c
Re: Intention to work on GSoC project
Hi, Thank you for your reply. On Tuesday, April 16, 2024 2:11:16 PM IST Eugenio Perez Martin wrote: > [...] > > After re-reading the linked articles, I think I have got some more > > clarity. One confusion was related to the difference between vdpa > > and vhost-vdpa. > > > > So far what I have understood is that L0 acts as the host and L1 > > acts as the guest in this setup. I understand that the guest can't > > see the vDPA device. > > > > I now also understand that vdpa_sim is not a PCI device. I am also > > under the impression that vdpa refers to the vdpa bus while > > vhost-vdpa is the device. Is my understanding correct? > > > > After running the commands in the blog [1], I see that there's a > > vhost-vdpa-0 device under /dev. > > > > I also have an entry "vdpa0" under /sys/bus/vdpa/devices/ which > > is a symlink to /sys/devices/vdpa0. There's a dir "vhost-vdpa-0" > > under "/sys/devices/vdpa0". Hypothetically, if vhost-vdpa-0 had > > been a PCI device, then it would have been present under > > /sys/bus/pci/devices, right? > > Right. You'll check that scenario with the vp_vdpa one. > > > Another source of confusion was the pci.0 option passed to the > > qemu-kvm command. But I have understood this as well now: > > "-device virtio-net-pci" is a pci device. > > > > > > There's one more thing. In "use case 1" of "Running traffic with > > > > vhost_vdpa in Guest" [1], running "modprobe pktgen" in the L1 VM > > > > > > > > gives an error: > > > > > module pktgen couldn't be found in > > > > > /lib/modules/6.5.6-300.fc39.x86_64. > > > > > > > > The kernel version is 6.5.6-300.fc39.x86_64. I haven't tried building > > > > pktgen manually in L1. I'll try that and will check if vdpa_sim works > > > > as expected after that. > > > > > > Did you install kernel-modules-internal? > > > > I just realized I had the wrong version of kernel-modules-internal > > installed. It works after installing the right version. > > Good! So you can move to vp_vdpa, or do you have more doubts > about vdpa_sim? I am ready to move on to vp_vdpa. I don't have any more doubts about vdpa_sim as of now. Thanks, Sahil
Re: Intention to work on GSoC project
Hi, Thank you for your reply. On Monday, April 15, 2024 2:27:36 PM IST Eugenio Perez Martin wrote: > [...] > > I have one question though. One of the options (use case 1 in [1]) > > > > given to the "qemu-kvm" command is: > > > -device virtio-net-pci,netdev=vhost-vdpa0,bus=pcie.0,addr=0x7\ > > > ,disable-modern=off,page-per-vq=on > > > > This gives an error: > > > Bus "pcie.0" not found > > > > Does pcie refer to PCI Express? Changing this to pci.0 works. > > Yes, you don't need to mess with pcie stuff so this solution is > totally valid. I think we need to change that part in the tutorial. > Understood. > > I read through the "device buses" section in QEMU's user > > documentation [5], but I have still not understood this. > > > > "ls /sys/bus/pci/devices/* | grep vdpa" does not give any results. > > Replacing pci with pci_express doesn't give any results either. How > > does one know which pci bus the vdpa device is connected to? > > I have gone through the "vDPA bus drivers" section of the "vDPA > > kernel framework" article [6] but I haven't managed to find an > > answer yet. Am I missing something here? > > You cannot see the vDPA device from the guest. From the guest POV is a > regular virtio over PCI bus. > > From the host, vdpa_sim is not a PCI device either, so you cannot see > under /sys/bus. Do you have a vdpa* entry under > /sys/bus/vdpa/devices/? > After re-reading the linked articles, I think I have got some more clarity. One confusion was related to the difference between vdpa and vhost-vdpa. So far what I have understood is that L0 acts as the host and L1 acts as the guest in this setup. I understand that the guest can't see the vDPA device. I now also understand that vdpa_sim is not a PCI device. I am also under the impression that vdpa refers to the vdpa bus while vhost-vdpa is the device. Is my understanding correct? After running the commands in the blog [1], I see that there's a vhost-vdpa-0 device under /dev. I also have an entry "vdpa0" under /sys/bus/vdpa/devices/ which is a symlink to /sys/devices/vdpa0. There's a dir "vhost-vdpa-0" under "/sys/devices/vdpa0". Hypothetically, if vhost-vdpa-0 had been a PCI device, then it would have been present under /sys/bus/pci/devices, right? Another source of confusion was the pci.0 option passed to the qemu-kvm command. But I have understood this as well now: "-device virtio-net-pci" is a pci device. > > There's one more thing. In "use case 1" of "Running traffic with > > vhost_vdpa in Guest" [1], running "modprobe pktgen" in the L1 VM > > > > gives an error: > > > module pktgen couldn't be found in /lib/modules/6.5.6-300.fc39.x86_64. > > > > The kernel version is 6.5.6-300.fc39.x86_64. I haven't tried building > > pktgen manually in L1. I'll try that and will check if vdpa_sim works > > as expected after that. > > Did you install kernel-modules-internal? I just realized I had the wrong version of kernel-modules-internal installed. It works after installing the right version. Thanks, Sahil [1] https://www.redhat.com/en/blog/hands-vdpa-what-do-you-do-when-you-aint-got-hardware-part-1
Re: Intention to work on GSoC project
Hi, On Friday, April 5, 2024 12:36:02 AM IST Sahil wrote: > [...] > I'll set up this environment as well. I would like to post an update here. I spent the last week trying to set up the environment as described in the blog [1]. I initially tried to get the L1 VM running on my host machine (Arch Linux). However, I was unable to use virt-sysprep or virt-cutomize to install packages in the qcow2 image. It wasn't able to resolve the hosts while downloading the packages. According to the logs, /etc/resolv.conf was a dangling symlink. I tried to use "virt-rescue" to configure DNS resolution. I tried following these sections [2], [3] in the Arch wiki but that didn't work either. I tried using qemu-nbd as well following this section [4] to access the image. While I managed gain access to the image, I wasn't able to install packages after performing a chroot. One workaround was to set this environment up in a VM. I decided to set up the environment with a Fedora image in virtualbox acting as L0. I have managed to set up an L1 VM in this environment and I can load it using qemu-kvm. I have one question though. One of the options (use case 1 in [1]) given to the "qemu-kvm" command is: > -device virtio-net-pci,netdev=vhost-vdpa0,bus=pcie.0,addr=0x7\ > ,disable-modern=off,page-per-vq=on This gives an error: > Bus "pcie.0" not found Does pcie refer to PCI Express? Changing this to pci.0 works. I read through the "device buses" section in QEMU's user documentation [5], but I have still not understood this. "ls /sys/bus/pci/devices/* | grep vdpa" does not give any results. Replacing pci with pci_express doesn't give any results either. How does one know which pci bus the vdpa device is connected to? I have gone through the "vDPA bus drivers" section of the "vDPA kernel framework" article [6] but I haven't managed to find an answer yet. Am I missing something here? There's one more thing. In "use case 1" of "Running traffic with vhost_vdpa in Guest" [1], running "modprobe pktgen" in the L1 VM gives an error: > module pktgen couldn't be found in /lib/modules/6.5.6-300.fc39.x86_64. The kernel version is 6.5.6-300.fc39.x86_64. I haven't tried building pktgen manually in L1. I'll try that and will check if vdpa_sim works as expected after that. [1] https://www.redhat.com/en/blog/hands-vdpa-what-do-you-do-when-you-aint-got-hardware-part-1 [2] https://wiki.archlinux.org/title/QEMU#User-mode_networking [3] https://wiki.archlinux.org/title/Systemd-networkd#Required_services_and_setup [4] https://wiki.archlinux.org/title/QEMU#Mounting_a_partition_from_a_qcow2_image [5] https://qemu-project.gitlab.io/qemu/system/device-emulation.html [6] https://www.redhat.com/en/blog/vdpa-kernel-framework-part-1-vdpa-bus-abstracting-hardware Thanks, Sahil
Re: Intention to work on GSoC project
Hi, On Thursday, April 4, 2024 12:07:49 AM IST Eugenio Perez Martin wrote: > On Wed, Apr 3, 2024 at 4:36 PM Sahil wrote: > [...] > > I would like to clarify one thing in the figure "Full two-entries > > descriptor table". The driver can only overwrite a used descriptor in the > > descriptor ring, right? > > Except for the first round, the driver can only write to used entries > in the descriptor table. In other words, their avail and used flags > must be equal. > > > And likewise for the device? > > Yes, but with avail descs. I think you got this already, but I want to > be as complete as possible here. > > > So in the figure, the driver will have to wait until descriptor[1] is > > used before it can overwrite it? > > Yes, but I think it is easier to think that both descriptor id 0 and 1 > are available already. The descriptor id must be less than virtqueue > size. > > An entry with a valid buffer and length must be invalid because of the > descriptor id in that situation, either because it is a number > vq > length or because it is a descriptor already available. I didn't think of it in that way. This makes sense now. > > Suppose the device marks descriptor[0] as used. I think the driver will > > not be able to overwrite that descriptor entry because it has to go in > > order and is at descriptor[1]. Is that correct? > > The device must write one descriptor as used, either 0 or 1, at > descriptors[0] as all the descriptors are available. > > Now, it does not matter if the device marks as used one or the two > descriptors: the driver must write its next available descriptor at > descriptor[1]. This is not because descriptor[1] contains a special > field or data, but because the driver must write the avail descriptors > sequentially, so the device knows the address to poll or check after a > notification. > > In other words, descriptor[1] is just a buffer space from the driver > to communicate an available descriptor to the device. It does not > matter what it contained before the writing, as the driver must > process that information before writing the new available descriptor. > > > Is it possible for the driver > > to go "backwards" in the descriptor ring? > > Nope, under any circumstance. Understood. Thank you for the clarification. > [...] > > Q1. > > In the paragraph just above Figure 6, there is the following line: > > > the vhost kernel thread and QEMU may run in different CPU threads, > > > so these writes must be synchronized with QEMU cleaning of the dirty > > > bitmap, and this write must be seen strictly after the modifications of > > > the guest memory by the QEMU thread. > > > > I am not clear on the last part of the statement. The modification of > > guest memory is being done by the vhost device and not by the QEMU > > thread, right? > > QEMU also writes to the bitmap cleaning it, so it knows the memory > does not need to be resent. Oh, I thought, from figure 6, the bitmap is a part of QEMU's memory but is separate from the guest's memory. > Feel free to ask questions about this, but you don't need to interact > with the dirty bitmap in the project. Understood, I won't go off on a tangent in that case. > [...] > > Regarding the implementation of this project, can the project be broken > > down into two parts: > > 1. implementing packed virtqueues in QEMU, and > > Right, but let me expand on this: QEMU already supports packed > virtqueue in an emulated device (hw/virtio/virtio.c). The missing part > is the "driver" one, to be able to communicate with a vDPA device, at > hw/virtio/vhost-shadow-virtqueue.c. Got it. I'll take a look at "hw/virtio/virtio.c". > [...] > > My plan is to also understand how split virtqueue has been implemented > > in QEMU. I think that'll be helpful when moving the kernel's implementation > > to QEMU. > > Sure, the split virtqueue is implemented in the same file > vhost_shadow_virtqueue.c. If you deploy vhost_vdpa +vdpa_sim or > vp_vdpa [1][2], you can: > * Run QEMU with -netdev type=vhost-vdpa,x-svq=on > * Set GDB breakpoint in interesting functions like > vhost_handle_guest_kick and vhost_svq_flush. I'll set up this environment as well. Thanks, Sahil
Re: Intention to work on GSoC project
lso not very sure about what happens when notifications are > > disabled. > > I'll have to read up on that again. But I believe the driver still won't > > be able to touch #0 until the device uses it. > > If one side disables notification it needs to check the indexes or the > flags by its own means: Timers, read the memory in a busy loop, etc. Understood. Thank you for the clarification. I have some questions from the "Virtio live migration technical deep dive" article [1]. Q1. In the paragraph just above Figure 6, there is the following line: > the vhost kernel thread and QEMU may run in different CPU threads, > so these writes must be synchronized with QEMU cleaning of the dirty > bitmap, and this write must be seen strictly after the modifications of > the guest memory by the QEMU thread. I am not clear on the last part of the statement. The modification of guest memory is being done by the vhost device and not by the QEMU thread, right? Q2. In the first point of the "Dynamic device state: virtqueue state" section: >The guest makes available N descriptors at the source of the migration, >so its avail idx member in the avail idx is N. I think there's a typo here: "...avail idx member in the avail ring is N" instead of "...avail idx is N". Regarding the implementation of this project, can the project be broken down into two parts: 1. implementing packed virtqueues in QEMU, and 2. providing mechanisms for (live) migration to work with packed virtqueues. I am ready to start working on the implementation. In one of your previous emails you had talked about moving packed virtqueue related implementation from the kernel's drivers/virtio/virtio_ring.c into vhost_shadow_virtqueue.c. My plan is to also understand how split virtqueue has been implemented in QEMU. I think that'll be helpful when moving the kernel's implementation to QEMU. Please let me know if I should change my approach. Thanks, Sahil
Re: Intention to work on GSoC project
Hi,
On Monday, April 1, 2024 11:53:11 PM IST daleyoung4...@gmail.com wrote:
> Hi,
>
> On Monday, March 25, 2024 21:20:32 CST Sahil wrote:
> > Q1.
> > Section 2.7.4 of the virtio spec [3] states that in an available
> > descriptor, the "Element Length" stores the length of the buffer element.
> > In the next few lines, it also states that the "Element Length" is
> > reserved for used descriptors and is ignored by drivers. This sounds a
> > little contradictory given that drivers write available desciptors in the
> > descriptor ring.
> When VIRTQ_DESC_F_WRITE is set, the device will use "Element Length" to
> specify the length it writes. When VIRTQ_DESC_F_WRITE is not set, which
> means the buffer is read-only for the device, "Element Length" will not be
> changed by the device, so drivers just ignore it.
Thank you for the clarification. I think I misunderstood what I had read
in the virtio spec. What I have understood now is that "Element Length"
has different meanings for available and used descriptors.
Correct me if I am wrong - for available descriptors, it represents the
length of the buffer. For used descriptors, it represents the length of
the buffer that is written to by the device if it's write-only, otherwise it
has no meaning and hence can be ignored by drivers.
> > Q2.
> > In the Red Hat article, just below the first listing ("Memory layout of a
> > packed virtqueue descriptor"), there's the following line referring to the
> > buffer id in
> >
> > "virtq_desc":
> > > This time, the id field is not an index for the device to look for the
> > > buffer: it is an opaque value for it, only has meaning for the driver.
> >
> > But the device returns the buffer id when it writes the used descriptor to
> > the descriptor ring. The "only has meaning for the driver" part has got me
> > a little confused. Which buffer id is this that the device returns? Is it
> > related to the buffer id in the available descriptor?
>
> In my understanding, buffer id is the element that avail descriptor marks to
> identify when adding descriptors to table. Device will returns the buffer
> id in the processed descriptor or the last descriptor in a chain, and write
> it to the descriptor that used idx refers to (first one in the chain). Then
> used idx increments.
>
> The Packed Virtqueue blog [1] is helpful, but some details in the examples
> are making me confused.
>
> Q1.
> In the last step of the two-entries descriptor table example, it says both
> buffers #0 and #1 are available for the device. I understand descriptor[0]
> is available and descriptor[1] is not, but there is no ID #0 now. So does
> the device got buffer #0 by notification beforehand? If so, does it mean
> buffer #0 will be lost when notifications are disabled?
>
I too have a similar question and understanding the relation between buffer
ids in the used and available descriptors might give more insight into this. For
available descriptors, the buffer id is used to associate descriptors with a
particular buffer. I am still not very sure about ids in used descriptors.
Regarding Q1, both buffers #0 and #1 are available. In the mentioned figure,
both descriptor[0] and descriptor[1] are available. This figure follows the
figure
with the caption "Using first buffer out of order". So in the first figure the
device
reads buffer #1 and writes the used descriptor but it still has buffer #0 to
read.
That still belongs to the device while buffer #1 can now be handled by the
driver
once again. So in the next figure, the driver makes buffer #1 available again.
The
device can still read buffer #0 from the previous batch of available
descriptors.
Based on what I have understood, the driver can't touch the descriptor
corresponding to buffer #0 until the device acknowledges it. I did find the
figure a little confusing as well. I think once the meaning of buffer id is
clear
from the driver's and device's perspective, it'll be easier to understand the
figure.
I am also not very sure about what happens when notifications are disabled.
I'll have to read up on that again. But I believe the driver still won't be
able to
touch #0 until the device uses it.
I think going through the source should better explain these concepts.
Thanks,
Sahil
Re: Intention to work on GSoC project
quot;drivers/virtio/*" are enabled when linux is being run in a guest VM?
>
> For PCI devices, as long as it detects a device with vendor == Red
> Hat, Inc. (0x1AF4) and device ID 0x1000 through 0x107F inclusive, yes.
> You can also load and unload manually with modprobe as other drivers.
Got it. The linux documentation in "documentation/driver-api/virtio/virtio.rst"
covers this as well. I must have missed this before sending the email.
> Let me know if you have more doubts. Thanks!
I have also read through the "Packed virtqueue: How to reduce overhead
with virtio" article [1] and the relevant section from the virtio specification
[2].
This has been slightly difficult to grasp. Going through the source in
"drivers/virtio/virtio_ring.c" is helping a bit but I am still confused about
a few things.
Q1.
Section 2.7.4 of the virtio spec [3] states that in an available descriptor, the
"Element Length" stores the length of the buffer element. In the next few lines,
it also states that the "Element Length" is reserved for used descriptors and is
ignored by drivers. This sounds a little contradictory given that drivers write
available desciptors in the descriptor ring.
Q2.
In the Red Hat article, just below the first listing ("Memory layout of a packed
virtqueue descriptor"), there's the following line referring to the buffer id in
"virtq_desc":
> This time, the id field is not an index for the device to look for the
> buffer: it is
> an opaque value for it, only has meaning for the driver.
But the device returns the buffer id when it writes the used descriptor to the
descriptor ring. The "only has meaning for the driver" part has got me a little
confused. Which buffer id is this that the device returns? Is it related to the
buffer id in the available descriptor?
Thanks,
Sahil
[1] https://www.redhat.com/en/blog/packed-virtqueue-how-reduce-overhead-virtio
[2]
https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-610007
[3]
https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-650004
Re: Intention to work on GSoC project
Hi, I was reading the "Virtqueues and virtio ring: How the data travels" article [1]. There are a few things that I have not understood in the "avail rings" section. Q1. Step 2 in the "Process to make a buffer available" diagram depicts how the virtio driver writes the descriptor index in the avail ring. In the example, the descriptor index #0 is written in the first entry. But in figure 2, the number 0 is in the 4th position in the avail ring. Is the avail ring queue an array of "struct virtq_avail" which maintains metadata such as the number of descriptor indexes in the header? Also, in the second position, the number changes from 0 (figure 1) to 1 (figure 2). I haven't understood what idx, 0 (later 1) and ring[] represent in the figures. Does this number represent the number of descriptors that are currently in the avail ring? Q2. There's this paragraph in the article right below the above mentioned diagram: > The avail ring must be able to hold the same number of descriptors > as the descriptor area, and the descriptor area must have a size power > of two, so idx wraps naturally at some point. For example, if the ring > size is 256 entries, idx 1 references the same descriptor as idx 257, 513... > And it will wrap at a 16 bit boundary. This way, neither side needs to > worry about processing an invalid idx: They are all valid. I haven't really understood this. I have understood that idx is calculated as idx mod queue_length. But I haven't understood the "16 bit boundary" part. I am also not very clear on how a queue length that is not a power of 2 might cause trouble. Could you please expand on this? Q3. I have started going through the source code in "drivers/virtio/virtio_ring.c". I have understood that the virtio driver runs in the guest's kernel. Does that mean the drivers in "drivers/virtio/*" are enabled when linux is being run in a guest VM? Thanks, Sahil [1] https://www.redhat.com/en/blog/virtqueues-and-virtio-ring-how-data-travels
Re: Intention to work on GSoC project
Hi, Thank you for your reply. On Friday, March 15, 2024 4:57:39 PM IST Eugenio Perez Martin wrote: > [...] > > Some sections in the above docs were difficult to grasp. For the time > > being, I have focused on those parts that I thought were relevant > > to the project. > > Please feel free to ask any questions, maybe we can improve the doc :). I understood the introductory sections of the documentation such as the "About QEMU" section and the first half of the "system emulation". Sections and subsections that went into greater detail were a little overwhelming such as the "QEMU virtio-net standby" subsection [1] or the "migration features" [2] subsection. But the red hat blogs and deep-dive articles helped cover a lot of ground conceptually. I feel once I start getting my hands dirty, I'll be able to absorb these concepts much better. I did have two questions that I would like to ask. Q1. Regarding the "Deep dive into Virtio-networking and vhost-net" article [3], the "Introduction" subsection of the "Vhost protocol" section mentions that sending the available buffer notification involves a vCPU interrupt (4th bullet point). But in figure 2, the arrow for the "available buffer notification" indicates a PCI interrupt. Initially I thought they were two different interrupts but I am a little confused about this now. Q2. In the "Virtio-net failover operation" section of the "Virtio-net failover: An introduction" article [4], there are five bullet points under the first figure. The second point states that the guest kernel needs the ability to switch between the VFIO device and the vfio-net device. I was wondering if "vfio-net" is a typo and if it should be "virtio-net" instead. > [...] > There is a post before the first in the series: > https://www.redhat.com/en/blog/virtio-devices-and-drivers-overview-headjack- > and-phone Got it. I didn't know this was the first in the series. I have now covered this as well, so I can move on to "Virtqueues and virtio ring: How the data travels" [3] :) > > 1. Virtqueues and virtio ring: How the data travels [8] > > 2. Packed virtqueue: How to reduce overhead with virtio [9] > > 3. Virtio live migration technical deep dive [10] > > 4. Hands on vDPA: what do you do when you ain't got the hardware v2 (Part > > 1) [11] > I think it's a good plan! > > If you feel like you're reading a lot of theory and want to get your > hands dirty already, you can also start messing with the code with the > blogs you already read. Or, maybe, after reading the Packed virtqueue > one, your call. > > In a very brute-forced description, you can start trying to copy all > the *packed* stuff of kernel's drivers/virtio/virtio_ring.c into > vhost_shadow_virtqueue.c. I would love to start with some hands-on tasks. I'll take a look at the kernel's "drivers/virtio/virtio_ring.c". I think I should also start going through the "vhost_shadow_virtqueue.c" [4] source code. > There is a lot more in the task, and I can get into more detail > if you want either here or in a meeting. Thank you. Either means of communication works for me although the latter will require some coordination. > If you prefer to continue with the theory it is ok too. A good balance of theory and practice would be nice at this stage. It'll keep my brains from getting too muddled up. Thanks, Sahil [1] https://www.qemu.org/docs/master/system/virtio-net-failover.html [2] https://www.qemu.org/docs/master/devel/migration/features.html [3] https://www.redhat.com/en/blog/deep-dive-virtio-networking-and-vhost-net [4] https://www.redhat.com/en/blog/virtio-net-failover-introduction [5] https://www.redhat.com/en/blog/virtqueues-and-virtio-ring-how-data-travels
Re: Intention to work on GSoC project
Hi, Thank you for your email. On Thursday, March 14, 2024 8:39:45 PM IST Eugenio Perez Martin wrote: > Hi Sahil, > > It's being hard to find a good self-contained small task related to > the project to be honest. As it would be out of SVQ, would it be ok > for you if we start straight to the task of adding the packed vq > format to SVQ? > > Thanks! Sure, this works too! I would love to get started with the project. I have a small update as well. I have read through a few docs and articles to familiarize myself with the relevant terminology and technicalities. 1. "About", "system emulation" and "user mode emulation" sections of the user documentation [1] 2. The migration subsystem [2] Some sections in the above docs were difficult to grasp. For the time being, I have focused on those parts that I thought were relevant to the project. I have also read through the following articles: 1. Introduction to virtio-networking and vhost-net [3] 2. Deep dive into Virtio-networking and vhost-net [4] 3. Virtualized Hardware Devices [5] 4. VFIO - "Virtual Function I/O" (Just the introduction) [6] 5. Virtio-net failover: An introduction [7] I hope I haven't gone off on a tangent. I was planning to finish reading up on the following articles as well: 1. Virtqueues and virtio ring: How the data travels [8] 2. Packed virtqueue: How to reduce overhead with virtio [9] 3. Virtio live migration technical deep dive [10] 4. Hands on vDPA: what do you do when you ain't got the hardware v2 (Part 1) [11] I believe the hands-on vPDA article will have me set up a development environment for the project as well. Please let me know if I should amend my roadmap. I am excited to get started :) Thanks, Sahil [1] https://www.qemu.org/docs/master/index.html [2] https://www.qemu.org/docs/master/devel/migration/index.html [3] https://www.redhat.com/en/blog/introduction-virtio-networking-and-vhost-net [4] https://www.redhat.com/en/blog/deep-dive-virtio-networking-and-vhost-net [5] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_getting_started_guide/sec-virtualization_getting_started-products-virtualized-hardware-devices [6] https://www.kernel.org/doc/html/latest/driver-api/vfio.html [7] https://www.redhat.com/en/blog/virtio-net-failover-introduction [8] https://www.redhat.com/en/blog/virtqueues-and-virtio-ring-how-data-travels [9] https://developers.redhat.com/articles/2024/02/21/virtio-live-migration-technical-deep-dive [10] https://www.redhat.com/en/blog/packed-virtqueue-how-reduce-overhead-virtio [11] https://www.redhat.com/en/blog/hands-vdpa-what-do-you-do-when-you-aint-got-hardware-part-1
Re: Intention to work on GSoC project
Hi, On Friday, March 1, 2024 1:10:49 PM IST you wrote: > [...] > You can add the recently published > "virtio-live-migration-technical-deep-dive" :) [1]. > > [1] > https://developers.redhat.com/articles/2024/02/21/virtio-live-migration-technical-deep-dive Thank you. This resource will help me cover a lot of ground. > [...] > There are few other tasks pending for QEMU in general and SVQ in > particular. I'll add them as gitlab issues by today. > That would be nice. I'll keep an eye out for the new issues. Thanks, Sahil
Re: Intention to work on GSoC project
Hi, Thank you for your reply. On Thursday, February 29, 2024 2:02:25 PM IST you wrote: > [...] > The first project is for sure related with migration. While vhost-user > memory isolation is not really related to migration, but both are > related to virtio devices. > [...] > Yeah, "bite-size" issues should be better to understand how to > contribute to QEMU. Feel free to work on any issue, doing the work > or helping to complete old patches. Got it. In that case it'll be best to complete the bite-sized task related to virtio [1]. A patch [2] had been long ago but it hasn't been completed yet. I'll work on getting it merged. I have already started reading the documentation on the migration subsystem and the hands-on blogs in the project description. That will help me get started with the first project. > Feel free to reach us if you have more questions on the projects. Yes, I'll definitely do that. Thanks, Sahil [1] https://gitlab.com/qemu-project/qemu/-/issues/230 [2] https://patchwork.kernel.org/project/qemu-devel/patch/20220414112902.41390-1-codeguy.mot...@gmail.com/
Intention to work on GSoC project
Hi, My name is Sahil and I go by the pseudonym 'valdaarhun' on Github. I have never contributed to QEMU before but I have used it a few times as an end user. I developed an interest in virtualization during my internship at VMware and would like to dive deeper in this subfield. My current full-time job does not allow me to take part in external programs that are paid. I would like to work on one of the proposed projects outside of GSoC. I have gone through QEMU's list of GSoC '24 projects [1] and am interested in two of them: 1. Add packed virtqueue to Shadow Virtqueue 2. vhost-user memory isolation Based on what I have understood, they are somewhat related and are part of the migration subsystem. I feel the learning curve of the first project will be less steep and will make me better prepared to tackle the second project as well. I have read the "Getting Started for Developers" [2] wiki page. I have also built QEMU from source. I think my next step should be to read the documentation on the migration subsystem [3], the blog posts attached in the first project's description and virtqueue's implementation. Would you also recommend that I work on a QEMU issue that is open on Gitlab and related to virtqueues/virtio to familiarize myself with the codebase? I went through the issues tagged as "device:virtio" [4] but can't really tell if any of them are good for beginners. One of them has the "bite-size" tag [5]. It also has a patch attached but hasn't been merged. Shall I work on getting that merged? I have worked on a few smaller systems programming issues in other organizations (eg: strace [6], htop [7]) in the past. I look forward to hearing from you. Regards, Sahil https://github.com/valdaarhun [1] https://wiki.qemu.org/Google_Summer_of_Code_2024 [2] https://wiki.qemu.org/Documentation/GettingStartedDevelopers [3] https://qemu-project.gitlab.io/qemu/devel/migration/index.html [4] https://gitlab.com/qemu-project/qemu/-/issues/? label_name%5B%5D=device%3Avirtio [5] https://gitlab.com/qemu-project/qemu/-/issues/230 [6] https://github.com/strace/strace/commits?author=valdaarhun [7] https://github.com/htop-dev/htop/commits?author=valdaarhun
