Re: [PATCH v2] Shared memory device with interrupt support
Anthony Liguori wrote: Avi Kivity wrote: Anthony Liguori wrote: I'd strongly recommend working these patches on qemu-devel and lkml. I suspect Avi may disagree with me, but in order for this to be eventually merged in either place, you're going to have additional requirements put on you. I don't disagree with the fact that there will be additional requirements, but I might disagree with some of those additional requirements themselves. It actually works out better than I think you expect it to... Can you explain why? You haven't addressed my concerns the last time around. We can't use mmap() directly. With the new RAM allocation scheme, I think it's pretty reasonable to now allow portions of ram to come from files that get mmap() (sort of like -mem-path). This RAM area could be setup as a BAR. That's what Cam's patch does, and what you objected to. In particular I think your proposal was unimplementable; I would like to see how how you can address my concerns. I don't remember what my proposal was to be perfectly honest :-) I think I suggested registering a guest allocated portion of memory as a sharable region via virtio? Yes. Why is that unimplementable? Bad choice of words - it's implementable, just not very usable. You can't share 1GB in a 256MB guest, will fragment host vmas, no guarantee the guest can actually allocate all that memory, doesn't work with large pages, what happens on freeing, etc. I don't think bulk memory sharing and the current transactional virtio mechanisms are a good fit for each other; but if we were to add a BAR-like capability to virtio that would address the compatibility requirement (though it might be difficult to implement on s390 with its requirement on contiguous host virtual address space). It doesn't necessarily have to be virtio if that's not what makes sense. The problem is not virtio, it's the transient scatter gather dma model that virtio supports. If virtio were to support BARs like Christian's patch proposes, then it could be easily done with virtio. Maybe we should call it something else though to avoid confusion. The QEMU bits and the device model bits are actually relatively simple. The part that I think needs more deep thought is the guest-visible interface. A char device is probably not the best interface. I think you want something like tmpfs/hugetlbfs. Yes those are so wonderful to work with. Another question is whether you want a guest to be able to share a portion of it's memory with another guest or have everything setup by the host. I think we want host setup. That way you have symmetry among the guests. If everything is setup by the host, hot plug is important. It is. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Shared memory device with interrupt support
Avi Kivity wrote: Anthony Liguori wrote: Avi Kivity wrote: Anthony Liguori wrote: I'd strongly recommend working these patches on qemu-devel and lkml. I suspect Avi may disagree with me, but in order for this to be eventually merged in either place, you're going to have additional requirements put on you. I don't disagree with the fact that there will be additional requirements, but I might disagree with some of those additional requirements themselves. It actually works out better than I think you expect it to... Can you explain why? You haven't addressed my concerns the last time around. Because of the qemu_ram_alloc() patches. We no longer have a contiguous phys_ram_base so we don't have to deal with mmap(MAP_FIXED). We can also more practically do memory hot-add which is more or less a requirement of this work. It also means we could do shared memory through more traditional means too like sys v ipc or whatever is the native mechanism on the underlying platform. That means we could even support Win32 (although I wouldn't make that an initial requirement). We can't use mmap() directly. With the new RAM allocation scheme, I think it's pretty reasonable to now allow portions of ram to come from files that get mmap() (sort of like -mem-path). This RAM area could be setup as a BAR. That's what Cam's patch does, and what you objected to. I'm flexible. BARs are pretty unattractive because of the size requirements. The actual transport implementation is the least important part though IMHO. The guest interface and how it's implemented within QEMU is much more important to get right the first time. Why is that unimplementable? Bad choice of words - it's implementable, just not very usable. You can't share 1GB in a 256MB guest, will fragment host vmas, no guarantee the guest can actually allocate all that memory, doesn't work with large pages, what happens on freeing, etc. You can share 1GB with a PCI BAR today. You're limited to 32-bit addresses which admittedly we could fix. Any reason to bother with BARs instead of just picking unused physical addresses? Does Windows do anything special with BAR addresses? The QEMU bits and the device model bits are actually relatively simple. The part that I think needs more deep thought is the guest-visible interface. A char device is probably not the best interface. I think you want something like tmpfs/hugetlbfs. Yes those are so wonderful to work with. qemu -ivshmem file=/dev/shm/ring.shared,name=shared-ring,size=1G,notify=/path/to/socket /path/to/socket is used to pass an eventfd Within the guest, you'd have: /dev/ivshmemfs/shared-ring An app would mmap() that file, and then could do something like an ioctl() to get an eventfd. Alternatively, you could have something like: /dev/ivshmemfs/mem/shared-ring /dev/ivshmemfs/notify/shared-ring Where notify/shared-ring behaves like an eventfd(). Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Shared memory device with interrupt support
Anthony Liguori wrote: I'd strongly recommend working these patches on qemu-devel and lkml. I suspect Avi may disagree with me, but in order for this to be eventually merged in either place, you're going to have additional requirements put on you. I don't disagree with the fact that there will be additional requirements, but I might disagree with some of those additional requirements themselves. It actually works out better than I think you expect it to... Can you explain why? You haven't addressed my concerns the last time around. Because of the qemu_ram_alloc() patches. We no longer have a contiguous phys_ram_base so we don't have to deal with mmap(MAP_FIXED). We can also more practically do memory hot-add which is more or less a requirement of this work. I think you're arguing my side. If the guest specifies the memory to be shared via an add_buf() sglist allocated from its free memory, you have to use MAP_FIXED (since the gpa-hva mapping is already fixed for guest memory). If it's provided as a BAR or equivalent, we can use a variant of qemu_ram_alloc() which binds to the shared segment instead of allocating. It also means we could do shared memory through more traditional means too like sys v ipc or whatever is the native mechanism on the underlying platform. That means we could even support Win32 (although I wouldn't make that an initial requirement). Not with add_buf() memory... We can't use mmap() directly. With the new RAM allocation scheme, I think it's pretty reasonable to now allow portions of ram to come from files that get mmap() (sort of like -mem-path). This RAM area could be setup as a BAR. That's what Cam's patch does, and what you objected to. I'm flexible. BARs are pretty unattractive because of the size requirements. What size requirements? The PCI memory hole? Those requirements are easily lifted. The actual transport implementation is the least important part though IMHO. The guest interface and how it's implemented within QEMU is much more important to get right the first time. I agree, with much more emphasis on the guest/host interface. Why is that unimplementable? Bad choice of words - it's implementable, just not very usable. You can't share 1GB in a 256MB guest, will fragment host vmas, no guarantee the guest can actually allocate all that memory, doesn't work with large pages, what happens on freeing, etc. You can share 1GB with a PCI BAR today. You're limited to 32-bit addresses which admittedly we could fix. Any reason to bother with BARs instead of just picking unused physical addresses? Does Windows do anything special with BAR addresses? If you use a BAR you let the host kernel know what you're doing. No doubt you could do the same thing yourself (the PCI support functions call the raw support functions), but if you use a BAR, everything from the BIOS onwards is plumbed down. Sure we could do something independent a la vbus, but my preference has always been to behave like real hardware. Oh, and if it's a BAR you can use device assignment. You can't assign a device that exposes memory the host doesn't know about. The QEMU bits and the device model bits are actually relatively simple. The part that I think needs more deep thought is the guest-visible interface. A char device is probably not the best interface. I think you want something like tmpfs/hugetlbfs. Yes those are so wonderful to work with. qemu -ivshmem file=/dev/shm/ring.shared,name=shared-ring,size=1G,notify=/path/to/socket /path/to/socket is used to pass an eventfd Within the guest, you'd have: /dev/ivshmemfs/shared-ring An app would mmap() that file, and then could do something like an ioctl() to get an eventfd. Alternatively, you could have something like: /dev/ivshmemfs/mem/shared-ring /dev/ivshmemfs/notify/shared-ring Where notify/shared-ring behaves like an eventfd(). Being the traditionalist that I am, I'd much prefer it to be a char device and use udev rules to get a meaningful name if needed. That's how every other real device works. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v2] Shared memory device with interrupt support
Cam, is it somehow possible to generate a local APIC interrupt from one VM to
another? I guess it shouldn't be as the LAPIC interrupts generated in one VM
will go to the VCPUs of the same VM...
Regards,
Bhaskar.
-Original Message-
From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of
Kumar, Venkat
Sent: Tuesday, May 19, 2009 9:22 AM
To: Cam Macdonell
Cc: kvm@vger.kernel.org list
Subject: RE: [PATCH v2] Shared memory device with interrupt support
I had tried all syntaxes other than this :).
Interrupts work now.
Thx,
Venkat
-Original Message-
From: Cam Macdonell [mailto:c...@cs.ualberta.ca]
Sent: Monday, May 18, 2009 9:51 PM
To: Kumar, Venkat
Cc: kvm@vger.kernel.org list
Subject: Re: [PATCH v2] Shared memory device with interrupt support
Kumar, Venkat wrote:
Cam - I got your patch to work but without notifications. I could share
memory using the patch but notifications aren't working.
I bring up two VM's with option -ivshmem shrmem,1024,/dev/shm/shrmem,server
and -ivshmem shrmem,1024,/dev/shm/shrmem respectively.
Ok, I guess I need to do more error checking of arguments :) You need
to specify unix: on the path. So your options should look like this
-ivshmem shrmem,1024,unix:/dev/shm/shrmem,server
-ivshmem shrmem,1024,unix:/dev/shm/shrmem
That should help.
Cam
When I make an ioctl from one of the VM's to inject an interrupt to the
other VM, I get an error in qemu_chr_write and return value is -1.
write call in send_all is failing with return value -1.
Am I missing something here?
Thx,
Venkat
-Original Message-
From: Cam Macdonell [mailto:c...@cs.ualberta.ca]
Sent: Saturday, May 16, 2009 9:01 AM
To: Kumar, Venkat
Cc: kvm@vger.kernel.org list
Subject: Re: [PATCH v2] Shared memory device with interrupt support
On 15-May-09, at 8:54 PM, Kumar, Venkat wrote:
Cam,
A questions on interrupts as well.
What is unix:path that needs to be passed in the argument list?
Can it be any string?
It has to be a valid path on the host. It will create a unix domain
socket on that path.
If my understanding is correct both the VM's who wants to
communicate would gives this path in the command line with one of
them specifying as server.
Exactly, the one with the server in the parameter list will wait for
a connection before booting.
Cam
Thx,
Venkat
Support an inter-vm shared memory device that maps a shared-
memory object
as a PCI device in the guest. This patch also supports interrupts
between
guest by communicating over a unix domain socket. This patch
applies to the
qemu-kvm repository.
This device now creates a qemu character device and sends 1-bytes
messages to
trigger interrupts. Writes are trigger by writing to the Doorbell
register
on the shared memory PCI device. The lower 8-bits of the value
written to this
register are sent as the 1-byte message so different meanings of
interrupts can
be supported.
Interrupts are only supported between 2 VMs currently. One VM must
act as the
server by adding server to the command-line argument. Shared
memory devices
are created with the following command-line:
-ivhshmem shm object,size in MB,[unix:path][,server]
Interrupts can also be used between host and guest as well by
implementing a
listener on the host.
Cam
---
Makefile.target |3 +
hw/ivshmem.c| 421 ++
+
hw/pc.c |6 +
hw/pc.h |3 +
qemu-options.hx | 14 ++
sysemu.h|8 +
vl.c| 14 ++
7 files changed, 469 insertions(+), 0 deletions(-)
create mode 100644 hw/ivshmem.c
diff --git a/Makefile.target b/Makefile.target
index b68a689..3190bba 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -643,6 +643,9 @@ OBJS += pcnet.o
OBJS += rtl8139.o
OBJS += e1000.o
+# Inter-VM PCI shared memory
+OBJS += ivshmem.o
+
# Generic watchdog support and some watchdog devices
OBJS += watchdog.o
OBJS += wdt_ib700.o wdt_i6300esb.o
diff --git a/hw/ivshmem.c b/hw/ivshmem.c
new file mode 100644
index 000..95e2268
--- /dev/null
+++ b/hw/ivshmem.c
@@ -0,0 +1,421 @@
+/*
+ * Inter-VM Shared Memory PCI device.
+ *
+ * Author:
+ * Cam Macdonell c...@cs.ualberta.ca
+ *
+ * Based On: cirrus_vga.c and rtl8139.c
+ *
+ * This code is licensed under the GNU GPL v2.
+ */
+
+#include hw.h
+#include console.h
+#include pc.h
+#include pci.h
+#include sysemu.h
+
+#include qemu-common.h
+#include sys/mman.h
+
+#define PCI_COMMAND_IOACCESS0x0001
+#define PCI_COMMAND_MEMACCESS 0x0002
+#define PCI_COMMAND_BUSMASTER 0x0004
+
+//#define DEBUG_IVSHMEM
+
+#ifdef DEBUG_IVSHMEM
+#define IVSHMEM_DPRINTF(fmt, args...)\
+do {printf(IVSHMEM: fmt, ##args); } while (0)
+#else
+#define IVSHMEM_DPRINTF(fmt, args...)
+#endif
+
+typedef struct IVShmemState {
+uint16_t intrmask;
+uint16_t
Re: [PATCH v2] Shared memory device with interrupt support
Jayaraman, Bhaskar wrote: Cam, is it somehow possible to generate a local APIC interrupt from one VM to another? I guess it shouldn't be as the LAPIC interrupts generated in one VM will go to the VCPUs of the same VM... Regards, Bhaskar. The closest thing to this is the irqfd+iosignalfd thing I mentioned the other day. With this model, a PIO/MMIO write in the src guest will directly inject an interrupt into the dst guest's LAPIC. However, as Avi points out, this is just an optimization. You can also do it by first taking a hop through each guests userspace as well. HTH -Greg signature.asc Description: OpenPGP digital signature
Re: [PATCH v2] Shared memory device with interrupt support
Avi Kivity wrote: Anthony Liguori wrote: I'd strongly recommend working these patches on qemu-devel and lkml. I suspect Avi may disagree with me, but in order for this to be eventually merged in either place, you're going to have additional requirements put on you. I don't disagree with the fact that there will be additional requirements, but I might disagree with some of those additional requirements themselves. It actually works out better than I think you expect it to... We can't use mmap() directly. With the new RAM allocation scheme, I think it's pretty reasonable to now allow portions of ram to come from files that get mmap() (sort of like -mem-path). This RAM area could be setup as a BAR. In particular I think your proposal was unimplementable; I would like to see how how you can address my concerns. I don't remember what my proposal was to be perfectly honest :-) I think I suggested registering a guest allocated portion of memory as a sharable region via virtio? Why is that unimplementable? I don't think bulk memory sharing and the current transactional virtio mechanisms are a good fit for each other; but if we were to add a BAR-like capability to virtio that would address the compatibility requirement (though it might be difficult to implement on s390 with its requirement on contiguous host virtual address space). It doesn't necessarily have to be virtio if that's not what makes sense. The QEMU bits and the device model bits are actually relatively simple. The part that I think needs more deep thought is the guest-visible interface. A char device is probably not the best interface. I think you want something like tmpfs/hugetlbfs. Another question is whether you want a guest to be able to share a portion of it's memory with another guest or have everything setup by the host. If everything is setup by the host, hot plug is important. Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Shared memory device with interrupt support
Avi Kivity wrote: Cam Macdonell wrote: If my understanding is correct both the VM's who wants to communicate would gives this path in the command line with one of them specifying as server. Exactly, the one with the server in the parameter list will wait for a connection before booting. hm, we may be able to eliminate the server from the fast path, at the cost of some complexity. When a guest connects to the server, the server creates an eventfd and passes using SCM_RIGHTS to all other connected guests. The server also passes the eventfds of currently connected guests to the new guest. From now on, the server does not participate in anything; when a quest wants to send an interrupt to one or more other guests, its qemu just writes to the eventfds() of the corresponding guests; their qemus will inject the interrupt, without any server involvement. Now, anyone who has been paying attention will have their alarms going off at the word eventfd. And yes, if the host supports irqfd, the various qemus can associate those eventfds with an irq and pretty much forget about them. When a qemu triggers an irqfd, the interrupt will be injected directly without the target qemu's involvement. I'll just add that you could tie the irqfd to an iosignalfd to eliminate the involvement of qemu on either side as well. I'm not sure if that really works with the design of this particular device (e.g. perhaps qemu is needed for other reasons besides signaling), but it is a neat demonstration of the flexibility of the newly emerging kvm-eventfd interfaces. -Greg signature.asc Description: OpenPGP digital signature
Re: [PATCH v2] Shared memory device with interrupt support
Gregory Haskins wrote: I'll just add that you could tie the irqfd to an iosignalfd to eliminate the involvement of qemu on either side as well. I'm not sure if that really works with the design of this particular device (e.g. perhaps qemu is needed for other reasons besides signaling), but it is a neat demonstration of the flexibility of the newly emerging kvm-eventfd interfaces. If we have an iosignalfd for point-to-point (say, a pio port with the guest ID) we can do direct guest-to-guest signalling. For broadcast or multicast, we need to exit to qemu to handle the loop. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v2] Shared memory device with interrupt support
Cam - I got your patch to work but without notifications. I could share memory
using the patch but notifications aren't working.
I bring up two VM's with option -ivshmem shrmem,1024,/dev/shm/shrmem,server
and -ivshmem shrmem,1024,/dev/shm/shrmem respectively.
When I make an ioctl from one of the VM's to inject an interrupt to the other
VM, I get an error in qemu_chr_write and return value is -1. write call
in send_all is failing with return value -1.
Am I missing something here?
Thx,
Venkat
-Original Message-
From: Cam Macdonell [mailto:c...@cs.ualberta.ca]
Sent: Saturday, May 16, 2009 9:01 AM
To: Kumar, Venkat
Cc: kvm@vger.kernel.org list
Subject: Re: [PATCH v2] Shared memory device with interrupt support
On 15-May-09, at 8:54 PM, Kumar, Venkat wrote:
Cam,
A questions on interrupts as well.
What is unix:path that needs to be passed in the argument list?
Can it be any string?
It has to be a valid path on the host. It will create a unix domain
socket on that path.
If my understanding is correct both the VM's who wants to
communicate would gives this path in the command line with one of
them specifying as server.
Exactly, the one with the server in the parameter list will wait for
a connection before booting.
Cam
Thx,
Venkat
Support an inter-vm shared memory device that maps a shared-
memory object
as a PCI device in the guest. This patch also supports interrupts
between
guest by communicating over a unix domain socket. This patch
applies to the
qemu-kvm repository.
This device now creates a qemu character device and sends 1-bytes
messages to
trigger interrupts. Writes are trigger by writing to the Doorbell
register
on the shared memory PCI device. The lower 8-bits of the value
written to this
register are sent as the 1-byte message so different meanings of
interrupts can
be supported.
Interrupts are only supported between 2 VMs currently. One VM must
act as the
server by adding server to the command-line argument. Shared
memory devices
are created with the following command-line:
-ivhshmem shm object,size in MB,[unix:path][,server]
Interrupts can also be used between host and guest as well by
implementing a
listener on the host.
Cam
---
Makefile.target |3 +
hw/ivshmem.c| 421 ++
+
hw/pc.c |6 +
hw/pc.h |3 +
qemu-options.hx | 14 ++
sysemu.h|8 +
vl.c| 14 ++
7 files changed, 469 insertions(+), 0 deletions(-)
create mode 100644 hw/ivshmem.c
diff --git a/Makefile.target b/Makefile.target
index b68a689..3190bba 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -643,6 +643,9 @@ OBJS += pcnet.o
OBJS += rtl8139.o
OBJS += e1000.o
+# Inter-VM PCI shared memory
+OBJS += ivshmem.o
+
# Generic watchdog support and some watchdog devices
OBJS += watchdog.o
OBJS += wdt_ib700.o wdt_i6300esb.o
diff --git a/hw/ivshmem.c b/hw/ivshmem.c
new file mode 100644
index 000..95e2268
--- /dev/null
+++ b/hw/ivshmem.c
@@ -0,0 +1,421 @@
+/*
+ * Inter-VM Shared Memory PCI device.
+ *
+ * Author:
+ * Cam Macdonell c...@cs.ualberta.ca
+ *
+ * Based On: cirrus_vga.c and rtl8139.c
+ *
+ * This code is licensed under the GNU GPL v2.
+ */
+
+#include hw.h
+#include console.h
+#include pc.h
+#include pci.h
+#include sysemu.h
+
+#include qemu-common.h
+#include sys/mman.h
+
+#define PCI_COMMAND_IOACCESS0x0001
+#define PCI_COMMAND_MEMACCESS 0x0002
+#define PCI_COMMAND_BUSMASTER 0x0004
+
+//#define DEBUG_IVSHMEM
+
+#ifdef DEBUG_IVSHMEM
+#define IVSHMEM_DPRINTF(fmt, args...)\
+do {printf(IVSHMEM: fmt, ##args); } while (0)
+#else
+#define IVSHMEM_DPRINTF(fmt, args...)
+#endif
+
+typedef struct IVShmemState {
+uint16_t intrmask;
+uint16_t intrstatus;
+uint16_t doorbell;
+uint8_t *ivshmem_ptr;
+unsigned long ivshmem_offset;
+unsigned int ivshmem_size;
+unsigned long bios_offset;
+unsigned int bios_size;
+target_phys_addr_t base_ctrl;
+int it_shift;
+PCIDevice *pci_dev;
+CharDriverState * chr;
+unsigned long map_addr;
+unsigned long map_end;
+int ivshmem_mmio_io_addr;
+} IVShmemState;
+
+typedef struct PCI_IVShmemState {
+PCIDevice dev;
+IVShmemState ivshmem_state;
+} PCI_IVShmemState;
+
+typedef struct IVShmemDesc {
+char name[1024];
+char * chrdev;
+int size;
+} IVShmemDesc;
+
+
+/* registers for the Inter-VM shared memory device */
+enum ivshmem_registers {
+IntrMask = 0,
+IntrStatus = 16,
+Doorbell = 32
+};
+
+static int num_ivshmem_devices = 0;
+static IVShmemDesc ivshmem_desc;
+
+static void ivshmem_map(PCIDevice *pci_dev, int region_num,
+uint32_t addr, uint32_t size, int type)
+{
+PCI_IVShmemState *d = (PCI_IVShmemState *)pci_dev;
+IVShmemState *s = d
Re: [PATCH v2] Shared memory device with interrupt support
Kumar, Venkat wrote:
Cam - I got your patch to work but without notifications. I could share memory
using the patch but notifications aren't working.
I bring up two VM's with option -ivshmem shrmem,1024,/dev/shm/shrmem,server and
-ivshmem shrmem,1024,/dev/shm/shrmem respectively.
Ok, I guess I need to do more error checking of arguments :) You need
to specify unix: on the path. So your options should look like this
-ivshmem shrmem,1024,unix:/dev/shm/shrmem,server
-ivshmem shrmem,1024,unix:/dev/shm/shrmem
That should help.
Cam
When I make an ioctl from one of the VM's to inject an interrupt to the other VM, I get an error in qemu_chr_write
and return value is -1. write call in send_all is failing with return value -1.
Am I missing something here?
Thx,
Venkat
-Original Message-
From: Cam Macdonell [mailto:c...@cs.ualberta.ca]
Sent: Saturday, May 16, 2009 9:01 AM
To: Kumar, Venkat
Cc: kvm@vger.kernel.org list
Subject: Re: [PATCH v2] Shared memory device with interrupt support
On 15-May-09, at 8:54 PM, Kumar, Venkat wrote:
Cam,
A questions on interrupts as well.
What is unix:path that needs to be passed in the argument list?
Can it be any string?
It has to be a valid path on the host. It will create a unix domain
socket on that path.
If my understanding is correct both the VM's who wants to
communicate would gives this path in the command line with one of
them specifying as server.
Exactly, the one with the server in the parameter list will wait for
a connection before booting.
Cam
Thx,
Venkat
Support an inter-vm shared memory device that maps a shared-
memory object
as a PCI device in the guest. This patch also supports interrupts
between
guest by communicating over a unix domain socket. This patch
applies to the
qemu-kvm repository.
This device now creates a qemu character device and sends 1-bytes
messages to
trigger interrupts. Writes are trigger by writing to the Doorbell
register
on the shared memory PCI device. The lower 8-bits of the value
written to this
register are sent as the 1-byte message so different meanings of
interrupts can
be supported.
Interrupts are only supported between 2 VMs currently. One VM must
act as the
server by adding server to the command-line argument. Shared
memory devices
are created with the following command-line:
-ivhshmem shm object,size in MB,[unix:path][,server]
Interrupts can also be used between host and guest as well by
implementing a
listener on the host.
Cam
---
Makefile.target |3 +
hw/ivshmem.c| 421 ++
+
hw/pc.c |6 +
hw/pc.h |3 +
qemu-options.hx | 14 ++
sysemu.h|8 +
vl.c| 14 ++
7 files changed, 469 insertions(+), 0 deletions(-)
create mode 100644 hw/ivshmem.c
diff --git a/Makefile.target b/Makefile.target
index b68a689..3190bba 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -643,6 +643,9 @@ OBJS += pcnet.o
OBJS += rtl8139.o
OBJS += e1000.o
+# Inter-VM PCI shared memory
+OBJS += ivshmem.o
+
# Generic watchdog support and some watchdog devices
OBJS += watchdog.o
OBJS += wdt_ib700.o wdt_i6300esb.o
diff --git a/hw/ivshmem.c b/hw/ivshmem.c
new file mode 100644
index 000..95e2268
--- /dev/null
+++ b/hw/ivshmem.c
@@ -0,0 +1,421 @@
+/*
+ * Inter-VM Shared Memory PCI device.
+ *
+ * Author:
+ * Cam Macdonell c...@cs.ualberta.ca
+ *
+ * Based On: cirrus_vga.c and rtl8139.c
+ *
+ * This code is licensed under the GNU GPL v2.
+ */
+
+#include hw.h
+#include console.h
+#include pc.h
+#include pci.h
+#include sysemu.h
+
+#include qemu-common.h
+#include sys/mman.h
+
+#define PCI_COMMAND_IOACCESS0x0001
+#define PCI_COMMAND_MEMACCESS 0x0002
+#define PCI_COMMAND_BUSMASTER 0x0004
+
+//#define DEBUG_IVSHMEM
+
+#ifdef DEBUG_IVSHMEM
+#define IVSHMEM_DPRINTF(fmt, args...)\
+do {printf(IVSHMEM: fmt, ##args); } while (0)
+#else
+#define IVSHMEM_DPRINTF(fmt, args...)
+#endif
+
+typedef struct IVShmemState {
+uint16_t intrmask;
+uint16_t intrstatus;
+uint16_t doorbell;
+uint8_t *ivshmem_ptr;
+unsigned long ivshmem_offset;
+unsigned int ivshmem_size;
+unsigned long bios_offset;
+unsigned int bios_size;
+target_phys_addr_t base_ctrl;
+int it_shift;
+PCIDevice *pci_dev;
+CharDriverState * chr;
+unsigned long map_addr;
+unsigned long map_end;
+int ivshmem_mmio_io_addr;
+} IVShmemState;
+
+typedef struct PCI_IVShmemState {
+PCIDevice dev;
+IVShmemState ivshmem_state;
+} PCI_IVShmemState;
+
+typedef struct IVShmemDesc {
+char name[1024];
+char * chrdev;
+int size;
+} IVShmemDesc;
+
+
+/* registers for the Inter-VM shared memory device */
+enum ivshmem_registers {
+IntrMask = 0,
+IntrStatus = 16,
+Doorbell = 32
+};
+
+static int num_ivshmem_devices = 0;
+static IVShmemDesc ivshmem_desc;
+
+static void ivshmem_map(PCIDevice *pci_dev, int region_num
Re: [PATCH v2] Shared memory device with interrupt support
Cam Macdonell wrote: My usual noob questions: Do I need to run Greg's tree on the host for the necessary irqfd/eventfd suppport? Yes (though irqfd will be merged real soon, and iosignalfd somewhat afterwards). Are there any examples to work from aside from Greg's unit tests? No. You don't really need to irqfd and iosignalfd to implement all of this, they're just optimizations. You can still pass the eventfds around. If you don't have irqfd, have qemu poll the eventfd, and when something happens, inject and interrupt. Likewise, if you don't have iosignalfd, register a pio/mmio handler for the command register, and when they fire touch all of the relevant irqfds. You'll need that anyway for backwards compatibility and for level-triggered pci interrupts, which irqfd doesn't support. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Shared memory device with interrupt support
Hi Cam, Cam Macdonell wrote: Support an inter-vm shared memory device that maps a shared-memory object as a PCI device in the guest. This patch also supports interrupts between guest by communicating over a unix domain socket. This patch applies to the qemu-kvm repository. This device now creates a qemu character device and sends 1-bytes messages to trigger interrupts. Writes are trigger by writing to the Doorbell register on the shared memory PCI device. The lower 8-bits of the value written to this register are sent as the 1-byte message so different meanings of interrupts can be supported. Interrupts are only supported between 2 VMs currently. One VM must act as the server by adding server to the command-line argument. Shared memory devices are created with the following command-line: -ivhshmem shm object,size in MB,[unix:path][,server] Interrupts can also be used between host and guest as well by implementing a listener on the host. Cam I'd strongly recommend working these patches on qemu-devel and lkml. I suspect Avi may disagree with me, but in order for this to be eventually merged in either place, you're going to have additional requirements put on you. If it goes in via qemu-kvm.git, there's a possibility that you'll be forced into an ABI break down the road (consider the old hypercall and balloon drivers). Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v2] Shared memory device with interrupt support
I had tried all syntaxes other than this :).
Interrupts work now.
Thx,
Venkat
-Original Message-
From: Cam Macdonell [mailto:c...@cs.ualberta.ca]
Sent: Monday, May 18, 2009 9:51 PM
To: Kumar, Venkat
Cc: kvm@vger.kernel.org list
Subject: Re: [PATCH v2] Shared memory device with interrupt support
Kumar, Venkat wrote:
Cam - I got your patch to work but without notifications. I could share
memory using the patch but notifications aren't working.
I bring up two VM's with option -ivshmem shrmem,1024,/dev/shm/shrmem,server
and -ivshmem shrmem,1024,/dev/shm/shrmem respectively.
Ok, I guess I need to do more error checking of arguments :) You need
to specify unix: on the path. So your options should look like this
-ivshmem shrmem,1024,unix:/dev/shm/shrmem,server
-ivshmem shrmem,1024,unix:/dev/shm/shrmem
That should help.
Cam
When I make an ioctl from one of the VM's to inject an interrupt to the
other VM, I get an error in qemu_chr_write and return value is -1.
write call in send_all is failing with return value -1.
Am I missing something here?
Thx,
Venkat
-Original Message-
From: Cam Macdonell [mailto:c...@cs.ualberta.ca]
Sent: Saturday, May 16, 2009 9:01 AM
To: Kumar, Venkat
Cc: kvm@vger.kernel.org list
Subject: Re: [PATCH v2] Shared memory device with interrupt support
On 15-May-09, at 8:54 PM, Kumar, Venkat wrote:
Cam,
A questions on interrupts as well.
What is unix:path that needs to be passed in the argument list?
Can it be any string?
It has to be a valid path on the host. It will create a unix domain
socket on that path.
If my understanding is correct both the VM's who wants to
communicate would gives this path in the command line with one of
them specifying as server.
Exactly, the one with the server in the parameter list will wait for
a connection before booting.
Cam
Thx,
Venkat
Support an inter-vm shared memory device that maps a shared-
memory object
as a PCI device in the guest. This patch also supports interrupts
between
guest by communicating over a unix domain socket. This patch
applies to the
qemu-kvm repository.
This device now creates a qemu character device and sends 1-bytes
messages to
trigger interrupts. Writes are trigger by writing to the Doorbell
register
on the shared memory PCI device. The lower 8-bits of the value
written to this
register are sent as the 1-byte message so different meanings of
interrupts can
be supported.
Interrupts are only supported between 2 VMs currently. One VM must
act as the
server by adding server to the command-line argument. Shared
memory devices
are created with the following command-line:
-ivhshmem shm object,size in MB,[unix:path][,server]
Interrupts can also be used between host and guest as well by
implementing a
listener on the host.
Cam
---
Makefile.target |3 +
hw/ivshmem.c| 421 ++
+
hw/pc.c |6 +
hw/pc.h |3 +
qemu-options.hx | 14 ++
sysemu.h|8 +
vl.c| 14 ++
7 files changed, 469 insertions(+), 0 deletions(-)
create mode 100644 hw/ivshmem.c
diff --git a/Makefile.target b/Makefile.target
index b68a689..3190bba 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -643,6 +643,9 @@ OBJS += pcnet.o
OBJS += rtl8139.o
OBJS += e1000.o
+# Inter-VM PCI shared memory
+OBJS += ivshmem.o
+
# Generic watchdog support and some watchdog devices
OBJS += watchdog.o
OBJS += wdt_ib700.o wdt_i6300esb.o
diff --git a/hw/ivshmem.c b/hw/ivshmem.c
new file mode 100644
index 000..95e2268
--- /dev/null
+++ b/hw/ivshmem.c
@@ -0,0 +1,421 @@
+/*
+ * Inter-VM Shared Memory PCI device.
+ *
+ * Author:
+ * Cam Macdonell c...@cs.ualberta.ca
+ *
+ * Based On: cirrus_vga.c and rtl8139.c
+ *
+ * This code is licensed under the GNU GPL v2.
+ */
+
+#include hw.h
+#include console.h
+#include pc.h
+#include pci.h
+#include sysemu.h
+
+#include qemu-common.h
+#include sys/mman.h
+
+#define PCI_COMMAND_IOACCESS0x0001
+#define PCI_COMMAND_MEMACCESS 0x0002
+#define PCI_COMMAND_BUSMASTER 0x0004
+
+//#define DEBUG_IVSHMEM
+
+#ifdef DEBUG_IVSHMEM
+#define IVSHMEM_DPRINTF(fmt, args...)\
+do {printf(IVSHMEM: fmt, ##args); } while (0)
+#else
+#define IVSHMEM_DPRINTF(fmt, args...)
+#endif
+
+typedef struct IVShmemState {
+uint16_t intrmask;
+uint16_t intrstatus;
+uint16_t doorbell;
+uint8_t *ivshmem_ptr;
+unsigned long ivshmem_offset;
+unsigned int ivshmem_size;
+unsigned long bios_offset;
+unsigned int bios_size;
+target_phys_addr_t base_ctrl;
+int it_shift;
+PCIDevice *pci_dev;
+CharDriverState * chr;
+unsigned long map_addr;
+unsigned long map_end;
+int ivshmem_mmio_io_addr;
+} IVShmemState;
+
+typedef struct PCI_IVShmemState {
+PCIDevice dev
Re: [PATCH v2] Shared memory device with interrupt support
Anthony Liguori wrote: I'd strongly recommend working these patches on qemu-devel and lkml. I suspect Avi may disagree with me, but in order for this to be eventually merged in either place, you're going to have additional requirements put on you. I don't disagree with the fact that there will be additional requirements, but I might disagree with some of those additional requirements themselves. In particular I think your proposal was unimplementable; I would like to see how how you can address my concerns. I don't think bulk memory sharing and the current transactional virtio mechanisms are a good fit for each other; but if we were to add a BAR-like capability to virtio that would address the compatibility requirement (though it might be difficult to implement on s390 with its requirement on contiguous host virtual address space). A model which does fit the current virtio capabilities is that of a DMA engine - guest A specifies an sglist to copy; guest B specifies an sglist to receive the copy; the host does the copy, using a real DMA engine if available. Note A == B is a possibility, and is a way to expose a DMA engine to a single guest for its own use in moving memory around. I think both models could prove useful. If it goes in via qemu-kvm.git, there's a possibility that you'll be forced into an ABI break down the road (consider the old hypercall and balloon drivers). I agree this is best merged upstream first. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Shared memory device with interrupt support
Cam Macdonell wrote: I don't think so. With the mmap call, I specify MAP_FIXED which requires that the memory in the shared memory object be mapped to the address given in the first parameter (s-ivshmem_ptr). If MAP_FIXED is not specified then mmap would allocate the memory and map on to it, but with MAP_FIXED it maps onto the already reserved space that ivshmem_ptr points to and was allocated with qemu_ram_alloc(). It might be nice to have a variant of qemu_ram_alloc() that takes a pointer to existing memory, so we don't have to play these MAP_FIXED games. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Shared memory device with interrupt support
Cam Macdonell wrote: If my understanding is correct both the VM's who wants to communicate would gives this path in the command line with one of them specifying as server. Exactly, the one with the server in the parameter list will wait for a connection before booting. hm, we may be able to eliminate the server from the fast path, at the cost of some complexity. When a guest connects to the server, the server creates an eventfd and passes using SCM_RIGHTS to all other connected guests. The server also passes the eventfds of currently connected guests to the new guest. From now on, the server does not participate in anything; when a quest wants to send an interrupt to one or more other guests, its qemu just writes to the eventfds() of the corresponding guests; their qemus will inject the interrupt, without any server involvement. Now, anyone who has been paying attention will have their alarms going off at the word eventfd. And yes, if the host supports irqfd, the various qemus can associate those eventfds with an irq and pretty much forget about them. When a qemu triggers an irqfd, the interrupt will be injected directly without the target qemu's involvement. I like it. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v2] Shared memory device with interrupt support
Hi Cam, I have gone through you latest shared memory patch.
I have a few questions and comments.
Comment:-
+if (ivshmem_enabled) {
+ivshmem_init(ivshmem_device);
+ram_size += ivshmem_get_size();
+}
+
In your initial patch this part of the patch is
+if (ivshmem_enabled) {
+ivshmem_init(ivshmem_device);
+phys_ram_size += ivshmem_get_size();
+}
I think the phys_ram_size += ivshmem_get_size(); is correct.
Question:-
You are giving the desired virtual address for mmaping the shared memory object
as s-ivshmem_ptr which is phys_ram_base + s-ivshmem_offset. This desired
virtual address is nothing but the base virtual address of the memory that you
are allocating after incrementing phys_ram_size. So now s-ivshmem_ptr would
point to a new set of memory, which is the shared memory region instead of
memory allocated through qemu_alloc_physram, which means if pages are allocated
for sh-ivshmem_ptr virtual address range then those pages can never be
addressed again. Correct me if my understanding is wrong.
Thx,
Venkat
-Original Message-
From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of
Cam Macdonell
Sent: Thursday, May 07, 2009 9:47 PM
To: kvm@vger.kernel.org
Cc: Cam Macdonell
Subject: [PATCH v2] Shared memory device with interrupt support
Support an inter-vm shared memory device that maps a shared-memory object
as a PCI device in the guest. This patch also supports interrupts between
guest by communicating over a unix domain socket. This patch applies to the
qemu-kvm repository.
This device now creates a qemu character device and sends 1-bytes messages to
trigger interrupts. Writes are trigger by writing to the Doorbell register
on the shared memory PCI device. The lower 8-bits of the value written to this
register are sent as the 1-byte message so different meanings of interrupts can
be supported.
Interrupts are only supported between 2 VMs currently. One VM must act as the
server by adding server to the command-line argument. Shared memory devices
are created with the following command-line:
-ivhshmem shm object,size in MB,[unix:path][,server]
Interrupts can also be used between host and guest as well by implementing a
listener on the host.
Cam
---
Makefile.target |3 +
hw/ivshmem.c| 421 +++
hw/pc.c |6 +
hw/pc.h |3 +
qemu-options.hx | 14 ++
sysemu.h|8 +
vl.c| 14 ++
7 files changed, 469 insertions(+), 0 deletions(-)
create mode 100644 hw/ivshmem.c
diff --git a/Makefile.target b/Makefile.target
index b68a689..3190bba 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -643,6 +643,9 @@ OBJS += pcnet.o
OBJS += rtl8139.o
OBJS += e1000.o
+# Inter-VM PCI shared memory
+OBJS += ivshmem.o
+
# Generic watchdog support and some watchdog devices
OBJS += watchdog.o
OBJS += wdt_ib700.o wdt_i6300esb.o
diff --git a/hw/ivshmem.c b/hw/ivshmem.c
new file mode 100644
index 000..95e2268
--- /dev/null
+++ b/hw/ivshmem.c
@@ -0,0 +1,421 @@
+/*
+ * Inter-VM Shared Memory PCI device.
+ *
+ * Author:
+ * Cam Macdonell c...@cs.ualberta.ca
+ *
+ * Based On: cirrus_vga.c and rtl8139.c
+ *
+ * This code is licensed under the GNU GPL v2.
+ */
+
+#include hw.h
+#include console.h
+#include pc.h
+#include pci.h
+#include sysemu.h
+
+#include qemu-common.h
+#include sys/mman.h
+
+#define PCI_COMMAND_IOACCESS0x0001
+#define PCI_COMMAND_MEMACCESS 0x0002
+#define PCI_COMMAND_BUSMASTER 0x0004
+
+//#define DEBUG_IVSHMEM
+
+#ifdef DEBUG_IVSHMEM
+#define IVSHMEM_DPRINTF(fmt, args...)\
+do {printf(IVSHMEM: fmt, ##args); } while (0)
+#else
+#define IVSHMEM_DPRINTF(fmt, args...)
+#endif
+
+typedef struct IVShmemState {
+uint16_t intrmask;
+uint16_t intrstatus;
+uint16_t doorbell;
+uint8_t *ivshmem_ptr;
+unsigned long ivshmem_offset;
+unsigned int ivshmem_size;
+unsigned long bios_offset;
+unsigned int bios_size;
+target_phys_addr_t base_ctrl;
+int it_shift;
+PCIDevice *pci_dev;
+CharDriverState * chr;
+unsigned long map_addr;
+unsigned long map_end;
+int ivshmem_mmio_io_addr;
+} IVShmemState;
+
+typedef struct PCI_IVShmemState {
+PCIDevice dev;
+IVShmemState ivshmem_state;
+} PCI_IVShmemState;
+
+typedef struct IVShmemDesc {
+char name[1024];
+char * chrdev;
+int size;
+} IVShmemDesc;
+
+
+/* registers for the Inter-VM shared memory device */
+enum ivshmem_registers {
+IntrMask = 0,
+IntrStatus = 16,
+Doorbell = 32
+};
+
+static int num_ivshmem_devices = 0;
+static IVShmemDesc ivshmem_desc;
+
+static void ivshmem_map(PCIDevice *pci_dev, int region_num,
+uint32_t addr, uint32_t size, int type)
+{
+PCI_IVShmemState *d = (PCI_IVShmemState *)pci_dev;
+IVShmemState *s = d-ivshmem_state;
+
+IVSHMEM_DPRINTF(addr = %u size =
Re: [PATCH v2] Shared memory device with interrupt support
On 15-May-09, at 8:45 PM, Kumar, Venkat wrote:
Hi Cam, I have gone through you latest shared memory patch.
I have a few questions and comments.
Comment:-
+if (ivshmem_enabled) {
+ivshmem_init(ivshmem_device);
+ram_size += ivshmem_get_size();
+}
+
In your initial patch this part of the patch is
+if (ivshmem_enabled) {
+ivshmem_init(ivshmem_device);
+phys_ram_size += ivshmem_get_size();
+}
I think the phys_ram_size += ivshmem_get_size(); is correct.
Hi Venkat,
Not with the newer qemu that qemu-kvm uses. The newer patch is for
qemu-kvm, not kvm-userspace. There is no longer a variable named
phys_ram_size in pc.c in qemu-kvm.
Question:-
You are giving the desired virtual address for mmaping the shared
memory object as s-ivshmem_ptr which is phys_ram_base + s-
ivshmem_offset. This desired virtual address is nothing but the
base virtual address of the memory that you are allocating after
incrementing phys_ram_size. So now s-ivshmem_ptr would point to a
new set of memory, which is the shared memory region instead of
memory allocated through qemu_alloc_physram, which means if pages
are allocated for sh-ivshmem_ptr virtual address range then those
pages can never be addressed again. Correct me if my understanding
is wrong.
I don't think so. With the mmap call, I specify MAP_FIXED which
requires that the memory in the shared memory object be mapped to the
address given in the first parameter (s-ivshmem_ptr). If MAP_FIXED
is not specified then mmap would allocate the memory and map on to it,
but with MAP_FIXED it maps onto the already reserved space that
ivshmem_ptr points to and was allocated with qemu_ram_alloc().
I hope that answers your question,
Cam
-Original Message-
From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org]
On Behalf Of Cam Macdonell
Sent: Thursday, May 07, 2009 9:47 PM
To: kvm@vger.kernel.org
Cc: Cam Macdonell
Subject: [PATCH v2] Shared memory device with interrupt support
Support an inter-vm shared memory device that maps a shared-
memory object as a PCI device in the guest. This patch also
supports interrupts between guest by communicating over a unix
domain socket. This patch applies to the qemu-kvm repository.
This device now creates a qemu character device and sends 1-bytes
messages to trigger interrupts. Writes are trigger by writing to
the Doorbell register on the shared memory PCI device. The lower
8-bits of the value written to this register are sent as the 1-byte
message so different meanings of interrupts can be supported.
Interrupts are only supported between 2 VMs currently. One VM must
act as the server by adding server to the command-line argument.
Shared memory devices are created with the following command-line:
-ivhshmem shm object,size in MB,[unix:path][,server]
Interrupts can also be used between host and guest as well by
implementing a listener on the host.
Cam
---
Makefile.target |3 +
hw/ivshmem.c| 421 ++
+
hw/pc.c |6 +
hw/pc.h |3 +
qemu-options.hx | 14 ++
sysemu.h|8 +
vl.c| 14 ++
7 files changed, 469 insertions(+), 0 deletions(-)
create mode 100644 hw/ivshmem.c
diff --git a/Makefile.target b/Makefile.target
index b68a689..3190bba 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -643,6 +643,9 @@ OBJS += pcnet.o
OBJS += rtl8139.o
OBJS += e1000.o
+# Inter-VM PCI shared memory
+OBJS += ivshmem.o
+
# Generic watchdog support and some watchdog devices
OBJS += watchdog.o
OBJS += wdt_ib700.o wdt_i6300esb.o
diff --git a/hw/ivshmem.c b/hw/ivshmem.c
new file mode 100644
index 000..95e2268
--- /dev/null
+++ b/hw/ivshmem.c
@@ -0,0 +1,421 @@
+/*
+ * Inter-VM Shared Memory PCI device.
+ *
+ * Author:
+ * Cam Macdonell c...@cs.ualberta.ca
+ *
+ * Based On: cirrus_vga.c and rtl8139.c
+ *
+ * This code is licensed under the GNU GPL v2.
+ */
+
+#include hw.h
+#include console.h
+#include pc.h
+#include pci.h
+#include sysemu.h
+
+#include qemu-common.h
+#include sys/mman.h
+
+#define PCI_COMMAND_IOACCESS0x0001
+#define PCI_COMMAND_MEMACCESS 0x0002
+#define PCI_COMMAND_BUSMASTER 0x0004
+
+//#define DEBUG_IVSHMEM
+
+#ifdef DEBUG_IVSHMEM
+#define IVSHMEM_DPRINTF(fmt, args...)\
+do {printf(IVSHMEM: fmt, ##args); } while (0)
+#else
+#define IVSHMEM_DPRINTF(fmt, args...)
+#endif
+
+typedef struct IVShmemState {
+uint16_t intrmask;
+uint16_t intrstatus;
+uint16_t doorbell;
+uint8_t *ivshmem_ptr;
+unsigned long ivshmem_offset;
+unsigned int ivshmem_size;
+unsigned long bios_offset;
+unsigned int bios_size;
+target_phys_addr_t base_ctrl;
+int it_shift;
+PCIDevice *pci_dev;
+CharDriverState * chr;
+unsigned long map_addr;
+unsigned long map_end;
+int ivshmem_mmio_io_addr;
+} IVShmemState;
+
Re: [PATCH v2] Shared memory device with interrupt support
On 15-May-09, at 8:54 PM, Kumar, Venkat wrote:
Cam,
A questions on interrupts as well.
What is unix:path that needs to be passed in the argument list?
Can it be any string?
It has to be a valid path on the host. It will create a unix domain
socket on that path.
If my understanding is correct both the VM's who wants to
communicate would gives this path in the command line with one of
them specifying as server.
Exactly, the one with the server in the parameter list will wait for
a connection before booting.
Cam
Thx,
Venkat
Support an inter-vm shared memory device that maps a shared-
memory object
as a PCI device in the guest. This patch also supports interrupts
between
guest by communicating over a unix domain socket. This patch
applies to the
qemu-kvm repository.
This device now creates a qemu character device and sends 1-bytes
messages to
trigger interrupts. Writes are trigger by writing to the Doorbell
register
on the shared memory PCI device. The lower 8-bits of the value
written to this
register are sent as the 1-byte message so different meanings of
interrupts can
be supported.
Interrupts are only supported between 2 VMs currently. One VM must
act as the
server by adding server to the command-line argument. Shared
memory devices
are created with the following command-line:
-ivhshmem shm object,size in MB,[unix:path][,server]
Interrupts can also be used between host and guest as well by
implementing a
listener on the host.
Cam
---
Makefile.target |3 +
hw/ivshmem.c| 421 ++
+
hw/pc.c |6 +
hw/pc.h |3 +
qemu-options.hx | 14 ++
sysemu.h|8 +
vl.c| 14 ++
7 files changed, 469 insertions(+), 0 deletions(-)
create mode 100644 hw/ivshmem.c
diff --git a/Makefile.target b/Makefile.target
index b68a689..3190bba 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -643,6 +643,9 @@ OBJS += pcnet.o
OBJS += rtl8139.o
OBJS += e1000.o
+# Inter-VM PCI shared memory
+OBJS += ivshmem.o
+
# Generic watchdog support and some watchdog devices
OBJS += watchdog.o
OBJS += wdt_ib700.o wdt_i6300esb.o
diff --git a/hw/ivshmem.c b/hw/ivshmem.c
new file mode 100644
index 000..95e2268
--- /dev/null
+++ b/hw/ivshmem.c
@@ -0,0 +1,421 @@
+/*
+ * Inter-VM Shared Memory PCI device.
+ *
+ * Author:
+ * Cam Macdonell c...@cs.ualberta.ca
+ *
+ * Based On: cirrus_vga.c and rtl8139.c
+ *
+ * This code is licensed under the GNU GPL v2.
+ */
+
+#include hw.h
+#include console.h
+#include pc.h
+#include pci.h
+#include sysemu.h
+
+#include qemu-common.h
+#include sys/mman.h
+
+#define PCI_COMMAND_IOACCESS0x0001
+#define PCI_COMMAND_MEMACCESS 0x0002
+#define PCI_COMMAND_BUSMASTER 0x0004
+
+//#define DEBUG_IVSHMEM
+
+#ifdef DEBUG_IVSHMEM
+#define IVSHMEM_DPRINTF(fmt, args...)\
+do {printf(IVSHMEM: fmt, ##args); } while (0)
+#else
+#define IVSHMEM_DPRINTF(fmt, args...)
+#endif
+
+typedef struct IVShmemState {
+uint16_t intrmask;
+uint16_t intrstatus;
+uint16_t doorbell;
+uint8_t *ivshmem_ptr;
+unsigned long ivshmem_offset;
+unsigned int ivshmem_size;
+unsigned long bios_offset;
+unsigned int bios_size;
+target_phys_addr_t base_ctrl;
+int it_shift;
+PCIDevice *pci_dev;
+CharDriverState * chr;
+unsigned long map_addr;
+unsigned long map_end;
+int ivshmem_mmio_io_addr;
+} IVShmemState;
+
+typedef struct PCI_IVShmemState {
+PCIDevice dev;
+IVShmemState ivshmem_state;
+} PCI_IVShmemState;
+
+typedef struct IVShmemDesc {
+char name[1024];
+char * chrdev;
+int size;
+} IVShmemDesc;
+
+
+/* registers for the Inter-VM shared memory device */
+enum ivshmem_registers {
+IntrMask = 0,
+IntrStatus = 16,
+Doorbell = 32
+};
+
+static int num_ivshmem_devices = 0;
+static IVShmemDesc ivshmem_desc;
+
+static void ivshmem_map(PCIDevice *pci_dev, int region_num,
+uint32_t addr, uint32_t size, int type)
+{
+PCI_IVShmemState *d = (PCI_IVShmemState *)pci_dev;
+IVShmemState *s = d-ivshmem_state;
+
+IVSHMEM_DPRINTF(addr = %u size = %u\n, addr, size);
+cpu_register_physical_memory(addr, s-ivshmem_size, s-
ivshmem_offset);
+
+}
+
+void ivshmem_init(const char * optarg) {
+
+char * temp;
+char * ivshmem_sz;
+int size;
+
+num_ivshmem_devices++;
+
+/* currently we only support 1 device */
+if (num_ivshmem_devices MAX_IVSHMEM_DEVICES) {
+return;
+}
+
+temp = strdup(optarg);
+snprintf(ivshmem_desc.name, 1024, /%s, strsep(temp,,));
+ivshmem_sz=strsep(temp,,);
+if (ivshmem_sz != NULL){
+size = atol(ivshmem_sz);
+} else {
+size = -1;
+}
+
+ivshmem_desc.chrdev = strsep(temp,\0);
+
+if ( size == -1) {
+ivshmem_desc.size = TARGET_PAGE_SIZE;
+} else {
+ivshmem_desc.size = size*1024*1024;
+}
+
