Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-17 Thread David Stevens
> > > Of course only virtio drivers would try step (2), other drivers (when > > > sharing buffers between intel gvt device and virtio-gpu for example) > > > would go straight to (3). > > > > For virtio-gpu as it is today, it's not clear to me that they're > > equivalent. As I read it, the virtio-gp

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-16 Thread Gerd Hoffmann
Hi, > > Of course only virtio drivers would try step (2), other drivers (when > > sharing buffers between intel gvt device and virtio-gpu for example) > > would go straight to (3). > > For virtio-gpu as it is today, it's not clear to me that they're > equivalent. As I read it, the virtio-gpu sp

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-12 Thread David Stevens
> > > Without buffer sharing support the driver importing a virtio-gpu dma-buf > > > can send the buffer scatter list to the host. So both virtio-gpu and > > > the other device would actually access the same guest pages, but they > > > are not aware that the buffer is shared between devices. > > >

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-12 Thread Gerd Hoffmann
On Thu, Dec 12, 2019 at 09:26:32PM +0900, David Stevens wrote: > > > > Second I think it is a bad idea > > > > from the security point of view. When explicitly exporting buffers it > > > > is easy to restrict access to the actual exports. > > > > > > Restricting access to actual exports could perh

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-12 Thread David Stevens
> > > Second I think it is a bad idea > > > from the security point of view. When explicitly exporting buffers it > > > is easy to restrict access to the actual exports. > > > > Restricting access to actual exports could perhaps help catch bugs. > > However, I don't think it provides any security

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-12 Thread Gerd Hoffmann
Hi, > > First the addressing is non-trivial, especially with the "transport > > specific device address" in the tuple. > > There is complexity here, but I think it would also be present in the > buffer sharing device case. With a buffer sharing device, the same > identifying information would n

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-11 Thread David Stevens
> First the addressing is non-trivial, especially with the "transport > specific device address" in the tuple. There is complexity here, but I think it would also be present in the buffer sharing device case. With a buffer sharing device, the same identifying information would need to be provided

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-11 Thread Enrico Granata
On Wed, Dec 11, 2019 at 1:26 AM Gerd Hoffmann wrote: > > Hi, > > > None of the proposals directly address the use case of sharing host > > allocated buffers between devices, but I think they can be extended to > > support it. Host buffers can be identified by the following tuple: > > (transport

Re: guest / host buffer sharing ...

2019-12-11 Thread Gerd Hoffmann
Hi, > None of the proposals directly address the use case of sharing host > allocated buffers between devices, but I think they can be extended to > support it. Host buffers can be identified by the following tuple: > (transport type enum, transport specific device address, shmid, > offset). I t

Re: guest / host buffer sharing ...

2019-12-10 Thread David Stevens
There are three issues being discussed here that aren't being clearly delineated: sharing guest allocated memory with the host, sharing host allocated memory with the guest, and sharing buffers between devices. Right now, guest allocated memory can be shared with the host through the virtqueues or

Re: guest / host buffer sharing ...

2019-12-04 Thread Dylan Reid
On Thu, Nov 21, 2019 at 4:59 PM Tomasz Figa wrote: > > On Thu, Nov 21, 2019 at 6:41 AM Geoffrey McRae wrote: > > > > > > > > On 2019-11-20 23:13, Tomasz Figa wrote: > > > Hi Geoffrey, > > > > > > On Thu, Nov 7, 2019 at 7:28 AM Geoffrey McRae > > > wrote: > > >> > > >> > > >> > > >> On 2019-11-06

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-26 Thread Gerd Hoffmann
> > I'm not convinced this is useful for audio ... > > > > I basically see two modes of operation which are useful: > > > > (1) send audio data via virtqueue. > > (2) map host audio buffers into the guest address space. > > > > The audio driver api (i.e. alsa) typically allows to mmap() the

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-25 Thread Liam Girdwood
On Wed, 2019-11-20 at 10:53 +0100, Gerd Hoffmann wrote: > Hi, > > > > > DSP FW has no access to userspace so we would need some > > > > additional > > > > API > > > > on top of DMA_BUF_SET_NAME etc to get physical hardware pages ? > > > > > > The dma-buf api currently can share guest memory sg-

Re: guest / host buffer sharing ...

2019-11-20 Thread Tomasz Figa
On Thu, Nov 21, 2019 at 6:41 AM Geoffrey McRae wrote: > > > > On 2019-11-20 23:13, Tomasz Figa wrote: > > Hi Geoffrey, > > > > On Thu, Nov 7, 2019 at 7:28 AM Geoffrey McRae > > wrote: > >> > >> > >> > >> On 2019-11-06 23:41, Gerd Hoffmann wrote: > >> > On Wed, Nov 06, 2019 at 05:36:22PM +0900, Da

Re: guest / host buffer sharing ...

2019-11-20 Thread Geoffrey McRae
On 2019-11-20 23:13, Tomasz Figa wrote: Hi Geoffrey, On Thu, Nov 7, 2019 at 7:28 AM Geoffrey McRae wrote: On 2019-11-06 23:41, Gerd Hoffmann wrote: > On Wed, Nov 06, 2019 at 05:36:22PM +0900, David Stevens wrote: >> > (1) The virtio device >> > = >> > >> > Has a sing

Re: guest / host buffer sharing ...

2019-11-20 Thread Tomasz Figa
Hi Geoffrey, On Thu, Nov 7, 2019 at 7:28 AM Geoffrey McRae wrote: > > > > On 2019-11-06 23:41, Gerd Hoffmann wrote: > > On Wed, Nov 06, 2019 at 05:36:22PM +0900, David Stevens wrote: > >> > (1) The virtio device > >> > = > >> > > >> > Has a single virtio queue, so the guest ca

Re: guest / host buffer sharing ...

2019-11-20 Thread Tomasz Figa
On Wed, Nov 6, 2019 at 6:51 PM Gerd Hoffmann wrote: > > Hi, > > > > Reason is: Meanwhile I'm wondering whenever "just use virtio-gpu > > > resources" is really a good answer for all the different use cases > > > we have collected over time. Maybe it is better to have a dedicated > > > buffer s

Re: guest / host buffer sharing ...

2019-11-20 Thread Tomasz Figa
Hi Frank, On Fri, Nov 8, 2019 at 12:10 AM Frank Yang wrote: > > So I'm not really sure why people are having issues sharing buffers that live > on the GPU. Doesn't that show up as some integer ID on the host, and some > $GuestFramework (dmabuf, gralloc) ID on the guest, and it all works out due

Re: guest / host buffer sharing ...

2019-11-20 Thread Tomasz Figa
Hi Stefan, On Mon, Nov 18, 2019 at 7:21 PM Stefan Hajnoczi wrote: > > On Sat, Nov 9, 2019 at 3:12 PM Tomasz Figa wrote: > > > > On Sat, Nov 9, 2019 at 9:08 PM Stefan Hajnoczi wrote: > > > > > > On Sat, Nov 9, 2019 at 12:17 PM Tomasz Figa wrote: > > > > On Sat, Nov 9, 2019 at 7:12 PM Stefan Haj

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-20 Thread Gerd Hoffmann
Hi, > > > DSP FW has no access to userspace so we would need some additional > > > API > > > on top of DMA_BUF_SET_NAME etc to get physical hardware pages ? > > > > The dma-buf api currently can share guest memory sg-lists. > > Ok, IIUC buffers can either be shared using the GPU proposed APIs

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-19 Thread Gurchetan Singh
On Tue, Nov 19, 2019 at 7:31 AM Liam Girdwood wrote: > > On Tue, 2019-11-12 at 14:55 -0800, Gurchetan Singh wrote: > > On Tue, Nov 12, 2019 at 5:56 AM Liam Girdwood > > wrote: > > > > > > On Mon, 2019-11-11 at 16:54 -0800, Gurchetan Singh wrote: > > > > On Tue, Nov 5, 2019 at 2:55 AM Gerd Hoffman

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-19 Thread Liam Girdwood
On Tue, 2019-11-12 at 14:55 -0800, Gurchetan Singh wrote: > On Tue, Nov 12, 2019 at 5:56 AM Liam Girdwood > wrote: > > > > On Mon, 2019-11-11 at 16:54 -0800, Gurchetan Singh wrote: > > > On Tue, Nov 5, 2019 at 2:55 AM Gerd Hoffmann > > > wrote: > > > > Each buffer also has some properties to car

Re: guest / host buffer sharing ...

2019-11-18 Thread Stefan Hajnoczi
On Sat, Nov 9, 2019 at 3:12 PM Tomasz Figa wrote: > > On Sat, Nov 9, 2019 at 9:08 PM Stefan Hajnoczi wrote: > > > > On Sat, Nov 9, 2019 at 12:17 PM Tomasz Figa wrote: > > > On Sat, Nov 9, 2019 at 7:12 PM Stefan Hajnoczi wrote: > > > > On Sat, Nov 9, 2019 at 2:41 AM Stéphane Marchesin > > > >

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-12 Thread Gurchetan Singh
On Tue, Nov 12, 2019 at 5:56 AM Liam Girdwood wrote: > > On Mon, 2019-11-11 at 16:54 -0800, Gurchetan Singh wrote: > > On Tue, Nov 5, 2019 at 2:55 AM Gerd Hoffmann > > wrote: > > > Each buffer also has some properties to carry metadata, some fixed > > > (id, size, application), but > > > also all

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-12 Thread Liam Girdwood
On Mon, 2019-11-11 at 16:54 -0800, Gurchetan Singh wrote: > On Tue, Nov 5, 2019 at 2:55 AM Gerd Hoffmann > wrote: > > Each buffer also has some properties to carry metadata, some fixed > > (id, size, application), but > > also allow free form (name = value, framebuffers would have > > width/height

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-11 Thread Gurchetan Singh
On Tue, Nov 5, 2019 at 2:55 AM Gerd Hoffmann wrote: > Each buffer also has some properties to carry metadata, some fixed (id, size, > application), but > also allow free form (name = value, framebuffers would have > width/height/stride/format for example). Sounds a lot like the recently added DM

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-11 Thread Liam Girdwood
On Mon, 2019-11-11 at 12:04 +0900, David Stevens wrote: > Having a centralized buffer allocator device is one way to deal with > sharing buffers, since it gives a definitive buffer identifier that > can be used by all drivers/devices to refer to the buffer. That being > said, I think the device as

Re: guest / host buffer sharing ...

2019-11-10 Thread David Stevens
> My question would be "what is the actual problem you are trying to > solve?". One problem that needs to be solved is sharing buffers between devices. With the out-of-tree Wayland device, to share virtio-gpu buffers we've been using the virtio resource id. However, that approach isn't necessarily

Re: guest / host buffer sharing ...

2019-11-09 Thread Tomasz Figa
On Sat, Nov 9, 2019 at 9:08 PM Stefan Hajnoczi wrote: > > On Sat, Nov 9, 2019 at 12:17 PM Tomasz Figa wrote: > > On Sat, Nov 9, 2019 at 7:12 PM Stefan Hajnoczi wrote: > > > On Sat, Nov 9, 2019 at 2:41 AM Stéphane Marchesin > > > wrote: > > > > On Thu, Nov 7, 2019 at 11:35 PM Stefan Hajnoczi

Re: guest / host buffer sharing ...

2019-11-09 Thread Tomasz Figa
On Sat, Nov 9, 2019 at 7:12 PM Stefan Hajnoczi wrote: > > On Sat, Nov 9, 2019 at 2:41 AM Stéphane Marchesin > wrote: > > > > On Thu, Nov 7, 2019 at 11:35 PM Stefan Hajnoczi wrote: > > > > > > On Fri, Nov 8, 2019 at 8:22 AM Gerd Hoffmann wrote: > > > > > > Adding a list of common properties to

Re: guest / host buffer sharing ...

2019-11-09 Thread Stefan Hajnoczi
On Sat, Nov 9, 2019 at 12:17 PM Tomasz Figa wrote: > On Sat, Nov 9, 2019 at 7:12 PM Stefan Hajnoczi wrote: > > On Sat, Nov 9, 2019 at 2:41 AM Stéphane Marchesin > > wrote: > > > On Thu, Nov 7, 2019 at 11:35 PM Stefan Hajnoczi > > > wrote: > > > > On Fri, Nov 8, 2019 at 8:22 AM Gerd Hoffmann

Re: guest / host buffer sharing ...

2019-11-09 Thread Stefan Hajnoczi
On Sat, Nov 9, 2019 at 2:41 AM Stéphane Marchesin wrote: > > On Thu, Nov 7, 2019 at 11:35 PM Stefan Hajnoczi wrote: > > > > On Fri, Nov 8, 2019 at 8:22 AM Gerd Hoffmann wrote: > > > > > Adding a list of common properties to the spec certainly makes sense, > > > > > so everybody uses the same nam

Re: guest / host buffer sharing ...

2019-11-08 Thread Stéphane Marchesin
On Thu, Nov 7, 2019 at 11:35 PM Stefan Hajnoczi wrote: > > On Fri, Nov 8, 2019 at 8:22 AM Gerd Hoffmann wrote: > > > > Adding a list of common properties to the spec certainly makes sense, > > > > so everybody uses the same names. Adding struct-ed properties for > > > > common use cases might be

Re: guest / host buffer sharing ...

2019-11-07 Thread Stefan Hajnoczi
On Fri, Nov 8, 2019 at 8:22 AM Gerd Hoffmann wrote: > > > Adding a list of common properties to the spec certainly makes sense, > > > so everybody uses the same names. Adding struct-ed properties for > > > common use cases might be useful too. > > > > Why not define VIRTIO devices for wayland and

Re: guest / host buffer sharing ...

2019-11-07 Thread Gerd Hoffmann
Hi, > > Adding a list of common properties to the spec certainly makes sense, > > so everybody uses the same names. Adding struct-ed properties for > > common use cases might be useful too. > > Why not define VIRTIO devices for wayland and friends? There is an out-of-tree implementation of th

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-07 Thread Gerd Hoffmann
On Thu, Nov 07, 2019 at 11:16:18AM +, Dr. David Alan Gilbert wrote: > * Gerd Hoffmann (kra...@redhat.com) wrote: > > Hi, > > > > > > This is not about host memory, buffers are in guest ram, everything else > > > > would make sharing those buffers between drivers inside the guest (as > > > >

Re: guest / host buffer sharing ...

2019-11-07 Thread Frank Yang
So I'm not really sure why people are having issues sharing buffers that live on the GPU. Doesn't that show up as some integer ID on the host, and some $GuestFramework (dmabuf, gralloc) ID on the guest, and it all works out due to maintaining the correspondence in your particular stack of virtual d

Re: guest / host buffer sharing ...

2019-11-07 Thread Stefan Hajnoczi
On Wed, Nov 6, 2019 at 1:50 PM Gerd Hoffmann wrote: > > In the graphics buffer sharing use case, how does the other side > > determine how to interpret this data? > > The idea is to have free form properties (name=value, with value being > a string) for that kind of metadata. > > > Shouldn't there

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-07 Thread Dr. David Alan Gilbert
* Gerd Hoffmann (kra...@redhat.com) wrote: > Hi, > > > > This is not about host memory, buffers are in guest ram, everything else > > > would make sharing those buffers between drivers inside the guest (as > > > dma-buf) quite difficult. > > > > Given it's just guest memory, can the guest just

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-07 Thread Gerd Hoffmann
Hi, > > This is not about host memory, buffers are in guest ram, everything else > > would make sharing those buffers between drivers inside the guest (as > > dma-buf) quite difficult. > > Given it's just guest memory, can the guest just have a virt queue on > which it places pointers to the me

Re: guest / host buffer sharing ...

2019-11-06 Thread Gerd Hoffmann
> On 2019-11-06 23:41, Gerd Hoffmann wrote: > > On Wed, Nov 06, 2019 at 05:36:22PM +0900, David Stevens wrote: > > > > (1) The virtio device > > > > = > > > > > > > > Has a single virtio queue, so the guest can send commands to register > > > > and unregister buffers. Buffers a

Re: guest / host buffer sharing ...

2019-11-06 Thread Geoffrey McRae
On 2019-11-06 23:41, Gerd Hoffmann wrote: On Wed, Nov 06, 2019 at 05:36:22PM +0900, David Stevens wrote: > (1) The virtio device > = > > Has a single virtio queue, so the guest can send commands to register > and unregister buffers. Buffers are allocated in guest ram. Ea

Re: guest / host buffer sharing ...

2019-11-06 Thread David Stevens
> (1) The virtio device > = > > Has a single virtio queue, so the guest can send commands to register > and unregister buffers. Buffers are allocated in guest ram. Each buffer > has a list of memory ranges for the data. Each buffer also has some Allocating from guest ram woul

Re: guest / host buffer sharing ...

2019-11-06 Thread Gerd Hoffmann
Hi, > In the graphics buffer sharing use case, how does the other side > determine how to interpret this data? The idea is to have free form properties (name=value, with value being a string) for that kind of metadata. > Shouldn't there be a VIRTIO > device spec for the messaging so compatible

Re: guest / host buffer sharing ...

2019-11-06 Thread Gerd Hoffmann
On Wed, Nov 06, 2019 at 05:36:22PM +0900, David Stevens wrote: > > (1) The virtio device > > = > > > > Has a single virtio queue, so the guest can send commands to register > > and unregister buffers. Buffers are allocated in guest ram. Each buffer > > has a list of memory ran

Re: guest / host buffer sharing ...

2019-11-06 Thread Stefan Hajnoczi
On Wed, Nov 6, 2019 at 10:51 AM Gerd Hoffmann wrote: > > > Reason is: Meanwhile I'm wondering whenever "just use virtio-gpu > > > resources" is really a good answer for all the different use cases > > > we have collected over time. Maybe it is better to have a dedicated > > > buffer sharing virt

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-11-06 Thread Dr. David Alan Gilbert
* Gerd Hoffmann (kra...@redhat.com) wrote: > Hi, > > > > Reason is: Meanwhile I'm wondering whenever "just use virtio-gpu > > > resources" is really a good answer for all the different use cases > > > we have collected over time. Maybe it is better to have a dedicated > > > buffer sharing virt

Re: guest / host buffer sharing ...

2019-11-06 Thread Gerd Hoffmann
Hi, > > Reason is: Meanwhile I'm wondering whenever "just use virtio-gpu > > resources" is really a good answer for all the different use cases > > we have collected over time. Maybe it is better to have a dedicated > > buffer sharing virtio device? Here is the rough idea: > > My concern is

Re: guest / host buffer sharing ...

2019-11-06 Thread Stefan Hajnoczi
On Tue, Nov 05, 2019 at 11:54:56AM +0100, Gerd Hoffmann wrote: > The issue of sharing buffers between guests and hosts keeps poping > up again and again in different contexts. Most recently here: > > https://www.mail-archive.com/qemu-devel@nongnu.org/msg656685.html > > So, I'm grabbing the recip

Re: guest / host buffer sharing ...

2019-11-05 Thread Gerd Hoffmann
> > (1) The virtio device > > = > > > > Has a single virtio queue, so the guest can send commands to register > > and unregister buffers. Buffers are allocated in guest ram. Each > > buffer > > has a list of memory ranges for the data. Each buffer also has some > > propertie

Re: guest / host buffer sharing ...

2019-11-05 Thread Geoffrey McRae
Hi Gerd. On 2019-11-05 21:54, Gerd Hoffmann wrote: Hi folks, The issue of sharing buffers between guests and hosts keeps poping up again and again in different contexts. Most recently here: https://www.mail-archive.com/qemu-devel@nongnu.org/msg656685.html So, I'm grabbing the recipient list