Re: [PATCH 1/2] drm/shmem: add support for per object dma api operations
On Mon, Jun 15, 2020 at 12:21 AM Gerd Hoffmann wrote: > > On Fri, Jun 12, 2020 at 11:54:54AM -0700, Gurchetan Singh wrote: > > > Plus, I just realized the virtio dma ops and ops used by drm shmem are > > different, so virtio would have to unconditionally have to skip the > > shmem path. Perhaps the best policy is to revert d323bb44e4d2? > > Reverting d323bb44e4d2 should work given that virtio-gpu doesn't support > dma-buf imports, but when doing so please add a comment to the code > explaining why virtio-gpu handles this different than everybody else. Done -- sent out "drm/virtio: Revert "drm/virtio: Call the right shmem helpers" ". > > thanks, > Gerd > ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH 1/2] drm/shmem: add support for per object dma api operations
On Fri, Jun 12, 2020 at 11:54:54AM -0700, Gurchetan Singh wrote: > Plus, I just realized the virtio dma ops and ops used by drm shmem are > different, so virtio would have to unconditionally have to skip the > shmem path. Perhaps the best policy is to revert d323bb44e4d2? Reverting d323bb44e4d2 should work given that virtio-gpu doesn't support dma-buf imports, but when doing so please add a comment to the code explaining why virtio-gpu handles this different than everybody else. thanks, Gerd ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH 1/2] drm/shmem: add support for per object dma api operations
Hi Am 12.06.20 um 20:54 schrieb Gurchetan Singh: > On Fri, Jun 12, 2020 at 3:17 AM Gerd Hoffmann wrote: >> >> On Fri, Jun 12, 2020 at 11:47:55AM +0200, Thomas Zimmermann wrote: >>> Hi >>> >>> Am 12.06.20 um 03:36 schrieb Gurchetan Singh: This is useful for the next patch. Also, should we only unmap the amount entries that we mapped with the dma-api? >>> >>> It looks like you're moving virtio code into shmem. >> >> Well, not really. >> >> virtio has -- for historical reasons -- the oddity that it may or may >> not need to dma_map resources, depending on device configuration. >> Initially virtio went with "it's just a vm, lets simply operate on >> physical ram addresses". That shortcut turned out to be a bad idea >> later on, especially with the arrival of iommu emulation support in >> qemu. But we couldn't just scratch it for backward compatibility >> reasons. See virtio_has_iommu_quirk(). >> >> This just allows to enable/disable dma_map, I guess to fix some fallout >> from recent shmem helper changes > > Yes, the main goal is to fix the double free. > > diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c > b/drivers/gpu/drm/virtio/virtgpu_object.c > index 346cef5ce251..2f7b6cd25a4b 100644 > --- a/drivers/gpu/drm/virtio/virtgpu_object.c > +++ b/drivers/gpu/drm/virtio/virtgpu_object.c > @@ -78,7 +78,6 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo) > shmem->mapped = 0; > } > > - sg_free_table(shmem->pages); > shmem->pages = NULL; > drm_gem_shmem_unpin(&bo->base.base); > } > > Doing only that on it's own causes log spam though > > [ 10.368385] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 > bytes), total 0 (slots), used 0 (slots) > [ 10.384957] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 > bytes), total 0 (slots), used 0 (slots) > [ 10.389920] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 > bytes), total 0 (slots), used 0 (slots) > [ 10.396859] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 > bytes), total 0 (slots), used 0 (slots) > [ 10.401954] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 > bytes), total 0 (slots), used 0 (slots) > [ 10.406694] virtio_gpu virtio5: swiotlb buffer is full (sz: 8192 > bytes), total 0 (slots), used 0 (slots) > [ 10.495744] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 > bytes), total 0 (slots), used 0 (slots) > > Plus, I just realized the virtio dma ops and ops used by drm shmem are > different, so virtio would have to unconditionally have to skip the > shmem path. Perhaps the best policy is to revert d323bb44e4d2? Can you fork the shmem code into virtio and modify it according to your needs? I know that code splitting is unpopular, but at least it's a clean solution. For some GEM object functions, you might still be able to share code with shmem helpers. Best regards Thomas > >> (Gurchetan, that kind of stuff should >> be mentioned in cover letter and commit messages). > > Good tip. > >> >> I'm not sure virtio actually needs that patch though. I *think* doing >> the dma_map+dma_unmap unconditionally, but then ignore the result in >> case we don't need it should work. And it shouldn't be a horrible >> performance hit either, in case we don't have a (virtual) iommu in the >> VM dma mapping is essentially a nop ... >> >> take care, >> Gerd >> -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer signature.asc Description: OpenPGP digital signature ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH 1/2] drm/shmem: add support for per object dma api operations
On Fri, Jun 12, 2020 at 3:17 AM Gerd Hoffmann wrote: > > On Fri, Jun 12, 2020 at 11:47:55AM +0200, Thomas Zimmermann wrote: > > Hi > > > > Am 12.06.20 um 03:36 schrieb Gurchetan Singh: > > > This is useful for the next patch. Also, should we only unmap the > > > amount entries that we mapped with the dma-api? > > > > It looks like you're moving virtio code into shmem. > > Well, not really. > > virtio has -- for historical reasons -- the oddity that it may or may > not need to dma_map resources, depending on device configuration. > Initially virtio went with "it's just a vm, lets simply operate on > physical ram addresses". That shortcut turned out to be a bad idea > later on, especially with the arrival of iommu emulation support in > qemu. But we couldn't just scratch it for backward compatibility > reasons. See virtio_has_iommu_quirk(). > > This just allows to enable/disable dma_map, I guess to fix some fallout > from recent shmem helper changes Yes, the main goal is to fix the double free. diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c index 346cef5ce251..2f7b6cd25a4b 100644 --- a/drivers/gpu/drm/virtio/virtgpu_object.c +++ b/drivers/gpu/drm/virtio/virtgpu_object.c @@ -78,7 +78,6 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo) shmem->mapped = 0; } - sg_free_table(shmem->pages); shmem->pages = NULL; drm_gem_shmem_unpin(&bo->base.base); } Doing only that on it's own causes log spam though [ 10.368385] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 bytes), total 0 (slots), used 0 (slots) [ 10.384957] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 bytes), total 0 (slots), used 0 (slots) [ 10.389920] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 bytes), total 0 (slots), used 0 (slots) [ 10.396859] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 bytes), total 0 (slots), used 0 (slots) [ 10.401954] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 bytes), total 0 (slots), used 0 (slots) [ 10.406694] virtio_gpu virtio5: swiotlb buffer is full (sz: 8192 bytes), total 0 (slots), used 0 (slots) [ 10.495744] virtio_gpu virtio5: swiotlb buffer is full (sz: 4096 bytes), total 0 (slots), used 0 (slots) Plus, I just realized the virtio dma ops and ops used by drm shmem are different, so virtio would have to unconditionally have to skip the shmem path. Perhaps the best policy is to revert d323bb44e4d2? > (Gurchetan, that kind of stuff should > be mentioned in cover letter and commit messages). Good tip. > > I'm not sure virtio actually needs that patch though. I *think* doing > the dma_map+dma_unmap unconditionally, but then ignore the result in > case we don't need it should work. And it shouldn't be a horrible > performance hit either, in case we don't have a (virtual) iommu in the > VM dma mapping is essentially a nop ... > > take care, > Gerd > ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH 1/2] drm/shmem: add support for per object dma api operations
On Fri, Jun 12, 2020 at 12:16 PM Gerd Hoffmann wrote: > > On Fri, Jun 12, 2020 at 11:47:55AM +0200, Thomas Zimmermann wrote: > > Hi > > > > Am 12.06.20 um 03:36 schrieb Gurchetan Singh: > > > This is useful for the next patch. Also, should we only unmap the > > > amount entries that we mapped with the dma-api? > > > > It looks like you're moving virtio code into shmem. > > Well, not really. > > virtio has -- for historical reasons -- the oddity that it may or may > not need to dma_map resources, depending on device configuration. > Initially virtio went with "it's just a vm, lets simply operate on > physical ram addresses". That shortcut turned out to be a bad idea > later on, especially with the arrival of iommu emulation support in > qemu. But we couldn't just scratch it for backward compatibility > reasons. See virtio_has_iommu_quirk(). > > This just allows to enable/disable dma_map, I guess to fix some fallout > from recent shmem helper changes (Gurchetan, that kind of stuff should > be mentioned in cover letter and commit messages). > > I'm not sure virtio actually needs that patch though. I *think* doing > the dma_map+dma_unmap unconditionally, but then ignore the result in > case we don't need it should work. And it shouldn't be a horrible > performance hit either, in case we don't have a (virtual) iommu in the > VM dma mapping is essentially a nop ... Yeah that sounds a lot more like a clean solution, instead of adding random flags and stuff all over helpers for each edge-case. The sgtable still has the struct pages, so just picking the right one in virtio code seems a lot cleaner separation of concerns. -Daniel > > take care, > Gerd > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH 1/2] drm/shmem: add support for per object dma api operations
On Fri, Jun 12, 2020 at 11:47:55AM +0200, Thomas Zimmermann wrote: > Hi > > Am 12.06.20 um 03:36 schrieb Gurchetan Singh: > > This is useful for the next patch. Also, should we only unmap the > > amount entries that we mapped with the dma-api? > > It looks like you're moving virtio code into shmem. Well, not really. virtio has -- for historical reasons -- the oddity that it may or may not need to dma_map resources, depending on device configuration. Initially virtio went with "it's just a vm, lets simply operate on physical ram addresses". That shortcut turned out to be a bad idea later on, especially with the arrival of iommu emulation support in qemu. But we couldn't just scratch it for backward compatibility reasons. See virtio_has_iommu_quirk(). This just allows to enable/disable dma_map, I guess to fix some fallout from recent shmem helper changes (Gurchetan, that kind of stuff should be mentioned in cover letter and commit messages). I'm not sure virtio actually needs that patch though. I *think* doing the dma_map+dma_unmap unconditionally, but then ignore the result in case we don't need it should work. And it shouldn't be a horrible performance hit either, in case we don't have a (virtual) iommu in the VM dma mapping is essentially a nop ... take care, Gerd ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH 1/2] drm/shmem: add support for per object dma api operations
Hi Am 12.06.20 um 03:36 schrieb Gurchetan Singh: > This is useful for the next patch. Also, should we only unmap the > amount entries that we mapped with the dma-api? It looks like you're moving virtio code into shmem. It would be nice to have a cover letter explaining the series. > > Signed-off-by: Gurchetan Singh > --- > drivers/gpu/drm/drm_gem_shmem_helper.c | 16 +++- > include/drm/drm_gem_shmem_helper.h | 10 ++ > 2 files changed, 21 insertions(+), 5 deletions(-) > > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c > b/drivers/gpu/drm/drm_gem_shmem_helper.c > index 0a7e3b664bc2..d439074ad7b5 100644 > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c > @@ -124,8 +124,10 @@ void drm_gem_shmem_free_object(struct drm_gem_object > *obj) > drm_prime_gem_destroy(obj, shmem->sgt); > } else { > if (shmem->sgt) { > - dma_unmap_sg(obj->dev->dev, shmem->sgt->sgl, > - shmem->sgt->nents, DMA_BIDIRECTIONAL); > + if (!shmem->skip_dma_api) > + dma_unmap_sg(obj->dev->dev, shmem->sgt->sgl, > + shmem->dma_map_count, > + DMA_BIDIRECTIONAL); > sg_free_table(shmem->sgt); > kfree(shmem->sgt); > } > @@ -422,8 +424,9 @@ void drm_gem_shmem_purge_locked(struct drm_gem_object > *obj) > > WARN_ON(!drm_gem_shmem_is_purgeable(shmem)); > > - dma_unmap_sg(obj->dev->dev, shmem->sgt->sgl, > - shmem->sgt->nents, DMA_BIDIRECTIONAL); > + if (!shmem->skip_dma_api) > + dma_unmap_sg(obj->dev->dev, shmem->sgt->sgl, > + shmem->dma_map_count, DMA_BIDIRECTIONAL); > sg_free_table(shmem->sgt); > kfree(shmem->sgt); > shmem->sgt = NULL; > @@ -695,7 +698,10 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct > drm_gem_object *obj) > goto err_put_pages; > } > /* Map the pages for use by the h/w. */ > - dma_map_sg(obj->dev->dev, sgt->sgl, sgt->nents, DMA_BIDIRECTIONAL); > + if (!shmem->skip_dma_api) > + shmem->dma_map_count = dma_map_sg(obj->dev->dev, sgt->sgl, > + sgt->nents, > + DMA_BIDIRECTIONAL); > > shmem->sgt = sgt; > > diff --git a/include/drm/drm_gem_shmem_helper.h > b/include/drm/drm_gem_shmem_helper.h > index 5381f0c8cf6f..2669d87cbfdd 100644 > --- a/include/drm/drm_gem_shmem_helper.h > +++ b/include/drm/drm_gem_shmem_helper.h > @@ -101,6 +101,16 @@ struct drm_gem_shmem_object { >* @map_cached: map object cached (instead of using writecombine). >*/ > bool map_cached; > + > + /** > + * @skip_dma_api: skip using dma api in certain places. > + */ > + bool skip_dma_api; This looks like an under-documented workaround for something. > + > + /** > + * @skip_dma_api: number of pages mapped by dma-api. > + */ > + bool dma_map_count; The documentation comment doesn't match the field name. Best regards Thomas > }; > > #define to_drm_gem_shmem_obj(obj) \ > -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer signature.asc Description: OpenPGP digital signature ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
[PATCH 1/2] drm/shmem: add support for per object dma api operations
This is useful for the next patch. Also, should we only unmap the amount entries that we mapped with the dma-api? Signed-off-by: Gurchetan Singh --- drivers/gpu/drm/drm_gem_shmem_helper.c | 16 +++- include/drm/drm_gem_shmem_helper.h | 10 ++ 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c index 0a7e3b664bc2..d439074ad7b5 100644 --- a/drivers/gpu/drm/drm_gem_shmem_helper.c +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c @@ -124,8 +124,10 @@ void drm_gem_shmem_free_object(struct drm_gem_object *obj) drm_prime_gem_destroy(obj, shmem->sgt); } else { if (shmem->sgt) { - dma_unmap_sg(obj->dev->dev, shmem->sgt->sgl, -shmem->sgt->nents, DMA_BIDIRECTIONAL); + if (!shmem->skip_dma_api) + dma_unmap_sg(obj->dev->dev, shmem->sgt->sgl, +shmem->dma_map_count, +DMA_BIDIRECTIONAL); sg_free_table(shmem->sgt); kfree(shmem->sgt); } @@ -422,8 +424,9 @@ void drm_gem_shmem_purge_locked(struct drm_gem_object *obj) WARN_ON(!drm_gem_shmem_is_purgeable(shmem)); - dma_unmap_sg(obj->dev->dev, shmem->sgt->sgl, -shmem->sgt->nents, DMA_BIDIRECTIONAL); + if (!shmem->skip_dma_api) + dma_unmap_sg(obj->dev->dev, shmem->sgt->sgl, +shmem->dma_map_count, DMA_BIDIRECTIONAL); sg_free_table(shmem->sgt); kfree(shmem->sgt); shmem->sgt = NULL; @@ -695,7 +698,10 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_object *obj) goto err_put_pages; } /* Map the pages for use by the h/w. */ - dma_map_sg(obj->dev->dev, sgt->sgl, sgt->nents, DMA_BIDIRECTIONAL); + if (!shmem->skip_dma_api) + shmem->dma_map_count = dma_map_sg(obj->dev->dev, sgt->sgl, + sgt->nents, + DMA_BIDIRECTIONAL); shmem->sgt = sgt; diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h index 5381f0c8cf6f..2669d87cbfdd 100644 --- a/include/drm/drm_gem_shmem_helper.h +++ b/include/drm/drm_gem_shmem_helper.h @@ -101,6 +101,16 @@ struct drm_gem_shmem_object { * @map_cached: map object cached (instead of using writecombine). */ bool map_cached; + + /** +* @skip_dma_api: skip using dma api in certain places. +*/ + bool skip_dma_api; + + /** +* @skip_dma_api: number of pages mapped by dma-api. +*/ + bool dma_map_count; }; #define to_drm_gem_shmem_obj(obj) \ -- 2.25.1 ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel