Re: [PATCH] drm/panfrost: Fix dma_resv deadlock at drm object pin time
On 4/21/24 19:39, Adrián Larumbe wrote:
> When Panfrost must pin an object that is being prepared a dma-buf
> attachment for on behalf of another driver, the core drm gem object pinning
> code already takes a lock on the object's dma reservation.
>
> However, Panfrost GEM object's pinning callback would eventually try taking
> the lock on the same dma reservation when delegating pinning of the object
> onto the shmem subsystem, which led to a deadlock.
>
> This can be shown by enabling CONFIG_DEBUG_WW_MUTEX_SLOWPATH, which throws
> the following recursive locking situation:
>
> weston/3440 is trying to acquire lock:
> 00e235a0 (reservation_ww_class_mutex){+.+.}-{3:3}, at:
> drm_gem_shmem_pin+0x34/0xb8 [drm_shmem_helper]
> but task is already holding lock:
> 00e235a0 (reservation_ww_class_mutex){+.+.}-{3:3}, at:
> drm_gem_pin+0x2c/0x80 [drm]
>
> Fix it by assuming the object's reservation had already been locked by the
> time we reach panfrost_gem_pin.
>
> Cc: Thomas Zimmermann
> Cc: Dmitry Osipenko
> Cc: Boris Brezillon
> Cc: Steven Price
> Fixes: a78027847226 ("drm/gem: Acquire reservation lock in
> drm_gem_{pin/unpin}()")
> Signed-off-by: Adrián Larumbe
> ---
> drivers/gpu/drm/panfrost/panfrost_gem.c | 7 ++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c
> b/drivers/gpu/drm/panfrost/panfrost_gem.c
> index d47b40b82b0b..6c26652d425d 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem.c
> @@ -192,7 +192,12 @@ static int panfrost_gem_pin(struct drm_gem_object *obj)
> if (bo->is_heap)
> return -EINVAL;
>
> - return drm_gem_shmem_pin(>base);
> + /*
> + * Pinning can only happen in response to a prime attachment request
> from
> + * another driver, but that's already being handled by drm_gem_pin
> + */
> + drm_WARN_ON(obj->dev, obj->import_attach);
> + return drm_gem_shmem_pin_locked(>base);
> }
Will be better to use drm_gem_shmem_object_pin() to avoid such problem
in future
Please also fix the Lima driver
--
Best regards,
Dmitry
Re: [PATCH] drm/virtio: add driver_priv validation in virtio_gpu_create_context
On 3/28/24 16:43, Maxim Korotkov wrote:
> The pointer file->driver_priv was dereferenced without checking
> against NULL, but in the "virtio_gpu_transfer_to_host_ioctl" function
> it was checked against NULL after calling virtio_gpu_create_context
> function.
>
> Found by Security Code and Linux Verification Center(linuxtesting.org)
> Fixes: 72b48ae800da ("drm/virtio: enqueue virtio_gpu_create_context after the
> first 3D ioctl")
> Signed-off-by: Maxim Korotkov
> ---
> drivers/gpu/drm/virtio/virtgpu_ioctl.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
> b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
> index e4f76f315550..98fe9ad4ed15 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
> +++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
> @@ -64,6 +64,9 @@ void virtio_gpu_create_context(struct drm_device *dev,
> struct drm_file *file)
> struct virtio_gpu_device *vgdev = dev->dev_private;
> struct virtio_gpu_fpriv *vfpriv = file->driver_priv;
>
> + if (!vfpriv)
> + return;
> +
> mutex_lock(>context_lock);
> if (vfpriv->context_created)
> goto out_unlock;
NULL check in virtio_gpu_transfer_to_host_ioctl() is bogus, vfpriv can't
be NULL there.
--
Best regards,
Dmitry
Re: [PATCH v19 17/30] drm/panfrost: Fix the error path in panfrost_mmu_map_fault_addr()
On 1/5/24 21:46, Dmitry Osipenko wrote:
> From: Boris Brezillon
>
> If some the pages or sgt allocation failed, we shouldn't release the
> pages ref we got earlier, otherwise we will end up with unbalanced
> get/put_pages() calls. We should instead leave everything in place
> and let the BO release function deal with extra cleanup when the object
> is destroyed, or let the fault handler try again next time it's called.
>
> Fixes: 187d2929206e ("drm/panfrost: Add support for GPU heap allocations")
> Cc:
> Signed-off-by: Boris Brezillon
> Co-developed-by: Dmitry Osipenko
> Signed-off-by: Dmitry Osipenko
> ---
> drivers/gpu/drm/panfrost/panfrost_mmu.c | 13 +
> 1 file changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c
> b/drivers/gpu/drm/panfrost/panfrost_mmu.c
> index bd5a0073009d..4a0b4bf03f1a 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
> @@ -502,11 +502,18 @@ static int panfrost_mmu_map_fault_addr(struct
> panfrost_device *pfdev, int as,
> mapping_set_unevictable(mapping);
>
> for (i = page_offset; i < page_offset + NUM_FAULT_PAGES; i++) {
> + /* Can happen if the last fault only partially filled this
> + * section of the pages array before failing. In that case
> + * we skip already filled pages.
> + */
> + if (pages[i])
> + continue;
> +
> pages[i] = shmem_read_mapping_page(mapping, i);
> if (IS_ERR(pages[i])) {
> ret = PTR_ERR(pages[i]);
> pages[i] = NULL;
> - goto err_pages;
> + goto err_unlock;
> }
> }
>
> @@ -514,7 +521,7 @@ static int panfrost_mmu_map_fault_addr(struct
> panfrost_device *pfdev, int as,
> ret = sg_alloc_table_from_pages(sgt, pages + page_offset,
> NUM_FAULT_PAGES, 0, SZ_2M, GFP_KERNEL);
> if (ret)
> - goto err_pages;
> + goto err_unlock;
>
> ret = dma_map_sgtable(pfdev->dev, sgt, DMA_BIDIRECTIONAL, 0);
> if (ret)
> @@ -537,8 +544,6 @@ static int panfrost_mmu_map_fault_addr(struct
> panfrost_device *pfdev, int as,
>
> err_map:
> sg_free_table(sgt);
> -err_pages:
> - drm_gem_shmem_put_pages_locked(>base);
> err_unlock:
> dma_resv_unlock(obj->resv);
> err_bo:
Applied to misc-fixes
Forgot that this patch doesn't depend on others in this series, sorry
for not doing it earlier
--
Best regards,
Dmitry
Re: [PATCH 19/43] drm/virtio: Use fbdev-shmem
On 3/12/24 18:45, Thomas Zimmermann wrote: > Implement fbdev emulation with fbdev-shmem. Avoids the overhead of > fbdev-generic's additional shadow buffering. No functional changes. > > Signed-off-by: Thomas Zimmermann > Cc: David Airlie > Cc: Gerd Hoffmann > Cc: Gurchetan Singh > Cc: Chia-I Wu > --- > drivers/gpu/drm/virtio/virtgpu_drv.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c > b/drivers/gpu/drm/virtio/virtgpu_drv.c > index 9539aa28937fa..3d626bbaab9e4 100644 > --- a/drivers/gpu/drm/virtio/virtgpu_drv.c > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.c > @@ -35,7 +35,7 @@ > #include > #include > #include > -#include > +#include > #include > > #include "virtgpu_drv.h" > @@ -103,7 +103,7 @@ static int virtio_gpu_probe(struct virtio_device *vdev) > if (ret) > goto err_deinit; > > - drm_fbdev_generic_setup(vdev->priv, 32); > + drm_fbdev_shmem_setup(vdev->priv, 32); > return 0; > > err_deinit: Tested-by: Dmitry Osipenko -- Best regards, Dmitry
Re: [PATCH 00/13] drm: Fix reservation locking for pin/unpin and console
On 2/27/24 13:14, Thomas Zimmermann wrote:
> Dma-buf locking semantics require the caller of pin and unpin to hold
> the buffer's reservation lock. Fix DRM to adhere to the specs. This
> enables to fix the locking in DRM's console emulation. Similar changes
> for vmap and mmap have been posted at [1][2]
>
> Most DRM drivers and memory managers acquire the buffer object's
> reservation lock within their GEM pin and unpin callbacks. This
> violates dma-buf locking semantics. We get away with it because PRIME
> does not provide pin/unpin, but attach/detach, for which the locking
> semantics is correct.
>
> Patches 1 to 8 rework DRM GEM code in various implementations to
> acquire the reservation lock when entering the pin and unpin callbacks.
> This prepares them for the next patch. Drivers that are not affected
> by these patches either don't acquire the reservation lock (amdgpu)
> or don't need preparation (loongson).
>
> Patch 9 moves reservation locking from the GEM pin/unpin callbacks
> into drm_gem_pin() and drm_gem_unpin(). As PRIME uses these functions
> internally it still gets the reservation lock.
>
> With the updated GEM callbacks, the rest of the patchset fixes the
> fbdev emulation's buffer locking. Fbdev emulation needs to keep its
> GEM buffer object inplace while updating its content. This required
> a implicit pinning and apparently amdgpu didn't do this at all.
>
> Patch 10 introduces drm_client_buffer_vmap_local() and _vunmap_local().
> The former function map a GEM buffer into the kernel's address space
> with regular vmap operations, but keeps holding the reservation lock.
> The _vunmap_local() helper undoes the vmap and releases the lock. The
> updated GEM callbacks make this possible. Between the two calls, the
> fbdev emulation can update the buffer content without have the buffer
> moved or evicted. Update fbdev-generic to use vmap_local helpers,
> which fix amdgpu. The idea of adding a "local vmap" has previously been
> attempted at [3] in a different form.
>
> Patch 11 adds implicit pinning to the DRM client's regular vmap
> helper so that long-term vmap'ed buffers won't be evicted. This only
> affects fbdev-dma, but GEM DMA helpers don't require pinning. So
> there are no practical changes.
>
> Patches 12 and 13 remove implicit pinning from the vmap and vunmap
> operations in gem-vram and qxl. These pin operations are not supposed
> to be part of vmap code, but were required to keep the buffers in place
> for fbdev emulation. With the conversion o ffbdev-generic to to
> vmap_local helpers, that code can finally be removed.
>
> Tested with amdgpu, nouveau, radeon, simpledrm and vc4.
>
> [1] https://patchwork.freedesktop.org/series/106371/
> [2] https://patchwork.freedesktop.org/series/116001/
> [3] https://patchwork.freedesktop.org/series/84732/
>
> Thomas Zimmermann (13):
> drm/gem-shmem: Acquire reservation lock in GEM pin/unpin callbacks
> drm/gem-vram: Acquire reservation lock in GEM pin/unpin callbacks
> drm/msm: Provide msm_gem_get_pages_locked()
> drm/msm: Acquire reservation lock in GEM pin/unpin callback
> drm/nouveau: Provide nouveau_bo_{pin,unpin}_locked()
> drm/nouveau: Acquire reservation lock in GEM pin/unpin callbacks
> drm/qxl: Provide qxl_bo_{pin,unpin}_locked()
> drm/qxl: Acquire reservation lock in GEM pin/unpin callbacks
> drm/gem: Acquire reservation lock in drm_gem_{pin/unpin}()
> drm/fbdev-generic: Fix locking with drm_client_buffer_vmap_local()
> drm/client: Pin vmap'ed GEM buffers
> drm/gem-vram: Do not pin buffer objects for vmap
> drm/qxl: Do not pin buffer objects for vmap
The patches look good. I gave them fbtest on virtio-gpu, no problems
spotted.
Reviewed-by: Dmitry Osipenko
Tested-by: Dmitry Osipenko # virtio-gpu
--
Best regards,
Dmitry
Re: [PATCH 00/13] drm: Fix reservation locking for pin/unpin and console
On 2/28/24 11:19, Thomas Zimmermann wrote: > Hi > > Am 27.02.24 um 19:14 schrieb Dmitry Osipenko: >> Hello, >> >> Thank you for the patches! >> >> On 2/27/24 13:14, Thomas Zimmermann wrote: >>> Dma-buf locking semantics require the caller of pin and unpin to hold >>> the buffer's reservation lock. Fix DRM to adhere to the specs. This >>> enables to fix the locking in DRM's console emulation. Similar changes >>> for vmap and mmap have been posted at [1][2] >>> >>> Most DRM drivers and memory managers acquire the buffer object's >>> reservation lock within their GEM pin and unpin callbacks. This >>> violates dma-buf locking semantics. We get away with it because PRIME >>> does not provide pin/unpin, but attach/detach, for which the locking >>> semantics is correct. >>> >>> Patches 1 to 8 rework DRM GEM code in various implementations to >>> acquire the reservation lock when entering the pin and unpin callbacks. >>> This prepares them for the next patch. Drivers that are not affected >>> by these patches either don't acquire the reservation lock (amdgpu) >>> or don't need preparation (loongson). >>> >>> Patch 9 moves reservation locking from the GEM pin/unpin callbacks >>> into drm_gem_pin() and drm_gem_unpin(). As PRIME uses these functions >>> internally it still gets the reservation lock. >>> >>> With the updated GEM callbacks, the rest of the patchset fixes the >>> fbdev emulation's buffer locking. Fbdev emulation needs to keep its >>> GEM buffer object inplace while updating its content. This required >>> a implicit pinning and apparently amdgpu didn't do this at all. >>> >>> Patch 10 introduces drm_client_buffer_vmap_local() and _vunmap_local(). >>> The former function map a GEM buffer into the kernel's address space >>> with regular vmap operations, but keeps holding the reservation lock. >>> The _vunmap_local() helper undoes the vmap and releases the lock. The >>> updated GEM callbacks make this possible. Between the two calls, the >>> fbdev emulation can update the buffer content without have the buffer >>> moved or evicted. Update fbdev-generic to use vmap_local helpers, >>> which fix amdgpu. The idea of adding a "local vmap" has previously been >>> attempted at [3] in a different form. >>> >>> Patch 11 adds implicit pinning to the DRM client's regular vmap >>> helper so that long-term vmap'ed buffers won't be evicted. This only >>> affects fbdev-dma, but GEM DMA helpers don't require pinning. So >>> there are no practical changes. >>> >>> Patches 12 and 13 remove implicit pinning from the vmap and vunmap >>> operations in gem-vram and qxl. These pin operations are not supposed >>> to be part of vmap code, but were required to keep the buffers in place >>> for fbdev emulation. With the conversion o ffbdev-generic to to >>> vmap_local helpers, that code can finally be removed. >> Isn't it a common behaviour for all DRM drivers to implicitly pin BO >> while it's vmapped? I was sure it should be common /o\ > > That's what I originally thought as well, but the intention is for pin > and vmap to be distinct operation. So far each driver has been > different, as you probably know best from your vmap refactoring. :) > >> >> Why would you want to kmap BO that isn't pinned? > > Pinning places the buffer object for the GPU. As a side effect, the > buffer is then kept in place, which enables vmap. So pinning only makes > sense for buffer objects that never move (shmem, dma). That's what patch > 11 is for. > >> >> Shouldn't TTM's vmap() be changed to do the pinning? > > I don't think so. One problem is that pinning needs a memory area (vram, > GTT, system ram, etc) specified, which vmap simply doesn't know about. > That has been a problem for fbdev emulation at some point. Our fbdev > code tried to pin as part of vmap, but chose the wrong area and suddenly > the GPU could not see the buffer object any longer. So the next best > thing for vmap was to pin the buffer object where ever it is currently > located. That is what gem-vram and qxl did so far. And of course, the > fbdev code needs to unpin and vunmap the buffer object quickly, so that > it can be relocated if the GPU needs it. Hence, the vmap_local > interface removes such short-term pinning in favor of holding the > reservation lock. > >> >> I missed that TTM doesn't pin BO on vmap() and now surprised to see it. >> It should be a rather serious problem requiring backporting of the >> fixes, but I don't see the fixes tags on the patches (?) > > No chance TBH. The old code has worked for years and backporting all > this would require your vmap patches at a minimum. > > Except maybe for amdgpu. It uses fbdev-generic, which requires pinning, > but amdgpu doesn't pin. That looks fishy, but I'm not aware of any bug > reports either. I guess, a quick workaround could fix older amdgpu if > necessary. Thanks! I'll make another pass on the patches on Monday -- Best regards, Dmitry
Re: [PATCH 00/13] drm: Fix reservation locking for pin/unpin and console
Hello, Thank you for the patches! On 2/27/24 13:14, Thomas Zimmermann wrote: > Dma-buf locking semantics require the caller of pin and unpin to hold > the buffer's reservation lock. Fix DRM to adhere to the specs. This > enables to fix the locking in DRM's console emulation. Similar changes > for vmap and mmap have been posted at [1][2] > > Most DRM drivers and memory managers acquire the buffer object's > reservation lock within their GEM pin and unpin callbacks. This > violates dma-buf locking semantics. We get away with it because PRIME > does not provide pin/unpin, but attach/detach, for which the locking > semantics is correct. > > Patches 1 to 8 rework DRM GEM code in various implementations to > acquire the reservation lock when entering the pin and unpin callbacks. > This prepares them for the next patch. Drivers that are not affected > by these patches either don't acquire the reservation lock (amdgpu) > or don't need preparation (loongson). > > Patch 9 moves reservation locking from the GEM pin/unpin callbacks > into drm_gem_pin() and drm_gem_unpin(). As PRIME uses these functions > internally it still gets the reservation lock. > > With the updated GEM callbacks, the rest of the patchset fixes the > fbdev emulation's buffer locking. Fbdev emulation needs to keep its > GEM buffer object inplace while updating its content. This required > a implicit pinning and apparently amdgpu didn't do this at all. > > Patch 10 introduces drm_client_buffer_vmap_local() and _vunmap_local(). > The former function map a GEM buffer into the kernel's address space > with regular vmap operations, but keeps holding the reservation lock. > The _vunmap_local() helper undoes the vmap and releases the lock. The > updated GEM callbacks make this possible. Between the two calls, the > fbdev emulation can update the buffer content without have the buffer > moved or evicted. Update fbdev-generic to use vmap_local helpers, > which fix amdgpu. The idea of adding a "local vmap" has previously been > attempted at [3] in a different form. > > Patch 11 adds implicit pinning to the DRM client's regular vmap > helper so that long-term vmap'ed buffers won't be evicted. This only > affects fbdev-dma, but GEM DMA helpers don't require pinning. So > there are no practical changes. > > Patches 12 and 13 remove implicit pinning from the vmap and vunmap > operations in gem-vram and qxl. These pin operations are not supposed > to be part of vmap code, but were required to keep the buffers in place > for fbdev emulation. With the conversion o ffbdev-generic to to > vmap_local helpers, that code can finally be removed. Isn't it a common behaviour for all DRM drivers to implicitly pin BO while it's vmapped? I was sure it should be common /o\ Why would you want to kmap BO that isn't pinned? Shouldn't TTM's vmap() be changed to do the pinning? I missed that TTM doesn't pin BO on vmap() and now surprised to see it. It should be a rather serious problem requiring backporting of the fixes, but I don't see the fixes tags on the patches (?) -- Best regards, Dmitry
Re: [PATCH v19 09/30] drm/shmem-helper: Add and use lockless drm_gem_shmem_get_pages()
On 1/30/24 13:10, Boris Brezillon wrote:
> On Tue, 30 Jan 2024 09:34:29 +0100
> Daniel Vetter wrote:
>
>> On Fri, Jan 26, 2024 at 07:43:29PM +0300, Dmitry Osipenko wrote:
>>> On 1/26/24 13:18, Boris Brezillon wrote:
>>>> On Thu, 25 Jan 2024 18:24:04 +0100
>>>> Daniel Vetter wrote:
>>>>
>>>>> On Fri, Jan 05, 2024 at 09:46:03PM +0300, Dmitry Osipenko wrote:
>>>>>> Add lockless drm_gem_shmem_get_pages() helper that skips taking
>>>>>> reservation
>>>>>> lock if pages_use_count is non-zero, leveraging from atomicity of the
>>>>>> refcount_t. Make drm_gem_shmem_mmap() to utilize the new helper.
>>>>>>
>>>>>> Acked-by: Maxime Ripard
>>>>>> Reviewed-by: Boris Brezillon
>>>>>> Suggested-by: Boris Brezillon
>>>>>> Signed-off-by: Dmitry Osipenko
>>>>>> ---
>>>>>> drivers/gpu/drm/drm_gem_shmem_helper.c | 19 +++
>>>>>> 1 file changed, 15 insertions(+), 4 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>>> b/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>>> index cacf0f8c42e2..1c032513abf1 100644
>>>>>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>>> @@ -226,6 +226,20 @@ void drm_gem_shmem_put_pages_locked(struct
>>>>>> drm_gem_shmem_object *shmem)
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
>>>>>>
>>>>>> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>>>>> +{
>>>>>> +int ret;
>>>>>
>>>>> Just random drive-by comment: a might_lock annotation here might be good,
>>>>> or people could hit some really interesting bugs that are rather hard to
>>>>> reproduce ...
>>>>
>>>> Actually, being able to acquire a ref in a dma-signalling path on an
>>>> object we know for sure already has refcount >= 1 (because we previously
>>>> acquired a ref in a path where dma_resv_lock() was allowed), was the
>>>> primary reason I suggested moving to this atomic-refcount approach.
>>>>
>>>> In the meantime, drm_gpuvm has evolved in a way that allows me to not
>>>> take the ref in the dma-signalling path (the gpuvm_bo object now holds
>>>> the ref, and it's acquired/released outside the dma-signalling path).
>>>>
>>>> Not saying we shouldn't add this might_lock(), but others might have
>>>> good reasons to have this function called in a path where locking
>>>> is not allowed.
>>>
>>> For Panthor the might_lock indeed won't be a appropriate, thanks for
>>> reminding about it. I'll add explanatory comment to the code.
>>
>> Hm these kind of tricks feel very dangerous to me. I think it would be
>> good to split up the two cases into two functions:
>>
>> 1. first one does only the atomic_inc and splats if the refcount is zero.
>> I think something in the name that denotes that we're incrementing a
>> borrowed pages reference would be good here, so like get_borrowed_pages
>> (there's not really a naming convention for these in the kernel).
>> Unfortunately no rust so we can't enforce that you provide the right kind
>> of borrowed reference at compile time.
>
> Yeah, I also considered adding a dedicated function for that use case
> at some point, instead of abusing get_pages(). Given I no longer need
> it, we can probably add this might_lock() and defer the addition of this
> get_borrowed_pages() helper until someone actually needs it.
Ack, I'll add the might_lock() then. Missed previously that you don't
need to use get_pages() anymore. Thanks
--
Best regards,
Dmitry
Re: [PATCH v2] drm/msm/gem: Fix double resv lock aquire
On 1/31/24 04:15, Rob Clark wrote:
> From: Rob Clark
>
> Since commit 56e5abba8c3e ("dma-buf: Add unlocked variant of vmapping
> functions"), the resv lock is already held in the prime vmap path, so
> don't try to grab it again.
>
> v2: This applies to vunmap path as well
>
> Fixes: 56e5abba8c3e ("dma-buf: Add unlocked variant of vmapping functions")
> Signed-off-by: Rob Clark
Nit: the offending commit should be 79e2cf2e7a19 ("drm/gem: Take
reservation lock for vmap/vunmap operations")
> ---
> drivers/gpu/drm/msm/msm_gem_prime.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem_prime.c
> b/drivers/gpu/drm/msm/msm_gem_prime.c
> index 5f68e31a3e4e..0915f3b68752 100644
> --- a/drivers/gpu/drm/msm/msm_gem_prime.c
> +++ b/drivers/gpu/drm/msm/msm_gem_prime.c
> @@ -26,7 +26,7 @@ int msm_gem_prime_vmap(struct drm_gem_object *obj, struct
> iosys_map *map)
> {
> void *vaddr;
>
> - vaddr = msm_gem_get_vaddr(obj);
> + vaddr = msm_gem_get_vaddr_locked(obj);
> if (IS_ERR(vaddr))
> return PTR_ERR(vaddr);
> iosys_map_set_vaddr(map, vaddr);
> @@ -36,7 +36,7 @@ int msm_gem_prime_vmap(struct drm_gem_object *obj, struct
> iosys_map *map)
>
> void msm_gem_prime_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
> {
> - msm_gem_put_vaddr(obj);
> + msm_gem_put_vaddr_locked(obj);
> }
>
> struct drm_gem_object *msm_gem_prime_import_sg_table(struct drm_device *dev,
Looks good otherwise
--
Best regards,
Dmitry
Re: [PATCH v19 09/30] drm/shmem-helper: Add and use lockless drm_gem_shmem_get_pages()
On 1/30/24 11:34, Daniel Vetter wrote:
> On Fri, Jan 26, 2024 at 07:43:29PM +0300, Dmitry Osipenko wrote:
>> On 1/26/24 13:18, Boris Brezillon wrote:
>>> On Thu, 25 Jan 2024 18:24:04 +0100
>>> Daniel Vetter wrote:
>>>
>>>> On Fri, Jan 05, 2024 at 09:46:03PM +0300, Dmitry Osipenko wrote:
>>>>> Add lockless drm_gem_shmem_get_pages() helper that skips taking
>>>>> reservation
>>>>> lock if pages_use_count is non-zero, leveraging from atomicity of the
>>>>> refcount_t. Make drm_gem_shmem_mmap() to utilize the new helper.
>>>>>
>>>>> Acked-by: Maxime Ripard
>>>>> Reviewed-by: Boris Brezillon
>>>>> Suggested-by: Boris Brezillon
>>>>> Signed-off-by: Dmitry Osipenko
>>>>> ---
>>>>> drivers/gpu/drm/drm_gem_shmem_helper.c | 19 +++
>>>>> 1 file changed, 15 insertions(+), 4 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>> b/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>> index cacf0f8c42e2..1c032513abf1 100644
>>>>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>> @@ -226,6 +226,20 @@ void drm_gem_shmem_put_pages_locked(struct
>>>>> drm_gem_shmem_object *shmem)
>>>>> }
>>>>> EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
>>>>>
>>>>> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>>>> +{
>>>>> + int ret;
>>>>
>>>> Just random drive-by comment: a might_lock annotation here might be good,
>>>> or people could hit some really interesting bugs that are rather hard to
>>>> reproduce ...
>>>
>>> Actually, being able to acquire a ref in a dma-signalling path on an
>>> object we know for sure already has refcount >= 1 (because we previously
>>> acquired a ref in a path where dma_resv_lock() was allowed), was the
>>> primary reason I suggested moving to this atomic-refcount approach.
>>>
>>> In the meantime, drm_gpuvm has evolved in a way that allows me to not
>>> take the ref in the dma-signalling path (the gpuvm_bo object now holds
>>> the ref, and it's acquired/released outside the dma-signalling path).
>>>
>>> Not saying we shouldn't add this might_lock(), but others might have
>>> good reasons to have this function called in a path where locking
>>> is not allowed.
>>
>> For Panthor the might_lock indeed won't be a appropriate, thanks for
>> reminding about it. I'll add explanatory comment to the code.
>
> Hm these kind of tricks feel very dangerous to me. I think it would be
> good to split up the two cases into two functions:
>
> 1. first one does only the atomic_inc and splats if the refcount is zero.
> I think something in the name that denotes that we're incrementing a
> borrowed pages reference would be good here, so like get_borrowed_pages
> (there's not really a naming convention for these in the kernel).
> Unfortunately no rust so we can't enforce that you provide the right kind
> of borrowed reference at compile time.
>
> 2. second one has the might_lock.
>
> This way you force callers to think what they're doing and ideally
> document where the borrowed reference is from, and ideally document that
> in the code. Otherwise we'll end up with way too much "works in testing,
> but is a nice CVE" code :-/
We indeed can have both variants of the borrowed/non-borrowed functions.
Thanks again for the suggestions
--
Best regards,
Dmitry
Re: [PATCH v19 22/30] drm/shmem-helper: Add common memory shrinker
On 1/30/24 11:39, Daniel Vetter wrote: > On Thu, Jan 25, 2024 at 10:07:03AM +0100, Boris Brezillon wrote: >> On Fri, 5 Jan 2024 21:46:16 +0300 >> Dmitry Osipenko wrote: >> >>> * >>> * This function Increases the use count and allocates the backing pages if >>> * use-count equals to zero. >>> + * >>> + * Note that this function doesn't pin pages in memory. If your driver >>> + * uses drm-shmem shrinker, then it's free to relocate pages to swap. >>> + * Getting pages only guarantees that pages are allocated, and not that >>> + * pages reside in memory. In order to pin pages use drm_gem_shmem_pin(). >> >> I still find this explanation confusing, if pages are allocated, they >> reside in memory. The only difference between drm_gem_shmem_get_pages() >> and drm_gem_shmem_pin_pages() is that the former lets the system >> reclaim the memory if the buffer is idle (no unsignalled fence attached >> to the dma_resv). >> >> We also need to describe the workflow for GEM validation (that's the >> TTM term for the swapin process happening when a GPU job is submitted). >> >> 1. Prepare the GPU job and initialize its fence >> 2. Lock the GEM resv >> 3. Add the GPU job fence to the resv object >> 4. If the GEM is evicted >>a. call drm_gem_shmem_swapin_locked() >>b. get the new sgt with drm_gem_shmem_get_pages_sgt_locked() >>c. repopulate the MMU table (driver internals) > > Might be good to explain where to call drm_sched_job_arm() here for > drivers using drm/sched, since that also needs to be at a very specific > point. Probably best to flesh out the details here by linking to the > relevant drm/sched and gpuvm functions as examples. > >> 5. Unlock the GEM dma_resv >> 6. Submit the GPU job >> >> With this sequence, the GEM pages are guaranteed to stay around until >> the GPU job is finished. > > Yeah I think the comment needs to explain how this ties together with > dma_resv locking and dma_resv fences, otherwise it just doesn't make much > sense. > > This holds even more so given that some of the earlier drivers derived > from i915-gem code (and i915-gem itself) use _pin() both for these more > permanent pinnings, and also to temporarily put the memory in place before > it all gets fenced and then unpinned > > So would be really good to have the sharpest possible nomeclatura here we > can get, and link between all the related concepts and functions in the > kerneldoc. > > Some overview flow like Boris sketched above in a DOC: section would also > be great. Thank you all for the feedback! I'll add all this doc in the next version -- Best regards, Dmitry
Re: [PATCH v19 22/30] drm/shmem-helper: Add common memory shrinker
On 1/29/24 12:01, Boris Brezillon wrote:
> On Fri, 5 Jan 2024 21:46:16 +0300
> Dmitry Osipenko wrote:
>
>> +/**
>> + * drm_gem_shmem_swapin_locked() - Moves shmem GEM back to memory and
>> enables
>> + * hardware access to the memory.
>> + * @shmem: shmem GEM object
>> + *
>> + * This function moves shmem GEM back to memory if it was previously evicted
>> + * by the memory shrinker. The GEM is ready to use on success.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_swapin_locked(struct drm_gem_shmem_object *shmem)
>> +{
>> +int err;
>> +
>> +dma_resv_assert_held(shmem->base.resv);
>> +
>> +if (!shmem->evicted)
>> +return 0;
>
> Shouldn't we have a drm_gem_shmem_shrinker_update_lru_locked() even if
> the object wasn't evicted, such that idle->busy transition moves the BO
> to the list tail?
Seems so, good catch. I'll double-check and remove it in the next version.
--
Best regards,
Dmitry
Re: [PATCH RESEND] drm/virtio: set segment size for virtio_gpu device
On 1/23/24 21:14, Sebastian Ott wrote: > drm/virtio: set segment size for virtio_gpu device > > Set the segment size of the virtio_gpu device to the value > used by the drm helpers when allocating sg lists to fix the > following complaint from DMA_API debug code: > DMA-API: virtio-pci :07:00.0: mapping sg segment longer than device > claims to support [len=262144] [max=65536] > > Signed-off-by: Sebastian Ott > --- > drivers/gpu/drm/virtio/virtgpu_drv.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c > b/drivers/gpu/drm/virtio/virtgpu_drv.c > index 4334c7608408..74b2cb3295af 100644 > --- a/drivers/gpu/drm/virtio/virtgpu_drv.c > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.c > @@ -94,6 +94,7 @@ static int virtio_gpu_probe(struct virtio_device *vdev) > goto err_free; > } > > + dma_set_max_seg_size(dev->dev, dma_max_mapping_size(dev->dev) ? : > UINT_MAX); > ret = virtio_gpu_init(vdev, dev); > if (ret) > goto err_free; Added stable tag and applied to misc-fixes. Please use `git send-email` next time. Thanks -- Best regards, Dmitry
Re: [PATCH v19 22/30] drm/shmem-helper: Add common memory shrinker
On 1/26/24 21:12, Boris Brezillon wrote:
> On Fri, 26 Jan 2024 19:27:49 +0300
> Dmitry Osipenko wrote:
>
>> On 1/26/24 12:55, Boris Brezillon wrote:
>>> On Fri, 26 Jan 2024 00:56:47 +0300
>>> Dmitry Osipenko wrote:
>>>
>>>> On 1/25/24 13:19, Boris Brezillon wrote:
>>>>> On Fri, 5 Jan 2024 21:46:16 +0300
>>>>> Dmitry Osipenko wrote:
>>>>>
>>>>>> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object
>>>>>> *shmem)
>>>>>> +{
>>>>>> +return (shmem->madv >= 0) && shmem->base.funcs->evict &&
>>>>>> +refcount_read(>pages_use_count) &&
>>>>>> +!refcount_read(>pages_pin_count) &&
>>>>>> +!shmem->base.dma_buf && !shmem->base.import_attach &&
>>>>>> +!shmem->evicted;
>>>>>
>>>>> Are we missing
>>>>>
>>>>> && dma_resv_test_signaled(shmem->base.resv,
>>>>> DMA_RESV_USAGE_BOOKKEEP)
>>>>>
>>>>> to make sure the GPU is done using the BO?
>>>>> The same applies to drm_gem_shmem_is_purgeable() BTW.
>>>>>
>>>>> If you don't want to do this test here, we need a way to let drivers
>>>>> provide a custom is_{evictable,purgeable}() test.
>>>>>
>>>>> I guess we should also expose drm_gem_shmem_shrinker_update_lru_locked()
>>>>> to let drivers move the GEMs that were used most recently (those
>>>>> referenced by a GPU job) at the end of the evictable LRU.
>>>>
>>>> We have the signaled-check in the common drm_gem_evict() helper:
>>>>
>>>> https://elixir.bootlin.com/linux/v6.8-rc1/source/drivers/gpu/drm/drm_gem.c#L1496
>>>>
>>>
>>> Ah, indeed. I'll need DMA_RESV_USAGE_BOOKKEEP instead of
>>> DMA_RESV_USAGE_READ in panthor, but I can add it in the driver specific
>>> ->evict() hook (though that means calling dma_resv_test_signaled()
>>> twice, which is not great, oh well).
>>
>> Maybe we should change drm_gem_evict() to use BOOKKEEP. The
>> test_signaled(BOOKKEEP) should be a "stronger" check than
>> test_signaled(READ)?
>
> It is, just wondering if some users have a good reason to want
> READ here.
>
>>
>>> The problem about the evictable LRU remains though: we need a way to let
>>> drivers put their BOs at the end of the list when the BO has been used
>>> by the GPU, don't we?
>>
>> If BO is use, then it won't be evicted, while idling BOs will be
>> evicted. Hence, the used BOs will be naturally moved down the LRU list
>> each time shrinker is invoked.
>>
>
> That only do the trick if the BOs being used most often are busy when
> the shrinker kicks in though. Let's take this scenario:
>
>
> BO 1 BO 2
> shinker
>
> busy
> idle (first-pos-in-evictable-LRU)
>
> busy
> idle (second-pos-in-evictable-LRU)
>
> busy
> idle
>
> busy
> idle
>
> busy
> idle
>
>
> find a BO to evict
>
> pick BO 2
>
> busy (swapin)
> idle
>
> If the LRU had been updated at each busy event, BO 1 should have
> been picked for eviction. But we evicted the BO that was first
> recorded idle instead of the one that was least recently
> recorded busy.
You have to swapin(BO) every time BO goes to busy state, and swapin does
drm_gem_lru_move_tail(BO). Hence, each time BO goes idle->busy, it's moved down
the LRU list.
For example, please see patch #29 where virtio-gpu invokes swapin for each
job's BO in the submit()->virtio_gpu_array_prepare() code path.
--
Best regards,
Dmitry
Re: [PATCH v19 09/30] drm/shmem-helper: Add and use lockless drm_gem_shmem_get_pages()
On 1/26/24 13:18, Boris Brezillon wrote:
> On Thu, 25 Jan 2024 18:24:04 +0100
> Daniel Vetter wrote:
>
>> On Fri, Jan 05, 2024 at 09:46:03PM +0300, Dmitry Osipenko wrote:
>>> Add lockless drm_gem_shmem_get_pages() helper that skips taking reservation
>>> lock if pages_use_count is non-zero, leveraging from atomicity of the
>>> refcount_t. Make drm_gem_shmem_mmap() to utilize the new helper.
>>>
>>> Acked-by: Maxime Ripard
>>> Reviewed-by: Boris Brezillon
>>> Suggested-by: Boris Brezillon
>>> Signed-off-by: Dmitry Osipenko
>>> ---
>>> drivers/gpu/drm/drm_gem_shmem_helper.c | 19 +++
>>> 1 file changed, 15 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
>>> b/drivers/gpu/drm/drm_gem_shmem_helper.c
>>> index cacf0f8c42e2..1c032513abf1 100644
>>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>>> @@ -226,6 +226,20 @@ void drm_gem_shmem_put_pages_locked(struct
>>> drm_gem_shmem_object *shmem)
>>> }
>>> EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
>>>
>>> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>> +{
>>> + int ret;
>>
>> Just random drive-by comment: a might_lock annotation here might be good,
>> or people could hit some really interesting bugs that are rather hard to
>> reproduce ...
>
> Actually, being able to acquire a ref in a dma-signalling path on an
> object we know for sure already has refcount >= 1 (because we previously
> acquired a ref in a path where dma_resv_lock() was allowed), was the
> primary reason I suggested moving to this atomic-refcount approach.
>
> In the meantime, drm_gpuvm has evolved in a way that allows me to not
> take the ref in the dma-signalling path (the gpuvm_bo object now holds
> the ref, and it's acquired/released outside the dma-signalling path).
>
> Not saying we shouldn't add this might_lock(), but others might have
> good reasons to have this function called in a path where locking
> is not allowed.
For Panthor the might_lock indeed won't be a appropriate, thanks for
reminding about it. I'll add explanatory comment to the code.
--
Best regards,
Dmitry
Re: [PATCH v19 22/30] drm/shmem-helper: Add common memory shrinker
On 1/26/24 12:55, Boris Brezillon wrote:
> On Fri, 26 Jan 2024 00:56:47 +0300
> Dmitry Osipenko wrote:
>
>> On 1/25/24 13:19, Boris Brezillon wrote:
>>> On Fri, 5 Jan 2024 21:46:16 +0300
>>> Dmitry Osipenko wrote:
>>>
>>>> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
>>>> +{
>>>> + return (shmem->madv >= 0) && shmem->base.funcs->evict &&
>>>> + refcount_read(>pages_use_count) &&
>>>> + !refcount_read(>pages_pin_count) &&
>>>> + !shmem->base.dma_buf && !shmem->base.import_attach &&
>>>> + !shmem->evicted;
>>>
>>> Are we missing
>>>
>>> && dma_resv_test_signaled(shmem->base.resv,
>>> DMA_RESV_USAGE_BOOKKEEP)
>>>
>>> to make sure the GPU is done using the BO?
>>> The same applies to drm_gem_shmem_is_purgeable() BTW.
>>>
>>> If you don't want to do this test here, we need a way to let drivers
>>> provide a custom is_{evictable,purgeable}() test.
>>>
>>> I guess we should also expose drm_gem_shmem_shrinker_update_lru_locked()
>>> to let drivers move the GEMs that were used most recently (those
>>> referenced by a GPU job) at the end of the evictable LRU.
>>
>> We have the signaled-check in the common drm_gem_evict() helper:
>>
>> https://elixir.bootlin.com/linux/v6.8-rc1/source/drivers/gpu/drm/drm_gem.c#L1496
>
> Ah, indeed. I'll need DMA_RESV_USAGE_BOOKKEEP instead of
> DMA_RESV_USAGE_READ in panthor, but I can add it in the driver specific
> ->evict() hook (though that means calling dma_resv_test_signaled()
> twice, which is not great, oh well).
Maybe we should change drm_gem_evict() to use BOOKKEEP. The
test_signaled(BOOKKEEP) should be a "stronger" check than
test_signaled(READ)?
> The problem about the evictable LRU remains though: we need a way to let
> drivers put their BOs at the end of the list when the BO has been used
> by the GPU, don't we?
If BO is use, then it won't be evicted, while idling BOs will be
evicted. Hence, the used BOs will be naturally moved down the LRU list
each time shrinker is invoked.
--
Best regards,
Dmitry
Re: [PATCH v19 22/30] drm/shmem-helper: Add common memory shrinker
On 1/25/24 13:19, Boris Brezillon wrote:
> On Fri, 5 Jan 2024 21:46:16 +0300
> Dmitry Osipenko wrote:
>
>> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +return (shmem->madv >= 0) && shmem->base.funcs->evict &&
>> +refcount_read(>pages_use_count) &&
>> +!refcount_read(>pages_pin_count) &&
>> +!shmem->base.dma_buf && !shmem->base.import_attach &&
>> +!shmem->evicted;
>
> Are we missing
>
> && dma_resv_test_signaled(shmem->base.resv,
> DMA_RESV_USAGE_BOOKKEEP)
>
> to make sure the GPU is done using the BO?
> The same applies to drm_gem_shmem_is_purgeable() BTW.
>
> If you don't want to do this test here, we need a way to let drivers
> provide a custom is_{evictable,purgeable}() test.
>
> I guess we should also expose drm_gem_shmem_shrinker_update_lru_locked()
> to let drivers move the GEMs that were used most recently (those
> referenced by a GPU job) at the end of the evictable LRU.
We have the signaled-check in the common drm_gem_evict() helper:
https://elixir.bootlin.com/linux/v6.8-rc1/source/drivers/gpu/drm/drm_gem.c#L1496
--
Best regards,
Dmitry
Re: [PATCH v19 09/30] drm/shmem-helper: Add and use lockless drm_gem_shmem_get_pages()
On 1/25/24 20:24, Daniel Vetter wrote:
> On Fri, Jan 05, 2024 at 09:46:03PM +0300, Dmitry Osipenko wrote:
>> Add lockless drm_gem_shmem_get_pages() helper that skips taking reservation
>> lock if pages_use_count is non-zero, leveraging from atomicity of the
>> refcount_t. Make drm_gem_shmem_mmap() to utilize the new helper.
>>
>> Acked-by: Maxime Ripard
>> Reviewed-by: Boris Brezillon
>> Suggested-by: Boris Brezillon
>> Signed-off-by: Dmitry Osipenko
>> ---
>> drivers/gpu/drm/drm_gem_shmem_helper.c | 19 +++
>> 1 file changed, 15 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
>> b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> index cacf0f8c42e2..1c032513abf1 100644
>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> @@ -226,6 +226,20 @@ void drm_gem_shmem_put_pages_locked(struct
>> drm_gem_shmem_object *shmem)
>> }
>> EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
>>
>> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +int ret;
>
> Just random drive-by comment: a might_lock annotation here might be good,
> or people could hit some really interesting bugs that are rather hard to
> reproduce ...
> -Sima
Thanks for the suggestion!
--
Best regards,
Dmitry
Re: [PATCH v19 17/30] drm/panfrost: Fix the error path in panfrost_mmu_map_fault_addr()
On 1/26/24 00:41, Dmitry Osipenko wrote:
> On 1/5/24 21:46, Dmitry Osipenko wrote:
>> for (i = page_offset; i < page_offset + NUM_FAULT_PAGES; i++) {
>> +/* Can happen if the last fault only partially filled this
>> + * section of the pages array before failing. In that case
>> + * we skip already filled pages.
>> + */
>> +if (pages[i])
>> +continue;
>> +
>> pages[i] = shmem_read_mapping_page(mapping, i);
>
> Although, the shmem_read_mapping_page() should return same page if it
> was already allocated, isn't it? I.e. there was no bug here and the
> fixes/stable tags not needed.
Scratch that, I forgot that the patch is about the unbalanced
get/put_pages
--
Best regards,
Dmitry
Re: [PATCH v19 17/30] drm/panfrost: Fix the error path in panfrost_mmu_map_fault_addr()
On 1/5/24 21:46, Dmitry Osipenko wrote:
> for (i = page_offset; i < page_offset + NUM_FAULT_PAGES; i++) {
> + /* Can happen if the last fault only partially filled this
> + * section of the pages array before failing. In that case
> + * we skip already filled pages.
> + */
> + if (pages[i])
> + continue;
> +
> pages[i] = shmem_read_mapping_page(mapping, i);
Although, the shmem_read_mapping_page() should return same page if it
was already allocated, isn't it? I.e. there was no bug here and the
fixes/stable tags not needed.
--
Best regards,
Dmitry
Re: [PATCH v19 30/30] drm/panfrost: Switch to generic memory shrinker
On 1/25/24 12:49, Boris Brezillon wrote:
> On Fri, 5 Jan 2024 21:46:24 +0300
> Dmitry Osipenko wrote:
>
>> --- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
>> +++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
>> @@ -328,6 +328,7 @@ int panfrost_mmu_map(struct panfrost_gem_mapping
>> *mapping)
>> struct panfrost_device *pfdev = to_panfrost_device(obj->dev);
>> struct sg_table *sgt;
>> int prot = IOMMU_READ | IOMMU_WRITE;
>> +int ret = 0;
>>
>> if (WARN_ON(mapping->active))
>> return 0;
>> @@ -335,15 +336,32 @@ int panfrost_mmu_map(struct panfrost_gem_mapping
>> *mapping)
>> if (bo->noexec)
>> prot |= IOMMU_NOEXEC;
>>
>> +if (!obj->import_attach) {
>> +/*
>> + * Don't allow shrinker to move pages while pages are mapped.
>> + * It's fine to move pages afterwards because shrinker will
>> + * take care of unmapping pages during eviction.
>> + */
>
> That's not exactly what this shmem_pin() is about, is it? I think it's
> here to meet the drm_gem_shmem_get_pages_sgt() rule stating that pages
> must be pinned while the sgt returned by drm_gem_shmem_get_pages_sgt()
> is manipulated. You actually unpin the GEM just after the mmu_map_sg()
> call, which means pages could very well be reclaimed while the MMU
> still has a mapping referencing those physical pages. And that's fine,
> because what's supposed to protect against that is the fence we
> register to the GEM resv at job submission time.
The comment indeed needs to be improved, thanks.
s/are mapped/in process of mapping creation/
--
Best regards,
Dmitry
[PATCH v19 30/30] drm/panfrost: Switch to generic memory shrinker
Replace Panfrost's custom memory shrinker with a common drm-shmem
memory shrinker.
Co-developed-by: Boris Brezillon
Signed-off-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c| 4 +-
drivers/gpu/drm/panfrost/Makefile | 1 -
drivers/gpu/drm/panfrost/panfrost_device.h| 4 -
drivers/gpu/drm/panfrost/panfrost_drv.c | 29 ++--
drivers/gpu/drm/panfrost/panfrost_gem.c | 60
drivers/gpu/drm/panfrost/panfrost_gem.h | 9 --
.../gpu/drm/panfrost/panfrost_gem_shrinker.c | 140 --
drivers/gpu/drm/panfrost/panfrost_job.c | 18 ++-
drivers/gpu/drm/panfrost/panfrost_mmu.c | 24 ++-
include/drm/drm_gem_shmem_helper.h| 7 -
10 files changed, 83 insertions(+), 213 deletions(-)
delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 7d2fe12bd793..56e88378079b 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -89,8 +89,6 @@ __drm_gem_shmem_create(struct drm_device *dev, size_t size,
bool private)
if (ret)
goto err_release;
- INIT_LIST_HEAD(>madv_list);
-
if (!private) {
/*
* Our buffers are kept pinned, so allocating them
@@ -619,6 +617,8 @@ void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object
*shmem)
{
struct drm_gem_object *obj = >base;
+ drm_WARN_ON_ONCE(obj->dev, !drm_gem_shmem_is_purgeable(shmem));
+
drm_gem_shmem_shrinker_put_pages_locked(shmem);
drm_gem_free_mmap_offset(obj);
diff --git a/drivers/gpu/drm/panfrost/Makefile
b/drivers/gpu/drm/panfrost/Makefile
index 2c01c1e7523e..f2cb1ab0a32d 100644
--- a/drivers/gpu/drm/panfrost/Makefile
+++ b/drivers/gpu/drm/panfrost/Makefile
@@ -5,7 +5,6 @@ panfrost-y := \
panfrost_device.o \
panfrost_devfreq.o \
panfrost_gem.o \
- panfrost_gem_shrinker.o \
panfrost_gpu.o \
panfrost_job.o \
panfrost_mmu.o \
diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h
b/drivers/gpu/drm/panfrost/panfrost_device.h
index 62f7e3527385..cea6df9cd650 100644
--- a/drivers/gpu/drm/panfrost/panfrost_device.h
+++ b/drivers/gpu/drm/panfrost/panfrost_device.h
@@ -140,10 +140,6 @@ struct panfrost_device {
atomic_t pending;
} reset;
- struct mutex shrinker_lock;
- struct list_head shrinker_list;
- struct shrinker *shrinker;
-
struct panfrost_devfreq pfdevfreq;
struct {
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c
b/drivers/gpu/drm/panfrost/panfrost_drv.c
index a15d62f19afb..5c730d15a24d 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -171,7 +171,6 @@ panfrost_lookup_bos(struct drm_device *dev,
break;
}
- atomic_inc(>gpu_usecount);
job->mappings[i] = mapping;
}
@@ -397,7 +396,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev,
void *data,
{
struct panfrost_file_priv *priv = file_priv->driver_priv;
struct drm_panfrost_madvise *args = data;
- struct panfrost_device *pfdev = dev->dev_private;
struct drm_gem_object *gem_obj;
struct panfrost_gem_object *bo;
int ret = 0;
@@ -410,11 +408,15 @@ static int panfrost_ioctl_madvise(struct drm_device *dev,
void *data,
bo = to_panfrost_bo(gem_obj);
+ if (bo->is_heap) {
+ args->retained = 1;
+ goto out_put_object;
+ }
+
ret = dma_resv_lock_interruptible(bo->base.base.resv, NULL);
if (ret)
goto out_put_object;
- mutex_lock(>shrinker_lock);
mutex_lock(>mappings.lock);
if (args->madv == PANFROST_MADV_DONTNEED) {
struct panfrost_gem_mapping *first;
@@ -440,17 +442,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev,
void *data,
args->retained = drm_gem_shmem_madvise_locked(>base, args->madv);
- if (args->retained) {
- if (args->madv == PANFROST_MADV_DONTNEED)
- list_move_tail(>base.madv_list,
- >shrinker_list);
- else if (args->madv == PANFROST_MADV_WILLNEED)
- list_del_init(>base.madv_list);
- }
-
out_unlock_mappings:
mutex_unlock(>mappings.lock);
- mutex_unlock(>shrinker_lock);
dma_resv_unlock(bo->base.base.resv);
out_put_object:
drm_gem_object_put(gem_obj);
@@ -635,9 +628,6 @@ static int panfrost_probe(struct platform_device *pdev)
ddev->dev_private = pfdev;
pfdev->ddev = ddev;
- mutex_init(>shrinker_lock);
- INIT_LIS
[PATCH v19 29/30] drm/virtio: Support shmem shrinking
Support generic drm-shmem memory shrinker and add new madvise IOCTL to
the VirtIO-GPU driver. BO cache manager of Mesa driver will mark BOs as
"don't need" using the new IOCTL to let shrinker purge the marked BOs on
OOM, the shrinker will also evict unpurgeable shmem BOs from memory if
guest supports SWAP file or partition.
Link: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15278
Acked-by: Gerd Hoffmann
Signed-off-by: Daniel Almeida
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/virtio/virtgpu_drv.h| 13 +-
drivers/gpu/drm/virtio/virtgpu_gem.c| 48 +--
drivers/gpu/drm/virtio/virtgpu_ioctl.c | 25 ++
drivers/gpu/drm/virtio/virtgpu_kms.c| 8
drivers/gpu/drm/virtio/virtgpu_object.c | 61 +
drivers/gpu/drm/virtio/virtgpu_vq.c | 40
include/uapi/drm/virtgpu_drm.h | 14 ++
7 files changed, 204 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h
b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 1837dc7ea9fb..37188c00e161 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -283,7 +283,7 @@ struct virtio_gpu_fpriv {
};
/* virtgpu_ioctl.c */
-#define DRM_VIRTIO_NUM_IOCTLS 12
+#define DRM_VIRTIO_NUM_IOCTLS 13
extern struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS];
void virtio_gpu_create_context(struct drm_device *dev, struct drm_file *file);
@@ -321,6 +321,8 @@ void virtio_gpu_array_put_free_delayed(struct
virtio_gpu_device *vgdev,
void virtio_gpu_array_put_free_work(struct work_struct *work);
int virtio_gpu_array_prepare(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object_array *objs);
+int virtio_gpu_gem_host_mem_release(struct virtio_gpu_object *bo);
+int virtio_gpu_gem_madvise(struct virtio_gpu_object *obj, int madv);
int virtio_gpu_gem_pin(struct virtio_gpu_object *bo);
void virtio_gpu_gem_unpin(struct virtio_gpu_object *bo);
@@ -334,6 +336,8 @@ void virtio_gpu_cmd_create_resource(struct
virtio_gpu_device *vgdev,
struct virtio_gpu_fence *fence);
void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object *bo);
+int virtio_gpu_cmd_release_resource(struct virtio_gpu_device *vgdev,
+ struct virtio_gpu_object *bo);
void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
uint64_t offset,
uint32_t width, uint32_t height,
@@ -354,6 +358,9 @@ void virtio_gpu_object_attach(struct virtio_gpu_device
*vgdev,
struct virtio_gpu_object *obj,
struct virtio_gpu_mem_entry *ents,
unsigned int nents);
+void virtio_gpu_object_detach(struct virtio_gpu_device *vgdev,
+ struct virtio_gpu_object *obj,
+ struct virtio_gpu_fence *fence);
void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,
struct virtio_gpu_output *output);
int virtio_gpu_cmd_get_display_info(struct virtio_gpu_device *vgdev);
@@ -497,4 +504,8 @@ void virtio_gpu_vram_unmap_dma_buf(struct device *dev,
int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
struct drm_file *file);
+/* virtgpu_gem_shrinker.c */
+int virtio_gpu_gem_shrinker_init(struct virtio_gpu_device *vgdev);
+void virtio_gpu_gem_shrinker_fini(struct virtio_gpu_device *vgdev);
+
#endif
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c
b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 97e67064c97e..68d27ae582ba 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -147,10 +147,20 @@ void virtio_gpu_gem_object_close(struct drm_gem_object
*obj,
struct virtio_gpu_device *vgdev = obj->dev->dev_private;
struct virtio_gpu_fpriv *vfpriv = file->driver_priv;
struct virtio_gpu_object_array *objs;
+ struct virtio_gpu_object *bo;
if (!vgdev->has_virgl_3d)
return;
+ bo = gem_to_virtio_gpu_obj(obj);
+
+ /*
+* Purged BO was already detached and released, the resource ID
+* is invalid by now.
+*/
+ if (!virtio_gpu_gem_madvise(bo, VIRTGPU_MADV_WILLNEED))
+ return;
+
objs = virtio_gpu_array_alloc(1);
if (!objs)
return;
@@ -305,16 +315,46 @@ int virtio_gpu_array_prepare(struct virtio_gpu_device
*vgdev,
for (i = 0; i < objs->nents; i++) {
bo = gem_to_virtio_gpu_obj(objs->objs[i]);
- if (virtio_gpu_is_shmem(bo) && bo->detached) {
- ret = virtio_gpu_reattach_shmem_object_locked(bo);
- if (
[PATCH v19 26/30] drm/shmem-helper: Turn warnings about imported GEM into errors
Turn sanity warnings about DRM-SHMEM API misuse into a error conditions
for cases where imported GEM is used when it shouldn't be used.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 0d95d723b90d..7d2fe12bd793 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -409,7 +409,8 @@ int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
struct drm_gem_object *obj = >base;
int ret;
- drm_WARN_ON(obj->dev, obj->import_attach);
+ if (drm_WARN_ON(obj->dev, obj->import_attach))
+ return -EINVAL;
if (refcount_inc_not_zero(>pages_pin_count))
return 0;
@@ -872,7 +873,8 @@ struct sg_table *drm_gem_shmem_get_sg_table(struct
drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
- drm_WARN_ON(obj->dev, obj->import_attach);
+ if (drm_WARN_ON(obj->dev, obj->import_attach))
+ return ERR_PTR(-EINVAL);
if (drm_WARN_ON(obj->dev, !shmem->pages))
return ERR_PTR(-ENOMEM);
@@ -909,7 +911,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt_locked(struct
drm_gem_shmem_object
if (shmem->sgt)
return shmem->sgt;
- drm_WARN_ON(obj->dev, obj->import_attach);
+ if (drm_WARN_ON(obj->dev, obj->import_attach))
+ return ERR_PTR(-EINVAL);
sgt = drm_gem_shmem_get_sg_table(shmem);
if (IS_ERR(sgt))
--
2.43.0
[PATCH v19 27/30] drm/virtio: Pin display framebuffer BO
Prepare to addition of memory shrinker support by pinning display
framebuffer BO pages in memory while they are in use by display on host.
Shrinker is free to relocate framebuffer BO pages if it doesn't know that
pages are in use, thus pin the pages to disallow shrinker to move them.
Acked-by: Gerd Hoffmann
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/virtio/virtgpu_drv.h | 2 ++
drivers/gpu/drm/virtio/virtgpu_gem.c | 19 +++
drivers/gpu/drm/virtio/virtgpu_plane.c | 17 +++--
3 files changed, 36 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h
b/drivers/gpu/drm/virtio/virtgpu_drv.h
index bb7d86a0c6a1..83d1e4622292 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -318,6 +318,8 @@ void virtio_gpu_array_put_free(struct
virtio_gpu_object_array *objs);
void virtio_gpu_array_put_free_delayed(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object_array *objs);
void virtio_gpu_array_put_free_work(struct work_struct *work);
+int virtio_gpu_gem_pin(struct virtio_gpu_object *bo);
+void virtio_gpu_gem_unpin(struct virtio_gpu_object *bo);
/* virtgpu_vq.c */
int virtio_gpu_alloc_vbufs(struct virtio_gpu_device *vgdev);
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c
b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 7db48d17ee3a..625c05d625bf 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -294,3 +294,22 @@ void virtio_gpu_array_put_free_work(struct work_struct
*work)
}
spin_unlock(>obj_free_lock);
}
+
+int virtio_gpu_gem_pin(struct virtio_gpu_object *bo)
+{
+ int err;
+
+ if (virtio_gpu_is_shmem(bo)) {
+ err = drm_gem_shmem_pin(>base);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+void virtio_gpu_gem_unpin(struct virtio_gpu_object *bo)
+{
+ if (virtio_gpu_is_shmem(bo))
+ drm_gem_shmem_unpin(>base);
+}
diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c
b/drivers/gpu/drm/virtio/virtgpu_plane.c
index a72a2dbda031..162fb8a44d71 100644
--- a/drivers/gpu/drm/virtio/virtgpu_plane.c
+++ b/drivers/gpu/drm/virtio/virtgpu_plane.c
@@ -248,20 +248,28 @@ static int virtio_gpu_plane_prepare_fb(struct drm_plane
*plane,
struct virtio_gpu_device *vgdev = dev->dev_private;
struct virtio_gpu_framebuffer *vgfb;
struct virtio_gpu_object *bo;
+ int err;
if (!new_state->fb)
return 0;
vgfb = to_virtio_gpu_framebuffer(new_state->fb);
bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
- if (!bo || (plane->type == DRM_PLANE_TYPE_PRIMARY && !bo->guest_blob))
+
+ err = virtio_gpu_gem_pin(bo);
+ if (err)
+ return err;
+
+ if (plane->type == DRM_PLANE_TYPE_PRIMARY && !bo->guest_blob)
return 0;
if (bo->dumb && (plane->state->fb != new_state->fb)) {
vgfb->fence = virtio_gpu_fence_alloc(vgdev,
vgdev->fence_drv.context,
0);
- if (!vgfb->fence)
+ if (!vgfb->fence) {
+ virtio_gpu_gem_unpin(bo);
return -ENOMEM;
+ }
}
return 0;
@@ -271,15 +279,20 @@ static void virtio_gpu_plane_cleanup_fb(struct drm_plane
*plane,
struct drm_plane_state *state)
{
struct virtio_gpu_framebuffer *vgfb;
+ struct virtio_gpu_object *bo;
if (!state->fb)
return;
vgfb = to_virtio_gpu_framebuffer(state->fb);
+ bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
+
if (vgfb->fence) {
dma_fence_put(>fence->f);
vgfb->fence = NULL;
}
+
+ virtio_gpu_gem_unpin(bo);
}
static void virtio_gpu_cursor_plane_update(struct drm_plane *plane,
--
2.43.0
[PATCH v19 28/30] drm/virtio: Attach shmem BOs dynamically
Prepare for addition of memory shrinker support by attaching shmem pages
to host dynamically on first use. Previously the attachment vq command
wasn't fenced and there was no vq kick made in the BO creation code path,
hence the attachment already was happening dynamically, but implicitly.
Making attachment explicitly dynamic will allow to simplify and reuse more
code when shrinker will be added. The virtio_gpu_object_shmem_init() now
works under the held reservation lock, which will be important to have for
shrinker to avoid moving pages while they are in active use by the driver.
Acked-by: Gerd Hoffmann
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/virtio/virtgpu_drv.h| 7 +++
drivers/gpu/drm/virtio/virtgpu_gem.c| 26 +
drivers/gpu/drm/virtio/virtgpu_ioctl.c | 32 +++
drivers/gpu/drm/virtio/virtgpu_object.c | 73 -
drivers/gpu/drm/virtio/virtgpu_submit.c | 15 -
5 files changed, 125 insertions(+), 28 deletions(-)
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h
b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 83d1e4622292..1837dc7ea9fb 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -92,6 +92,7 @@ struct virtio_gpu_object {
uint32_t hw_res_handle;
bool dumb;
bool created;
+ bool detached;
bool host3d_blob, guest_blob;
uint32_t blob_mem, blob_flags;
@@ -318,6 +319,8 @@ void virtio_gpu_array_put_free(struct
virtio_gpu_object_array *objs);
void virtio_gpu_array_put_free_delayed(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object_array *objs);
void virtio_gpu_array_put_free_work(struct work_struct *work);
+int virtio_gpu_array_prepare(struct virtio_gpu_device *vgdev,
+struct virtio_gpu_object_array *objs);
int virtio_gpu_gem_pin(struct virtio_gpu_object *bo);
void virtio_gpu_gem_unpin(struct virtio_gpu_object *bo);
@@ -458,6 +461,10 @@ int virtio_gpu_object_create(struct virtio_gpu_device
*vgdev,
bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
+int virtio_gpu_reattach_shmem_object_locked(struct virtio_gpu_object *bo);
+
+int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
+
int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
uint32_t *resid);
/* virtgpu_prime.c */
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c
b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 625c05d625bf..97e67064c97e 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -295,6 +295,26 @@ void virtio_gpu_array_put_free_work(struct work_struct
*work)
spin_unlock(>obj_free_lock);
}
+int virtio_gpu_array_prepare(struct virtio_gpu_device *vgdev,
+struct virtio_gpu_object_array *objs)
+{
+ struct virtio_gpu_object *bo;
+ int ret = 0;
+ u32 i;
+
+ for (i = 0; i < objs->nents; i++) {
+ bo = gem_to_virtio_gpu_obj(objs->objs[i]);
+
+ if (virtio_gpu_is_shmem(bo) && bo->detached) {
+ ret = virtio_gpu_reattach_shmem_object_locked(bo);
+ if (ret)
+ break;
+ }
+ }
+
+ return ret;
+}
+
int virtio_gpu_gem_pin(struct virtio_gpu_object *bo)
{
int err;
@@ -303,6 +323,12 @@ int virtio_gpu_gem_pin(struct virtio_gpu_object *bo)
err = drm_gem_shmem_pin(>base);
if (err)
return err;
+
+ err = virtio_gpu_reattach_shmem_object(bo);
+ if (err) {
+ drm_gem_shmem_unpin(>base);
+ return err;
+ }
}
return 0;
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index e4f76f315550..c7da22006149 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -256,6 +256,10 @@ static int virtio_gpu_transfer_from_host_ioctl(struct
drm_device *dev,
if (ret != 0)
goto err_put_free;
+ ret = virtio_gpu_array_prepare(vgdev, objs);
+ if (ret)
+ goto err_unlock;
+
fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
if (!fence) {
ret = -ENOMEM;
@@ -298,11 +302,25 @@ static int virtio_gpu_transfer_to_host_ioctl(struct
drm_device *dev, void *data,
goto err_put_free;
}
+ ret = virtio_gpu_array_lock_resv(objs);
+ if (ret != 0)
+ goto err_put_free;
+
+ ret = virtio_gpu_array_prepare(vgdev, objs);
+ if (ret)
+ goto err_unlock;
+
+ fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
+ if (!fence) {
+ ret = -ENOMEM;
+ goto err_unlock;
+ }
[PATCH v19 25/30] drm/shmem-helper: Don't free refcounted GEM
Don't free shmem object if it has pages that are in use at the time of the GEM's freeing if DRM driver doesn't manage GEM/pages lifetime properly. This prevents memory corruption due to the use-after-free bug in exchange to leaking GEM. Signed-off-by: Dmitry Osipenko --- drivers/gpu/drm/drm_gem_shmem_helper.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c index e6e6e693ab95..0d95d723b90d 100644 --- a/drivers/gpu/drm/drm_gem_shmem_helper.c +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c @@ -205,9 +205,15 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem) if (obj->import_attach) drm_prime_gem_destroy(obj, shmem->sgt); - drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count)); - drm_WARN_ON(obj->dev, refcount_read(>pages_use_count)); - drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count)); + /* +* Prevent memory corruption caused by the use-after-free bug in a +* case where shmem user erroneously holds reference to pages while +* GEM is freed by leaking the GEM. +*/ + if (drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count)) || + drm_WARN_ON(obj->dev, refcount_read(>pages_use_count)) || + drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count))) + return; drm_gem_object_release(obj); kfree(shmem); -- 2.43.0
[PATCH v19 24/30] drm/shmem-helper: Optimize unlocked get_pages_sgt()
SGT isn't refcounted. Once SGT pointer has been obtained, it remains the same for both locked and unlocked get_pages_sgt(). Return cached SGT directly without taking a potentially expensive lock. Signed-off-by: Dmitry Osipenko --- drivers/gpu/drm/drm_gem_shmem_helper.c | 12 1 file changed, 12 insertions(+) diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c index 8fd7851c088b..e6e6e693ab95 100644 --- a/drivers/gpu/drm/drm_gem_shmem_helper.c +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c @@ -962,6 +962,18 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem) drm_WARN_ON(obj->dev, drm_gem_shmem_is_purgeable(shmem))) return ERR_PTR(-EBUSY); + /* +* Drivers that use shrinker should take into account that shrinker +* may relocate BO, thus invalidating the returned SGT pointer. +* Such drivers should pin GEM while they use SGT. +* +* Drivers that don't use shrinker should take into account that +* SGT is released together with the GEM pages. Pages should be kept +* alive while SGT is used. +*/ + if (shmem->sgt) + return shmem->sgt; + ret = dma_resv_lock_interruptible(shmem->base.resv, NULL); if (ret) return ERR_PTR(ret); -- 2.43.0
[PATCH v19 23/30] drm/shmem-helper: Export drm_gem_shmem_get_pages_sgt_locked()
Export drm_gem_shmem_get_pages_sgt_locked() that will be used by virtio-gpu
shrinker during GEM swap-in operation done under the held reservation lock.
Reviewed-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 22 +-
include/drm/drm_gem_shmem_helper.h | 1 +
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 59cebd1e35af..8fd7851c088b 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -875,12 +875,31 @@ struct sg_table *drm_gem_shmem_get_sg_table(struct
drm_gem_shmem_object *shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_get_sg_table);
-static struct sg_table *drm_gem_shmem_get_pages_sgt_locked(struct
drm_gem_shmem_object *shmem)
+/**
+ * drm_gem_shmem_get_pages_sgt_locked - Provide a scatter/gather table of
pinned
+ * pages for a shmem GEM object
+ * @shmem: shmem GEM object
+ *
+ * This is a locked version of @drm_gem_shmem_get_sg_table that exports a
+ * scatter/gather table suitable for PRIME usage by calling the standard
+ * DMA mapping API.
+ *
+ * Drivers must hold GEM's reservation lock when using this function.
+ *
+ * Drivers who need to acquire an scatter/gather table for objects need to call
+ * drm_gem_shmem_get_pages_sgt() instead.
+ *
+ * Returns:
+ * A pointer to the scatter/gather table of pinned pages or error pointer on
failure.
+ */
+struct sg_table *drm_gem_shmem_get_pages_sgt_locked(struct
drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
int ret;
struct sg_table *sgt;
+ dma_resv_assert_held(shmem->base.resv);
+
if (shmem->sgt)
return shmem->sgt;
@@ -904,6 +923,7 @@ static struct sg_table
*drm_gem_shmem_get_pages_sgt_locked(struct drm_gem_shmem_
kfree(sgt);
return ERR_PTR(ret);
}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_get_pages_sgt_locked);
/**
* drm_gem_shmem_get_pages_sgt - Pin pages, dma map them, and return a
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index df97c11fc99a..167f00f089de 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -149,6 +149,7 @@ void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object
*shmem);
struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object
*shmem);
struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object
*shmem);
+struct sg_table *drm_gem_shmem_get_pages_sgt_locked(struct
drm_gem_shmem_object *shmem);
void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
struct drm_printer *p, unsigned int indent);
--
2.43.0
[PATCH v19 21/30] drm/shmem-helper: Change sgt allocation policy
In a preparation to addition of drm-shmem memory shrinker support, change
the SGT allocation policy in this way:
1. SGT can be allocated only if shmem pages are pinned at the
time of allocation, otherwise allocation fails.
2. Drivers must ensure that pages are pinned during the time of SGT usage
and should get new SGT if pages were unpinned.
This new policy is required by the shrinker because it will move pages
to/from SWAP unless pages are pinned, invalidating SGT pointer once pages
are relocated.
Previous patches prepared drivers to the new policy.
Reviewed-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 55 ++
1 file changed, 29 insertions(+), 26 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index c7357110ca76..ff5437ab2c95 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -133,6 +133,14 @@ drm_gem_shmem_free_pages(struct drm_gem_shmem_object
*shmem)
{
struct drm_gem_object *obj = >base;
+ if (shmem->sgt) {
+ dma_unmap_sgtable(obj->dev->dev, shmem->sgt,
+ DMA_BIDIRECTIONAL, 0);
+ sg_free_table(shmem->sgt);
+ kfree(shmem->sgt);
+ shmem->sgt = NULL;
+ }
+
#ifdef CONFIG_X86
if (shmem->map_wc)
set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
@@ -155,24 +163,12 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object
*shmem)
{
struct drm_gem_object *obj = >base;
- if (obj->import_attach) {
+ if (obj->import_attach)
drm_prime_gem_destroy(obj, shmem->sgt);
- } else {
- drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count));
- if (shmem->sgt) {
- dma_unmap_sgtable(obj->dev->dev, shmem->sgt,
- DMA_BIDIRECTIONAL, 0);
- sg_free_table(shmem->sgt);
- kfree(shmem->sgt);
- }
- if (shmem->pages &&
- refcount_dec_and_test(>pages_use_count))
- drm_gem_shmem_free_pages(shmem);
-
- drm_WARN_ON(obj->dev, refcount_read(>pages_use_count));
- drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count));
- }
+ drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count));
+ drm_WARN_ON(obj->dev, refcount_read(>pages_use_count));
+ drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count));
drm_gem_object_release(obj);
kfree(shmem);
@@ -722,6 +718,9 @@ struct sg_table *drm_gem_shmem_get_sg_table(struct
drm_gem_shmem_object *shmem)
drm_WARN_ON(obj->dev, obj->import_attach);
+ if (drm_WARN_ON(obj->dev, !shmem->pages))
+ return ERR_PTR(-ENOMEM);
+
return drm_prime_pages_to_sg(obj->dev, shmem->pages, obj->size >>
PAGE_SHIFT);
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_get_sg_table);
@@ -737,15 +736,10 @@ static struct sg_table
*drm_gem_shmem_get_pages_sgt_locked(struct drm_gem_shmem_
drm_WARN_ON(obj->dev, obj->import_attach);
- ret = drm_gem_shmem_get_pages_locked(shmem);
- if (ret)
- return ERR_PTR(ret);
-
sgt = drm_gem_shmem_get_sg_table(shmem);
- if (IS_ERR(sgt)) {
- ret = PTR_ERR(sgt);
- goto err_put_pages;
- }
+ if (IS_ERR(sgt))
+ return sgt;
+
/* Map the pages for use by the h/w. */
ret = dma_map_sgtable(obj->dev->dev, sgt, DMA_BIDIRECTIONAL, 0);
if (ret)
@@ -758,8 +752,6 @@ static struct sg_table
*drm_gem_shmem_get_pages_sgt_locked(struct drm_gem_shmem_
err_free_sgt:
sg_free_table(sgt);
kfree(sgt);
-err_put_pages:
- drm_gem_shmem_put_pages_locked(shmem);
return ERR_PTR(ret);
}
@@ -776,6 +768,17 @@ static struct sg_table
*drm_gem_shmem_get_pages_sgt_locked(struct drm_gem_shmem_
* and difference between dma-buf imported and natively allocated objects.
* drm_gem_shmem_get_sg_table() should not be directly called by drivers.
*
+ * Drivers should adhere to these SGT usage rules:
+ *
+ * 1. SGT should be allocated only if shmem pages are pinned at the
+ *time of allocation, otherwise allocation will fail.
+ *
+ * 2. Drivers should ensure that pages are pinned during the time of
+ *SGT usage and should get new SGT if pages were unpinned.
+ *
+ * Drivers don't own returned SGT and must take care of the SGT pointer
+ * lifetime. SGT is valid as long as GEM pages that backing SGT are pinned.
+ *
* Returns:
* A pointer to the scatter/gather table of pinned pages or errno on failure.
*/
--
2.43.0
[PATCH v19 22/30] drm/shmem-helper: Add common memory shrinker
Introduce common drm-shmem shrinker for DRM drivers.
To start using drm-shmem shrinker drivers should do the following:
1. Implement evict() callback of GEM object where driver should check
whether object is purgeable or evictable using drm-shmem helpers and
perform the shrinking action
2. Initialize drm-shmem internals using drmm_gem_shmem_init(drm_device),
which will register drm-shmem shrinker
3. Implement madvise IOCTL that will use drm_gem_shmem_madvise()
Signed-off-by: Daniel Almeida
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c| 365 +-
drivers/gpu/drm/panfrost/panfrost_gem.c | 3 +-
.../gpu/drm/panfrost/panfrost_gem_shrinker.c | 13 +-
include/drm/drm_device.h | 10 +-
include/drm/drm_gem_shmem_helper.h| 68 +++-
5 files changed, 433 insertions(+), 26 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index ff5437ab2c95..59cebd1e35af 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -20,6 +20,7 @@
#include
#include
#include
+#include
#include
#include
@@ -128,11 +129,49 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct
drm_device *dev, size_t
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
+static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
+{
+ return (shmem->madv >= 0) && shmem->base.funcs->evict &&
+ refcount_read(>pages_use_count) &&
+ !refcount_read(>pages_pin_count) &&
+ !shmem->base.dma_buf && !shmem->base.import_attach &&
+ !shmem->evicted;
+}
+
+static void
+drm_gem_shmem_shrinker_update_lru_locked(struct drm_gem_shmem_object *shmem)
+{
+ struct drm_gem_object *obj = >base;
+ struct drm_gem_shmem *shmem_mm = obj->dev->shmem_mm;
+ struct drm_gem_shmem_shrinker *shmem_shrinker = _mm->shrinker;
+
+ dma_resv_assert_held(shmem->base.resv);
+
+ if (!shmem_shrinker || obj->import_attach)
+ return;
+
+ if (shmem->madv < 0)
+ drm_gem_lru_remove(>base);
+ else if (drm_gem_shmem_is_evictable(shmem) ||
drm_gem_shmem_is_purgeable(shmem))
+ drm_gem_lru_move_tail(_shrinker->lru_evictable,
>base);
+ else if (shmem->evicted)
+ drm_gem_lru_move_tail(_shrinker->lru_evicted,
>base);
+ else if (!shmem->pages)
+ drm_gem_lru_remove(>base);
+ else
+ drm_gem_lru_move_tail(_shrinker->lru_pinned,
>base);
+}
+
static void
drm_gem_shmem_free_pages(struct drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
+ if (!shmem->pages) {
+ drm_WARN_ON(obj->dev, !shmem->evicted && shmem->madv >= 0);
+ return;
+ }
+
if (shmem->sgt) {
dma_unmap_sgtable(obj->dev->dev, shmem->sgt,
DMA_BIDIRECTIONAL, 0);
@@ -175,15 +214,26 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object
*shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
-static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
+static int
+drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
struct page **pages;
+ if (drm_WARN_ON(obj->dev, obj->import_attach))
+ return -EINVAL;
+
dma_resv_assert_held(shmem->base.resv);
- if (refcount_inc_not_zero(>pages_use_count))
+ if (shmem->madv < 0) {
+ drm_WARN_ON(obj->dev, shmem->pages);
+ return -ENOMEM;
+ }
+
+ if (shmem->pages) {
+ drm_WARN_ON(obj->dev, !shmem->evicted);
return 0;
+ }
pages = drm_gem_get_pages(obj);
if (IS_ERR(pages)) {
@@ -204,8 +254,29 @@ static int drm_gem_shmem_get_pages_locked(struct
drm_gem_shmem_object *shmem)
shmem->pages = pages;
+ return 0;
+}
+
+static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
+{
+ int err;
+
+ dma_resv_assert_held(shmem->base.resv);
+
+ if (shmem->madv < 0)
+ return -ENOMEM;
+
+ if (refcount_inc_not_zero(>pages_use_count))
+ return 0;
+
+ err = drm_gem_shmem_acquire_pages(shmem);
+ if (err)
+ return err;
+
refcount_set(>pages_use_count, 1);
+ drm_gem_shmem_shrinker_update_lru_locked(shmem);
+
return 0;
}
@@ -222,6 +293,8 @@ void drm_gem_shmem_put_pages_locked(struct
drm_gem_shmem_object *shmem)
if (refcount_dec_and_test(>pages_use_count))
drm_gem_shmem_free_pages(shmem);
+
[PATCH v19 19/30] drm/virtio: Explicitly get and put drm-shmem pages
We're moving away from implicit get_pages() that is done by
get_pages_sgt() to simplify the refcnt handling. Drivers will have
to pin pages while they use sgt. VirtIO-GPU doesn't support shrinker,
hence pages are pinned and sgt is valid as long as pages' use-count > 0.
Reviewed-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/virtio/virtgpu_object.c | 9 -
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c
b/drivers/gpu/drm/virtio/virtgpu_object.c
index c7e74cf13022..e58528c562ef 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -67,6 +67,7 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo)
virtio_gpu_resource_id_put(vgdev, bo->hw_res_handle);
if (virtio_gpu_is_shmem(bo)) {
+ drm_gem_shmem_put_pages(>base);
drm_gem_shmem_free(>base);
} else if (virtio_gpu_is_vram(bo)) {
struct virtio_gpu_object_vram *vram = to_virtio_gpu_vram(bo);
@@ -196,9 +197,13 @@ int virtio_gpu_object_create(struct virtio_gpu_device
*vgdev,
return PTR_ERR(shmem_obj);
bo = gem_to_virtio_gpu_obj(_obj->base);
+ ret = drm_gem_shmem_get_pages(shmem_obj);
+ if (ret)
+ goto err_free_gem;
+
ret = virtio_gpu_resource_id_get(vgdev, >hw_res_handle);
if (ret < 0)
- goto err_free_gem;
+ goto err_put_pages;
bo->dumb = params->dumb;
@@ -243,6 +248,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device
*vgdev,
kvfree(ents);
err_put_id:
virtio_gpu_resource_id_put(vgdev, bo->hw_res_handle);
+err_put_pages:
+ drm_gem_shmem_put_pages(shmem_obj);
err_free_gem:
drm_gem_shmem_free(shmem_obj);
return ret;
--
2.43.0
[PATCH v19 20/30] drm/v3d: Explicitly get and put drm-shmem pages
To simplify the drm-shmem refcnt handling, we're moving away from the implicit get_pages() that is used by get_pages_sgt(). From now on drivers will have to pin pages while they use sgt. V3D driver doesn't support shrinker, hence pages are pinned and sgt is valid as long as pages' use-count > 0. Reviewed-by: Boris Brezillon Signed-off-by: Dmitry Osipenko --- drivers/gpu/drm/v3d/v3d_bo.c | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/v3d/v3d_bo.c b/drivers/gpu/drm/v3d/v3d_bo.c index 1bdfac8beafd..ccf04ce93e8c 100644 --- a/drivers/gpu/drm/v3d/v3d_bo.c +++ b/drivers/gpu/drm/v3d/v3d_bo.c @@ -50,6 +50,9 @@ void v3d_free_object(struct drm_gem_object *obj) /* GPU execution may have dirtied any pages in the BO. */ bo->base.pages_mark_dirty_on_put = true; + if (!obj->import_attach) + drm_gem_shmem_put_pages(>base); + drm_gem_shmem_free(>base); } @@ -139,12 +142,18 @@ struct v3d_bo *v3d_bo_create(struct drm_device *dev, struct drm_file *file_priv, bo = to_v3d_bo(_obj->base); bo->vaddr = NULL; - ret = v3d_bo_create_finish(_obj->base); + ret = drm_gem_shmem_get_pages(shmem_obj); if (ret) goto free_obj; + ret = v3d_bo_create_finish(_obj->base); + if (ret) + goto put_pages; + return bo; +put_pages: + drm_gem_shmem_put_pages(shmem_obj); free_obj: drm_gem_shmem_free(shmem_obj); return ERR_PTR(ret); -- 2.43.0
[PATCH v19 13/30] drm/shmem-helper: Make drm_gem_shmem_get_pages() public
We're going to move away from having implicit get_pages() done by
get_pages_sgt() to simplify refcnt handling. Drivers will manage
get/put_pages() by themselves. Expose the drm_gem_shmem_get_pages()
in a public drm-shmem API.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 10 +-
include/drm/drm_gem_shmem_helper.h | 1 +
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 799a3c5015ad..dc416a4bce1b 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -228,7 +228,14 @@ void drm_gem_shmem_put_pages_locked(struct
drm_gem_shmem_object *shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
-static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
+/*
+ * drm_gem_shmem_get_pages - Increase use count on the backing pages for a
shmem GEM object
+ * @shmem: shmem GEM object
+ *
+ * This function Increases the use count and allocates the backing pages if
+ * use-count equals to zero.
+ */
+int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
{
int ret;
@@ -241,6 +248,7 @@ static int drm_gem_shmem_get_pages(struct
drm_gem_shmem_object *shmem)
return ret;
}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_get_pages);
static int drm_gem_shmem_pin_locked(struct drm_gem_shmem_object *shmem)
{
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index 18020f653d7e..6dedc0739fbc 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -110,6 +110,7 @@ struct drm_gem_shmem_object {
struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev,
size_t size);
void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem);
+int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem);
void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem);
int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem);
void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem);
--
2.43.0
[PATCH v19 18/30] drm/panfrost: Explicitly get and put drm-shmem pages
To simplify the drm-shmem refcnt handling, we're moving away from
the implicit get_pages() that is used by get_pages_sgt(). From now on
drivers will have to pin pages while they use sgt. Panfrost's shrinker
doesn't support swapping out BOs, hence pages are pinned and sgt is valid
as long as pages' use-count > 0.
In Panfrost, panfrost_gem_mapping, which is the object representing a
GPU mapping of a BO, owns a pages ref. This guarantees that any BO being
mapped GPU side has its pages retained till the mapping is destroyed.
Since pages are no longer guaranteed to stay pinned for the BO lifetime,
and MADVISE(DONT_NEED) flagging remains after the GEM handle has been
destroyed, we need to add an extra 'is_purgeable' check in
panfrost_gem_purge(), to make sure we're not trying to purge a BO that
already had its pages released.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/panfrost/panfrost_gem.c | 63 ++-
.../gpu/drm/panfrost/panfrost_gem_shrinker.c | 6 ++
2 files changed, 52 insertions(+), 17 deletions(-)
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c
b/drivers/gpu/drm/panfrost/panfrost_gem.c
index f268bd5c2884..7edfc12f7c1f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gem.c
@@ -35,20 +35,6 @@ static void panfrost_gem_free_object(struct drm_gem_object
*obj)
*/
WARN_ON_ONCE(!list_empty(>mappings.list));
- if (bo->sgts) {
- int i;
- int n_sgt = bo->base.base.size / SZ_2M;
-
- for (i = 0; i < n_sgt; i++) {
- if (bo->sgts[i].sgl) {
- dma_unmap_sgtable(pfdev->dev, >sgts[i],
- DMA_BIDIRECTIONAL, 0);
- sg_free_table(>sgts[i]);
- }
- }
- kvfree(bo->sgts);
- }
-
drm_gem_shmem_free(>base);
}
@@ -85,11 +71,40 @@ panfrost_gem_teardown_mapping(struct panfrost_gem_mapping
*mapping)
static void panfrost_gem_mapping_release(struct kref *kref)
{
- struct panfrost_gem_mapping *mapping;
-
- mapping = container_of(kref, struct panfrost_gem_mapping, refcount);
+ struct panfrost_gem_mapping *mapping =
+ container_of(kref, struct panfrost_gem_mapping, refcount);
+ struct panfrost_gem_object *bo = mapping->obj;
+ struct panfrost_device *pfdev = bo->base.base.dev->dev_private;
panfrost_gem_teardown_mapping(mapping);
+
+ /* On heap BOs, release the sgts created in the fault handler path. */
+ if (bo->sgts) {
+ int i, n_sgt = bo->base.base.size / SZ_2M;
+
+ for (i = 0; i < n_sgt; i++) {
+ if (bo->sgts[i].sgl) {
+ dma_unmap_sgtable(pfdev->dev, >sgts[i],
+ DMA_BIDIRECTIONAL, 0);
+ sg_free_table(>sgts[i]);
+ }
+ }
+ kvfree(bo->sgts);
+ }
+
+ /* Pages ref is owned by the panfrost_gem_mapping object. We must
+* release our pages ref (if any), before releasing the object
+* ref.
+* Non-heap BOs acquired the pages at panfrost_gem_mapping creation
+* time, and heap BOs may have acquired pages if the fault handler
+* was called, in which case bo->sgts should be non-NULL.
+*/
+ if (!bo->base.base.import_attach && (!bo->is_heap || bo->sgts) &&
+ bo->base.madv >= 0) {
+ drm_gem_shmem_put_pages(>base);
+ bo->sgts = NULL;
+ }
+
drm_gem_object_put(>obj->base.base);
panfrost_mmu_ctx_put(mapping->mmu);
kfree(mapping);
@@ -125,6 +140,20 @@ int panfrost_gem_open(struct drm_gem_object *obj, struct
drm_file *file_priv)
if (!mapping)
return -ENOMEM;
+ if (!bo->is_heap && !bo->base.base.import_attach) {
+ /* Pages ref is owned by the panfrost_gem_mapping object.
+* For non-heap BOs, we request pages at mapping creation
+* time, such that the panfrost_mmu_map() call, further down in
+* this function, is guaranteed to have pages_use_count > 0
+* when drm_gem_shmem_get_pages_sgt() is called.
+*/
+ ret = drm_gem_shmem_get_pages(>base);
+ if (ret) {
+ kfree(mapping);
+ return ret;
+ }
+ }
+
INIT_LIST_HEAD(>node);
kref_init(>refcount);
drm_gem_object_get(obj);
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
index 02b60ea1433a..d4fb0854cf2f 100644
--- a/drivers/gpu/
[PATCH v19 17/30] drm/panfrost: Fix the error path in panfrost_mmu_map_fault_addr()
From: Boris Brezillon
If some the pages or sgt allocation failed, we shouldn't release the
pages ref we got earlier, otherwise we will end up with unbalanced
get/put_pages() calls. We should instead leave everything in place
and let the BO release function deal with extra cleanup when the object
is destroyed, or let the fault handler try again next time it's called.
Fixes: 187d2929206e ("drm/panfrost: Add support for GPU heap allocations")
Cc:
Signed-off-by: Boris Brezillon
Co-developed-by: Dmitry Osipenko
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/panfrost/panfrost_mmu.c | 13 +
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c
b/drivers/gpu/drm/panfrost/panfrost_mmu.c
index bd5a0073009d..4a0b4bf03f1a 100644
--- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
+++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
@@ -502,11 +502,18 @@ static int panfrost_mmu_map_fault_addr(struct
panfrost_device *pfdev, int as,
mapping_set_unevictable(mapping);
for (i = page_offset; i < page_offset + NUM_FAULT_PAGES; i++) {
+ /* Can happen if the last fault only partially filled this
+* section of the pages array before failing. In that case
+* we skip already filled pages.
+*/
+ if (pages[i])
+ continue;
+
pages[i] = shmem_read_mapping_page(mapping, i);
if (IS_ERR(pages[i])) {
ret = PTR_ERR(pages[i]);
pages[i] = NULL;
- goto err_pages;
+ goto err_unlock;
}
}
@@ -514,7 +521,7 @@ static int panfrost_mmu_map_fault_addr(struct
panfrost_device *pfdev, int as,
ret = sg_alloc_table_from_pages(sgt, pages + page_offset,
NUM_FAULT_PAGES, 0, SZ_2M, GFP_KERNEL);
if (ret)
- goto err_pages;
+ goto err_unlock;
ret = dma_map_sgtable(pfdev->dev, sgt, DMA_BIDIRECTIONAL, 0);
if (ret)
@@ -537,8 +544,6 @@ static int panfrost_mmu_map_fault_addr(struct
panfrost_device *pfdev, int as,
err_map:
sg_free_table(sgt);
-err_pages:
- drm_gem_shmem_put_pages_locked(>base);
err_unlock:
dma_resv_unlock(obj->resv);
err_bo:
--
2.43.0
[PATCH v19 15/30] drm/shmem-helper: Avoid lockdep warning when pages are released
All drivers will be moved to get/put pages explicitly and then the last
put_pages() will be invoked during gem_free() time by some drivers.
We can't touch reservation lock when GEM is freed because that will cause
a spurious warning from lockdep when shrinker support will be added.
Lockdep doesn't know that fs_reclaim isn't functioning for a freed object,
and thus, can't deadlock. Release pages directly without taking reservation
lock if GEM is freed and its refcount is zero.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 16
1 file changed, 16 insertions(+)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index f5ed64f78648..c7357110ca76 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -242,6 +242,22 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object
*shmem)
if (refcount_dec_not_one(>pages_use_count))
return;
+ /*
+* Destroying the object is a special case because acquiring
+* the obj lock can cause a locking order inversion between
+* reservation_ww_class_mutex and fs_reclaim.
+*
+* This deadlock is not actually possible, because no one should
+* be already holding the lock when GEM is released. Unfortunately
+* lockdep is not aware of this detail. So when the refcount drops
+* to zero, we pretend it is already locked.
+*/
+ if (!kref_read(>base.refcount)) {
+ if (refcount_dec_and_test(>pages_use_count))
+ drm_gem_shmem_free_pages(shmem);
+ return;
+ }
+
dma_resv_lock(shmem->base.resv, NULL);
drm_gem_shmem_put_pages_locked(shmem);
dma_resv_unlock(shmem->base.resv);
--
2.43.0
[PATCH v19 14/30] drm/shmem-helper: Add drm_gem_shmem_put_pages()
We're going to move away from having implicit get_pages() done by
get_pages_sgt() to ease simplify refcnt handling. Drivers will manage
get/put_pages() by themselves. Add drm_gem_shmem_put_pages().
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 20
include/drm/drm_gem_shmem_helper.h | 1 +
2 files changed, 21 insertions(+)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index dc416a4bce1b..f5ed64f78648 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -218,6 +218,7 @@ static int drm_gem_shmem_get_pages_locked(struct
drm_gem_shmem_object *shmem)
* @shmem: shmem GEM object
*
* This function decreases the use count and puts the backing pages when use
drops to zero.
+ * Caller must hold GEM's reservation lock.
*/
void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem)
{
@@ -228,6 +229,25 @@ void drm_gem_shmem_put_pages_locked(struct
drm_gem_shmem_object *shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
+/*
+ * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a
shmem GEM object
+ * @shmem: shmem GEM object
+ *
+ * This function decreases the use count and puts the backing pages when use
drops to zero.
+ * It's unlocked version of drm_gem_shmem_put_pages_locked(), caller must not
hold
+ * GEM's reservation lock.
+ */
+void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
+{
+ if (refcount_dec_not_one(>pages_use_count))
+ return;
+
+ dma_resv_lock(shmem->base.resv, NULL);
+ drm_gem_shmem_put_pages_locked(shmem);
+ dma_resv_unlock(shmem->base.resv);
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages);
+
/*
* drm_gem_shmem_get_pages - Increase use count on the backing pages for a
shmem GEM object
* @shmem: shmem GEM object
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index 6dedc0739fbc..525480488451 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -111,6 +111,7 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct
drm_device *dev, size_t
void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem);
int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem);
+void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem);
void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem);
int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem);
void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem);
--
2.43.0
[PATCH v19 16/30] drm/lima: Explicitly get and put drm-shmem pages
To simplify the drm-shmem refcnt handling, we're moving away from
the implicit get_pages() that is used by get_pages_sgt(). From now on
drivers will have to pin pages while they use sgt. Lima driver doesn't
have shrinker, hence pages are pinned and sgt is valid as long as pages'
use-count > 0.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/lima/lima_gem.c | 15 +--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index 2a97aa85416b..9c3e34a7fbed 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -115,6 +115,7 @@ int lima_gem_create_handle(struct drm_device *dev, struct
drm_file *file,
return PTR_ERR(shmem);
obj = >base;
+ bo = to_lima_bo(obj);
/* Mali Utgard GPU can only support 32bit address space */
mask = mapping_gfp_mask(obj->filp->f_mapping);
@@ -123,13 +124,17 @@ int lima_gem_create_handle(struct drm_device *dev, struct
drm_file *file,
mapping_set_gfp_mask(obj->filp->f_mapping, mask);
if (is_heap) {
- bo = to_lima_bo(obj);
err = lima_heap_alloc(bo, NULL);
if (err)
goto out;
} else {
- struct sg_table *sgt = drm_gem_shmem_get_pages_sgt(shmem);
+ struct sg_table *sgt;
+ err = drm_gem_shmem_get_pages(shmem);
+ if (err)
+ goto out;
+
+ sgt = drm_gem_shmem_get_pages_sgt(shmem);
if (IS_ERR(sgt)) {
err = PTR_ERR(sgt);
goto out;
@@ -139,6 +144,9 @@ int lima_gem_create_handle(struct drm_device *dev, struct
drm_file *file,
err = drm_gem_handle_create(file, obj, handle);
out:
+ if (err && refcount_read(>base.pages_use_count))
+ drm_gem_shmem_put_pages(shmem);
+
/* drop reference from allocate - handle holds it now */
drm_gem_object_put(obj);
@@ -152,6 +160,9 @@ static void lima_gem_free_object(struct drm_gem_object *obj)
if (!list_empty(>va))
dev_err(obj->dev->dev, "lima gem free bo still has va\n");
+ if (refcount_read(>base.pages_use_count))
+ drm_gem_shmem_put_pages(>base);
+
drm_gem_shmem_free(>base);
}
--
2.43.0
[PATCH v19 12/30] drm/shmem-helper: Prepare drm_gem_shmem_free() to shrinker addition
Prepare drm_gem_shmem_free() to addition of memory shrinker support
to drm-shmem by adding and using variant of put_pages() that doesn't
touch reservation lock. Reservation shouldn't be touched because lockdep
will trigger a bogus warning about locking contention with fs_reclaim
code paths that can't happen during the time when GEM is freed and
lockdep doesn't know about that.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 40 ++
1 file changed, 21 insertions(+), 19 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 3403700780c3..799a3c5015ad 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -128,6 +128,22 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct
drm_device *dev, size_t
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
+static void
+drm_gem_shmem_free_pages(struct drm_gem_shmem_object *shmem)
+{
+ struct drm_gem_object *obj = >base;
+
+#ifdef CONFIG_X86
+ if (shmem->map_wc)
+ set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
+#endif
+
+ drm_gem_put_pages(obj, shmem->pages,
+ shmem->pages_mark_dirty_on_put,
+ shmem->pages_mark_accessed_on_put);
+ shmem->pages = NULL;
+}
+
/**
* drm_gem_shmem_free - Free resources associated with a shmem GEM object
* @shmem: shmem GEM object to free
@@ -142,8 +158,6 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
if (obj->import_attach) {
drm_prime_gem_destroy(obj, shmem->sgt);
} else {
- dma_resv_lock(shmem->base.resv, NULL);
-
drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count));
if (shmem->sgt) {
@@ -152,13 +166,12 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object
*shmem)
sg_free_table(shmem->sgt);
kfree(shmem->sgt);
}
- if (shmem->pages)
- drm_gem_shmem_put_pages_locked(shmem);
+ if (shmem->pages &&
+ refcount_dec_and_test(>pages_use_count))
+ drm_gem_shmem_free_pages(shmem);
drm_WARN_ON(obj->dev, refcount_read(>pages_use_count));
drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count));
-
- dma_resv_unlock(shmem->base.resv);
}
drm_gem_object_release(obj);
@@ -208,21 +221,10 @@ static int drm_gem_shmem_get_pages_locked(struct
drm_gem_shmem_object *shmem)
*/
void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem)
{
- struct drm_gem_object *obj = >base;
-
dma_resv_assert_held(shmem->base.resv);
- if (refcount_dec_and_test(>pages_use_count)) {
-#ifdef CONFIG_X86
- if (shmem->map_wc)
- set_pages_array_wb(shmem->pages, obj->size >>
PAGE_SHIFT);
-#endif
-
- drm_gem_put_pages(obj, shmem->pages,
- shmem->pages_mark_dirty_on_put,
- shmem->pages_mark_accessed_on_put);
- shmem->pages = NULL;
- }
+ if (refcount_dec_and_test(>pages_use_count))
+ drm_gem_shmem_free_pages(shmem);
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
--
2.43.0
[PATCH v19 11/30] drm/shmem-helper: Use refcount_t for vmap_use_count
Use refcount_t helper for vmap_use_count to make refcounting consistent
with pages_use_count and pages_pin_count that use refcount_t. This also
makes vmapping to benefit from the refcount_t's overflow checks.
Acked-by: Maxime Ripard
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 28 +++---
include/drm/drm_gem_shmem_helper.h | 2 +-
2 files changed, 13 insertions(+), 17 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 9c89183f81b7..3403700780c3 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -144,7 +144,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
} else {
dma_resv_lock(shmem->base.resv, NULL);
- drm_WARN_ON(obj->dev, shmem->vmap_use_count);
+ drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count));
if (shmem->sgt) {
dma_unmap_sgtable(obj->dev->dev, shmem->sgt,
@@ -344,23 +344,25 @@ int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object
*shmem,
dma_resv_assert_held(shmem->base.resv);
- if (shmem->vmap_use_count++ > 0) {
+ if (refcount_inc_not_zero(>vmap_use_count)) {
iosys_map_set_vaddr(map, shmem->vaddr);
return 0;
}
ret = drm_gem_shmem_pin_locked(shmem);
if (ret)
- goto err_zero_use;
+ return ret;
if (shmem->map_wc)
prot = pgprot_writecombine(prot);
shmem->vaddr = vmap(shmem->pages, obj->size >> PAGE_SHIFT,
VM_MAP, prot);
- if (!shmem->vaddr)
+ if (!shmem->vaddr) {
ret = -ENOMEM;
- else
+ } else {
iosys_map_set_vaddr(map, shmem->vaddr);
+ refcount_set(>vmap_use_count, 1);
+ }
}
if (ret) {
@@ -373,8 +375,6 @@ int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object
*shmem,
err_put_pages:
if (!obj->import_attach)
drm_gem_shmem_unpin_locked(shmem);
-err_zero_use:
- shmem->vmap_use_count = 0;
return ret;
}
@@ -402,14 +402,10 @@ void drm_gem_shmem_vunmap_locked(struct
drm_gem_shmem_object *shmem,
} else {
dma_resv_assert_held(shmem->base.resv);
- if (drm_WARN_ON_ONCE(obj->dev, !shmem->vmap_use_count))
- return;
-
- if (--shmem->vmap_use_count > 0)
- return;
-
- vunmap(shmem->vaddr);
- drm_gem_shmem_unpin_locked(shmem);
+ if (refcount_dec_and_test(>vmap_use_count)) {
+ vunmap(shmem->vaddr);
+ drm_gem_shmem_unpin_locked(shmem);
+ }
}
shmem->vaddr = NULL;
@@ -655,7 +651,7 @@ void drm_gem_shmem_print_info(const struct
drm_gem_shmem_object *shmem,
drm_printf_indent(p, indent, "pages_pin_count=%u\n",
refcount_read(>pages_pin_count));
drm_printf_indent(p, indent, "pages_use_count=%u\n",
refcount_read(>pages_use_count));
- drm_printf_indent(p, indent, "vmap_use_count=%u\n",
shmem->vmap_use_count);
+ drm_printf_indent(p, indent, "vmap_use_count=%u\n",
refcount_read(>vmap_use_count));
drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_print_info);
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index 80623b897803..18020f653d7e 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -82,7 +82,7 @@ struct drm_gem_shmem_object {
* Reference count on the virtual address.
* The address are un-mapped when the count reaches zero.
*/
- unsigned int vmap_use_count;
+ refcount_t vmap_use_count;
/**
* @pages_mark_dirty_on_put:
--
2.43.0
[PATCH v19 09/30] drm/shmem-helper: Add and use lockless drm_gem_shmem_get_pages()
Add lockless drm_gem_shmem_get_pages() helper that skips taking reservation
lock if pages_use_count is non-zero, leveraging from atomicity of the
refcount_t. Make drm_gem_shmem_mmap() to utilize the new helper.
Acked-by: Maxime Ripard
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 19 +++
1 file changed, 15 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index cacf0f8c42e2..1c032513abf1 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -226,6 +226,20 @@ void drm_gem_shmem_put_pages_locked(struct
drm_gem_shmem_object *shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
+static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
+{
+ int ret;
+
+ if (refcount_inc_not_zero(>pages_use_count))
+ return 0;
+
+ dma_resv_lock(shmem->base.resv, NULL);
+ ret = drm_gem_shmem_get_pages_locked(shmem);
+ dma_resv_unlock(shmem->base.resv);
+
+ return ret;
+}
+
static int drm_gem_shmem_pin_locked(struct drm_gem_shmem_object *shmem)
{
int ret;
@@ -609,10 +623,7 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem,
struct vm_area_struct
return ret;
}
- dma_resv_lock(shmem->base.resv, NULL);
- ret = drm_gem_shmem_get_pages_locked(shmem);
- dma_resv_unlock(shmem->base.resv);
-
+ ret = drm_gem_shmem_get_pages(shmem);
if (ret)
return ret;
--
2.43.0
[PATCH v19 10/30] drm/shmem-helper: Switch drm_gem_shmem_vmap/vunmap to use pin/unpin
The vmapped pages shall be pinned in memory and previously get/put_pages()
were implicitly hard-pinning/unpinning the pages. This will no longer be
the case with addition of memory shrinker because pages_use_count > 0 won't
determine anymore whether pages are hard-pinned (they will be soft-pinned),
while the new pages_pin_count will do the hard-pinning. Switch the
vmap/vunmap() to use pin/unpin() functions in a preparation of addition
of the memory shrinker support to drm-shmem.
Acked-by: Maxime Ripard
Reviewed-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 19 ---
include/drm/drm_gem_shmem_helper.h | 2 +-
2 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 1c032513abf1..9c89183f81b7 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -256,6 +256,14 @@ static int drm_gem_shmem_pin_locked(struct
drm_gem_shmem_object *shmem)
return ret;
}
+static void drm_gem_shmem_unpin_locked(struct drm_gem_shmem_object *shmem)
+{
+ dma_resv_assert_held(shmem->base.resv);
+
+ if (refcount_dec_and_test(>pages_pin_count))
+ drm_gem_shmem_put_pages_locked(shmem);
+}
+
/**
* drm_gem_shmem_pin - Pin backing pages for a shmem GEM object
* @shmem: shmem GEM object
@@ -303,10 +311,7 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object
*shmem)
return;
dma_resv_lock(shmem->base.resv, NULL);
-
- if (refcount_dec_and_test(>pages_pin_count))
- drm_gem_shmem_put_pages_locked(shmem);
-
+ drm_gem_shmem_unpin_locked(shmem);
dma_resv_unlock(shmem->base.resv);
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_unpin);
@@ -344,7 +349,7 @@ int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object
*shmem,
return 0;
}
- ret = drm_gem_shmem_get_pages_locked(shmem);
+ ret = drm_gem_shmem_pin_locked(shmem);
if (ret)
goto err_zero_use;
@@ -367,7 +372,7 @@ int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object
*shmem,
err_put_pages:
if (!obj->import_attach)
- drm_gem_shmem_put_pages_locked(shmem);
+ drm_gem_shmem_unpin_locked(shmem);
err_zero_use:
shmem->vmap_use_count = 0;
@@ -404,7 +409,7 @@ void drm_gem_shmem_vunmap_locked(struct
drm_gem_shmem_object *shmem,
return;
vunmap(shmem->vaddr);
- drm_gem_shmem_put_pages_locked(shmem);
+ drm_gem_shmem_unpin_locked(shmem);
}
shmem->vaddr = NULL;
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index 2c5dc62df20c..80623b897803 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -124,7 +124,7 @@ int drm_gem_shmem_madvise_locked(struct
drm_gem_shmem_object *shmem, int madv);
static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object
*shmem)
{
return (shmem->madv > 0) &&
- !shmem->vmap_use_count && shmem->sgt &&
+ !refcount_read(>pages_pin_count) && shmem->sgt &&
!shmem->base.dma_buf && !shmem->base.import_attach;
}
--
2.43.0
[PATCH v19 07/30] drm/shmem-helper: Add and use pages_pin_count
Add separate pages_pin_count for tracking of whether drm-shmem pages are
moveable or not. With the addition of memory shrinker support to drm-shmem,
the pages_use_count will no longer determine whether pages are hard-pinned
in memory, but whether pages exist and are soft-pinned (and could be swapped
out). The pages_pin_count > 1 will hard-pin pages in memory.
Acked-by: Maxime Ripard
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 25 +
include/drm/drm_gem_shmem_helper.h | 11 +++
2 files changed, 28 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 1f0a66386415..55b9dd3d4b18 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -156,6 +156,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
drm_gem_shmem_put_pages_locked(shmem);
drm_WARN_ON(obj->dev, shmem->pages_use_count);
+ drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count));
dma_resv_unlock(shmem->base.resv);
}
@@ -234,18 +235,16 @@ static int drm_gem_shmem_pin_locked(struct
drm_gem_shmem_object *shmem)
dma_resv_assert_held(shmem->base.resv);
+ if (refcount_inc_not_zero(>pages_pin_count))
+ return 0;
+
ret = drm_gem_shmem_get_pages_locked(shmem);
+ if (!ret)
+ refcount_set(>pages_pin_count, 1);
return ret;
}
-static void drm_gem_shmem_unpin_locked(struct drm_gem_shmem_object *shmem)
-{
- dma_resv_assert_held(shmem->base.resv);
-
- drm_gem_shmem_put_pages_locked(shmem);
-}
-
/**
* drm_gem_shmem_pin - Pin backing pages for a shmem GEM object
* @shmem: shmem GEM object
@@ -263,6 +262,9 @@ int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
drm_WARN_ON(obj->dev, obj->import_attach);
+ if (refcount_inc_not_zero(>pages_pin_count))
+ return 0;
+
ret = dma_resv_lock_interruptible(shmem->base.resv, NULL);
if (ret)
return ret;
@@ -286,8 +288,14 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object
*shmem)
drm_WARN_ON(obj->dev, obj->import_attach);
+ if (refcount_dec_not_one(>pages_pin_count))
+ return;
+
dma_resv_lock(shmem->base.resv, NULL);
- drm_gem_shmem_unpin_locked(shmem);
+
+ if (refcount_dec_and_test(>pages_pin_count))
+ drm_gem_shmem_put_pages_locked(shmem);
+
dma_resv_unlock(shmem->base.resv);
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_unpin);
@@ -632,6 +640,7 @@ void drm_gem_shmem_print_info(const struct
drm_gem_shmem_object *shmem,
if (shmem->base.import_attach)
return;
+ drm_printf_indent(p, indent, "pages_pin_count=%u\n",
refcount_read(>pages_pin_count));
drm_printf_indent(p, indent, "pages_use_count=%u\n",
shmem->pages_use_count);
drm_printf_indent(p, indent, "vmap_use_count=%u\n",
shmem->vmap_use_count);
drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index 9e83212becbb..c708a9f45cbd 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -39,6 +39,17 @@ struct drm_gem_shmem_object {
*/
unsigned int pages_use_count;
+ /**
+* @pages_pin_count:
+*
+* Reference count on the pinned pages table.
+*
+* Pages are hard-pinned and reside in memory if count
+* greater than zero. Otherwise, when count is zero, the pages are
+* allowed to be evicted and purged by memory shrinker.
+*/
+ refcount_t pages_pin_count;
+
/**
* @madv: State for madvise
*
--
2.43.0
[PATCH v19 08/30] drm/shmem-helper: Use refcount_t for pages_use_count
Use atomic refcount_t helper for pages_use_count to optimize pin/unpin
functions by skipping reservation locking while GEM's pin refcount > 1.
Acked-by: Maxime Ripard
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 33 +++--
drivers/gpu/drm/lima/lima_gem.c | 2 +-
drivers/gpu/drm/panfrost/panfrost_mmu.c | 2 +-
include/drm/drm_gem_shmem_helper.h | 2 +-
4 files changed, 18 insertions(+), 21 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 55b9dd3d4b18..cacf0f8c42e2 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -155,7 +155,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
if (shmem->pages)
drm_gem_shmem_put_pages_locked(shmem);
- drm_WARN_ON(obj->dev, shmem->pages_use_count);
+ drm_WARN_ON(obj->dev, refcount_read(>pages_use_count));
drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count));
dma_resv_unlock(shmem->base.resv);
@@ -173,14 +173,13 @@ static int drm_gem_shmem_get_pages_locked(struct
drm_gem_shmem_object *shmem)
dma_resv_assert_held(shmem->base.resv);
- if (shmem->pages_use_count++ > 0)
+ if (refcount_inc_not_zero(>pages_use_count))
return 0;
pages = drm_gem_get_pages(obj);
if (IS_ERR(pages)) {
drm_dbg_kms(obj->dev, "Failed to get pages (%ld)\n",
PTR_ERR(pages));
- shmem->pages_use_count = 0;
return PTR_ERR(pages);
}
@@ -196,6 +195,8 @@ static int drm_gem_shmem_get_pages_locked(struct
drm_gem_shmem_object *shmem)
shmem->pages = pages;
+ refcount_set(>pages_use_count, 1);
+
return 0;
}
@@ -211,21 +212,17 @@ void drm_gem_shmem_put_pages_locked(struct
drm_gem_shmem_object *shmem)
dma_resv_assert_held(shmem->base.resv);
- if (drm_WARN_ON_ONCE(obj->dev, !shmem->pages_use_count))
- return;
-
- if (--shmem->pages_use_count > 0)
- return;
-
+ if (refcount_dec_and_test(>pages_use_count)) {
#ifdef CONFIG_X86
- if (shmem->map_wc)
- set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
+ if (shmem->map_wc)
+ set_pages_array_wb(shmem->pages, obj->size >>
PAGE_SHIFT);
#endif
- drm_gem_put_pages(obj, shmem->pages,
- shmem->pages_mark_dirty_on_put,
- shmem->pages_mark_accessed_on_put);
- shmem->pages = NULL;
+ drm_gem_put_pages(obj, shmem->pages,
+ shmem->pages_mark_dirty_on_put,
+ shmem->pages_mark_accessed_on_put);
+ shmem->pages = NULL;
+ }
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
@@ -552,8 +549,8 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct
*vma)
* mmap'd, vm_open() just grabs an additional reference for the new
* mm the vma is getting copied into (ie. on fork()).
*/
- if (!drm_WARN_ON_ONCE(obj->dev, !shmem->pages_use_count))
- shmem->pages_use_count++;
+ drm_WARN_ON_ONCE(obj->dev,
+!refcount_inc_not_zero(>pages_use_count));
dma_resv_unlock(shmem->base.resv);
@@ -641,7 +638,7 @@ void drm_gem_shmem_print_info(const struct
drm_gem_shmem_object *shmem,
return;
drm_printf_indent(p, indent, "pages_pin_count=%u\n",
refcount_read(>pages_pin_count));
- drm_printf_indent(p, indent, "pages_use_count=%u\n",
shmem->pages_use_count);
+ drm_printf_indent(p, indent, "pages_use_count=%u\n",
refcount_read(>pages_use_count));
drm_printf_indent(p, indent, "vmap_use_count=%u\n",
shmem->vmap_use_count);
drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
}
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index 433bda72e59b..2a97aa85416b 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -47,7 +47,7 @@ int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm)
}
bo->base.pages = pages;
- bo->base.pages_use_count = 1;
+ refcount_set(>base.pages_use_count, 1);
mapping_set_unevictable(mapping);
}
diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c
b/drivers/gpu/drm/panfrost/panfrost_mmu.c
index 1ab081bd81a8..bd5a0073009d 100644
--- a/drivers/gpu/drm/panfrost/panfrost_mmu
[PATCH v19 01/30] drm/gem: Change locked/unlocked postfix of drm_gem_v/unmap() function names
Make drm/gem API function names consistent by having locked function
use the _locked postfix in the name, while the unlocked variants don't
use the _unlocked postfix. Rename drm_gem_v/unmap() function names to
make them consistent with the rest of the API functions.
Acked-by: Maxime Ripard
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_client.c | 6 +++---
drivers/gpu/drm/drm_gem.c| 20 ++--
drivers/gpu/drm/drm_gem_framebuffer_helper.c | 6 +++---
drivers/gpu/drm/drm_internal.h | 4 ++--
drivers/gpu/drm/drm_prime.c | 4 ++--
drivers/gpu/drm/lima/lima_sched.c| 4 ++--
drivers/gpu/drm/panfrost/panfrost_dump.c | 4 ++--
drivers/gpu/drm/panfrost/panfrost_perfcnt.c | 6 +++---
include/drm/drm_gem.h| 4 ++--
9 files changed, 29 insertions(+), 29 deletions(-)
diff --git a/drivers/gpu/drm/drm_client.c b/drivers/gpu/drm/drm_client.c
index 9403b3f576f7..7ee9baf46eaa 100644
--- a/drivers/gpu/drm/drm_client.c
+++ b/drivers/gpu/drm/drm_client.c
@@ -255,7 +255,7 @@ void drm_client_dev_restore(struct drm_device *dev)
static void drm_client_buffer_delete(struct drm_client_buffer *buffer)
{
if (buffer->gem) {
- drm_gem_vunmap_unlocked(buffer->gem, >map);
+ drm_gem_vunmap(buffer->gem, >map);
drm_gem_object_put(buffer->gem);
}
@@ -339,7 +339,7 @@ drm_client_buffer_vmap(struct drm_client_buffer *buffer,
* fd_install step out of the driver backend hooks, to make that
* final step optional for internal users.
*/
- ret = drm_gem_vmap_unlocked(buffer->gem, map);
+ ret = drm_gem_vmap(buffer->gem, map);
if (ret)
return ret;
@@ -361,7 +361,7 @@ void drm_client_buffer_vunmap(struct drm_client_buffer
*buffer)
{
struct iosys_map *map = >map;
- drm_gem_vunmap_unlocked(buffer->gem, map);
+ drm_gem_vunmap(buffer->gem, map);
}
EXPORT_SYMBOL(drm_client_buffer_vunmap);
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 44a948b80ee1..95327b003692 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1175,7 +1175,7 @@ void drm_gem_unpin(struct drm_gem_object *obj)
obj->funcs->unpin(obj);
}
-int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
+int drm_gem_vmap_locked(struct drm_gem_object *obj, struct iosys_map *map)
{
int ret;
@@ -1192,9 +1192,9 @@ int drm_gem_vmap(struct drm_gem_object *obj, struct
iosys_map *map)
return 0;
}
-EXPORT_SYMBOL(drm_gem_vmap);
+EXPORT_SYMBOL(drm_gem_vmap_locked);
-void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
+void drm_gem_vunmap_locked(struct drm_gem_object *obj, struct iosys_map *map)
{
dma_resv_assert_held(obj->resv);
@@ -1207,27 +1207,27 @@ void drm_gem_vunmap(struct drm_gem_object *obj, struct
iosys_map *map)
/* Always set the mapping to NULL. Callers may rely on this. */
iosys_map_clear(map);
}
-EXPORT_SYMBOL(drm_gem_vunmap);
+EXPORT_SYMBOL(drm_gem_vunmap_locked);
-int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
+int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
{
int ret;
dma_resv_lock(obj->resv, NULL);
- ret = drm_gem_vmap(obj, map);
+ ret = drm_gem_vmap_locked(obj, map);
dma_resv_unlock(obj->resv);
return ret;
}
-EXPORT_SYMBOL(drm_gem_vmap_unlocked);
+EXPORT_SYMBOL(drm_gem_vmap);
-void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
+void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
{
dma_resv_lock(obj->resv, NULL);
- drm_gem_vunmap(obj, map);
+ drm_gem_vunmap_locked(obj, map);
dma_resv_unlock(obj->resv);
}
-EXPORT_SYMBOL(drm_gem_vunmap_unlocked);
+EXPORT_SYMBOL(drm_gem_vunmap);
/**
* drm_gem_lock_reservations - Sets up the ww context and acquires
diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
index 3bdb6ba37ff4..3808f47310bf 100644
--- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
+++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
@@ -362,7 +362,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb, struct
iosys_map *map,
ret = -EINVAL;
goto err_drm_gem_vunmap;
}
- ret = drm_gem_vmap_unlocked(obj, [i]);
+ ret = drm_gem_vmap(obj, [i]);
if (ret)
goto err_drm_gem_vunmap;
}
@@ -384,7 +384,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb, struct
iosys_map *map,
obj = drm_gem_fb_get_obj(fb, i);
if (!obj)
con
[PATCH v19 05/30] drm/shmem-helper: Refactor locked/unlocked functions
Add locked and remove unlocked postfixes from drm-shmem function names,
making names consistent with the drm/gem core code.
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c| 60 +--
drivers/gpu/drm/lima/lima_gem.c | 6 +-
drivers/gpu/drm/panfrost/panfrost_drv.c | 2 +-
drivers/gpu/drm/panfrost/panfrost_gem.c | 2 +-
.../gpu/drm/panfrost/panfrost_gem_shrinker.c | 2 +-
drivers/gpu/drm/panfrost/panfrost_mmu.c | 2 +-
include/drm/drm_gem_shmem_helper.h| 28 -
7 files changed, 51 insertions(+), 51 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 0d61f2b3e213..043e8e3b129c 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -153,7 +153,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
kfree(shmem->sgt);
}
if (shmem->pages)
- drm_gem_shmem_put_pages(shmem);
+ drm_gem_shmem_put_pages_locked(shmem);
drm_WARN_ON(obj->dev, shmem->pages_use_count);
@@ -165,7 +165,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
-static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
+static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
struct page **pages;
@@ -199,12 +199,12 @@ static int drm_gem_shmem_get_pages(struct
drm_gem_shmem_object *shmem)
}
/*
- * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a
shmem GEM object
+ * drm_gem_shmem_put_pages_locked - Decrease use count on the backing pages
for a shmem GEM object
* @shmem: shmem GEM object
*
* This function decreases the use count and puts the backing pages when use
drops to zero.
*/
-void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
+void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
@@ -226,7 +226,7 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object
*shmem)
shmem->pages_mark_accessed_on_put);
shmem->pages = NULL;
}
-EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
static int drm_gem_shmem_pin_locked(struct drm_gem_shmem_object *shmem)
{
@@ -234,7 +234,7 @@ static int drm_gem_shmem_pin_locked(struct
drm_gem_shmem_object *shmem)
dma_resv_assert_held(shmem->base.resv);
- ret = drm_gem_shmem_get_pages(shmem);
+ ret = drm_gem_shmem_get_pages_locked(shmem);
return ret;
}
@@ -243,7 +243,7 @@ static void drm_gem_shmem_unpin_locked(struct
drm_gem_shmem_object *shmem)
{
dma_resv_assert_held(shmem->base.resv);
- drm_gem_shmem_put_pages(shmem);
+ drm_gem_shmem_put_pages_locked(shmem);
}
/**
@@ -293,7 +293,7 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
EXPORT_SYMBOL_GPL(drm_gem_shmem_unpin);
/*
- * drm_gem_shmem_vmap - Create a virtual mapping for a shmem GEM object
+ * drm_gem_shmem_vmap_locked - Create a virtual mapping for a shmem GEM object
* @shmem: shmem GEM object
* @map: Returns the kernel virtual address of the SHMEM GEM object's backing
* store.
@@ -302,13 +302,13 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_unpin);
* exists for the buffer backing the shmem GEM object. It hides the differences
* between dma-buf imported and natively allocated objects.
*
- * Acquired mappings should be cleaned up by calling drm_gem_shmem_vunmap().
+ * Acquired mappings should be cleaned up by calling
drm_gem_shmem_vunmap_locked().
*
* Returns:
* 0 on success or a negative error code on failure.
*/
-int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
- struct iosys_map *map)
+int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
+ struct iosys_map *map)
{
struct drm_gem_object *obj = >base;
int ret = 0;
@@ -331,7 +331,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
return 0;
}
- ret = drm_gem_shmem_get_pages(shmem);
+ ret = drm_gem_shmem_get_pages_locked(shmem);
if (ret)
goto err_zero_use;
@@ -354,28 +354,28 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
err_put_pages:
if (!obj->import_attach)
- drm_gem_shmem_put_pages(shmem);
+ drm_gem_shmem_put_pages_locked(shmem);
err_zero_use:
shmem->vmap_use_count = 0;
return ret;
}
-EXPORT_SYMBOL_GPL(drm_gem_shmem_vmap);
+EXPO
[PATCH v19 06/30] drm/shmem-helper: Remove obsoleted is_iomem test
Everything that uses the mapped buffer should be agnostic to is_iomem.
The only reason for the is_iomem test is that we're setting shmem->vaddr
to the returned map->vaddr. Now that the shmem->vaddr code is gone, remove
the obsoleted is_iomem test to clean up the code.
Acked-by: Maxime Ripard
Suggested-by: Thomas Zimmermann
Reviewed-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 6 --
1 file changed, 6 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 043e8e3b129c..1f0a66386415 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -315,12 +315,6 @@ int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object
*shmem,
if (obj->import_attach) {
ret = dma_buf_vmap(obj->import_attach->dmabuf, map);
- if (!ret) {
- if (drm_WARN_ON(obj->dev, map->is_iomem)) {
- dma_buf_vunmap(obj->import_attach->dmabuf, map);
- return -EIO;
- }
- }
} else {
pgprot_t prot = PAGE_KERNEL;
--
2.43.0
[PATCH v19 04/30] drm/shmem-helper: Make all exported symbols GPL
Make all drm-shmem exported symbols GPL to make them consistent with
the rest of drm-shmem symbols.
Acked-by: Maxime Ripard
Reviewed-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index e435f986cd13..0d61f2b3e213 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -226,7 +226,7 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object
*shmem)
shmem->pages_mark_accessed_on_put);
shmem->pages = NULL;
}
-EXPORT_SYMBOL(drm_gem_shmem_put_pages);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages);
static int drm_gem_shmem_pin_locked(struct drm_gem_shmem_object *shmem)
{
@@ -271,7 +271,7 @@ int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
return ret;
}
-EXPORT_SYMBOL(drm_gem_shmem_pin);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_pin);
/**
* drm_gem_shmem_unpin - Unpin backing pages for a shmem GEM object
@@ -290,7 +290,7 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
drm_gem_shmem_unpin_locked(shmem);
dma_resv_unlock(shmem->base.resv);
}
-EXPORT_SYMBOL(drm_gem_shmem_unpin);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_unpin);
/*
* drm_gem_shmem_vmap - Create a virtual mapping for a shmem GEM object
@@ -360,7 +360,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
return ret;
}
-EXPORT_SYMBOL(drm_gem_shmem_vmap);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_vmap);
/*
* drm_gem_shmem_vunmap - Unmap a virtual mapping for a shmem GEM object
@@ -396,7 +396,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object
*shmem,
shmem->vaddr = NULL;
}
-EXPORT_SYMBOL(drm_gem_shmem_vunmap);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_vunmap);
static int
drm_gem_shmem_create_with_handle(struct drm_file *file_priv,
@@ -435,7 +435,7 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object
*shmem, int madv)
return (madv >= 0);
}
-EXPORT_SYMBOL(drm_gem_shmem_madvise);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_madvise);
void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
{
@@ -467,7 +467,7 @@ void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0,
(loff_t)-1);
}
-EXPORT_SYMBOL(drm_gem_shmem_purge);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
/**
* drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
@@ -642,7 +642,7 @@ void drm_gem_shmem_print_info(const struct
drm_gem_shmem_object *shmem,
drm_printf_indent(p, indent, "vmap_use_count=%u\n",
shmem->vmap_use_count);
drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
}
-EXPORT_SYMBOL(drm_gem_shmem_print_info);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_print_info);
/**
* drm_gem_shmem_get_sg_table - Provide a scatter/gather table of pinned
--
2.43.0
[PATCH v19 03/30] drm/gem: Document locking rule of vmap and evict callbacks
The vmap/vunmap/evict GEM callbacks are always invoked with a held GEM's
reservation lock. Document this locking rule for clarity.
Signed-off-by: Dmitry Osipenko
---
include/drm/drm_gem.h | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index f835fdee6a5e..021f64371056 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -156,7 +156,8 @@ struct drm_gem_object_funcs {
* @vmap:
*
* Returns a virtual address for the buffer. Used by the
-* drm_gem_dmabuf_vmap() helper.
+* drm_gem_dmabuf_vmap() helper. Called with a held GEM reservation
+* lock.
*
* This callback is optional.
*/
@@ -166,7 +167,8 @@ struct drm_gem_object_funcs {
* @vunmap:
*
* Releases the address previously returned by @vmap. Used by the
-* drm_gem_dmabuf_vunmap() helper.
+* drm_gem_dmabuf_vunmap() helper. Called with a held GEM reservation
+* lock.
*
* This callback is optional.
*/
@@ -189,7 +191,8 @@ struct drm_gem_object_funcs {
* @evict:
*
* Evicts gem object out from memory. Used by the drm_gem_object_evict()
-* helper. Returns 0 on success, -errno otherwise.
+* helper. Returns 0 on success, -errno otherwise. Called with a held
+* GEM reservation lock.
*
* This callback is optional.
*/
--
2.43.0
[PATCH v19 02/30] drm/gem: Add _locked postfix to functions that have unlocked counterpart
Add _locked postfix to drm_gem functions that have unlocked counterpart
functions to make GEM functions naming more consistent and intuitive in
regards to the locking requirements.
Acked-by: Maxime Ripard
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem.c | 6 +++---
include/drm/drm_gem.h | 2 +-
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 95327b003692..4523cd40fb2f 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1490,10 +1490,10 @@ drm_gem_lru_scan(struct drm_gem_lru *lru,
EXPORT_SYMBOL(drm_gem_lru_scan);
/**
- * drm_gem_evict - helper to evict backing pages for a GEM object
+ * drm_gem_evict_locked - helper to evict backing pages for a GEM object
* @obj: obj in question
*/
-int drm_gem_evict(struct drm_gem_object *obj)
+int drm_gem_evict_locked(struct drm_gem_object *obj)
{
dma_resv_assert_held(obj->resv);
@@ -1505,4 +1505,4 @@ int drm_gem_evict(struct drm_gem_object *obj)
return 0;
}
-EXPORT_SYMBOL(drm_gem_evict);
+EXPORT_SYMBOL(drm_gem_evict_locked);
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index decb19ffb2c8..f835fdee6a5e 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -551,7 +551,7 @@ unsigned long drm_gem_lru_scan(struct drm_gem_lru *lru,
unsigned long *remaining,
bool (*shrink)(struct drm_gem_object *obj));
-int drm_gem_evict(struct drm_gem_object *obj);
+int drm_gem_evict_locked(struct drm_gem_object *obj);
#ifdef CONFIG_LOCKDEP
/**
--
2.43.0
[PATCH v19 00/30] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers
put() in GEM-creation error code paths, which is now required by drm-shmem and was missed in a previous patch versions. - Virtio-GPU now attaches shmem pages to host on first use and not when BO is created. In older patch versions there was a potential race condition in the BO creation code path where both get_sgt()+object_attach() should've been made under same resv lock, otherwise pages could be evicted before attachment is invoked. - Virtio-GPU and drm-shmem shrinker patches are split into smaller ones. v14:- All the prerequisite reservation locking patches landed upstream, previously were a part of this series in v13 and older. https://lore.kernel.org/dri-devel/20230529223935.2672495-1-dmitry.osipe...@collabora.com/ - Added patches to improve locked/unlocked function names, like was suggested by Boris Brezillon for v13. - Made all exported drm-shmem symbols GPL, like was previously discussed with Thomas Zimmermann on this series. - Improved virtio-gpu shrinker patch. Now it won't detach purged BO when userspace closes GEM. Crosvm (and not qemu) checks res_id on CMD_CTX_DETACH_RESOURCE and prints noisy error message if ID is invalid, which wasn't noticed before. v13:- Updated virtio-gpu shrinker patch to use drm_gem_shmem_object_pin() directly instead of drm_gem_pin() and dropped patch that exported drm_gem_pin() functions, like was requested by Thomas Zimmermann in v12. v12:- Fixed the "no previous prototype for function" warning reported by kernel build bot for v11. - Fixed the missing reservation lock reported by Intel CI for VGEM driver. Other drivers using drm-shmem were affected similarly to VGEM. The problem was in the dma-buf attachment code path that led to drm-shmem pinning function which assumed the held reservation lock by drm_gem_pin(). In the past that code path was causing trouble for i915 driver and we've changed the locking scheme for the attachment code path in the dma-buf core to let exporters to handle the locking themselves. After a closer investigation, I realized that my assumption about testing of dma-buf export code path using Panfrost driver was incorrect. Now I created additional local test to exrecise the Panfrost export path. I also reproduced the issue reported by the Intel CI for v10. It's all fixed now by making the drm_gem_shmem_pin() to take the resv lock by itself. - Patches are based on top of drm-tip, CC'd intel-gfx CI for testing. v11:- Rebased on a recent linux-next. Added new patch as a result: drm/shmem-helper: Export drm_gem_shmem_get_pages_sgt_locked() It's needed by the virtio-gpu driver to swap-in/unevict shmem object, previously get_pages_sgt() didn't use locking. - Separated the "Add memory shrinker" patch into smaller parts to ease the reviewing, as was requested by Thomas Zimmermann: drm/shmem-helper: Factor out pages alloc/release from drm_gem_shmem_get/put_pages() drm/shmem-helper: Add pages_pin_count field drm/shmem-helper: Switch drm_gem_shmem_vmap/vunmap to use pin/unpin drm/shmem-helper: Factor out unpinning part from drm_gem_shmem_purge() - Addessed the v10 review comments from Thomas Zimmermann: return errno instead of bool, sort code alphabetically, rename function and etc minor changes. - Added new patch to remove the "map->is_iomem" from drm-shmem, as was suggested by Thomas Zimmermann. - Added acks and r-b's that were given to v10. v10:- Was partially applied to misc-fixes/next. https://lore.kernel.org/dri-devel/6c16f303-81df-7ebe-85e9-51bb40a8b...@collabora.com/T/ Boris Brezillon (1): drm/panfrost: Fix the error path in panfrost_mmu_map_fault_addr() Dmitry Osipenko (29): drm/gem: Change locked/unlocked postfix of drm_gem_v/unmap() function names drm/gem: Add _locked postfix to functions that have unlocked counterpart drm/gem: Document locking rule of vmap and evict callbacks drm/shmem-helper: Make all exported symbols GPL drm/shmem-helper: Refactor locked/unlocked functions drm/shmem-helper: Remove obsoleted is_iomem test drm/shmem-helper: Add and use pages_pin_count drm/shmem-helper: Use refcount_t for pages_use_count drm/shmem-helper: Add and use lockless drm_gem_shmem_get_pages() drm/shmem-helper: Switch drm_gem_shmem_vmap/vunmap to use pin/unpin drm/shmem-helper: Use refcount_t for vmap_use_count drm/shmem-helper: Prepare drm_gem_shmem_free() to shrinker addition drm/shmem-helper: Make drm_gem_shmem_get_pages() public drm/shmem-helper: Add drm_gem_shmem_put_pages() drm/shmem-helper: Avoid lockdep warning when pages are released drm/lima: Explicitly get and put drm-shmem pages drm/panfrost: Explicitly get and put drm-shmem pages
Re: [PATCH 1/1] drm/virtio: Implement RESOURCE_GET_LAYOUT ioctl
On 12/21/23 13:00, Julia Zhang wrote:
> From: Daniel Stone
>
> Add a new ioctl to allow the guest VM to discover how the guest
> actually allocated the underlying buffer, which allows buffers to
> be used for GL<->Vulkan interop and through standard window systems.
> It's also a step towards properly supporting modifiers in the guest.
>
> Signed-off-by: Daniel Stone
> Co-developed-by: Julia Zhang # support query
> stride before it's created
> Signed-off-by: Julia Zhang
> ---
> drivers/gpu/drm/virtio/virtgpu_drv.c | 1 +
> drivers/gpu/drm/virtio/virtgpu_drv.h | 22 -
> drivers/gpu/drm/virtio/virtgpu_ioctl.c | 66 ++
> drivers/gpu/drm/virtio/virtgpu_kms.c | 8 +++-
> drivers/gpu/drm/virtio/virtgpu_vq.c| 63
> include/uapi/drm/virtgpu_drm.h | 21
> include/uapi/linux/virtio_gpu.h| 30
> 7 files changed, 208 insertions(+), 3 deletions(-)
...
> +static int virtio_gpu_resource_query_layout_ioctl(struct drm_device *dev,
> + void *data,
> + struct drm_file *file)
> +{
> + struct drm_virtgpu_resource_query_layout *args = data;
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> + struct drm_gem_object *obj = NULL;
> + struct virtio_gpu_object *bo = NULL;
> + struct virtio_gpu_query_info bo_info = {0};
> + int ret = 0;
> + int i;
> +
> + if (!vgdev->has_resource_query_layout) {
> + DRM_ERROR("failing: no RQL on host\n");
Please remove this message
> + return -EINVAL;
return -ENOSYS
> + }
> +
> + if (args->handle > 0) {
> + obj = drm_gem_object_lookup(file, args->handle);
> + if (obj == NULL) {
> + DRM_ERROR("invalid handle 0x%x\n", args->handle);
> + return -ENOENT;
> + }
> + bo = gem_to_virtio_gpu_obj(obj);
> + }
> +
> + ret = virtio_gpu_cmd_get_resource_layout(vgdev, _info, args->width,
> + args->height, args->format,
> + args->bind, bo ?
> bo->hw_res_handle : 0);
What this special hw_res_handle=0 is doing? Why is it needed?
> + if (ret)
> + goto out;
> +
> + ret = wait_event_timeout(vgdev->resp_wq,
> + atomic_read(_info.is_valid),
> + 5 * HZ);
> + if (!ret)
> + goto out;
> +
> +valid:
> + smp_rmb();
> + WARN_ON(atomic_read(_info.is_valid));
Please remove this WARN_ON and fix the kernelbot report
> + args->num_planes = bo_info.num_planes;
> + args->modifier = bo_info.modifier;
> + for (i = 0; i < args->num_planes; i++) {
> + args->planes[i].offset = bo_info.planes[i].offset;
> + args->planes[i].stride = bo_info.planes[i].stride;
> + }
> + for (; i < VIRTIO_GPU_MAX_RESOURCE_PLANES; i++) {
> + args->planes[i].offset = 0;
> + args->planes[i].stride = 0;
> + }
> + ret = 0;
ret is already 0 here
> +out:
> + if (obj)
> + drm_gem_object_put(obj);
> + return ret;
> +}
...
> diff --git a/include/uapi/linux/virtio_gpu.h b/include/uapi/linux/virtio_gpu.h
> index f556fde07b76..547575232376 100644
> --- a/include/uapi/linux/virtio_gpu.h
> +++ b/include/uapi/linux/virtio_gpu.h
> @@ -65,6 +65,11 @@
> */
> #define VIRTIO_GPU_F_CONTEXT_INIT4
>
> +/*
> + * VIRTIO_GPU_CMD_RESOURCE_QUERY_LAYOUT
> + */
> +#define VIRTIO_GPU_F_RESOURCE_QUERY_LAYOUT 5
> +
> enum virtio_gpu_ctrl_type {
> VIRTIO_GPU_UNDEFINED = 0,
>
> @@ -95,6 +100,7 @@ enum virtio_gpu_ctrl_type {
> VIRTIO_GPU_CMD_SUBMIT_3D,
> VIRTIO_GPU_CMD_RESOURCE_MAP_BLOB,
> VIRTIO_GPU_CMD_RESOURCE_UNMAP_BLOB,
> + VIRTIO_GPU_CMD_RESOURCE_QUERY_LAYOUT,
>
> /* cursor commands */
> VIRTIO_GPU_CMD_UPDATE_CURSOR = 0x0300,
> @@ -108,6 +114,7 @@ enum virtio_gpu_ctrl_type {
> VIRTIO_GPU_RESP_OK_EDID,
> VIRTIO_GPU_RESP_OK_RESOURCE_UUID,
> VIRTIO_GPU_RESP_OK_MAP_INFO,
> + VIRTIO_GPU_RESP_OK_RESOURCE_LAYOUT,
>
> /* error responses */
> VIRTIO_GPU_RESP_ERR_UNSPEC = 0x1200,
> @@ -453,4 +460,27 @@ struct virtio_gpu_resource_unmap_blob {
> __le32 padding;
> };
>
> +/* VIRTIO_GPU_CMD_RESOURCE_QUERY_LAYOUT */
> +struct virtio_gpu_resource_query_layout {
> + struct virtio_gpu_ctrl_hdr hdr;
> + __le32 resource_id;
> + __le32 width;
> + __le32 height;
> + __le32 format;
> + __le32 bind;
64b pad missing
> +};
> +
> +
> +/* VIRTIO_GPU_RESP_OK_RESOURCE_LAYOUT */
> +#define VIRTIO_GPU_RES_MAX_PLANES 4
> +struct virtio_gpu_resp_resource_layout {
> + struct virtio_gpu_ctrl_hdr hdr;
> + __le64 modifier;
> + __le32 num_planes;
> + struct virtio_gpu_resource_plane {
> + __le64 offset;
> +
Re: [PATCH v1 1/1] drm/virtio: Spelling fixes
On 12/19/23 18:19, Andy Shevchenko wrote: > While making a spelling mistake myself for `git grep kvalloc` > I found that the only file has such a typo. Fix it and update > to the standard de facto of how we refer to the functions. > Also spell usr-out as user-out, it seems this driver uses its > own terminology nobody else can decypher, make it more readable. > > Signed-off-by: Andy Shevchenko > --- > drivers/gpu/drm/virtio/virtgpu_submit.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/drivers/gpu/drm/virtio/virtgpu_submit.c > b/drivers/gpu/drm/virtio/virtgpu_submit.c > index 5c514946bbad..1c7c7f61a222 100644 > --- a/drivers/gpu/drm/virtio/virtgpu_submit.c > +++ b/drivers/gpu/drm/virtio/virtgpu_submit.c > @@ -99,8 +99,8 @@ virtio_gpu_parse_deps(struct virtio_gpu_submit *submit) > return 0; > > /* > - * kvalloc at first tries to allocate memory using kmalloc and > - * falls back to vmalloc only on failure. It also uses __GFP_NOWARN > + * kvmalloc() at first tries to allocate memory using kmalloc() and > + * falls back to vmalloc() only on failure. It also uses __GFP_NOWARN >* internally for allocations larger than a page size, preventing >* storm of KMSG warnings. >*/ > @@ -529,7 +529,7 @@ int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, > void *data, > virtio_gpu_submit(); > > /* > - * Set up usr-out data after submitting the job to optimize > + * Set up user-out data after submitting the job to optimize >* the job submission path. >*/ > virtio_gpu_install_out_fence_fd(); Applied to misc-next, thanks -- Best regards, Dmitry
Re: [PATCH] drm/virtio: add definition for venus capset
On 11/19/23 06:46, Dmitry Osipenko wrote:
> On 9/21/23 00:16, Dmitry Osipenko wrote:
>> On 9/15/23 13:59, Huang Rui wrote:
>>> This definition is used fro qemu, and qemu imports this marco in the
>>> headers to enable venus for virtio gpu. So it should add it even kernel
>>> doesn't use this.
>>>
>>> Signed-off-by: Huang Rui
>>> ---
>>>
>>> Hi all,
>>>
>>> We would like to add a new definition for venus capset, it will be used for
>>> qemu. Please see details on below discussion:
>>>
>>> https://lore.kernel.org/qemu-devel/b82982aa-5b9e-481e-9491-b9313877b...@daynix.com/
>>>
>>> Thanks,
>>> Ray
>>>
>>> include/uapi/linux/virtio_gpu.h | 2 ++
>>> 1 file changed, 2 insertions(+)
>>>
>>> diff --git a/include/uapi/linux/virtio_gpu.h
>>> b/include/uapi/linux/virtio_gpu.h
>>> index f556fde07b76..0e21f3998108 100644
>>> --- a/include/uapi/linux/virtio_gpu.h
>>> +++ b/include/uapi/linux/virtio_gpu.h
>>> @@ -309,6 +309,8 @@ struct virtio_gpu_cmd_submit {
>>>
>>> #define VIRTIO_GPU_CAPSET_VIRGL 1
>>> #define VIRTIO_GPU_CAPSET_VIRGL2 2
>>> +/* 3 is reserved for gfxstream */
>>> +#define VIRTIO_GPU_CAPSET_VENUS 4
>>
>> Could you please add all other capsets, so we won't needed to do it
>> again in the future
>
> I've opened request to update virtio-spec with the corrected/updated
> capsets https://github.com/oasis-tcs/virtio-spec/issues/182. I'm
> expecting that it will take some time until spec change will be merged
> and now leaning to apply this v1 patch to not hold the Venus work.
>
> Gerd, do you have objections? R-b/ack?
Applied patch to misc-next with edited commit message. Updating spec
taking much time, not worth to hold this change longer. We'll add the
rest of capsets later on. Thanks, Rui!
--
Best regards,
Dmitry
Re: [PATCH v18 04/26] drm/shmem-helper: Refactor locked/unlocked functions
On 12/14/23 21:16, Maxime Ripard wrote:
> On Tue, Dec 05, 2023 at 02:43:16PM +0300, Dmitry Osipenko wrote:
>> On 12/4/23 15:55, Maxime Ripard wrote:
>>>> Okay, that means s/_locked/_nolock/ in drm_gem_shmem_helpers.{c,h}, I
>>>> guess.
>>
>> DRM subsys and majority of kernel uses common _locked postfix. We should
>> retain the old naming scheme by using _locked() in DRM. It's not
>> worthwhile changing the name to a much less popular variant for a no
>> good reason.
>>
>> Maxime, are you okay with keeping the _locked name?
>
> Yeah... I still don't really like it, but you're right that it's best to
> remain consistent over my opinion :)
Thanks for the review!
Best regards,
Dmitry
Re: [PATCH v18 04/26] drm/shmem-helper: Refactor locked/unlocked functions
On 12/4/23 15:55, Maxime Ripard wrote:
>> Okay, that means s/_locked/_nolock/ in drm_gem_shmem_helpers.{c,h}, I
>> guess.
DRM subsys and majority of kernel uses common _locked postfix. We should
retain the old naming scheme by using _locked() in DRM. It's not
worthwhile changing the name to a much less popular variant for a no
good reason.
Maxime, are you okay with keeping the _locked name?
--
Best regards,
Dmitry
Re: [PATCH v18 04/26] drm/shmem-helper: Refactor locked/unlocked functions
On 11/29/23 10:53, Boris Brezillon wrote:
> On Wed, 29 Nov 2023 01:05:14 +0300
> Dmitry Osipenko wrote:
>
>> On 11/28/23 15:37, Boris Brezillon wrote:
>>> On Tue, 28 Nov 2023 12:14:42 +0100
>>> Maxime Ripard wrote:
>>>
>>>> Hi,
>>>>
>>>> On Fri, Nov 24, 2023 at 11:59:11AM +0100, Boris Brezillon wrote:
>>>>> On Fri, 24 Nov 2023 11:40:06 +0100
>>>>> Maxime Ripard wrote:
>>>>>
>>>>>> On Mon, Oct 30, 2023 at 02:01:43AM +0300, Dmitry Osipenko wrote:
>>>>>>> Add locked and remove unlocked postfixes from drm-shmem function names,
>>>>>>> making names consistent with the drm/gem core code.
>>>>>>>
>>>>>>> Reviewed-by: Boris Brezillon
>>>>>>> Suggested-by: Boris Brezillon
>>>>>>> Signed-off-by: Dmitry Osipenko
>>>>>>
>>>>>> This contradicts my earlier ack on a patch but...
>>>>>>
>>>>>>> ---
>>>>>>> drivers/gpu/drm/drm_gem_shmem_helper.c| 64 +--
>>>>>>> drivers/gpu/drm/lima/lima_gem.c | 8 +--
>>>>>>> drivers/gpu/drm/panfrost/panfrost_drv.c | 2 +-
>>>>>>> drivers/gpu/drm/panfrost/panfrost_gem.c | 6 +-
>>>>>>> .../gpu/drm/panfrost/panfrost_gem_shrinker.c | 2 +-
>>>>>>> drivers/gpu/drm/panfrost/panfrost_mmu.c | 2 +-
>>>>>>> drivers/gpu/drm/v3d/v3d_bo.c | 4 +-
>>>>>>> drivers/gpu/drm/virtio/virtgpu_object.c | 4 +-
>>>>>>> include/drm/drm_gem_shmem_helper.h| 36 +--
>>>>>>> 9 files changed, 64 insertions(+), 64 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>>>> b/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>>>> index 0d61f2b3e213..154585ddae08 100644
>>>>>>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>>>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>>>> @@ -43,8 +43,8 @@ static const struct drm_gem_object_funcs
>>>>>>> drm_gem_shmem_funcs = {
>>>>>>> .pin = drm_gem_shmem_object_pin,
>>>>>>> .unpin = drm_gem_shmem_object_unpin,
>>>>>>> .get_sg_table = drm_gem_shmem_object_get_sg_table,
>>>>>>> - .vmap = drm_gem_shmem_object_vmap,
>>>>>>> - .vunmap = drm_gem_shmem_object_vunmap,
>>>>>>> + .vmap = drm_gem_shmem_object_vmap_locked,
>>>>>>> + .vunmap = drm_gem_shmem_object_vunmap_locked,
>>>>>>
>>>>>> While I think we should indeed be consistent with the names, I would
>>>>>> also expect helpers to get the locking right by default.
>>>>>
>>>>> Wait, actually I think this patch does what you suggest already. The
>>>>> _locked() prefix tells the caller: "you should take care of the locking,
>>>>> I expect the lock to be held when this is hook/function is called". So
>>>>> helpers without the _locked() prefix take care of the locking (which I
>>>>> guess matches your 'helpers get the locking right' expectation), and
>>>>> those with the _locked() prefix don't.
>>>>
>>>> What I meant by "getting the locking right" is indeed a bit ambiguous,
>>>> sorry. What I'm trying to say I guess is that, in this particular case,
>>>> I don't think you can expect the vmap implementation to be called with
>>>> or without the locks held. The doc for that function will say that it's
>>>> either one or the other, but not both.
>>>>
>>>> So helpers should follow what is needed to provide a default vmap/vunmap
>>>> implementation, including what locking is expected from a vmap/vunmap
>>>> implementation.
>>>
>>> Hm, yeah, I think that's a matter of taste. When locking is often
>>> deferrable, like it is in DRM, I find it beneficial for funcions and
>>> function pointers to reflect the locking scheme, rather than relying on
>>> people properly reading the doc, especially when this is the only
>>> outlier in the group of drm_gem_object_funcs we already have, and it's
>>
Re: [PATCH v18 04/26] drm/shmem-helper: Refactor locked/unlocked functions
On 11/28/23 15:37, Boris Brezillon wrote:
> On Tue, 28 Nov 2023 12:14:42 +0100
> Maxime Ripard wrote:
>
>> Hi,
>>
>> On Fri, Nov 24, 2023 at 11:59:11AM +0100, Boris Brezillon wrote:
>>> On Fri, 24 Nov 2023 11:40:06 +0100
>>> Maxime Ripard wrote:
>>>
>>>> On Mon, Oct 30, 2023 at 02:01:43AM +0300, Dmitry Osipenko wrote:
>>>>> Add locked and remove unlocked postfixes from drm-shmem function names,
>>>>> making names consistent with the drm/gem core code.
>>>>>
>>>>> Reviewed-by: Boris Brezillon
>>>>> Suggested-by: Boris Brezillon
>>>>> Signed-off-by: Dmitry Osipenko
>>>>
>>>> This contradicts my earlier ack on a patch but...
>>>>
>>>>> ---
>>>>> drivers/gpu/drm/drm_gem_shmem_helper.c| 64 +--
>>>>> drivers/gpu/drm/lima/lima_gem.c | 8 +--
>>>>> drivers/gpu/drm/panfrost/panfrost_drv.c | 2 +-
>>>>> drivers/gpu/drm/panfrost/panfrost_gem.c | 6 +-
>>>>> .../gpu/drm/panfrost/panfrost_gem_shrinker.c | 2 +-
>>>>> drivers/gpu/drm/panfrost/panfrost_mmu.c | 2 +-
>>>>> drivers/gpu/drm/v3d/v3d_bo.c | 4 +-
>>>>> drivers/gpu/drm/virtio/virtgpu_object.c | 4 +-
>>>>> include/drm/drm_gem_shmem_helper.h| 36 +--
>>>>> 9 files changed, 64 insertions(+), 64 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>> b/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>> index 0d61f2b3e213..154585ddae08 100644
>>>>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>>>>> @@ -43,8 +43,8 @@ static const struct drm_gem_object_funcs
>>>>> drm_gem_shmem_funcs = {
>>>>> .pin = drm_gem_shmem_object_pin,
>>>>> .unpin = drm_gem_shmem_object_unpin,
>>>>> .get_sg_table = drm_gem_shmem_object_get_sg_table,
>>>>> - .vmap = drm_gem_shmem_object_vmap,
>>>>> - .vunmap = drm_gem_shmem_object_vunmap,
>>>>> + .vmap = drm_gem_shmem_object_vmap_locked,
>>>>> + .vunmap = drm_gem_shmem_object_vunmap_locked,
>>>>
>>>> While I think we should indeed be consistent with the names, I would
>>>> also expect helpers to get the locking right by default.
>>>
>>> Wait, actually I think this patch does what you suggest already. The
>>> _locked() prefix tells the caller: "you should take care of the locking,
>>> I expect the lock to be held when this is hook/function is called". So
>>> helpers without the _locked() prefix take care of the locking (which I
>>> guess matches your 'helpers get the locking right' expectation), and
>>> those with the _locked() prefix don't.
>>
>> What I meant by "getting the locking right" is indeed a bit ambiguous,
>> sorry. What I'm trying to say I guess is that, in this particular case,
>> I don't think you can expect the vmap implementation to be called with
>> or without the locks held. The doc for that function will say that it's
>> either one or the other, but not both.
>>
>> So helpers should follow what is needed to provide a default vmap/vunmap
>> implementation, including what locking is expected from a vmap/vunmap
>> implementation.
>
> Hm, yeah, I think that's a matter of taste. When locking is often
> deferrable, like it is in DRM, I find it beneficial for funcions and
> function pointers to reflect the locking scheme, rather than relying on
> people properly reading the doc, especially when this is the only
> outlier in the group of drm_gem_object_funcs we already have, and it's
> not event documented at the drm_gem_object_funcs level [1] :P.
>
>>
>> If that means that vmap is always called with the locks taken, then
>> drm_gem_shmem_object_vmap can just assume that it will be called with
>> the locks taken and there's no need to mention it in the name (and you
>> can probably sprinkle a couple of lockdep assertion to make sure the
>> locking is indeed consistent).
>
> Things get very confusing when you end up having drm_gem_shmem helpers
> that are suffixed with _locked() to encode the fact locking is the
> caller's responsibility and no suffix for the
> callee-takes-care-of-the-locking semantics, while other helpers that are
> not suffixed at all actually implement the
> caller-should
Re: [PATCH v18 22/26] drm/shmem-helper: Don't free refcounted GEM
On 11/23/23 12:08, Boris Brezillon wrote: > On Thu, 23 Nov 2023 01:30:24 +0300 > Dmitry Osipenko wrote: > >> On 11/13/23 12:54, Boris Brezillon wrote: >>> On Mon, 30 Oct 2023 02:02:01 +0300 >>> Dmitry Osipenko wrote: >>> >>>> Don't free refcounted shmem object to prevent use-after-free bug that >>>> is worse than a memory leak. >>>> >>>> Signed-off-by: Dmitry Osipenko >>>> --- >>>> drivers/gpu/drm/drm_gem_shmem_helper.c | 7 --- >>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>> >>>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c >>>> b/drivers/gpu/drm/drm_gem_shmem_helper.c >>>> index 6dd087f19ea3..4253c367dc07 100644 >>>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c >>>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c >>>> @@ -203,9 +203,10 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object >>>> *shmem) >>>>if (obj->import_attach) >>>>drm_prime_gem_destroy(obj, shmem->sgt); >>>> >>>> - drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count)); >>>> - drm_WARN_ON(obj->dev, refcount_read(>pages_use_count)); >>>> - drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count)); >>>> + if (drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count)) || >>>> + drm_WARN_ON(obj->dev, refcount_read(>pages_use_count)) || >>>> + drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count))) >>>> + return; >>> >>> I guess you're worried about ->sgt being referenced by the driver after >>> the GEM is destroyed. If we assume drivers don't cache the sgt and >>> always call get_pages_sgt() when they need it that shouldn't be an >>> issue. What we really don't want to release is the pages themselves, >>> but the GPU MMU might still have active mappings pointing to these >>> pages. >>> >>> In any case, I'm not against leaking the GEM object when any of these >>> counters are not zero, but can we at least have a comment in the >>> code explaining why we're doing that, so people don't have to go look >>> at the git history to figure it out. >> >> This patch is a minor improvement, it doesn't address any specific >> issue. This should be a common pattern in kernel. If you're giving a >> warning and know about the inevitable catastrophe, then avoid it if you can. > > Sure, I'm just asking that we add a comment to explain why we leak > memory here. Is that too much to ask? Will add the comment. The reason why I added this patch was unrelated to the sgt, that's what I'm talking about. -- Best regards, Dmitry
Re: [PATCH v18 15/26] drm/panfrost: Explicitly get and put drm-shmem pages
On 11/23/23 12:05, Boris Brezillon wrote:
> On Thu, 23 Nov 2023 01:04:56 +0300
> Dmitry Osipenko wrote:
>
>> On 11/10/23 13:53, Boris Brezillon wrote:
>>> Hm, there was no drm_gem_shmem_get_pages_sgt() call here, why should we
>>> add a drm_gem_shmem_get_pages()? What we should do instead is add a
>>> drm_gem_shmem_get_pages() for each drm_gem_shmem_get_pages_sgt() we
>>> have in the driver (in panfrost_mmu_map()), and add
>>> drm_gem_shmem_put_pages() calls where they are missing
>>> (panfrost_mmu_unmap()).
>>>
>>>> + if (err)
>>>> + goto err_free;
>>>> + }
>>>> +
>>>>return bo;
>>>> +
>>>> +err_free:
>>>> + drm_gem_shmem_free(>base);
>>>> +
>>>> + return ERR_PTR(err);
>>>> }
>>>>
>>>> struct drm_gem_object *
>>>> diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c
>>>> b/drivers/gpu/drm/panfrost/panfrost_mmu.c
>>>> index 770dab1942c2..ac145a98377b 100644
>>>> --- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
>>>> +++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
>>>> @@ -504,7 +504,7 @@ static int panfrost_mmu_map_fault_addr(struct
>>>> panfrost_device *pfdev, int as,
>>>>if (IS_ERR(pages[i])) {
>>>>ret = PTR_ERR(pages[i]);
>>>>pages[i] = NULL;
>>>> - goto err_pages;
>>>> + goto err_unlock;
>>>>}
>>>>}
>>>>
>>>> @@ -512,7 +512,7 @@ static int panfrost_mmu_map_fault_addr(struct
>>>> panfrost_device *pfdev, int as,
>>>>ret = sg_alloc_table_from_pages(sgt, pages + page_offset,
>>>>NUM_FAULT_PAGES, 0, SZ_2M, GFP_KERNEL);
>>>>if (ret)
>>>> - goto err_pages;
>>>> + goto err_unlock;
>>> Feels like the panfrost_gem_mapping object should hold a ref on the BO
>>> pages, not the BO itself, because, ultimately, the user of the BO is
>>> the GPU. This matches what I was saying about moving get/put_pages() to
>>> panfrost_mmu_map/unmap(): everytime a panfrost_gem_mapping becomes
>>> active, to want to take a pages ref, every time it becomes inactive,
>>> you should release the pages ref.
>>
>> The panfrost_mmu_unmap() is also used by shrinker when BO is purged. I'm
>> unhappy with how icky it all becomes if unmap is made to put pages.
>
> Why, that's exactly what's supposed to happen. If you mmu_unmap(), that
> means you no longer need the pages ref you got.
The drm_gem_shmem_purge() frees the pages. If mmu_unmap() frees pages too, then
it becomes odd for drm_gem_shmem_purge() that it needs to free pages that were
already freed.
>> Previously map() was implicitly allocating pages with get_sgt() and then
>> pages were implicitly released by drm_gem_shmem_free(). A non-heap BO is
>> mapped when it's created by Panfrost, hence the actual lifetime of pages
>> is kept unchanged by this patch.
>
> But the whole point of making it explicit is to control when pages are
> needed or not, isn't it. The fact we mmu_map() the BO at open time, and
> keep it mapped until it's not longer referenced is an implementation
> choice, and I don't think having pages_put() in mmu_unmap() changes
> that.
Previously, when the last mmu_unmap() was done, the pages were not released.
If you'll make unmap to put pages, then you can't map BO again because pages
are released by the last put() of unmap. In order to keep the old pages
allocation logic unchanged, the pages must be referenced while BO is alive, not
while mapping is alive.
Technically, the code can be changed to put pages on unmap. This requires
adding special quirk to drm_gem_shmem_purge() and then for Panfrost pages
should have the same lifetime as BO, hence why bother?
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 5ee98b6f0c94..5492610802a1 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -600,7 +600,9 @@ drm_gem_shmem_shrinker_put_pages_locked(struct
drm_gem_shmem_object *shmem)
if (shmem->evicted)
return;
- drm_gem_shmem_free_pages(shmem);
+ if (refcount_read(>pages_use_count))
+ drm_gem_shmem_free_pages(shmem);
+
drm_vma_node_unmap(>vma_node, dev->anon_inode->i_mapping);
}
@@ -608,7 +610,8 @@ void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object
*shmem)
{
Re: [PATCH v18 24/26] drm/virtio: Attach shmem BOs dynamically
On 11/23/23 01:37, Dmitry Osipenko wrote: > On 11/13/23 12:57, Boris Brezillon wrote: >> On Mon, 30 Oct 2023 02:02:03 +0300 >> Dmitry Osipenko wrote: >> >>> Prepare for addition of memory shrinker support by attaching shmem pages >>> to host dynamically on first use. Previously the attachment vq command >>> wasn't fenced and there was no vq kick made in the BO creation code path, >>> hence the attachment already was happening dynamically, but implicitly. >>> Making attachment explicitly dynamic will allow to simplify and reuse more >>> code when shrinker will be added. The virtio_gpu_object_shmem_init() now >>> works under the held reservation lock, which will be important to have for >>> shrinker to avoid moving pages while they are in active use by the driver. >> Ah, this commit might actually help getting rid of the workaround >> introduced in "drm/shmem-helper: Add common memory shrinker". > > Not really. The virtio_gpu_free_object() is unchanged, it's only > allocation that is being deferred and it's only done for a one BO type > (virtio-gpu has multiple BO types). s/allocation/attachment/ Pages are still allocated by virtio_gpu_object_create(). -- Best regards, Dmitry
Re: [PATCH v18 24/26] drm/virtio: Attach shmem BOs dynamically
On 11/13/23 12:57, Boris Brezillon wrote: > On Mon, 30 Oct 2023 02:02:03 +0300 > Dmitry Osipenko wrote: > >> Prepare for addition of memory shrinker support by attaching shmem pages >> to host dynamically on first use. Previously the attachment vq command >> wasn't fenced and there was no vq kick made in the BO creation code path, >> hence the attachment already was happening dynamically, but implicitly. >> Making attachment explicitly dynamic will allow to simplify and reuse more >> code when shrinker will be added. The virtio_gpu_object_shmem_init() now >> works under the held reservation lock, which will be important to have for >> shrinker to avoid moving pages while they are in active use by the driver. > Ah, this commit might actually help getting rid of the workaround > introduced in "drm/shmem-helper: Add common memory shrinker". Not really. The virtio_gpu_free_object() is unchanged, it's only allocation that is being deferred and it's only done for a one BO type (virtio-gpu has multiple BO types). -- Best regards, Dmitry
Re: [PATCH v18 22/26] drm/shmem-helper: Don't free refcounted GEM
On 11/13/23 12:54, Boris Brezillon wrote: > On Mon, 30 Oct 2023 02:02:01 +0300 > Dmitry Osipenko wrote: > >> Don't free refcounted shmem object to prevent use-after-free bug that >> is worse than a memory leak. >> >> Signed-off-by: Dmitry Osipenko >> --- >> drivers/gpu/drm/drm_gem_shmem_helper.c | 7 --- >> 1 file changed, 4 insertions(+), 3 deletions(-) >> >> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c >> b/drivers/gpu/drm/drm_gem_shmem_helper.c >> index 6dd087f19ea3..4253c367dc07 100644 >> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c >> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c >> @@ -203,9 +203,10 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object >> *shmem) >> if (obj->import_attach) >> drm_prime_gem_destroy(obj, shmem->sgt); >> >> -drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count)); >> -drm_WARN_ON(obj->dev, refcount_read(>pages_use_count)); >> -drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count)); >> +if (drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count)) || >> +drm_WARN_ON(obj->dev, refcount_read(>pages_use_count)) || >> +drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count))) >> +return; > > I guess you're worried about ->sgt being referenced by the driver after > the GEM is destroyed. If we assume drivers don't cache the sgt and > always call get_pages_sgt() when they need it that shouldn't be an > issue. What we really don't want to release is the pages themselves, > but the GPU MMU might still have active mappings pointing to these > pages. > > In any case, I'm not against leaking the GEM object when any of these > counters are not zero, but can we at least have a comment in the > code explaining why we're doing that, so people don't have to go look > at the git history to figure it out. This patch is a minor improvement, it doesn't address any specific issue. This should be a common pattern in kernel. If you're giving a warning and know about the inevitable catastrophe, then avoid it if you can. Actually, there are other similar cases in drm-shmem that can be improved. -- Best regards, Dmitry
Re: [PATCH v18 15/26] drm/panfrost: Explicitly get and put drm-shmem pages
On 11/10/23 13:53, Boris Brezillon wrote:
> Hm, there was no drm_gem_shmem_get_pages_sgt() call here, why should we
> add a drm_gem_shmem_get_pages()? What we should do instead is add a
> drm_gem_shmem_get_pages() for each drm_gem_shmem_get_pages_sgt() we
> have in the driver (in panfrost_mmu_map()), and add
> drm_gem_shmem_put_pages() calls where they are missing
> (panfrost_mmu_unmap()).
>
>> +if (err)
>> +goto err_free;
>> +}
>> +
>> return bo;
>> +
>> +err_free:
>> +drm_gem_shmem_free(>base);
>> +
>> +return ERR_PTR(err);
>> }
>>
>> struct drm_gem_object *
>> diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c
>> b/drivers/gpu/drm/panfrost/panfrost_mmu.c
>> index 770dab1942c2..ac145a98377b 100644
>> --- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
>> +++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
>> @@ -504,7 +504,7 @@ static int panfrost_mmu_map_fault_addr(struct
>> panfrost_device *pfdev, int as,
>> if (IS_ERR(pages[i])) {
>> ret = PTR_ERR(pages[i]);
>> pages[i] = NULL;
>> -goto err_pages;
>> +goto err_unlock;
>> }
>> }
>>
>> @@ -512,7 +512,7 @@ static int panfrost_mmu_map_fault_addr(struct
>> panfrost_device *pfdev, int as,
>> ret = sg_alloc_table_from_pages(sgt, pages + page_offset,
>> NUM_FAULT_PAGES, 0, SZ_2M, GFP_KERNEL);
>> if (ret)
>> -goto err_pages;
>> +goto err_unlock;
> Feels like the panfrost_gem_mapping object should hold a ref on the BO
> pages, not the BO itself, because, ultimately, the user of the BO is
> the GPU. This matches what I was saying about moving get/put_pages() to
> panfrost_mmu_map/unmap(): everytime a panfrost_gem_mapping becomes
> active, to want to take a pages ref, every time it becomes inactive,
> you should release the pages ref.
The panfrost_mmu_unmap() is also used by shrinker when BO is purged. I'm
unhappy with how icky it all becomes if unmap is made to put pages.
Previously map() was implicitly allocating pages with get_sgt() and then
pages were implicitly released by drm_gem_shmem_free(). A non-heap BO is
mapped when it's created by Panfrost, hence the actual lifetime of pages
is kept unchanged by this patch. The implicit allocation is turned into
explicit one, i.e. pages are explicitly allocated before BO is mapped.
--
Best regards,
Dmitry
Re: [PATCH v18 11/26] drm/shmem-helper: Prepare drm_gem_shmem_free() to shrinker addition
On 11/20/23 14:19, Boris Brezillon wrote:
...
- dma_resv_lock(shmem->base.resv, NULL);
-
drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count));
if (shmem->sgt) {
@@ -157,8 +171,6 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object
*shmem)
>>> If you drop the dma_resv_lock/unlock(), you should also replace the
>>> drm_gem_shmem_put_pages_locked() by a drm_gem_shmem_free_pages() in this
>>> commit.
>>
>> drm_gem_shmem_put_pages_locked() is exported by a later patch of this
>> series, it's not worthwhile to remove this function
>
> I'm not talking about removing drm_gem_shmem_put_pages_locked(), but
> replacing the drm_gem_shmem_put_pages_locked() call you have in
> drm_gem_shmem_free() by a drm_gem_shmem_free_pages(), so you don't end
> up with a lockdep warning when you stop exactly here in the patch
> series, which is important if we want to keep things bisectable.
Indeed, there is assert_locked() there. Thanks for the clarification :)
--
Best regards,
Dmitry
Re: [PATCH v18 11/26] drm/shmem-helper: Prepare drm_gem_shmem_free() to shrinker addition
On 11/10/23 13:16, Boris Brezillon wrote:
> On Mon, 30 Oct 2023 02:01:50 +0300
> Dmitry Osipenko wrote:
>
>> Prepare drm_gem_shmem_free() to addition of memory shrinker support
>> to drm-shmem by adding and using variant of put_pages() that doesn't
>> touch reservation lock. Reservation shouldn't be touched because lockdep
>> will trigger a bogus warning about locking contention with fs_reclaim
>> code paths that can't happen during the time when GEM is freed and
>> lockdep doesn't know about that.
>>
>> Signed-off-by: Dmitry Osipenko
>> ---
>> drivers/gpu/drm/drm_gem_shmem_helper.c | 35 +-
>> 1 file changed, 18 insertions(+), 17 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
>> b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> index 08b5a57c59d8..24ff2b99e75b 100644
>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> @@ -128,6 +128,22 @@ struct drm_gem_shmem_object
>> *drm_gem_shmem_create(struct drm_device *dev, size_t
>> }
>> EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>>
>> +static void
>> +drm_gem_shmem_free_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +struct drm_gem_object *obj = >base;
>> +
>> +#ifdef CONFIG_X86
>> +if (shmem->map_wc)
>> +set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> +#endif
>> +
>> +drm_gem_put_pages(obj, shmem->pages,
>> + shmem->pages_mark_dirty_on_put,
>> + shmem->pages_mark_accessed_on_put);
>> +shmem->pages = NULL;
>> +}
>> +
>> /**
>> * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>> * @shmem: shmem GEM object to free
>> @@ -142,8 +158,6 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object
>> *shmem)
>> if (obj->import_attach) {
>> drm_prime_gem_destroy(obj, shmem->sgt);
>> } else {
>> -dma_resv_lock(shmem->base.resv, NULL);
>> -
>> drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count));
>>
>> if (shmem->sgt) {
>> @@ -157,8 +171,6 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object
>> *shmem)
>>
> If you drop the dma_resv_lock/unlock(), you should also replace the
> drm_gem_shmem_put_pages_locked() by a drm_gem_shmem_free_pages() in this
> commit.
drm_gem_shmem_put_pages_locked() is exported by a later patch of this
series, it's not worthwhile to remove this function
--
Best regards,
Dmitry
Re: [PATCH] drm/virtio: add definition for venus capset
On 9/21/23 00:16, Dmitry Osipenko wrote:
> On 9/15/23 13:59, Huang Rui wrote:
>> This definition is used fro qemu, and qemu imports this marco in the
>> headers to enable venus for virtio gpu. So it should add it even kernel
>> doesn't use this.
>>
>> Signed-off-by: Huang Rui
>> ---
>>
>> Hi all,
>>
>> We would like to add a new definition for venus capset, it will be used for
>> qemu. Please see details on below discussion:
>>
>> https://lore.kernel.org/qemu-devel/b82982aa-5b9e-481e-9491-b9313877b...@daynix.com/
>>
>> Thanks,
>> Ray
>>
>> include/uapi/linux/virtio_gpu.h | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/include/uapi/linux/virtio_gpu.h
>> b/include/uapi/linux/virtio_gpu.h
>> index f556fde07b76..0e21f3998108 100644
>> --- a/include/uapi/linux/virtio_gpu.h
>> +++ b/include/uapi/linux/virtio_gpu.h
>> @@ -309,6 +309,8 @@ struct virtio_gpu_cmd_submit {
>>
>> #define VIRTIO_GPU_CAPSET_VIRGL 1
>> #define VIRTIO_GPU_CAPSET_VIRGL2 2
>> +/* 3 is reserved for gfxstream */
>> +#define VIRTIO_GPU_CAPSET_VENUS 4
>
> Could you please add all other capsets, so we won't needed to do it
> again in the future
I've opened request to update virtio-spec with the corrected/updated
capsets https://github.com/oasis-tcs/virtio-spec/issues/182. I'm
expecting that it will take some time until spec change will be merged
and now leaning to apply this v1 patch to not hold the Venus work.
Gerd, do you have objections? R-b/ack?
--
Best regards,
Dmitry
Re: [PATCH v1] drm/virtio: Fix return value for VIRTGPU_CONTEXT_PARAM_DEBUG_NAME
On 11/12/23 01:42, Dmitry Osipenko wrote:
> The strncpy_from_user() returns number of copied bytes and not zero on
> success. The non-zero return value of ioctl is treated as error. Return
> zero on success instead of the number of copied bytes.
>
> Fixes: 7add80126bce ("drm/uapi: add explicit virtgpu context debug name")
> Signed-off-by: Dmitry Osipenko
> ---
> drivers/gpu/drm/virtio/virtgpu_ioctl.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
> b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
> index 1e2042419f95..e4f76f315550 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
> +++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
> @@ -665,6 +665,7 @@ static int virtio_gpu_context_init_ioctl(struct
> drm_device *dev,
> goto out_unlock;
>
> vfpriv->explicit_debug_name = true;
> + ret = 0;
> break;
> default:
> ret = -EINVAL;
Applied to misc-next
--
Best regards,
Dmitry
Re: [RFC PATCH v2 1/1] drm/virtio: new fence for every plane update
On 10/23/23 20:31, Kim, Dongwon wrote: ... >> Please write a guide how to test it. Are you using spice for the >> multi-display >> viewer? > > [DW] Yeah, let me come up with a simple test case. So we use virtio-gpu as > KMS device. It is used to share the guest frame with QEMU. > SPICE is one of client solutions we use but we primarily use QEMU GTK-UI w/ > multi displays (QEMU with this params '-device > virtio-vga,max_outputs=2,blob=true'). > Thanks! I'm having trouble wit h reproducing the issue. For GTK-UI you should be using virtio-vga-gl, shouldn't you? I assume you meant that your simple case is reproducible using dGPU, correct? I need a test case that is reproducing using virgl and no dGPU. Using virtio-vga-gl+virgl+max_outputs=2 I don't see any issues. In the same time switching to the second virtual display doesn't work with Qemu's GTK UI, I'm getting black screen for the second display. In the KDE settings I have two displays and it says both are active. For the first display that works, I don't see wrong frames ordering. Please give me a test case that is reproducing using latest version of vanilla/upstream Qemu. -- Best regards, Dmitry
Re: [PATCH 2/2] drm/virtio: Modify RESOURCE_GET_LAYOUT ioctl
On 11/10/23 10:16, Julia Zhang wrote: > Modify RESOURCE_GET_LAYOUT ioctl to handle the use case that query > correct stride for guest linear resource before it is created. > > Signed-off-by: Julia Zhang > --- > drivers/gpu/drm/virtio/virtgpu_drv.h | 26 -- > drivers/gpu/drm/virtio/virtgpu_ioctl.c | 47 -- > drivers/gpu/drm/virtio/virtgpu_vq.c| 35 +++ > include/uapi/drm/virtgpu_drm.h | 6 ++-- > include/uapi/linux/virtio_gpu.h| 8 ++--- > 5 files changed, 66 insertions(+), 56 deletions(-) 1. Please squash this all into a single patch. For upstream kernel it's not acceptable to have subsequent commits modifying previous commits. To commit message add your s-o-b, your co-developed-by tags and a brief comment explaining changes you've done to the original patch. Signed-off-by: Daniel Stone Co-developed-by: Julia Zhang # query correct stride for guest linear resource before it's created Signed-off-by: Julia Zhang 2. Make sure that patch passes `scripts/checkpatch.pl` 3. Add link to the commit message for the relevant Mesa MR that makes use of the new ioctl. The MR should be already merged or ready to be merged. Link: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/123456 -- Best regards, Dmitry
[PATCH v1] drm/virtio: Fix return value for VIRTGPU_CONTEXT_PARAM_DEBUG_NAME
The strncpy_from_user() returns number of copied bytes and not zero on
success. The non-zero return value of ioctl is treated as error. Return
zero on success instead of the number of copied bytes.
Fixes: 7add80126bce ("drm/uapi: add explicit virtgpu context debug name")
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/virtio/virtgpu_ioctl.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index 1e2042419f95..e4f76f315550 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -665,6 +665,7 @@ static int virtio_gpu_context_init_ioctl(struct drm_device
*dev,
goto out_unlock;
vfpriv->explicit_debug_name = true;
+ ret = 0;
break;
default:
ret = -EINVAL;
--
2.41.0
Re: [PATCH v3 2/2] drm/uapi: add explicit virtgpu context debug name
On 10/18/23 21:17, Gurchetan Singh wrote:
> + case VIRTGPU_CONTEXT_PARAM_DEBUG_NAME:
> + if (vfpriv->explicit_debug_name) {
> + ret = -EINVAL;
> + goto out_unlock;
> + }
> +
> + ret = strncpy_from_user(vfpriv->debug_name,
> + u64_to_user_ptr(value),
> + DEBUG_NAME_MAX_LEN - 1);
> +
> + if (ret < 0) {
> + ret = -EFAULT;
> + goto out_unlock;
> + }
> +
> + vfpriv->explicit_debug_name = true;
> + break;
Spotted a problem here. The ret needs to be set to zero on success. I'll
send the fix shortly. Gurchetan you should've been getting the
DRM_IOCTL_VIRTGPU_CONTEXT_INIT failure from gfxstream when you tested
this patch, haven't you?
Also noticed that the patch title says "drm/uapi" instead of
"drm/virtio". My bad for not noticing it earlier. Please be more careful
next time too :)
--
Best regards,
Dmitry
Re: [PATCH v3 2/2] drm/uapi: add explicit virtgpu context debug name
On 10/18/23 21:17, Gurchetan Singh wrote:
> There are two problems with the current method of determining the
> virtio-gpu debug name.
>
> 1) TASK_COMM_LEN is defined to be 16 bytes only, and this is a
>Linux kernel idiom (see PR_SET_NAME + PR_GET_NAME). Though,
>Android/FreeBSD get around this via setprogname(..)/getprogname(..)
>in libc.
>
>On Android, names longer than 16 bytes are common. For example,
>one often encounters a program like "com.android.systemui".
>
>The virtio-gpu spec allows the debug name to be up to 64 bytes, so
>ideally userspace should be able to set debug names up to 64 bytes.
>
> 2) The current implementation determines the debug name using whatever
>task initiated virtgpu. This is could be a "RenderThread" of a
>larger program, when we actually want to propagate the debug name
>of the program.
>
> To fix these issues, add a new CONTEXT_INIT param that allows userspace
> to set the debug name when creating a context.
>
> It takes a null-terminated C-string as the param value. The length of the
> string (excluding the terminator) **should** be <= 64 bytes. Otherwise,
> the debug_name will be truncated to 64 bytes.
>
> Link to open-source userspace:
> https://android-review.googlesource.com/c/platform/hardware/google/gfxstream/+/2787176
>
> Signed-off-by: Gurchetan Singh
> Reviewed-by: Josh Simonot
> ---
> Fixes suggested by Dmitry Osipenko
> v2:
> - Squash implementation and UAPI change into one commit
> - Avoid unnecessary casts
> - Use bool when necessary
> v3:
> - Use DEBUG_NAME_MAX_LEN - 1 when copying string
>
> drivers/gpu/drm/virtio/virtgpu_drv.h | 5
> drivers/gpu/drm/virtio/virtgpu_ioctl.c | 39 ++
> include/uapi/drm/virtgpu_drm.h | 2 ++
> 3 files changed, 40 insertions(+), 6 deletions(-)
...
> + ret = strncpy_from_user(vfpriv->debug_name,
> + u64_to_user_ptr(value),
> + DEBUG_NAME_MAX_LEN - 1);
> +
> + if (ret < 0) {
> + ret = -EFAULT;
> + goto out_unlock;
> + }
The strncpy_from_user() itself returns -EFAULT. I changed code to return
the "ret" directly and applied to misc-next.
Gerd, let us know if have any objections to this patch.
--
Best regards,
Dmitry
Re: [PATCH v18 25/26] drm/virtio: Support shmem shrinking
On 11/4/23 01:55, Gurchetan Singh wrote: > On Sun, Oct 29, 2023 at 4:03 PM Dmitry Osipenko < > dmitry.osipe...@collabora.com> wrote: > >> Support generic drm-shmem memory shrinker and add new madvise IOCTL to >> the VirtIO-GPU driver. BO cache manager of Mesa driver will mark BOs as >> "don't need" using the new IOCTL to let shrinker purge the marked BOs on >> OOM, the shrinker will also evict unpurgeable shmem BOs from memory if >> guest supports SWAP file or partition. >> >> Acked-by: Gerd Hoffmann >> Signed-off-by: Daniel Almeida >> Signed-off-by: Dmitry Osipenko >> --- >> drivers/gpu/drm/virtio/virtgpu_drv.h| 13 +- >> drivers/gpu/drm/virtio/virtgpu_gem.c| 35 ++ >> drivers/gpu/drm/virtio/virtgpu_ioctl.c | 25 ++ >> drivers/gpu/drm/virtio/virtgpu_kms.c| 8 >> drivers/gpu/drm/virtio/virtgpu_object.c | 61 + >> drivers/gpu/drm/virtio/virtgpu_vq.c | 40 >> include/uapi/drm/virtgpu_drm.h | 14 ++ >> 7 files changed, 195 insertions(+), 1 deletion(-) ... > > Link to open-source userspace? https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15278 I'll add it to the commit message > Also, don't you need a VIRTGPU_PARAM_MADVISE_SUPPORTED or is the plan to > use a DRM version? The ioctl() returns error if DRM_VIRTGPU_MADVISE unsupported, extra flags not needed by userspace > Overall, the series for a generic shrinker seems useful for a wide variety > of DRM drivers, such as Panfrost. > > For virtio-gpu, it could be a tad VIRGIL specific: since other context > types (gfxstream gles, DRM, vk contexts) decrease memory consumption by > either not allocating shadow buffers for textures/buffers, or using blob > memory. > > Maybe we need to design with blob in mind, since we expect virgl to be > deprecated. On Android, it basically is with ANGLE and native contexts. > On Linux, Zink looks good too. Even with memory issues fixed, virgl is > unattractive compared to those solutions. We should finish shmem first since started with it, then move to blobs. My rough idea for the blobs is to use a timer-based approach to avoid frequent virtio syncing with host that will be bad for performance otherwise. Virtio-gpu kernel driver will maintain a list of purgeable blobs and will send the list of idling blobs down to host after a period of inactivity. Virgl may be not interesting for CrOS, but Qemu will continue supporting it. I also expect that today's ARM Chromebooks which use Virgl and only support GL will continue to use Virgl for the next 4 years. > Android specific idea: I wonder if we could tie GEM blob buffers usage to > the lmkd and kill based on that ... ? > > https://source.android.com/docs/core/perf/lmkd > > Is there GEM buffer accounting infrastructure already? Are you talking about killing a guest APP that uses host blobs when host is under pressure? I'm not familiar with how arcvm works, but may assume that it runs a VM per APP. In that case VM is just another process from the lmkd perspective that will be taken down on OOM, i.e. blob buffers already should be seen as a part of a VM's memory by lmkd when they reside in sysmem. -- Best regards, Dmitry
[PATCH v18 14/26] drm/lima: Explicitly get and put drm-shmem pages
To simplify the drm-shmem refcnt handling, we're moving away from
the implicit get_pages() that is used by get_pages_sgt(). From now on
drivers will have to pin pages while they use sgt. Lima driver doesn't
have shrinker, hence pages are pinned and sgt is valid as long as pages'
use-count > 0.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/lima/lima_gem.c | 18 --
drivers/gpu/drm/lima/lima_gem.h | 1 +
2 files changed, 17 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index 988e74f67465..d255f5775dac 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -46,6 +46,7 @@ int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm)
return -ENOMEM;
}
+ bo->put_pages = true;
bo->base.pages = pages;
refcount_set(>base.pages_use_count, 1);
@@ -115,6 +116,7 @@ int lima_gem_create_handle(struct drm_device *dev, struct
drm_file *file,
return PTR_ERR(shmem);
obj = >base;
+ bo = to_lima_bo(obj);
/* Mali Utgard GPU can only support 32bit address space */
mask = mapping_gfp_mask(obj->filp->f_mapping);
@@ -123,13 +125,19 @@ int lima_gem_create_handle(struct drm_device *dev, struct
drm_file *file,
mapping_set_gfp_mask(obj->filp->f_mapping, mask);
if (is_heap) {
- bo = to_lima_bo(obj);
err = lima_heap_alloc(bo, NULL);
if (err)
goto out;
} else {
- struct sg_table *sgt = drm_gem_shmem_get_pages_sgt(shmem);
+ struct sg_table *sgt;
+
+ err = drm_gem_shmem_get_pages(shmem);
+ if (err)
+ goto out;
+
+ bo->put_pages = true;
+ sgt = drm_gem_shmem_get_pages_sgt(shmem);
if (IS_ERR(sgt)) {
err = PTR_ERR(sgt);
goto out;
@@ -139,6 +147,9 @@ int lima_gem_create_handle(struct drm_device *dev, struct
drm_file *file,
err = drm_gem_handle_create(file, obj, handle);
out:
+ if (err && bo->put_pages)
+ drm_gem_shmem_put_pages(shmem);
+
/* drop reference from allocate - handle holds it now */
drm_gem_object_put(obj);
@@ -152,6 +163,9 @@ static void lima_gem_free_object(struct drm_gem_object *obj)
if (!list_empty(>va))
dev_err(obj->dev->dev, "lima gem free bo still has va\n");
+ if (bo->put_pages)
+ drm_gem_shmem_put_pages(>base);
+
drm_gem_shmem_free(>base);
}
diff --git a/drivers/gpu/drm/lima/lima_gem.h b/drivers/gpu/drm/lima/lima_gem.h
index ccea06142f4b..dc5a6d465c80 100644
--- a/drivers/gpu/drm/lima/lima_gem.h
+++ b/drivers/gpu/drm/lima/lima_gem.h
@@ -16,6 +16,7 @@ struct lima_bo {
struct list_head va;
size_t heap_size;
+ bool put_pages;
};
static inline struct lima_bo *
--
2.41.0
[PATCH v18 25/26] drm/virtio: Support shmem shrinking
Support generic drm-shmem memory shrinker and add new madvise IOCTL to
the VirtIO-GPU driver. BO cache manager of Mesa driver will mark BOs as
"don't need" using the new IOCTL to let shrinker purge the marked BOs on
OOM, the shrinker will also evict unpurgeable shmem BOs from memory if
guest supports SWAP file or partition.
Acked-by: Gerd Hoffmann
Signed-off-by: Daniel Almeida
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/virtio/virtgpu_drv.h| 13 +-
drivers/gpu/drm/virtio/virtgpu_gem.c| 35 ++
drivers/gpu/drm/virtio/virtgpu_ioctl.c | 25 ++
drivers/gpu/drm/virtio/virtgpu_kms.c| 8
drivers/gpu/drm/virtio/virtgpu_object.c | 61 +
drivers/gpu/drm/virtio/virtgpu_vq.c | 40
include/uapi/drm/virtgpu_drm.h | 14 ++
7 files changed, 195 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h
b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 421f524ae1de..33a78b24c272 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -278,7 +278,7 @@ struct virtio_gpu_fpriv {
};
/* virtgpu_ioctl.c */
-#define DRM_VIRTIO_NUM_IOCTLS 12
+#define DRM_VIRTIO_NUM_IOCTLS 13
extern struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS];
void virtio_gpu_create_context(struct drm_device *dev, struct drm_file *file);
@@ -316,6 +316,8 @@ void virtio_gpu_array_put_free_delayed(struct
virtio_gpu_device *vgdev,
void virtio_gpu_array_put_free_work(struct work_struct *work);
int virtio_gpu_array_prepare(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object_array *objs);
+int virtio_gpu_gem_host_mem_release(struct virtio_gpu_object *bo);
+int virtio_gpu_gem_madvise(struct virtio_gpu_object *obj, int madv);
int virtio_gpu_gem_pin(struct virtio_gpu_object *bo);
void virtio_gpu_gem_unpin(struct virtio_gpu_object *bo);
@@ -329,6 +331,8 @@ void virtio_gpu_cmd_create_resource(struct
virtio_gpu_device *vgdev,
struct virtio_gpu_fence *fence);
void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object *bo);
+int virtio_gpu_cmd_release_resource(struct virtio_gpu_device *vgdev,
+ struct virtio_gpu_object *bo);
void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
uint64_t offset,
uint32_t width, uint32_t height,
@@ -349,6 +353,9 @@ void virtio_gpu_object_attach(struct virtio_gpu_device
*vgdev,
struct virtio_gpu_object *obj,
struct virtio_gpu_mem_entry *ents,
unsigned int nents);
+void virtio_gpu_object_detach(struct virtio_gpu_device *vgdev,
+ struct virtio_gpu_object *obj,
+ struct virtio_gpu_fence *fence);
void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,
struct virtio_gpu_output *output);
int virtio_gpu_cmd_get_display_info(struct virtio_gpu_device *vgdev);
@@ -492,4 +499,8 @@ void virtio_gpu_vram_unmap_dma_buf(struct device *dev,
int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
struct drm_file *file);
+/* virtgpu_gem_shrinker.c */
+int virtio_gpu_gem_shrinker_init(struct virtio_gpu_device *vgdev);
+void virtio_gpu_gem_shrinker_fini(struct virtio_gpu_device *vgdev);
+
#endif
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c
b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 97e67064c97e..748f7bbb0e6d 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -147,10 +147,20 @@ void virtio_gpu_gem_object_close(struct drm_gem_object
*obj,
struct virtio_gpu_device *vgdev = obj->dev->dev_private;
struct virtio_gpu_fpriv *vfpriv = file->driver_priv;
struct virtio_gpu_object_array *objs;
+ struct virtio_gpu_object *bo;
if (!vgdev->has_virgl_3d)
return;
+ bo = gem_to_virtio_gpu_obj(obj);
+
+ /*
+* Purged BO was already detached and released, the resource ID
+* is invalid by now.
+*/
+ if (!virtio_gpu_gem_madvise(bo, VIRTGPU_MADV_WILLNEED))
+ return;
+
objs = virtio_gpu_array_alloc(1);
if (!objs)
return;
@@ -315,6 +325,31 @@ int virtio_gpu_array_prepare(struct virtio_gpu_device
*vgdev,
return ret;
}
+int virtio_gpu_gem_madvise(struct virtio_gpu_object *bo, int madv)
+{
+ if (virtio_gpu_is_shmem(bo))
+ return drm_gem_shmem_object_madvise(>base.base, madv);
+
+ return 1;
+}
+
+int virtio_gpu_gem_host_mem_release(struct virtio_gpu_object *bo)
+{
+ struct virtio_gpu_device *vgdev = bo->base.base.dev-&
[PATCH v18 08/26] drm/shmem-helper: Add and use lockless drm_gem_shmem_get_pages()
Add lockless drm_gem_shmem_get_pages() helper that skips taking reservation
lock if pages_use_count is non-zero, leveraging from atomicity of the
refcount_t. Make drm_gem_shmem_mmap() to utilize the new helper.
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 19 +++
1 file changed, 15 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 6e02643ed87e..41b749bedb11 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -226,6 +226,20 @@ void drm_gem_shmem_put_pages_locked(struct
drm_gem_shmem_object *shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
+static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
+{
+ int ret;
+
+ if (refcount_inc_not_zero(>pages_use_count))
+ return 0;
+
+ dma_resv_lock(shmem->base.resv, NULL);
+ ret = drm_gem_shmem_get_pages_locked(shmem);
+ dma_resv_unlock(shmem->base.resv);
+
+ return ret;
+}
+
static int drm_gem_shmem_pin_locked(struct drm_gem_shmem_object *shmem)
{
int ret;
@@ -609,10 +623,7 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem,
struct vm_area_struct
return ret;
}
- dma_resv_lock(shmem->base.resv, NULL);
- ret = drm_gem_shmem_get_pages_locked(shmem);
- dma_resv_unlock(shmem->base.resv);
-
+ ret = drm_gem_shmem_get_pages(shmem);
if (ret)
return ret;
--
2.41.0
[PATCH v18 19/26] drm/shmem-helper: Add common memory shrinker
Introduce common drm-shmem shrinker for DRM drivers.
To start using drm-shmem shrinker drivers should do the following:
1. Implement evict() callback of GEM object where driver should check
whether object is purgeable or evictable using drm-shmem helpers and
perform the shrinking action
2. Initialize drm-shmem internals using drmm_gem_shmem_init(drm_device),
which will register drm-shmem shrinker
3. Implement madvise IOCTL that will use drm_gem_shmem_madvise()
Signed-off-by: Daniel Almeida
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c| 386 +-
.../gpu/drm/panfrost/panfrost_gem_shrinker.c | 9 +-
include/drm/drm_device.h | 10 +-
include/drm/drm_gem_shmem_helper.h| 68 ++-
4 files changed, 450 insertions(+), 23 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 1420d2166b76..007521bea302 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -20,6 +20,7 @@
#include
#include
#include
+#include
#include
#include
@@ -88,8 +89,6 @@ __drm_gem_shmem_create(struct drm_device *dev, size_t size,
bool private)
if (ret)
goto err_release;
- INIT_LIST_HEAD(>madv_list);
-
if (!private) {
/*
* Our buffers are kept pinned, so allocating them
@@ -128,11 +127,49 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct
drm_device *dev, size_t
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
+static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
+{
+ return (shmem->madv >= 0) && shmem->base.funcs->evict &&
+ refcount_read(>pages_use_count) &&
+ !refcount_read(>pages_pin_count) &&
+ !shmem->base.dma_buf && !shmem->base.import_attach &&
+ !shmem->evicted;
+}
+
+static void
+drm_gem_shmem_shrinker_update_lru_locked(struct drm_gem_shmem_object *shmem)
+{
+ struct drm_gem_object *obj = >base;
+ struct drm_gem_shmem *shmem_mm = obj->dev->shmem_mm;
+ struct drm_gem_shmem_shrinker *shmem_shrinker = _mm->shrinker;
+
+ dma_resv_assert_held(shmem->base.resv);
+
+ if (!shmem_shrinker || obj->import_attach)
+ return;
+
+ if (shmem->madv < 0)
+ drm_gem_lru_remove(>base);
+ else if (drm_gem_shmem_is_evictable(shmem) ||
drm_gem_shmem_is_purgeable(shmem))
+ drm_gem_lru_move_tail(_shrinker->lru_evictable,
>base);
+ else if (shmem->evicted)
+ drm_gem_lru_move_tail(_shrinker->lru_evicted,
>base);
+ else if (!shmem->pages)
+ drm_gem_lru_remove(>base);
+ else
+ drm_gem_lru_move_tail(_shrinker->lru_pinned,
>base);
+}
+
static void
drm_gem_shmem_free_pages(struct drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
+ if (!shmem->pages) {
+ drm_WARN_ON(obj->dev, !shmem->evicted && shmem->madv >= 0);
+ return;
+ }
+
if (shmem->sgt) {
dma_unmap_sgtable(obj->dev->dev, shmem->sgt,
DMA_BIDIRECTIONAL, 0);
@@ -175,15 +212,25 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object
*shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
-static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
+static int
+drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
struct page **pages;
+ drm_WARN_ON(obj->dev, obj->import_attach);
+
dma_resv_assert_held(shmem->base.resv);
- if (refcount_inc_not_zero(>pages_use_count))
+ if (shmem->madv < 0) {
+ drm_WARN_ON(obj->dev, shmem->pages);
+ return -ENOMEM;
+ }
+
+ if (shmem->pages) {
+ drm_WARN_ON(obj->dev, !shmem->evicted);
return 0;
+ }
pages = drm_gem_get_pages(obj);
if (IS_ERR(pages)) {
@@ -204,8 +251,29 @@ static int drm_gem_shmem_get_pages_locked(struct
drm_gem_shmem_object *shmem)
shmem->pages = pages;
+ return 0;
+}
+
+static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
+{
+ int err;
+
+ dma_resv_assert_held(shmem->base.resv);
+
+ if (shmem->madv < 0)
+ return -ENOMEM;
+
+ if (refcount_inc_not_zero(>pages_use_count))
+ return 0;
+
+ err = drm_gem_shmem_acquire_pages(shmem);
+ if (err)
+ return err;
+
refcount_set(>pages_use_count, 1);
+ drm_gem_shmem_shrinker_update_lru_locked(shm
[PATCH v18 13/26] drm/shmem-helper: Add drm_gem_shmem_put_pages()
We're going to move away from having implicit get_pages() done by
get_pages_sgt() to ease simplify refcnt handling. Drivers will manage
get/put_pages() by themselves. Add drm_gem_shmem_put_pages().
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 20
include/drm/drm_gem_shmem_helper.h | 1 +
2 files changed, 21 insertions(+)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index ca6f422c0dfc..f371ebc6f85c 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -217,6 +217,7 @@ static int drm_gem_shmem_get_pages_locked(struct
drm_gem_shmem_object *shmem)
* @shmem: shmem GEM object
*
* This function decreases the use count and puts the backing pages when use
drops to zero.
+ * Caller must hold GEM's reservation lock.
*/
void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem)
{
@@ -227,6 +228,25 @@ void drm_gem_shmem_put_pages_locked(struct
drm_gem_shmem_object *shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
+/*
+ * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a
shmem GEM object
+ * @shmem: shmem GEM object
+ *
+ * This function decreases the use count and puts the backing pages when use
drops to zero.
+ * It's unlocked version of drm_gem_shmem_put_pages_locked(), caller must not
hold
+ * GEM's reservation lock.
+ */
+void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
+{
+ if (refcount_dec_not_one(>pages_use_count))
+ return;
+
+ dma_resv_lock(shmem->base.resv, NULL);
+ drm_gem_shmem_put_pages_locked(shmem);
+ dma_resv_unlock(shmem->base.resv);
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages);
+
/*
* drm_gem_shmem_get_pages - Increase use count on the backing pages for a
shmem GEM object
* @shmem: shmem GEM object
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index 45cd293e10a4..6aad3e27d7ee 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -111,6 +111,7 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct
drm_device *dev, size_t
void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem);
int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem);
+void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem);
void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem);
int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem);
void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem);
--
2.41.0
[PATCH v18 12/26] drm/shmem-helper: Make drm_gem_shmem_get_pages() public
We're going to move away from having implicit get_pages() done by
get_pages_sgt() to ease simplify refcnt handling. Drivers will manage
get/put_pages() by themselves. Expose the drm_gem_shmem_get_pages()
in a public drm-shmem API.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 10 +-
include/drm/drm_gem_shmem_helper.h | 1 +
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 24ff2b99e75b..ca6f422c0dfc 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -227,7 +227,14 @@ void drm_gem_shmem_put_pages_locked(struct
drm_gem_shmem_object *shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
-static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
+/*
+ * drm_gem_shmem_get_pages - Increase use count on the backing pages for a
shmem GEM object
+ * @shmem: shmem GEM object
+ *
+ * This function Increases the use count and allocates the backing pages if
+ * use-count equals to zero.
+ */
+int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
{
int ret;
@@ -240,6 +247,7 @@ static int drm_gem_shmem_get_pages(struct
drm_gem_shmem_object *shmem)
return ret;
}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_get_pages);
static int drm_gem_shmem_pin_locked(struct drm_gem_shmem_object *shmem)
{
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index e7b3f4c02bf5..45cd293e10a4 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -110,6 +110,7 @@ struct drm_gem_shmem_object {
struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev,
size_t size);
void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem);
+int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem);
void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem);
int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem);
void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem);
--
2.41.0
[PATCH v18 18/26] drm/shmem-helper: Change sgt allocation policy
In a preparation to addition of drm-shmem memory shrinker support, change
the SGT allocation policy in this way:
1. SGT can be allocated only if shmem pages are pinned at the
time of allocation, otherwise allocation fails.
2. Drivers must ensure that pages are pinned during the time of SGT usage
and should get new SGT if pages were unpinned.
This new policy is required by the shrinker because it will move pages
to/from SWAP unless pages are pinned, invalidating SGT pointer once pages
are relocated.
Previous patches prepared drivers to the new policy.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 51 +-
1 file changed, 26 insertions(+), 25 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index f371ebc6f85c..1420d2166b76 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -133,6 +133,14 @@ drm_gem_shmem_free_pages(struct drm_gem_shmem_object
*shmem)
{
struct drm_gem_object *obj = >base;
+ if (shmem->sgt) {
+ dma_unmap_sgtable(obj->dev->dev, shmem->sgt,
+ DMA_BIDIRECTIONAL, 0);
+ sg_free_table(shmem->sgt);
+ kfree(shmem->sgt);
+ shmem->sgt = NULL;
+ }
+
#ifdef CONFIG_X86
if (shmem->map_wc)
set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
@@ -155,23 +163,12 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object
*shmem)
{
struct drm_gem_object *obj = >base;
- if (obj->import_attach) {
+ if (obj->import_attach)
drm_prime_gem_destroy(obj, shmem->sgt);
- } else {
- drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count));
-
- if (shmem->sgt) {
- dma_unmap_sgtable(obj->dev->dev, shmem->sgt,
- DMA_BIDIRECTIONAL, 0);
- sg_free_table(shmem->sgt);
- kfree(shmem->sgt);
- }
- if (shmem->pages)
- drm_gem_shmem_put_pages_locked(shmem);
- drm_WARN_ON(obj->dev, refcount_read(>pages_use_count));
- drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count));
- }
+ drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count));
+ drm_WARN_ON(obj->dev, refcount_read(>pages_use_count));
+ drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count));
drm_gem_object_release(obj);
kfree(shmem);
@@ -705,6 +702,9 @@ struct sg_table *drm_gem_shmem_get_sg_table(struct
drm_gem_shmem_object *shmem)
drm_WARN_ON(obj->dev, obj->import_attach);
+ if (drm_WARN_ON(obj->dev, !shmem->pages))
+ return ERR_PTR(-ENOMEM);
+
return drm_prime_pages_to_sg(obj->dev, shmem->pages, obj->size >>
PAGE_SHIFT);
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_get_sg_table);
@@ -720,15 +720,10 @@ static struct sg_table
*drm_gem_shmem_get_pages_sgt_locked(struct drm_gem_shmem_
drm_WARN_ON(obj->dev, obj->import_attach);
- ret = drm_gem_shmem_get_pages_locked(shmem);
- if (ret)
- return ERR_PTR(ret);
-
sgt = drm_gem_shmem_get_sg_table(shmem);
- if (IS_ERR(sgt)) {
- ret = PTR_ERR(sgt);
- goto err_put_pages;
- }
+ if (IS_ERR(sgt))
+ return sgt;
+
/* Map the pages for use by the h/w. */
ret = dma_map_sgtable(obj->dev->dev, sgt, DMA_BIDIRECTIONAL, 0);
if (ret)
@@ -741,8 +736,6 @@ static struct sg_table
*drm_gem_shmem_get_pages_sgt_locked(struct drm_gem_shmem_
err_free_sgt:
sg_free_table(sgt);
kfree(sgt);
-err_put_pages:
- drm_gem_shmem_put_pages_locked(shmem);
return ERR_PTR(ret);
}
@@ -759,6 +752,14 @@ static struct sg_table
*drm_gem_shmem_get_pages_sgt_locked(struct drm_gem_shmem_
* and difference between dma-buf imported and natively allocated objects.
* drm_gem_shmem_get_sg_table() should not be directly called by drivers.
*
+ * Drivers should adhere to these SGT usage rules:
+ *
+ * 1. SGT should be allocated only if shmem pages are pinned at the
+ *time of allocation, otherwise allocation will fail.
+ *
+ * 2. Drivers should ensure that pages are pinned during the time of
+ *SGT usage and should get new SGT if pages were unpinned.
+ *
* Returns:
* A pointer to the scatter/gather table of pinned pages or errno on failure.
*/
--
2.41.0
[PATCH v18 26/26] drm/panfrost: Switch to generic memory shrinker
Replace Panfrost's custom memory shrinker with a common drm-shmem
memory shrinker.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/panfrost/Makefile | 1 -
drivers/gpu/drm/panfrost/panfrost_device.h| 4 -
drivers/gpu/drm/panfrost/panfrost_drv.c | 27 ++--
drivers/gpu/drm/panfrost/panfrost_gem.c | 34 +++--
drivers/gpu/drm/panfrost/panfrost_gem.h | 9 --
.../gpu/drm/panfrost/panfrost_gem_shrinker.c | 129 --
drivers/gpu/drm/panfrost/panfrost_job.c | 18 ++-
drivers/gpu/drm/panfrost/panfrost_mmu.c | 18 ++-
include/drm/drm_gem_shmem_helper.h| 7 -
9 files changed, 66 insertions(+), 181 deletions(-)
delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
diff --git a/drivers/gpu/drm/panfrost/Makefile
b/drivers/gpu/drm/panfrost/Makefile
index 2c01c1e7523e..f2cb1ab0a32d 100644
--- a/drivers/gpu/drm/panfrost/Makefile
+++ b/drivers/gpu/drm/panfrost/Makefile
@@ -5,7 +5,6 @@ panfrost-y := \
panfrost_device.o \
panfrost_devfreq.o \
panfrost_gem.o \
- panfrost_gem_shrinker.o \
panfrost_gpu.o \
panfrost_job.o \
panfrost_mmu.o \
diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h
b/drivers/gpu/drm/panfrost/panfrost_device.h
index 1e85656dc2f7..2b24a0d4f85e 100644
--- a/drivers/gpu/drm/panfrost/panfrost_device.h
+++ b/drivers/gpu/drm/panfrost/panfrost_device.h
@@ -117,10 +117,6 @@ struct panfrost_device {
atomic_t pending;
} reset;
- struct mutex shrinker_lock;
- struct list_head shrinker_list;
- struct shrinker shrinker;
-
struct panfrost_devfreq pfdevfreq;
struct {
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c
b/drivers/gpu/drm/panfrost/panfrost_drv.c
index 7f2aba96d5b9..ef520d2cc1d2 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -171,7 +171,6 @@ panfrost_lookup_bos(struct drm_device *dev,
break;
}
- atomic_inc(>gpu_usecount);
job->mappings[i] = mapping;
}
@@ -397,7 +396,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev,
void *data,
{
struct panfrost_file_priv *priv = file_priv->driver_priv;
struct drm_panfrost_madvise *args = data;
- struct panfrost_device *pfdev = dev->dev_private;
struct drm_gem_object *gem_obj;
struct panfrost_gem_object *bo;
int ret = 0;
@@ -410,11 +408,15 @@ static int panfrost_ioctl_madvise(struct drm_device *dev,
void *data,
bo = to_panfrost_bo(gem_obj);
+ if (bo->is_heap) {
+ args->retained = 1;
+ goto out_put_object;
+ }
+
ret = dma_resv_lock_interruptible(bo->base.base.resv, NULL);
if (ret)
goto out_put_object;
- mutex_lock(>shrinker_lock);
mutex_lock(>mappings.lock);
if (args->madv == PANFROST_MADV_DONTNEED) {
struct panfrost_gem_mapping *first;
@@ -440,17 +442,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev,
void *data,
args->retained = drm_gem_shmem_madvise_locked(>base, args->madv);
- if (args->retained) {
- if (args->madv == PANFROST_MADV_DONTNEED)
- list_move_tail(>base.madv_list,
- >shrinker_list);
- else if (args->madv == PANFROST_MADV_WILLNEED)
- list_del_init(>base.madv_list);
- }
-
out_unlock_mappings:
mutex_unlock(>mappings.lock);
- mutex_unlock(>shrinker_lock);
dma_resv_unlock(bo->base.base.resv);
out_put_object:
drm_gem_object_put(gem_obj);
@@ -635,9 +628,6 @@ static int panfrost_probe(struct platform_device *pdev)
ddev->dev_private = pfdev;
pfdev->ddev = ddev;
- mutex_init(>shrinker_lock);
- INIT_LIST_HEAD(>shrinker_list);
-
err = panfrost_device_init(pfdev);
if (err) {
if (err != -EPROBE_DEFER)
@@ -659,10 +649,14 @@ static int panfrost_probe(struct platform_device *pdev)
if (err < 0)
goto err_out1;
- panfrost_gem_shrinker_init(ddev);
+ err = drmm_gem_shmem_init(ddev);
+ if (err < 0)
+ goto err_out2;
return 0;
+err_out2:
+ drm_dev_unregister(ddev);
err_out1:
pm_runtime_disable(pfdev->dev);
panfrost_device_fini(pfdev);
@@ -678,7 +672,6 @@ static void panfrost_remove(struct platform_device *pdev)
struct drm_device *ddev = pfdev->ddev;
drm_dev_unregister(ddev);
- panfrost_gem_shrinker_cleanup(ddev);
pm_runtime_get_sync(pfdev->dev);
pm_runtime_disable(pfdev->dev);
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c
b/drivers/gpu/drm/panfrost/panfrost_gem.c
index b
[PATCH v18 06/26] drm/shmem-helper: Add and use pages_pin_count
Add separate pages_pin_count for tracking of whether drm-shmem pages are
moveable or not. With the addition of memory shrinker support to drm-shmem,
the pages_use_count will no longer determine whether pages are hard-pinned
in memory, but whether pages exist and are soft-pinned (and could be swapped
out). The pages_pin_count > 1 will hard-pin pages in memory.
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 25 +
include/drm/drm_gem_shmem_helper.h | 11 +++
2 files changed, 28 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 2cc0601865f6..b9b71a1a563a 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -156,6 +156,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
drm_gem_shmem_put_pages_locked(shmem);
drm_WARN_ON(obj->dev, shmem->pages_use_count);
+ drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count));
dma_resv_unlock(shmem->base.resv);
}
@@ -234,18 +235,16 @@ static int drm_gem_shmem_pin_locked(struct
drm_gem_shmem_object *shmem)
dma_resv_assert_held(shmem->base.resv);
+ if (refcount_inc_not_zero(>pages_pin_count))
+ return 0;
+
ret = drm_gem_shmem_get_pages_locked(shmem);
+ if (!ret)
+ refcount_set(>pages_pin_count, 1);
return ret;
}
-static void drm_gem_shmem_unpin_locked(struct drm_gem_shmem_object *shmem)
-{
- dma_resv_assert_held(shmem->base.resv);
-
- drm_gem_shmem_put_pages_locked(shmem);
-}
-
/**
* drm_gem_shmem_pin - Pin backing pages for a shmem GEM object
* @shmem: shmem GEM object
@@ -263,6 +262,9 @@ int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
drm_WARN_ON(obj->dev, obj->import_attach);
+ if (refcount_inc_not_zero(>pages_pin_count))
+ return 0;
+
ret = dma_resv_lock_interruptible(shmem->base.resv, NULL);
if (ret)
return ret;
@@ -286,8 +288,14 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object
*shmem)
drm_WARN_ON(obj->dev, obj->import_attach);
+ if (refcount_dec_not_one(>pages_pin_count))
+ return;
+
dma_resv_lock(shmem->base.resv, NULL);
- drm_gem_shmem_unpin_locked(shmem);
+
+ if (refcount_dec_and_test(>pages_pin_count))
+ drm_gem_shmem_put_pages_locked(shmem);
+
dma_resv_unlock(shmem->base.resv);
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_unpin);
@@ -632,6 +640,7 @@ void drm_gem_shmem_print_info(const struct
drm_gem_shmem_object *shmem,
if (shmem->base.import_attach)
return;
+ drm_printf_indent(p, indent, "pages_pin_count=%u\n",
refcount_read(>pages_pin_count));
drm_printf_indent(p, indent, "pages_use_count=%u\n",
shmem->pages_use_count);
drm_printf_indent(p, indent, "vmap_use_count=%u\n",
shmem->vmap_use_count);
drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index 6ee4a4046980..5088bd623518 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -39,6 +39,17 @@ struct drm_gem_shmem_object {
*/
unsigned int pages_use_count;
+ /**
+* @pages_pin_count:
+*
+* Reference count on the pinned pages table.
+*
+* Pages are hard-pinned and reside in memory if count
+* greater than zero. Otherwise, when count is zero, the pages are
+* allowed to be evicted and purged by memory shrinker.
+*/
+ refcount_t pages_pin_count;
+
/**
* @madv: State for madvise
*
--
2.41.0
[PATCH v18 22/26] drm/shmem-helper: Don't free refcounted GEM
Don't free refcounted shmem object to prevent use-after-free bug that is worse than a memory leak. Signed-off-by: Dmitry Osipenko --- drivers/gpu/drm/drm_gem_shmem_helper.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c index 6dd087f19ea3..4253c367dc07 100644 --- a/drivers/gpu/drm/drm_gem_shmem_helper.c +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c @@ -203,9 +203,10 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem) if (obj->import_attach) drm_prime_gem_destroy(obj, shmem->sgt); - drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count)); - drm_WARN_ON(obj->dev, refcount_read(>pages_use_count)); - drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count)); + if (drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count)) || + drm_WARN_ON(obj->dev, refcount_read(>pages_use_count)) || + drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count))) + return; drm_gem_object_release(obj); kfree(shmem); -- 2.41.0
[PATCH v18 24/26] drm/virtio: Attach shmem BOs dynamically
Prepare for addition of memory shrinker support by attaching shmem pages
to host dynamically on first use. Previously the attachment vq command
wasn't fenced and there was no vq kick made in the BO creation code path,
hence the attachment already was happening dynamically, but implicitly.
Making attachment explicitly dynamic will allow to simplify and reuse more
code when shrinker will be added. The virtio_gpu_object_shmem_init() now
works under the held reservation lock, which will be important to have for
shrinker to avoid moving pages while they are in active use by the driver.
Acked-by: Gerd Hoffmann
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/virtio/virtgpu_drv.h| 7 +++
drivers/gpu/drm/virtio/virtgpu_gem.c| 26 +
drivers/gpu/drm/virtio/virtgpu_ioctl.c | 32 +++
drivers/gpu/drm/virtio/virtgpu_object.c | 73 -
drivers/gpu/drm/virtio/virtgpu_submit.c | 15 -
5 files changed, 125 insertions(+), 28 deletions(-)
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h
b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 56269814fb6d..421f524ae1de 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -89,6 +89,7 @@ struct virtio_gpu_object {
uint32_t hw_res_handle;
bool dumb;
bool created;
+ bool detached;
bool host3d_blob, guest_blob;
uint32_t blob_mem, blob_flags;
@@ -313,6 +314,8 @@ void virtio_gpu_array_put_free(struct
virtio_gpu_object_array *objs);
void virtio_gpu_array_put_free_delayed(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object_array *objs);
void virtio_gpu_array_put_free_work(struct work_struct *work);
+int virtio_gpu_array_prepare(struct virtio_gpu_device *vgdev,
+struct virtio_gpu_object_array *objs);
int virtio_gpu_gem_pin(struct virtio_gpu_object *bo);
void virtio_gpu_gem_unpin(struct virtio_gpu_object *bo);
@@ -453,6 +456,10 @@ int virtio_gpu_object_create(struct virtio_gpu_device
*vgdev,
bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
+int virtio_gpu_reattach_shmem_object_locked(struct virtio_gpu_object *bo);
+
+int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
+
int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
uint32_t *resid);
/* virtgpu_prime.c */
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c
b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 625c05d625bf..97e67064c97e 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -295,6 +295,26 @@ void virtio_gpu_array_put_free_work(struct work_struct
*work)
spin_unlock(>obj_free_lock);
}
+int virtio_gpu_array_prepare(struct virtio_gpu_device *vgdev,
+struct virtio_gpu_object_array *objs)
+{
+ struct virtio_gpu_object *bo;
+ int ret = 0;
+ u32 i;
+
+ for (i = 0; i < objs->nents; i++) {
+ bo = gem_to_virtio_gpu_obj(objs->objs[i]);
+
+ if (virtio_gpu_is_shmem(bo) && bo->detached) {
+ ret = virtio_gpu_reattach_shmem_object_locked(bo);
+ if (ret)
+ break;
+ }
+ }
+
+ return ret;
+}
+
int virtio_gpu_gem_pin(struct virtio_gpu_object *bo)
{
int err;
@@ -303,6 +323,12 @@ int virtio_gpu_gem_pin(struct virtio_gpu_object *bo)
err = drm_gem_shmem_pin(>base);
if (err)
return err;
+
+ err = virtio_gpu_reattach_shmem_object(bo);
+ if (err) {
+ drm_gem_shmem_unpin(>base);
+ return err;
+ }
}
return 0;
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index b24b11f25197..070c29cea26a 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -246,6 +246,10 @@ static int virtio_gpu_transfer_from_host_ioctl(struct
drm_device *dev,
if (ret != 0)
goto err_put_free;
+ ret = virtio_gpu_array_prepare(vgdev, objs);
+ if (ret)
+ goto err_unlock;
+
fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
if (!fence) {
ret = -ENOMEM;
@@ -288,11 +292,25 @@ static int virtio_gpu_transfer_to_host_ioctl(struct
drm_device *dev, void *data,
goto err_put_free;
}
+ ret = virtio_gpu_array_lock_resv(objs);
+ if (ret != 0)
+ goto err_put_free;
+
+ ret = virtio_gpu_array_prepare(vgdev, objs);
+ if (ret)
+ goto err_unlock;
+
+ fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
+ if (!fence) {
+ ret = -ENOMEM;
+ goto err_unlock;
+ }
[PATCH v18 11/26] drm/shmem-helper: Prepare drm_gem_shmem_free() to shrinker addition
Prepare drm_gem_shmem_free() to addition of memory shrinker support
to drm-shmem by adding and using variant of put_pages() that doesn't
touch reservation lock. Reservation shouldn't be touched because lockdep
will trigger a bogus warning about locking contention with fs_reclaim
code paths that can't happen during the time when GEM is freed and
lockdep doesn't know about that.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 35 +-
1 file changed, 18 insertions(+), 17 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 08b5a57c59d8..24ff2b99e75b 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -128,6 +128,22 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct
drm_device *dev, size_t
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
+static void
+drm_gem_shmem_free_pages(struct drm_gem_shmem_object *shmem)
+{
+ struct drm_gem_object *obj = >base;
+
+#ifdef CONFIG_X86
+ if (shmem->map_wc)
+ set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
+#endif
+
+ drm_gem_put_pages(obj, shmem->pages,
+ shmem->pages_mark_dirty_on_put,
+ shmem->pages_mark_accessed_on_put);
+ shmem->pages = NULL;
+}
+
/**
* drm_gem_shmem_free - Free resources associated with a shmem GEM object
* @shmem: shmem GEM object to free
@@ -142,8 +158,6 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
if (obj->import_attach) {
drm_prime_gem_destroy(obj, shmem->sgt);
} else {
- dma_resv_lock(shmem->base.resv, NULL);
-
drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count));
if (shmem->sgt) {
@@ -157,8 +171,6 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
drm_WARN_ON(obj->dev, refcount_read(>pages_use_count));
drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count));
-
- dma_resv_unlock(shmem->base.resv);
}
drm_gem_object_release(obj);
@@ -208,21 +220,10 @@ static int drm_gem_shmem_get_pages_locked(struct
drm_gem_shmem_object *shmem)
*/
void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem)
{
- struct drm_gem_object *obj = >base;
-
dma_resv_assert_held(shmem->base.resv);
- if (refcount_dec_and_test(>pages_use_count)) {
-#ifdef CONFIG_X86
- if (shmem->map_wc)
- set_pages_array_wb(shmem->pages, obj->size >>
PAGE_SHIFT);
-#endif
-
- drm_gem_put_pages(obj, shmem->pages,
- shmem->pages_mark_dirty_on_put,
- shmem->pages_mark_accessed_on_put);
- shmem->pages = NULL;
- }
+ if (refcount_dec_and_test(>pages_use_count))
+ drm_gem_shmem_free_pages(shmem);
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
--
2.41.0
[PATCH v18 05/26] drm/shmem-helper: Remove obsoleted is_iomem test
Everything that uses the mapped buffer should be agnostic to is_iomem.
The only reason for the is_iomem test is that we're setting shmem->vaddr
to the returned map->vaddr. Now that the shmem->vaddr code is gone, remove
the obsoleted is_iomem test to clean up the code.
Suggested-by: Thomas Zimmermann
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 6 --
1 file changed, 6 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 154585ddae08..2cc0601865f6 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -315,12 +315,6 @@ int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object
*shmem,
if (obj->import_attach) {
ret = dma_buf_vmap(obj->import_attach->dmabuf, map);
- if (!ret) {
- if (drm_WARN_ON(obj->dev, map->is_iomem)) {
- dma_buf_vunmap(obj->import_attach->dmabuf, map);
- return -EIO;
- }
- }
} else {
pgprot_t prot = PAGE_KERNEL;
--
2.41.0
[PATCH v18 17/26] drm/v3d: Explicitly get and put drm-shmem pages
To simplify the drm-shmem refcnt handling, we're moving away from the implicit get_pages() that is used by get_pages_sgt(). From now on drivers will have to pin pages while they use sgt. V3D driver doesn't support shrinker, hence pages are pinned and sgt is valid as long as pages' use-count > 0. Signed-off-by: Dmitry Osipenko --- drivers/gpu/drm/v3d/v3d_bo.c | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/v3d/v3d_bo.c b/drivers/gpu/drm/v3d/v3d_bo.c index 42cd874f6810..0597c6b01b6c 100644 --- a/drivers/gpu/drm/v3d/v3d_bo.c +++ b/drivers/gpu/drm/v3d/v3d_bo.c @@ -47,6 +47,9 @@ void v3d_free_object(struct drm_gem_object *obj) /* GPU execution may have dirtied any pages in the BO. */ bo->base.pages_mark_dirty_on_put = true; + if (!obj->import_attach) + drm_gem_shmem_put_pages(>base); + drm_gem_shmem_free(>base); } @@ -135,12 +138,18 @@ struct v3d_bo *v3d_bo_create(struct drm_device *dev, struct drm_file *file_priv, return ERR_CAST(shmem_obj); bo = to_v3d_bo(_obj->base); - ret = v3d_bo_create_finish(_obj->base); + ret = drm_gem_shmem_get_pages(shmem_obj); if (ret) goto free_obj; + ret = v3d_bo_create_finish(_obj->base); + if (ret) + goto put_pages; + return bo; +put_pages: + drm_gem_shmem_put_pages(shmem_obj); free_obj: drm_gem_shmem_free(shmem_obj); return ERR_PTR(ret); -- 2.41.0
[PATCH v18 20/26] drm/shmem-helper: Export drm_gem_shmem_get_pages_sgt_locked()
Export drm_gem_shmem_get_pages_sgt_locked() that will be used by virtio-gpu
shrinker during GEM swap-in operation done under the held reservation lock.
Reviewed-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 22 +-
include/drm/drm_gem_shmem_helper.h | 1 +
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 007521bea302..560ce565f376 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -872,12 +872,31 @@ struct sg_table *drm_gem_shmem_get_sg_table(struct
drm_gem_shmem_object *shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_get_sg_table);
-static struct sg_table *drm_gem_shmem_get_pages_sgt_locked(struct
drm_gem_shmem_object *shmem)
+/**
+ * drm_gem_shmem_get_pages_sgt_locked - Provide a scatter/gather table of
pinned
+ * pages for a shmem GEM object
+ * @shmem: shmem GEM object
+ *
+ * This is a locked version of @drm_gem_shmem_get_sg_table that exports a
+ * scatter/gather table suitable for PRIME usage by calling the standard
+ * DMA mapping API.
+ *
+ * Drivers must hold GEM's reservation lock when using this function.
+ *
+ * Drivers who need to acquire an scatter/gather table for objects need to call
+ * drm_gem_shmem_get_pages_sgt() instead.
+ *
+ * Returns:
+ * A pointer to the scatter/gather table of pinned pages or error pointer on
failure.
+ */
+struct sg_table *drm_gem_shmem_get_pages_sgt_locked(struct
drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
int ret;
struct sg_table *sgt;
+ dma_resv_assert_held(shmem->base.resv);
+
if (shmem->sgt)
return shmem->sgt;
@@ -901,6 +920,7 @@ static struct sg_table
*drm_gem_shmem_get_pages_sgt_locked(struct drm_gem_shmem_
kfree(sgt);
return ERR_PTR(ret);
}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_get_pages_sgt_locked);
/**
* drm_gem_shmem_get_pages_sgt - Pin pages, dma map them, and return a
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index 3bb70616d095..6ac77c2082ed 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -149,6 +149,7 @@ void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object
*shmem);
struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object
*shmem);
struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object
*shmem);
+struct sg_table *drm_gem_shmem_get_pages_sgt_locked(struct
drm_gem_shmem_object *shmem);
void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
struct drm_printer *p, unsigned int indent);
--
2.41.0
[PATCH v18 07/26] drm/shmem-helper: Use refcount_t for pages_use_count
Use atomic refcount_t helper for pages_use_count to optimize pin/unpin
functions by skipping reservation locking while GEM's pin refcount > 1.
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 33 +++--
drivers/gpu/drm/lima/lima_gem.c | 2 +-
drivers/gpu/drm/panfrost/panfrost_mmu.c | 2 +-
include/drm/drm_gem_shmem_helper.h | 2 +-
4 files changed, 18 insertions(+), 21 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index b9b71a1a563a..6e02643ed87e 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -155,7 +155,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
if (shmem->pages)
drm_gem_shmem_put_pages_locked(shmem);
- drm_WARN_ON(obj->dev, shmem->pages_use_count);
+ drm_WARN_ON(obj->dev, refcount_read(>pages_use_count));
drm_WARN_ON(obj->dev, refcount_read(>pages_pin_count));
dma_resv_unlock(shmem->base.resv);
@@ -173,14 +173,13 @@ static int drm_gem_shmem_get_pages_locked(struct
drm_gem_shmem_object *shmem)
dma_resv_assert_held(shmem->base.resv);
- if (shmem->pages_use_count++ > 0)
+ if (refcount_inc_not_zero(>pages_use_count))
return 0;
pages = drm_gem_get_pages(obj);
if (IS_ERR(pages)) {
drm_dbg_kms(obj->dev, "Failed to get pages (%ld)\n",
PTR_ERR(pages));
- shmem->pages_use_count = 0;
return PTR_ERR(pages);
}
@@ -196,6 +195,8 @@ static int drm_gem_shmem_get_pages_locked(struct
drm_gem_shmem_object *shmem)
shmem->pages = pages;
+ refcount_set(>pages_use_count, 1);
+
return 0;
}
@@ -211,21 +212,17 @@ void drm_gem_shmem_put_pages_locked(struct
drm_gem_shmem_object *shmem)
dma_resv_assert_held(shmem->base.resv);
- if (drm_WARN_ON_ONCE(obj->dev, !shmem->pages_use_count))
- return;
-
- if (--shmem->pages_use_count > 0)
- return;
-
+ if (refcount_dec_and_test(>pages_use_count)) {
#ifdef CONFIG_X86
- if (shmem->map_wc)
- set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
+ if (shmem->map_wc)
+ set_pages_array_wb(shmem->pages, obj->size >>
PAGE_SHIFT);
#endif
- drm_gem_put_pages(obj, shmem->pages,
- shmem->pages_mark_dirty_on_put,
- shmem->pages_mark_accessed_on_put);
- shmem->pages = NULL;
+ drm_gem_put_pages(obj, shmem->pages,
+ shmem->pages_mark_dirty_on_put,
+ shmem->pages_mark_accessed_on_put);
+ shmem->pages = NULL;
+ }
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
@@ -552,8 +549,8 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct
*vma)
* mmap'd, vm_open() just grabs an additional reference for the new
* mm the vma is getting copied into (ie. on fork()).
*/
- if (!drm_WARN_ON_ONCE(obj->dev, !shmem->pages_use_count))
- shmem->pages_use_count++;
+ drm_WARN_ON_ONCE(obj->dev,
+!refcount_inc_not_zero(>pages_use_count));
dma_resv_unlock(shmem->base.resv);
@@ -641,7 +638,7 @@ void drm_gem_shmem_print_info(const struct
drm_gem_shmem_object *shmem,
return;
drm_printf_indent(p, indent, "pages_pin_count=%u\n",
refcount_read(>pages_pin_count));
- drm_printf_indent(p, indent, "pages_use_count=%u\n",
shmem->pages_use_count);
+ drm_printf_indent(p, indent, "pages_use_count=%u\n",
refcount_read(>pages_use_count));
drm_printf_indent(p, indent, "vmap_use_count=%u\n",
shmem->vmap_use_count);
drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
}
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index 62d4a409faa8..988e74f67465 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -47,7 +47,7 @@ int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm)
}
bo->base.pages = pages;
- bo->base.pages_use_count = 1;
+ refcount_set(>base.pages_use_count, 1);
mapping_set_unevictable(mapping);
}
diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c
b/drivers/gpu/drm/panfrost/panfrost_mmu.c
index 9fd4a89c52dd..770dab1942c2 100644
--- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
+++ b/drivers/gpu
[PATCH v18 21/26] drm/shmem-helper: Optimize unlocked get_pages_sgt()
SGT isn't refcounted. Once SGT pointer has been obtained, it remains the same for both locked and unlocked get_pages_sgt(). Return cached SGT directly without taking a potentially expensive lock. Signed-off-by: Dmitry Osipenko --- drivers/gpu/drm/drm_gem_shmem_helper.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c index 560ce565f376..6dd087f19ea3 100644 --- a/drivers/gpu/drm/drm_gem_shmem_helper.c +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c @@ -955,6 +955,9 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem) drm_WARN_ON(obj->dev, drm_gem_shmem_is_evictable(shmem)); drm_WARN_ON(obj->dev, drm_gem_shmem_is_purgeable(shmem)); + if (shmem->sgt) + return shmem->sgt; + ret = dma_resv_lock_interruptible(shmem->base.resv, NULL); if (ret) return ERR_PTR(ret); -- 2.41.0
[PATCH v18 09/26] drm/shmem-helper: Switch drm_gem_shmem_vmap/vunmap to use pin/unpin
The vmapped pages shall be pinned in memory and previously get/put_pages()
were implicitly hard-pinning/unpinning the pages. This will no longer be
the case with addition of memory shrinker because pages_use_count > 0 won't
determine anymore whether pages are hard-pinned (they will be soft-pinned),
while the new pages_pin_count will do the hard-pinning. Switch the
vmap/vunmap() to use pin/unpin() functions in a preparation of addition
of the memory shrinker support to drm-shmem.
Reviewed-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 19 ---
include/drm/drm_gem_shmem_helper.h | 2 +-
2 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 41b749bedb11..6f963c2c1ecc 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -256,6 +256,14 @@ static int drm_gem_shmem_pin_locked(struct
drm_gem_shmem_object *shmem)
return ret;
}
+static void drm_gem_shmem_unpin_locked(struct drm_gem_shmem_object *shmem)
+{
+ dma_resv_assert_held(shmem->base.resv);
+
+ if (refcount_dec_and_test(>pages_pin_count))
+ drm_gem_shmem_put_pages_locked(shmem);
+}
+
/**
* drm_gem_shmem_pin - Pin backing pages for a shmem GEM object
* @shmem: shmem GEM object
@@ -303,10 +311,7 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object
*shmem)
return;
dma_resv_lock(shmem->base.resv, NULL);
-
- if (refcount_dec_and_test(>pages_pin_count))
- drm_gem_shmem_put_pages_locked(shmem);
-
+ drm_gem_shmem_unpin_locked(shmem);
dma_resv_unlock(shmem->base.resv);
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_unpin);
@@ -344,7 +349,7 @@ int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object
*shmem,
return 0;
}
- ret = drm_gem_shmem_get_pages_locked(shmem);
+ ret = drm_gem_shmem_pin_locked(shmem);
if (ret)
goto err_zero_use;
@@ -367,7 +372,7 @@ int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object
*shmem,
err_put_pages:
if (!obj->import_attach)
- drm_gem_shmem_put_pages_locked(shmem);
+ drm_gem_shmem_unpin_locked(shmem);
err_zero_use:
shmem->vmap_use_count = 0;
@@ -404,7 +409,7 @@ void drm_gem_shmem_vunmap_locked(struct
drm_gem_shmem_object *shmem,
return;
vunmap(shmem->vaddr);
- drm_gem_shmem_put_pages_locked(shmem);
+ drm_gem_shmem_unpin_locked(shmem);
}
shmem->vaddr = NULL;
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index bd3596e54abe..a6de11001048 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -124,7 +124,7 @@ int drm_gem_shmem_madvise_locked(struct
drm_gem_shmem_object *shmem, int madv);
static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object
*shmem)
{
return (shmem->madv > 0) &&
- !shmem->vmap_use_count && shmem->sgt &&
+ !refcount_read(>pages_pin_count) && shmem->sgt &&
!shmem->base.dma_buf && !shmem->base.import_attach;
}
--
2.41.0
[PATCH v18 23/26] drm/virtio: Pin display framebuffer BO
Prepare to addition of memory shrinker support by pinning display
framebuffer BO pages in memory while they are in use by display on host.
Shrinker is free to relocate framebuffer BO pages if it doesn't know that
pages are in use, thus pin the pages to disallow shrinker to move them.
Acked-by: Gerd Hoffmann
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/virtio/virtgpu_drv.h | 2 ++
drivers/gpu/drm/virtio/virtgpu_gem.c | 19 +++
drivers/gpu/drm/virtio/virtgpu_plane.c | 17 +++--
3 files changed, 36 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h
b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 96365a772f77..56269814fb6d 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -313,6 +313,8 @@ void virtio_gpu_array_put_free(struct
virtio_gpu_object_array *objs);
void virtio_gpu_array_put_free_delayed(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object_array *objs);
void virtio_gpu_array_put_free_work(struct work_struct *work);
+int virtio_gpu_gem_pin(struct virtio_gpu_object *bo);
+void virtio_gpu_gem_unpin(struct virtio_gpu_object *bo);
/* virtgpu_vq.c */
int virtio_gpu_alloc_vbufs(struct virtio_gpu_device *vgdev);
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c
b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 7db48d17ee3a..625c05d625bf 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -294,3 +294,22 @@ void virtio_gpu_array_put_free_work(struct work_struct
*work)
}
spin_unlock(>obj_free_lock);
}
+
+int virtio_gpu_gem_pin(struct virtio_gpu_object *bo)
+{
+ int err;
+
+ if (virtio_gpu_is_shmem(bo)) {
+ err = drm_gem_shmem_pin(>base);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+void virtio_gpu_gem_unpin(struct virtio_gpu_object *bo)
+{
+ if (virtio_gpu_is_shmem(bo))
+ drm_gem_shmem_unpin(>base);
+}
diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c
b/drivers/gpu/drm/virtio/virtgpu_plane.c
index a2e045f3a000..def57b01a826 100644
--- a/drivers/gpu/drm/virtio/virtgpu_plane.c
+++ b/drivers/gpu/drm/virtio/virtgpu_plane.c
@@ -238,20 +238,28 @@ static int virtio_gpu_plane_prepare_fb(struct drm_plane
*plane,
struct virtio_gpu_device *vgdev = dev->dev_private;
struct virtio_gpu_framebuffer *vgfb;
struct virtio_gpu_object *bo;
+ int err;
if (!new_state->fb)
return 0;
vgfb = to_virtio_gpu_framebuffer(new_state->fb);
bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
- if (!bo || (plane->type == DRM_PLANE_TYPE_PRIMARY && !bo->guest_blob))
+
+ err = virtio_gpu_gem_pin(bo);
+ if (err)
+ return err;
+
+ if (plane->type == DRM_PLANE_TYPE_PRIMARY && !bo->guest_blob)
return 0;
if (bo->dumb && (plane->state->fb != new_state->fb)) {
vgfb->fence = virtio_gpu_fence_alloc(vgdev,
vgdev->fence_drv.context,
0);
- if (!vgfb->fence)
+ if (!vgfb->fence) {
+ virtio_gpu_gem_unpin(bo);
return -ENOMEM;
+ }
}
return 0;
@@ -261,15 +269,20 @@ static void virtio_gpu_plane_cleanup_fb(struct drm_plane
*plane,
struct drm_plane_state *state)
{
struct virtio_gpu_framebuffer *vgfb;
+ struct virtio_gpu_object *bo;
if (!state->fb)
return;
vgfb = to_virtio_gpu_framebuffer(state->fb);
+ bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
+
if (vgfb->fence) {
dma_fence_put(>fence->f);
vgfb->fence = NULL;
}
+
+ virtio_gpu_gem_unpin(bo);
}
static void virtio_gpu_cursor_plane_update(struct drm_plane *plane,
--
2.41.0
[PATCH v18 16/26] drm/virtio: Explicitly get and put drm-shmem pages
We're moving away from implicit get_pages() that is done by
get_pages_sgt() to simplify the refcnt handling. Drivers will have
to pin pages while they use sgt. VirtIO-GPU doesn't support shrinker,
hence pages are pinned and sgt is valid as long as pages' use-count > 0.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/virtio/virtgpu_object.c | 9 -
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c
b/drivers/gpu/drm/virtio/virtgpu_object.c
index ee5d2a70656b..998f8b05ceb1 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -67,6 +67,7 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo)
virtio_gpu_resource_id_put(vgdev, bo->hw_res_handle);
if (virtio_gpu_is_shmem(bo)) {
+ drm_gem_shmem_put_pages(>base);
drm_gem_shmem_free(>base);
} else if (virtio_gpu_is_vram(bo)) {
struct virtio_gpu_object_vram *vram = to_virtio_gpu_vram(bo);
@@ -196,9 +197,13 @@ int virtio_gpu_object_create(struct virtio_gpu_device
*vgdev,
return PTR_ERR(shmem_obj);
bo = gem_to_virtio_gpu_obj(_obj->base);
+ ret = drm_gem_shmem_get_pages(shmem_obj);
+ if (ret)
+ goto err_free_gem;
+
ret = virtio_gpu_resource_id_get(vgdev, >hw_res_handle);
if (ret < 0)
- goto err_free_gem;
+ goto err_put_pages;
bo->dumb = params->dumb;
@@ -243,6 +248,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device
*vgdev,
kvfree(ents);
err_put_id:
virtio_gpu_resource_id_put(vgdev, bo->hw_res_handle);
+err_put_pages:
+ drm_gem_shmem_put_pages(shmem_obj);
err_free_gem:
drm_gem_shmem_free(shmem_obj);
return ret;
--
2.41.0
[PATCH v18 10/26] drm/shmem-helper: Use refcount_t for vmap_use_count
Use refcount_t helper for vmap_use_count to make refcounting consistent
with pages_use_count and pages_pin_count that use refcount_t. This also
makes vmapping to benefit from the refcount_t's overflow checks.
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 28 +++---
include/drm/drm_gem_shmem_helper.h | 2 +-
2 files changed, 13 insertions(+), 17 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 6f963c2c1ecc..08b5a57c59d8 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -144,7 +144,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
} else {
dma_resv_lock(shmem->base.resv, NULL);
- drm_WARN_ON(obj->dev, shmem->vmap_use_count);
+ drm_WARN_ON(obj->dev, refcount_read(>vmap_use_count));
if (shmem->sgt) {
dma_unmap_sgtable(obj->dev->dev, shmem->sgt,
@@ -344,23 +344,25 @@ int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object
*shmem,
dma_resv_assert_held(shmem->base.resv);
- if (shmem->vmap_use_count++ > 0) {
+ if (refcount_inc_not_zero(>vmap_use_count)) {
iosys_map_set_vaddr(map, shmem->vaddr);
return 0;
}
ret = drm_gem_shmem_pin_locked(shmem);
if (ret)
- goto err_zero_use;
+ return ret;
if (shmem->map_wc)
prot = pgprot_writecombine(prot);
shmem->vaddr = vmap(shmem->pages, obj->size >> PAGE_SHIFT,
VM_MAP, prot);
- if (!shmem->vaddr)
+ if (!shmem->vaddr) {
ret = -ENOMEM;
- else
+ } else {
iosys_map_set_vaddr(map, shmem->vaddr);
+ refcount_set(>vmap_use_count, 1);
+ }
}
if (ret) {
@@ -373,8 +375,6 @@ int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object
*shmem,
err_put_pages:
if (!obj->import_attach)
drm_gem_shmem_unpin_locked(shmem);
-err_zero_use:
- shmem->vmap_use_count = 0;
return ret;
}
@@ -402,14 +402,10 @@ void drm_gem_shmem_vunmap_locked(struct
drm_gem_shmem_object *shmem,
} else {
dma_resv_assert_held(shmem->base.resv);
- if (drm_WARN_ON_ONCE(obj->dev, !shmem->vmap_use_count))
- return;
-
- if (--shmem->vmap_use_count > 0)
- return;
-
- vunmap(shmem->vaddr);
- drm_gem_shmem_unpin_locked(shmem);
+ if (refcount_dec_and_test(>vmap_use_count)) {
+ vunmap(shmem->vaddr);
+ drm_gem_shmem_unpin_locked(shmem);
+ }
}
shmem->vaddr = NULL;
@@ -655,7 +651,7 @@ void drm_gem_shmem_print_info(const struct
drm_gem_shmem_object *shmem,
drm_printf_indent(p, indent, "pages_pin_count=%u\n",
refcount_read(>pages_pin_count));
drm_printf_indent(p, indent, "pages_use_count=%u\n",
refcount_read(>pages_use_count));
- drm_printf_indent(p, indent, "vmap_use_count=%u\n",
shmem->vmap_use_count);
+ drm_printf_indent(p, indent, "vmap_use_count=%u\n",
refcount_read(>vmap_use_count));
drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_print_info);
diff --git a/include/drm/drm_gem_shmem_helper.h
b/include/drm/drm_gem_shmem_helper.h
index a6de11001048..e7b3f4c02bf5 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -82,7 +82,7 @@ struct drm_gem_shmem_object {
* Reference count on the virtual address.
* The address are un-mapped when the count reaches zero.
*/
- unsigned int vmap_use_count;
+ refcount_t vmap_use_count;
/**
* @pages_mark_dirty_on_put:
--
2.41.0
[PATCH v18 15/26] drm/panfrost: Explicitly get and put drm-shmem pages
To simplify the drm-shmem refcnt handling, we're moving away from
the implicit get_pages() that is used by get_pages_sgt(). From now on
drivers will have to pin pages while they use sgt. Panfrost's shrinker
doesn't support swapping out BOs, hence pages are pinned and sgt is valid
as long as pages' use-count > 0.
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/panfrost/panfrost_gem.c | 17 +
drivers/gpu/drm/panfrost/panfrost_mmu.c | 6 ++
2 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c
b/drivers/gpu/drm/panfrost/panfrost_gem.c
index 6b77d8cebcb2..bb9d43cf7c3c 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gem.c
@@ -47,8 +47,13 @@ static void panfrost_gem_free_object(struct drm_gem_object
*obj)
}
}
kvfree(bo->sgts);
+
+ drm_gem_shmem_put_pages(>base);
}
+ if (!bo->is_heap && !obj->import_attach)
+ drm_gem_shmem_put_pages(>base);
+
drm_gem_shmem_free(>base);
}
@@ -269,6 +274,7 @@ panfrost_gem_create(struct drm_device *dev, size_t size,
u32 flags)
{
struct drm_gem_shmem_object *shmem;
struct panfrost_gem_object *bo;
+ int err;
/* Round up heap allocations to 2MB to keep fault handling simple */
if (flags & PANFROST_BO_HEAP)
@@ -282,7 +288,18 @@ panfrost_gem_create(struct drm_device *dev, size_t size,
u32 flags)
bo->noexec = !!(flags & PANFROST_BO_NOEXEC);
bo->is_heap = !!(flags & PANFROST_BO_HEAP);
+ if (!bo->is_heap) {
+ err = drm_gem_shmem_get_pages(shmem);
+ if (err)
+ goto err_free;
+ }
+
return bo;
+
+err_free:
+ drm_gem_shmem_free(>base);
+
+ return ERR_PTR(err);
}
struct drm_gem_object *
diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c
b/drivers/gpu/drm/panfrost/panfrost_mmu.c
index 770dab1942c2..ac145a98377b 100644
--- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
+++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
@@ -504,7 +504,7 @@ static int panfrost_mmu_map_fault_addr(struct
panfrost_device *pfdev, int as,
if (IS_ERR(pages[i])) {
ret = PTR_ERR(pages[i]);
pages[i] = NULL;
- goto err_pages;
+ goto err_unlock;
}
}
@@ -512,7 +512,7 @@ static int panfrost_mmu_map_fault_addr(struct
panfrost_device *pfdev, int as,
ret = sg_alloc_table_from_pages(sgt, pages + page_offset,
NUM_FAULT_PAGES, 0, SZ_2M, GFP_KERNEL);
if (ret)
- goto err_pages;
+ goto err_unlock;
ret = dma_map_sgtable(pfdev->dev, sgt, DMA_BIDIRECTIONAL, 0);
if (ret)
@@ -535,8 +535,6 @@ static int panfrost_mmu_map_fault_addr(struct
panfrost_device *pfdev, int as,
err_map:
sg_free_table(sgt);
-err_pages:
- drm_gem_shmem_put_pages_locked(>base);
err_unlock:
dma_resv_unlock(obj->resv);
err_bo:
--
2.41.0
[PATCH v18 03/26] drm/shmem-helper: Make all exported symbols GPL
Make all drm-shmem exported symbols GPL to make them consistent with
the rest of drm-shmem symbols.
Reviewed-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index e435f986cd13..0d61f2b3e213 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -226,7 +226,7 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object
*shmem)
shmem->pages_mark_accessed_on_put);
shmem->pages = NULL;
}
-EXPORT_SYMBOL(drm_gem_shmem_put_pages);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages);
static int drm_gem_shmem_pin_locked(struct drm_gem_shmem_object *shmem)
{
@@ -271,7 +271,7 @@ int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
return ret;
}
-EXPORT_SYMBOL(drm_gem_shmem_pin);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_pin);
/**
* drm_gem_shmem_unpin - Unpin backing pages for a shmem GEM object
@@ -290,7 +290,7 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
drm_gem_shmem_unpin_locked(shmem);
dma_resv_unlock(shmem->base.resv);
}
-EXPORT_SYMBOL(drm_gem_shmem_unpin);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_unpin);
/*
* drm_gem_shmem_vmap - Create a virtual mapping for a shmem GEM object
@@ -360,7 +360,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
return ret;
}
-EXPORT_SYMBOL(drm_gem_shmem_vmap);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_vmap);
/*
* drm_gem_shmem_vunmap - Unmap a virtual mapping for a shmem GEM object
@@ -396,7 +396,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object
*shmem,
shmem->vaddr = NULL;
}
-EXPORT_SYMBOL(drm_gem_shmem_vunmap);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_vunmap);
static int
drm_gem_shmem_create_with_handle(struct drm_file *file_priv,
@@ -435,7 +435,7 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object
*shmem, int madv)
return (madv >= 0);
}
-EXPORT_SYMBOL(drm_gem_shmem_madvise);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_madvise);
void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
{
@@ -467,7 +467,7 @@ void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0,
(loff_t)-1);
}
-EXPORT_SYMBOL(drm_gem_shmem_purge);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
/**
* drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
@@ -642,7 +642,7 @@ void drm_gem_shmem_print_info(const struct
drm_gem_shmem_object *shmem,
drm_printf_indent(p, indent, "vmap_use_count=%u\n",
shmem->vmap_use_count);
drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
}
-EXPORT_SYMBOL(drm_gem_shmem_print_info);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_print_info);
/**
* drm_gem_shmem_get_sg_table - Provide a scatter/gather table of pinned
--
2.41.0
[PATCH v18 04/26] drm/shmem-helper: Refactor locked/unlocked functions
Add locked and remove unlocked postfixes from drm-shmem function names,
making names consistent with the drm/gem core code.
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_gem_shmem_helper.c| 64 +--
drivers/gpu/drm/lima/lima_gem.c | 8 +--
drivers/gpu/drm/panfrost/panfrost_drv.c | 2 +-
drivers/gpu/drm/panfrost/panfrost_gem.c | 6 +-
.../gpu/drm/panfrost/panfrost_gem_shrinker.c | 2 +-
drivers/gpu/drm/panfrost/panfrost_mmu.c | 2 +-
drivers/gpu/drm/v3d/v3d_bo.c | 4 +-
drivers/gpu/drm/virtio/virtgpu_object.c | 4 +-
include/drm/drm_gem_shmem_helper.h| 36 +--
9 files changed, 64 insertions(+), 64 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c
b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 0d61f2b3e213..154585ddae08 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -43,8 +43,8 @@ static const struct drm_gem_object_funcs drm_gem_shmem_funcs
= {
.pin = drm_gem_shmem_object_pin,
.unpin = drm_gem_shmem_object_unpin,
.get_sg_table = drm_gem_shmem_object_get_sg_table,
- .vmap = drm_gem_shmem_object_vmap,
- .vunmap = drm_gem_shmem_object_vunmap,
+ .vmap = drm_gem_shmem_object_vmap_locked,
+ .vunmap = drm_gem_shmem_object_vunmap_locked,
.mmap = drm_gem_shmem_object_mmap,
.vm_ops = _gem_shmem_vm_ops,
};
@@ -153,7 +153,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
kfree(shmem->sgt);
}
if (shmem->pages)
- drm_gem_shmem_put_pages(shmem);
+ drm_gem_shmem_put_pages_locked(shmem);
drm_WARN_ON(obj->dev, shmem->pages_use_count);
@@ -165,7 +165,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
}
EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
-static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
+static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
struct page **pages;
@@ -199,12 +199,12 @@ static int drm_gem_shmem_get_pages(struct
drm_gem_shmem_object *shmem)
}
/*
- * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a
shmem GEM object
+ * drm_gem_shmem_put_pages_locked - Decrease use count on the backing pages
for a shmem GEM object
* @shmem: shmem GEM object
*
* This function decreases the use count and puts the backing pages when use
drops to zero.
*/
-void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
+void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem)
{
struct drm_gem_object *obj = >base;
@@ -226,7 +226,7 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object
*shmem)
shmem->pages_mark_accessed_on_put);
shmem->pages = NULL;
}
-EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages);
+EXPORT_SYMBOL_GPL(drm_gem_shmem_put_pages_locked);
static int drm_gem_shmem_pin_locked(struct drm_gem_shmem_object *shmem)
{
@@ -234,7 +234,7 @@ static int drm_gem_shmem_pin_locked(struct
drm_gem_shmem_object *shmem)
dma_resv_assert_held(shmem->base.resv);
- ret = drm_gem_shmem_get_pages(shmem);
+ ret = drm_gem_shmem_get_pages_locked(shmem);
return ret;
}
@@ -243,7 +243,7 @@ static void drm_gem_shmem_unpin_locked(struct
drm_gem_shmem_object *shmem)
{
dma_resv_assert_held(shmem->base.resv);
- drm_gem_shmem_put_pages(shmem);
+ drm_gem_shmem_put_pages_locked(shmem);
}
/**
@@ -293,7 +293,7 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
EXPORT_SYMBOL_GPL(drm_gem_shmem_unpin);
/*
- * drm_gem_shmem_vmap - Create a virtual mapping for a shmem GEM object
+ * drm_gem_shmem_vmap_locked - Create a virtual mapping for a shmem GEM object
* @shmem: shmem GEM object
* @map: Returns the kernel virtual address of the SHMEM GEM object's backing
* store.
@@ -302,13 +302,13 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_unpin);
* exists for the buffer backing the shmem GEM object. It hides the differences
* between dma-buf imported and natively allocated objects.
*
- * Acquired mappings should be cleaned up by calling drm_gem_shmem_vunmap().
+ * Acquired mappings should be cleaned up by calling
drm_gem_shmem_vunmap_locked().
*
* Returns:
* 0 on success or a negative error code on failure.
*/
-int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
- struct iosys_map *map)
+int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
+ struct iosys_map *map)
{
struct drm_gem_object *obj = >base;
int ret = 0;
@@ -331,7 +331,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
[PATCH v18 01/26] drm/gem: Change locked/unlocked postfix of drm_gem_v/unmap() function names
Make drm/gem API function names consistent by having locked function
use the _locked postfix in the name, while the unlocked variants don't
use the _unlocked postfix. Rename drm_gem_v/unmap() function names to
make them consistent with the rest of the API functions.
Reviewed-by: Boris Brezillon
Suggested-by: Boris Brezillon
Signed-off-by: Dmitry Osipenko
---
drivers/gpu/drm/drm_client.c | 6 +++---
drivers/gpu/drm/drm_gem.c| 20 ++--
drivers/gpu/drm/drm_gem_framebuffer_helper.c | 6 +++---
drivers/gpu/drm/drm_internal.h | 4 ++--
drivers/gpu/drm/drm_prime.c | 4 ++--
drivers/gpu/drm/lima/lima_sched.c| 4 ++--
drivers/gpu/drm/panfrost/panfrost_dump.c | 4 ++--
drivers/gpu/drm/panfrost/panfrost_perfcnt.c | 6 +++---
include/drm/drm_gem.h| 4 ++--
9 files changed, 29 insertions(+), 29 deletions(-)
diff --git a/drivers/gpu/drm/drm_client.c b/drivers/gpu/drm/drm_client.c
index 2762572f286e..c935db1ba918 100644
--- a/drivers/gpu/drm/drm_client.c
+++ b/drivers/gpu/drm/drm_client.c
@@ -265,7 +265,7 @@ void drm_client_dev_restore(struct drm_device *dev)
static void drm_client_buffer_delete(struct drm_client_buffer *buffer)
{
if (buffer->gem) {
- drm_gem_vunmap_unlocked(buffer->gem, >map);
+ drm_gem_vunmap(buffer->gem, >map);
drm_gem_object_put(buffer->gem);
}
@@ -349,7 +349,7 @@ drm_client_buffer_vmap(struct drm_client_buffer *buffer,
* fd_install step out of the driver backend hooks, to make that
* final step optional for internal users.
*/
- ret = drm_gem_vmap_unlocked(buffer->gem, map);
+ ret = drm_gem_vmap(buffer->gem, map);
if (ret)
return ret;
@@ -371,7 +371,7 @@ void drm_client_buffer_vunmap(struct drm_client_buffer
*buffer)
{
struct iosys_map *map = >map;
- drm_gem_vunmap_unlocked(buffer->gem, map);
+ drm_gem_vunmap(buffer->gem, map);
}
EXPORT_SYMBOL(drm_client_buffer_vunmap);
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 44a948b80ee1..95327b003692 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1175,7 +1175,7 @@ void drm_gem_unpin(struct drm_gem_object *obj)
obj->funcs->unpin(obj);
}
-int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
+int drm_gem_vmap_locked(struct drm_gem_object *obj, struct iosys_map *map)
{
int ret;
@@ -1192,9 +1192,9 @@ int drm_gem_vmap(struct drm_gem_object *obj, struct
iosys_map *map)
return 0;
}
-EXPORT_SYMBOL(drm_gem_vmap);
+EXPORT_SYMBOL(drm_gem_vmap_locked);
-void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
+void drm_gem_vunmap_locked(struct drm_gem_object *obj, struct iosys_map *map)
{
dma_resv_assert_held(obj->resv);
@@ -1207,27 +1207,27 @@ void drm_gem_vunmap(struct drm_gem_object *obj, struct
iosys_map *map)
/* Always set the mapping to NULL. Callers may rely on this. */
iosys_map_clear(map);
}
-EXPORT_SYMBOL(drm_gem_vunmap);
+EXPORT_SYMBOL(drm_gem_vunmap_locked);
-int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
+int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
{
int ret;
dma_resv_lock(obj->resv, NULL);
- ret = drm_gem_vmap(obj, map);
+ ret = drm_gem_vmap_locked(obj, map);
dma_resv_unlock(obj->resv);
return ret;
}
-EXPORT_SYMBOL(drm_gem_vmap_unlocked);
+EXPORT_SYMBOL(drm_gem_vmap);
-void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
+void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
{
dma_resv_lock(obj->resv, NULL);
- drm_gem_vunmap(obj, map);
+ drm_gem_vunmap_locked(obj, map);
dma_resv_unlock(obj->resv);
}
-EXPORT_SYMBOL(drm_gem_vunmap_unlocked);
+EXPORT_SYMBOL(drm_gem_vunmap);
/**
* drm_gem_lock_reservations - Sets up the ww context and acquires
diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
index 3bdb6ba37ff4..3808f47310bf 100644
--- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
+++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
@@ -362,7 +362,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb, struct
iosys_map *map,
ret = -EINVAL;
goto err_drm_gem_vunmap;
}
- ret = drm_gem_vmap_unlocked(obj, [i]);
+ ret = drm_gem_vmap(obj, [i]);
if (ret)
goto err_drm_gem_vunmap;
}
@@ -384,7 +384,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb, struct
iosys_map *map,
obj = drm_gem_fb_get_obj(fb, i);
if (!obj)
con
