Re: [Intel-gfx] [PATCH v5 5/5] drm/tinydrm: Switch from CMA to shmem buffers
Den 29.10.2018 10.07, skrev Daniel Vetter: On Sun, Oct 28, 2018 at 09:46:43PM +0100, Noralf Trønnes wrote: Den 28.10.2018 21.21, skrev David Lechner: On 10/26/2018 05:38 PM, Noralf Trønnes wrote: Den 17.10.2018 15.04, skrev Noralf Trønnes: This move makes tinydrm useful for more drivers. tinydrm doesn't need continuous memory, but at the time it was convenient to use the CMA library. The spi core can do dma on is_vmalloc() addresses making this possible. Cc: David Lechner Signed-off-by: Noralf Trønnes Acked-by: David Lechner Tested-by: David Lechner --- David, FYI This series is scratched. See the shmem helper patch thread for details. Yes, I saw that. Thank you. I don't suppose there is a way to configure the DMA controller to do the byte swapping? Not that I know of. "Proper" SPI hw can do 16-bit transfers and for them there's no problem. The DMA capable SPI block on the Pi can only do 8-bit, hence the swapping. But that wasn't the problem, the byteswapping actually papered over the problem. I'm no -mm expert so I don't know why the problem onyl showed up when using the virtual address of the buffer used by fbcon and not on mmap'ed fbdev as Daniel suggested would happen. Hm, I missed that detail. This sounds like one of the mappings ended up being write-combining (which avoids all the issues with dirty cpu cache lines), while the broken one was not. Or we ended up with a flush somewhere by accident. Either way shmem not being coherent is a problem on the Pi, even though I expected the DMA streaming API called by the spi core to flush "things". It should do that for you. At least if it's using dma_map/unmap_sg and friends. I just watched a talk by the i2c maintainer about his challenges with supporting DMA buffers and DMA only host controllers: DMA Safety in Buffers for Linux Kernel Device Drivers - Wolfram Sang https://www.youtube.com/watch?v=JDwaMClvV-s In passing he mentions this from Documentation/spi/spi-summary: - I/O buffers use the usual Linux rules, and must be DMA-safe. You'd normally allocate them from the heap or free page pool. So how can the SPI core ensure DMA safety for its DMA mapping of the underlying pages of virtual addresses? This is what I relied on when doing the shmem helper. Doing further digging I came across these: Vignesh R says: https://www.spinics.net/lists/kernel/msg2687381.html SPI core does try to DMA into underlying physical pages of vmalloc'd buffer. But this has two problems: 1. Does not work well with VIVT caches[1]. 2. On ARM LPAE systems, vmalloc'd buffers can be from highmem region that are not addressable using 32 bit addresses and is backed by LPAE. So, a 32 bit DMA cannot access these buffers. Both these issues lead to random crashes and errors with UBIFS and JFFS2 flash file systems which this patch series tries to address using bounce buffer [1] https://patchwork.kernel.org/patch/9579553/ Russel King provides some more info: https://patchwork.kernel.org/patch/9579553/#20139929 SPI is rather another special case - rather than SPI following the established mechanism of passing data references via scatterlists or similar, it also passes them via virtual addresses, which means SPI can directly access the vmalloc area when performing PIO. This really makes the problem more complex, because it means that if you do have a SPI driver that does that, it's going to be reading/writing direct from vmalloc space. That's not a problem as long as the data is only accessed via vmalloc space, but it will definitely go totally wrong if the data is subsequently mapped into userspace. So even if I could make the shmem helper work on the Raspberry Pi with tinydrm and SPI, it probably wouldn't work on all systems. I also learned that using devm_kmalloc, which tinydrm does for a tx buffer, is not DMA safe because it prepends a devres struct to the buffer making the buffer itself not cache line aligned. [Question] devm_kmalloc() for DMA ? https://linux-arm-kernel.infradead.narkive.com/vyJqy0RQ/question-devm-kmalloc-for-dma I'll turn this into a plain kmalloc when I return to my work on supporting device unplug which will deal with device resource lifetime issues. Noralf. ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [PATCH v5 5/5] drm/tinydrm: Switch from CMA to shmem buffers
On Sun, Oct 28, 2018 at 09:46:43PM +0100, Noralf Trønnes wrote: > > Den 28.10.2018 21.21, skrev David Lechner: > > On 10/26/2018 05:38 PM, Noralf Trønnes wrote: > > > Den 17.10.2018 15.04, skrev Noralf Trønnes: > > > > This move makes tinydrm useful for more drivers. tinydrm doesn't need > > > > continuous memory, but at the time it was convenient to use the CMA > > > > library. The spi core can do dma on is_vmalloc() addresses making this > > > > possible. > > > > > > > > Cc: David Lechner > > > > Signed-off-by: Noralf Trønnes > > > > Acked-by: David Lechner > > > > Tested-by: David Lechner > > > > --- > > > David, > > > FYI This series is scratched. > > > See the shmem helper patch thread for details. > > > > > Yes, I saw that. Thank you. > > > > I don't suppose there is a way to configure the DMA controller to do > > the byte swapping? > > > > Not that I know of. > "Proper" SPI hw can do 16-bit transfers and for them there's no problem. > The DMA capable SPI block on the Pi can only do 8-bit, hence the swapping. > > But that wasn't the problem, the byteswapping actually papered over the > problem. I'm no -mm expert so I don't know why the problem onyl showed > up when using the virtual address of the buffer used by fbcon and not on > mmap'ed fbdev as Daniel suggested would happen. Hm, I missed that detail. This sounds like one of the mappings ended up being write-combining (which avoids all the issues with dirty cpu cache lines), while the broken one was not. Or we ended up with a flush somewhere by accident. > Either way shmem not being coherent is a problem on the Pi, even though > I expected the DMA streaming API called by the spi core to flush "things". It should do that for you. At least if it's using dma_map/unmap_sg and friends. > The solution I'm aiming for is to make it easy for tinydrm drivers to use > the > buffer type they want, instead of having one type they all have to use. General recommendation for this is "less midlayer, more helper". Which was the goal of all this ... Oh well :-/ Maybe we can progress in other areas meanwhile. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [PATCH v5 5/5] drm/tinydrm: Switch from CMA to shmem buffers
Den 28.10.2018 21.21, skrev David Lechner: On 10/26/2018 05:38 PM, Noralf Trønnes wrote: Den 17.10.2018 15.04, skrev Noralf Trønnes: This move makes tinydrm useful for more drivers. tinydrm doesn't need continuous memory, but at the time it was convenient to use the CMA library. The spi core can do dma on is_vmalloc() addresses making this possible. Cc: David Lechner Signed-off-by: Noralf Trønnes Acked-by: David Lechner Tested-by: David Lechner --- David, FYI This series is scratched. See the shmem helper patch thread for details. Yes, I saw that. Thank you. I don't suppose there is a way to configure the DMA controller to do the byte swapping? Not that I know of. "Proper" SPI hw can do 16-bit transfers and for them there's no problem. The DMA capable SPI block on the Pi can only do 8-bit, hence the swapping. But that wasn't the problem, the byteswapping actually papered over the problem. I'm no -mm expert so I don't know why the problem onyl showed up when using the virtual address of the buffer used by fbcon and not on mmap'ed fbdev as Daniel suggested would happen. Either way shmem not being coherent is a problem on the Pi, even though I expected the DMA streaming API called by the spi core to flush "things". The solution I'm aiming for is to make it easy for tinydrm drivers to use the buffer type they want, instead of having one type they all have to use. Noralf. ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [PATCH v5 5/5] drm/tinydrm: Switch from CMA to shmem buffers
On 10/26/2018 05:38 PM, Noralf Trønnes wrote: > > Den 17.10.2018 15.04, skrev Noralf Trønnes: >> This move makes tinydrm useful for more drivers. tinydrm doesn't need >> continuous memory, but at the time it was convenient to use the CMA >> library. The spi core can do dma on is_vmalloc() addresses making this >> possible. >> >> Cc: David Lechner >> Signed-off-by: Noralf Trønnes >> Acked-by: David Lechner >> Tested-by: David Lechner >> --- > > David, > FYI This series is scratched. > See the shmem helper patch thread for details. > Yes, I saw that. Thank you. I don't suppose there is a way to configure the DMA controller to do the byte swapping? ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [PATCH v5 5/5] drm/tinydrm: Switch from CMA to shmem buffers
Den 17.10.2018 15.04, skrev Noralf Trønnes:
This move makes tinydrm useful for more drivers. tinydrm doesn't need
continuous memory, but at the time it was convenient to use the CMA
library. The spi core can do dma on is_vmalloc() addresses making this
possible.
Cc: David Lechner
Signed-off-by: Noralf Trønnes
Acked-by: David Lechner
Tested-by: David Lechner
---
David,
FYI This series is scratched.
See the shmem helper patch thread for details.
Noralf.
drivers/gpu/drm/tinydrm/Kconfig| 2 +-
drivers/gpu/drm/tinydrm/core/tinydrm-core.c| 92 +++---
drivers/gpu/drm/tinydrm/core/tinydrm-helpers.c | 5 ++
drivers/gpu/drm/tinydrm/ili9225.c | 14 ++--
drivers/gpu/drm/tinydrm/ili9341.c | 6 +-
drivers/gpu/drm/tinydrm/mi0283qt.c | 6 +-
drivers/gpu/drm/tinydrm/mipi-dbi.c | 38 ---
drivers/gpu/drm/tinydrm/repaper.c | 24 +++
drivers/gpu/drm/tinydrm/st7586.c | 15 +++--
drivers/gpu/drm/tinydrm/st7735r.c | 6 +-
include/drm/tinydrm/tinydrm.h | 36 +++---
11 files changed, 91 insertions(+), 153 deletions(-)
diff --git a/drivers/gpu/drm/tinydrm/Kconfig b/drivers/gpu/drm/tinydrm/Kconfig
index 16f4b5c91f1b..aa0cabba5ace 100644
--- a/drivers/gpu/drm/tinydrm/Kconfig
+++ b/drivers/gpu/drm/tinydrm/Kconfig
@@ -2,7 +2,7 @@ menuconfig DRM_TINYDRM
tristate "Support for simple displays"
depends on DRM
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_SHMEM_HELPER
help
Choose this option if you have a tinydrm supported display.
If M is selected the module will be called tinydrm.
diff --git a/drivers/gpu/drm/tinydrm/core/tinydrm-core.c
b/drivers/gpu/drm/tinydrm/core/tinydrm-core.c
index 255341ee4eb9..38ba361d1af2 100644
--- a/drivers/gpu/drm/tinydrm/core/tinydrm-core.c
+++ b/drivers/gpu/drm/tinydrm/core/tinydrm-core.c
@@ -12,6 +12,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -23,7 +24,7 @@
*
* It is based on _simple_display_pipe coupled with a _connector which
* has only one fixed _display_mode. The framebuffers are backed by the
- * cma helper and have support for framebuffer flushing (dirty).
+ * shmem buffers and have support for framebuffer flushing (dirty).
* fbdev support is also included.
*
*/
@@ -37,84 +38,41 @@
*/
/**
- * tinydrm_gem_cma_prime_import_sg_table - Produce a CMA GEM object from
- * another driver's scatter/gather table of pinned pages
- * @drm: DRM device to import into
- * @attach: DMA-BUF attachment
- * @sgt: Scatter/gather table of pinned pages
+ * tinydrm_fb_destroy - Destroy framebuffer
+ * @fb: Framebuffer
*
- * This function imports a scatter/gather table exported via DMA-BUF by
- * another driver using drm_gem_cma_prime_import_sg_table(). It sets the
- * kernel virtual address on the CMA object. Drivers should use this as their
- * _driver->gem_prime_import_sg_table callback if they need the virtual
- * address. tinydrm_gem_cma_free_object() should be used in combination with
- * this function.
- *
- * Returns:
- * A pointer to a newly created GEM object or an ERR_PTR-encoded negative
- * error code on failure.
+ * This function unmaps the virtual address on the backing buffer and destroys
the framebuffer.
+ * Drivers should use this as their _framebuffer_funcs->destroy callback.
*/
-struct drm_gem_object *
-tinydrm_gem_cma_prime_import_sg_table(struct drm_device *drm,
- struct dma_buf_attachment *attach,
- struct sg_table *sgt)
+void tinydrm_fb_destroy(struct drm_framebuffer *fb)
{
- struct drm_gem_cma_object *cma_obj;
- struct drm_gem_object *obj;
- void *vaddr;
+ struct drm_gem_object *gem = drm_gem_fb_get_obj(fb, 0);
+ struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(gem);
- vaddr = dma_buf_vmap(attach->dmabuf);
- if (!vaddr) {
- DRM_ERROR("Failed to vmap PRIME buffer\n");
- return ERR_PTR(-ENOMEM);
- }
-
- obj = drm_gem_cma_prime_import_sg_table(drm, attach, sgt);
- if (IS_ERR(obj)) {
- dma_buf_vunmap(attach->dmabuf, vaddr);
- return obj;
- }
-
- cma_obj = to_drm_gem_cma_obj(obj);
- cma_obj->vaddr = vaddr;
-
- return obj;
+ drm_gem_vunmap(gem, shmem->vaddr);
+ drm_gem_fb_destroy(fb);
}
-EXPORT_SYMBOL(tinydrm_gem_cma_prime_import_sg_table);
-
-/**
- * tinydrm_gem_cma_free_object - Free resources associated with a CMA GEM
- * object
- * @gem_obj: GEM object to free
- *
- * This function frees the backing memory of the CMA GEM object, cleans up the
- * GEM object state and frees the memory used to store the object itself using
- * drm_gem_cma_free_object(). It also handles PRIME buffers
[Intel-gfx] [PATCH v5 5/5] drm/tinydrm: Switch from CMA to shmem buffers
This move makes tinydrm useful for more drivers. tinydrm doesn't need
continuous memory, but at the time it was convenient to use the CMA
library. The spi core can do dma on is_vmalloc() addresses making this
possible.
Cc: David Lechner
Signed-off-by: Noralf Trønnes
Acked-by: David Lechner
Tested-by: David Lechner
---
drivers/gpu/drm/tinydrm/Kconfig| 2 +-
drivers/gpu/drm/tinydrm/core/tinydrm-core.c| 92 +++---
drivers/gpu/drm/tinydrm/core/tinydrm-helpers.c | 5 ++
drivers/gpu/drm/tinydrm/ili9225.c | 14 ++--
drivers/gpu/drm/tinydrm/ili9341.c | 6 +-
drivers/gpu/drm/tinydrm/mi0283qt.c | 6 +-
drivers/gpu/drm/tinydrm/mipi-dbi.c | 38 ---
drivers/gpu/drm/tinydrm/repaper.c | 24 +++
drivers/gpu/drm/tinydrm/st7586.c | 15 +++--
drivers/gpu/drm/tinydrm/st7735r.c | 6 +-
include/drm/tinydrm/tinydrm.h | 36 +++---
11 files changed, 91 insertions(+), 153 deletions(-)
diff --git a/drivers/gpu/drm/tinydrm/Kconfig b/drivers/gpu/drm/tinydrm/Kconfig
index 16f4b5c91f1b..aa0cabba5ace 100644
--- a/drivers/gpu/drm/tinydrm/Kconfig
+++ b/drivers/gpu/drm/tinydrm/Kconfig
@@ -2,7 +2,7 @@ menuconfig DRM_TINYDRM
tristate "Support for simple displays"
depends on DRM
select DRM_KMS_HELPER
- select DRM_KMS_CMA_HELPER
+ select DRM_GEM_SHMEM_HELPER
help
Choose this option if you have a tinydrm supported display.
If M is selected the module will be called tinydrm.
diff --git a/drivers/gpu/drm/tinydrm/core/tinydrm-core.c
b/drivers/gpu/drm/tinydrm/core/tinydrm-core.c
index 255341ee4eb9..38ba361d1af2 100644
--- a/drivers/gpu/drm/tinydrm/core/tinydrm-core.c
+++ b/drivers/gpu/drm/tinydrm/core/tinydrm-core.c
@@ -12,6 +12,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -23,7 +24,7 @@
*
* It is based on _simple_display_pipe coupled with a _connector which
* has only one fixed _display_mode. The framebuffers are backed by the
- * cma helper and have support for framebuffer flushing (dirty).
+ * shmem buffers and have support for framebuffer flushing (dirty).
* fbdev support is also included.
*
*/
@@ -37,84 +38,41 @@
*/
/**
- * tinydrm_gem_cma_prime_import_sg_table - Produce a CMA GEM object from
- * another driver's scatter/gather table of pinned pages
- * @drm: DRM device to import into
- * @attach: DMA-BUF attachment
- * @sgt: Scatter/gather table of pinned pages
+ * tinydrm_fb_destroy - Destroy framebuffer
+ * @fb: Framebuffer
*
- * This function imports a scatter/gather table exported via DMA-BUF by
- * another driver using drm_gem_cma_prime_import_sg_table(). It sets the
- * kernel virtual address on the CMA object. Drivers should use this as their
- * _driver->gem_prime_import_sg_table callback if they need the virtual
- * address. tinydrm_gem_cma_free_object() should be used in combination with
- * this function.
- *
- * Returns:
- * A pointer to a newly created GEM object or an ERR_PTR-encoded negative
- * error code on failure.
+ * This function unmaps the virtual address on the backing buffer and destroys
the framebuffer.
+ * Drivers should use this as their _framebuffer_funcs->destroy callback.
*/
-struct drm_gem_object *
-tinydrm_gem_cma_prime_import_sg_table(struct drm_device *drm,
- struct dma_buf_attachment *attach,
- struct sg_table *sgt)
+void tinydrm_fb_destroy(struct drm_framebuffer *fb)
{
- struct drm_gem_cma_object *cma_obj;
- struct drm_gem_object *obj;
- void *vaddr;
+ struct drm_gem_object *gem = drm_gem_fb_get_obj(fb, 0);
+ struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(gem);
- vaddr = dma_buf_vmap(attach->dmabuf);
- if (!vaddr) {
- DRM_ERROR("Failed to vmap PRIME buffer\n");
- return ERR_PTR(-ENOMEM);
- }
-
- obj = drm_gem_cma_prime_import_sg_table(drm, attach, sgt);
- if (IS_ERR(obj)) {
- dma_buf_vunmap(attach->dmabuf, vaddr);
- return obj;
- }
-
- cma_obj = to_drm_gem_cma_obj(obj);
- cma_obj->vaddr = vaddr;
-
- return obj;
+ drm_gem_vunmap(gem, shmem->vaddr);
+ drm_gem_fb_destroy(fb);
}
-EXPORT_SYMBOL(tinydrm_gem_cma_prime_import_sg_table);
-
-/**
- * tinydrm_gem_cma_free_object - Free resources associated with a CMA GEM
- * object
- * @gem_obj: GEM object to free
- *
- * This function frees the backing memory of the CMA GEM object, cleans up the
- * GEM object state and frees the memory used to store the object itself using
- * drm_gem_cma_free_object(). It also handles PRIME buffers which has the
kernel
- * virtual address set by tinydrm_gem_cma_prime_import_sg_table(). Drivers
- * can use this as their _driver->gem_free_object_unlocked callback.
- */
