Re: [Nouveau] [PATCH v9 07/10] mm: Device exclusive memory access
On Wed, Jun 02, 2021 at 06:50:37PM +1000, Balbir Singh wrote: > On Wed, May 26, 2021 at 12:17:18AM -0700, John Hubbard wrote: > > On 5/25/21 4:51 AM, Balbir Singh wrote: > > ... > > > > How beneficial is this code to nouveau users? I see that it permits a > > > > part of OpenCL to be implemented, but how useful/important is this in > > > > the real world? > > > > > > That is a very good question! I've not reviewed the code, but a sample > > > program with the described use case would make things easy to parse. > > > I suspect that is not easy to build at the moment? > > > > > > > The cover letter says this: > > > > This has been tested with upstream Mesa 21.1.0 and a simple OpenCL program > > which checks that GPU atomic accesses to system memory are atomic. Without > > this series the test fails as there is no way of write-protecting the page > > mapping which results in the device clobbering CPU writes. For reference > > the test is available at https://ozlabs.org/~apopple/opencl_svm_atomics/ > > > > Further testing has been performed by adding support for testing exclusive > > access to the hmm-tests kselftests. > > > > ...so that seems to cover the "sample program" request, at least. > > Thanks, I'll take a look > > > > > > I wonder how we co-ordinate all the work the mm is doing, page migration, > > > reclaim with device exclusive access? Do we have any numbers for the worst > > > case page fault latency when something is marked away for exclusive > > > access? > > > > CPU page fault latency is approximately "terrible", if a page is resident on > > the GPU. We have to spin up a DMA engine on the GPU and have it copy the > > page > > over the PCIe bus, after all. > > > > > I presume for now this is anonymous memory only? SWP_DEVICE_EXCLUSIVE > > > would > > > > Yes, for now. > > > > > only impact the address space of programs using the GPU. Should the > > > exclusively > > > marked range live in the unreclaimable list and recycled back to > > > active/in-active > > > to account for the fact that > > > > > > 1. It is not reclaimable and reclaim will only hurt via page faults? > > > 2. It ages the page correctly or at-least allows for that possibility > > > when the > > > page is used by the GPU. > > > > I'm not sure that that is *necessarily* something we can conclude. It > > depends upon > > access patterns of each program. For example, a "reduction" parallel > > program sends > > over lots of data to the GPU, and only a tiny bit of (reduced!) data comes > > back > > to the CPU. In that case, freeing the physical page on the CPU is actually > > the > > best decision for the OS to make (if the OS is sufficiently prescient). > > > > With a shared device or a device exclusive range, it would be good to get the > device > usage pattern and update the mm with that knowledge, so that the LRU can be > better > maintained. With your comment you seem to suggest that a page used by the GPU > might > be a good candidate for reclaim based on the CPU's understanding of the age of > the page should not account for use by the device > (are GPU workloads - access once and discard?) Hmm, besides the aging info, this reminded me: do we need to isolate the page from lru too when marking device exclusive access? Afaict the current patch didn't do that so I think it's reclaimable. If we still have the rmap then we'll get a mmu notify CLEAR when unmapping that special pte, so device driver should be able to drop the ownership. However we dropped the rmap when marking exclusive. Now I don't know whether and how it'll work if page reclaim runs with the page being exclusively owned if without isolating the page.. -- Peter Xu ___ Nouveau mailing list Nouveau@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [Nouveau] [PATCH v9 07/10] mm: Device exclusive memory access
On Wed, May 26, 2021 at 12:17:18AM -0700, John Hubbard wrote: > On 5/25/21 4:51 AM, Balbir Singh wrote: > ... > > > How beneficial is this code to nouveau users? I see that it permits a > > > part of OpenCL to be implemented, but how useful/important is this in > > > the real world? > > > > That is a very good question! I've not reviewed the code, but a sample > > program with the described use case would make things easy to parse. > > I suspect that is not easy to build at the moment? > > > > The cover letter says this: > > This has been tested with upstream Mesa 21.1.0 and a simple OpenCL program > which checks that GPU atomic accesses to system memory are atomic. Without > this series the test fails as there is no way of write-protecting the page > mapping which results in the device clobbering CPU writes. For reference > the test is available at https://ozlabs.org/~apopple/opencl_svm_atomics/ > > Further testing has been performed by adding support for testing exclusive > access to the hmm-tests kselftests. > > ...so that seems to cover the "sample program" request, at least. Thanks, I'll take a look > > > I wonder how we co-ordinate all the work the mm is doing, page migration, > > reclaim with device exclusive access? Do we have any numbers for the worst > > case page fault latency when something is marked away for exclusive access? > > CPU page fault latency is approximately "terrible", if a page is resident on > the GPU. We have to spin up a DMA engine on the GPU and have it copy the page > over the PCIe bus, after all. > > > I presume for now this is anonymous memory only? SWP_DEVICE_EXCLUSIVE would > > Yes, for now. > > > only impact the address space of programs using the GPU. Should the > > exclusively > > marked range live in the unreclaimable list and recycled back to > > active/in-active > > to account for the fact that > > > > 1. It is not reclaimable and reclaim will only hurt via page faults? > > 2. It ages the page correctly or at-least allows for that possibility when > > the > > page is used by the GPU. > > I'm not sure that that is *necessarily* something we can conclude. It depends > upon > access patterns of each program. For example, a "reduction" parallel program > sends > over lots of data to the GPU, and only a tiny bit of (reduced!) data comes > back > to the CPU. In that case, freeing the physical page on the CPU is actually the > best decision for the OS to make (if the OS is sufficiently prescient). > With a shared device or a device exclusive range, it would be good to get the device usage pattern and update the mm with that knowledge, so that the LRU can be better maintained. With your comment you seem to suggest that a page used by the GPU might be a good candidate for reclaim based on the CPU's understanding of the age of the page should not account for use by the device (are GPU workloads - access once and discard?) Balbir Singh. ___ Nouveau mailing list Nouveau@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/nouveau
[Nouveau] [RESEND 00/26] Rid W=1 warnings from GPU
Some off these patches have been knocking around for a while.
Who will hoover them up please?
This set is part of a larger effort attempting to clean-up W=1
kernel builds, which are currently overwhelmingly riddled with
niggly little warnings.
Lee Jones (26):
drm/mediatek/mtk_disp_color: Strip incorrect doc and demote header
drm/mediatek/mtk_disp_gamma: Strip and demote non-conformant
kernel-doc header
drm/mediatek/mtk_disp_ovl: Strip and demote non-conformant header
drm/mediatek/mtk_disp_rdma: Strip and demote non-conformant kernel-doc
header
drm/sti/sti_hdmi_tx3g4c28phy: Provide function names for kernel-doc
headers
drm/sti/sti_hda: Provide missing function names
drm/sti/sti_tvout: Provide a bunch of missing function names
drm/sti/sti_hqvdp: Fix incorrectly named function 'sti_hqvdp_vtg_cb()'
drm/msm/disp/dpu1/dpu_encoder_phys_cmd: Remove unused variable
'cmd_enc'
drm/msm/disp/dpu1/dpu_hw_interrupts: Demote a bunch of kernel-doc
abuses
drm/msm/disp/dpu1/dpu_plane: Fix a couple of naming issues
drm/msm/msm_gem: Demote kernel-doc abuses
drm/msm/dp/dp_catalog: Correctly document param 'dp_catalog'
drm/msm/dp/dp_link: Fix some potential doc-rot
drm/nouveau/nvkm/subdev/mc/tu102: Make functions called by reference
static
drm/amd/display/dc/dce/dce_transform: Remove superfluous
re-initialisation of DCFE_MEM_LIGHT_SLEEP_CNTL,
drm/xlnx/zynqmp_disp: Fix incorrectly named enum
'zynqmp_disp_layer_id'
drm/xlnx/zynqmp_dp: Fix incorrectly name function 'zynqmp_dp_train()'
drm/ttm/ttm_tt: Demote non-conformant kernel-doc header
drm/panel/panel-raspberrypi-touchscreen: Demote kernel-doc abuse
drm/panel/panel-sitronix-st7701: Demote kernel-doc abuse
drm/vgem/vgem_drv: Standard comment blocks should not use kernel-doc
format
drm/exynos/exynos7_drm_decon: Fix incorrect naming of
'decon_shadow_protect_win()'
drm/exynos/exynos_drm_ipp: Fix documentation for
'exynos_drm_ipp_get_{caps,res}_ioctl()'
drm/vboxvideo/hgsmi_base: Place function names into headers
drm/vboxvideo/modesetting: Provide function names for prototype
headers
.../drm/amd/display/dc/dce/dce_transform.h| 3 +-
drivers/gpu/drm/exynos/exynos7_drm_decon.c| 2 +-
drivers/gpu/drm/exynos/exynos_drm_ipp.c | 4 +--
drivers/gpu/drm/mediatek/mtk_disp_color.c | 3 +-
drivers/gpu/drm/mediatek/mtk_disp_gamma.c | 4 +--
drivers/gpu/drm/mediatek/mtk_disp_ovl.c | 3 +-
drivers/gpu/drm/mediatek/mtk_disp_rdma.c | 4 +--
.../drm/msm/disp/dpu1/dpu_encoder_phys_cmd.c | 4 ---
.../gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.c | 32 +--
drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c | 4 +--
drivers/gpu/drm/msm/dp/dp_catalog.c | 2 +-
drivers/gpu/drm/msm/dp/dp_link.c | 6 ++--
drivers/gpu/drm/msm/msm_gem.c | 4 +--
.../gpu/drm/nouveau/nvkm/subdev/mc/tu102.c| 6 ++--
.../drm/panel/panel-raspberrypi-touchscreen.c | 2 +-
drivers/gpu/drm/panel/panel-sitronix-st7701.c | 2 +-
drivers/gpu/drm/sti/sti_hda.c | 6 ++--
drivers/gpu/drm/sti/sti_hdmi_tx3g4c28phy.c| 4 +--
drivers/gpu/drm/sti/sti_hqvdp.c | 2 +-
drivers/gpu/drm/sti/sti_tvout.c | 18 +--
drivers/gpu/drm/ttm/ttm_tt.c | 2 +-
drivers/gpu/drm/vboxvideo/hgsmi_base.c| 19 +++
drivers/gpu/drm/vboxvideo/modesetting.c | 20 +++-
drivers/gpu/drm/vgem/vgem_drv.c | 2 +-
drivers/gpu/drm/xlnx/zynqmp_disp.c| 2 +-
drivers/gpu/drm/xlnx/zynqmp_dp.c | 2 +-
26 files changed, 80 insertions(+), 82 deletions(-)
Cc: Adam Jackson
Cc: Ajay Kumar
Cc: Akshu Agarwal
Cc: Alex Deucher
Cc: Alistair Popple
Cc: amd-...@lists.freedesktop.org
Cc: AngeloGioacchino Del Regno
Cc: Benjamin Gaignard
Cc: Ben Skeggs
Cc: Ben Widawsky
Cc: Chandan Uddaraju
Cc: Christian Koenig
Cc: "Christian König"
Cc: Chun-Kuang Hu
Cc: Daniel Vetter
Cc: David Airlie
Cc: dri-de...@lists.freedesktop.org
Cc: Eric Anholt
Cc: Fabien Dessenne
Cc: freedr...@lists.freedesktop.org
Cc: Hans de Goede
Cc: Harry Wentland
Cc: Huang Rui
Cc: Hyun Kwon
Cc: Inki Dae
Cc: Jagan Teki
Cc: Joonyoung Shim
Cc: Krishna Manikandan
Cc: Krzysztof Kozlowski
Cc: Kuogee Hsieh
Cc: Kyungmin Park
Cc: Laurent Pinchart
Cc: Leo Li
Cc: linaro-mm-...@lists.linaro.org
Cc: linux-arm-ker...@lists.infradead.org
Cc: linux-arm-...@vger.kernel.org
Cc: linux-media...@lists.infradead.org
Cc: linux-me...@vger.kernel.org
Cc: linux-samsung-...@vger.kernel.org
Cc: Marek Szyprowski
Cc: Matthias Brugger
Cc: Mauro Rossi
Cc: Michal Simek
Cc: nouveau@lists.freedesktop.org
Cc: Philipp Zabel
Cc: Rob Clark
Cc: Sam Ravnborg
Cc: Sean Paul
Cc: Seung-Woo Kim
Cc: Stephen Boyd
Cc: Sumit Semwal
Cc: Thierry Reding
Cc: Vincent Abriou
--
2.31.1
___
Nouveau mailing list
Nouveau@lists.freedesktop.org
[Nouveau] [RESEND 15/26] drm/nouveau/nvkm/subdev/mc/tu102: Make functions called by reference static
Fixes the following W=1 kernel build warning(s):
drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c:50:1: warning: no previous
prototype for ‘tu102_mc_intr_unarm’ [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c:62:1: warning: no previous
prototype for ‘tu102_mc_intr_rearm’ [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c:74:1: warning: no previous
prototype for ‘tu102_mc_intr_mask’ [-Wmissing-prototypes]
Cc: Ben Skeggs
Cc: David Airlie
Cc: Daniel Vetter
Cc: Alistair Popple
Cc: dri-de...@lists.freedesktop.org
Cc: nouveau@lists.freedesktop.org
Signed-off-by: Lee Jones
---
drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c
b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c
index 58db83ebadc5f..a96084b34a788 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/tu102.c
@@ -46,7 +46,7 @@ tu102_mc_intr_update(struct tu102_mc *mc)
nvkm_wr32(device, 0xb81610, 0x6);
}
-void
+static void
tu102_mc_intr_unarm(struct nvkm_mc *base)
{
struct tu102_mc *mc = tu102_mc(base);
@@ -58,7 +58,7 @@ tu102_mc_intr_unarm(struct nvkm_mc *base)
spin_unlock_irqrestore(>lock, flags);
}
-void
+static void
tu102_mc_intr_rearm(struct nvkm_mc *base)
{
struct tu102_mc *mc = tu102_mc(base);
@@ -70,7 +70,7 @@ tu102_mc_intr_rearm(struct nvkm_mc *base)
spin_unlock_irqrestore(>lock, flags);
}
-void
+static void
tu102_mc_intr_mask(struct nvkm_mc *base, u32 mask, u32 intr)
{
struct tu102_mc *mc = tu102_mc(base);
--
2.31.1
___
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau
