Re: [PATCH] drm/msm: Only enable A6xx LLCC code on A6xx
Hi Rob, Konrad, On 2021-01-07 22:56, Rob Clark wrote: > On Wed, Jan 6, 2021 at 8:50 PM Sai Prakash Ranjan > wrote: >> >> On 2021-01-05 01:00, Konrad Dybcio wrote: >> > Using this code on A5xx (and probably older too) causes a >> > smmu bug. >> > >> > Fixes: 474dadb8b0d5 ("drm/msm/a6xx: Add support for using system >> > cache(LLC)") >> > Signed-off-by: Konrad Dybcio >> > Tested-by: AngeloGioacchino Del Regno >> > >> > --- >> >> Reviewed-by: Sai Prakash Ranjan >> >> > drivers/gpu/drm/msm/adreno/adreno_gpu.c | 21 - >> > drivers/gpu/drm/msm/adreno/adreno_gpu.h | 5 + >> > 2 files changed, 17 insertions(+), 9 deletions(-) >> > >> > diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c >> > b/drivers/gpu/drm/msm/adreno/adreno_gpu.c >> > index 6cf9975e951e..f09175698827 100644 >> > --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c >> > +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c >> > @@ -191,8 +191,6 @@ adreno_iommu_create_address_space(struct msm_gpu >> > *gpu, >> > struct platform_device *pdev) >> > { >> > struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); >> > - struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); >> > - struct io_pgtable_domain_attr pgtbl_cfg; >> > struct iommu_domain *iommu; >> > struct msm_mmu *mmu; > > struct msm_gem_address_space *aspace; >> > @@ -202,13 +200,18 @@ adreno_iommu_create_address_space(struct msm_gpu >> > *gpu, >> > if (!iommu) >> > return NULL; >> > >> > - /* >> > - * This allows GPU to set the bus attributes required to use system >> > - * cache on behalf of the iommu page table walker. >> > - */ >> > - if (!IS_ERR(a6xx_gpu->htw_llc_slice)) { >> > - pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA; >> > - iommu_domain_set_attr(iommu, DOMAIN_ATTR_IO_PGTABLE_CFG, >> > _cfg); >> > + >> > + if (adreno_is_a6xx(adreno_gpu)) { >> > + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); >> > + struct io_pgtable_domain_attr pgtbl_cfg; >> > + /* >> > + * This allows GPU to set the bus attributes required to use >> > system >> > + * cache on behalf of the iommu page table walker. >> > + */ >> > + if (!IS_ERR(a6xx_gpu->htw_llc_slice)) { >> > + pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA; >> > + iommu_domain_set_attr(iommu, >> > DOMAIN_ATTR_IO_PGTABLE_CFG, >> > _cfg); >> > + } > > I'm applying for -fixes as this is an obvious problem.. But kinda > thinking that we should try to move it into an a6xx specific > create_address_space() (or wrapper for the generic fxn) > > Sai/Jordan, could I talk one of you into trying to clean this up > better for next cycle? > Looking more closely(sorry I should have before), the quirk setting is already guarded by htw_llc_slice check but what is happening here is that check is not proper when LLCC is disabled i.e., CONFIG_QCOM_LLCC=n. When LLCC is disabled, htw_llc_slice is set to NULL and the !IS_ERR check passes because it doesn't take care of NULL and quirk is set causing bugs. So the proper fix would be to use IS_ERR_OR_NULL for the check. Konrad, can you please test this below change without your change? diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 130661898546..3b798e883f82 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -1117,7 +1117,7 @@ static void a6xx_llc_slices_init(struct platform_device *pdev, a6xx_gpu->llc_slice = llcc_slice_getd(LLCC_GPU); a6xx_gpu->htw_llc_slice = llcc_slice_getd(LLCC_GPUHTW); - if (IS_ERR(a6xx_gpu->llc_slice) && IS_ERR(a6xx_gpu->htw_llc_slice)) + if (IS_ERR_OR_NULL(a6xx_gpu->llc_slice) && IS_ERR_OR_NULL(a6xx_gpu->htw_llc_slice)) a6xx_gpu->llc_mmio = ERR_PTR(-EINVAL); } diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c index 6cf9975e951e..dbd5cacddb9c 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c @@ -206,7 +206,7 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu, * This allows GPU to set the bus attributes required to use system * cache on behalf of the iommu page table walker. */ - if (!IS_ERR(a6xx_gpu->htw_llc_slice)) { + if (!IS_ERR_OR_NULL(a6xx_gpu->htw_llc_slice)) { pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA; iommu_domain_set_attr(iommu, DOMAIN_ATTR_IO_PGTABLE_CFG, _cfg); } Thanks, Sai -- QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH] drm/msm: Only enable A6xx LLCC code on A6xx
On 2021-01-08 19:09, Konrad Dybcio wrote: Konrad, can you please test this below change without your change? This brings no difference, a BUG still happens. We're still calling to_a6xx_gpu on ANY device that's probed! Too bad it won't turn my A330 into an A640.. Also, relying on disabling LLCC in the config is out of question as it makes the arm32 kernel not compile with DRM/MSM and it just removes the functionality on devices with a6xx.. (unless somebody removes the dependency on it, which in my opinion is even worse and will cause more problems for developers!). Disabling LLCC is not the suggestion, I was under the impression that was the cause here for the smmu bug. Anyways, the check for llc slice in case llcc is disabled is not correct as well. I will send a patch for that as well. The bigger question is how and why did that piece of code ever make it to adreno_gpu.c and not a6xx_gpu.c? My mistake, I will move it. To solve it in a cleaner way I propose to move it to an a6xx-specific file, or if it's going to be used with next-gen GPUs, perhaps manage calling of this code via an adreno quirk/feature in adreno_device.c. Now that I think about it, A5xx GPMU en/disable could probably managed like that, instead of using tons of if-statements for each GPU model that has it.. While we're at it, do ALL (and I truly do mean ALL, including the low-end ones, this will be important later on) A6xx GPUs make use of that feature? I do not have a list of all A6XX GPUs with me currently, but from what I know, A618, A630, A640, A650 has the support. Thanks, Sai -- QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH] drm/msm: Only enable A6xx LLCC code on A6xx
> Konrad, can you please test this below change without your change? This brings no difference, a BUG still happens. We're still calling to_a6xx_gpu on ANY device that's probed! Too bad it won't turn my A330 into an A640.. Also, relying on disabling LLCC in the config is out of question as it makes the arm32 kernel not compile with DRM/MSM and it just removes the functionality on devices with a6xx.. (unless somebody removes the dependency on it, which in my opinion is even worse and will cause more problems for developers!). The bigger question is how and why did that piece of code ever make it to adreno_gpu.c and not a6xx_gpu.c? To solve it in a cleaner way I propose to move it to an a6xx-specific file, or if it's going to be used with next-gen GPUs, perhaps manage calling of this code via an adreno quirk/feature in adreno_device.c. Now that I think about it, A5xx GPMU en/disable could probably managed like that, instead of using tons of if-statements for each GPU model that has it.. While we're at it, do ALL (and I truly do mean ALL, including the low-end ones, this will be important later on) A6xx GPUs make use of that feature? Konrad ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH] drm/msm: Only enable A6xx LLCC code on A6xx
On Wed, Jan 6, 2021 at 8:50 PM Sai Prakash Ranjan wrote: > > On 2021-01-05 01:00, Konrad Dybcio wrote: > > Using this code on A5xx (and probably older too) causes a > > smmu bug. > > > > Fixes: 474dadb8b0d5 ("drm/msm/a6xx: Add support for using system > > cache(LLC)") > > Signed-off-by: Konrad Dybcio > > Tested-by: AngeloGioacchino Del Regno > > > > --- > > Reviewed-by: Sai Prakash Ranjan > > > drivers/gpu/drm/msm/adreno/adreno_gpu.c | 21 - > > drivers/gpu/drm/msm/adreno/adreno_gpu.h | 5 + > > 2 files changed, 17 insertions(+), 9 deletions(-) > > > > diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c > > b/drivers/gpu/drm/msm/adreno/adreno_gpu.c > > index 6cf9975e951e..f09175698827 100644 > > --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c > > +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c > > @@ -191,8 +191,6 @@ adreno_iommu_create_address_space(struct msm_gpu > > *gpu, > > struct platform_device *pdev) > > { > > struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); > > - struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > > - struct io_pgtable_domain_attr pgtbl_cfg; > > struct iommu_domain *iommu; > > struct msm_mmu *mmu; > > struct msm_gem_address_space *aspace; > > @@ -202,13 +200,18 @@ adreno_iommu_create_address_space(struct msm_gpu > > *gpu, > > if (!iommu) > > return NULL; > > > > - /* > > - * This allows GPU to set the bus attributes required to use system > > - * cache on behalf of the iommu page table walker. > > - */ > > - if (!IS_ERR(a6xx_gpu->htw_llc_slice)) { > > - pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA; > > - iommu_domain_set_attr(iommu, DOMAIN_ATTR_IO_PGTABLE_CFG, > > _cfg); > > + > > + if (adreno_is_a6xx(adreno_gpu)) { > > + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > > + struct io_pgtable_domain_attr pgtbl_cfg; > > + /* > > + * This allows GPU to set the bus attributes required to use > > system > > + * cache on behalf of the iommu page table walker. > > + */ > > + if (!IS_ERR(a6xx_gpu->htw_llc_slice)) { > > + pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA; > > + iommu_domain_set_attr(iommu, > > DOMAIN_ATTR_IO_PGTABLE_CFG, > > _cfg); > > + } I'm applying for -fixes as this is an obvious problem.. But kinda thinking that we should try to move it into an a6xx specific create_address_space() (or wrapper for the generic fxn) Sai/Jordan, could I talk one of you into trying to clean this up better for next cycle? BR, -R > > } > > > > mmu = msm_iommu_new(>dev, iommu); > > diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h > > b/drivers/gpu/drm/msm/adreno/adreno_gpu.h > > index 4574d85c5680..08421fa54a50 100644 > > --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h > > +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h > > @@ -226,6 +226,11 @@ static inline int adreno_is_a540(struct adreno_gpu > > *gpu) > > return gpu->revn == 540; > > } > > > > +static inline bool adreno_is_a6xx(struct adreno_gpu *gpu) > > +{ > > + return ((gpu->revn < 700 && gpu->revn > 599)); > > +} > > + > > static inline int adreno_is_a618(struct adreno_gpu *gpu) > > { > > return gpu->revn == 618; > > -- > QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a > member > of Code Aurora Forum, hosted by The Linux Foundation ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH] drm/msm: Only enable A6xx LLCC code on A6xx
On 2021-01-05 01:00, Konrad Dybcio wrote: Using this code on A5xx (and probably older too) causes a smmu bug. Fixes: 474dadb8b0d5 ("drm/msm/a6xx: Add support for using system cache(LLC)") Signed-off-by: Konrad Dybcio Tested-by: AngeloGioacchino Del Regno --- Reviewed-by: Sai Prakash Ranjan drivers/gpu/drm/msm/adreno/adreno_gpu.c | 21 - drivers/gpu/drm/msm/adreno/adreno_gpu.h | 5 + 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c index 6cf9975e951e..f09175698827 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c @@ -191,8 +191,6 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu, struct platform_device *pdev) { struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); - struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); - struct io_pgtable_domain_attr pgtbl_cfg; struct iommu_domain *iommu; struct msm_mmu *mmu; struct msm_gem_address_space *aspace; @@ -202,13 +200,18 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu, if (!iommu) return NULL; - /* -* This allows GPU to set the bus attributes required to use system -* cache on behalf of the iommu page table walker. -*/ - if (!IS_ERR(a6xx_gpu->htw_llc_slice)) { - pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA; - iommu_domain_set_attr(iommu, DOMAIN_ATTR_IO_PGTABLE_CFG, _cfg); + + if (adreno_is_a6xx(adreno_gpu)) { + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct io_pgtable_domain_attr pgtbl_cfg; + /* + * This allows GPU to set the bus attributes required to use system + * cache on behalf of the iommu page table walker. + */ + if (!IS_ERR(a6xx_gpu->htw_llc_slice)) { + pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA; + iommu_domain_set_attr(iommu, DOMAIN_ATTR_IO_PGTABLE_CFG, _cfg); + } } mmu = msm_iommu_new(>dev, iommu); diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h index 4574d85c5680..08421fa54a50 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h @@ -226,6 +226,11 @@ static inline int adreno_is_a540(struct adreno_gpu *gpu) return gpu->revn == 540; } +static inline bool adreno_is_a6xx(struct adreno_gpu *gpu) +{ + return ((gpu->revn < 700 && gpu->revn > 599)); +} + static inline int adreno_is_a618(struct adreno_gpu *gpu) { return gpu->revn == 618; -- QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH] drm/msm: Only enable A6xx LLCC code on A6xx
On Mon, Jan 04, 2021 at 08:30:41PM +0100, Konrad Dybcio wrote: > Using this code on A5xx (and probably older too) causes a > smmu bug. > > Fixes: 474dadb8b0d5 ("drm/msm/a6xx: Add support for using system cache(LLC)") > Signed-off-by: Konrad Dybcio > Tested-by: AngeloGioacchino Del Regno > Yep, I can see how this would be not ideal. Reviewed-by: Jordan Crouse > --- > drivers/gpu/drm/msm/adreno/adreno_gpu.c | 21 - > drivers/gpu/drm/msm/adreno/adreno_gpu.h | 5 + > 2 files changed, 17 insertions(+), 9 deletions(-) > > diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c > b/drivers/gpu/drm/msm/adreno/adreno_gpu.c > index 6cf9975e951e..f09175698827 100644 > --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c > +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c > @@ -191,8 +191,6 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu, > struct platform_device *pdev) > { > struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); > - struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > - struct io_pgtable_domain_attr pgtbl_cfg; > struct iommu_domain *iommu; > struct msm_mmu *mmu; > struct msm_gem_address_space *aspace; > @@ -202,13 +200,18 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu, > if (!iommu) > return NULL; > > - /* > - * This allows GPU to set the bus attributes required to use system > - * cache on behalf of the iommu page table walker. > - */ > - if (!IS_ERR(a6xx_gpu->htw_llc_slice)) { > - pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA; > - iommu_domain_set_attr(iommu, DOMAIN_ATTR_IO_PGTABLE_CFG, > _cfg); > + > + if (adreno_is_a6xx(adreno_gpu)) { > + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > + struct io_pgtable_domain_attr pgtbl_cfg; > + /* > + * This allows GPU to set the bus attributes required to use > system > + * cache on behalf of the iommu page table walker. > + */ > + if (!IS_ERR(a6xx_gpu->htw_llc_slice)) { > + pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA; > + iommu_domain_set_attr(iommu, > DOMAIN_ATTR_IO_PGTABLE_CFG, _cfg); > + } > } > > mmu = msm_iommu_new(>dev, iommu); > diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h > b/drivers/gpu/drm/msm/adreno/adreno_gpu.h > index 4574d85c5680..08421fa54a50 100644 > --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h > +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h > @@ -226,6 +226,11 @@ static inline int adreno_is_a540(struct adreno_gpu *gpu) > return gpu->revn == 540; > } > > +static inline bool adreno_is_a6xx(struct adreno_gpu *gpu) > +{ > + return ((gpu->revn < 700 && gpu->revn > 599)); > +} > + > static inline int adreno_is_a618(struct adreno_gpu *gpu) > { > return gpu->revn == 618; > -- > 2.29.2 > -- The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel