Re: [PATCHv3] drm/amdkfd: Fix unaligned 64-bit doorbell warning
On 9/6/2023 9:09 PM, Mukul Joshi wrote:
This patch fixes the following unaligned 64-bit doorbell
warning seen when submitting packets on HIQ on GFX v9.4.3
by making the HIQ doorbell 64-bit aligned.
The warning is seen when GPU is loaded in any mode other
than SPX mode.
[ +0.000301] [ cut here ]
[ +0.03] Unaligned 64-bit doorbell
[ +0.30] WARNING: /amdkfd/kfd_doorbell.c:339
write_kernel_doorbell64+0x72/0x80
[ +0.03] RIP: 0010:write_kernel_doorbell64+0x72/0x80
[ +0.04] RSP: 0018:c90004287730 EFLAGS: 00010246
[ +0.05] RAX: RBX: RCX:
[ +0.03] RDX: 0001 RSI: 82837c71 RDI:
[ +0.03] RBP: c90004287748 R08: 0003 R09: 0001
[ +0.02] R10: 001a R11: 88a034008198 R12: c900013bd004
[ +0.03] R13: 0008 R14: c900042877b0 R15: 007f
[ +0.03] FS: 7fa8c7b62000() GS:889f8840()
knlGS:
[ +0.04] CS: 0010 DS: ES: CR0: 80050033
[ +0.03] CR2: 56111c45aaf0 CR3: 0001414f2002 CR4: 00770ee0
[ +0.03] PKRU: 5554
[ +0.02] Call Trace:
[ +0.04]
[ +0.06] kq_submit_packet+0x45/0x50 [amdgpu]
[ +0.000524] pm_send_set_resources+0x7f/0xc0 [amdgpu]
[ +0.000500] set_sched_resources+0xe4/0x160 [amdgpu]
[ +0.000503] start_cpsch+0x1c5/0x2a0 [amdgpu]
[ +0.000497] kgd2kfd_device_init.cold+0x816/0xb42 [amdgpu]
[ +0.000743] amdgpu_amdkfd_device_init+0x15f/0x1f0 [amdgpu]
[ +0.000602] amdgpu_device_init.cold+0x1813/0x2176 [amdgpu]
[ +0.000684] ? pci_bus_read_config_word+0x4a/0x80
[ +0.12] ? do_pci_enable_device+0xdc/0x110
[ +0.08] amdgpu_driver_load_kms+0x1a/0x110 [amdgpu]
[ +0.000545] amdgpu_pci_probe+0x197/0x400 [amdgpu]
Fixes: cfeaeb3c0ce7 ("drm/amdgpu: use doorbell mgr for kfd kernel doorbells")
Signed-off-by: Mukul Joshi
---
v1->v2:
- Update the logic to make it work with both 32 bit
64 bit doorbells.
- Add the Fixed tag
v2->v3:
- Revert to the original change to align it with whats done in
amdgpu_doorbell_index_on_bar.
drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
index c2e0b79dcc6d..7b38537c7c99 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
@@ -162,6 +162,7 @@ void __iomem *kfd_get_kernel_doorbell(struct kfd_dev *kfd,
return NULL;
*doorbell_off = amdgpu_doorbell_index_on_bar(kfd->adev, kfd->doorbells, inx);
+ inx *= 2;
To be more clear, suggest to use a macro here and at
amdgpu_doorbell_index_on_bar() -
DOORBELL_INDX2ABS()/DOORBELL_ABS2INDX()
Thanks,
Lijo
pr_debug("Get kernel queue doorbell\n"
" doorbell offset == 0x%08X\n"
@@ -176,6 +177,7 @@ void kfd_release_kernel_doorbell(struct kfd_dev *kfd, u32
__iomem *db_addr)
unsigned int inx;
inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr);
+ inx /= 2;
mutex_lock(>doorbell_mutex);
__clear_bit(inx, kfd->doorbell_bitmap);
Re: [PATCHv3] drm/amdkfd: Fix unaligned 64-bit doorbell warning
On 2023-09-06 11:39, Mukul Joshi wrote:
This patch fixes the following unaligned 64-bit doorbell
warning seen when submitting packets on HIQ on GFX v9.4.3
by making the HIQ doorbell 64-bit aligned.
The warning is seen when GPU is loaded in any mode other
than SPX mode.
[ +0.000301] [ cut here ]
[ +0.03] Unaligned 64-bit doorbell
[ +0.30] WARNING: /amdkfd/kfd_doorbell.c:339
write_kernel_doorbell64+0x72/0x80
[ +0.03] RIP: 0010:write_kernel_doorbell64+0x72/0x80
[ +0.04] RSP: 0018:c90004287730 EFLAGS: 00010246
[ +0.05] RAX: RBX: RCX:
[ +0.03] RDX: 0001 RSI: 82837c71 RDI:
[ +0.03] RBP: c90004287748 R08: 0003 R09: 0001
[ +0.02] R10: 001a R11: 88a034008198 R12: c900013bd004
[ +0.03] R13: 0008 R14: c900042877b0 R15: 007f
[ +0.03] FS: 7fa8c7b62000() GS:889f8840()
knlGS:
[ +0.04] CS: 0010 DS: ES: CR0: 80050033
[ +0.03] CR2: 56111c45aaf0 CR3: 0001414f2002 CR4: 00770ee0
[ +0.03] PKRU: 5554
[ +0.02] Call Trace:
[ +0.04]
[ +0.06] kq_submit_packet+0x45/0x50 [amdgpu]
[ +0.000524] pm_send_set_resources+0x7f/0xc0 [amdgpu]
[ +0.000500] set_sched_resources+0xe4/0x160 [amdgpu]
[ +0.000503] start_cpsch+0x1c5/0x2a0 [amdgpu]
[ +0.000497] kgd2kfd_device_init.cold+0x816/0xb42 [amdgpu]
[ +0.000743] amdgpu_amdkfd_device_init+0x15f/0x1f0 [amdgpu]
[ +0.000602] amdgpu_device_init.cold+0x1813/0x2176 [amdgpu]
[ +0.000684] ? pci_bus_read_config_word+0x4a/0x80
[ +0.12] ? do_pci_enable_device+0xdc/0x110
[ +0.08] amdgpu_driver_load_kms+0x1a/0x110 [amdgpu]
[ +0.000545] amdgpu_pci_probe+0x197/0x400 [amdgpu]
Fixes: cfeaeb3c0ce7 ("drm/amdgpu: use doorbell mgr for kfd kernel doorbells")
Signed-off-by: Mukul Joshi
Reviewed-by: Felix Kuehling
---
v1->v2:
- Update the logic to make it work with both 32 bit
64 bit doorbells.
- Add the Fixed tag
v2->v3:
- Revert to the original change to align it with whats done in
amdgpu_doorbell_index_on_bar.
drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
index c2e0b79dcc6d..7b38537c7c99 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
@@ -162,6 +162,7 @@ void __iomem *kfd_get_kernel_doorbell(struct kfd_dev *kfd,
return NULL;
*doorbell_off = amdgpu_doorbell_index_on_bar(kfd->adev, kfd->doorbells, inx);
+ inx *= 2;
pr_debug("Get kernel queue doorbell\n"
" doorbell offset == 0x%08X\n"
@@ -176,6 +177,7 @@ void kfd_release_kernel_doorbell(struct kfd_dev *kfd, u32
__iomem *db_addr)
unsigned int inx;
inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr);
+ inx /= 2;
mutex_lock(>doorbell_mutex);
__clear_bit(inx, kfd->doorbell_bitmap);
Re: [PATCHv3] drm/amdkfd: Fix unaligned 64-bit doorbell warning
+ Shashank
On Wed, Sep 6, 2023 at 11:45 AM Mukul Joshi wrote:
>
> This patch fixes the following unaligned 64-bit doorbell
> warning seen when submitting packets on HIQ on GFX v9.4.3
> by making the HIQ doorbell 64-bit aligned.
> The warning is seen when GPU is loaded in any mode other
> than SPX mode.
>
> [ +0.000301] [ cut here ]
> [ +0.03] Unaligned 64-bit doorbell
> [ +0.30] WARNING: /amdkfd/kfd_doorbell.c:339
> write_kernel_doorbell64+0x72/0x80
> [ +0.03] RIP: 0010:write_kernel_doorbell64+0x72/0x80
> [ +0.04] RSP: 0018:c90004287730 EFLAGS: 00010246
> [ +0.05] RAX: RBX: RCX:
>
> [ +0.03] RDX: 0001 RSI: 82837c71 RDI:
>
> [ +0.03] RBP: c90004287748 R08: 0003 R09:
> 0001
> [ +0.02] R10: 001a R11: 88a034008198 R12:
> c900013bd004
> [ +0.03] R13: 0008 R14: c900042877b0 R15:
> 007f
> [ +0.03] FS: 7fa8c7b62000() GS:889f8840()
> knlGS:
> [ +0.04] CS: 0010 DS: ES: CR0: 80050033
> [ +0.03] CR2: 56111c45aaf0 CR3: 0001414f2002 CR4:
> 00770ee0
> [ +0.03] PKRU: 5554
> [ +0.02] Call Trace:
> [ +0.04]
> [ +0.06] kq_submit_packet+0x45/0x50 [amdgpu]
> [ +0.000524] pm_send_set_resources+0x7f/0xc0 [amdgpu]
> [ +0.000500] set_sched_resources+0xe4/0x160 [amdgpu]
> [ +0.000503] start_cpsch+0x1c5/0x2a0 [amdgpu]
> [ +0.000497] kgd2kfd_device_init.cold+0x816/0xb42 [amdgpu]
> [ +0.000743] amdgpu_amdkfd_device_init+0x15f/0x1f0 [amdgpu]
> [ +0.000602] amdgpu_device_init.cold+0x1813/0x2176 [amdgpu]
> [ +0.000684] ? pci_bus_read_config_word+0x4a/0x80
> [ +0.12] ? do_pci_enable_device+0xdc/0x110
> [ +0.08] amdgpu_driver_load_kms+0x1a/0x110 [amdgpu]
> [ +0.000545] amdgpu_pci_probe+0x197/0x400 [amdgpu]
>
> Fixes: cfeaeb3c0ce7 ("drm/amdgpu: use doorbell mgr for kfd kernel doorbells")
> Signed-off-by: Mukul Joshi
> ---
> v1->v2:
> - Update the logic to make it work with both 32 bit
> 64 bit doorbells.
> - Add the Fixed tag
> v2->v3:
> - Revert to the original change to align it with whats done in
> amdgpu_doorbell_index_on_bar.
>
> drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
> b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
> index c2e0b79dcc6d..7b38537c7c99 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
> @@ -162,6 +162,7 @@ void __iomem *kfd_get_kernel_doorbell(struct kfd_dev *kfd,
> return NULL;
>
> *doorbell_off = amdgpu_doorbell_index_on_bar(kfd->adev,
> kfd->doorbells, inx);
> + inx *= 2;
>
> pr_debug("Get kernel queue doorbell\n"
> " doorbell offset == 0x%08X\n"
> @@ -176,6 +177,7 @@ void kfd_release_kernel_doorbell(struct kfd_dev *kfd, u32
> __iomem *db_addr)
> unsigned int inx;
>
> inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr);
> + inx /= 2;
>
> mutex_lock(>doorbell_mutex);
> __clear_bit(inx, kfd->doorbell_bitmap);
> --
> 2.35.1
>
[PATCHv3] drm/amdkfd: Fix unaligned 64-bit doorbell warning
This patch fixes the following unaligned 64-bit doorbell
warning seen when submitting packets on HIQ on GFX v9.4.3
by making the HIQ doorbell 64-bit aligned.
The warning is seen when GPU is loaded in any mode other
than SPX mode.
[ +0.000301] [ cut here ]
[ +0.03] Unaligned 64-bit doorbell
[ +0.30] WARNING: /amdkfd/kfd_doorbell.c:339
write_kernel_doorbell64+0x72/0x80
[ +0.03] RIP: 0010:write_kernel_doorbell64+0x72/0x80
[ +0.04] RSP: 0018:c90004287730 EFLAGS: 00010246
[ +0.05] RAX: RBX: RCX:
[ +0.03] RDX: 0001 RSI: 82837c71 RDI:
[ +0.03] RBP: c90004287748 R08: 0003 R09: 0001
[ +0.02] R10: 001a R11: 88a034008198 R12: c900013bd004
[ +0.03] R13: 0008 R14: c900042877b0 R15: 007f
[ +0.03] FS: 7fa8c7b62000() GS:889f8840()
knlGS:
[ +0.04] CS: 0010 DS: ES: CR0: 80050033
[ +0.03] CR2: 56111c45aaf0 CR3: 0001414f2002 CR4: 00770ee0
[ +0.03] PKRU: 5554
[ +0.02] Call Trace:
[ +0.04]
[ +0.06] kq_submit_packet+0x45/0x50 [amdgpu]
[ +0.000524] pm_send_set_resources+0x7f/0xc0 [amdgpu]
[ +0.000500] set_sched_resources+0xe4/0x160 [amdgpu]
[ +0.000503] start_cpsch+0x1c5/0x2a0 [amdgpu]
[ +0.000497] kgd2kfd_device_init.cold+0x816/0xb42 [amdgpu]
[ +0.000743] amdgpu_amdkfd_device_init+0x15f/0x1f0 [amdgpu]
[ +0.000602] amdgpu_device_init.cold+0x1813/0x2176 [amdgpu]
[ +0.000684] ? pci_bus_read_config_word+0x4a/0x80
[ +0.12] ? do_pci_enable_device+0xdc/0x110
[ +0.08] amdgpu_driver_load_kms+0x1a/0x110 [amdgpu]
[ +0.000545] amdgpu_pci_probe+0x197/0x400 [amdgpu]
Fixes: cfeaeb3c0ce7 ("drm/amdgpu: use doorbell mgr for kfd kernel doorbells")
Signed-off-by: Mukul Joshi
---
v1->v2:
- Update the logic to make it work with both 32 bit
64 bit doorbells.
- Add the Fixed tag
v2->v3:
- Revert to the original change to align it with whats done in
amdgpu_doorbell_index_on_bar.
drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
index c2e0b79dcc6d..7b38537c7c99 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
@@ -162,6 +162,7 @@ void __iomem *kfd_get_kernel_doorbell(struct kfd_dev *kfd,
return NULL;
*doorbell_off = amdgpu_doorbell_index_on_bar(kfd->adev, kfd->doorbells,
inx);
+ inx *= 2;
pr_debug("Get kernel queue doorbell\n"
" doorbell offset == 0x%08X\n"
@@ -176,6 +177,7 @@ void kfd_release_kernel_doorbell(struct kfd_dev *kfd, u32
__iomem *db_addr)
unsigned int inx;
inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr);
+ inx /= 2;
mutex_lock(>doorbell_mutex);
__clear_bit(inx, kfd->doorbell_bitmap);
--
2.35.1
