RE: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov
[AMD Official Use Only - Internal Distribution Only] Reviewed-by: Monk Liu Thanks -- Monk Liu | Cloud-GPU Core team -- -Original Message- From: Deng, Emily Sent: Thursday, April 1, 2021 2:04 PM To: Liu, Monk Cc: amd-gfx@lists.freedesktop.org Subject: RE: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov [AMD Official Use Only - Internal Distribution Only] Hi Monk, Could you help to review this patch? Best wishes Emily Deng >-Original Message- >From: Deng, Emily >Sent: Wednesday, March 31, 2021 5:02 PM >To: Deng, Emily ; amd-gfx@lists.freedesktop.org >Subject: RE: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov > >[AMD Official Use Only - Internal Distribution Only] > >Ping .. > >>-Original Message- >>From: Emily Deng >>Sent: Tuesday, March 30, 2021 5:43 PM >>To: amd-gfx@lists.freedesktop.org >>Cc: Deng, Emily >>Subject: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov >> >>From: "Emily.Deng" >> >>For vf assigned to guest VM, after FLR, the msix table will be reset. >>As the flr is done on host driver. The qemu and vfio driver don't know >>this, and the msix is still enable from qemu and vfio driver side. >>So if want to re-setup the msix table, first need to disable and >>re-enable the msix from guest VM side or the qemu will do nothing as >>it thought the msix is already enabled. >> >>v2: >>Change name with amdgpu_irq prefix, remove #ifdef. >> >>Signed-off-by: Emily.Deng >>--- >> drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 14 ++ >> 1 file changed, 14 insertions(+) >> >>diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>index 03412543427a..3045f52e613d 100644 >>--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>@@ -277,6 +277,17 @@ static bool amdgpu_msi_ok(struct amdgpu_device >>*adev) >> return true; >> } >> >>+static void amdgpu_irq_restore_msix(struct amdgpu_device *adev) { >>+u16 ctrl; >>+ >>+pci_read_config_word(adev->pdev, adev->pdev->msix_cap + >>PCI_MSIX_FLAGS, ); >>+ctrl &= ~PCI_MSIX_FLAGS_ENABLE; >>+pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >>PCI_MSIX_FLAGS, ctrl); >>+ctrl |= PCI_MSIX_FLAGS_ENABLE; >>+pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >>+PCI_MSIX_FLAGS, ctrl); } >>+ >> /** >> * amdgpu_irq_init - initialize interrupt handling >> * >>@@ -558,6 +569,9 @@ void amdgpu_irq_gpu_reset_resume_helper(struct >>amdgpu_device *adev) { >> int i, j, k; >> >>+if (amdgpu_sriov_vf(adev)) >>+amdgpu_irq_restore_msix(adev); >>+ >> for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if >>(!adev->irq.client[i].sources) continue; >>-- >>2.25.1 > ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov
Acked-by: Nirmoy Das On 3/30/21 11:42 AM, Emily Deng wrote: From: "Emily.Deng" For vf assigned to guest VM, after FLR, the msix table will be reset. As the flr is done on host driver. The qemu and vfio driver don't know this, and the msix is still enable from qemu and vfio driver side. So if want to re-setup the msix table, first need to disable and re-enable the msix from guest VM side or the qemu will do nothing as it thought the msix is already enabled. v2: Change name with amdgpu_irq prefix, remove #ifdef. Signed-off-by: Emily.Deng --- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 03412543427a..3045f52e613d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -277,6 +277,17 @@ static bool amdgpu_msi_ok(struct amdgpu_device *adev) return true; } +static void amdgpu_irq_restore_msix(struct amdgpu_device *adev) +{ + u16 ctrl; + + pci_read_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, ); + ctrl &= ~PCI_MSIX_FLAGS_ENABLE; + pci_write_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, ctrl); + ctrl |= PCI_MSIX_FLAGS_ENABLE; + pci_write_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, ctrl); +} + /** * amdgpu_irq_init - initialize interrupt handling * @@ -558,6 +569,9 @@ void amdgpu_irq_gpu_reset_resume_helper(struct amdgpu_device *adev) { int i, j, k; + if (amdgpu_sriov_vf(adev)) + amdgpu_irq_restore_msix(adev); + for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
RE: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov
[AMD Official Use Only - Internal Distribution Only] Hi Monk, Could you help to review this patch? Best wishes Emily Deng >-Original Message- >From: Deng, Emily >Sent: Wednesday, March 31, 2021 5:02 PM >To: Deng, Emily ; amd-gfx@lists.freedesktop.org >Subject: RE: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov > >[AMD Official Use Only - Internal Distribution Only] > >Ping .. > >>-Original Message- >>From: Emily Deng >>Sent: Tuesday, March 30, 2021 5:43 PM >>To: amd-gfx@lists.freedesktop.org >>Cc: Deng, Emily >>Subject: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov >> >>From: "Emily.Deng" >> >>For vf assigned to guest VM, after FLR, the msix table will be reset. >>As the flr is done on host driver. The qemu and vfio driver don't know >>this, and the msix is still enable from qemu and vfio driver side. >>So if want to re-setup the msix table, first need to disable and >>re-enable the msix from guest VM side or the qemu will do nothing as it >>thought the msix is already enabled. >> >>v2: >>Change name with amdgpu_irq prefix, remove #ifdef. >> >>Signed-off-by: Emily.Deng >>--- >> drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 14 ++ >> 1 file changed, 14 insertions(+) >> >>diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>index 03412543427a..3045f52e613d 100644 >>--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>@@ -277,6 +277,17 @@ static bool amdgpu_msi_ok(struct amdgpu_device >>*adev) >> return true; >> } >> >>+static void amdgpu_irq_restore_msix(struct amdgpu_device *adev) { >>+u16 ctrl; >>+ >>+pci_read_config_word(adev->pdev, adev->pdev->msix_cap + >>PCI_MSIX_FLAGS, ); >>+ctrl &= ~PCI_MSIX_FLAGS_ENABLE; >>+pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >>PCI_MSIX_FLAGS, ctrl); >>+ctrl |= PCI_MSIX_FLAGS_ENABLE; >>+pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >>+PCI_MSIX_FLAGS, ctrl); } >>+ >> /** >> * amdgpu_irq_init - initialize interrupt handling >> * >>@@ -558,6 +569,9 @@ void amdgpu_irq_gpu_reset_resume_helper(struct >>amdgpu_device *adev) { >> int i, j, k; >> >>+if (amdgpu_sriov_vf(adev)) >>+amdgpu_irq_restore_msix(adev); >>+ >> for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if >>(!adev->irq.client[i].sources) continue; >>-- >>2.25.1 > ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
RE: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov
[AMD Official Use Only - Internal Distribution Only] Ping .. >-Original Message- >From: Emily Deng >Sent: Tuesday, March 30, 2021 5:43 PM >To: amd-gfx@lists.freedesktop.org >Cc: Deng, Emily >Subject: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov > >From: "Emily.Deng" > >For vf assigned to guest VM, after FLR, the msix table will be reset. >As the flr is done on host driver. The qemu and vfio driver don't know this, >and the msix is still enable from qemu and vfio driver side. >So if want to re-setup the msix table, first need to disable and re-enable the >msix from guest VM side or the qemu will do nothing as it thought the msix is >already enabled. > >v2: >Change name with amdgpu_irq prefix, remove #ifdef. > >Signed-off-by: Emily.Deng >--- > drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 14 ++ > 1 file changed, 14 insertions(+) > >diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >index 03412543427a..3045f52e613d 100644 >--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >@@ -277,6 +277,17 @@ static bool amdgpu_msi_ok(struct amdgpu_device >*adev) > return true; > } > >+static void amdgpu_irq_restore_msix(struct amdgpu_device *adev) { >+u16 ctrl; >+ >+pci_read_config_word(adev->pdev, adev->pdev->msix_cap + >PCI_MSIX_FLAGS, ); >+ctrl &= ~PCI_MSIX_FLAGS_ENABLE; >+pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >PCI_MSIX_FLAGS, ctrl); >+ctrl |= PCI_MSIX_FLAGS_ENABLE; >+pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >+PCI_MSIX_FLAGS, ctrl); } >+ > /** > * amdgpu_irq_init - initialize interrupt handling > * >@@ -558,6 +569,9 @@ void amdgpu_irq_gpu_reset_resume_helper(struct >amdgpu_device *adev) { > int i, j, k; > >+if (amdgpu_sriov_vf(adev)) >+amdgpu_irq_restore_msix(adev); >+ > for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { > if (!adev->irq.client[i].sources) > continue; >-- >2.25.1 ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
RE: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov
[AMD Official Use Only - Internal Distribution Only] Ping .. >-Original Message- >From: Emily Deng >Sent: Tuesday, March 30, 2021 5:43 PM >To: amd-gfx@lists.freedesktop.org >Cc: Deng, Emily >Subject: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov > >From: "Emily.Deng" > >For vf assigned to guest VM, after FLR, the msix table will be reset. >As the flr is done on host driver. The qemu and vfio driver don't know this, >and the msix is still enable from qemu and vfio driver side. >So if want to re-setup the msix table, first need to disable and re-enable the >msix from guest VM side or the qemu will do nothing as it thought the msix is >already enabled. > >v2: >Change name with amdgpu_irq prefix, remove #ifdef. > >Signed-off-by: Emily.Deng >--- > drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 14 ++ > 1 file changed, 14 insertions(+) > >diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >index 03412543427a..3045f52e613d 100644 >--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >@@ -277,6 +277,17 @@ static bool amdgpu_msi_ok(struct amdgpu_device >*adev) > return true; > } > >+static void amdgpu_irq_restore_msix(struct amdgpu_device *adev) { >+u16 ctrl; >+ >+pci_read_config_word(adev->pdev, adev->pdev->msix_cap + >PCI_MSIX_FLAGS, ); >+ctrl &= ~PCI_MSIX_FLAGS_ENABLE; >+pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >PCI_MSIX_FLAGS, ctrl); >+ctrl |= PCI_MSIX_FLAGS_ENABLE; >+pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >+PCI_MSIX_FLAGS, ctrl); } >+ > /** > * amdgpu_irq_init - initialize interrupt handling > * >@@ -558,6 +569,9 @@ void amdgpu_irq_gpu_reset_resume_helper(struct >amdgpu_device *adev) { > int i, j, k; > >+if (amdgpu_sriov_vf(adev)) >+amdgpu_irq_restore_msix(adev); >+ > for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { > if (!adev->irq.client[i].sources) > continue; >-- >2.25.1 ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
RE: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov
[AMD Official Use Only - Internal Distribution Only] >-Original Message- >From: Das, Nirmoy >Sent: Tuesday, March 30, 2021 5:34 PM >To: Deng, Emily ; Das, Nirmoy >; amd-gfx@lists.freedesktop.org >Subject: Re: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov > > >On 3/30/21 11:29 AM, Deng, Emily wrote: >> [AMD Official Use Only - Internal Distribution Only] >> >>> -Original Message- >>> From: Das, Nirmoy >>> Sent: Tuesday, March 30, 2021 4:59 PM >>> To: Deng, Emily ; amd-gfx@lists.freedesktop.org >>> Subject: Re: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov >>> >>> >>> On 3/30/21 10:14 AM, Emily Deng wrote: >>>> From: "Emily.Deng" >>>> >>>> After FLR, the msix will be cleared, so need to toggle it for sriov. >>>> >>>> v2: >>>> Change name with amdgpu_irq prefix, remove #ifdef. >>>> >>>> Signed-off-by: Emily.Deng >>>> --- >>>>drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 14 ++ >>>>1 file changed, 14 insertions(+) >>>> >>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>>> index 03412543427a..3045f52e613d 100644 >>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >>>> @@ -277,6 +277,17 @@ static bool amdgpu_msi_ok(struct >amdgpu_device >>> *adev) >>>>return true; >>>>} >>>> >>>> +static void amdgpu_irq_restore_msix(struct amdgpu_device *adev) { >>>> +u16 ctrl; >>>> + >>>> +pci_read_config_word(adev->pdev, adev->pdev->msix_cap + >>> PCI_MSIX_FLAGS, ); >>>> +ctrl &= ~PCI_MSIX_FLAGS_ENABLE; >>>> +pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >>> PCI_MSIX_FLAGS, ctrl); >>>> +ctrl |= PCI_MSIX_FLAGS_ENABLE; >>>> +pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >>>> +PCI_MSIX_FLAGS, ctrl); >>> >>> Why write 1st clear and then set the msix flag if we know that msix >>> is already cleared >> For vf assigned to guest VM, after FLR, the msix table will be reset. >> As the flr is done on host driver. The qemu and vfio driver don't know >> this, and the msix is still enable from qemu and vfio driver side. So if >> want to >re-setup the msix table, first need to disable and re-enable the msix from >guest VM side or the qemu will do nothing as it thought the msix is already >enabled. > > >Thanks for the detailed explanation, Emily. Please add a comment so that we >know/remember why we are doing this. Ok, will do this. Thanks. > > >Nirmoy > > >>> >>> >>>> +} >>>> + >>>>/** >>>> * amdgpu_irq_init - initialize interrupt handling >>>> * >>>> @@ -558,6 +569,9 @@ void >amdgpu_irq_gpu_reset_resume_helper(struct >>> amdgpu_device *adev) >>>>{ >>>>int i, j, k; >>>> >>>> +if (amdgpu_sriov_vf(adev)) >>>> +amdgpu_irq_restore_msix(adev); >>> >>> Is it possible to load amdgpu on guest without msix ? If so then we need >>> to probe if msix is enabled. It is decided by host driver, not guest driver. >>> >>> >>> Nirmoy >>> >>> >>>> + >>>>for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { >>>>if (!adev->irq.client[i].sources) >>>>continue; ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov
On 3/30/21 11:29 AM, Deng, Emily wrote: [AMD Official Use Only - Internal Distribution Only] -Original Message- From: Das, Nirmoy Sent: Tuesday, March 30, 2021 4:59 PM To: Deng, Emily ; amd-gfx@lists.freedesktop.org Subject: Re: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov On 3/30/21 10:14 AM, Emily Deng wrote: From: "Emily.Deng" After FLR, the msix will be cleared, so need to toggle it for sriov. v2: Change name with amdgpu_irq prefix, remove #ifdef. Signed-off-by: Emily.Deng --- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 03412543427a..3045f52e613d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -277,6 +277,17 @@ static bool amdgpu_msi_ok(struct amdgpu_device *adev) return true; } +static void amdgpu_irq_restore_msix(struct amdgpu_device *adev) { +u16 ctrl; + +pci_read_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, ); +ctrl &= ~PCI_MSIX_FLAGS_ENABLE; +pci_write_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, ctrl); +ctrl |= PCI_MSIX_FLAGS_ENABLE; +pci_write_config_word(adev->pdev, adev->pdev->msix_cap + +PCI_MSIX_FLAGS, ctrl); Why write 1st clear and then set the msix flag if we know that msix is already cleared For vf assigned to guest VM, after FLR, the msix table will be reset. As the flr is done on host driver. The qemu and vfio driver don't know this, and the msix is still enable from qemu and vfio driver side. So if want to re-setup the msix table, first need to disable and re-enable the msix from guest VM side or the qemu will do nothing as it thought the msix is already enabled. Thanks for the detailed explanation, Emily. Please add a comment so that we know/remember why we are doing this. Nirmoy +} + /** * amdgpu_irq_init - initialize interrupt handling * @@ -558,6 +569,9 @@ void amdgpu_irq_gpu_reset_resume_helper(struct amdgpu_device *adev) { int i, j, k; +if (amdgpu_sriov_vf(adev)) +amdgpu_irq_restore_msix(adev); Is it possible to load amdgpu on guest without msix ? If so then we need to probe if msix is enabled. Nirmoy + for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
RE: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov
[AMD Official Use Only - Internal Distribution Only] >-Original Message- >From: Das, Nirmoy >Sent: Tuesday, March 30, 2021 4:59 PM >To: Deng, Emily ; amd-gfx@lists.freedesktop.org >Subject: Re: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov > > >On 3/30/21 10:14 AM, Emily Deng wrote: >> From: "Emily.Deng" >> >> After FLR, the msix will be cleared, so need to toggle it for sriov. >> >> v2: >> Change name with amdgpu_irq prefix, remove #ifdef. >> >> Signed-off-by: Emily.Deng >> --- >> drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 14 ++ >> 1 file changed, 14 insertions(+) >> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >> b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >> index 03412543427a..3045f52e613d 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c >> @@ -277,6 +277,17 @@ static bool amdgpu_msi_ok(struct amdgpu_device >*adev) >> return true; >> } >> >> +static void amdgpu_irq_restore_msix(struct amdgpu_device *adev) { >> +u16 ctrl; >> + >> +pci_read_config_word(adev->pdev, adev->pdev->msix_cap + >PCI_MSIX_FLAGS, ); >> +ctrl &= ~PCI_MSIX_FLAGS_ENABLE; >> +pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >PCI_MSIX_FLAGS, ctrl); >> +ctrl |= PCI_MSIX_FLAGS_ENABLE; >> +pci_write_config_word(adev->pdev, adev->pdev->msix_cap + >> +PCI_MSIX_FLAGS, ctrl); > > >Why write 1st clear and then set the msix flag if we know that msix is already >cleared For vf assigned to guest VM, after FLR, the msix table will be reset. As the flr is done on host driver. The qemu and vfio driver don't know this, and the msix is still enable from qemu and vfio driver side. So if want to re-setup the msix table, first need to disable and re-enable the msix from guest VM side or the qemu will do nothing as it thought the msix is already enabled. > > > >> +} >> + >> /** >>* amdgpu_irq_init - initialize interrupt handling >>* >> @@ -558,6 +569,9 @@ void amdgpu_irq_gpu_reset_resume_helper(struct >amdgpu_device *adev) >> { >> int i, j, k; >> >> +if (amdgpu_sriov_vf(adev)) >> +amdgpu_irq_restore_msix(adev); > > >Is it possible to load amdgpu on guest without msix ? If so then we need >to probe if msix is enabled. > > >Nirmoy > > >> + >> for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { >> if (!adev->irq.client[i].sources) >> continue; ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [PATCH] drm/amdgpu: Toggle msix after FLR for sriov
On 3/30/21 10:14 AM, Emily Deng wrote: From: "Emily.Deng" After FLR, the msix will be cleared, so need to toggle it for sriov. v2: Change name with amdgpu_irq prefix, remove #ifdef. Signed-off-by: Emily.Deng --- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 03412543427a..3045f52e613d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -277,6 +277,17 @@ static bool amdgpu_msi_ok(struct amdgpu_device *adev) return true; } +static void amdgpu_irq_restore_msix(struct amdgpu_device *adev) +{ + u16 ctrl; + + pci_read_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, ); + ctrl &= ~PCI_MSIX_FLAGS_ENABLE; + pci_write_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, ctrl); + ctrl |= PCI_MSIX_FLAGS_ENABLE; + pci_write_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, ctrl); Why write 1st clear and then set the msix flag if we know that msix is already cleared. +} + /** * amdgpu_irq_init - initialize interrupt handling * @@ -558,6 +569,9 @@ void amdgpu_irq_gpu_reset_resume_helper(struct amdgpu_device *adev) { int i, j, k; + if (amdgpu_sriov_vf(adev)) + amdgpu_irq_restore_msix(adev); Is it possible to load amdgpu on guest without msix ? If so then we need to probe if msix is enabled. Nirmoy + for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx