AMD General Cc @Feng, Kenneth,
Best Regards, Kevin > -----Original Message----- > From: Wang, Kevin > Sent: Wednesday, September 30, 2026 2:09 PM > To: 'Lazar, Lijo' <[email protected]>; Zhang, Yifan > <[email protected]>; Deucher, Alexander > <[email protected]>; Koenig, Christian > <[email protected]>; [email protected] > Cc: Yuan, Perry <[email protected]> > Subject: RE: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 > reset on gfx12 > > > On a second look, the patch doesn't seem right. Reset sequence is > > supposed to be complete by the time FW responds back to the mode-1 > reset message. > > During reset sequence, we need Mem space to be enabled to poll success > > status of the message and that's most volatile window during the whole > > sequence. > > That is not right. When the OS driver polls and sees the response register > become "1", it does **NOT** mean the mode1 sequence is completed. > The PMFW will continue forward the message to PSP and wait for PMFW > ucode reload to finish at a 2nd-half stage. > > Best Regards, > Kevin > > > -----Original Message----- > > From: amd-gfx <[email protected]> On Behalf Of > > Lazar, Lijo > > Sent: Wednesday, September 30, 2026 1:41 PM > > To: Zhang, Yifan <[email protected]>; Deucher, Alexander > > <[email protected]>; Koenig, Christian > > <[email protected]>; [email protected] > > Cc: Yuan, Perry <[email protected]> > > Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 > > reset on gfx12 > > > > On a second look, the patch doesn't seem right. Reset sequence is > > supposed to be complete by the time FW responds back to the mode-1 > reset message. > > During reset sequence, we need Mem space to be enabled to poll success > > status of the message and that's most volatile window during the whole > > sequence. > > > > Disabling the memory space after reset sequence is complete looks to > > be the wrong diagnosis of the issue. Access during reinit sequence > > after reset is safe and shouldn't cause any issue in general. > > > > That said, we should try to resolve the issue without touching any PCI > > config settings. PCI config space is open for others as well. > > > > Thanks, > > Lijo > > > > On 30-Sep-26 10:57 AM, Zhang, Yifan wrote: > > > AMD General > > > > > > How about disable UE during mode1 reset ? E.g. > > > > > > Before: > > > pcie_capability_read_word(pdev, PCI_EXP_DEVCTL, &saved_devctl); > > > pcie_capability_write_word(pdev, PCI_EXP_DEVCTL, > > > saved_devctl & ~PCI_EXP_DEVCTL_URRE); > > > pci_read_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK, > > > &saved_uemsk); > > > pci_write_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK, > > > saved_uemsk | > > > PCI_ERR_UNC_UNSUP); > > > > > > > > > After: > > > pci_write_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK, > > > saved_uemsk); > > > pcie_capability_write_word(pdev, PCI_EXP_DEVCTL, > > > saved_devctl); > > > > > > -----Original Message----- > > > From: Lazar, Lijo <[email protected]> > > > Sent: Wednesday, September 30, 2026 11:37 AM > > > To: Zhang, Yifan <[email protected]>; Deucher, Alexander > > > <[email protected]>; Koenig, Christian > > > <[email protected]>; [email protected] > > > Cc: Yuan, Perry <[email protected]> > > > Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > > > mode1 reset on gfx12 > > > > > > > > > This is the behaviour of this bit - > > > > > > Memory Space Enable - Controls a Function's response to > > > Memory Space accesses. When this bit is Clear, all received Memory > > > Space > > accesses are caused to be handled as "Unsupported Requests" > > > > > > On AER enabled systems, depending on how the platform is configured > > > to > > classify the severity of UR errors, that could result in System error. > > > > > > On other hand, if you are sure that there is no such access during > > > mode-1 reset, we don't need this patch in the first place. > > > > > > Thanks, > > > Lijo > > > > > > > > > On 30-Sep-26 8:53 AM, Zhang, Yifan wrote: > > >> AMD General > > >> > > >> Hi Lijo, > > >> > > >> Why does clearing PCI_COMMAND_MEMORY cause AER errors on data > > center GPUs? Are their PCIe BARs still being accessed during a mode1 reset? > > >> > > >> Best Regards, > > >> Yifan > > >> > > >> -----Original Message----- > > >> From: Lazar, Lijo <[email protected]> > > >> Sent: Wednesday, September 30, 2026 11:06 AM > > >> To: Zhang, Yifan <[email protected]>; Deucher, Alexander > > >> <[email protected]>; Koenig, Christian > > >> <[email protected]>; [email protected] > > >> Cc: Yuan, Perry <[email protected]> > > >> Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > > >> mode1 reset on gfx12 > > >> > > >> This doesn't work for all (at least on some datacenter GPUs). With > > >> AER > > enabled, this will result in AER errors. Depending on the platform > > config, some may classify this access as fatal. > > >> > > >> Thanks, > > >> Lijo > > >> > > >> On 30-Sep-26 8:25 AM, Zhang, Yifan wrote: > > >>> Public > > >>> > > >>> > > >>> Hi Alex, > > >>> > > >>> I think it is reasonable to apply the patch to all ASICs. I will > > >>> sync with the hardware team about this change; so far we have only > > >>> tested it on Navi31. > > >>> > > >>> How about adding a common helper and calling it from ASIC-specific > > >>> code, starting with Navi31? For example: > > >>> > > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > > >>> b/drivers/gpu/ drm/amd/amdgpu/amdgpu_device.c > > >>> > > >>> index b690a07e31ab..075de681ab86 100644 > > >>> > > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > > >>> > > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > > >>> > > >>> @@ -4931,6 +4931,28 @@ bool > > amdgpu_device_should_recover_gpu(struct > > >>> amdgpu_device *adev) > > >>> > > >>> return false; > > >>> > > >>> } > > >>> > > >>> +void amdgpu_device_mode1_reset_wait(struct amdgpu_device *adev, > > >>> > > >>> + unsigned int delay_ms) > > >>> > > >>> +{ > > >>> > > >>> + u16 saved_cmd, cmd; > > >>> > > >>> + > > >>> > > >>> + pci_read_config_word(adev->pdev, PCI_COMMAND, > &saved_cmd); > > >>> > > >>> + cmd = saved_cmd & ~PCI_COMMAND_MEMORY; > > >>> > > >>> + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd); > > >>> > > >>> + > > >>> > > >>> + msleep(delay_ms); > > >>> > > >>> + > > >>> > > >>> + pci_write_config_word(adev->pdev, PCI_COMMAND, saved_cmd); > > >>> > > >>> +} > > >>> > > >>> + > > >>> > > >>> diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > > >>> b/drivers/ gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > > >>> > > >>> index b250c5e41e45..356e09b7eef9 100644 > > >>> > > >>> --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > > >>> > > >>> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > > >>> > > >>> @@ -2242,7 +2242,8 @@ int smu_v13_0_mode1_reset(struct > > smu_context > > >>> *smu) > > >>> > > >>> ret = smu_cmn_send_smc_msg(smu, SMU_MSG_Mode1Reset, > > >>> NULL); > > >>> > > >>> if (!ret) > > >>> > > >>> - msleep(SMU13_MODE1_RESET_WAIT_TIME_IN_MS); > > >>> > > >>> + amdgpu_device_mode1_reset_wait(smu->adev, > > >>> > > >>> + > > >>> SMU13_MODE1_RESET_WAIT_TIME_IN_MS); > > >>> > > >>> return ret; > > >>> > > >>> } > > >>> > > >>> Best Regards, > > >>> > > >>> Yifan > > >>> > > >>> *From:*Deucher, Alexander <[email protected]> > > >>> *Sent:* Wednesday, September 30, 2026 2:59 AM > > >>> *To:* Zhang, Yifan <[email protected]>; Koenig, Christian > > >>> <[email protected]>; [email protected] > > >>> *Cc:* Yuan, Perry <[email protected]> > > >>> *Subject:* Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > > >>> mode1 reset on gfx12 > > >>> > > >>> Public > > >>> > > >>> I think this should be asic agnostic. Any asic doing mode1 reset > > >>> should do this? > > >>> > > >>> Alex > > >>> > > >>> ------------------------------------------------------------------ > > >>> -- > > >>> - > > >>> - > > >>> -- > > >>> > > >>> *From:*Zhang, Yifan <[email protected] > > >>> <mailto:[email protected]>> > > >>> *Sent:* Tuesday, September 29, 2026 5:36 AM > > >>> *To:* Koenig, Christian <[email protected] > > >>> <mailto:[email protected]>>; [email protected] > > >>> <mailto:[email protected]> > > >>> <[email protected] > > >>> <mailto:[email protected]>> > > >>> *Cc:* Deucher, Alexander <[email protected] > > >>> <mailto:[email protected]>>; Yuan, Perry > > <[email protected] > > >>> <mailto:[email protected]>> > > >>> *Subject:* RE: [PATCH] amdgpu/pm: block GPU PCI bar access during > > >>> mode1 reset on gfx12 > > >>> > > >>> Public > > >>> > > >>> Hi Christian, > > >>> > > >>> The no_hw_access flag was first set in commit > > >>> 0de604d0357d0d22cbf03af1077d174b641707b6. This was intended to > fix > > >>> the Mode 1 reset hang, but it proved insufficient. > > >>> I think the flag was set in the SMU code so that it would apply > > >>> only to > > >>> SMU13/14 ASICs, without affecting the Mode 1 reset process on > > >>> other > > ASICs. > > >>> I’m not sure whether we can drop the no_hw_access flag after > > >>> disabling PCI_COMMAND_MEMORY. The PCI hang occurs intermittently, > > so > > >>> we may need another round of stress testing to confirm that the > > >>> flag is no longer needed. Should we remove no_hw_access in a > > >>> separate > > patch? > > >>> > > >>> Best Regards, > > >>> Yifan > > >>> > > >>> -----Original Message----- > > >>> From: Koenig, Christian <[email protected] > > >>> <mailto:[email protected]>> > > >>> Sent: Tuesday, September 29, 2026 5:06 PM > > >>> To: Zhang, Yifan <[email protected] > > >>> <mailto:[email protected]>>; [email protected] > > >>> <mailto:[email protected]> > > >>> Cc: Deucher, Alexander <[email protected] > > >>> <mailto:[email protected]>>; Yuan, Perry > > <[email protected] > > >>> <mailto:[email protected]>> > > >>> Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > > >>> mode1 reset on gfx12 > > >>> > > >>> On 9/29/26 04:25, Yifan Zhang wrote: > > >>>> block GPU PCI bar access during mode1 reset on gfx12, to resolve > > >>>> the random mode1 reset hang issue. > > >>> > > >>> Interesting idea, that is much simpler than trying to mess with > > >>> CPU page tables in the GPU reset path. > > >>> > > >>> The only downside I can see is that we now drop VRAM and doorbell > > >>> writes, but I think that won't matter since VRAM is lost by a > > >>> mode1 reset anyway. > > >>> > > >>> MMIO writes are dropped as well which means that we can also drop > > >>> the no_hw_access flag. > > >>> > > >>> And BTW the SMU code should absolutely *NOT* touch that > > no_hw_access > > >>> flag in any way. Who added that there? > > >>> > > >>> Regards, > > >>> Christian. > > >>> > > >>>> > > >>>> Signed-off-by: Yifan Zhang <[email protected] > > >>>> <mailto:[email protected]>> > > >>>> --- > > >>>> .../drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c | 29 > > >>>> ++++++++++++------- > > >>>> 1 file changed, 19 insertions(+), 10 deletions(-) > > >>>> > > >>>> diff --git > > a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > > >>>> b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > > >>>> index 072d9c4e202d..25b9085ead71 100644 > > >>>> --- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > > >>>> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > > >>>> @@ -2078,21 +2078,30 @@ static int > > >>>> smu_v14_0_2_set_df_cstate(struct smu_context *smu, > > >>>> > > >>>> static int smu_v14_0_2_mode1_reset(struct smu_context *smu) { > > >>>> + struct amdgpu_device *adev = smu->adev; > > >>>> + u16 saved_cmd, cmd; > > >>>> int ret = 0; > > >>>> > > >>>> ret = smu_cmn_send_debug_smc_msg(smu, > > DEBUGSMC_MSG_Mode1Reset); > > >>>> - if (!ret) { > > >>>> - if (amdgpu_emu_mode == 1) { > > >>>> - msleep(50000); > > >>>> - } else { > > >>>> - /* disable mmio access while doing mode 1 > > >>>> reset*/ > > >>>> - smu->adev->no_hw_access = true; > > >>>> - /* ensure no_hw_access is globally visible > > >>>> before any MMIO */ > > >>>> - smp_mb(); > > >>>> - msleep(1000); > > >>>> - } > > >>>> + if (ret) > > >>>> + return ret; > > >>>> + > > >>>> + pci_read_config_word(adev->pdev, PCI_COMMAND, > &saved_cmd); > > >>>> + cmd = saved_cmd & ~PCI_COMMAND_MEMORY; > > >>>> + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd); > > >>>> + > > >>>> + if (amdgpu_emu_mode == 1) { > > >>>> + msleep(50000); > > >>>> + } else { > > >>>> + /* disable mmio access while doing mode 1 reset*/ > > >>>> + smu->adev->no_hw_access = true; > > >>>> + /* ensure no_hw_access is globally visible before > > >>>> +any MMIO */ > > >>>> + smp_mb(); > > >>>> + msleep(1000); > > >>>> } > > >>>> > > >>>> + pci_write_config_word(adev->pdev, PCI_COMMAND, saved_cmd); > > >>>> + > > >>>> return ret; > > >>>> } > > >>>> > > >>> > > >> > > >
