AMD General Yes. In addition, the polling happens inside smu_cmn_send_smc_msg(), which runs before PCI_COMMAND_MEMORY is cleared. During the mode1 reset process there should be no traffic to the PCIe BARs; otherwise, the PCI bus may hang.
Also, every PCI config write in the patch targets only the GPU device itself (pci_write_config_dword is per device), so no other PCI devices are affected. -----Original Message----- From: Wang, Kevin <[email protected]> Sent: Wednesday, September 30, 2026 2:09 PM To: Lazar, Lijo <[email protected]>; Zhang, Yifan <[email protected]>; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; [email protected] Cc: Yuan, Perry <[email protected]> Subject: RE: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 reset on gfx12 AMD General > On a second look, the patch doesn't seem right. Reset sequence is > supposed to be complete by the time FW responds back to the mode-1 reset > message. > During reset sequence, we need Mem space to be enabled to poll success > status of the message and that's most volatile window during the whole > sequence. That is not right. When the OS driver polls and sees the response register become "1", it does **NOT** mean the mode1 sequence is completed. The PMFW will continue forward the message to PSP and wait for PMFW ucode reload to finish at a 2nd-half stage. Best Regards, Kevin > -----Original Message----- > From: amd-gfx <[email protected]> On Behalf Of > Lazar, Lijo > Sent: Wednesday, September 30, 2026 1:41 PM > To: Zhang, Yifan <[email protected]>; Deucher, Alexander > <[email protected]>; Koenig, Christian > <[email protected]>; [email protected] > Cc: Yuan, Perry <[email protected]> > Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 > reset on gfx12 > > On a second look, the patch doesn't seem right. Reset sequence is > supposed to be complete by the time FW responds back to the mode-1 reset > message. > During reset sequence, we need Mem space to be enabled to poll success > status of the message and that's most volatile window during the whole > sequence. > > Disabling the memory space after reset sequence is complete looks to > be the wrong diagnosis of the issue. Access during reinit sequence > after reset is safe and shouldn't cause any issue in general. > > That said, we should try to resolve the issue without touching any PCI > config settings. PCI config space is open for others as well. > > Thanks, > Lijo > > On 30-Sep-26 10:57 AM, Zhang, Yifan wrote: > > AMD General > > > > How about disable UE during mode1 reset ? E.g. > > > > Before: > > pcie_capability_read_word(pdev, PCI_EXP_DEVCTL, &saved_devctl); > > pcie_capability_write_word(pdev, PCI_EXP_DEVCTL, > > saved_devctl & ~PCI_EXP_DEVCTL_URRE); > > pci_read_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK, > > &saved_uemsk); > > pci_write_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK, > > saved_uemsk | > > PCI_ERR_UNC_UNSUP); > > > > > > After: > > pci_write_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK, > > saved_uemsk); > > pcie_capability_write_word(pdev, PCI_EXP_DEVCTL, > > saved_devctl); > > > > -----Original Message----- > > From: Lazar, Lijo <[email protected]> > > Sent: Wednesday, September 30, 2026 11:37 AM > > To: Zhang, Yifan <[email protected]>; Deucher, Alexander > > <[email protected]>; Koenig, Christian > > <[email protected]>; [email protected] > > Cc: Yuan, Perry <[email protected]> > > Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > > mode1 reset on gfx12 > > > > > > This is the behaviour of this bit - > > > > Memory Space Enable - Controls a Function's response to > > Memory Space accesses. When this bit is Clear, all received Memory > > Space > accesses are caused to be handled as "Unsupported Requests" > > > > On AER enabled systems, depending on how the platform is configured > > to > classify the severity of UR errors, that could result in System error. > > > > On other hand, if you are sure that there is no such access during > > mode-1 reset, we don't need this patch in the first place. > > > > Thanks, > > Lijo > > > > > > On 30-Sep-26 8:53 AM, Zhang, Yifan wrote: > >> AMD General > >> > >> Hi Lijo, > >> > >> Why does clearing PCI_COMMAND_MEMORY cause AER errors on data > center GPUs? Are their PCIe BARs still being accessed during a mode1 reset? > >> > >> Best Regards, > >> Yifan > >> > >> -----Original Message----- > >> From: Lazar, Lijo <[email protected]> > >> Sent: Wednesday, September 30, 2026 11:06 AM > >> To: Zhang, Yifan <[email protected]>; Deucher, Alexander > >> <[email protected]>; Koenig, Christian > >> <[email protected]>; [email protected] > >> Cc: Yuan, Perry <[email protected]> > >> Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > >> mode1 reset on gfx12 > >> > >> This doesn't work for all (at least on some datacenter GPUs). With > >> AER > enabled, this will result in AER errors. Depending on the platform > config, some may classify this access as fatal. > >> > >> Thanks, > >> Lijo > >> > >> On 30-Sep-26 8:25 AM, Zhang, Yifan wrote: > >>> Public > >>> > >>> > >>> Hi Alex, > >>> > >>> I think it is reasonable to apply the patch to all ASICs. I will > >>> sync with the hardware team about this change; so far we have only > >>> tested it on Navi31. > >>> > >>> How about adding a common helper and calling it from ASIC-specific > >>> code, starting with Navi31? For example: > >>> > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> b/drivers/gpu/ drm/amd/amdgpu/amdgpu_device.c > >>> > >>> index b690a07e31ab..075de681ab86 100644 > >>> > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> > >>> @@ -4931,6 +4931,28 @@ bool > amdgpu_device_should_recover_gpu(struct > >>> amdgpu_device *adev) > >>> > >>> return false; > >>> > >>> } > >>> > >>> +void amdgpu_device_mode1_reset_wait(struct amdgpu_device *adev, > >>> > >>> + unsigned int delay_ms) > >>> > >>> +{ > >>> > >>> + u16 saved_cmd, cmd; > >>> > >>> + > >>> > >>> + pci_read_config_word(adev->pdev, PCI_COMMAND, &saved_cmd); > >>> > >>> + cmd = saved_cmd & ~PCI_COMMAND_MEMORY; > >>> > >>> + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd); > >>> > >>> + > >>> > >>> + msleep(delay_ms); > >>> > >>> + > >>> > >>> + pci_write_config_word(adev->pdev, PCI_COMMAND, saved_cmd); > >>> > >>> +} > >>> > >>> + > >>> > >>> diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > >>> b/drivers/ gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > >>> > >>> index b250c5e41e45..356e09b7eef9 100644 > >>> > >>> --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > >>> > >>> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > >>> > >>> @@ -2242,7 +2242,8 @@ int smu_v13_0_mode1_reset(struct > smu_context > >>> *smu) > >>> > >>> ret = smu_cmn_send_smc_msg(smu, SMU_MSG_Mode1Reset, > >>> NULL); > >>> > >>> if (!ret) > >>> > >>> - msleep(SMU13_MODE1_RESET_WAIT_TIME_IN_MS); > >>> > >>> + amdgpu_device_mode1_reset_wait(smu->adev, > >>> > >>> + > >>> SMU13_MODE1_RESET_WAIT_TIME_IN_MS); > >>> > >>> return ret; > >>> > >>> } > >>> > >>> Best Regards, > >>> > >>> Yifan > >>> > >>> *From:*Deucher, Alexander <[email protected]> > >>> *Sent:* Wednesday, September 30, 2026 2:59 AM > >>> *To:* Zhang, Yifan <[email protected]>; Koenig, Christian > >>> <[email protected]>; [email protected] > >>> *Cc:* Yuan, Perry <[email protected]> > >>> *Subject:* Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > >>> mode1 reset on gfx12 > >>> > >>> Public > >>> > >>> I think this should be asic agnostic. Any asic doing mode1 reset > >>> should do this? > >>> > >>> Alex > >>> > >>> ------------------------------------------------------------------ > >>> -- > >>> - > >>> - > >>> -- > >>> > >>> *From:*Zhang, Yifan <[email protected] > >>> <mailto:[email protected]>> > >>> *Sent:* Tuesday, September 29, 2026 5:36 AM > >>> *To:* Koenig, Christian <[email protected] > >>> <mailto:[email protected]>>; [email protected] > >>> <mailto:[email protected]> > >>> <[email protected] > >>> <mailto:[email protected]>> > >>> *Cc:* Deucher, Alexander <[email protected] > >>> <mailto:[email protected]>>; Yuan, Perry > <[email protected] > >>> <mailto:[email protected]>> > >>> *Subject:* RE: [PATCH] amdgpu/pm: block GPU PCI bar access during > >>> mode1 reset on gfx12 > >>> > >>> Public > >>> > >>> Hi Christian, > >>> > >>> The no_hw_access flag was first set in commit > >>> 0de604d0357d0d22cbf03af1077d174b641707b6. This was intended to fix > >>> the Mode 1 reset hang, but it proved insufficient. > >>> I think the flag was set in the SMU code so that it would apply > >>> only to > >>> SMU13/14 ASICs, without affecting the Mode 1 reset process on > >>> other > ASICs. > >>> I’m not sure whether we can drop the no_hw_access flag after > >>> disabling PCI_COMMAND_MEMORY. The PCI hang occurs intermittently, > so > >>> we may need another round of stress testing to confirm that the > >>> flag is no longer needed. Should we remove no_hw_access in a > >>> separate > patch? > >>> > >>> Best Regards, > >>> Yifan > >>> > >>> -----Original Message----- > >>> From: Koenig, Christian <[email protected] > >>> <mailto:[email protected]>> > >>> Sent: Tuesday, September 29, 2026 5:06 PM > >>> To: Zhang, Yifan <[email protected] > >>> <mailto:[email protected]>>; [email protected] > >>> <mailto:[email protected]> > >>> Cc: Deucher, Alexander <[email protected] > >>> <mailto:[email protected]>>; Yuan, Perry > <[email protected] > >>> <mailto:[email protected]>> > >>> Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > >>> mode1 reset on gfx12 > >>> > >>> On 9/29/26 04:25, Yifan Zhang wrote: > >>>> block GPU PCI bar access during mode1 reset on gfx12, to resolve > >>>> the random mode1 reset hang issue. > >>> > >>> Interesting idea, that is much simpler than trying to mess with > >>> CPU page tables in the GPU reset path. > >>> > >>> The only downside I can see is that we now drop VRAM and doorbell > >>> writes, but I think that won't matter since VRAM is lost by a > >>> mode1 reset anyway. > >>> > >>> MMIO writes are dropped as well which means that we can also drop > >>> the no_hw_access flag. > >>> > >>> And BTW the SMU code should absolutely *NOT* touch that > no_hw_access > >>> flag in any way. Who added that there? > >>> > >>> Regards, > >>> Christian. > >>> > >>>> > >>>> Signed-off-by: Yifan Zhang <[email protected] > >>>> <mailto:[email protected]>> > >>>> --- > >>>> .../drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c | 29 > >>>> ++++++++++++------- > >>>> 1 file changed, 19 insertions(+), 10 deletions(-) > >>>> > >>>> diff --git > a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > >>>> b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > >>>> index 072d9c4e202d..25b9085ead71 100644 > >>>> --- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > >>>> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > >>>> @@ -2078,21 +2078,30 @@ static int > >>>> smu_v14_0_2_set_df_cstate(struct smu_context *smu, > >>>> > >>>> static int smu_v14_0_2_mode1_reset(struct smu_context *smu) { > >>>> + struct amdgpu_device *adev = smu->adev; > >>>> + u16 saved_cmd, cmd; > >>>> int ret = 0; > >>>> > >>>> ret = smu_cmn_send_debug_smc_msg(smu, > DEBUGSMC_MSG_Mode1Reset); > >>>> - if (!ret) { > >>>> - if (amdgpu_emu_mode == 1) { > >>>> - msleep(50000); > >>>> - } else { > >>>> - /* disable mmio access while doing mode 1 > >>>> reset*/ > >>>> - smu->adev->no_hw_access = true; > >>>> - /* ensure no_hw_access is globally visible > >>>> before any MMIO */ > >>>> - smp_mb(); > >>>> - msleep(1000); > >>>> - } > >>>> + if (ret) > >>>> + return ret; > >>>> + > >>>> + pci_read_config_word(adev->pdev, PCI_COMMAND, &saved_cmd); > >>>> + cmd = saved_cmd & ~PCI_COMMAND_MEMORY; > >>>> + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd); > >>>> + > >>>> + if (amdgpu_emu_mode == 1) { > >>>> + msleep(50000); > >>>> + } else { > >>>> + /* disable mmio access while doing mode 1 reset*/ > >>>> + smu->adev->no_hw_access = true; > >>>> + /* ensure no_hw_access is globally visible before > >>>> +any MMIO */ > >>>> + smp_mb(); > >>>> + msleep(1000); > >>>> } > >>>> > >>>> + pci_write_config_word(adev->pdev, PCI_COMMAND, saved_cmd); > >>>> + > >>>> return ret; > >>>> } > >>>> > >>> > >> > >
