AMD General On Navi3x/Navi4x, the driver doesn't poll for a response. It sends the mode1 reset request through the SMU debug mailbox and waits a fixed interval (SMU13_MODE1_RESET_WAIT_TIME_IN_MS) for the reset to proceed. After that, it polls the PSP bootloader status and the memsize register to confirm the ASIC is out of reset. On ASICs where the regular SMU message path is used, the polled response only acknowledges the message; it doesn't indicate that the reset has completed.
From: Lazar, Lijo <[email protected]> Sent: Wednesday, September 30, 2026 2:28 PM To: Zhang, Yifan <[email protected]>; Wang, Kevin <[email protected]>; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; [email protected] Cc: Yuan, Perry <[email protected]> Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 reset on gfx12 AMD General The actual reset sequence is run by the FW while you are polling for the response. This patch doesn't make any sense in that way. Thanks, Lijo ________________________________ From: Zhang, Yifan <[email protected]<mailto:[email protected]>> Sent: Wednesday, 30 September 2026 11:47:46 To: Wang, Kevin <[email protected]<mailto:[email protected]>>; Lazar, Lijo <[email protected]<mailto:[email protected]>>; Deucher, Alexander <[email protected]<mailto:[email protected]>>; Koenig, Christian <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Cc: Yuan, Perry <[email protected]<mailto:[email protected]>> Subject: RE: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 reset on gfx12 AMD General Yes. In addition, the polling happens inside smu_cmn_send_smc_msg(), which runs before PCI_COMMAND_MEMORY is cleared. During the mode1 reset process there should be no traffic to the PCIe BARs; otherwise, the PCI bus may hang. Also, every PCI config write in the patch targets only the GPU device itself (pci_write_config_dword is per device), so no other PCI devices are affected. -----Original Message----- From: Wang, Kevin <[email protected]<mailto:[email protected]>> Sent: Wednesday, September 30, 2026 2:09 PM To: Lazar, Lijo <[email protected]<mailto:[email protected]>>; Zhang, Yifan <[email protected]<mailto:[email protected]>>; Deucher, Alexander <[email protected]<mailto:[email protected]>>; Koenig, Christian <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> Cc: Yuan, Perry <[email protected]<mailto:[email protected]>> Subject: RE: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 reset on gfx12 AMD General > On a second look, the patch doesn't seem right. Reset sequence is > supposed to be complete by the time FW responds back to the mode-1 reset > message. > During reset sequence, we need Mem space to be enabled to poll success > status of the message and that's most volatile window during the whole > sequence. That is not right. When the OS driver polls and sees the response register become "1", it does **NOT** mean the mode1 sequence is completed. The PMFW will continue forward the message to PSP and wait for PMFW ucode reload to finish at a 2nd-half stage. Best Regards, Kevin > -----Original Message----- > From: amd-gfx > <[email protected]<mailto:[email protected]>> > On Behalf Of > Lazar, Lijo > Sent: Wednesday, September 30, 2026 1:41 PM > To: Zhang, Yifan <[email protected]<mailto:[email protected]>>; > Deucher, Alexander > <[email protected]<mailto:[email protected]>>; Koenig, > Christian > <[email protected]<mailto:[email protected]>>; > [email protected]<mailto:[email protected]> > Cc: Yuan, Perry <[email protected]<mailto:[email protected]>> > Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 > reset on gfx12 > > On a second look, the patch doesn't seem right. Reset sequence is > supposed to be complete by the time FW responds back to the mode-1 reset > message. > During reset sequence, we need Mem space to be enabled to poll success > status of the message and that's most volatile window during the whole > sequence. > > Disabling the memory space after reset sequence is complete looks to > be the wrong diagnosis of the issue. Access during reinit sequence > after reset is safe and shouldn't cause any issue in general. > > That said, we should try to resolve the issue without touching any PCI > config settings. PCI config space is open for others as well. > > Thanks, > Lijo > > On 30-Sep-26 10:57 AM, Zhang, Yifan wrote: > > AMD General > > > > How about disable UE during mode1 reset ? E.g. > > > > Before: > > pcie_capability_read_word(pdev, PCI_EXP_DEVCTL, &saved_devctl); > > pcie_capability_write_word(pdev, PCI_EXP_DEVCTL, > > saved_devctl & ~PCI_EXP_DEVCTL_URRE); > > pci_read_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK, > > &saved_uemsk); > > pci_write_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK, > > saved_uemsk | > > PCI_ERR_UNC_UNSUP); > > > > > > After: > > pci_write_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK, > > saved_uemsk); > > pcie_capability_write_word(pdev, PCI_EXP_DEVCTL, > > saved_devctl); > > > > -----Original Message----- > > From: Lazar, Lijo <[email protected]<mailto:[email protected]>> > > Sent: Wednesday, September 30, 2026 11:37 AM > > To: Zhang, Yifan <[email protected]<mailto:[email protected]>>; > > Deucher, Alexander > > <[email protected]<mailto:[email protected]>>; Koenig, > > Christian > > <[email protected]<mailto:[email protected]>>; > > [email protected]<mailto:[email protected]> > > Cc: Yuan, Perry <[email protected]<mailto:[email protected]>> > > Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > > mode1 reset on gfx12 > > > > > > This is the behaviour of this bit - > > > > Memory Space Enable - Controls a Function's response to > > Memory Space accesses. When this bit is Clear, all received Memory > > Space > accesses are caused to be handled as "Unsupported Requests" > > > > On AER enabled systems, depending on how the platform is configured > > to > classify the severity of UR errors, that could result in System error. > > > > On other hand, if you are sure that there is no such access during > > mode-1 reset, we don't need this patch in the first place. > > > > Thanks, > > Lijo > > > > > > On 30-Sep-26 8:53 AM, Zhang, Yifan wrote: > >> AMD General > >> > >> Hi Lijo, > >> > >> Why does clearing PCI_COMMAND_MEMORY cause AER errors on data > center GPUs? Are their PCIe BARs still being accessed during a mode1 reset? > >> > >> Best Regards, > >> Yifan > >> > >> -----Original Message----- > >> From: Lazar, Lijo <[email protected]<mailto:[email protected]>> > >> Sent: Wednesday, September 30, 2026 11:06 AM > >> To: Zhang, Yifan <[email protected]<mailto:[email protected]>>; > >> Deucher, Alexander > >> <[email protected]<mailto:[email protected]>>; Koenig, > >> Christian > >> <[email protected]<mailto:[email protected]>>; > >> [email protected]<mailto:[email protected]> > >> Cc: Yuan, Perry <[email protected]<mailto:[email protected]>> > >> Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > >> mode1 reset on gfx12 > >> > >> This doesn't work for all (at least on some datacenter GPUs). With > >> AER > enabled, this will result in AER errors. Depending on the platform > config, some may classify this access as fatal. > >> > >> Thanks, > >> Lijo > >> > >> On 30-Sep-26 8:25 AM, Zhang, Yifan wrote: > >>> Public > >>> > >>> > >>> Hi Alex, > >>> > >>> I think it is reasonable to apply the patch to all ASICs. I will > >>> sync with the hardware team about this change; so far we have only > >>> tested it on Navi31. > >>> > >>> How about adding a common helper and calling it from ASIC-specific > >>> code, starting with Navi31? For example: > >>> > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> b/drivers/gpu/ drm/amd/amdgpu/amdgpu_device.c > >>> > >>> index b690a07e31ab..075de681ab86 100644 > >>> > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> > >>> @@ -4931,6 +4931,28 @@ bool > amdgpu_device_should_recover_gpu(struct > >>> amdgpu_device *adev) > >>> > >>> return false; > >>> > >>> } > >>> > >>> +void amdgpu_device_mode1_reset_wait(struct amdgpu_device *adev, > >>> > >>> + unsigned int delay_ms) > >>> > >>> +{ > >>> > >>> + u16 saved_cmd, cmd; > >>> > >>> + > >>> > >>> + pci_read_config_word(adev->pdev, PCI_COMMAND, &saved_cmd); > >>> > >>> + cmd = saved_cmd & ~PCI_COMMAND_MEMORY; > >>> > >>> + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd); > >>> > >>> + > >>> > >>> + msleep(delay_ms); > >>> > >>> + > >>> > >>> + pci_write_config_word(adev->pdev, PCI_COMMAND, saved_cmd); > >>> > >>> +} > >>> > >>> + > >>> > >>> diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > >>> b/drivers/ gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > >>> > >>> index b250c5e41e45..356e09b7eef9 100644 > >>> > >>> --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > >>> > >>> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > >>> > >>> @@ -2242,7 +2242,8 @@ int smu_v13_0_mode1_reset(struct > smu_context > >>> *smu) > >>> > >>> ret = smu_cmn_send_smc_msg(smu, SMU_MSG_Mode1Reset, > >>> NULL); > >>> > >>> if (!ret) > >>> > >>> - msleep(SMU13_MODE1_RESET_WAIT_TIME_IN_MS); > >>> > >>> + amdgpu_device_mode1_reset_wait(smu->adev, > >>> > >>> + > >>> SMU13_MODE1_RESET_WAIT_TIME_IN_MS); > >>> > >>> return ret; > >>> > >>> } > >>> > >>> Best Regards, > >>> > >>> Yifan > >>> > >>> *From:*Deucher, Alexander > >>> <[email protected]<mailto:[email protected]>> > >>> *Sent:* Wednesday, September 30, 2026 2:59 AM > >>> *To:* Zhang, Yifan <[email protected]<mailto:[email protected]>>; > >>> Koenig, Christian > >>> <[email protected]<mailto:[email protected]>>; > >>> [email protected]<mailto:[email protected]> > >>> *Cc:* Yuan, Perry <[email protected]<mailto:[email protected]>> > >>> *Subject:* Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > >>> mode1 reset on gfx12 > >>> > >>> Public > >>> > >>> I think this should be asic agnostic. Any asic doing mode1 reset > >>> should do this? > >>> > >>> Alex > >>> > >>> ------------------------------------------------------------------ > >>> -- > >>> - > >>> - > >>> -- > >>> > >>> *From:*Zhang, Yifan <[email protected] <mailto:[email protected]%0b>> >>> <mailto:[email protected]>> > >>> *Sent:* Tuesday, September 29, 2026 5:36 AM > >>> *To:* Koenig, Christian <[email protected] <mailto:[email protected]%0b>> >>> <mailto:[email protected]>>; [email protected]<mailto:[email protected]> > >>> <mailto:[email protected]> > >>> <[email protected] <mailto:[email protected]%0b>> >>> <mailto:[email protected]>> > >>> *Cc:* Deucher, Alexander <[email protected] <mailto:[email protected]%0b>> >>> <mailto:[email protected]>>; Yuan, Perry > <[email protected] <mailto:[email protected]%0b>> >>> <mailto:[email protected]>> > >>> *Subject:* RE: [PATCH] amdgpu/pm: block GPU PCI bar access during > >>> mode1 reset on gfx12 > >>> > >>> Public > >>> > >>> Hi Christian, > >>> > >>> The no_hw_access flag was first set in commit > >>> 0de604d0357d0d22cbf03af1077d174b641707b6. This was intended to fix > >>> the Mode 1 reset hang, but it proved insufficient. > >>> I think the flag was set in the SMU code so that it would apply > >>> only to > >>> SMU13/14 ASICs, without affecting the Mode 1 reset process on > >>> other > ASICs. > >>> I'm not sure whether we can drop the no_hw_access flag after > >>> disabling PCI_COMMAND_MEMORY. The PCI hang occurs intermittently, > so > >>> we may need another round of stress testing to confirm that the > >>> flag is no longer needed. Should we remove no_hw_access in a > >>> separate > patch? > >>> > >>> Best Regards, > >>> Yifan > >>> > >>> -----Original Message----- > >>> From: Koenig, Christian <[email protected] <mailto:[email protected]%0b>> >>> <mailto:[email protected]>> > >>> Sent: Tuesday, September 29, 2026 5:06 PM > >>> To: Zhang, Yifan <[email protected] <mailto:[email protected]%0b>> >>> <mailto:[email protected]>>; [email protected]<mailto:[email protected]> > >>> <mailto:[email protected]> > >>> Cc: Deucher, Alexander <[email protected] <mailto:[email protected]%0b>> >>> <mailto:[email protected]>>; Yuan, Perry > <[email protected] <mailto:[email protected]%0b>> >>> <mailto:[email protected]>> > >>> Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > >>> mode1 reset on gfx12 > >>> > >>> On 9/29/26 04:25, Yifan Zhang wrote: > >>>> block GPU PCI bar access during mode1 reset on gfx12, to resolve > >>>> the random mode1 reset hang issue. > >>> > >>> Interesting idea, that is much simpler than trying to mess with > >>> CPU page tables in the GPU reset path. > >>> > >>> The only downside I can see is that we now drop VRAM and doorbell > >>> writes, but I think that won't matter since VRAM is lost by a > >>> mode1 reset anyway. > >>> > >>> MMIO writes are dropped as well which means that we can also drop > >>> the no_hw_access flag. > >>> > >>> And BTW the SMU code should absolutely *NOT* touch that > no_hw_access > >>> flag in any way. Who added that there? > >>> > >>> Regards, > >>> Christian. > >>> > >>>> > >>>> Signed-off-by: Yifan Zhang <[email protected] <mailto:[email protected]%0b>> >>>> <mailto:[email protected]>> > >>>> --- > >>>> .../drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c | 29 > >>>> ++++++++++++------- > >>>> 1 file changed, 19 insertions(+), 10 deletions(-) > >>>> > >>>> diff --git > a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > >>>> b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > >>>> index 072d9c4e202d..25b9085ead71 100644 > >>>> --- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > >>>> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c > >>>> @@ -2078,21 +2078,30 @@ static int > >>>> smu_v14_0_2_set_df_cstate(struct smu_context *smu, > >>>> > >>>> static int smu_v14_0_2_mode1_reset(struct smu_context *smu) { > >>>> + struct amdgpu_device *adev = smu->adev; > >>>> + u16 saved_cmd, cmd; > >>>> int ret = 0; > >>>> > >>>> ret = smu_cmn_send_debug_smc_msg(smu, > DEBUGSMC_MSG_Mode1Reset); > >>>> - if (!ret) { > >>>> - if (amdgpu_emu_mode == 1) { > >>>> - msleep(50000); > >>>> - } else { > >>>> - /* disable mmio access while doing mode 1 > >>>> reset*/ > >>>> - smu->adev->no_hw_access = true; > >>>> - /* ensure no_hw_access is globally visible > >>>> before any MMIO */ > >>>> - smp_mb(); > >>>> - msleep(1000); > >>>> - } > >>>> + if (ret) > >>>> + return ret; > >>>> + > >>>> + pci_read_config_word(adev->pdev, PCI_COMMAND, &saved_cmd); > >>>> + cmd = saved_cmd & ~PCI_COMMAND_MEMORY; > >>>> + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd); > >>>> + > >>>> + if (amdgpu_emu_mode == 1) { > >>>> + msleep(50000); > >>>> + } else { > >>>> + /* disable mmio access while doing mode 1 reset*/ > >>>> + smu->adev->no_hw_access = true; > >>>> + /* ensure no_hw_access is globally visible before > >>>> +any MMIO */ > >>>> + smp_mb(); > >>>> + msleep(1000); > >>>> } > >>>> > >>>> + pci_write_config_word(adev->pdev, PCI_COMMAND, saved_cmd); > >>>> + > >>>> return ret; > >>>> } > >>>> > >>> > >> > >
