AMD General Hi Lijo,
Why does clearing PCI_COMMAND_MEMORY cause AER errors on data center GPUs? Are their PCIe BARs still being accessed during a mode1 reset? Best Regards, Yifan -----Original Message----- From: Lazar, Lijo <[email protected]> Sent: Wednesday, September 30, 2026 11:06 AM To: Zhang, Yifan <[email protected]>; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; [email protected] Cc: Yuan, Perry <[email protected]> Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 reset on gfx12 This doesn't work for all (at least on some datacenter GPUs). With AER enabled, this will result in AER errors. Depending on the platform config, some may classify this access as fatal. Thanks, Lijo On 30-Sep-26 8:25 AM, Zhang, Yifan wrote: > Public > > > Hi Alex, > > I think it is reasonable to apply the patch to all ASICs. I will sync > with the hardware team about this change; so far we have only tested > it on Navi31. > > How about adding a common helper and calling it from ASIC-specific > code, starting with Navi31? For example: > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/ > drm/amd/amdgpu/amdgpu_device.c > > index b690a07e31ab..075de681ab86 100644 > > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > > @@ -4931,6 +4931,28 @@ bool amdgpu_device_should_recover_gpu(struct > amdgpu_device *adev) > > return false; > > } > > +void amdgpu_device_mode1_reset_wait(struct amdgpu_device *adev, > > + unsigned int delay_ms) > > +{ > > + u16 saved_cmd, cmd; > > + > > + pci_read_config_word(adev->pdev, PCI_COMMAND, &saved_cmd); > > + cmd = saved_cmd & ~PCI_COMMAND_MEMORY; > > + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd); > > + > > + msleep(delay_ms); > > + > > + pci_write_config_word(adev->pdev, PCI_COMMAND, saved_cmd); > > +} > > + > > diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c b/drivers/ > gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > > index b250c5e41e45..356e09b7eef9 100644 > > --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > > +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c > > @@ -2242,7 +2242,8 @@ int smu_v13_0_mode1_reset(struct smu_context > *smu) > > ret = smu_cmn_send_smc_msg(smu, SMU_MSG_Mode1Reset, NULL); > > if (!ret) > > - msleep(SMU13_MODE1_RESET_WAIT_TIME_IN_MS); > > + amdgpu_device_mode1_reset_wait(smu->adev, > > + > SMU13_MODE1_RESET_WAIT_TIME_IN_MS); > > return ret; > > } > > Best Regards, > > Yifan > > *From:*Deucher, Alexander <[email protected]> > *Sent:* Wednesday, September 30, 2026 2:59 AM > *To:* Zhang, Yifan <[email protected]>; Koenig, Christian > <[email protected]>; [email protected] > *Cc:* Yuan, Perry <[email protected]> > *Subject:* Re: [PATCH] amdgpu/pm: block GPU PCI bar access during > mode1 reset on gfx12 > > Public > > I think this should be asic agnostic. Any asic doing mode1 reset > should do this? > > Alex > > ---------------------------------------------------------------------- > -- > > *From:*Zhang, Yifan <[email protected] > <mailto:[email protected]>> > *Sent:* Tuesday, September 29, 2026 5:36 AM > *To:* Koenig, Christian <[email protected] > <mailto:[email protected]>>; [email protected] > <mailto:[email protected]> <[email protected] > <mailto:[email protected]>> > *Cc:* Deucher, Alexander <[email protected] > <mailto:[email protected]>>; Yuan, Perry <[email protected] > <mailto:[email protected]>> > *Subject:* RE: [PATCH] amdgpu/pm: block GPU PCI bar access during > mode1 reset on gfx12 > > Public > > Hi Christian, > > The no_hw_access flag was first set in commit > 0de604d0357d0d22cbf03af1077d174b641707b6. This was intended to fix the > Mode 1 reset hang, but it proved insufficient. > I think the flag was set in the SMU code so that it would apply only > to > SMU13/14 ASICs, without affecting the Mode 1 reset process on other ASICs. > I’m not sure whether we can drop the no_hw_access flag after disabling > PCI_COMMAND_MEMORY. The PCI hang occurs intermittently, so we may need > another round of stress testing to confirm that the flag is no longer > needed. Should we remove no_hw_access in a separate patch? > > Best Regards, > Yifan > > -----Original Message----- > From: Koenig, Christian <[email protected] > <mailto:[email protected]>> > Sent: Tuesday, September 29, 2026 5:06 PM > To: Zhang, Yifan <[email protected] <mailto:[email protected]>>; > [email protected] <mailto:[email protected]> > Cc: Deucher, Alexander <[email protected] > <mailto:[email protected]>>; Yuan, Perry <[email protected] > <mailto:[email protected]>> > Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 > reset on gfx12 > > On 9/29/26 04:25, Yifan Zhang wrote: >> block GPU PCI bar access during mode1 reset on gfx12, to resolve the >> random mode1 reset hang issue. > > Interesting idea, that is much simpler than trying to mess with CPU > page tables in the GPU reset path. > > The only downside I can see is that we now drop VRAM and doorbell > writes, but I think that won't matter since VRAM is lost by a mode1 > reset anyway. > > MMIO writes are dropped as well which means that we can also drop the > no_hw_access flag. > > And BTW the SMU code should absolutely *NOT* touch that no_hw_access > flag in any way. Who added that there? > > Regards, > Christian. > >> >> Signed-off-by: Yifan Zhang <[email protected] >><mailto:[email protected]>> >> --- >> .../drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c | 29 >> ++++++++++++------- >> 1 file changed, 19 insertions(+), 10 deletions(-) >> >> diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c >> b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c >> index 072d9c4e202d..25b9085ead71 100644 >> --- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c >> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c >> @@ -2078,21 +2078,30 @@ static int smu_v14_0_2_set_df_cstate(struct >> smu_context *smu, >> >> static int smu_v14_0_2_mode1_reset(struct smu_context *smu) { >> + struct amdgpu_device *adev = smu->adev; >> + u16 saved_cmd, cmd; >> int ret = 0; >> >> ret = smu_cmn_send_debug_smc_msg(smu, DEBUGSMC_MSG_Mode1Reset); >> - if (!ret) { >> - if (amdgpu_emu_mode == 1) { >> - msleep(50000); >> - } else { >> - /* disable mmio access while doing mode 1 >>reset*/ >> - smu->adev->no_hw_access = true; >> - /* ensure no_hw_access is globally visible >>before any MMIO */ >> - smp_mb(); >> - msleep(1000); >> - } >> + if (ret) >> + return ret; >> + >> + pci_read_config_word(adev->pdev, PCI_COMMAND, &saved_cmd); >> + cmd = saved_cmd & ~PCI_COMMAND_MEMORY; >> + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd); >> + >> + if (amdgpu_emu_mode == 1) { >> + msleep(50000); >> + } else { >> + /* disable mmio access while doing mode 1 reset*/ >> + smu->adev->no_hw_access = true; >> + /* ensure no_hw_access is globally visible before any >> +MMIO */ >> + smp_mb(); >> + msleep(1000); >> } >> >> + pci_write_config_word(adev->pdev, PCI_COMMAND, saved_cmd); >> + >> return ret; >> } >> >
