AMD General
How about disable UE during mode1 reset ? E.g.
Before:
pcie_capability_read_word(pdev, PCI_EXP_DEVCTL, &saved_devctl);
pcie_capability_write_word(pdev, PCI_EXP_DEVCTL,
saved_devctl & ~PCI_EXP_DEVCTL_URRE);
pci_read_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK,
&saved_uemsk);
pci_write_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK,
saved_uemsk | PCI_ERR_UNC_UNSUP);
After:
pci_write_config_dword(pdev, aer + PCI_ERR_UNCOR_MASK,
saved_uemsk);
pcie_capability_write_word(pdev, PCI_EXP_DEVCTL, saved_devctl);
-----Original Message-----
From: Lazar, Lijo <[email protected]>
Sent: Wednesday, September 30, 2026 11:37 AM
To: Zhang, Yifan <[email protected]>; Deucher, Alexander
<[email protected]>; Koenig, Christian <[email protected]>;
[email protected]
Cc: Yuan, Perry <[email protected]>
Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1 reset on
gfx12
This is the behaviour of this bit -
Memory Space Enable - Controls a Function's response to Memory Space
accesses. When this bit is Clear, all received Memory Space accesses are caused
to be handled as "Unsupported Requests"
On AER enabled systems, depending on how the platform is configured to classify
the severity of UR errors, that could result in System error.
On other hand, if you are sure that there is no such access during
mode-1 reset, we don't need this patch in the first place.
Thanks,
Lijo
On 30-Sep-26 8:53 AM, Zhang, Yifan wrote:
> AMD General
>
> Hi Lijo,
>
> Why does clearing PCI_COMMAND_MEMORY cause AER errors on data center GPUs?
> Are their PCIe BARs still being accessed during a mode1 reset?
>
> Best Regards,
> Yifan
>
> -----Original Message-----
> From: Lazar, Lijo <[email protected]>
> Sent: Wednesday, September 30, 2026 11:06 AM
> To: Zhang, Yifan <[email protected]>; Deucher, Alexander
> <[email protected]>; Koenig, Christian
> <[email protected]>; [email protected]
> Cc: Yuan, Perry <[email protected]>
> Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1
> reset on gfx12
>
> This doesn't work for all (at least on some datacenter GPUs). With AER
> enabled, this will result in AER errors. Depending on the platform config,
> some may classify this access as fatal.
>
> Thanks,
> Lijo
>
> On 30-Sep-26 8:25 AM, Zhang, Yifan wrote:
>> Public
>>
>>
>> Hi Alex,
>>
>> I think it is reasonable to apply the patch to all ASICs. I will sync
>> with the hardware team about this change; so far we have only tested
>> it on Navi31.
>>
>> How about adding a common helper and calling it from ASIC-specific
>> code, starting with Navi31? For example:
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> b/drivers/gpu/ drm/amd/amdgpu/amdgpu_device.c
>>
>> index b690a07e31ab..075de681ab86 100644
>>
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>
>> @@ -4931,6 +4931,28 @@ bool amdgpu_device_should_recover_gpu(struct
>> amdgpu_device *adev)
>>
>> return false;
>>
>> }
>>
>> +void amdgpu_device_mode1_reset_wait(struct amdgpu_device *adev,
>>
>> + unsigned int delay_ms)
>>
>> +{
>>
>> + u16 saved_cmd, cmd;
>>
>> +
>>
>> + pci_read_config_word(adev->pdev, PCI_COMMAND, &saved_cmd);
>>
>> + cmd = saved_cmd & ~PCI_COMMAND_MEMORY;
>>
>> + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd);
>>
>> +
>>
>> + msleep(delay_ms);
>>
>> +
>>
>> + pci_write_config_word(adev->pdev, PCI_COMMAND, saved_cmd);
>>
>> +}
>>
>> +
>>
>> diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
>> b/drivers/ gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
>>
>> index b250c5e41e45..356e09b7eef9 100644
>>
>> --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
>>
>> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
>>
>> @@ -2242,7 +2242,8 @@ int smu_v13_0_mode1_reset(struct smu_context
>> *smu)
>>
>> ret = smu_cmn_send_smc_msg(smu, SMU_MSG_Mode1Reset, NULL);
>>
>> if (!ret)
>>
>> - msleep(SMU13_MODE1_RESET_WAIT_TIME_IN_MS);
>>
>> + amdgpu_device_mode1_reset_wait(smu->adev,
>>
>> +
>> SMU13_MODE1_RESET_WAIT_TIME_IN_MS);
>>
>> return ret;
>>
>> }
>>
>> Best Regards,
>>
>> Yifan
>>
>> *From:*Deucher, Alexander <[email protected]>
>> *Sent:* Wednesday, September 30, 2026 2:59 AM
>> *To:* Zhang, Yifan <[email protected]>; Koenig, Christian
>> <[email protected]>; [email protected]
>> *Cc:* Yuan, Perry <[email protected]>
>> *Subject:* Re: [PATCH] amdgpu/pm: block GPU PCI bar access during
>> mode1 reset on gfx12
>>
>> Public
>>
>> I think this should be asic agnostic. Any asic doing mode1 reset
>> should do this?
>>
>> Alex
>>
>> ---------------------------------------------------------------------
>> -
>> --
>>
>> *From:*Zhang, Yifan <[email protected]
>> <mailto:[email protected]>>
>> *Sent:* Tuesday, September 29, 2026 5:36 AM
>> *To:* Koenig, Christian <[email protected]
>> <mailto:[email protected]>>; [email protected]
>> <mailto:[email protected]> <[email protected]
>> <mailto:[email protected]>>
>> *Cc:* Deucher, Alexander <[email protected]
>> <mailto:[email protected]>>; Yuan, Perry <[email protected]
>> <mailto:[email protected]>>
>> *Subject:* RE: [PATCH] amdgpu/pm: block GPU PCI bar access during
>> mode1 reset on gfx12
>>
>> Public
>>
>> Hi Christian,
>>
>> The no_hw_access flag was first set in commit
>> 0de604d0357d0d22cbf03af1077d174b641707b6. This was intended to fix
>> the Mode 1 reset hang, but it proved insufficient.
>> I think the flag was set in the SMU code so that it would apply only
>> to
>> SMU13/14 ASICs, without affecting the Mode 1 reset process on other ASICs.
>> I’m not sure whether we can drop the no_hw_access flag after
>> disabling PCI_COMMAND_MEMORY. The PCI hang occurs intermittently, so
>> we may need another round of stress testing to confirm that the flag
>> is no longer needed. Should we remove no_hw_access in a separate patch?
>>
>> Best Regards,
>> Yifan
>>
>> -----Original Message-----
>> From: Koenig, Christian <[email protected]
>> <mailto:[email protected]>>
>> Sent: Tuesday, September 29, 2026 5:06 PM
>> To: Zhang, Yifan <[email protected]
>> <mailto:[email protected]>>; [email protected]
>> <mailto:[email protected]>
>> Cc: Deucher, Alexander <[email protected]
>> <mailto:[email protected]>>; Yuan, Perry <[email protected]
>> <mailto:[email protected]>>
>> Subject: Re: [PATCH] amdgpu/pm: block GPU PCI bar access during mode1
>> reset on gfx12
>>
>> On 9/29/26 04:25, Yifan Zhang wrote:
>>> block GPU PCI bar access during mode1 reset on gfx12, to resolve the
>>> random mode1 reset hang issue.
>>
>> Interesting idea, that is much simpler than trying to mess with CPU
>> page tables in the GPU reset path.
>>
>> The only downside I can see is that we now drop VRAM and doorbell
>> writes, but I think that won't matter since VRAM is lost by a mode1
>> reset anyway.
>>
>> MMIO writes are dropped as well which means that we can also drop the
>> no_hw_access flag.
>>
>> And BTW the SMU code should absolutely *NOT* touch that no_hw_access
>> flag in any way. Who added that there?
>>
>> Regards,
>> Christian.
>>
>>>
>>> Signed-off-by: Yifan Zhang <[email protected]
>>> <mailto:[email protected]>>
>>> ---
>>> .../drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c | 29
>>> ++++++++++++-------
>>> 1 file changed, 19 insertions(+), 10 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
>>> b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
>>> index 072d9c4e202d..25b9085ead71 100644
>>> --- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
>>> +++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
>>> @@ -2078,21 +2078,30 @@ static int smu_v14_0_2_set_df_cstate(struct
>>> smu_context *smu,
>>>
>>> static int smu_v14_0_2_mode1_reset(struct smu_context *smu) {
>>> + struct amdgpu_device *adev = smu->adev;
>>> + u16 saved_cmd, cmd;
>>> int ret = 0;
>>>
>>> ret = smu_cmn_send_debug_smc_msg(smu, DEBUGSMC_MSG_Mode1Reset);
>>> - if (!ret) {
>>> - if (amdgpu_emu_mode == 1) {
>>> - msleep(50000);
>>> - } else {
>>> - /* disable mmio access while doing mode 1
>>> reset*/
>>> - smu->adev->no_hw_access = true;
>>> - /* ensure no_hw_access is globally visible
>>> before any MMIO */
>>> - smp_mb();
>>> - msleep(1000);
>>> - }
>>> + if (ret)
>>> + return ret;
>>> +
>>> + pci_read_config_word(adev->pdev, PCI_COMMAND, &saved_cmd);
>>> + cmd = saved_cmd & ~PCI_COMMAND_MEMORY;
>>> + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd);
>>> +
>>> + if (amdgpu_emu_mode == 1) {
>>> + msleep(50000);
>>> + } else {
>>> + /* disable mmio access while doing mode 1 reset*/
>>> + smu->adev->no_hw_access = true;
>>> + /* ensure no_hw_access is globally visible before any
>>> +MMIO */
>>> + smp_mb();
>>> + msleep(1000);
>>> }
>>>
>>> + pci_write_config_word(adev->pdev, PCI_COMMAND, saved_cmd);
>>> +
>>> return ret;
>>> }
>>>
>>
>
