On Thu, Jun 20, 2024 at 10:58:56AM +0800, Zhenzhong Duan wrote:
> In some cases the detector of a Non-Fatal Error(NFE) is not the most
> appropriate agent to determine the type of the error. For example,
> when software performs a configuration read from a non-existent
> device or Function, completer will send an ERR_NONFATAL Message.
> On some platforms, ERR_NONFATAL results in a System Error, which
> breaks normal software probing.
>
> Advisory Non-Fatal Error(ANFE) is a special case that can be used
> in above scenario. It is predominantly determined by the role of the
> detecting agent (Requester, Completer, or Receiver) and the specific
> error. In such cases, an agent with AER signals the NFE (if enabled)
> by sending an ERR_COR Message as an advisory to software, instead of
> sending ERR_NONFATAL.
>
> When processing an ANFE, ideally both correctable error(CE) status and
> uncorrectable error(UE) status should be cleared. However, there is no
> way to fully identify the UE associated with ANFE. Even worse, Non-Fatal
> Error(NFE) may set the same UE status bit as ANFE. Treating an ANFE as
> NFE will reproduce above mentioned issue, i.e., breaking software probing;
> treating NFE as ANFE will make us ignore some UEs which need active
> recover operation. To avoid clearing UEs that are not ANFE by accident,
> the most conservative route is taken here: If any of the NFE Detected
> bits is set in Device Status, do not touch UE status, they should be
> cleared later by the UE handler. Otherwise, a specific set of UEs that
> may be raised as ANFE according to the PCIe specification will be cleared
> if their corresponding severity is Non-Fatal.
>
> To achieve above purpose, cache UNCOR_STATUS bits that might be ANFE
> in aer_err_info.anfe_status and clean them in pci_aer_handle_error().
> aer_err_info.anfe_status will also be used to print ANFE related bits
> in following patch.
>
> For instance, previously, when the kernel receives an ANFE with Poisoned
> TLP in OS native AER mode, only the status of CE will be reported and
> cleared:
>
> AER: Correctable error message received from 0000:b7:02.0
> PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
> device [8086:0db0] error status/mask=00002000/00000000
> [13] NonFatalErr
>
> If the kernel receives a Malformed TLP after that, two UEs will be
> reported, which is unexpected. The Malformed TLP Header is lost since
> the previous ANFE gated the TLP header logs:
>
> PCIe Bus Error: severity="Uncorrectable (Fatal), type=Transaction Layer,
> (Receiver ID)
> device [8086:0db0] error status/mask=00041000/00180020
> [12] TLP (First)
> [18] MalfTLP
>
> To handle this case properly, calculate potential ANFE related status bits
> and save in aer_err_info. Use this information to determine the status bits
> that need to be cleared.
>
> Now, for the previous scenario, both CE status and related UE status will
> be cleared after ANFE:
>
> AER: Correctable error message received from 0000:b7:02.0
> PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
> device [8086:0db0] error status/mask=00002000/00000000
> [13] NonFatalErr
>
> PCIe Bus Error: severity="Uncorrectable (Fatal), type=Transaction Layer,
> (Receiver ID)
> device [8086:0db0] error status/mask=00040000/00180020
> [18] MalfTLP (First)
>
> Tested-by: Yudong Wang <yudong.w...@intel.com>
> Co-developed-by: "Wang, Qingshun" <qingshun.w...@linux.intel.com>
> Signed-off-by: "Wang, Qingshun" <qingshun.w...@linux.intel.com>
> Signed-off-by: Zhenzhong Duan <zhenzhong.d...@intel.com>
> Reviewed-by: Jonathan Cameron <jonathan.came...@huawei.com>
> Reviewed-by: Kuppuswamy Sathyanarayanan
> <sathyanarayanan.kuppusw...@linux.intel.com>
This no longer applies cleanly; would you mind rebasing it to pci/main
(v6.17-rc1)? There have been recent AER changes; if they affect the
dmesg text, could you update that as well?
> +static void anfe_get_uc_status(struct pci_dev *dev, struct aer_err_info
> *info)
> +{
> + u32 uncor_mask, uncor_status, anfe_status;
> + u16 device_status;
> + int aer = dev->aer_cap;
> +
> + /*
> + * To avoid race between device status read and error status register
> read,
> + * cache uncorrectable error status before checking for NFE in device
> status
> + * register.
I can't tell for sure from the patch, but if this doesn't fit in 80
columns, can you rewrap it so it matches the rest of the file?
> + */
> + pci_read_config_dword(dev, aer + PCI_ERR_UNCOR_STATUS, &uncor_status);
> + pci_read_config_dword(dev, aer + PCI_ERR_UNCOR_MASK, &uncor_mask);
> + /*
> + * According to PCIe Base Specification Revision 6.1 Section 6.2.3.2.4,
> + * if an UNCOR error is raised as Advisory Non-Fatal error, it will
> + * match the following conditions:
> + * a. The severity of the error is Non-Fatal.
> + * b. The error is one of the following:
> + * 1. Poisoned TLP (Section 6.2.3.2.4.3)
> + * 2. Completion Timeout (Section 6.2.3.2.4.4)
> + * 3. Completer Abort (Section 6.2.3.2.4.1)
> + * 4. Unexpected Completion (Section 6.2.3.2.4.5)
> + * 5. Unsupported Request (Section 6.2.3.2.4.1)
> + */
Could you update the citation to PCIe 7.0, please?
Bjorn
