On 7/2/2025 8:20 PM, Bagas Sanjaya wrote: > On Wed, Jul 02, 2025 at 08:00:41PM +0900, Alexandre Courbot wrote: >> diff --git a/Documentation/gpu/nova/core/vbios.rst >> b/Documentation/gpu/nova/core/vbios.rst >> new file mode 100644 >> index >> 0000000000000000000000000000000000000000..55d7dd4a6658c2a20cc5617f96b278bc4ec2ba17 >> --- /dev/null >> +++ b/Documentation/gpu/nova/core/vbios.rst >> @@ -0,0 +1,180 @@ >> +.. SPDX-License-Identifier: (GPL-2.0+ OR MIT) >> + >> +========== >> +VBIOS >> +========== >> +This document describes the layout of the VBIOS image which is a series of >> concatenated >> +images in the ROM of the GPU. The VBIOS is mirrored onto the BAR 0 space >> and is read >> +by both Boot ROM firmware (also known as IFR or init-from-rom firmware) on >> the GPU to >> +boot strap various microcontrollers (PMU, SEC, GSP) with critical >> initialization before > > bootstrap > >> +the driver loads, as well as by the nova-core driver in the kernel to boot >> the GSP. >> + >> +The format of the images in the ROM follow the "BIOS Specification" part of >> the >> +PCI specification, with Nvidia-specific extensions. The ROM images of type >> FwSec >> +are the ones that contain Falcon ucode and what we are mainly looking for. >> + >> +As an example, the following are the different image types that can be >> found in the >> +VBIOS of an Ampere GA102 GPU which is supported by the nova-core driver. >> + >> +- PciAt Image (Type 0x00) - This is the standard PCI BIOS image, whose name >> + likely comes from the "IBM PC/AT" architecture. >> + >> +- EFI Image (Type 0x03) - This is the EFI BIOS image. It contains the UEFI >> GOP >> + driver that is used to display UEFI graphics output. >> + >> +- First FwSec Image (Type 0xE0) - The first FwSec image (Secure Firmware) >> + >> +- Second FwSec Image (Type 0xE0) - The second FwSec image (Secure Firmware) >> + contains various different microcodes (also known as an applications) >> that do > > various microcodes? The above 2 fixups sounds good to me, I'm guessing Danilo if you're pulling this then maybe apply the fixups directly to the patch since it is trivial? > >> + >> +.. note:: >> + This diagram is created based on an GA-102 Ampere GPU as an example and >> could >> + vary for future or other GPUs. >> + >> +Here is a block diagram of the VBIOS layout:: > > Above sentence (but not the note directive) is redundant, though. This suggestion I'd skip. But thanks for all the review! - Joel > >> + >> + >> +----------------------------------------------------------------------------+ >> + | VBIOS (Starting at ROM_OFFSET: 0x300000) >> | >> + >> +----------------------------------------------------------------------------+ >> + | +-----------------------------------------------+ >> | >> + | | PciAt Image (Type 0x00) | >> | >> + | +-----------------------------------------------+ >> | >> + | | +-------------------+ | >> | >> + | | | ROM Header | | >> | >> + | | | (Signature 0xAA55)| | >> | >> + | | +-------------------+ | >> | >> + | | | rom header's pci_data_struct_offset | >> | >> + | | | points to the PCIR structure | >> | >> + | | V | >> | >> + | | +-------------------+ | >> | >> + | | | PCIR Structure | | >> | >> + | | | (Signature "PCIR")| | >> | >> + | | | last_image: 0x80 | | >> | >> + | | | image_len: size | | >> | >> + | | | in 512-byte units | | >> | >> + | | +-------------------+ | >> | >> + | | | | >> | >> + | | | NPDE immediately follows PCIR | >> | >> + | | V | >> | >> + | | +-------------------+ | >> | >> + | | | NPDE Structure | | >> | >> + | | | (Signature "NPDE")| | >> | >> + | | | last_image: 0x00 | | >> | >> + | | +-------------------+ | >> | >> + | | | >> | >> + | | +-------------------+ | >> | >> + | | | BIT Header | (Signature scanning | >> | >> + | | | (Signature "BIT") | provides the location | >> | >> + | | +-------------------+ of the BIT table) | >> | >> + | | | header is | >> | >> + | | | followed by a table of tokens | >> | >> + | | V one of which is for falcon data. | >> | >> + | | +-------------------+ | >> | >> + | | | BIT Tokens | | >> | >> + | | | ______________ | | >> | >> + | | | | Falcon Data | | | >> | >> + | | | | Token (0x70)|---+------------>------------+--+ >> | >> + | | | +-------------+ | falcon_data_ptr() | | >> | >> + | | +-------------------+ | V >> | >> + | +-----------------------------------------------+ | >> | >> + | (no gap between images) | >> | >> + | +-----------------------------------------------+ | >> | >> + | | EFI Image (Type 0x03) | | >> | >> + | +-----------------------------------------------+ | >> | >> + | | Contains the UEFI GOP driver (Graphics Output)| | >> | >> + | | +-------------------+ | | >> | >> + | | | ROM Header | | | >> | >> + | | +-------------------+ | | >> | >> + | | | PCIR Structure | | | >> | >> + | | +-------------------+ | | >> | >> + | | | NPDE Structure | | | >> | >> + | | +-------------------+ | | >> | >> + | | | Image data | | | >> | >> + | | +-------------------+ | | >> | >> + | +-----------------------------------------------+ | >> | >> + | (no gap between images) | >> | >> + | +-----------------------------------------------+ | >> | >> + | | First FwSec Image (Type 0xE0) | | >> | >> + | +-----------------------------------------------+ | >> | >> + | | +-------------------+ | | >> | >> + | | | ROM Header | | | >> | >> + | | +-------------------+ | | >> | >> + | | | PCIR Structure | | | >> | >> + | | +-------------------+ | | >> | >> + | | | NPDE Structure | | | >> | >> + | | +-------------------+ | | >> | >> + | | | Image data | | | >> | >> + | | +-------------------+ | | >> | >> + | +-----------------------------------------------+ | >> | >> + | (no gap between images) | >> | >> + | +-----------------------------------------------+ | >> | >> + | | Second FwSec Image (Type 0xE0) | | >> | >> + | +-----------------------------------------------+ | >> | >> + | | +-------------------+ | | >> | >> + | | | ROM Header | | | >> | >> + | | +-------------------+ | | >> | >> + | | | PCIR Structure | | | >> | >> + | | +-------------------+ | | >> | >> + | | | NPDE Structure | | | >> | >> + | | +-------------------+ | | >> | >> + | | | | >> | >> + | | +-------------------+ | | >> | >> + | | | PMU Lookup Table | <- falcon_data_offset <----+ >> | >> + | | | +-------------+ | pmu_lookup_table | >> | >> + | | | | Entry 0x85 | | | >> | >> + | | | | FWSEC_PROD | | | >> | >> + | | | +-------------+ | | >> | >> + | | +-------------------+ | >> | >> + | | | | >> | >> + | | | points to | >> | >> + | | V | >> | >> + | | +-------------------+ | >> | >> + | | | FalconUCodeDescV3 | <- falcon_ucode_offset | >> | >> + | | | (FWSEC Firmware) | fwsec_header() | >> | >> + | | +-------------------+ | >> | >> + | | | immediately followed by... | >> | >> + | | V | >> | >> + | | +----------------------------+ | >> | >> + | | | Signatures + FWSEC Ucode | | >> | >> + | | | fwsec_sigs(), fwsec_ucode()| | >> | >> + | | +----------------------------+ | >> | >> + | +-----------------------------------------------+ >> | >> + | >> | >> + >> +----------------------------------------------------------------------------+ >> + >> +Falcon data Lookup >> +------------------ >> +A key part of the VBIOS extraction code (vbios.rs) is to find the location >> of the >> +Falcon data in the VBIOS which contains the PMU lookup table. This lookup >> table is >> +used to find the required Falcon ucode based on an application ID. >> + >> +The location of the PMU lookup table is found by scanning the BIT (`BIOS >> Information Table`_) >> +tokens for a token with the id `BIT_TOKEN_ID_FALCON_DATA` (0x70) which >> indicates the >> +offset of the same from the start of the VBIOS image. Unfortunately, the >> offset >> +does not account for the EFI image located between the PciAt and FwSec >> images. >> +The `vbios.rs` code compensates for this with appropriate arithmetic. >> + >> +.. _`BIOS Information Table`: >> https://download.nvidia.com/open-gpu-doc/BIOS-Information-Table/1/BIOS-Information-Table.html > > The rest looks good. > > Thanks. >
