While not terribly complicated, a little bit of documentation will help augment the code for this very important mechanism.
Signed-off-by: Joel Fernandes <[email protected]> --- Documentation/gpu/nova/core/pramin.rst | 113 +++++++++++++++++++++++++ Documentation/gpu/nova/index.rst | 1 + 2 files changed, 114 insertions(+) create mode 100644 Documentation/gpu/nova/core/pramin.rst diff --git a/Documentation/gpu/nova/core/pramin.rst b/Documentation/gpu/nova/core/pramin.rst new file mode 100644 index 000000000000..19615e504db9 --- /dev/null +++ b/Documentation/gpu/nova/core/pramin.rst @@ -0,0 +1,113 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================= +PRAMIN aperture mechanism +========================= + +.. note:: + The following description is approximate and current as of the Ampere family. + It may change for future generations and is intended to assist in understanding + the driver code. + +Introduction +============ + +PRAMIN is a hardware aperture mechanism that provides CPU access to GPU Video RAM (VRAM) before +the GPU's Memory Management Unit (MMU) and page tables are initialized. This 1MB sliding window, +located at a fixed offset within BAR0, is essential for setting up page tables and other critical +GPU data structures without relying on the GPU's MMU. + +Architecture Overview +===================== + +Logically, the PRAMIN aperture mechanism is implemented by the GPU's PBUS (PCIe Bus Controller Unit) +and provides a CPU-accessible window into VRAM through the PCIe interface:: + + +-----------------+ PCIe +------------------------------+ + | CPU |<----------->| GPU | + +-----------------+ | | + | +----------------------+ | + | | PBUS | | + | | (Bus Controller) | | + | | | | + | | +--------------.<------------ (window always starts at + | | | PRAMIN | | | BAR0 + 0x700000) + | | | Window | | | + | | | (1MB) | | | + | | +--------------+ | | + | | | | | + | +---------|------------+ | + | | | + | v | + | .----------------------.<------------ (Program PRAMIN to any + | | VRAM | | 64KB VRAM physical boundary) + | | (Several GBs) | | + | | | | + | | FB[0x000000000000] | | + | | ... | | + | | FB[0x7FFFFFFFFFF] | | + | +----------------------+ | + +------------------------------+ + +PBUS (PCIe Bus Controller) among other things is responsible in the GPU for handling MMIO +accesses to the BAR registers. + +PRAMIN Window Operation +======================= + +The PRAMIN window provides a 1MB sliding aperture that can be repositioned over +the entire VRAM address space using the NV_PBUS_BAR0_WINDOW register. + +Window Control Mechanism +------------------------- + +The window position is controlled via the PBUS BAR0_WINDOW register:: + + NV_PBUS_BAR0_WINDOW Register + +-----+-----+--------------------------------------+ + |31-26|25-24| 23-0 | + | |TARG | BASE_ADDR | + | | ET | (bits 39:16 of VRAM address) | + +-----+-----+--------------------------------------+ + + TARGET field values: + - 0x0: VID_MEM (Video Memory / VRAM) + - 0x1: SYS_MEM_COHERENT (Coherent system memory) + - 0x2: SYS_MEM_NONCOHERENT (Non-coherent system memory) + +64KB Alignment Requirement +--------------------------- + +The PRAMIN window must be aligned to 64KB boundaries in VRAM. This is enforced +by the BASE_ADDR field representing bits [39:16] of the target address:: + + VRAM Address Calculation: + actual_vram_addr = (BASE_ADDR << 16) + pramin_offset + Where: + - BASE_ADDR: 24-bit value from NV_PBUS_BAR0_WINDOW[23:0] + - pramin_offset: 20-bit offset within PRAMIN window [0x00000-0xFFFFF] + Example Window Positioning: + +---------------------------------------------------------+ + | VRAM Space | + | | + | 0x000000000 +-----------------+ <-- 64KB aligned | + | | PRAMIN Window | | + | | (1MB) | | + | 0x0000FFFFF +-----------------+ | + | | + | | ^ | + | | | Window can slide | + | v | to any 64KB boundary | + | | + | 0x123400000 +-----------------+ <-- 64KB aligned | + | | PRAMIN Window | | + | | (1MB) | | + | 0x1234FFFFF +-----------------+ | + | | + | ... | + | | + | 0x7FFFF0000 +-----------------+ <-- 64KB aligned | + | | PRAMIN Window | | + | | (1MB) | | + | 0x7FFFFFFFF +-----------------+ | + +---------------------------------------------------------+ diff --git a/Documentation/gpu/nova/index.rst b/Documentation/gpu/nova/index.rst index 46302daace34..e77d3ee336a4 100644 --- a/Documentation/gpu/nova/index.rst +++ b/Documentation/gpu/nova/index.rst @@ -33,3 +33,4 @@ vGPU manager VFIO driver and the nova-drm driver. core/fwsec core/falcon core/msgq + core/pramin -- 2.34.1
