On 10/20/25 11:55 AM, Joel Fernandes wrote:
> While not terribly complicated, a little bit of documentation will help
> augment the code for this very important mechanism.
>
> Signed-off-by: Joel Fernandes <[email protected]>
> ---
> Documentation/gpu/nova/core/pramin.rst | 113 +++++++++++++++++++++++++
> Documentation/gpu/nova/index.rst | 1 +
> 2 files changed, 114 insertions(+)
> create mode 100644 Documentation/gpu/nova/core/pramin.rst
Hi Joel,
Here's a more thorough review.
>
> diff --git a/Documentation/gpu/nova/core/pramin.rst
> b/Documentation/gpu/nova/core/pramin.rst
> new file mode 100644
> index 000000000000..19615e504db9
> --- /dev/null
> +++ b/Documentation/gpu/nova/core/pramin.rst
> @@ -0,0 +1,113 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=========================
> +PRAMIN aperture mechanism
> +=========================
> +
> +.. note::
> + The following description is approximate and current as of the Ampere
> family.
> + It may change for future generations and is intended to assist in
> understanding
> + the driver code.
> +
> +Introduction
> +============
> +
> +PRAMIN is a hardware aperture mechanism that provides CPU access to GPU
> Video RAM (VRAM) before
> +the GPU's Memory Management Unit (MMU) and page tables are initialized. This
> 1MB sliding window,
> +located at a fixed offset within BAR0, is essential for setting up page
> tables and other critical
> +GPU data structures without relying on the GPU's MMU.
> +
> +Architecture Overview
> +=====================
> +
> +Logically, the PRAMIN aperture mechanism is implemented by the GPU's PBUS
> (PCIe Bus Controller Unit)
How about:
The PRAMIN aperture mechanism is logically implemented by the GPU's PBUS (PCIe
Bus Controller Unit)
> +and provides a CPU-accessible window into VRAM through the PCIe interface::
> +
> + +-----------------+ PCIe +------------------------------+
> + | CPU |<----------->| GPU |
> + +-----------------+ | |
> + | +----------------------+ |
> + | | PBUS | |
> + | | (Bus Controller) | |
> + | | | |
> + | | +--------------.<------------
> (window always starts at
> + | | | PRAMIN | | |
> BAR0 + 0x700000)
> + | | | Window | | |
> + | | | (1MB) | | |
> + | | +--------------+ | |
> + | | | | |
> + | +---------|------------+ |
> + | | |
> + | v |
> + | .----------------------.<------------
> (Program PRAMIN to any
> + | | VRAM | | 64KB
> VRAM physical boundary)
> + | | (Several GBs) | |
> + | | | |
> + | | FB[0x000000000000] | |
> + | | ... | |
> + | | FB[0x7FFFFFFFFFF] | |
> + | +----------------------+ |
> + +------------------------------+
> +
> +PBUS (PCIe Bus Controller) among other things is responsible in the GPU for
> handling MMIO
How about:
PBUS (PCIe Bus Controller) is responsible for, among other things, handling MMIO
> +accesses to the BAR registers.
> +
> +PRAMIN Window Operation
> +=======================
> +
> +The PRAMIN window provides a 1MB sliding aperture that can be repositioned
> over
> +the entire VRAM address space using the NV_PBUS_BAR0_WINDOW register.
> +
> +Window Control Mechanism
> +-------------------------
> +
> +The window position is controlled via the PBUS BAR0_WINDOW register::
> +
> + NV_PBUS_BAR0_WINDOW Register
> + +-----+-----+--------------------------------------+
> + |31-26|25-24| 23-0 |
Should we use ":" notation here?
31:26 | 25:24 | 23:0
> + | |TARG | BASE_ADDR |
> + | | ET | (bits 39:16 of VRAM address) |
> + +-----+-----+--------------------------------------+
> +
> + TARGET field values:
> + - 0x0: VID_MEM (Video Memory / VRAM)
> + - 0x1: SYS_MEM_COHERENT (Coherent system memory)
> + - 0x2: SYS_MEM_NONCOHERENT (Non-coherent system memory)
These SYS_MEM_* items imply to the reader that PRAMIN can target
sysmem. However, pramin.rs in this series hard-codes this field
to VID_MEM:
let window_reg =
regs::NV_PBUS_BAR0_WINDOW::default().set_window_addr(fb_offset);
It's not clear why that is done (I seem to recall a potential deadlock
in older chips, in similar situations, but that's probably stale info).
If, however, there is some limitation that prevents using SYS_MEM*, then
we should document it, probably here.
> +
> +64KB Alignment Requirement
> +---------------------------
> +
> +The PRAMIN window must be aligned to 64KB boundaries in VRAM. This is
> enforced
> +by the BASE_ADDR field representing bits [39:16] of the target address::
> +
> + VRAM Address Calculation:
> + actual_vram_addr = (BASE_ADDR << 16) + pramin_offset
> + Where:
> + - BASE_ADDR: 24-bit value from NV_PBUS_BAR0_WINDOW[23:0]
> + - pramin_offset: 20-bit offset within PRAMIN window [0x00000-0xFFFFF]
How about:
- pramin_offset: 20-bit offset within the PRAMIN window [0x00000-0xFFFFF]
> + Example Window Positioning:
> + +---------------------------------------------------------+
> + | VRAM Space |
> + | |
> + | 0x000000000 +-----------------+ <-- 64KB aligned |
> + | | PRAMIN Window | |
> + | | (1MB) | |
> + | 0x0000FFFFF +-----------------+ |
> + | |
> + | | ^ |
> + | | | Window can slide |
> + | v | to any 64KB boundary |
Maybe:
s/any 64KB boundary/any 64KB-aligned boundary/
> + | |
> + | 0x123400000 +-----------------+ <-- 64KB aligned |
> + | | PRAMIN Window | |
> + | | (1MB) | |
> + | 0x1234FFFFF +-----------------+ |
> + | |
> + | ... |
> + | |
> + | 0x7FFFF0000 +-----------------+ <-- 64KB aligned |
> + | | PRAMIN Window | |
> + | | (1MB) | |
> + | 0x7FFFFFFFF +-----------------+ |
> + +---------------------------------------------------------+
> diff --git a/Documentation/gpu/nova/index.rst
> b/Documentation/gpu/nova/index.rst
> index 46302daace34..e77d3ee336a4 100644
> --- a/Documentation/gpu/nova/index.rst
> +++ b/Documentation/gpu/nova/index.rst
> @@ -33,3 +33,4 @@ vGPU manager VFIO driver and the nova-drm driver.
> core/fwsec
> core/falcon
> core/msgq
> + core/pramin
thanks,
--
John Hubbard
