On Tue, Jun 16, 2026 at 09:00:08PM +0300, Petre Rodan wrote:
>
> hello Jonathan,
>
> On Tue, Jun 16, 2026 at 04:49:18PM +1000, Jonathan Gray wrote:
> > On Mon, Jun 15, 2026 at 11:07:36PM +0300, Petre Rodan wrote:
> > > On Mon, Jun 15, 2026 at 02:09:13PM +1000, Jonathan Gray wrote:
> > > > On Sun, Jun 14, 2026 at 09:52:53PM +0300, Petre Rodan wrote:
> > > > > 9:0:0: ATI Navi 21
> > > > > 0x0000: Vendor ID: 1002, Product ID: 73bf
> > > > > 0x0004: Command: 0006, Status: 0010
> > > > > 0x0008: Class: 03 Display, Subclass: 00 VGA,
> > > > > Interface: 00, Revision: c3
> > > > > 0x000c: BIST: 00, Header Type: 80, Latency Timer: 00,
> > > > > Cache Line Size: 10
> > > > > 0x0010: BAR mem prefetchable 64bit addr:
> > > > > 0x0000007800000000/0x400000000
> > > > > 0x0018: BAR mem prefetchable 64bit addr:
> > > > > 0x0000007c00000000/0x10000000
> > > > > 0x0020: BAR io addr: 0x0000e000/0x0100
> > > > > 0x0024: BAR mem 32bit addr: 0xfca00000/0x00100000
> > > >
> > > > framebuffer memory is described by pci bar 0x10
> > > >
> > > > the window size here is already 16GB, even without resizing the bar
> > > >
> > > > can you try this diff to force visible vram to 512MB?
> > > >
> > > > Index: sys/dev/pci/drm/amd/amdgpu/amdgpu_gmc.c
> > > > ===================================================================
> > > > RCS file: /cvs/src/sys/dev/pci/drm/amd/amdgpu/amdgpu_gmc.c,v
> > > > diff -u -p -r1.20 amdgpu_gmc.c
> > > > --- sys/dev/pci/drm/amd/amdgpu/amdgpu_gmc.c 25 May 2026 08:56:32
> > > > -0000 1.20
> > > > +++ sys/dev/pci/drm/amd/amdgpu/amdgpu_gmc.c 15 Jun 2026 03:53:34
> > > > -0000
> > > > @@ -216,6 +216,8 @@ void amdgpu_gmc_vram_location(struct amd
> > > > uint64_t vis_limit = (uint64_t)amdgpu_vis_vram_limit << 20;
> > > > uint64_t limit = (uint64_t)amdgpu_vram_limit << 20;
> > > >
> > > > + vis_limit = 512ULL << 20;
> > > > +
> > > > mc->vram_start = base;
> > > > mc->vram_end = mc->vram_start + mc->mc_vram_size - 1;
> > > > if (limit < mc->real_vram_size)
> > >
> > > I compared the amdgpu driver behaviour in OpenBSD vs Linux kernel 6.18.35
> > > and to me it looks like the errors start at this bsd specific #ifdef
> > > block:
> > >
> > > RCS file: /cvs/src/sys/dev/pci/drm/amd/amdgpu/amdgpu_ttm.c,v
> > > diff -u -p -r1.30 amdgpu_ttm.c
> > > --- amdgpu_ttm.c 9 Mar 2026 23:57:53 -0000 1.30
> > > +++ amdgpu_ttm.c 15 Jun 2026 19:00:39 -0000
> > > @@ -2022,x +2022,y @@ int amdgpu_ttm_init(struct amdgpu_device
> > > #else
> > > if (bus_space_map(adev->memt, adev->gmc.aper_base,
> > > adev->gmc.visible_vram_size,
> > > BUS_SPACE_MAP_LINEAR | BUS_SPACE_MAP_PREFETCHABLE,
> > > &adev->mman.aper_bsh)) {
> > > adev->mman.aper_base_kaddr = NULL;
> > > + printf("~~~ explicit NULL, visible_vram_size 0x%llX\n",
> > > adev->gmc.visible_vram_size);
> > > } else {
> > > adev->mman.aper_base_kaddr = bus_space_vaddr(adev->memt,
> > > adev->mman.aper_bsh);
> > > + printf("~~~ aper_base_kaddr %p, visible_vram_size 0x%llX\n",
> > > adev->mman.aper_base_kaddr,
> > > + adev->gmc.visible_vram_size);
> > > }
> > > #endif
> > >
> > > <TLDR>
> > > in OpenBSD if bus_space_map() is true then aper_base_kaddr is explicitly
> > > NULLed out,
> > > and for some reason this happens if visible_vram_size > 2GB.
> > > in Linux aper_base_kaddr is a non-NULL pointer in all my tests.
> > > </TLDR>
> > >
> > > # in OpenBSD with vis_limit between 512-2048MB, startx is happy to use
> > > the modsetting driver:
> > >
> > > VRAM: 16368M 0x0000008000000000 - 0x00000083FEFFFFFF (16368M used)
> > > GART: 512M 0x0000000000000000 - 0x000000001FFFFFFF
> > > ~~~ aper_base_kaddr 0xffff8000699ea000, visible_vram_size 0x80000000
> > > amdgpu0: SIENNA_CICHLID GC 10.3.0 60 CU rev 0x01
> > > amdgpu0: 1920x1080, 32bpp
> > > wsdisplay0 at amdgpu0 mux 1
> > >
> > >
> > > # in OpenBSD with vis_limit >= 2560UL<<20, aper_base_kaddr is explicitly
> > > set to NULL and the init failure chain happens:
> > >
> > > VRAM: 16368M 0x0000008000000000 - 0x00000083FEFFFFFF (16368M used)
> > > GART: 512M 0x0000000000000000 - 0x000000001FFFFFFF
> > > ~~~ explicit NULL, visible_vram_size 0xA0000000
> > > [drm] *ERROR* v11 visible_vram_size a0000000 or aper_base_kaddr 0x0 is
> > > not initialized.
> > > drm:pid0:psp_sw_init *ERROR* Failed to process memory training!
> > >
> > > drm:pid0:amdgpu_device_ip_init *ERROR* sw_init of IP block <psp> failed
> > > -22
> > > drm:pid0:amdgpu_device_init *ERROR* amdgpu_device_ip_init failed
> > >
> > > drm:pid0:amdgpu_driver_load_kms *ERROR* Fatal error during GPU init
> > > WARNING !amdgpu_irq_enabled(adev, src, type) failed at
> > > /sys/dev/pci/drm/amd/amdgpu/amdgpu_irq.c:650
> > >
> > >
> > > # in linux, even with the default unlimited vis_limit, aper_base_kaddr
> > > does not get NULLed out:
> > >
> > > [ 8.101797] amdgpu 0000:09:00.0: amdgpu: VRAM: 16368M
> > > 0x0000008000000000 - 0x00000083FEFFFFFF (16368M used)
> > > [ 8.101799] amdgpu 0000:09:00.0: amdgpu: GART: 512M 0x0000000000000000
> > > - 0x000000001FFFFFFF
> > > [ 8.101809] [drm] Detected VRAM RAM=16368M, BAR=16384M
> > > [ 8.101810] [drm] RAM width 256bits GDDR6
> > > [ 8.101908] amdgpu: ~~~ aper_base_kaddr 00000000dbe1887b,
> > > visible_vram_size 0x3FF000000
> > > [ 8.101937] amdgpu 0000:09:00.0: amdgpu: amdgpu: 16368M of VRAM memory
> > > ready
> > > [ 8.101939] amdgpu 0000:09:00.0: amdgpu: amdgpu: 16002M of GTT memory
> > > ready.
> > >
> > > am I getting warmer?
> >
> > My 8GB Vega 56 shows up with a 256M window so I can't test on hardware.
> > Can you try the below diff and report back the dmesg changes?
>
> my pleasure. thank you for taking your time.
>
> > 33:0:0: ATI Radeon Rx Vega
> > 0x0000: Vendor ID: 1002, Product ID: 687f
> > 0x0004: Command: 0007, Status: 0010
> > 0x0008: Class: 03 Display, Subclass: 00 VGA,
> > Interface: 00, Revision: c3
> > 0x000c: BIST: 00, Header Type: 80, Latency Timer: 00,
> > Cache Line Size: 10
> > 0x0010: BAR mem prefetchable 64bit addr: 0x00000000e0000000/0x10000000
> > 0x0018: BAR mem prefetchable 64bit addr: 0x00000000f0000000/0x00200000
> > 0x0020: BAR io addr: 0x0000e000/0x0100
> > 0x0024: BAR mem 32bit addr: 0xfcb00000/0x00080000
> >
> > Index: sys/dev/pci/drm/amd/amdgpu/amdgpu_drv.c
> > ===================================================================
> > RCS file: /cvs/src/sys/dev/pci/drm/amd/amdgpu/amdgpu_drv.c,v
> > diff -u -p -r1.73 amdgpu_drv.c
> > --- sys/dev/pci/drm/amd/amdgpu/amdgpu_drv.c 15 Jun 2026 02:46:12 -0000
> > 1.73
> > +++ sys/dev/pci/drm/amd/amdgpu/amdgpu_drv.c 16 Jun 2026 05:29:53 -0000
> > @@ -3409,6 +3409,8 @@ amdgpu_attach(struct device *parent, str
> > printf(": can't get framebuffer info\n");
> > return;
> > }
> > +printf("\n%s pci_mapreg_info fb_aper_offset 0x%lx fb_aper_size 0x%lx\n",
> > + __func__, adev->fb_aper_offset, adev->fb_aper_size);
> >
> > if (adev->fb_aper_offset == 0) {
> > bus_size_t start, end, pci_mem_end;
> > @@ -3432,6 +3434,8 @@ amdgpu_attach(struct device *parent, str
> > pci_conf_write(pa->pa_pc, pa->pa_tag,
> > AMDGPU_PCI_MEM + 4, (uint64_t)base >> 32);
> > adev->fb_aper_offset = base;
> > +printf("\n%s fallback fb_aper_offset 0x%lx fb_aper_size 0x%lx\n",
> > + __func__, adev->fb_aper_offset, adev->fb_aper_size);
> > }
> >
> > if (adev->family >= CHIP_BONAIRE)
> > Index: sys/arch/amd64/amd64/bus_space.c
> > ===================================================================
> > RCS file: /cvs/src/sys/arch/amd64/amd64/bus_space.c,v
> > diff -u -p -r1.31 bus_space.c
> > --- sys/arch/amd64/amd64/bus_space.c 4 Jun 2026 05:22:04 -0000
> > 1.31
> > +++ sys/arch/amd64/amd64/bus_space.c 16 Jun 2026 05:45:39 -0000
> > @@ -381,8 +381,10 @@ bus_space_map(bus_space_tag_t t, bus_add
> > */
> > error = extent_alloc_region(ex, bpa, size,
> > EX_NOWAIT | (ioport_malloc_safe ? EX_MALLOCOK : 0));
> > - if (error)
> > + if (error) {
> > +printf("\n%s extent_alloc_region error %d\n", __func__, error);
> > return (error);
> > + }
> >
> > /*
> > * For I/O space, that's all she wrote.
> > @@ -414,6 +416,7 @@ bus_space_map(bus_space_tag_t t, bus_add
> > bpa, size);
> > printf("bus_space_map: can't free region\n");
> > }
> > +printf("\n%s x86_mem_add_mapping error %d\n", __func__, error);
> > }
> >
> > return (error);
> > @@ -522,8 +525,10 @@ x86_mem_add_mapping(bus_addr_t bpa, bus_
> > map_size = endpa - pa;
> >
> > va = (vaddr_t)km_alloc(map_size, &kv_any, &kp_none, &kd_nowait);
> > - if (va == 0)
> > + if (va == 0) {
> > +printf("\n%s km_alloc ENOMEM\n", __func__);
> > return (ENOMEM);
> > + }
> >
> > *bshp = (bus_space_handle_t)(va + (bpa & PGOFSET));
>
> I took the liberty of adding a few more printfs in order to dig a bit deeper.
> and I got stuck in uvm_addr_invoke(), which calls an unknown to me
> uaddr_select function via a pointer.
> the particular pointer address being used does not seem to belong to the
> _select functions within uvm_addr.c however - see the dmesg outputs. so maybe
> there is a place where &kv_any's _select function used was created from
> scratch?
>
> attached you'll find 3 files:
> - current patch against /sys OPENBSD_7_9 for printf context
> - dmesg with vis_limit 4GB, resizebar and above 4G decoding enabled in bios
> - dmesg with vis_limit 4GB, resizebar and above 4G decoding disabled in bios
>
> I guess you're no longer interested in the 2GB vis_limit since it will not
> trigger any additional err messages.
> also I see no notable changes if I enable/disable those 2 options in bios, I
> will leave them on from now on.
>
> I am used to write code for 16bit microcontrollers so please allow me to
> accommodate a bit with this virtual memory mapping madness :)
Thanks. The problem is the size of the kernel virtual address space
on amd64.
#define VM_MIN_KERNEL_ADDRESS 0xffff800000000000
#define VM_MAX_KERNEL_ADDRESS 0xffff800100000000
Is 4GB.
This should be increased, but that may require further changes in the
part of the kernel involved in memory mapping (the pmap).
So as a first step, I will commit a change to limit the visible vram
amdgpu is using. Something like the amdgpu_gmc_vram_location() patch
you previously tested.
