On 2025-09-04 at 05:45 +1000, John Hubbard <jhubb...@nvidia.com> wrote...
> On 9/1/25 4:55 PM, Alistair Popple wrote:
> > On 2025-08-30 at 09:55 +1000, John Hubbard <jhubb...@nvidia.com> wrote...
> > > On 8/27/25 1:19 AM, Alistair Popple wrote:
> ...
> > > > + // SAFETY: No DMA allocations have been made yet
> > > > + unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<48>())?
> > > > };
> > >
> > > Eventually, should be 52 bits wide, rather than 48. Or so I believe from
> > > looking at various drivers, including Nouveau (which uses 52-bit for
> > > Blackwell) and mlx* (which use a 64-bit mask).
> > >
> > > However, it works for the current series, because this series only
> > > supports
> > > Ampere GPUs, and 48 bits suffices for those.
> >
> > Actually based on both Nouveau and our internal docs this should be
> > 47-bits. I
>
> Yes. Which is why I wrote "48 bits suffices".
Sufficies most of the time, but is wrong none the less! Knowing my luck I would
run into just such a unicorn system where this would cause problems :-) So
thanks for looking at this.
> > suspect I just chose 48 during initial bring-up because that's what most
> > CPUs
> > support but neglected to add the TODO to actually go and check this. So
> > will fix
> > for v2.
> >
> > > So, you could leave this patch as-is, and I'll change 48 --> 52 in the
> > > upcoming Hopper/Blackwell series. Or you can change it now.
> >
> > We can't of course just change this to 52 bits because this needs to reflect
> > what the GPU HW supports. So ideally this needs to come from the HAL. I left
>
> I should have been more precise. I meant, "use 52 bits, via HAL, just
> like Nouveau does".
Aha. I wasn't sure so thanks for the clarification.
> > this hard-coded because in the short-term leaving it as 47 bits even for
> > Blackwell won't cause any issues. It may force usage of an IOMMU to address
> > physical addresses greater than 47-bits when it otherwise wouldn't for
> > Hopper/Blackwell (it would always have to for Ampere/Turing), but
> > short-term I
> > doubt many systems actually have physical memory above 47-bits anyway.
> >
> > In other words you could leave this as 47 bits in the upcoming
> > Hopper/Blackwell
> > series or use the HAL we have come up with (if that is available) to obtain
> > the
> > optimal value.
>
> Yes. I'm planning to match Nouveau's HAL approach for this, in the
> upcoming Hopper/Blackwell series.
Excellent, that sounds perfect.
>
> thanks,
> --
> John Hubbard
>
