Re: AMD64 buffer cache 4GB cap anything new, multiqueueing plans? ("64bit DMA on amd64" cont)

On Tue, Nov 6, 2018 at 9:51 PM Joseph Mayer wrote: > Previously there was a years-long thread about a 4GB (32bit) buffer > cache constraint on AMD64, ref > https://marc.info/?t=14682443664=1=2 . > > What I gather is, > > * The problematique is that on AMD64, DMA is limited to 32bit >

AMD64 buffer cache 4GB cap anything new, multiqueueing plans? ("64bit DMA on amd64" cont)

a system be configured to use 64bit DMA on AMD64 and be expected to work presuming recent or higher-quality / well-selected hardware?) * The OS asks the disk hardware to load disk data to give memory locations via DMA, and then userland fread() and mmap() is fed with that data

Re: 64bit DMA on amd64

On Sat, Nov 11, 2017 at 4:22 PM, wrote: > Theo 2016-07-11 15:09:48, https://marc.info/?l=openbsd- > tech=146824981122013=2 , > https://marc.info/?l=openbsd-tech=146825098022380=2 : > > And bufs don't need it either. Have you actually cranked your buffer > > cache that

Re: 64bit DMA on amd64

Theo 2016-07-11 15:09:48, https://marc.info/?l=openbsd-tech=146824981122013=2 , https://marc.info/?l=openbsd-tech=146825098022380=2 : > And bufs don't need it either. Have you actually cranked your buffer > cache that high? I have test this, on sparc64 which has unlimited DMA > reach due to the

Re: 64bit DMA on amd64

(Reply to misc@ I presume.) Hi Theo / list, Some humble followup questions regarding the previous buffer cache conversation. In particular curious what the crawl was that you saw in the very large buffer cache test you made on Sparc64? On 2016-07-12 00:16, Theo de Raadt wrote: [...] The

Re: 64bit DMA on amd64

> Except that the flipper isn't enabled yet and that the backpressure > mechanism is busted somewhow. At least that is what the recent > experiment with cranking up the buffer cache limit showed us. > People screamed and we backed the change out again. And there were > problems on amd64 and

Re: 64bit DMA on amd64

> From: "Theo de Raadt" > Date: Mon, 11 Jul 2016 09:29:16 -0600 > > > > And bufs don't need it either. Have you actually cranked your buffer > > > cache that high? I have test this, on sparc64 which has unlimited DMA > > > reach due to the iommu. The system comes to a

Re: 64bit DMA on amd64

> On Mon, 11 Jul 2016, Theo de Raadt wrote: > > > No, I didn't know that. I assumed that having a few more GBs of bufcache > > > would help the performance. Until that is the case, 64bit dma does not > > > make much sense. > > > > BTW, my tests were on a 128GB sun4v machine. Sun T5140. They

Re: 64bit DMA on amd64

On Mon, 11 Jul 2016, Theo de Raadt wrote: > > No, I didn't know that. I assumed that having a few more GBs of bufcache > > would help the performance. Until that is the case, 64bit dma does not > > make much sense. > > BTW, my tests were on a 128GB sun4v machine. Sun T5140. They are >

Re: 64bit DMA on amd64

> > And bufs don't need it either. Have you actually cranked your buffer > > cache that high? I have test this, on sparc64 which has unlimited DMA > > reach due to the iommu. The system comes to a crawl when there are > > too many mbufs or bufs, probably due to management structures unable > >

Re: 64bit DMA on amd64

On Mon, 11 Jul 2016, Theo de Raadt wrote: > > Openbsd on amd64 assumes that DMA is only possible to the lower 4GB. > > Not exactly. On an architecture-by-architecture basis, OpenBSD is > capable of insisting DMA reachable memory only lands in a smaller zone > of memory -- because it makes the

Re: 64bit DMA on amd64

> BTW, for usb devices, it probably depends on the host controller if 64bit > dma is possible or not. I guess most xhci controllers will be able to do > it. The 4GB limitation is a simple solution to a wide variety of problems. Please describe a situation where 4GB of dma memory is a

Re: 64bit DMA on amd64

> Openbsd on amd64 assumes that DMA is only possible to the lower 4GB. Not exactly. On an architecture-by-architecture basis, OpenBSD is capable of insisting DMA reachable memory only lands in a smaller zone of memory -- because it makes the other layers of code easier. > More interesting would

Re: 64bit DMA on amd64

On Mon, 11 Jul 2016, Ted Unangst wrote: > Stefan Fritsch wrote: > > On Mon, 11 Jul 2016, Reyk Floeter wrote: > > > The intentional 4GB limit is for forwarding: what if you forward mbufs > > > from a 64bit-capable interface to another one that doesn't support 64bit > > > DMA? And even if you

Re: 64bit DMA on amd64

> From: "Ted Unangst" > Date: Mon, 11 Jul 2016 10:45:19 -0400 > > Stefan Fritsch wrote: > > On Mon, 11 Jul 2016, Reyk Floeter wrote: > > > The intentional 4GB limit is for forwarding: what if you forward mbufs > > > from a 64bit-capable interface to another one that doesn't

Re: 64bit DMA on amd64

Stefan Fritsch wrote: > On Mon, 11 Jul 2016, Reyk Floeter wrote: > > The intentional 4GB limit is for forwarding: what if you forward mbufs > > from a 64bit-capable interface to another one that doesn't support 64bit > > DMA? And even if you would only enable it if all interfaces are > >

Re: 64bit DMA on amd64

> Date: Mon, 11 Jul 2016 16:10:04 +0200 (CEST) > From: Stefan Fritsch > > On Mon, 11 Jul 2016, Reyk Floeter wrote: > > The intentional 4GB limit is for forwarding: what if you forward mbufs > > from a 64bit-capable interface to another one that doesn't support 64bit > > DMA?

Re: 64bit DMA on amd64

On Mon, 11 Jul 2016, Reyk Floeter wrote: > The intentional 4GB limit is for forwarding: what if you forward mbufs > from a 64bit-capable interface to another one that doesn't support 64bit > DMA? And even if you would only enable it if all interfaces are > 64bit-capable, what if you plug in a

Re: 64bit DMA on amd64

Hi, The intentional 4GB limit is for forwarding: what if you forward mbufs from a 64bit-capable interface to another one that doesn't support 64bit DMA? And even if you would only enable it if all interfaces are 64bit-capable, what if you plug in a 32bit USB/hotplug interface? We did not want

64bit DMA on amd64

Hi, following the discussion about mbufs, I have some questions about 64bit DMA in general. Openbsd on amd64 assumes that DMA is only possible to the lower 4GB. But there are many devices (PCIe, virtio, ...) that can do DMA to the whole memory. Is it feasible to have known good devices opt-in