Re: contigmalloc() breaking Xorg
On 2012-Aug-06 10:16:13 -0400, John Baldwin wrote:
>On Thursday, July 12, 2012 8:26:05 am John Baldwin wrote:
>> However, rather add a wiredmalloc(), I think you should just have
>> bus_dmamem_alloc() call kmem_alloc_attr() directly in this case. One of the
>> things I've been meaning to add to bus_dma is a way to allocate other memory
>> types (e.g. WC memory), and in that case it would be best to just call
>> kmem_alloc_attr() directly instead.
>
>After my recent changes, I think this would do the above. What do
>you think of this, and if it looks ok, can you test it?
>
>Index: busdma_machdep.c
>===
>--- busdma_machdep.c (revision 239020)
>+++ busdma_machdep.c (working copy)
>@@ -533,13 +533,14 @@ bus_dmamem_alloc(bus_dma_tag_t dmat, void** vaddr,
> dmat->lowaddr >= ptoa((vm_paddr_t)Maxmem) &&
> attr == VM_MEMATTR_DEFAULT) {
> *vaddr = malloc(dmat->maxsize, M_DEVBUF, mflags);
>+ } else if (dmat->nsegments >= btoc(dmat->maxsize) &&
>+ dmat->alignment <= PAGE_SIZE &&
>+ (dmat->boundary == 0 || dmat->boundary >= dmat->lowaddr)) {
>+ /* Page-based multi-segment allocations allowed */
>+ *vaddr = (void *)kmem_alloc_attr(kernel_map, dmat->maxsize,
>+ mflags, 0ul, dmat->lowaddr, attr);
>+ *mapp = &contig_dmamap;
> } else {
>- /*
>- * XXX Use Contigmalloc until it is merged into this facility
>- * and handles multi-seg allocations. Nobody is doing
>- * multi-seg allocations yet though.
>- * XXX Certain AGP hardware does.
>- */
> *vaddr = (void *)kmem_alloc_contig(kernel_map, dmat->maxsize,
> mflags, 0ul, dmat->lowaddr, dmat->alignment ?
> dmat->alignment : 1ul, dmat->boundary, attr);
I've tried this on a -current/amd64 box (r239130) that has 2GB RAM and
ZFS. I looped tar(1)s of several bits of /usr/src whilst doing a "-j2
buildworld" and killing X every 30s for about 7 hours. (The Xserver
grabs a 8192-page dmamem tag when it starts). It all seemed to work OK.
I haven't tried it on the box where I originally saw the problem
because that's running 8.x. I'll have a look at backporting your
and alc@'s fixes when I get some spare time.
--
Peter Jeremy
pgpNNsdrw2Bk7.pgp
Description: PGP signature
Re: contigmalloc() breaking Xorg
On Thursday, July 12, 2012 8:26:05 am John Baldwin wrote:
> However, rather add a wiredmalloc(), I think you should just have
> bus_dmamem_alloc() call kmem_alloc_attr() directly in this case. One of the
> things I've been meaning to add to bus_dma is a way to allocate other memory
> types (e.g. WC memory), and in that case it would be best to just call
> kmem_alloc_attr() directly instead.
After my recent changes, I think this would do the above. What do you think of
this, and if it looks ok, can you test it?
Index: busdma_machdep.c
===
--- busdma_machdep.c(revision 239020)
+++ busdma_machdep.c(working copy)
@@ -533,13 +533,14 @@ bus_dmamem_alloc(bus_dma_tag_t dmat, void** vaddr,
dmat->lowaddr >= ptoa((vm_paddr_t)Maxmem) &&
attr == VM_MEMATTR_DEFAULT) {
*vaddr = malloc(dmat->maxsize, M_DEVBUF, mflags);
+ } else if (dmat->nsegments >= btoc(dmat->maxsize) &&
+ dmat->alignment <= PAGE_SIZE &&
+ (dmat->boundary == 0 || dmat->boundary >= dmat->lowaddr)) {
+ /* Page-based multi-segment allocations allowed */
+ *vaddr = (void *)kmem_alloc_attr(kernel_map, dmat->maxsize,
+ mflags, 0ul, dmat->lowaddr, attr);
+ *mapp = &contig_dmamap;
} else {
- /*
-* XXX Use Contigmalloc until it is merged into this facility
-* and handles multi-seg allocations. Nobody is doing
-* multi-seg allocations yet though.
-* XXX Certain AGP hardware does.
-*/
*vaddr = (void *)kmem_alloc_contig(kernel_map, dmat->maxsize,
mflags, 0ul, dmat->lowaddr, dmat->alignment ?
dmat->alignment : 1ul, dmat->boundary, attr);
--
John Baldwin
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: contigmalloc() breaking Xorg
On 07/12/2012 07:26, John Baldwin wrote:
[ Adding alc@ for VM stuff, Warner for arm/mips bus dma brokenness ]
When the code underlying contigmalloc() fails in its initial attempt to
allocate memory and proceeds to launder and reclaim pages, it should
almost certainly do as the page daemon does and invoke the vm_lowmem
handlers. In particular, this should coax the ZFS ARC into releasing
some of its hoard of wired memory. Try this:
Index: vm/vm_contig.c
===
--- vm/vm_contig.c (revision 238372)
+++ vm/vm_contig.c (working copy)
@@ -192,6 +192,18 @@ vm_contig_grow_cache(int tries, vm_paddr_t low, vm
{
int actl, actmax, inactl, inactmax;
+ if (tries > 0) {
+ /*
+* Decrease registered cache sizes.
+*/
+ EVENTHANDLER_INVOKE(vm_lowmem, 0);
+
+ /*
+* We do this explicitly after the caches have been drained
+* above.
+*/
+ uma_reclaim();
+ }
vm_page_lock_queues();
inactl = 0;
inactmax = tries < 1 ? 0 : cnt.v_inactive_count;
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: contigmalloc() breaking Xorg
[ Adding alc@ for VM stuff, Warner for arm/mips bus dma brokenness ] On Sunday, July 08, 2012 7:05:16 am Peter Jeremy wrote: > On 2012-Jul-03 21:17:53 +1000, Peter Jeremy wrote: > >I have a reasonably recent 8-stable/amd64 system (r237444) with a "ATI > >Radeon HD 2400 Pro", xorg-server-1.10.6,1 and xf86-video-ati-6.14.3_1 > >8GB RAM and ZFS. I'm seeing fairly consistent problems with Xorg > ... > >How difficult would it be to modify bus_dmamem_alloc() [at least on > >x86] to handle multi-segment allocations? > > I think I've managed to create an amd64 bus_dmamem_alloc() that allows > page-sized allocations as long as no boundary condition is specified > and no more than page-sized alignment is required (porting it to other > architectures would be trivial). I've given it a quick whirl inside a > VBox and no smoke came out but I'd appreciate someone with a better > understanding of bus_dma(9) and vm/vm_contig.c giving > http://www.rulingia.com/bugs/patch-wiredmalloc a once-over. Note that > this patch is against 8.x but there's only a trivial difference to head. > > BTW, the comment in busdma_machdep.c:bus_dmamem_alloc() >* XXX Use Contigmalloc until it is merged into this facility >* and handles multi-seg allocations. Nobody is doing >* multi-seg allocations yet though. >* XXX Certain AGP hardware does. > does not appear to be accurate. Apart from drm, quite a few drivers > call bus_dma_tag_create(9) with multiple segments and also call > bus_dmamem_alloc(9) [though I haven't verified that the calls share > the same bus_dma_tag, so I can't be absolutely certain]. I do think that all tags currently used with bus_dmamem_alloc() only use a single segment. It's a bit of an unfortunate part of the bus_dmamem API that the size of the allocate is determined by the tag (the tag should be used for determining the features and constraints of a DMA engine, not really the amount of memory to allocate). However, rather add a wiredmalloc(), I think you should just have bus_dmamem_alloc() call kmem_alloc_attr() directly in this case. One of the things I've been meaning to add to bus_dma is a way to allocate other memory types (e.g. WC memory), and in that case it would be best to just call kmem_alloc_attr() directly instead. > BTW(2): Whilst studying busdma_machdep.c for arm and mips, I've > noticed they appear to potentially allocate substantial kernel stack > under some conditions as several bus_dma(9) functions include: > bus_dma_segment_t dm_segments[dmat->nsegments]; > What prevents this overflowing the kernel stack? That does seem dubious. x86 stores the array in the tag instead. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: contigmalloc() breaking Xorg
On 2012-Jul-03 21:17:53 +1000, Peter Jeremy wrote: >I have a reasonably recent 8-stable/amd64 system (r237444) with a "ATI >Radeon HD 2400 Pro", xorg-server-1.10.6,1 and xf86-video-ati-6.14.3_1 >8GB RAM and ZFS. I'm seeing fairly consistent problems with Xorg ... >How difficult would it be to modify bus_dmamem_alloc() [at least on >x86] to handle multi-segment allocations? I think I've managed to create an amd64 bus_dmamem_alloc() that allows page-sized allocations as long as no boundary condition is specified and no more than page-sized alignment is required (porting it to other architectures would be trivial). I've given it a quick whirl inside a VBox and no smoke came out but I'd appreciate someone with a better understanding of bus_dma(9) and vm/vm_contig.c giving http://www.rulingia.com/bugs/patch-wiredmalloc a once-over. Note that this patch is against 8.x but there's only a trivial difference to head. BTW, the comment in busdma_machdep.c:bus_dmamem_alloc() * XXX Use Contigmalloc until it is merged into this facility * and handles multi-seg allocations. Nobody is doing * multi-seg allocations yet though. * XXX Certain AGP hardware does. does not appear to be accurate. Apart from drm, quite a few drivers call bus_dma_tag_create(9) with multiple segments and also call bus_dmamem_alloc(9) [though I haven't verified that the calls share the same bus_dma_tag, so I can't be absolutely certain]. BTW(2): Whilst studying busdma_machdep.c for arm and mips, I've noticed they appear to potentially allocate substantial kernel stack under some conditions as several bus_dma(9) functions include: bus_dma_segment_t dm_segments[dmat->nsegments]; What prevents this overflowing the kernel stack? -- Peter Jeremy pgp36kekzDWj0.pgp Description: PGP signature
Re: contigmalloc() breaking Xorg
On 2012-Jul-03 21:17:53 +1000, Peter Jeremy wrote: >Does anyone have a tool that can display physical RAM allocation? >This would at least allow me to identify offending allocations. >http://lists.freebsd.org/pipermail/freebsd-hackers/2011-February/thread.html >asks the same question but just peters out. That link should be http://lists.freebsd.org/pipermail/freebsd-hackers/2011-February/034321.html sorry about the mis-paste. -- Peter Jeremy pgp1iwrLyMm1P.pgp Description: PGP signature
contigmalloc() breaking Xorg
I have a reasonably recent 8-stable/amd64 system (r237444) with a "ATI Radeon HD 2400 Pro", xorg-server-1.10.6,1 and xf86-video-ati-6.14.3_1 8GB RAM and ZFS. I'm seeing fairly consistent problems with Xorg spinning in swwrt for long periods (I've seen ½hr) and then failing. The resultant Xorg.0.log shows (eg): [854259.962] (EE) RADEON(0): [pci] Out of memory (-12) That message comes from xf86-video-ati-6.14.3/src/radeon_dri.c:RADEONDRIPciInit() and the -12 indicates ENOMEM. That code (indirectly) issues DRM_IOCTL_SG_ALLOC and winds up in sys/dev/drm/drm_scatter.c:drm_sg_alloc(), which uses bus_dma_tag_create(), bus_dmamem_alloc() and bus_dmamap_load() to actually allocate memory below 4GB. Setting hw.dri.0.debug shows that it's trying to allocate 32MB: Jul 3 18:57:49 server kernel: [drm:pid72128:drm_ioctl] pid=72128, cmd=0xc0106438, nr=0x38, dev 0xff000246ee00, auth=1 Jul 3 18:57:49 server kernel: [drm:pid72128:drm_sg_alloc_ioctl] Jul 3 18:57:49 server kernel: [drm:pid72128:drm_sg_alloc] sg size=33554432 pages=8192 Jul 3 19:28:09 server kernel: [drm:pid72128:drm_ioctl] returning 12 [note the timestamps] Whilst drm_sg_alloc() allows non-contiguous allocation (the code just wants 8192 pages), bus_dma(9) states: "The current implementation of bus_dmamem_alloc() will allocate all requests as a single segment." (and this is the same in 10-current). bus_dmamem_alloc() for a region greater than one page uses contigmalloc(). I believe that Xorg spinning in swwrt is a regression but I don't have a good idea for when it started (and http://lists.freebsd.org/pipermail/freebsd-stable/2011-February/061369.html suggests that it's been occurring for quite a while). For that matter contigmalloc() also seems to have a long history of causing problems with other parts of FreeBSD - I first crossed swords with it 7½ years ago (when it was causing panics in umass(4)). Previously, the work-around for contigmalloc() issues was to ensure that the appropriate contigmalloc() calls occurred shortly after a reboot - before RAM got too fragmented. That doesn't appear to work here because it looks like Xorg releases and (tries to) re-allocates the memory during a reset (ie on logout). It is a _serious_ nuisance having to reboot because I fumbled my password... Can anyone suggest a way forward? Note that additional RAM isn't an option for this box. How difficult would it be to modify bus_dmamem_alloc() [at least on x86] to handle multi-segment allocations? Does anyone have a tool that can display physical RAM allocation? This would at least allow me to identify offending allocations. http://lists.freebsd.org/pipermail/freebsd-hackers/2011-February/thread.html asks the same question but just peters out. -- Peter Jeremy pgpuclQzdFSg9.pgp Description: PGP signature
