On 10/14/13 11:52 PM, Matthew Ahrens wrote: > What happened was that the kernel addresses space got > heavily fragmented and the ARC wasn't feeling much pressure to contract, > so larger allocations started failing. > > > How did you come to that conclusion? E.g. how did you measure kernel > address space fragmentation? > > I wonder what exactly the OP meant by "VM slab allocator is quite > fragmented". It may mean that there are many buffers not in use (e.g. > as measured by ::kmastat's "buf total" - "buf in use"). This does not > indicate virtual address space fragmentation.
I came to this conclusion after analyzing the above noted deadlock situation. Also, I misspoke as to my hypothesis on what went wrong. This was recorded on a 64-bit machine that had fragmented *physical* memory and any attempt to KM_SLEEP allocate a larger buffer resulted in a deadlock (due to the chain I outlined previously). That having been said, I will freely admit that I haven't seen this happen in the field myself and I could be quite wrong on large memory allocations becoming a problem. > I see your fundamental premise as: > > In some situations, a single large memory allocation may fail, but > several smaller allocations (totaling the same amount of memory) would > succeed. This situation may occur when the zfs kernel module is loaded. > > Would you agree? Pretty much, yes. > To me, this premise is plausible in a virtual > address-constrained system (e.g. 32-bit), if the kernel module is loaded > some time after booting (e.g. on Linux). Are you addressing a 32-bit > only problem, or do you contend that the problem also exists on 64-bit > systems? My initial impression was that this problem might exist on 64-bit as well, given the above bug description. Simply pushing the memory request out to some other parts of the address space may not help due to there not *physically* being a large enough chunk. Yes, the memory manager can swap stuff out or move stuff around to fulfill it, but only if the request were KM_SLEEP (or from userspace). Perhaps simply switching over to a KM_SLEEP allocation might solve the problem (though I'd be wary of trying to allocate 1GB+ in a single request in any case). Cheers, -- Saso _______________________________________________ developer mailing list [email protected] http://lists.open-zfs.org/mailman/listinfo/developer
