Re: Unexpected out of memory kills when running parallel find instances over millions of files

2023-10-21 Thread Johnny Billquist
I think I reported this something like 20 years ago, but noone really seemed to care. I noticed it pretty much right away after NetBSD switched to the unified memory thing, where all free memory usually was grabbed as disk cache. It was not fun on VAX, but at the time it seem other platforms

Re: Unexpected out of memory kills when running parallel find instances over millions of files

2023-10-21 Thread Manuel Bouyer
On Fri, Oct 20, 2023 at 10:26:05PM +0200, Reinoud Zandijk wrote: > Hi, > > On Thu, Oct 19, 2023 at 11:20:02AM +0200, Mateusz Guzik wrote: > > Running 20 find(1) instances, where each has a "private" tree with > > million of files runs into trouble with the kernel killing them (and > > others): >

Re: Unexpected out of memory kills when running parallel find instances over millions of files

2023-10-21 Thread Reinoud Zandijk
Hi, On Thu, Oct 19, 2023 at 11:20:02AM +0200, Mateusz Guzik wrote: > Running 20 find(1) instances, where each has a "private" tree with > million of files runs into trouble with the kernel killing them (and > others): > [ 785.194378] UVM: pid 1998.1998 (find), uid 0 killed: out of swap > [