On Thu, Feb 08, 2018 at 07:50:22PM +0100, Ole Tange wrote: > In practice this means you need 2.3 GB of free RAM and 3.8 GB of free > virtual memory (RAM+swap) to run this. > > Given your output from 'free' I think that if you add 2 GB of swap, > then that will solve your issue. Just because Perl loves swap, it does > not mean that it will start swapping - it just wants to feel safe that > it could if it wanted to. I have seen the same behaviour of other > programs, so my guess is that you might get more performance from your > system simply by adding a little swap space. > > The other option is to use --cat as described on > http://lists.gnu.org/archive/html/parallel/2018-01/msg00027.html
Yes, but --cat creates temp files, which I can't have as it slows down things considerably. Anyway - looks like I simply can't use parallel. It's memory requirements are (from my perspective) impossible to handle, and I still don't understand what exactly is the benefit of buffering stuff in memory. depesz
