Re: How to limit amount of virtual memory used for files (was: Re: Tuning ZFS memory usage on NetBSD - call for advice)

2022-09-24 Thread Matthias Petermann
Hello all, On 22.09.22 18:38, Mike Pumford wrote: On 22/09/2022 06:44, Lloyd Parkes wrote: Can we put together a catalogue of clearly defined problems so that we can reproduce them and investigate further? While Håvard appears to have solved his problem, I'm pretty sure I have an unused G4

Re: How to limit amount of virtual memory used for files (was: Re: Tuning ZFS memory usage on NetBSD - call for advice)

2022-09-22 Thread Mike Pumford
On 22/09/2022 06:44, Lloyd Parkes wrote: Can we put together a catalogue of clearly defined problems so that we can reproduce them and investigate further? While Håvard appears to have solved his problem, I'm pretty sure I have an unused G4 Mac Mini of my own that I can try and reproduce

Re: How to limit amount of virtual memory used for files (was: Re: Tuning ZFS memory usage on NetBSD - call for advice)

2022-09-22 Thread Michael van Elst
ll...@must-have-coffee.gen.nz (Lloyd Parkes) writes: >HÃ¥vard's email about his G4 Mac Mini is an excellent example >of a problem. A problem I have experienced in the past was a program >failing with out of memory errors while processing 128MB of data on a >system with 256MB of RAM. A problem

Re: How to limit amount of virtual memory used for files (was: Re: Tuning ZFS memory usage on NetBSD - call for advice)

2022-09-21 Thread Lloyd Parkes
I wrote a quick and dirty program to trigger vast amounts of File memory usage only to remind myself of something important. Having File memory use all available RAM with only scraps left is not a problem by itself. The memory is there to be used and it is getting used. I used to have to

Re: How to limit amount of virtual memory used for files (was: Re: Tuning ZFS memory usage on NetBSD - call for advice)

2022-09-20 Thread RVP
On Tue, 20 Sep 2022, Matthias Petermann wrote: In theory this looked exactly what I was looking for... unfortunately I can observe the same effect like I was doing before with the simple redirection. "File" grows to a maximum, the system starts to swap. The system is still buffering even

Re: How to limit amount of virtual memory used for files (was: Re: Tuning ZFS memory usage on NetBSD - call for advice)

2022-09-20 Thread tlaronde
Hello, Le Tue, Sep 20, 2022 at 01:19:17PM +0200, Matthias Petermann a écrit : > Hi, > > Am 20.09.2022 um 08:27 schrieb RVP: > > On Tue, 20 Sep 2022, Matthias Petermann wrote: > > > > > I think I had answered this earlier (but I'm not quite sure) - the > > > problem only occurs when I write the

Re: How to limit amount of virtual memory used for files (was: Re: Tuning ZFS memory usage on NetBSD - call for advice)

2022-09-20 Thread Matthias Petermann
Hi, Am 20.09.2022 um 08:27 schrieb RVP: On Tue, 20 Sep 2022, Matthias Petermann wrote: I think I had answered this earlier (but I'm not quite sure) - the problem only occurs when I write the data obtained with "zfs send" to a local file system (e.g. the USB HDD). If I send the data to a

Re: How to limit amount of virtual memory used for files (was: Re: Tuning ZFS memory usage on NetBSD - call for advice)

2022-09-20 Thread RVP
On Tue, 20 Sep 2022, Matthias Petermann wrote: I think I had answered this earlier (but I'm not quite sure) - the problem only occurs when I write the data obtained with "zfs send" to a local file system (e.g. the USB HDD). If I send the data to a remote system with netcat instead, the "file

How to limit amount of virtual memory used for files (was: Re: Tuning ZFS memory usage on NetBSD - call for advice)

2022-09-20 Thread Matthias Petermann
Hello all, Am 31.08.2022 um 21:57 schrieb Lloyd Parkes: It might not be ZFS related. But it could be. Someone else reported excessive, ongoing, increasing "File" usage a while back and I was somewhat dismissive because they were running a truckload of apps at the same time (not in VMs). I

Re: Tuning ZFS memory usage on NetBSD - call for advice

2022-09-03 Thread Matthias Petermann
Hi Lloyd, On 31.08.22 21:57, Lloyd Parkes wrote: It might not be ZFS related. But it could be. Someone else reported excessive, ongoing, increasing "File" usage a while back and I was somewhat dismissive because they were running a truckload of apps at the same time (not in VMs). I did

Re: Tuning ZFS memory usage on NetBSD - call for advice

2022-08-31 Thread Lloyd Parkes
It might not be ZFS related. But it could be. Someone else reported excessive, ongoing, increasing "File" usage a while back and I was somewhat dismissive because they were running a truckload of apps at the same time (not in VMs). I did manage to reproduce his problem on an empty non-ZFS

Tuning ZFS memory usage on NetBSD - call for advice

2022-08-31 Thread Matthias Petermann
Hello all, under [1] is described in the section "Memory usage", which requirements ZFS has for the memory. It further mentions that the tunables that exist in FreeBSD do not exist in NetBSD. Especially for the size of the ARC there seems to be no limit for NetBSD: "vfs.zfs.arc_max -