Re: Re: [PATCH] Reduce memory usage

2021-10-02 Thread Jindřich Makovička via rsync
e even worse. > > RG > > > > > Original Message > On Sep 26, 2021, 13:54, Jindřich Makovička via rsync < > rsync@lists.samba.org> wrote: > Hi, > > When using rsync to back up the file system on my laptop, containing a > pretty much default

[PATCH] Reduce memory usage

2021-09-28 Thread Jindřich Makovička via rsync
In 2004, an allocation optimization has been added to the file list handling code, that preallocates 32k of file_struct pointers in a file_list. This optimization predates the incremental recursion feature, for which it is not appropriate anymore. When copying a tree containing a large number of

Re: [PATCH] Reduce memory usage

2021-09-28 Thread Jindřich Makovička via rsync
On Mon, 27 Sep 2021 22:03:01 +0200 Johannes Altmanninger wrote: > > On Sun, Sep 26, 2021 at 01:54:13PM +0200, Jindřich Makovička via > > rsync wrote: > > > > Applying the attached patch, which reduces the default allocation > > to 32 pointers, and preallocates

Re: [PATCH] Reduce memory usage

2021-09-27 Thread Jindřich Makovička via rsync
On Mon, 27 Sep 2021 13:38:22 +0200 Jindřich Makovička wrote: > On Sun, 26 Sep 2021 13:54:13 +0200 > Jindřich Makovička wrote: > > Hi, > > > > ... > > > > Applying the attached patch, which reduces the default allocation to > > 32 pointers, and preallocates 32K pointers only for the main file >

Re: [PATCH] Reduce memory usage

2021-09-27 Thread Jindřich Makovička via rsync
On Sun, 26 Sep 2021 13:54:13 +0200 Jindřich Makovička wrote: > Hi, > > ... > > Applying the attached patch, which reduces the default allocation to > 32 pointers, and preallocates 32K pointers only for the main file > lists in send_file_list and recv_file_list, reduces the peak memory > usage in

[PATCH] Reduce memory usage

2021-09-26 Thread Jindřich Makovička via rsync
Hi, When using rsync to back up the file system on my laptop, containing a pretty much default linux desktop, I was wondering how rsync uses over 100MB of RAM it allocates. It turned out that most of the memory is used for the arrays of file_struct pointers, most of which end up unused - much