Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-21 Thread David Kastrup
Junio C Hamano gits...@pobox.com writes: David Kastrup d...@gnu.org writes: Junio C Hamano gits...@pobox.com writes: Duy Nguyen pclo...@gmail.com writes: The only downside I see is large blobs will be packed undeltified, which could increase pack size if you have lots of them. I think

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-21 Thread David Kastrup
Jeff King p...@peff.net writes: On Wed, Mar 19, 2014 at 01:38:32PM +0100, David Kastrup wrote: The default of 16MiB causes serious thrashing for large delta chains combined with large files. Does it make much sense to bump this without also bumping MAX_DELTA_CACHE in sha1_file.c? In my

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-21 Thread Duy Nguyen
On Fri, Mar 21, 2014 at 1:04 PM, David Kastrup d...@gnu.org wrote: Hmm, doesn't packing need to read existing data? Judging from the frequent out-of-memory conditions of git gc --aggressive, packing is not restrained by deltaBaseCacheLimit. pack-objects memory usage is more controlled by

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-21 Thread David Kastrup
Junio C Hamano gits...@pobox.com writes: I know that the 512MiB default for the bitFileThreashold (aka forget about delta compression) came out of thin air. It was just 1GB is always too huge for anybody, so let's cut it in half and declare that value the initial version of a sane

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-21 Thread David Kastrup
Jeff King p...@peff.net writes: If you have before-and-after numbers for just this patch on some repository, that would be an interesting thing to put in the commit message. It's a hen-and-egg problem regarding the benchmarks. The most impressive benchmarks arise with the git-blame

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-20 Thread Junio C Hamano
Duy Nguyen pclo...@gmail.com writes: On Thu, Mar 20, 2014 at 5:11 AM, Junio C Hamano gits...@pobox.com wrote: ... I know that the 512MiB default for the bitFileThreashold (aka forget about delta compression) came out of thin air. It was just 1GB is always too huge for anybody, so let's cut

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-20 Thread David Kastrup
Junio C Hamano gits...@pobox.com writes: Duy Nguyen pclo...@gmail.com writes: The only downside I see is large blobs will be packed undeltified, which could increase pack size if you have lots of them. I think that is something that can be tweaked, unless the user tells us otherwise via

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-20 Thread Junio C Hamano
David Kastrup d...@gnu.org writes: Junio C Hamano gits...@pobox.com writes: Duy Nguyen pclo...@gmail.com writes: The only downside I see is large blobs will be packed undeltified, which could increase pack size if you have lots of them. I think that is something that can be tweaked,

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-20 Thread Jeff King
On Wed, Mar 19, 2014 at 01:38:32PM +0100, David Kastrup wrote: The default of 16MiB causes serious thrashing for large delta chains combined with large files. Does it make much sense to bump this without also bumping MAX_DELTA_CACHE in sha1_file.c? In my measurements of linux.git, bumping the

[PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-19 Thread David Kastrup
The default of 16MiB causes serious thrashing for large delta chains combined with large files. Signed-off-by: David Kastrup d...@gnu.org --- Forgot the signoff. For the rationale of this patch and the 128MiB choice, see the original patch. Documentation/config.txt | 2 +- environment.c

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-19 Thread Junio C Hamano
David Kastrup d...@gnu.org writes: The default of 16MiB causes serious thrashing for large delta chains combined with large files. Signed-off-by: David Kastrup d...@gnu.org --- Is that a good argument? Wouldn't the default of 128MiB burden smaller machines with bloated processes? Forgot

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-19 Thread David Kastrup
Junio C Hamano gits...@pobox.com writes: David Kastrup d...@gnu.org writes: The default of 16MiB causes serious thrashing for large delta chains combined with large files. Signed-off-by: David Kastrup d...@gnu.org --- Is that a good argument? Wouldn't the default of 128MiB burden

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-19 Thread Junio C Hamano
David Kastrup d...@gnu.org writes: Junio C Hamano gits...@pobox.com writes: David Kastrup d...@gnu.org writes: The default of 16MiB causes serious thrashing for large delta chains combined with large files. Signed-off-by: David Kastrup d...@gnu.org --- Is that a good argument?

Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

2014-03-19 Thread Duy Nguyen
On Thu, Mar 20, 2014 at 5:11 AM, Junio C Hamano gits...@pobox.com wrote: David Kastrup d...@gnu.org writes: Junio C Hamano gits...@pobox.com writes: David Kastrup d...@gnu.org writes: The default of 16MiB causes serious thrashing for large delta chains combined with large files.