Re: git grep parallelism question

2013-05-05 Thread Pete Wyckoff
torva...@linux-foundation.org wrote on Fri, 26 Apr 2013 13:31 -0700: Anyway, I think your patch is good if for no other reason that it allows this kind of testing, but at least for my machine, clearly the current default of eight threads is actually good enough. Maybe somebody with a very

Re: git grep parallelism question

2013-04-30 Thread John Keeping
On Mon, Apr 29, 2013 at 03:22:24PM -0700, Junio C Hamano wrote: John Keeping j...@keeping.me.uk writes: No, I was the one missing something (--root to be precise). But with TEST_OUTPUT_DIRECTORY you also get the result files in your temporary location, not just the trash directory. With

Re: git grep parallelism question

2013-04-30 Thread Jeff King
On Tue, Apr 30, 2013 at 09:08:49AM +0100, John Keeping wrote: With your patch, doesn't t-*.sh --root $there automatically use the fast $there temporary location as the result depot, too? No, the current code uses: $TEST_OUTPUT_DIRECTORY/$root/trash\ directory.t where we

Re: git grep parallelism question

2013-04-30 Thread John Keeping
On Tue, Apr 30, 2013 at 11:59:39AM -0400, Jeff King wrote: On Tue, Apr 30, 2013 at 09:08:49AM +0100, John Keeping wrote: With your patch, doesn't t-*.sh --root $there automatically use the fast $there temporary location as the result depot, too? No, the current code uses:

Re: git grep parallelism question

2013-04-30 Thread Jeff King
On Tue, Apr 30, 2013 at 05:12:08PM +0100, John Keeping wrote: So I'm not against TEST_OUTPUT_DIRECTORY as a concept, but I'm having trouble seeing how it is more useful than --root. I think the original intent of TEST_OUTPUT_DIRECTORY was to allow other users of the test framework (in

Re: git grep parallelism question

2013-04-29 Thread Ramkumar Ramachandra
Linus Torvalds wrote: Anyway, I think your patch is good if for no other reason that it allows this kind of testing, but at least for my machine, clearly the current default of eight threads is actually good enough. Maybe somebody with a very different machine might want to run the above

Re: git grep parallelism question

2013-04-29 Thread John Keeping
On Mon, Apr 29, 2013 at 07:35:01PM +0530, Ramkumar Ramachandra wrote: On a related note, one place that IO parallelism can provide massive benefits is in executing shell scripts. Accordingly, I always use the following commands to compile and test git respectively: make -j 8 CFLAGS=-g

Re: git grep parallelism question

2013-04-29 Thread Thomas Rast
John Keeping j...@keeping.me.uk writes: On Mon, Apr 29, 2013 at 07:35:01PM +0530, Ramkumar Ramachandra wrote: On a related note, one place that IO parallelism can provide massive benefits is in executing shell scripts. Accordingly, I always use the following commands to compile and test git

Re: git grep parallelism question

2013-04-29 Thread John Keeping
On Mon, Apr 29, 2013 at 08:04:10PM +0200, Thomas Rast wrote: John Keeping j...@keeping.me.uk writes: On Mon, Apr 29, 2013 at 07:35:01PM +0530, Ramkumar Ramachandra wrote: On a related note, one place that IO parallelism can provide massive benefits is in executing shell scripts.

Re: git grep parallelism question

2013-04-29 Thread Junio C Hamano
John Keeping j...@keeping.me.uk writes: On Mon, Apr 29, 2013 at 08:04:10PM +0200, Thomas Rast wrote: John Keeping j...@keeping.me.uk writes: On Mon, Apr 29, 2013 at 07:35:01PM +0530, Ramkumar Ramachandra wrote: On a related note, one place that IO parallelism can provide massive

Re: git grep parallelism question

2013-04-27 Thread Thomas Rast
Linus Torvalds torva...@linux-foundation.org writes: Anyway, I think your patch is good if for no other reason that it allows this kind of testing, but at least for my machine, clearly the current default of eight threads is actually good enough. Maybe somebody with a very different machine

git grep parallelism question

2013-04-26 Thread Linus Torvalds
Since I reboot fairly regularly to test new kernels, I don't *always* have the kernel source tree in my caches, so I still care about some cold-cache performance. And git grep tends to be the most noticeable one. Now, I have a SSD, and even the cold-cache case takes just five seconds or so, but

Re: git grep parallelism question

2013-04-26 Thread Junio C Hamano
Linus Torvalds torva...@linux-foundation.org writes: Wouldn't it be lovely if it was slightly smarter (something more akin to the index preloading that takes number of files into account) or at least allowed people to set the parallelism explicitly with a command line switch? Yeah, a

Re: git grep parallelism question

2013-04-26 Thread Linus Torvalds
On Fri, Apr 26, 2013 at 11:47 AM, Junio C Hamano gits...@pobox.com wrote: The real issue may be that we do not have a good estimate of how many paths are involved in the request before starting these threads, though. Yes. Also, I'm not sure if the 15% possible improvement on my SSD case is

Re: git grep parallelism question

2013-04-26 Thread Junio C Hamano
Linus Torvalds torva...@linux-foundation.org writes: Yes. Also, I'm not sure if the 15% possible improvement on my SSD case is even worth it for something that in the end isn't necessarily the common case. Cold cache being uncommon case would be forever true but more and more people are on

Re: git grep parallelism question

2013-04-26 Thread Linus Torvalds
On Fri, Apr 26, 2013 at 12:19 PM, Junio C Hamano gits...@pobox.com wrote: OK, you have to recompile at least once for experiment, so perhaps building the test binary with this patch may help. So here's my experiment on my machine with this patch and the kernel tree. First I did the warm-cache