Re: "git grep" parallelism question

2013-05-05 Thread Pete Wyckoff
torva...@linux-foundation.org wrote on Fri, 26 Apr 2013 13:31 -0700: > Anyway, I think your patch is good if for no other reason that it > allows this kind of testing, but at least for my machine, clearly the > current default of eight threads is actually "good enough". Maybe > somebody with a very

Re: "git grep" parallelism question

2013-04-30 Thread Jeff King
On Tue, Apr 30, 2013 at 05:12:08PM +0100, John Keeping wrote: > > So I'm not against TEST_OUTPUT_DIRECTORY as a concept, but I'm having > > trouble seeing how it is more useful than "--root". > > I think the original intent of TEST_OUTPUT_DIRECTORY was to allow other > users of the test framework

Re: "git grep" parallelism question

2013-04-30 Thread John Keeping
On Tue, Apr 30, 2013 at 11:59:39AM -0400, Jeff King wrote: > On Tue, Apr 30, 2013 at 09:08:49AM +0100, John Keeping wrote: > > > > With your patch, doesn't "t-*.sh --root $there" automatically > > > use the fast $there temporary location as the result depot, too? > > > > No, the current code

Re: "git grep" parallelism question

2013-04-30 Thread Jeff King
On Tue, Apr 30, 2013 at 09:08:49AM +0100, John Keeping wrote: > > With your patch, doesn't "t-*.sh --root $there" automatically > > use the fast $there temporary location as the result depot, too? > > No, the current code uses: > > $TEST_OUTPUT_DIRECTORY/$root/trash\ directory.t > >

Re: "git grep" parallelism question

2013-04-30 Thread John Keeping
On Mon, Apr 29, 2013 at 03:22:24PM -0700, Junio C Hamano wrote: > John Keeping writes: > > No, I was the one missing something (--root to be precise). But with > > TEST_OUTPUT_DIRECTORY you also get the result files in your temporary > > location, not just the trash directory. > > With your patc

Re: "git grep" parallelism question

2013-04-29 Thread Junio C Hamano
John Keeping writes: > On Mon, Apr 29, 2013 at 08:04:10PM +0200, Thomas Rast wrote: >> John Keeping writes: >> >> > On Mon, Apr 29, 2013 at 07:35:01PM +0530, Ramkumar Ramachandra wrote: >> >> On a related note, one place that IO parallelism can provide massive >> >> benefits is in executing she

Re: "git grep" parallelism question

2013-04-29 Thread John Keeping
On Mon, Apr 29, 2013 at 08:04:10PM +0200, Thomas Rast wrote: > John Keeping writes: > > > On Mon, Apr 29, 2013 at 07:35:01PM +0530, Ramkumar Ramachandra wrote: > >> On a related note, one place that IO parallelism can provide massive > >> benefits is in executing shell scripts. Accordingly, I al

Re: "git grep" parallelism question

2013-04-29 Thread Thomas Rast
John Keeping writes: > On Mon, Apr 29, 2013 at 07:35:01PM +0530, Ramkumar Ramachandra wrote: >> On a related note, one place that IO parallelism can provide massive >> benefits is in executing shell scripts. Accordingly, I always use the >> following commands to compile and test git respectively

Re: "git grep" parallelism question

2013-04-29 Thread John Keeping
On Mon, Apr 29, 2013 at 07:35:01PM +0530, Ramkumar Ramachandra wrote: > On a related note, one place that IO parallelism can provide massive > benefits is in executing shell scripts. Accordingly, I always use the > following commands to compile and test git respectively: > > make -j 8 CFLAGS=

Re: "git grep" parallelism question

2013-04-29 Thread Ramkumar Ramachandra
Linus Torvalds wrote: > Anyway, I think your patch is good if for no other reason that it > allows this kind of testing, but at least for my machine, clearly the > current default of eight threads is actually "good enough". Maybe > somebody with a very different machine might want to run the above

Re: "git grep" parallelism question

2013-04-27 Thread Thomas Rast
Linus Torvalds writes: > Anyway, I think your patch is good if for no other reason that it > allows this kind of testing, but at least for my machine, clearly the > current default of eight threads is actually "good enough". Maybe > somebody with a very different machine might want to run the abo

Re: "git grep" parallelism question

2013-04-26 Thread Linus Torvalds
On Fri, Apr 26, 2013 at 12:19 PM, Junio C Hamano wrote: > > OK, you have to recompile at least once for experiment, so perhaps > building the test binary with this patch may help. So here's my experiment on my machine with this patch and the kernel tree. First I did the warm-cache case: for i

Re: "git grep" parallelism question

2013-04-26 Thread Junio C Hamano
Linus Torvalds writes: > Yes. Also, I'm not sure if the 15% possible improvement on my SSD case > is even worth it for something that in the end isn't necessarily the > common case. Cold cache being uncommon case would be forever true but more and more people are on SSD, and 15% is not a trivial

Re: "git grep" parallelism question

2013-04-26 Thread Linus Torvalds
On Fri, Apr 26, 2013 at 11:47 AM, Junio C Hamano wrote: > > The real issue may be that we do not have a good estimate of how > many paths are involved in the request before starting these > threads, though. Yes. Also, I'm not sure if the 15% possible improvement on my SSD case is even worth it fo

Re: "git grep" parallelism question

2013-04-26 Thread Junio C Hamano
Linus Torvalds writes: > Wouldn't it be lovely if it was slightly smarter (something more akin > to the index preloading that takes number of files into account) or at > least allowed people to set the parallelism explicitly with a command > line switch? Yeah, a reasonable starting point for aut

"git grep" parallelism question

2013-04-26 Thread Linus Torvalds
Since I reboot fairly regularly to test new kernels, I don't *always* have the kernel source tree in my caches, so I still care about some cold-cache performance. And "git grep" tends to be the most noticeable one. Now, I have a SSD, and even the cold-cache case takes just five seconds or so, but