Re: Drastic jump in the time required for the test suite

On Fri, Oct 28, 2016 at 3:41 AM, Eric Wong wrote: > Johannes Schindelin wrote: >> I know you are a fan of testing things thoroughly in the test suite, but I >> have to say that it is getting out of hand, in particular due to our >> over-use of shell

Re: Drastic jump in the time required for the test suite

Johannes Schindelin wrote: > I know you are a fan of testing things thoroughly in the test suite, but I > have to say that it is getting out of hand, in particular due to our > over-use of shell script idioms (which really only run fast on Linux, not > a good idea for

Re: Drastic jump in the time required for the test suite

Am 21.10.2016 um 12:59 schrieb Duy Nguyen: On Thu, Oct 20, 2016 at 11:40 PM, René Scharfe wrote: I get this on WSL with prove -j8: Files=750, Tests=13657, 906 wallclock secs ( 8.51 usr 17.17 sys + 282.62 cusr 3731.85 csys = 4040.15 CPU) And this for a run on Debian inside a

Re: Drastic jump in the time required for the test suite

Am 21.10.2016 um 15:10 schrieb Matthieu Moy: > René Scharfe writes: > >> I get this on WSL with prove -j8: >> >> Files=750, Tests=13657, 906 wallclock secs ( 8.51 usr 17.17 sys + 282.62 >> cusr 3731.85 csys = 4040.15 CPU) >> >> And this for a run on Debian inside a Hyper-V VM on

Re: Drastic jump in the time required for the test suite

René Scharfe writes: > I get this on WSL with prove -j8: > > Files=750, Tests=13657, 906 wallclock secs ( 8.51 usr 17.17 sys + 282.62 cusr > 3731.85 csys = 4040.15 CPU) > > And this for a run on Debian inside a Hyper-V VM on the same system: > > Files=759, Tests=13895, 99

Re: Drastic jump in the time required for the test suite

On Fri, Oct 21, 2016 at 3:40 AM, Dennis Kaarsemaker wrote: > On Thu, 2016-10-20 at 08:31 -0400, Jeff King wrote: > >> I'm also not entirely convinced that the test suite being a shell script >> is the main culprit for its slowness. We run git a lot of times, and >> that's

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 11:40 PM, René Scharfe wrote: > Am 20.10.2016 um 13:02 schrieb Duy Nguyen: >> On Wed, Oct 19, 2016 at 4:18 PM, Johannes Schindelin >> wrote: >>> Hi Junio, >>> >>> I know you are a fan of testing things thoroughly in the test

Re: Drastic jump in the time required for the test suite

On Fri, Oct 21, 2016 at 07:27:40AM +0200, Johannes Sixt wrote: > Am 20.10.2016 um 23:38 schrieb Jeff King: > > test_cmp () { > > # optimize for common "they are the same" case > > # without any subshells or subprograms > > We do this already on Windows; it's the

Re: Drastic jump in the time required for the test suite

Am 20.10.2016 um 23:38 schrieb Jeff King: test_cmp () { # optimize for common "they are the same" case # without any subshells or subprograms We do this already on Windows; it's the function mingw_test_cmp in our test suite:

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 05:00:13PM -0400, Jeff King wrote: > > I am not an expert on perl nor tracing, but is it feasible to find out > > how many internal calls there are? i.e. either some shell script (rebase, > > submodule) calling git itself a couple of times or even from compile/git/git > >

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 02:53:36PM -0700, Stefan Beller wrote: > >> That said I really like the idea of having a helper that would eliminate > >> the cat > >> for you, e.g. : > >> > >> git_test_helper_equal_stdin_or_diff_and_die -C super_repo status > >> --porcelain=v2 --branch

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 05:38:03PM -0400, Jeff King wrote: > I think that helper still ends up using "cat" and "diff" under the hood, > unless you write those bits in pure shell. But at that point, I suspect > we could "cat" and "test_cmp" in pure shell, something like: > [...] > Those are both

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 2:38 PM, Jeff King wrote: > On Thu, Oct 20, 2016 at 12:54:32PM -0700, Stefan Beller wrote: > >> Maybe we should stop introducing un-optimized tests. >> [...] >> * heavy use of the "git -C " pattern. When applying that >> thouroughly we'd save spanning the

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 12:54:32PM -0700, Stefan Beller wrote: > Maybe we should stop introducing un-optimized tests. > [...] > * heavy use of the "git -C " pattern. When applying that > thouroughly we'd save spanning the subshells. Yeah, I imagine with some style changes we could drop quite a

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 10:38:23PM +0200, Johannes Sixt wrote: > Am 20.10.2016 um 14:31 schrieb Jeff King: > > Close to 1/3 of those processes are just invoking the bin-wrapper > > script to set up the EXEC_PATH, etc. I imagine it would not be too hard > > to just do that in the test script. In

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 09:30:27AM -0700, Stefan Beller wrote: > On Thu, Oct 20, 2016 at 5:31 AM, Jeff King wrote: > > > > > $ perl -lne '/execve\("(.*?)"/ and print $1' /tmp/foo.out | sort | uniq -c > > | sort -rn | head > > 152271 /home/peff/compile/git/git > > 57340

Re: Drastic jump in the time required for the test suite

On Thu, 2016-10-20 at 08:31 -0400, Jeff King wrote: > I'm also not entirely convinced that the test suite being a shell script > is the main culprit for its slowness. We run git a lot of times, and > that's inherent in testing it. I ran the whole test suite under > "strace -f -e execve". There

Re: Drastic jump in the time required for the test suite

Am 20.10.2016 um 14:31 schrieb Jeff King: Close to 1/3 of those processes are just invoking the bin-wrapper script to set up the EXEC_PATH, etc. I imagine it would not be too hard to just do that in the test script. In fact, it looks like: make prefix=/wherever install

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 3:50 AM, Johannes Schindelin wrote: > Hi peff, > > On Wed, 19 Oct 2016, Jeff King wrote: > >> On Wed, Oct 19, 2016 at 10:32:12AM -0700, Junio C Hamano wrote: >> >> > > Maybe we should start optimizing the tests... >> > Maybe we should stop

Re: Drastic jump in the time required for the test suite

Junio C Hamano writes: > Are you proposing to replace the tests written as shell scripts with > scripts in another language or framework that run equivalent > sequences of git commands that is as portable as, if not more, > Bourne shell? The language (/bin/sh) is probably not

Re: Drastic jump in the time required for the test suite

Am 20.10.2016 um 13:02 schrieb Duy Nguyen: > On Wed, Oct 19, 2016 at 4:18 PM, Johannes Schindelin > wrote: >> Hi Junio, >> >> I know you are a fan of testing things thoroughly in the test suite, but I >> have to say that it is getting out of hand, in particular due to

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 5:31 AM, Jeff King wrote: > > $ perl -lne '/execve\("(.*?)"/ and print $1' /tmp/foo.out | sort | uniq -c | > sort -rn | head > 152271 /home/peff/compile/git/git > 57340 /home/peff/compile/git/t/../bin-wrappers/git > 16865 /bin/sed > 12650 /bin/rm >

Re: Drastic jump in the time required for the test suite

Johannes Schindelin writes: > Of course, if you continue to resist (because the problem is obviously not > affecting you personally, so why would you care), I won't even try to find > the time to start on that project. Sorry, but I did not know I was resisting, as I

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 12:17:33PM +0200, Johannes Schindelin wrote: > If you want to know just how harmful this reliance on shell scripting is > to our goal of keeping Git portable: already moving from Linux to MacOSX > costs you roughly 3x as long to run the build & test (~12mins vs ~36mins >

Re: Drastic jump in the time required for the test suite

On Thu, Oct 20, 2016 at 12:50:32PM +0200, Johannes Schindelin wrote: > That reflects my findings, too. I want to add that I found preciously > little difference between running slow-to-fast and running in numeric > order, so I gave up on optimizing on that front. Interesting. It makes a 10-15%

Re: Drastic jump in the time required for the test suite

On Wed, Oct 19, 2016 at 4:18 PM, Johannes Schindelin wrote: > Hi Junio, > > I know you are a fan of testing things thoroughly in the test suite, but I > have to say that it is getting out of hand, in particular due to our > over-use of shell script idioms (which really

Re: Drastic jump in the time required for the test suite

Hi peff, On Wed, 19 Oct 2016, Jeff King wrote: > On Wed, Oct 19, 2016 at 10:32:12AM -0700, Junio C Hamano wrote: > > > > Maybe we should start optimizing the tests... > > > > Yup, two things that come to mind are to identify long ones and see if > > each of them can be split into two halves

Re: Drastic jump in the time required for the test suite

Hi Junio, On Wed, 19 Oct 2016, Junio C Hamano wrote: > Johannes Schindelin writes: > > > What I can also say for certain is that the version from yesterday (commit > > 4ef44ce) was the first one in a week that built successfully and hence > > reached the test phase,

Re: Drastic jump in the time required for the test suite

On Wed, Oct 19, 2016 at 10:32:12AM -0700, Junio C Hamano wrote: > > Maybe we should start optimizing the tests... > > Yup, two things that come to mind are to identify long ones and see > if each of them can be split into two halves that can be run in > parallel, and to go through the tests with

Re: Drastic jump in the time required for the test suite

Johannes Schindelin writes: > What I can also say for certain is that the version from yesterday (commit > 4ef44ce) was the first one in a week that built successfully and hence > reached the test phase, and it was the first version of `pu` ever to time > out after

Drastic jump in the time required for the test suite

Hi Junio, I know you are a fan of testing things thoroughly in the test suite, but I have to say that it is getting out of hand, in particular due to our over-use of shell script idioms (which really only run fast on Linux, not a good idea for a portable software). My builds of `pu` now time