Re: Request for testing v2.19.0-rc0 *with builtin stash/rebase*
Hi Bryan, On Fri, 24 Aug 2018, Bryan Turner wrote: > On Fri, Aug 24, 2018 at 5:14 AM Johannes Schindelin > wrote: > > > > For that reason, I was delighted to see that our Google Summer of Code > > pushed pretty hard in that direction. And I could not help myself so I had > > to test how much faster things got. Here is the result of my first, really > > quick and dirty test: > > > > without builtin stash/rebasewith builtin stash/rebase > > t3400 (rebase) 1m27s 32s > > t3404 (rebase -i) 13m15s 3m59s > > t3903 (stash) 8m37s 1m18s > > > > What can I say? Even if the numbers are off by as much as 10%, these are > > impressive improvements: keep in mind that there is a lot of churn going > > on in the test suite because it is itself implemented in Unix shell > > scripts (and hence I won't even bother to try more correct performance > > benchmarking because that is simply not possible when Unix shell scripts > > are in the equation). So the speed improvements of the stash/rebase > > commands are *even higher* than this. > > Thanks for taking the time to make these pre-releases available. I > appreciate the effort. And the same to Junio, for always posting > release candidates. We rely on them heavily to find changes that might > cause issues before admins start upgrading in the wild and find them > for us. > > I downloaded both the rc0.1 and rc0.2 builds, as well as 2.18.0, and > ran them all through Bitbucket Server's test suite a few times (to > ensure warm disk for comparable numbers). I added support for some > "simple" rebase cases a few releases ago, so we have a set of tests > that verify the rebase behaviors we use. (We don't use git stash, so > we don't have any tests in our suite for that.) > > Running our entire Git test suite (~1,600 tests) against Git for > Windows 2.18.0 takes ~5 minutes, and 2.19.0-rc0.1 produced an almost > identical duration. Running our tests against rc0.2 cut the duration > down to 4 minutes. There were no test failures on either pre-release > build. > > To try and get a better sense of the rebase performance improvement > specifically, I filtered down to a set of 14 specific tests which use > it and ran those. On Git 2.18, those 14 tests take just over 19 > seconds. On 2.19.0-rc0.2, they take just over 8 seconds. > > When they do ship, whether it's in 2.19 (by default or optional) or > later, the changes definitely offer some significant performance wins. > > Thanks again, to everyone involved, for all the effort that went into > designing, implementing, reviewing and releasing these improvements. > As someone who develops under Windows most of the time, they make a > big difference in my day to day work. And that's not to mention all > the Bitbucket Server and Bitbucket Data Center users who will enjoy a > snappier experience as these changes make their way out into the wild. Thank you for the thorough testing! As a heads-up: after Thomas Gummerer offered some cautious thoughts, I decided to work on the stash.useBuiltin support after all: the Git for Windows v2.19.0 installer will have builtin stash and rebase as two experimental options, both defaulting to off (to protect not only Git for Windows' users, but also the maintainer's inbox). Likewise, portable Git and MinGit will support stash.useBuiltin and rebase.useBuiltin, defaulting to `false` for both. Thanks, Johannes
Re: Request for testing v2.19.0-rc0 *with builtin stash/rebase*
On Fri, Aug 24, 2018 at 5:14 AM Johannes Schindelin wrote: > > For that reason, I was delighted to see that our Google Summer of Code > pushed pretty hard in that direction. And I could not help myself so I had > to test how much faster things got. Here is the result of my first, really > quick and dirty test: > > without builtin stash/rebasewith builtin stash/rebase > t3400 (rebase) 1m27s 32s > t3404 (rebase -i) 13m15s 3m59s > t3903 (stash) 8m37s 1m18s > > What can I say? Even if the numbers are off by as much as 10%, these are > impressive improvements: keep in mind that there is a lot of churn going > on in the test suite because it is itself implemented in Unix shell > scripts (and hence I won't even bother to try more correct performance > benchmarking because that is simply not possible when Unix shell scripts > are in the equation). So the speed improvements of the stash/rebase > commands are *even higher* than this. Thanks for taking the time to make these pre-releases available. I appreciate the effort. And the same to Junio, for always posting release candidates. We rely on them heavily to find changes that might cause issues before admins start upgrading in the wild and find them for us. I downloaded both the rc0.1 and rc0.2 builds, as well as 2.18.0, and ran them all through Bitbucket Server's test suite a few times (to ensure warm disk for comparable numbers). I added support for some "simple" rebase cases a few releases ago, so we have a set of tests that verify the rebase behaviors we use. (We don't use git stash, so we don't have any tests in our suite for that.) Running our entire Git test suite (~1,600 tests) against Git for Windows 2.18.0 takes ~5 minutes, and 2.19.0-rc0.1 produced an almost identical duration. Running our tests against rc0.2 cut the duration down to 4 minutes. There were no test failures on either pre-release build. To try and get a better sense of the rebase performance improvement specifically, I filtered down to a set of 14 specific tests which use it and ran those. On Git 2.18, those 14 tests take just over 19 seconds. On 2.19.0-rc0.2, they take just over 8 seconds. When they do ship, whether it's in 2.19 (by default or optional) or later, the changes definitely offer some significant performance wins. Thanks again, to everyone involved, for all the effort that went into designing, implementing, reviewing and releasing these improvements. As someone who develops under Windows most of the time, they make a big difference in my day to day work. And that's not to mention all the Bitbucket Server and Bitbucket Data Center users who will enjoy a snappier experience as these changes make their way out into the wild. Best regards, Bryan Turner
Re: [git-for-windows] Request for testing v2.19.0-rc0 *with builtin stash/rebase*
Team, On Fri, 24 Aug 2018, Johannes Schindelin wrote: > while this mail talks about Git for Windows, please keep in mind that we > try *very* hard to keep Git for Windows' master working correctly not only > on Windows but also on macOS and Linux. > > I, for one, run Git built from Git for Windows' `master` branch in my > Linux VMs all the time. > > As all of you know by now, the fact that Git was pretty much designed to > work well on Linux is not exactly helping Git for Windows. (Or for that > matter, macOS: Git is substantially slower on macOS than on Linux when > running on the same hardware.) The quickest pay-off comes from converting > Unix shell scripts (which are designed to spawn processes, whereas Windows > is more optimized for multi-threaded applications). > > For that reason, I was delighted to see that our Google Summer of Code > pushed pretty hard in that direction. And I could not help myself so I had > to test how much faster things got. Here is the result of my first, really > quick and dirty test: > > without builtin stash/rebasewith builtin stash/rebase > t3400 (rebase)1m27s 32s > t3404 (rebase -i) 13m15s 3m59s > t3903 (stash) 8m37s 1m18s > > What can I say? Even if the numbers are off by as much as 10%, these are > impressive improvements: keep in mind that there is a lot of churn going > on in the test suite because it is itself implemented in Unix shell > scripts (and hence I won't even bother to try more correct performance > benchmarking because that is simply not possible when Unix shell scripts > are in the equation). So the speed improvements of the stash/rebase > commands are *even higher* than this. > > So I really, really, really want those builtins in Git for Windows > v2.19.0. At *least* as an option. > > Therefore, after creating a pre-release of Git for Windows corresponding > to Git v2.19.0-rc0, I created another one (note the .2 at the end), *with* > those new builtins: > https://github.com/git-for-windows/git/releases/tag/v2.19.0-rc0.windows.2 > > I would like to ask you for help in dog-fooding this. In particular if you > are a heavy stash/rebase user (like I am), it would be helpful to really > kick those tires. > > Of course, those are only Windows binaries on that web page, but it should > be easy to compile from that tag on Linux and macOS. I could also build a > macOS installer and add it to that pre-release, is there interest in that? > > Currently I am uncertain whether I should spend the time to reinstate the > old scripted `git stash` guarded by `stash.useBuiltin` and the old > scripted `git rebase -i` (guarded by the same `rebase.useBuiltin` that > guards the scripted `git rebase`), as a fall-back (or make the new > builtins opt-ins instead of opt-outs). > > So far, I have not encountered any serious bug, but then, I did not really > have a chance to use it yet (I installed it, of course, but I have not > done any serious rebasing yet). So your help will be crucial in > determining where I need to spend time. Thomas Gummerer pointed out just how dangerously I want to live, so I did implement the `stash.useBuiltin` option (and extended the `rebase.useBuiltin` option to also cover `rebase -i`), and made the use of those builtins an opt-in, i.e. still default to the scripted versions. You can see my progress in this PR (which I mainly opened to have this tested on Windows, macOS and Linux): https://github.com/git-for-windows/git/pull/1800 Ciao, Johannes
Request for testing v2.19.0-rc0 *with builtin stash/rebase*
Team, while this mail talks about Git for Windows, please keep in mind that we try *very* hard to keep Git for Windows' master working correctly not only on Windows but also on macOS and Linux. I, for one, run Git built from Git for Windows' `master` branch in my Linux VMs all the time. As all of you know by now, the fact that Git was pretty much designed to work well on Linux is not exactly helping Git for Windows. (Or for that matter, macOS: Git is substantially slower on macOS than on Linux when running on the same hardware.) The quickest pay-off comes from converting Unix shell scripts (which are designed to spawn processes, whereas Windows is more optimized for multi-threaded applications). For that reason, I was delighted to see that our Google Summer of Code pushed pretty hard in that direction. And I could not help myself so I had to test how much faster things got. Here is the result of my first, really quick and dirty test: without builtin stash/rebasewith builtin stash/rebase t3400 (rebase) 1m27s 32s t3404 (rebase -i) 13m15s 3m59s t3903 (stash) 8m37s 1m18s What can I say? Even if the numbers are off by as much as 10%, these are impressive improvements: keep in mind that there is a lot of churn going on in the test suite because it is itself implemented in Unix shell scripts (and hence I won't even bother to try more correct performance benchmarking because that is simply not possible when Unix shell scripts are in the equation). So the speed improvements of the stash/rebase commands are *even higher* than this. So I really, really, really want those builtins in Git for Windows v2.19.0. At *least* as an option. Therefore, after creating a pre-release of Git for Windows corresponding to Git v2.19.0-rc0, I created another one (note the .2 at the end), *with* those new builtins: https://github.com/git-for-windows/git/releases/tag/v2.19.0-rc0.windows.2 I would like to ask you for help in dog-fooding this. In particular if you are a heavy stash/rebase user (like I am), it would be helpful to really kick those tires. Of course, those are only Windows binaries on that web page, but it should be easy to compile from that tag on Linux and macOS. I could also build a macOS installer and add it to that pre-release, is there interest in that? Currently I am uncertain whether I should spend the time to reinstate the old scripted `git stash` guarded by `stash.useBuiltin` and the old scripted `git rebase -i` (guarded by the same `rebase.useBuiltin` that guards the scripted `git rebase`), as a fall-back (or make the new builtins opt-ins instead of opt-outs). So far, I have not encountered any serious bug, but then, I did not really have a chance to use it yet (I installed it, of course, but I have not done any serious rebasing yet). So your help will be crucial in determining where I need to spend time. Thanks, Johannes
