Re: Git very slow ?
On Sun, Mar 08, 2015 at 05:21:22PM +0100, David Kastrup wrote: Kevin D m...@ikke.info writes: On Sat, Mar 07, 2015 at 01:30:07AM +, Ken Moffat wrote: Hi, please CC me if that is not your usual fashion, because I am not subscribed. I use git for my build scripts - those are accessed over nfs. Since I started using 2.1 and later (I don't think I used 2.0) commands such as 'commit' take a long time before anything happens. I assumed that the newer version meant this would take longer. But today^Wyesterday I was bisecting the kernel on a local filesystem - even when the number of revisions left to test was in the single digits, git bisect took a long time to decide which revision should be the next one to test. The filesystems are ext4. Is this sort of delay normal now? What really prompted me to ask is that I ran git blame on a script, to see when I made a particular change so that I could add that information to a ticket, and almost gave up waiting because it felt as if it was taking for ever. What kind of repository are we talking about? How many files, how big? Git should not have become significantly slower recently. Particularly not git-blame in 2.1. I should be quite surprised to see any git-blame call running noticeably slower in 2.1 than in any preceding version. What may have happened is that the repository recently got repacked aggressively and thus any access to older revisions got slower. However, that change would be mostly tied to the repository rather than the version of Git you access it with. That is possible - well, not recently-recently, but I might have repacked my repo of buildscripts some time last year. Running ls -al .git in that repository gives me: drwxr-xr-x 8 ken 100 4096 Mar 8 16:08 . drwxr-xr-x 48 ken 100 4096 Mar 8 03:05 .. -rw-r--r-- 1 ken 100220 May 12 2014 BRANCH_DESCRIPTION drwxr-xr-x 2 ken 100 4096 Apr 13 2010 branches -rw-r--r-- 1 ken 100470 Mar 8 16:08 COMMIT_EDITMSG -rw-r--r-- 1 ken 100566 May 17 2014 config -rw-r--r-- 1 ken 100 73 May 1 2010 description -rw-r--r-- 1 ken 100 196439 Sep 17 21:56 gitk.cache -rw-rw-rw- 1 ken 100 29 Feb 8 22:19 HEAD drwxr-xr-x 2 ken 100 4096 May 1 2010 hooks -rw-r--r-- 1 ken 100 218255 Mar 8 16:07 index drwxr-xr-x 2 ken 100 4096 Sep 16 2013 info drwxr-xr-x 3 ken 100 4096 Sep 16 2013 logs drwxr-xr-x 260 ken 100 4096 Nov 12 2013 objects -rw-r--r-- 1 ken 100 41 Nov 11 06:05 ORIG_HEAD -rw-r--r-- 1 ken 100 1879 Sep 16 2013 packed-refs drwxr-xr-x 5 ken 100 4096 May 20 2014 refs -rw-r--r-- 1 ken 100 41 Dec 7 2010 RENAMED-REF Running git blame on a script which dates back to when the repo was created takes between 5 and 6 seconds to show the first screen, running it on a file first created last August took about 3 seconds. I have another small repo also on nfs with only a few files, started last year, and there git blame takes about a second. In my clone of linus' tree, running git blame scripts/headers.sh (a random example,I do not normally run git blame there) took 10 seconds to return the first screen. All of those timings are from preparing the comand, watching my desktop clock until it changes the second, and hitting enter - so, only accurate to approx one second. ĸen -- Nanny Ogg usually went to bed early. After all, she was an old lady. Sometimes she went to bed as early as 6 a.m. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git very slow ?
On Sun, Mar 08, 2015 at 04:51:36PM +0100, Kevin D wrote: On Sat, Mar 07, 2015 at 01:30:07AM +, Ken Moffat wrote: Hi, please CC me if that is not your usual fashion, because I am not subscribed. I use git for my build scripts - those are accessed over nfs. Since I started using 2.1 and later (I don't think I used 2.0) commands such as 'commit' take a long time before anything happens. I assumed that the newer version meant this would take longer. But today^Wyesterday I was bisecting the kernel on a local filesystem - even when the number of revisions left to test was in the single digits, git bisect took a long time to decide which revision should be the next one to test. The filesystems are ext4. Is this sort of delay normal now? What really prompted me to ask is that I ran git blame on a script, to see when I made a particular change so that I could add that information to a ticket, and almost gave up waiting because it felt as if it was taking for ever. What kind of repository are we talking about? How many files, how big? Git should not have become significantly slower recently. The comments on git bisect were for linus'skernel tree, on a local disk. 2.3GB of repo, just under 57000 files. My own repo of build scripts, where I have noticed the delay before git commit lets me type in the message, is an nfs v3 mount from another of my machines in the same room - ping between them gives times of 0.25 to 0.3 seconds and I think the nfs part is irrelevant. Here, the size is 70MB and 12133 files [ about 1500 scripts total, so the rest is from the commits ]. Some of this might be the drives - on the desktop with linus's tree the machine only supports SATA2 (3GB/S), but the machine serving my scripts goes back further and probably only supports SATA1 (1.5GB/S) Also, might there be anti-virus software that slows down file access? No, this is all local access on linux machines. ĸen -- Nanny Ogg usually went to bed early. After all, she was an old lady. Sometimes she went to bed as early as 6 a.m. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git very slow ?
On Sun, Mar 08, 2015 at 12:39:10PM -0700, Linus Torvalds wrote: On Sun, Mar 8, 2015 at 12:02 PM, Ken Moffat zarniwh...@ntlworld.com wrote: The comments on git bisect were for linus'skernel tree, on a local disk. 2.3GB of repo, just under 57000 files. Ugh. I hope you are talking about checked-out size. [torvalds@i7 linux]$ du -sh .git 850M .git because otherwise it sounds like that repo hasn't been repacked in forever. Yes - I had finished bisecting, with the build still present. To really pack things (which can slow things down for old history as people said, but on the whole it tends to be a big win due to less IO), do git repack -adf --window=200 --depth=200 and go away for a while. Oh, and make sure your machine has enough memory and CPU to make that for a while not be *too* long. For that, many thanks - this desktop has about 7GB (integrated graphics steal a bit), current AMD desktop, and the repack of my scripts repo took about 56 seconds. I'll do that on my copy of the kernel tomorrow (it's on another machine). You should have a few hundred files (just a few tens of files directly after the repack) and that roughly 850MB of space for the repository information itself. But yeah, fully checked out and built with all the modules etc, you would have much more. That said, if you have something fairly that is consistently really slow (like the git commit you mentioned), *before* doing the repack, do strace -o ../trace-file -Ttt git commit and we can get a much better guess about why it's so slow. Send it to me in private email if you don't want to make it public, and I can take a look. I don't think you need to look - it was taking most of the time (about 8 seconds) looping through the many files below .git/objects. The trace was just over 9000 lines, repeating after the repack was less than 1300 lines. It's available (97K after using xz) if you think it would be useful, but I think you have already diagnosed the problem and solution. ping between them gives times of 0.25 to 0.3 seconds .. and I *really* hope that was not seconds, but ms. Otherwise your nfsv3 setup is going to be really really painful. Linus Yes, I'm not always good at knowing the right units. Thanks for the help. ĸen -- Nanny Ogg usually went to bed early. After all, she was an old lady. Sometimes she went to bed as early as 6 a.m. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Git very slow ?
Hi, please CC me if that is not your usual fashion, because I am not subscribed. I use git for my build scripts - those are accessed over nfs. Since I started using 2.1 and later (I don't think I used 2.0) commands such as 'commit' take a long time before anything happens. I assumed that the newer version meant this would take longer. But today^Wyesterday I was bisecting the kernel on a local filesystem - even when the number of revisions left to test was in the single digits, git bisect took a long time to decide which revision should be the next one to test. The filesystems are ext4. Is this sort of delay normal now? What really prompted me to ask is that I ran git blame on a script, to see when I made a particular change so that I could add that information to a ticket, and almost gave up waiting because it felt as if it was taking for ever. ĸen -- Nanny Ogg usually went to bed early. After all, she was an old lady. Sometimes she went to bed as early as 6 a.m. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Git v1.9-rc0
On Wed, Jan 22, 2014 at 10:10:18AM -0800, Junio C Hamano wrote: Vicent Martí tan...@gmail.com writes: Do these consume CPU every time somebody asks for a tarball? That might be considered wrong depending on the view. No, our infrastructure caches frequently requested tarballs so they don't have to be regenerated on the fly. Thanks. That is certainly good enough for consumers, and better than having to manually create and upload for me ;-) Two questions: Does regenerating (e.g. if the tarball has dropped out of the cache) change its sums (md5sum or similar) ? In (beyond) linuxfromscratch we use md5sums to verify that a tarball has not changed. Also, will there be links for manpages and htmldocs tarballs ? I note that all of these *are* still available at googlecode for the moment : https://code.google.com/p/git-core/downloads/list ĸen -- das eine Mal als Tragödie, dieses Mal als Farce -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Git v1.9-rc0
On Wed, Jan 22, 2014 at 01:04:02PM -0800, Junio C Hamano wrote: Ken Moffat zarniwh...@ntlworld.com writes: I note that all of these *are* still available at googlecode for the moment : https://code.google.com/p/git-core/downloads/list As I said, Cgc is not the ony download site. The end of http://git-blame.blogspot.com/p/git-public-repositories.html lists the two sites that currently have the material. I may replace Cgc with something else (and add it/them to the list), but in the meantime I do not think k.org will go out of business in anytime soon, so... OK, thanks for the pointer to https://www.kernel.org/pub/software/scm/git/ for released tarballs. ĸen -- das eine Mal als Tragödie, dieses Mal als Farce -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
