Re: Regression and failure to clone/fetch with new code Re: git-svn performance

2014-10-30 Thread Hin-Tak Leung
(apology for the empty message again - sticky fingers in smart phone...) -- On Thu, Oct 30, 2014 08:46 GMT Eric Wong wrote: >Thanks, I'm not able to reproduce the issue, but can you try the >following? > >diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm >index 75cd

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

2014-10-30 Thread Hin-Tak Leung
-- On Thu, Oct 30, 2014 08:46 GMT Eric Wong wrote: >Hin-Tak Leung wrote: >> Here is the data dumper info . I tried the dumper code on the R repo >> as well, and saw that against the virtual box repo, there is one >> curious difference - $self->{last_rev} is a str

Re: differences between old clone and new Re: git-svn performance

2014-10-30 Thread Eric Wong
Hin-Tak Leung wrote: > That's quite straight-forward, I think - except for the recent burst (I am > essentially > adapting the git 2.1.0 release shipped by the upcoming fedora 21 scheduled > for christmas) > I tend to update to the latest fedora release about a week or two after > release; > f

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

2014-10-30 Thread Eric Wong
Hin-Tak Leung wrote: > Here is the data dumper info . I tried the dumper code on the R repo > as well, and saw that against the virtual box repo, there is one > curious difference - $self->{last_rev} is a string rather than a number. > I tried hacking around doing "$x += 0;" to coerce last_rev >

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

2014-10-29 Thread Hin-Tak Leung
Here is the data dumper info . I tried the dumper code on the R repo as well, and saw that against the virtual box repo, there is one curious difference - $self->{last_rev} is a string rather than a number. I tried hacking around doing "$x += 0;" to coerce last_rev to a number at various places bu

Re: differences between old clone and new Re: git-svn performance

2014-10-29 Thread Hin-Tak Leung
On Thu, 30/10/14, Eric Wong wrote: > The missing merge on branch "R-2-14-branch" is: > > commit 93af4d4cc3a5e0039944dd4e340d26995be8a252 > Merge: 121990f 6ff1b87 > Author: ripley > Date:   Wed Feb 22 13:45:34 2012 + > >     port r584

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

2014-10-29 Thread Eric Wong
Hin-Tak Leung wrote: > Argh, sorry. I thought I included the info but I didn't. Thanks. I'll try a different version of svn later. > What do you think were missing in my e-mails? I was skimming and missed the part about Debian packages :) -- To unsubscribe from this list: send the line "unsubsc

Re: differences between old clone and new Re: git-svn performance

2014-10-29 Thread Eric Wong
Hin-Tak Leung wrote: > On Tue, 28/10/14, Eric Wong wrote: > > > So both merges > are correct, but we lose one, and gain one? > I'll try to check more closely tomorrow.  > Can you point out > the exact revisions in the > R repo?  Thanks. > > > The missing merge on branch "R-2-14-branch" is

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

2014-10-29 Thread Hin-Tak Leung
nces of new clone against old is a missing merge in at R-2-14-branch@58454 , and two extra merges at djm-parseRd@46925 and djm-parseRd@46906 . On Wed, 29/10/14, Eric Wong wrote: Subject: Re: Regression and failure to clone/fetch with new code Re:

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

2014-10-29 Thread Eric Wong
Hin-Tak Leung wrote: > Hi, I patched my system git with the recent git-svn improvements, and just use > it for general use; so theses are the patches, against 2.1.0. > > 0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch > 0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patc

Re: differences between old clone and new Re: git-svn performance

2014-10-28 Thread Hin-Tak Leung
On Tue, 28/10/14, Eric Wong wrote: > So both merges are correct, but we lose one, and gain one? I'll try to check more closely tomorrow.  Can you point out the exact revisions in the R repo?  Thanks. The missing merge on branch "R-2-14-branch" is: commit 93af4d4cc3a5e0039944dd4e340d26995b

Regression and failure to clone/fetch with new code Re: git-svn performance

2014-10-28 Thread Hin-Tak Leung
Hi, I patched my system git with the recent git-svn improvements, and just use it for general use; so theses are the patches, against 2.1.0. 0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch 0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patch 0003-git-svn-reduce-check_cher

Re: Anomaly with the new code - Re: git-svn performance

2014-10-28 Thread Eric Wong
Hin-Tak Leung wrote: > >Eric Wong wrote: > >> Which SVN version are you using?  I'm cloning (currently on r373xx) > >> https://svn.r-project.org/R using --stdlayout and > >> unable to see memory growth of the git-svn Perl process beyond 40M > >> (on a 32-bit system). > > > >git-svn hit 45M and to

Re: differences between old clone and new Re: git-svn performance

2014-10-28 Thread Eric Wong
Hin-Tak Leung wrote: > To compare the old clone with the new, I did: > > git branch -r | sort | xargs -n 1 git log --decorate=full -n 1 > > It turned out other than the empty vs 3 word commit messages > about two years ago on trunk (which are inherited in all the newer > branches), there are two

differences between old clone and new Re: git-svn performance

2014-10-27 Thread Hin-Tak Leung
To compare the old clone with the new, I did: git branch -r | sort | xargs -n 1 git log --decorate=full -n 1 It turned out other than the empty vs 3 word commit messages about two years ago on trunk (which are inherited in all the newer branches), there are two other groups of differences. One b

Re: Anomaly with the new code - Re: git-svn performance

2014-10-27 Thread Hin-Tak Leung
-- On Mon, Oct 27, 2014 16:56 GMT Eric Wong wrote: >Eric Wong wrote: >> Which SVN version are you using?  I'm cloning (currently on r373xx) >> https://svn.r-project.org/R using --stdlayout and >> unable to see memory growth of the git-svn Perl process beyond 40M >> (on

Re: Anomaly with the new code - Re: git-svn performance

2014-10-27 Thread Hin-Tak Leung
-- On Mon, Oct 27, 2014 06:38 GMT Eric Wong wrote: >Which SVN version are you using?  I'm cloning (currently on r373xx) >https://svn.r-project.org/R using --stdlayout and >unable to see memory growth of the git-svn Perl process beyond 40M >(on a 32-bit system). > >I als

Re: Anomaly with the new code - Re: git-svn performance

2014-10-27 Thread Eric Wong
Eric Wong wrote: > Which SVN version are you using? I'm cloning (currently on r373xx) > https://svn.r-project.org/R using --stdlayout and > unable to see memory growth of the git-svn Perl process beyond 40M > (on a 32-bit system). git-svn hit 45M and took 11:44 to finish. My ping times to svn.

Re: Anomaly with the new code - Re: git-svn performance

2014-10-26 Thread Eric Wong
Hin-Tak Leung wrote: > On Sat, Oct 25, 2014 00:34 BST Eric Wong wrote: > >0006 is insufficient and incompatible with older SVN. > >I pushed "git-svn: reload RA every log-window-size" > >(commit dfa72fdb96befbd790f623bb2909a347176753c2) instead > >which saves much more memory: > > it is fetching ag

Re: git-svn performance

2014-10-25 Thread Hin-Tak Leung
-- On Sat, Oct 25, 2014 06:32 BST Eric Wong wrote: >Hin-Tak Leung wrote: >> the old didn't missing a revision - just a revision 'message' - blank >> instead of 3 words, above the git svn id. I supppse it is possible >> some power problem or etc caused this. I'll chec

Re: Anomaly with the new code - Re: git-svn performance

2014-10-25 Thread Hin-Tak Leung
-- On Sat, Oct 25, 2014 00:34 BST Eric Wong wrote: >Hin-Tak Leung wrote: >> I keep tabs of a particular svn repository over many years >> and run "git svn fetch --all" every few days. So that's the old clone. >> Since this discussion started, I made a new one with

Re: git-svn performance

2014-10-25 Thread Hin-Tak Leung
-- On Sat, Oct 25, 2014 01:02 BST Eric Wong wrote: >Hin-Tak Leung wrote: >> Comparing trunk of old and new, I see one difference - One short >> commit message is missing in the *old* (the "Add checkPoFiles etc." part) >> and so all the sha1 afterwards differed. Is t

single-word commit messages - Re: git-svn performance

2014-10-25 Thread Hin-Tak Leung
On Sat, 25/10/14, Eric Wong wrote: > Probably an SVN hook preventing it.  git-svn test cases such as t/t9118-git-svn-funky-branch-names.sh do single word commits. Thanks - I see indeed - at least that clears that up. -- To unsubscribe from this

Re: git-svn performance

2014-10-25 Thread Eric Wong
Hin-Tak Leung wrote: > btw, git svn seems to disallow single word commit messages (or is it a > svn config?). i found that i could not do git svn dcommit, when i had > merely did git commit -m 'typos', for example, for an svn repo i have > write access to. (I don't have them many such things, so i

Re: Anomaly with the new code - Re: git-svn performance

2014-10-25 Thread Eric Wong
Hin-Tak Leung wrote: > On Sat, Oct 25, 2014 00:34 BST Eric Wong wrote: > >Hin-Tak Leung wrote: > >> 0006-git-svn-clear-global-SVN-pool-between-get_log-invoca.patch > > > >0006 is insufficient and incompatible with older SVN. > >I pushed "git-svn: reload RA every log-window-size" > >(commit dfa

Re: git-svn performance

2014-10-25 Thread Eric Wong
Hin-Tak Leung wrote: > the old didn't missing a revision - just a revision 'message' - blank > instead of 3 words, above the git svn id. I supppse it is possible > some power problem or etc caused this. I'll check the other branches > as well, and possibly clone again to be sure. ( The new clone d

Re: git-svn performance

2014-10-24 Thread Eric Wong
Hin-Tak Leung wrote: > Comparing trunk of old and new, I see one difference - One short > commit message is missing in the *old* (the "Add checkPoFiles etc." part) > and so all the sha1 afterwards differed. Is that an old bug that's fixed > and therefore I should throw away the old clone? I don

Re: Anomaly with the new code - Re: git-svn performance

2014-10-24 Thread Eric Wong
Hin-Tak Leung wrote: > I keep tabs of a particular svn repository over many years > and run "git svn fetch --all" every few days. So that's the old clone. > Since this discussion started, I made a new one with git 2.1.0 patched > with the first two patches below, a couple of weeks ago. And I ran >

Anomaly with the new code - Re: git-svn performance

2014-10-24 Thread Hin-Tak Leung
I keep tabs of a particular svn repository over many years and run "git svn fetch --all" every few days. So that's the old clone. Since this discussion started, I made a new one with git 2.1.0 patched with the first two patches below, a couple of weeks ago. And I ran 'git svn fetch --all' on both e

Re: git-svn performance

2014-10-22 Thread Hin-Tak Leung
-- On Tue, Oct 21, 2014 10:00 BST Eric Wong wrote: >Jakob Stoklund Olesen wrote: >> Yes, but I think you can remove cached_mergeinfo_rev too. > >Thanks, pushed the patch at the bottom, too. >Also started working on some memory reductions here: > http://mid.gmane.org/2

Re: git-svn performance

2014-10-21 Thread Eric Wong
Jakob Stoklund Olesen wrote: > Yes, but I think you can remove cached_mergeinfo_rev too. Thanks, pushed the patch at the bottom, too. Also started working on some memory reductions here: http://mid.gmane.org/20141021033912.ga27...@dcvr.yhbt.net But there seem to be more problems :< ---

Re: git-svn performance

2014-10-20 Thread Jakob Stoklund Olesen
> On Oct 19, 2014, at 18:16, Eric Wong wrote: > > Jakob Stoklund Olesen wrote: >> If cached_mergeinfo is using too much memory, you can probably drop >> that cache entirely. IIRC, it didn't give that much of a speed up. >> >> I am surprised that it is using a lot of memory, though. There is on

Re: git-svn performance

2014-10-19 Thread Eric Wong
Jakob Stoklund Olesen wrote: > If cached_mergeinfo is using too much memory, you can probably drop > that cache entirely. IIRC, it didn't give that much of a speed up. > > I am surprised that it is using a lot of memory, though. There is only > one entry per SVN branch. Something like the below?

Re: git-svn performance

2014-10-19 Thread Jakob Stoklund Olesen
> On Oct 18, 2014, at 19:33, Eric Wong wrote: > > Eric Wong wrote: >> This reduces hash lookups for looking up cache data and will >> simplify tying data to disk in the next commit. > > I considered the following, but GDBM might not be readily available on > non-POSIX platforms. I think the o

Re: git-svn performance

2014-10-19 Thread Fabian Schmied
On Sun, Oct 19, 2014 at 2:32 AM, Eric Wong wrote: > Fabian Schmied wrote: >> Hi, >> >> I'm currently migrating an SVN repository to Git using git-svn (Git >> for Windows 1.8.3-preview20130601), and I'm experiencing severe >> performance problems with "git svn fetch". Commits to the SVN "trunk" >>

Re: git-svn performance

2014-10-18 Thread Eric Wong
Eric Wong wrote: > This reduces hash lookups for looking up cache data and will > simplify tying data to disk in the next commit. I considered the following, but GDBM might not be readily available on non-POSIX platforms. I think the other problem is the existing caches are still in memory (whet

Re: git-svn performance

2014-10-18 Thread Eric Wong
Eric Wong wrote: > Hin-Tak (Cc-ed) reported good improvements with them, but also > a large memory increase: This might reduce the pathname and internal hash overheads: 8<--- From: Eric Wong Date: Sun, 19 Oct 2014 02:26:53 + Subject: [PATCH] git-sv

Re: git-svn performance

2014-10-18 Thread Eric Wong
Fabian Schmied wrote: > Hi, > > I'm currently migrating an SVN repository to Git using git-svn (Git > for Windows 1.8.3-preview20130601), and I'm experiencing severe > performance problems with "git svn fetch". Commits to the SVN "trunk" > are fetched very fast (a few seconds or so per SVN revisi

git-svn performance

2014-10-17 Thread Fabian Schmied
Hi, I'm currently migrating an SVN repository to Git using git-svn (Git for Windows 1.8.3-preview20130601), and I'm experiencing severe performance problems with "git svn fetch". Commits to the SVN "trunk" are fetched very fast (a few seconds or so per SVN revision), but commits to some branches (

Re: [PATCH v2] improve git svn performance

2014-01-22 Thread Eric Wong
manjian2...@gmail.com wrote: > * perl/Git/SVN.pm > Modified according to Eric Wong > > >Hi, I'm interested in this. How much did performance improve by > >(and how many revisions is the repository)> > Our svn server are built in a LAN,15152 revisions.Not optimized > git-svn used 10 hours or m

[PATCH v2] improve git svn performance

2014-01-22 Thread manjian2006
From: manjian2006 * perl/Git/SVN.pm Modified according to Eric Wong >Hi, I'm interested in this. How much did performance improve by >(and how many revisions is the repository)> Our svn server are built in a LAN,15152 revisions.Not optimized git-svn used 10 hours or more to accomplish, whi

Re: [PATCH] improve git svn performance

2014-01-21 Thread Eric Wong
manjian2...@gmail.com wrote: > From: linzj > I am trying to improve git svn's performance according to some > profiling data.As the data showed,_rev_list subroutine and rebuild > subroutine are consuming a large proportion of time.So I improve > _rev_list's performance by memoize its resul

[PATCH] improve git svn performance

2014-01-19 Thread manjian2006
From: linzj Hi, I am trying to improve git svn's performance according to some profiling data.As the data showed,_rev_list subroutine and rebuild subroutine are consuming a large proportion of time.So I improve _rev_list's performance by memoize its results,and avoid subprocess invocation by

[PATCH] improve git svn performance Hi, I am trying to improve git svn's performance according to some profiling data.As the data showed,_rev_list subroutine and rebuild subroutine are consuming a l

2014-01-19 Thread manjian2006
From: linzj --- perl/Git/SVN.pm | 63 - 1 file changed, 49 insertions(+), 14 deletions(-) diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm index 5273ee8..3cd1c8f 100644 --- a/perl/Git/SVN.pm +++ b/perl/Git/SVN.pm @@ -1599,27 +1599,36 @@ sub