This memoization appears unneeded as the check_cherry_pick2 cache is in front of it does enough.
With this change applied, importing from local svn+ssh and http copies of the R repo[1] takes only 2:00 (2 hours) on my system and the git-svn process never uses more than 60MB RSS on my x86-64 GNU/Linux system[2]. This 60M measurement is only for the git-svn Perl process itself and does not include memory used by git subprocesses accessing large packs (subprocess memory usage _is_ measured by my time(1) tool). Before this change, an import took longer (2:20) on svn+ssh:// but git-svn used around 240MB during the imports. Worse yet, git-svn ballooned to over 400M when writing out the cache to the filesystem. I also tried removing memoization for `has_no_changes', too, but a local copy of the R repository(*) was not close to finishing within 10 hours on my system. [1] http://svn.r-project.org/R [2] file:// repos causes libsvn to use more memory internally Signed-off-by: Eric Wong <normalper...@yhbt.net> Cc: Hin-Tak Leung <ht...@users.sourceforge.net> --- This patch is in my "less-memo" branch of bogomips.org/git-svn.git Info here: (not intended as a pull request for Junio, yet) The following changes since commit fbecd99861ea5795aeba46faf2ac7a8c1b70d485: Update draft release notes to 2.2 (2014-10-24 15:02:17 -0700) are available in the git repository at: git://bogomips.org/git-svn.git less-memo for you to fetch changes up to 7676aff70973e617c3f58a8633db6d0e3ee99e45: git-svn: disable _rev_list memoization (2014-10-27 01:39:39 +0000) ---------------------------------------------------------------- Eric Wong (7): git-svn: reduce check_cherry_pick cache overhead git-svn: cache only mergeinfo revisions git-svn: remove mergeinfo rev caching git-svn: reload RA every log-window-size git-svn: remove unnecessary DESTROY override git-svn: save a little memory as fetch progresses git-svn: disable _rev_list memoization Jakob Stoklund Olesen (2): git-svn: only look at the new parts of svn:mergeinfo git-svn: only look at the root path for svn:mergeinfo Sveinung Kvilhaugsvik (1): git-svn.txt: advertise pushurl with dcommit Documentation/git-svn.txt | 4 ++ perl/Git/SVN.pm | 121 +++++++++++++++++++++++++++++----------------- perl/Git/SVN/Ra.pm | 27 ++++++----- 3 files changed, 94 insertions(+), 58 deletions(-) perl/Git/SVN.pm | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm index 4364506..5f9d469 100644 --- a/perl/Git/SVN.pm +++ b/perl/Git/SVN.pm @@ -1600,7 +1600,6 @@ sub tie_for_persistent_memoization { my %lookup_svn_merge_cache; my %check_cherry_pick2_cache; my %has_no_changes_cache; - my %_rev_list_cache; tie_for_persistent_memoization(\%lookup_svn_merge_cache, "$cache_path/lookup_svn_merge"); @@ -1622,14 +1621,6 @@ sub tie_for_persistent_memoization { SCALAR_CACHE => ['HASH' => \%has_no_changes_cache], LIST_CACHE => 'FAULT', ; - - tie_for_persistent_memoization(\%_rev_list_cache, - "$cache_path/_rev_list"); - memoize '_rev_list', - SCALAR_CACHE => 'FAULT', - LIST_CACHE => ['HASH' => \%_rev_list_cache], - ; - } sub unmemoize_svn_mergeinfo_functions { @@ -1639,7 +1630,6 @@ sub tie_for_persistent_memoization { Memoize::unmemoize 'lookup_svn_merge'; Memoize::unmemoize 'check_cherry_pick2'; Memoize::unmemoize 'has_no_changes'; - Memoize::unmemoize '_rev_list'; } sub clear_memoized_mergeinfo_caches { -- EW -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
