Re: [PATCH v2] git-svn: workaround for a bug in svn serf backend
Jonathan Nieder jrnie...@gmail.com wrote: Roman Kagan wrote: Subversion serf backend in versions 1.8.5 and below has a bug that the function creating the descriptor of a file change -- add_file() -- doesn't make a copy of its third argument when storing it on the returned descriptor. As a result, by the time this field is used (in transactions of file copying or renaming) it may well be released, and the memory reused. One of its possible manifestations is the svn assertion triggering on an invalid path, with a message svn_fspath__skip_ancestor: Assertion `svn_fspath__is_canonical(child_fspath)' failed. [...] Makes sense. Perhaps also worth mentioning that this is fixed by r1553376, but no need to reroll just for that. Thanks all, I noted this in an addendum to the commit: Subversion serf backend in versions 1.8.5 and below has a bug(*) that the ... * [ew: fixed in Subversion r1553376 as noted by Jonathan Nieder] Cc: Benjamin Pabst benjamin.pabs...@gmail.com Cc: Eric Wong normalper...@yhbt.net Cc: Jonathan Nieder jrnie...@gmail.com No need for these lines --- the mail header already keeps track of who is being cc-ed. I don't mind seeing it in history. At least I've gotten accustomed to it from the Linux kernel and tracking patch flow between dev - stable trees. Signed-off-by: Roman Kagan rka...@mail.ru Reviewed-by: Jonathan Nieder jrnie...@gmail.com Signed-off-by: Eric Wong normalper...@yhbt.net The following changes since commit 7794a680e63a2a11b73cb1194653662f2769a792: Sync with 1.8.5.2 (2013-12-17 14:12:17 -0800) are available in the git repository at: git://git.bogomips.org/git-svn.git master for you to fetch changes up to 2394e94e831991348688831a384b088a424c7ace: git-svn: workaround for a bug in svn serf backend (2013-12-27 20:22:19 +) Roman Kagan (1): git-svn: workaround for a bug in svn serf backend perl/Git/SVN/Editor.pm | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] improve git svn performance
manjian2...@gmail.com wrote:
From: linzj li...@ucweb.com
I am trying to improve git svn's performance according to some
profiling data.As the data showed,_rev_list subroutine and rebuild
subroutine are consuming a large proportion of time.So I improve
_rev_list's performance by memoize its results,and avoid subprocess
invocation by memoize rebuild subroutine's key data.Here's my patch:
Hi, I'm interested in this. How much did performance improve by
(and how many revisions is the repository)
However, I cannot accept the patch in the current state.
The commit message is inadequate; and you need a Signed-off-by and follow
existing coding convention/style. See Documentation/SubmittingPatches
and Documentation/CodingGuidelines for more info.
Some comments inline...
tie_for_persistent_memoization(\%lookup_svn_merge_cache,
- $cache_path/lookup_svn_merge);
+ $cache_path/lookup_svn_merge);
memoize 'lookup_svn_merge',
- SCALAR_CACHE = 'FAULT',
- LIST_CACHE = ['HASH' = \%lookup_svn_merge_cache],
- ;
+ SCALAR_CACHE = 'FAULT',
+ LIST_CACHE = ['HASH' =
\%lookup_svn_merge_cache],
+ ;
Please no extraneous whitespace changes.
+#define a global associate map to record rebuild status
+my %rebuildStatus;
We prefer snake_case variables, not camelCase.
sub rebuild {
my ($self) = @_;
my $map_path = $self-map_path;
my $partial = (-e $map_path ! -z $map_path);
- return unless ::verify_ref($self-refname.'^0');
+my $verifyKey = $self-refname.'^0';
+if (! exists $rebuildVerifyStatus{$verifyKey} || ! defined
$rebuildVerifyStatus{$verifyKey} ) {
+my $verifyResult = ::verify_ref($verifyKey);
+if ($verifyResult) {
+$rebuildVerifyStatus{$verifyKey} = 1;
+}
+}
+if (! exists $rebuildVerifyStatus{$verifyKey}) {
+return;
+}
Please use tabs for indentation to match surrounding code
(most of git is tabs).
Thanks.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] improve git svn performance
manjian2...@gmail.com wrote:
* perl/Git/SVN.pm
Modified according to Eric Wong normalper...@yhbt.net
Hi, I'm interested in this. How much did performance improve by
(and how many revisions is the repository)
Our svn server are built in a LAN,15152 revisions.Not optimized
git-svn used 10 hours or more to accomplish, while optimized one using
only 3-4 hours.
According to some profiling data,_rev_list subroutine and rebuild
subroutine are consuming a large proportion of time. So I improve
_rev_list's performance by memoize its results,and avoid subprocess
invocation by memoize rebuild subroutine's key data.
Impressive! Thanks for that info.
Signed-off-by: manjian2006 manjian2...@gmail.com
Real name is preferred by this project, I think.
A proper patch would start something like this:
---8
From: Your Name manjian2...@gmail.com
Subject: git-svn: memoize _rev_list and rebuild
According to profile data, _rev_list and rebuild consume a large
portion of time. Memoize the results of _rev_list and memoize
rebuild internals to avoid subprocess invocation.
When importing 15152 revisions on a LAN, time improved from 10
hours to 3-4 hours.
Signed-off-by: Your Name manjian2...@gmail.com
-- a few more comments below ---
sub rebuild {
my ($self) = @_;
my $map_path = $self-map_path;
my $partial = (-e $map_path ! -z $map_path);
- return unless ::verify_ref($self-refname.'^0');
+ my $verify_key = $self-refname.'^0';
+ if (! exists $rebuild_verify_status{$verify_key} || ! defined
$rebuild_verify_status{$verify_key} ) {
80 column wrap, please.
However, I think just having a single
!$rebuild_verify_status{$verify_key} check is enough, no need for
extra defined/exists checks for %rebuild_verify_status nor %rebuild_status.
Neither of them load untrusted data.
- command_output_pipe(qw/rev-list --pretty=raw --reverse/,
- ($head ? $head.. : ) . $self-refname,
+ command_output_pipe(qw/rev-list --pretty=raw --reverse/,
+ $key_value,
Please do not leave trailing whitespace. Thanks.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3] git-svn: memoize _rev_list and rebuild
manjian2...@gmail.com wrote: According to profile data, _rev_list and rebuild consume a large portion of time. Memoize the results of _rev_list and memoize rebuild internals to avoid subprocess invocation. When importing 15152 revisions on a LAN, time improved from 10 hours to 3-4 hours. Signed-off-by: lin zuojian manjian2...@gmail.com Thanks! Signed-off-by: Eric Wong normalper...@yhbt.net Pushed for Junio. The following changes since commit d9bb4be53bc5185244b4be9860562a012803bacb: Merge tag 'gitgui-0.19.0' of http://repo.or.cz/r/git-gui (2014-01-21 13:16:17 -0800) are available in the git repository at: git://git.bogomips.org/git-svn.git master for you to fetch changes up to ab0bcec9873f1fcef6c4b8825cc9e762c636ca9e: git-svn: memoize _rev_list and rebuild (2014-01-23 02:54:26 +) lin zuojian (1): git-svn: memoize _rev_list and rebuild perl/Git/SVN.pm | 41 ++--- 1 file changed, 38 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Added get sendmail from .mailrc
Brilliantov Kirill Vladimirovich brillian...@inbox.ru wrote: Signed-off-by: Brilliantov Kirill Vladimirovich brillian...@inbox.ru --- git-send-email.perl | 18 ++ 1 file changed, 18 insertions(+) Some documentation references to .mailrc and its format would be nice. --- a/git-send-email.perl +++ b/git-send-email.perl @@ -28,6 +28,7 @@ use File::Temp qw/ tempdir tempfile /; use File::Spec::Functions qw(catfile); use Error qw(:try); use Git; +use File::HomeDir; We should probably avoid a new dependency and also remain consistent with the rest of git handles home directories. Unfortunately, expand_user_path()/git_config_pathname() isn't currently exposed to scripters right now... -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Added get sendmail from .mailrc
Brilliantov Kirill Vladimirovich brillian...@inbox.ru wrote: On 2014-01-25 22:37:21, Eric Wong wrote: Brilliantov Kirill Vladimirovich brillian...@inbox.ru wrote: --- a/git-send-email.perl +++ b/git-send-email.perl @@ -28,6 +28,7 @@ use File::Temp qw/ tempdir tempfile /; use File::Spec::Functions qw(catfile); use Error qw(:try); use Git; +use File::HomeDir; We should probably avoid a new dependency and also remain consistent with the rest of git handles home directories. Unfortunately, expand_user_path()/git_config_pathname() isn't currently exposed to scripters right now... Ok, if new dependency is not allowed I see next ways: Not saying it's not allowed. I meant we should probably expose expand_user_path()/git_config_pathname() C functions to script helpers (so git-config or git-rev-parse can provide them to sh or perl scripts). -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 099/144] t9101-git-svn-props.sh: use the $( ... ) construct for command substitution
Elia Pinto gitter.spi...@gmail.com wrote: The Git CodingGuidelines prefer the $( ... ) construct for command substitution instead of using the back-quotes, or grave accents (`..`). I did not check very closely, but for the git-svn tests: Acked-by: Eric Wong normalper...@yhbt.net Thanks. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC/PATCHv3 3/3] Git 2.0: git svn: Set default --prefix='origin/' if --prefix is not given
Johan Herland jo...@herland.net wrote: --- a/Documentation/git-svn.txt +++ b/Documentation/git-svn.txt @@ -86,14 +86,7 @@ COMMANDS (refs/remotes/$remote/*). Setting a prefix is also useful if you wish to track multiple projects that share a common repository. -+ -NOTE: In Git v2.0, the default prefix will CHANGE from (no prefix) -to origin/. This is done to put SVN-tracking refs at -refs/remotes/origin/* instead of refs/remotes/*, and make them -more compatible with how Git's own remote-tracking refs are organized -(i.e. refs/remotes/$remote/*). You can enjoy the same benefits today, -by using the --prefix option. - + By default, the prefix is set to 'origin/'. We should maintain a description of =1.9 behavior in the manpage. Users on long-term-support systems are likely to continue using ancient git installations for some time (5-10 years, even?), but may come across the current documentation online. Otherwise the patch looks fine and I can push it up for Junio for 2.0-rc1. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn.txt: Retain a description og pre-v2.0 default prefix
Johan Herland jo...@herland.net wrote: Feel free to add/squash this on top. Thanks! Squashed and pushed. The following changes since commit cc291953df19aa4a97bee3590e708dc1fc557500: Git 2.0-rc0 (2014-04-18 11:21:43 -0700) are available in the git repository at: git://bogomips.org/git-svn.git master for you to fetch changes up to fe191fcaa58cb785c804465a0da9bcba9fd9e822: Git 2.0: git svn: Set default --prefix='origin/' if --prefix is not given (2014-04-19 11:30:13 +) Johan Herland (1): Git 2.0: git svn: Set default --prefix='origin/' if --prefix is not given Documentation/git-svn.txt| 20 ++--- git-svn.perl | 12 +- t/t9107-git-svn-migrate.sh | 54 t/t9114-git-svn-dcommit-merge.sh | 4 +- t/t9116-git-svn-log.sh | 46 ++-- t/t9117-git-svn-init-clone.sh| 16 +++ t/t9118-git-svn-funky-branch-names.sh| 20 - t/t9120-git-svn-clone-with-percent-escapes.sh| 14 +++--- t/t9125-git-svn-multi-glob-branch-names.sh | 6 +-- t/t9128-git-svn-cmd-branch.sh| 18 t/t9135-git-svn-moved-branch-empty-file.sh | 2 +- t/t9141-git-svn-multiple-branches.sh | 28 ++-- t/t9145-git-svn-master-branch.sh | 2 +- t/t9155-git-svn-fetch-deleted-tag.sh | 4 +- t/t9156-git-svn-fetch-deleted-tag-2.sh | 6 +-- t/t9161-git-svn-mergeinfo-push.sh| 22 +- t/t9163-git-svn-reset-clears-caches.sh | 4 +- t/t9165-git-svn-fetch-merge-branch-of-branch.sh | 2 +- t/t9166-git-svn-fetch-merge-branch-of-branch2.sh | 2 +- t/t9167-git-svn-cmd-branch-subproject.sh | 2 +- 20 files changed, 131 insertions(+), 153 deletions(-) -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/2] git-svn: only look at the new parts of svn:mergeinfo
Thanks! I still haven't gotten around to looking at svn:mergeinfo things, but this passes tests so I'm inclined to merge this unless somebody disagrees. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2] git-svn: only look at the root path for svn:mergeinfo
Jakob Stoklund Olesen stokl...@2pi.dk wrote: Subversion can put mergeinfo on any sub-directory to track cherry-picks. Since cherry-picks are not represented explicitly in git, git-svn should just ignore it. Hi, was git-svn trying to track cherry-picks as merge before? This changes behavior a bit, so two independent users of git-svn may not have identical histories as a result, correct? Can you add a test to ensure this behavior is preserved? Thanks. Sorry, I've never looked at mergeinfo myself, mainly relying on Sam + tests for this. [1] - Historically, git-svn (using defaults) has always tried to preserve identical histories for independent users across different git-svn versions. However, mergeinfo may be enough of a corner-case where we can make an exception. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-svn Rewrites Some Commits, but not All
Piotr Krukowiecki piotr.krukowie...@gmail.com wrote: On Mon, Apr 28, 2014 at 9:26 PM, Aaron Laws dartm...@gmail.com wrote: The way I understand it, when `git svn dcommit` is run, new commits are created (A' is created from A adding SVN information), then the current branch is moved to point to A'. Why don't we move any other refs that were pointing to A over to A' ? What would be the point of continuing to point to A? I'm interested in looking into coding this change to git-svn, but I would like to hear some feedback first. I think A' might not always be simply (A + SVN info). I think you can dcommit when you're not up to date. So A' will have a different parent than A (will be automatically rebased on top of current branch tip). Other refs pointing to A might be used as bookmarks, and moving them from A to A' would be a significant change. Right, I would not want git rebase (what dcommit uses internally) to scan all the branches in my repository and try to update them behind my back. It's far too surprising and potentially dangerous. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] config: preserve config file permissions on edits
Users may already store sensitive data such as imap.pass in
.git/config; making the file world-readable when git config
is called to edit means their password would be compromised
on a shared system.
Signed-off-by: Eric Wong normalper...@yhbt.net
---
config.c | 7 +++
t/t1300-repo-config.sh | 6 ++
2 files changed, 13 insertions(+)
diff --git a/config.c b/config.c
index a30cb5c..a0b6756 100644
--- a/config.c
+++ b/config.c
@@ -1636,6 +1636,13 @@ int git_config_set_multivar_in_file(const char
*config_filename,
MAP_PRIVATE, in_fd, 0);
close(in_fd);
+ if (fchmod(fd, st.st_mode 0) 0) {
+ error(fchmod on %s failed: %s,
+ lock-filename, strerror(errno));
+ ret = CONFIG_NO_WRITE;
+ goto out_free;
+ }
+
if (store.seen == 0)
store.seen = 1;
diff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh
index 58cd543..d87693e 100755
--- a/t/t1300-repo-config.sh
+++ b/t/t1300-repo-config.sh
@@ -1158,4 +1158,10 @@ test_expect_failure 'adding a key into an empty section
reuses header' '
test_cmp expect .git/config
'
+test_expect_success POSIXPERM,PERL 'preserves existing permissions' '
+ chmod 0600 .git/config
+ git config imap.pass Hunter2
+ perl -e die q(badperm) if ((stat(q(.git/config)))[2] 0) != 0600
+'
+
test_done
--
Eric Wong
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2] config: preserve config file permissions on edits
Users may already store sensitive data such as imap.pass in
.git/config; making the file world-readable when git config
is called to edit means their password would be compromised
on a shared system.
[v2: updated for section renames, as noted by Junio]
Signed-off-by: Eric Wong normalper...@yhbt.net
---
config.c | 16
t/t1300-repo-config.sh | 10 ++
2 files changed, 26 insertions(+)
diff --git a/config.c b/config.c
index a30cb5c..c227aa8 100644
--- a/config.c
+++ b/config.c
@@ -1636,6 +1636,13 @@ int git_config_set_multivar_in_file(const char
*config_filename,
MAP_PRIVATE, in_fd, 0);
close(in_fd);
+ if (fchmod(fd, st.st_mode 0) 0) {
+ error(fchmod on %s failed: %s,
+ lock-filename, strerror(errno));
+ ret = CONFIG_NO_WRITE;
+ goto out_free;
+ }
+
if (store.seen == 0)
store.seen = 1;
@@ -1784,6 +1791,7 @@ int git_config_rename_section_in_file(const char
*config_filename,
int out_fd;
char buf[1024];
FILE *config_file;
+ struct stat st;
if (new_name !section_name_is_ok(new_name)) {
ret = error(invalid section name: %s, new_name);
@@ -1805,6 +1813,14 @@ int git_config_rename_section_in_file(const char
*config_filename,
goto unlock_and_out;
}
+ fstat(fileno(config_file), st);
+
+ if (fchmod(out_fd, st.st_mode 0) 0) {
+ ret = error(fchmod on %s failed: %s,
+ lock-filename, strerror(errno));
+ goto out;
+ }
+
while (fgets(buf, sizeof(buf), config_file)) {
int i;
int length;
diff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh
index 58cd543..3f80ff0 100755
--- a/t/t1300-repo-config.sh
+++ b/t/t1300-repo-config.sh
@@ -1158,4 +1158,14 @@ test_expect_failure 'adding a key into an empty section
reuses header' '
test_cmp expect .git/config
'
+test_expect_success POSIXPERM,PERL 'preserves existing permissions' '
+ chmod 0600 .git/config
+ git config imap.pass Hunter2
+ perl -e \
+ die q(badset) if ((stat(q(.git/config)))[2] 0) != 0600
+ git config --rename-section imap pop
+ perl -e \
+ die q(badrename) if ((stat(q(.git/config)))[2] 0) != 0600
+'
+
test_done
--
Eric Wong
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[ANNOUNCE] ssoma - some sort of mail archiver (using git)
ssoma is a git-based mail archiver and transport. Email is injected via ssoma-mda(1) (MDA: mail delivery agent) on a server and may be shared (via git) and extracted to mbox, Maildir, or IMAP via ssoma(1). ssoma exists primarily as the mechanism (not policy) for public-inbox but may easily be used for other projects. Readers of public-inbox instances may install ssoma to extract messages into their favorite mail client for reading. See http://public-inbox.org/ for more information on how ssoma is used. Source code: git clone git://80x24.org/ssoma WWW: http://ssoma.public-inbox.org/ Installation info: http://ssoma.public-inbox.org/INSTALL.html Discussion: m...@public-inbox.org (no subscription required[1]) You may subscribe to m...@public-inbox.org via ssoma: LISTNAME is a name of your choosing: URL=git://public-inbox.org/meta LISTNAME=public-inbox # to initialize a maildir (this may be a new or existing maildir, # ssoma will not touch existing messages) # If you prefer mbox, use mbox:/path/to/mbox as the last argument ssoma add $LISTNAME $URL maildir:/path/to/maildir # read with your favorite MUA (only using mutt as an example) mutt -f /path/to/maildir # (or /path/to/mbox) # to keep your mbox or maildir up-to-date, periodically run the following: ssoma sync $LISTNAME # your MUA may modify and delete messages from the maildir or mbox, # this does not affect ssoma functionality at all # to sync all your ssoma subscriptions ssoma sync # You may wish to sync in your cronjob ssoma sync --cron If you prefer web browsers, you may also read the list at: http://public-inbox.org/meta/ without installing ssoma Or use an Atom feed reader: http://public-inbox.org/meta/atom.xml [1] - or possible at this point: public-inbox is an archives first, pull-based approach to mailing lists. Anybody may also run a push service on top of it, though. Mail repository format -- If you are uncomfortable running code in ssoma for any reason and would rather read directly from the git repository, the following document describes it: http://ssoma.public-inbox.org/ssoma_repository.txt Thanks for reading this far! -- Eric Wong -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v1 1/2] Remove 'git archimport'
Felipe Contreras felipe.contre...@gmail.com wrote: No updates since 2010, and no tests. Who benefits from this removal? Is this causing a maintenance burden for Junio? Plus, foreign SCM tools should live out-of-tree anyway. Even if so, there ought to be a transitionary period in case there are any users. We would need to warn potential users of its impending removal in the documentation and at runtime. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v1 1/2] Remove 'git archimport'
Felipe Contreras felipe.contre...@gmail.com wrote: Eric Wong wrote: Felipe Contreras felipe.contre...@gmail.com wrote: No updates since 2010, and no tests. Who benefits from this removal? Is this causing a maintenance burden for Junio? It is cruft that nobody uses and we are not even testing. We do not know nobody uses it. I have old GNU Arch projects I have not looked at in a decade. There is a small chance I may use archimport again (whether for nostalgia or contractual/legal reasons). Of course I know to extract archimport from history, but someone in the future may not know the existence of it. Plus, foreign SCM tools should live out-of-tree anyway. Even if so, there ought to be a transitionary period in case there are any users. We would need to warn potential users of its impending removal in the documentation and at runtime. All right, so you are OK with adding deprecation warnings whenever the tool is run, and a note in the documentation? No, I am not convinced existing foreign SCM tools should move out-of-tree. Perhaps something like the following would be helpful: Warning: Like GNU Arch, git-archimport is unmaintained. Please contact us at git@vger.kernel.org in case you run into problems, want to help, or just want to tell us it still works. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v1 1/2] Remove 'git archimport'
Felipe Contreras felipe.contre...@gmail.com wrote: As a minimal token that anybody might possibly be using it, I would like to see it work at least once. Since you said you have arch repos, can you confirm that it does something? Those repos are in offline/offsite storage and I do not have time to retrieve them. I've forgotten how to use tla to get a public repo, even. Eric Wong wrote: No, I am not convinced existing foreign SCM tools should move out-of-tree. Perhaps something like the following would be helpful: Tell that to Junio. If tools like git-remote-hg with tests and active maintanance and many users cannot be in the core, why should 'git archimport' be? Perhaps it's easier to deal with a benign, unmaintained tool than to deal with you as a maintainer? Would you at least be OK with a demotion to contrib/? I don't see the point of moving it around, even. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v1 1/2] Remove 'git archimport'
Thomas Adam tho...@xteddy.org wrote: I think I speak for everyone when I say: fuck off. I wouldn't put it so harshly... Felipe: I suggest you take a long vacation away from development. Whatever good you may be able to contribute today is drowned out by your behavior. The projects you are involved with will get by fine without you. To help you with your vacation, I shall start ignoring you. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] unblock and unignore SIGPIPE
Patrick Reynolds patrick.reyno...@github.com wrote: But in the real world, several real potential callers, including Perl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE ignored. s/Unicorn/Ruby/ But unicorn would ignore SIGPIPE it if Ruby did not; relying on SIGPIPE while doing any multiplexed I/O doesn't work well. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn.txt: Remove mentions of repack options
Lawrence Velázquez v...@larryv.me wrote: Git no longer seems to use these flags or their associated config keys; when they are present, git-svn outputs a message indicating that they are being ignored. Signed-off-by: Lawrence Velázquez v...@larryv.me Thanks, will queue. Signed-off-by: Eric Wong normalper...@yhbt.net -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2] git svn: info: correctly handle absolute path args
Calling git svn info $(pwd) would hit:
Reading from filehandle failed at ...
errors due to improper prefixing and canonicalization.
Strip the toplevel path from absolute filesystem paths to ensure
downstream canonicalization routines are only exposed to paths
tracked in git (or SVN).
Thanks to Andrej Manduch amand...@gmail.com for originally
noticing the issue and fixing my original version of this to handle
more corner cases such as /path/to/top/../top and
/path/to/top/../top/file as shown in the new test cases.
Signed-off-by: Andrej Manduch amand...@gmail.com
Signed-off-by: Eric Wong normalper...@yhbt.net
---
Eric Wong normalper...@yhbt.net wrote:
Thanks Andrej. I'll queue that on top of mine.
Can you turn that into a proper commit message with Subject?
Thanks.
I just squashed in your change and gave you credit.
Sorry, I forgot about this completely. Will test a bit more
and ask Junio to pull.
git-svn.perl| 22 --
t/t9119-git-svn-info.sh | 30 ++
2 files changed, 46 insertions(+), 6 deletions(-)
diff --git a/git-svn.perl b/git-svn.perl
index 1f41ee1..47cd6ea 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -1477,10 +1477,20 @@ sub cmd_commit_diff {
}
}
-
sub cmd_info {
- my $path = canonicalize_path(defined($_[0]) ? $_[0] : .);
- my $fullpath = canonicalize_path($cmd_dir_prefix . $path);
+ my $path_arg = defined($_[0]) ? $_[0] : '.';
+ my $path = $path_arg;
+ if ($path =~ m!\A/!) {
+ my $toplevel = eval {
+ my @cmd = qw/rev-parse --show-toplevel/;
+ command_oneline(\@cmd, STDERR = 0);
+ };
+ $path = canonicalize_path($path);
+ $path =~ s!\A\Q$toplevel\E/?!!;
+ $path = canonicalize_path($path);
+ } else {
+ $path = canonicalize_path($cmd_dir_prefix . $path);
+ }
if (exists $_[1]) {
die Too many arguments specified\n;
}
@@ -1501,14 +1511,14 @@ sub cmd_info {
# canonicalize_path() will return to make libsvn 1.5.x happy,
$path = . if $path eq ;
- my $full_url = canonicalize_url( add_path_to_url( $url, $fullpath ) );
+ my $full_url = canonicalize_url( add_path_to_url( $url, $path ) );
if ($_url) {
print $full_url\n;
return;
}
- my $result = Path: $path\n;
+ my $result = Path: $path_arg\n;
$result .= Name: . basename($path) . \n if $file_type ne dir;
$result .= URL: $full_url\n;
@@ -1539,7 +1549,7 @@ sub cmd_info {
}
my ($lc_author, $lc_rev, $lc_date_utc);
- my @args = Git::SVN::Log::git_svn_log_cmd($rev, $rev, --, $fullpath);
+ my @args = Git::SVN::Log::git_svn_log_cmd($rev, $rev, --, $path);
my $log = command_output_pipe(@args);
my $esc_color = qr/(?:\033\[(?:(?:\d+;)*\d*)?m)*/;
while ($log) {
diff --git a/t/t9119-git-svn-info.sh b/t/t9119-git-svn-info.sh
index ff19695..f16f323 100755
--- a/t/t9119-git-svn-info.sh
+++ b/t/t9119-git-svn-info.sh
@@ -74,6 +74,36 @@ test_expect_success 'info .'
test_cmp_info expected.info-dot actual.info-dot
+test_expect_success 'info $(pwd)' '
+ (cd svnwc; svn info $(pwd)) expected.info-pwd
+ (cd gitwc; git svn info $(pwd)) actual.info-pwd
+ grep -v ^Path: expected.info-pwd expected.info-np
+ grep -v ^Path: actual.info-pwd actual.info-np
+ test_cmp_info expected.info-np actual.info-np
+ test $(sed -ne \/^Path:/ s!/svnwc!!\ expected.info-pwd) = \
+$(sed -ne \/^Path:/ s!/gitwc!!\ actual.info-pwd)
+ '
+
+test_expect_success 'info $(pwd)/../___wc' '
+ (cd svnwc; svn info $(pwd)/../svnwc) expected.info-pwd
+ (cd gitwc; git svn info $(pwd)/../gitwc) actual.info-pwd
+ grep -v ^Path: expected.info-pwd expected.info-np
+ grep -v ^Path: actual.info-pwd actual.info-np
+ test_cmp_info expected.info-np actual.info-np
+ test $(sed -ne \/^Path:/ s!/svnwc!!\ expected.info-pwd) = \
+$(sed -ne \/^Path:/ s!/gitwc!!\ actual.info-pwd)
+ '
+
+test_expect_success 'info $(pwd)/../___wc//file' '
+ (cd svnwc; svn info $(pwd)/../svnwc//file) expected.info-pwd
+ (cd gitwc; git svn info $(pwd)/../gitwc//file) actual.info-pwd
+ grep -v ^Path: expected.info-pwd expected.info-np
+ grep -v ^Path: actual.info-pwd actual.info-np
+ test_cmp_info expected.info-np actual.info-np
+ test $(sed -ne \/^Path:/ s!/svnwc!!\ expected.info-pwd) = \
+$(sed -ne \/^Path:/ s!/gitwc!!\ actual.info-pwd)
+ '
+
test_expect_success 'info --url .' '
test $(cd gitwc; git svn info --url .) = $quoted_svnrepo
'
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info
[PATCH] git svn: find-rev allows short switches for near matches
Allow -B and -A to act as short aliases for --before and --after
options respectively. This reduces typing and hopefully allows
reuse of muscle memory for grep(1) users.
Signed-off-by: Eric Wong normalper...@yhbt.net
---
Will push to git://bogomips.org/git-svn.git
Documentation/git-svn.txt | 2 ++
git-svn.perl | 4 ++--
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index 14036cf..ef8ef1c 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -386,11 +386,13 @@ Any other arguments are passed directly to 'git log'
tree-ish to specify which branch should be searched). When given a
tree-ish, returns the corresponding SVN revision number.
+
+-B;;
--before;;
Don't require an exact match if given an SVN revision, instead find
the commit corresponding to the state of the SVN repository (on the
current branch) at the specified revision.
+
+-A;;
--after;;
Don't require an exact match if given an SVN revision; if there is
not an exact match return the closest match searching forward in the
diff --git a/git-svn.perl b/git-svn.perl
index 47cd6ea..a0ca31d 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -260,8 +260,8 @@ my %cmd = (
} ],
'find-rev' = [ \cmd_find_rev,
Translate between SVN revision numbers and tree-ish,
- { 'before' = \$_before,
- 'after' = \$_after } ],
+ { 'B|before' = \$_before,
+ 'A|after' = \$_after } ],
'rebase' = [ \cmd_rebase, Fetch and rebase your working directory,
{ 'merge|m|M' = \$_merge,
'verbose|v' = \$_verbose,
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3] git svn: info: correctly handle absolute path args
Thanks Johannes and brian. Diff against v2:
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -1480,13 +1480,30 @@ sub cmd_commit_diff {
sub cmd_info {
my $path_arg = defined($_[0]) ? $_[0] : '.';
my $path = $path_arg;
- if ($path =~ m!\A/!) {
+ if (File::Spec-file_name_is_absolute($path)) {
+ $path = canonicalize_path($path);
+
my $toplevel = eval {
my @cmd = qw/rev-parse --show-toplevel/;
command_oneline(\@cmd, STDERR = 0);
};
- $path = canonicalize_path($path);
- $path =~ s!\A\Q$toplevel\E/?!!;
+
+ # remove $toplevel from the absolute path:
+ my ($vol, $dirs, $file) = File::Spec-splitpath($path);
+ my (undef, $tdirs, $tfile) = File::Spec-splitpath($toplevel);
+ my @dirs = File::Spec-splitdir($dirs);
+ my @tdirs = File::Spec-splitdir($tdirs);
+ pop @dirs if $dirs[-1] eq '';
+ pop @tdirs if $tdirs[-1] eq '';
+ push @dirs, $file;
+ push @tdirs, $tfile;
+ while (@tdirs @dirs $tdirs[0] eq $dirs[0]) {
+ shift @dirs;
+ shift @tdirs;
+ }
+ $dirs = File::Spec-catdir(@dirs);
+ $path = File::Spec-catpath($vol, $dirs);
+
$path = canonicalize_path($path);
} else {
$path = canonicalize_path($cmd_dir_prefix . $path);
--8---
From: Eric Wong normalper...@yhbt.net
Subject: [PATCH v3] git svn: info: correctly handle absolute path args
Calling git svn info $(pwd) would hit:
Reading from filehandle failed at ...
errors due to improper prefixing and canonicalization.
Strip the toplevel path from absolute filesystem paths to ensure
downstream canonicalization routines are only exposed to paths
tracked in git (or SVN).
v2:
Thanks to Andrej Manduch for originally noticing the issue
and fixing my original version of this to handle
more corner cases such as /path/to/top/../top and
/path/to/top/../top/file as shown in the new test cases.
v3:
Fix pathname portability problems pointed out by Johannes Sixt
with a hint from brian m. carlson.
Cc: Johannes Sixt j...@kdbg.org
Cc: brian m. carlson sand...@crustytoothpaste.net
Signed-off-by: Andrej Manduch amand...@gmail.com
Signed-off-by: Eric Wong normalper...@yhbt.net
---
git-svn.perl| 39 +--
t/t9119-git-svn-info.sh | 30 ++
2 files changed, 63 insertions(+), 6 deletions(-)
diff --git a/git-svn.perl b/git-svn.perl
index 1f41ee1..40565cd 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -1477,10 +1477,37 @@ sub cmd_commit_diff {
}
}
-
sub cmd_info {
- my $path = canonicalize_path(defined($_[0]) ? $_[0] : .);
- my $fullpath = canonicalize_path($cmd_dir_prefix . $path);
+ my $path_arg = defined($_[0]) ? $_[0] : '.';
+ my $path = $path_arg;
+ if (File::Spec-file_name_is_absolute($path)) {
+ $path = canonicalize_path($path);
+
+ my $toplevel = eval {
+ my @cmd = qw/rev-parse --show-toplevel/;
+ command_oneline(\@cmd, STDERR = 0);
+ };
+
+ # remove $toplevel from the absolute path:
+ my ($vol, $dirs, $file) = File::Spec-splitpath($path);
+ my (undef, $tdirs, $tfile) = File::Spec-splitpath($toplevel);
+ my @dirs = File::Spec-splitdir($dirs);
+ my @tdirs = File::Spec-splitdir($tdirs);
+ pop @dirs if $dirs[-1] eq '';
+ pop @tdirs if $tdirs[-1] eq '';
+ push @dirs, $file;
+ push @tdirs, $tfile;
+ while (@tdirs @dirs $tdirs[0] eq $dirs[0]) {
+ shift @dirs;
+ shift @tdirs;
+ }
+ $dirs = File::Spec-catdir(@dirs);
+ $path = File::Spec-catpath($vol, $dirs);
+
+ $path = canonicalize_path($path);
+ } else {
+ $path = canonicalize_path($cmd_dir_prefix . $path);
+ }
if (exists $_[1]) {
die Too many arguments specified\n;
}
@@ -1501,14 +1528,14 @@ sub cmd_info {
# canonicalize_path() will return to make libsvn 1.5.x happy,
$path = . if $path eq ;
- my $full_url = canonicalize_url( add_path_to_url( $url, $fullpath ) );
+ my $full_url = canonicalize_url( add_path_to_url( $url, $path ) );
if ($_url) {
print $full_url\n;
return;
}
- my $result = Path: $path\n;
+ my $result = Path: $path_arg\n;
$result .= Name: . basename($path) . \n if $file_type ne dir;
$result .= URL: $full_url\n;
@@ -1539,7 +1566,7 @@ sub cmd_info {
}
my
[PATCH] git-svn: delay term initialization
On my Debian 7 system, this gives annoying warnings when the output
of git svn commands are redirected:
Unable to get Terminal Size. The TIOCGWINSZ ioctl didn't work.
The COLUMNS and LINES environment variables didn't work. The
resize program didn't work.
Signed-off-by: Eric Wong normalper...@yhbt.net
---
Also, manually tested to ensure dcommit --interactive works.
git-svn.perl | 18 +++---
1 file changed, 11 insertions(+), 7 deletions(-)
diff --git a/git-svn.perl b/git-svn.perl
index 40565cd..ce0d7e1 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -306,13 +306,16 @@ sub readline {
}
package main;
-my $term = eval {
- $ENV{GIT_SVN_NOTTY}
- ? new Term::ReadLine 'git-svn', \*STDIN, \*STDOUT
- : new Term::ReadLine 'git-svn';
-};
-if ($@) {
- $term = new FakeTerm $@: going non-interactive;
+my $term;
+sub term_init {
+ $term = eval {
+ $ENV{GIT_SVN_NOTTY}
+ ? new Term::ReadLine 'git-svn', \*STDIN, \*STDOUT
+ : new Term::ReadLine 'git-svn';
+ };
+ if ($@) {
+ $term = new FakeTerm $@: going non-interactive;
+ }
}
my $cmd;
@@ -424,6 +427,7 @@ sub ask {
my $default = $arg{default};
my $resp;
my $i = 0;
+ term_init() unless $term;
if ( !( defined($term-IN)
defined( fileno($term-IN) )
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn: delay term initialization
Eric Wong normalper...@yhbt.net wrote: On my Debian 7 system, this gives annoying warnings when the output s/gives/fixes/ -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PULL] git svn updates for master
Hi Junio, a couple of small changes and fixes. Most of these should be suitable for maint, too. The following changes since commit ce1d3a93a6405b8a0313491df3099919ed3d150f: Update draft release notes to 2.2 (2014-09-11 11:19:47 -0700) are available in the git repository at: git://bogomips.org/git-svn.git master for you to fetch changes up to 30d45f798d1a4b14759cd977b68be4476d66ea17: git-svn: delay term initialization (2014-09-14 08:08:54 +) Eric Wong (3): git svn: info: correctly handle absolute path args git svn: find-rev allows short switches for near matches git-svn: delay term initialization Lawrence Velázquez (1): git-svn.txt: Remove mentions of repack options Monard Vong (1): git-svn: branch: avoid systematic prompt for cert/pass Documentation/git-svn.txt | 17 ++--- git-svn.perl | 65 +++ t/t9119-git-svn-info.sh | 30 ++ 3 files changed, 81 insertions(+), 31 deletions(-) -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git svn's performance issue and strange pauses, and other thing
Hin-Tak Leung ht...@users.sourceforge.net wrote: (I am not on the list - please CC) Done, it is standard practice for git :) Thanks for git-svn - I use it instead of subversion itself for many years now. Just thought I'd ask/report a few issues I noticed for some time now, of tracking development of a particular subversion-based development project. Broadly speaking, I think there are 3 problems, especially noticeable against a particular repository, but to a lesser extent with some others too. - just doing git svn fetch --all seems to consume a lot of memory, for very little actual fetched changes. (in the 2GB+ region, sometimes). - git svn fetch --all also seems to take a long time too, for certain fetched changes. (in the minutes region). Jakob sent some patches a few months ago which seem to address the issue. Unfortunately we forgot about them :x Can you take a look at the following two mergeinfo-speedups in my repo? (git://bogomips.org/git-svn) Jakob Stoklund Olesen (2): git-svn: only look at the new parts of svn:mergeinfo git-svn: only look at the root path for svn:mergeinfo Also downloadable here: http://bogomips.org/git-svn.git/patch?id=9b258e721b30785357535 http://bogomips.org/git-svn.git/patch?id=73409a2145e93b436d74a Can you please give them a try? - I know I can probably just read the source, but I'd like to know why .git/svn/.caches is even larger than .git/objects (which supposedly contains everything that's of interest)? I hope this can be documented towards the end of the man-page, for example, of important parts of .git/svn (and what not to do with them...), without needing to 'read the source'. Here is part of du from a couple of days ago: 254816.git/objects 307056.git/svn/.caches 332452.git/svn 588064.git The actual .git/config is here - this should be sufficient info for somebody looking into experiencing the issues I mentioned above. IIRC, the caching is unique to mergeinfo, so perhaps Jakob's patches help, there, too. Sorry I don't understand the mergeinfo stuff more, I've never worked on a project which uses it. $ more .git/config [core] repositoryformatversion = 0 filemode = true bare = false logallrefupdates = true [svn-remote svn] url = https://svn.r-project.org/R fetch = trunk:refs/remotes/trunk branches = branches/*:refs/remotes/* tags = tags/*:refs/remotes/tags/* [pack] threads = 1 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git svn's performance issue and strange pauses, and other thing
Eric Wong normalper...@yhbt.net wrote: Jakob sent some patches a few months ago which seem to address the issue. Unfortunately we forgot about them :x Hin-Tak: have you tried Jakob's patches? I've taken another look, signed-off and pushed to my master. Can you take a look at the following two mergeinfo-speedups in my repo? (git://bogomips.org/git-svn) Jakob Stoklund Olesen (2): git-svn: only look at the new parts of svn:mergeinfo git-svn: only look at the root path for svn:mergeinfo Also downloadable here: http://bogomips.org/git-svn.git/patch?id=9b258e721b30785357535 http://bogomips.org/git-svn.git/patch?id=73409a2145e93b436d74a Can you please give them a try? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-svn performance
Fabian Schmied fabian.schm...@gmail.com wrote: Hi, I'm currently migrating an SVN repository to Git using git-svn (Git for Windows 1.8.3-preview20130601), and I'm experiencing severe performance problems with git svn fetch. Commits to the SVN trunk are fetched very fast (a few seconds or so per SVN revision), but commits to some branches (hotfix branches) are currently taking about 9 minutes per revision. I fear that the time per these commits is increasing and that indeed the migration might not be finishable at all. For the commits that take such a long time, git-svn always outputs lots of warnings about ignored SVN cherry-picks, and it tells me it can't find a revmap for the path being imported. (See [1].) AFAICS, the offending commits take place on some branches that include a lot of manually merged (SVN cherry-picked) revisions. Git-svn seems to be checking something (though I don't know what) that makes importing these revisions really slow. And it repeats this for every revision on these branches with increasing work to do. Is there anything I can do to speed this up? (I already tried increasing the --log-window-size to 500, didn't have any effect.) Can you take a look at the following two mergeinfo-speedups in my repo? (git://bogomips.org/git-svn) Jakob Stoklund Olesen (2): git-svn: only look at the new parts of svn:mergeinfo git-svn: only look at the root path for svn:mergeinfo Also downloadable here: http://bogomips.org/git-svn.git/patch?id=9b258e721b30785357535 http://bogomips.org/git-svn.git/patch?id=73409a2145e93b436d74a Hin-Tak (Cc-ed) reported good improvements with them, but also a large memory increase: http://mid.gmane.org/1412706046.90413.yahoomailba...@web172303.mail.ir2.yahoo.com Jakob (or anybody else): I suppose we could tie the new cached_mergeinfo* caches to disk-backed storage to avoid the memory bloat. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Specifying account profile in MSMTP
Robert Dailey rcdailey.li...@gmail.com wrote: Hey guys, I'm using MSMTP to define 2 accounts: Work email and personal email. If I send patches via email through Git at work, I want to use my work SMTP server and account information. Likewise at home for personal projects, I want to use my personal SMTP account. I put my .gitconfig in Dropbox and I share it across all of my machines, so I avoid putting my SMTP server information in my global config because I'd then have to use the same account everywhere. However, I do not define a default account in my MSMTP file (which is also in dropbox, in fact my whole home directory is in there). Is there a way to specify the MSMTP account to use at the command line when I run `git send-email`? If not, are there other good solutions to this problem? msmtprc can be configured to picks accounts based on the envelop sender specified by git send-email. I make sure my email address in the git commits is correct and use use envelopesender=auto for git send-email when using msmtp. Config examples below: --- ~/.gitconfig --- [sendemail] smtpserver = /usr/bin/msmtp envelopesender = auto /path/to/project_a/.git/config [user] email = a...@example.com /path/to/project_b/.git/config [user] email = b...@example.com ~/.msmtprc account a host ... from a...@example.com user a password ... auth ... account b host ... from b...@example.com user b password ... auth ... -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-svn performance
Eric Wong normalper...@yhbt.net wrote:
Hin-Tak (Cc-ed) reported good improvements with them, but also
a large memory increase:
This might reduce the pathname and internal hash overheads:
8---
From: Eric Wong normalper...@yhbt.net
Date: Sun, 19 Oct 2014 02:26:53 +
Subject: [PATCH] git-svn: simplify cached_mergeinfo layout
This reduces hash lookups for looking up cache data and will
simplify tying data to disk in the next commit.
Signed-off-by: Eric Wong normalper...@yhbt.net
---
perl/Git/SVN.pm | 20 ++--
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index b1a84d0..25dbcd5 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1708,15 +1708,17 @@ sub mergeinfo_changes {
my %minfo = map {split :, $_ } split \n, $mergeinfo_prop;
my $old_minfo = {};
+ # layout: $path = [ $rev, \%mergeinfo ]
+ my $cached_mergeinfo = $self-{cached_mergeinfo};
+
# Initialize cache on the first call.
- unless (defined $self-{cached_mergeinfo_rev}) {
- $self-{cached_mergeinfo_rev} = {};
- $self-{cached_mergeinfo} = {};
+ unless (defined $cached_mergeinfo) {
+ $cached_mergeinfo = $self-{cached_mergeinfo} = {};
}
- my $cached_rev = $self-{cached_mergeinfo_rev}{$old_path};
- if (defined $cached_rev $cached_rev == $old_rev) {
- $old_minfo = $self-{cached_mergeinfo}{$old_path};
+ my $cached = $cached_mergeinfo-{$old_path};
+ if (defined $cached $cached-[0] == $old_rev) {
+ $old_minfo = $cached-[1];
} else {
my $ra = $self-ra;
# Give up if $old_path isn't in the repo.
@@ -1733,13 +1735,11 @@ sub mergeinfo_changes {
$props-{svn:mergeinfo};
$old_minfo = \%omi;
}
- $self-{cached_mergeinfo}{$old_path} = $old_minfo;
- $self-{cached_mergeinfo_rev}{$old_path} = $old_rev;
+ $cached_mergeinfo-{$old_path} = [ $old_rev, $old_minfo ];
}
# Cache the new mergeinfo.
- $self-{cached_mergeinfo}{$path} = \%minfo;
- $self-{cached_mergeinfo_rev}{$path} = $rev;
+ $cached_mergeinfo-{$path} = [ $rev, \%minfo ];
my %changes = ();
foreach my $p (keys %minfo) {
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-svn performance
Eric Wong normalper...@yhbt.net wrote:
This reduces hash lookups for looking up cache data and will
simplify tying data to disk in the next commit.
I considered the following, but GDBM might not be readily available on
non-POSIX platforms. I think the other problem is the existing caches
are still in memory (whether YAML or Storable) even if disk-backed,
causing a large amount of memory usage anyways.
(Both patches on top of Jakob's)
-
Subject: [RFC] git-svn: tie cached_mergeinfo to a GDBM_File store
This should reduce per-instance memory usage by allowing
serialization to disk. Using the existing Memoize::Storable
or YAML backends does not allow fast lookups.
GDBM_File should be available in most Perl installations
and should not pose unnecessary burden
---
perl/Git/SVN.pm | 19 ---
1 file changed, 16 insertions(+), 3 deletions(-)
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index 25dbcd5..3e477c7 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -14,6 +14,7 @@ use IPC::Open3;
use Memoize; # core since 5.8.0, Jul 2002
use Memoize::Storable;
use POSIX qw(:signal_h);
+use Storable qw(freeze thaw);
use Git qw(
command
@@ -1713,10 +1714,21 @@ sub mergeinfo_changes {
# Initialize cache on the first call.
unless (defined $cached_mergeinfo) {
- $cached_mergeinfo = $self-{cached_mergeinfo} = {};
+ my %hash;
+ eval '
+ require File::Temp;
+ use GDBM_File;
+ my $fh = File::Temp-new(TEMPLATE = mergeinfo.);
+ $self-{cached_mergeinfo_fh} = $fh;
+ $fh-unlink_on_destroy(1);
+ tie %hash = GDBM_File, $fh-filename, GDBM_WRCREAT, 0600;
+ ';
+ $cached_mergeinfo = $self-{cached_mergeinfo} = \%hash;
}
my $cached = $cached_mergeinfo-{$old_path};
+ $cached = thaw($cached) if defined $cached;
+
if (defined $cached $cached-[0] == $old_rev) {
$old_minfo = $cached-[1];
} else {
@@ -1735,11 +1747,12 @@ sub mergeinfo_changes {
$props-{svn:mergeinfo};
$old_minfo = \%omi;
}
- $cached_mergeinfo-{$old_path} = [ $old_rev, $old_minfo ];
+ $cached_mergeinfo-{$old_path} =
+ freeze([ $old_rev, $old_minfo ]);
}
# Cache the new mergeinfo.
- $cached_mergeinfo-{$path} = [ $rev, \%minfo ];
+ $cached_mergeinfo-{$path} = freeze([ $rev, \%minfo ]);
my %changes = ();
foreach my $p (keys %minfo) {
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git svn's performance issue and strange pauses, and other thing
Hin-Tak Leung ht...@users.sourceforge.net wrote:
The new clone has:
--
$ ls -ltr .git/svn/.caches/
total 144788
-rw-rw-r--. 1 Hin-Tak Hin-Tak 1166138 Oct 7 13:44 lookup_svn_merge.yaml
-rw-rw-r--. 1 Hin-Tak Hin-Tak 72849741 Oct 7 13:48 check_cherry_pick.yaml
-rw-rw-r--. 1 Hin-Tak Hin-Tak 1133855 Oct 7 13:49 has_no_changes.yaml
-rw-rw-r--. 1 Hin-Tak Hin-Tak 73109005 Oct 7 13:53 _rev_list.yaml
--
The old clone has:
snip
-rw-rw-r--. 1 Hin-Tak Hin-Tak 40241189 Oct 5 16:42 lookup_svn_merge.yaml
-rw-rw-r--. 1 Hin-Tak Hin-Tak 225323456 Oct 5 16:49 check_cherry_pick.yaml
-rw-rw-r--. 1 Hin-Tak Hin-Tak242547 Oct 5 16:49 has_no_changes.yaml
-rw-rw-r--. 1 Hin-Tak Hin-Tak 24120007 Oct 5 16:50 _rev_list.yaml
--
I had to suspend somewhat around r59000 - but it is interesting to see
that the max memory consumption of the later part is almost double?
and it also runs at 100% rather than 60% overall; I don't know what
to make of that - probably just smaller changes versus
larger ones, or different time of day and network loads (yes,
I guess it is just bandwidth-limited?, since the bulk of CPU time is in system
rather than user).
git-svn memory usage is insane, and we need to reduce it.
(on Linux, fork() performance is reduced as memory size of the parent
grows, and I don't think we can easily call vfork() from Perl)
I am somwhat worry about the dramatic difference between the two .svn/.caches
-
check_cherry_pick.yaml is 225MB in one and 73MB in the other, and also
_rev_list.yaml is opposite - 24MB vs 73MB. How do I reconcile that?
Calling patterns changed, and it looks like Jakob's changes avoided some
calls. The main thing to care about:
Does the repository history look right?
The check_cherry_pick cache can be made smaller, too:
--- 8 -
From: Eric Wong normalper...@yhbt.net
Subject: [PATCH] git-svn: reduce check_cherry_pick cache overhead
We do not need to store entire lists of commits, only the
number of incomplete and the first commit for reference.
This reduces the amount of data we need to store in memory
and on disk stores.
Signed-off-by: Eric Wong normalper...@yhbt.net
---
perl/Git/SVN.pm | 28 +++-
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index 25dbcd5..b2d37cb 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1537,7 +1537,7 @@ sub _rev_list {
@rv;
}
-sub check_cherry_pick {
+sub check_cherry_pick2 {
my $base = shift;
my $tip = shift;
my $parents = shift;
@@ -1552,7 +1552,8 @@ sub check_cherry_pick {
delete $commits{$commit};
}
}
- return (keys %commits);
+ my @k = (keys %commits);
+ return (scalar @k, $k[0]);
}
sub has_no_changes {
@@ -1597,7 +1598,7 @@ sub tie_for_persistent_memoization {
mkpath([$cache_path]) unless -d $cache_path;
my %lookup_svn_merge_cache;
- my %check_cherry_pick_cache;
+ my %check_cherry_pick2_cache;
my %has_no_changes_cache;
my %_rev_list_cache;
@@ -1608,11 +1609,11 @@ sub tie_for_persistent_memoization {
LIST_CACHE = ['HASH' = \%lookup_svn_merge_cache],
;
- tie_for_persistent_memoization(\%check_cherry_pick_cache,
- $cache_path/check_cherry_pick);
- memoize 'check_cherry_pick',
+ tie_for_persistent_memoization(\%check_cherry_pick2_cache,
+ $cache_path/check_cherry_pick2);
+ memoize 'check_cherry_pick2',
SCALAR_CACHE = 'FAULT',
- LIST_CACHE = ['HASH' = \%check_cherry_pick_cache],
+ LIST_CACHE = ['HASH' = \%check_cherry_pick2_cache],
;
tie_for_persistent_memoization(\%has_no_changes_cache,
@@ -1636,7 +1637,7 @@ sub tie_for_persistent_memoization {
$memoized = 0;
Memoize::unmemoize 'lookup_svn_merge';
- Memoize::unmemoize 'check_cherry_pick';
+ Memoize::unmemoize 'check_cherry_pick2';
Memoize::unmemoize 'has_no_changes';
Memoize::unmemoize '_rev_list';
}
@@ -1648,7 +1649,8 @@ sub tie_for_persistent_memoization {
return unless -d $cache_path;
for my $cache_file (($cache_path/lookup_svn_merge,
-$cache_path/check_cherry_pick,
+$cache_path/check_cherry_pick, # old
+$cache_path/check_cherry_pick2,
$cache_path/has_no_changes)) {
for my $suffix (qw(yaml db)) {
my $file = $cache_file.$suffix;
@@ -1817,15 +1819,15 @@ sub find_extra_svn_parents
Re: git-svn performance
Jakob Stoklund Olesen stokl...@2pi.dk wrote:
If cached_mergeinfo is using too much memory, you can probably drop
that cache entirely. IIRC, it didn't give that much of a speed up.
I am surprised that it is using a lot of memory, though. There is only
one entry per SVN branch.
Something like the below? (on top of your original two patches)
Pushed to my master @ git://bogomips.org/git-svn.git
Eric Wong (2):
git-svn: reduce check_cherry_pick cache overhead
git-svn: cache only mergeinfo revisions
Jakob Stoklund Olesen (2):
git-svn: only look at the new parts of svn:mergeinfo
git-svn: only look at the root path for svn:mergeinfo
git-svn still seems to have some excessive memory usage problems,
even independenty of mergeinfo stuff.
--8
From: Eric Wong normalper...@yhbt.net
Date: Mon, 20 Oct 2014 01:02:53 +
Subject: [PATCH] git-svn: cache only mergeinfo revisions
This should reduce excessive memory usage from the new mergeinfo
caches without hurting performance too much, assuming reasonable
latency to the SVN server.
Cc: Hin-Tak Leung ht...@users.sourceforge.net
Suggested-by: Jakob Stoklund Olesen stokl...@2pi.dk
Signed-off-by: Eric Wong normalper...@yhbt.net
---
perl/Git/SVN.pm | 22 --
1 file changed, 8 insertions(+), 14 deletions(-)
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index 171af37..f8a75b1 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1713,13 +1713,10 @@ sub mergeinfo_changes {
# Initialize cache on the first call.
unless (defined $self-{cached_mergeinfo_rev}) {
$self-{cached_mergeinfo_rev} = {};
- $self-{cached_mergeinfo} = {};
}
my $cached_rev = $self-{cached_mergeinfo_rev}{$old_path};
- if (defined $cached_rev $cached_rev == $old_rev) {
- $old_minfo = $self-{cached_mergeinfo}{$old_path};
- } else {
+ unless (defined $cached_rev $cached_rev == $old_rev) {
my $ra = $self-ra;
# Give up if $old_path isn't in the repo.
# This is probably a merge on a subtree.
@@ -1728,19 +1725,16 @@ sub mergeinfo_changes {
directory didn't exist in r$old_rev\n;
return {};
}
- my (undef, undef, $props) =
- $self-ra-get_dir($old_path, $old_rev);
- if (defined $props-{svn:mergeinfo}) {
- my %omi = map {split :, $_ } split \n,
- $props-{svn:mergeinfo};
- $old_minfo = \%omi;
- }
- $self-{cached_mergeinfo}{$old_path} = $old_minfo;
- $self-{cached_mergeinfo_rev}{$old_path} = $old_rev;
}
+ my (undef, undef, $props) = $self-ra-get_dir($old_path, $old_rev);
+ if (defined $props-{svn:mergeinfo}) {
+ my %omi = map {split :, $_ } split \n,
+ $props-{svn:mergeinfo};
+ $old_minfo = \%omi;
+ }
+ $self-{cached_mergeinfo_rev}{$old_path} = $old_rev;
# Cache the new mergeinfo.
- $self-{cached_mergeinfo}{$path} = \%minfo;
$self-{cached_mergeinfo_rev}{$path} = $rev;
my %changes = ();
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn.txt: advertise pushurl with dcommit
Sveinung Kvilhaugsvik sveinun...@users.sourceforge.net wrote: Advertise that the svn-remote.name.pushurl config key allows specifying the commit URL for the entire SVN repository in the documenation of git svn's dcommit command. Signed-off-by: Sveinung Kvilhaugsvik sveinun...@users.sourceforge.net Thanks, minor grammar+spelling fix to the commit message: Advertise that the svn-remote.name.pushurl config key allows specifying the commit URL for the entire SVN repository in the documentation of the git svn dcommit command. Signed-off-by: Eric Wong normalper...@yhbt.net Pushed with updated commit message to master of git://bogomips.org/git-svn -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn: merge: fix rooturl/branchurl match check
Tommaso Colombo tommaso.colo...@outlook.com wrote:
When populating svn:mergeinfo, git-svn merge checks if the merge parent
of the merged branch is under the same root as the git-svn repository.
This was implemented comparing $gs-repos_root with the return value of
of cmt_metadata for the merge parent. However, the first may contain a
username, whereas the second does not. In this case the comparison
fails.
Remove the username from $gs-repos_root before performing the
comparison.
Thanks. Commit makes sense, but one of the test cases fails for me,
can you check it out?
$ make make t9161-git-svn-mergeinfo-push.sh -C t GIT_TEST_OPTS='-i -v'
snip successes
ok 11 - reintegration merge
expecting success:
mergeinfo=$(svn_cmd propget svn:mergeinfo $svnrepo/branches/svnb4)
test $mergeinfo = /branches/svnb1:2-4,7-9,13-18
/branches/svnb2:3,8,16-17
/branches/svnb3:4,9
/branches/svnb5:6,11
not ok 12 - check reintegration mergeinfo
#
# mergeinfo=$(svn_cmd propget svn:mergeinfo
$svnrepo/branches/svnb4)
# test $mergeinfo = /branches/svnb1:2-4,7-9,13-18
# /branches/svnb2:3,8,16-17
# /branches/svnb3:4,9
# /branches/svnb5:6,11
#
make: *** [t9161-git-svn-mergeinfo-push.sh] Error 1
make: Leaving directory `/home/ew/git-core/t'
You'll also need to sign-off (see Documentation/SubmittingPatches)
It is required (project policy, not mine)
@@ -729,7 +730,7 @@ sub populate_merge_info {
}
my $branchpath = $1;
- my $ra = Git::SVN::Ra-new($branchurl);
+ my $ra =
Git::SVN::Ra-new(add_path_to_url($gs-repos_root, $branchpath));
Also, please keep long lines wrapped to = 80 columns.
Thanks again.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-svn: clear global SVN pool between get_log invocations
During long fetches with many revisions, some SVN functions are not
always using local pools and some gets into the global pool. Ensure
we free that memory occasionally between SVN::RA::get_log calls to
reduce memory growth.
This reduces memory usage over the course of fetching 10K revisions
using a test repository created with the script at the end of this
commit message.
As reported by time(1) on my x86-64 system:
before: 54024k
after: 47344k
-8--
set -e
tmp=$(mktemp -d svntestrepo-)
svnadmin create $tmp
repo=file://$(cd $tmp pwd)
svn co $repo $tmp/wd
cd $tmp/wd
if ! test -f a
then
a
svn add a
svn commit -m 'A'
fi
nr=1
while test $nr -gt 0
do
echo $nr a
svn commit -q -m A
nr=$((nr - 1))
done
echo repository created in $repo
-8--
Signed-off-by: Eric Wong normalper...@yhbt.net
---
Pushed to master of git://bogomips.org/git-svn
However, memory usage still seems to grow infinitely even in this simple
fetch case, so are other memory leaks.
perl/Git/SVN/Ra.pm | 6 ++
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm
index a7b0119..3df4591 100644
--- a/perl/Git/SVN/Ra.pm
+++ b/perl/Git/SVN/Ra.pm
@@ -237,10 +237,6 @@ sub trees_match {
my $ctx = SVN::Client-new(auth = _auth_providers);
my $out = IO::File-new_tmpfile;
- # older SVN (1.1.x) doesn't take $pool as the last parameter for
- # $ctx-diff(), so we'll create a default one
- my $pool = SVN::Pool-new_default_sub;
-
$ra_invalid = 1; # this will open a new SVN::Ra connection to $url1
$ctx-diff([], $url1, $rev1, $url2, $rev2, 1, 1, 0, $out, $out);
$out-flush;
@@ -453,6 +449,7 @@ sub gs_fetch_loop_common {
$_[0] = undef;
$self = undef;
$RA = undef;
+ SVN::Core-gpool-clear;
$self = Git::SVN::Ra-new($ra_url);
$ra_invalid = undef;
}
@@ -468,6 +465,7 @@ sub gs_fetch_loop_common {
my $k = svn-remote.$g-{remote}.$g-{t}-maxRev;
Git::SVN::tmp_config($k, $max);
}
+ SVN::Core-gpool-clear;
last if $max = $head;
$min = $max + 1;
$max += $inc;
--
EW
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-svn performance
Jakob Stoklund Olesen stokl...@2pi.dk wrote:
Yes, but I think you can remove cached_mergeinfo_rev too.
Thanks, pushed the patch at the bottom, too.
Also started working on some memory reductions here:
http://mid.gmane.org/20141021033912.ga27...@dcvr.yhbt.net
But there seem to be more problems :
8-
From: Eric Wong normalper...@yhbt.net
Date: Tue, 21 Oct 2014 06:23:22 +
Subject: [PATCH] git-svn: remove mergeinfo rev caching
This should further reduce memory usage from the new mergeinfo
speedups without hurting performance too much, assuming
reasonable latency to the SVN server.
Cc: Hin-Tak Leung ht...@users.sourceforge.net
Suggested-by: Jakob Stoklund Olesen stokl...@2pi.dk
Signed-off-by: Eric Wong normalper...@yhbt.net
---
perl/Git/SVN.pm | 30 +-
1 file changed, 9 insertions(+), 21 deletions(-)
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index f8a75b1..4364506 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1710,32 +1710,20 @@ sub mergeinfo_changes {
my %minfo = map {split :, $_ } split \n, $mergeinfo_prop;
my $old_minfo = {};
- # Initialize cache on the first call.
- unless (defined $self-{cached_mergeinfo_rev}) {
- $self-{cached_mergeinfo_rev} = {};
- }
-
- my $cached_rev = $self-{cached_mergeinfo_rev}{$old_path};
- unless (defined $cached_rev $cached_rev == $old_rev) {
- my $ra = $self-ra;
- # Give up if $old_path isn't in the repo.
- # This is probably a merge on a subtree.
- if ($ra-check_path($old_path, $old_rev) != $SVN::Node::dir) {
- warn W: ignoring svn:mergeinfo on $old_path, ,
- directory didn't exist in r$old_rev\n;
- return {};
- }
- }
- my (undef, undef, $props) = $self-ra-get_dir($old_path, $old_rev);
+ my $ra = $self-ra;
+ # Give up if $old_path isn't in the repo.
+ # This is probably a merge on a subtree.
+ if ($ra-check_path($old_path, $old_rev) != $SVN::Node::dir) {
+ warn W: ignoring svn:mergeinfo on $old_path, ,
+ directory didn't exist in r$old_rev\n;
+ return {};
+ }
+ my (undef, undef, $props) = $ra-get_dir($old_path, $old_rev);
if (defined $props-{svn:mergeinfo}) {
my %omi = map {split :, $_ } split \n,
$props-{svn:mergeinfo};
$old_minfo = \%omi;
}
- $self-{cached_mergeinfo_rev}{$old_path} = $old_rev;
-
- # Cache the new mergeinfo.
- $self-{cached_mergeinfo_rev}{$path} = $rev;
my %changes = ();
foreach my $p (keys %minfo) {
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn: clear global SVN pool between get_log invocations
Eric Wong normalper...@yhbt.net wrote: + SVN::Core-gpool-clear; Unfortunately, SVN::Core-gpool is not available in older SVN, but I'm cooking a better patch which saves even more memory. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-svn: reload RA every log-window-size
Despite attempting to use local memory pools everywhere we can,
(including our call to SVN::Ra::do_update and all subsequent reporter
calls), there does not appear to be a way to force the Git::SVN::Fetcher
callbacks to use a pool other than the per-SVN::Ra pool.
Git::SVN::Fetcher ends up using the main RA pool which grows
monotonically in size for the lifetime of the RA object.
Thus the only way to free that memory appears to be to destroy and
recreate the RA connection for at every --log-window-size interval.
This reduces memory usage over the course of fetching 10K revisions
using a test repository created with the script at the end of this
commit message.
As reported by time(1) on my x86-64 system:
before: 54024k
after: 28680k
Unfortunately, there remains some yet-to-be-tracked-down slow memory
growth which would be evident as the `nr' parameter increases in
the repository generation script:
-8--
set -e
tmp=$(mktemp -d svntestrepo-)
svnadmin create $tmp
repo=file://$(cd $tmp pwd)
svn co $repo $tmp/wd
cd $tmp/wd
if ! test -f a
then
a
svn add a
svn commit -m 'A'
fi
nr=1
while test $nr -gt 0
do
echo $nr a
svn commit -q -m A
nr=$((nr - 1))
done
echo repository created in $repo
-8--
Signed-off-by: Eric Wong normalper...@yhbt.net
---
Eric Wong normalper...@yhbt.net wrote:
Eric Wong normalper...@yhbt.net wrote:
+ SVN::Core-gpool-clear;
Unfortunately, SVN::Core-gpool is not available in older SVN,
but I'm cooking a better patch which saves even more memory.
perl/Git/SVN/Ra.pm | 21 +
1 file changed, 13 insertions(+), 8 deletions(-)
diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm
index a7b0119..1828519 100644
--- a/perl/Git/SVN/Ra.pm
+++ b/perl/Git/SVN/Ra.pm
@@ -376,10 +376,19 @@ sub longest_common_path {
sub gs_fetch_loop_common {
my ($self, $base, $head, $gsv, $globs) = @_;
return if ($base $head);
+ my $gpool = SVN::Pool-new_default;
+ my $ra_url = $self-url;
+ my $reload_ra = sub {
+ $_[0] = undef;
+ $self = undef;
+ $RA = undef;
+ $gpool-clear;
+ $self = Git::SVN::Ra-new($ra_url);
+ $ra_invalid = undef;
+ };
my $inc = $_log_window_size;
my ($min, $max) = ($base, $head $base + $inc ? $head : $base + $inc);
my $longest_path = longest_common_path($gsv, $globs);
- my $ra_url = $self-url;
my $find_trailing_edge;
while (1) {
my %revs;
@@ -449,13 +458,7 @@ sub gs_fetch_loop_common {
$g-{t}-maxRev;
Git::SVN::tmp_config($k, $r);
}
- if ($ra_invalid) {
- $_[0] = undef;
- $self = undef;
- $RA = undef;
- $self = Git::SVN::Ra-new($ra_url);
- $ra_invalid = undef;
- }
+ $reload_ra-() if $ra_invalid;
}
# pre-fill the .rev_db since it'll eventually get filled in
# with '0' x40 if something new gets committed
@@ -472,6 +475,8 @@ sub gs_fetch_loop_common {
$min = $max + 1;
$max += $inc;
$max = $head if ($max $head);
+
+ $reload_ra-();
}
Git::SVN::gc();
}
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Anomaly with the new code - Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: I keep tabs of a particular svn repository over many years and run git svn fetch --all every few days. So that's the old clone. Since this discussion started, I made a new one with git 2.1.0 patched with the first two patches below, a couple of weeks ago. And I ran 'git svn fetch --all' on both every few days since. I have added a few more patches, so the whole list is the 6 below against 2.1.0. The latest fetch is really strange - the fetch against the new clone took almost twice as long and uses almost twice as much memory, vs against the old. 17 min, 800 MB vs 10 min 400MB. Details below. Maybe this is a performance issue about how the clones were made? Memory usage seems to grow with the amount of revisions fetched, see below. And higher memory means slower fork() on Linux systems. 0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch 0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patch 0003-git-svn-reduce-check_cherry_pick-cache-overhead.patch 0004-git-svn-cache-only-mergeinfo-revisions.patch 0006-git-svn-clear-global-SVN-pool-between-get_log-invoca.patch 0006 is insufficient and incompatible with older SVN. I pushed git-svn: reload RA every log-window-size (commit dfa72fdb96befbd790f623bb2909a347176753c2) instead which saves much more memory: http://mid.gmane.org/20141024225352.gb31...@dcvr.yhbt.net But there still seems to be some slow growth with many revisions which is not mergeinfo-related. 0007-git-svn-remove-mergeinfo-rev-caching.patch I think it is also safe to remove the _rev_list memoization since it uses a lot of memory. The remaining caches should be tiny (but useful, I think). -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: Comparing trunk of old and new, I see one difference - One short commit message is missing in the *old* (the Add checkPoFiles etc. part) and so all the sha1 afterwards differed. Is that an old bug that's fixed and therefore I should throw away the old clone? I don't recall a bug which would cause a revision to be skipped. I suppose it's alright now the new clone has that revision. Perhaps there was a power outage or improper shutdown? At least we can be glad current versions see this revision... -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: the old didn't missing a revision - just a revision 'message' - blank instead of 3 words, above the git svn id. I supppse it is possible some power problem or etc caused this. I'll check the other branches as well, and possibly clone again to be sure. ( The new clone did have one break...) Oh, there's a possibility the commit message in SVN was edited/added after-the-fact, but that depends on the SVN admin (most never allow or do it). -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-svn: remove unnecessary DESTROY override
This override was probably never necessary, but most likely a no-op
as it does not appear to do anything in SVN::Ra itself.
Signed-off-by: Eric Wong normalper...@yhbt.net
---
perl/Git/SVN/Ra.pm | 4
1 file changed, 4 deletions(-)
diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm
index 1828519..e326849 100644
--- a/perl/Git/SVN/Ra.pm
+++ b/perl/Git/SVN/Ra.pm
@@ -177,10 +177,6 @@ sub get_dir {
wantarray ? (\%dirents, $r, $props) : \%dirents;
}
-sub DESTROY {
- # do not call the real DESTROY since we store ourselves in $RA
-}
-
# get_log(paths, start, end, limit,
# discover_changed_paths, strict_node_history, receiver)
sub get_log {
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-svn: save a little memory as fetch progresses
There is no reason to keep entries in the %revs hash after we're
done processing a revision, so allow entries become freed as
processing continues.
Signed-off-by: Eric Wong normalper...@yhbt.net
---
perl/Git/SVN/Ra.pm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm
index e326849..5bc5b4e 100644
--- a/perl/Git/SVN/Ra.pm
+++ b/perl/Git/SVN/Ra.pm
@@ -431,7 +431,7 @@ sub gs_fetch_loop_common {
my %exists = map { $_-path = $_ } @$gsv;
foreach my $r (sort {$a = $b} keys %revs) {
- my ($paths, $logged) = @{$revs{$r}};
+ my ($paths, $logged) = @{delete $revs{$r}};
foreach my $gs ($self-match_globs(\%exists, $paths,
$globs, $r)) {
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3] git-svn: memoize _rev_list and rebuild
manjian2...@gmail.com wrote: From: lin zuojian manjian2...@gmail.com According to profile data, _rev_list and rebuild consume a large portion of time. Memoize the results of _rev_list and memoize rebuild internals to avoid subprocess invocation. Hi, I am considering reverting _rev_list memoization because it can increases memory usage a lot, which hurting fork() performance on Linux. Can you tell me if your original results are under Linux or another kernel? I may only disable _rev_list memoization under Linux and leave other OSes unaffected, too. I've merged some patches from Jakob to improve svn:mergeinfo performance along with some followup patches to reduce memory use. However memory bloat is still a problem. Currently in my master branch of git://bogomips.org/git-svn Eric Wong (4): git-svn: reduce check_cherry_pick cache overhead git-svn: cache only mergeinfo revisions git-svn: clear global SVN pool between get_log invocations git-svn: remove mergeinfo rev caching Jakob Stoklund Olesen (2): git-svn: only look at the new parts of svn:mergeinfo git-svn: only look at the root path for svn:mergeinfo -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Anomaly with the new code - Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: On Sat, Oct 25, 2014 00:34 BST Eric Wong wrote: Hin-Tak Leung ht...@users.sourceforge.net wrote: 0006-git-svn-clear-global-SVN-pool-between-get_log-invoca.patch 0006 is insufficient and incompatible with older SVN. I pushed git-svn: reload RA every log-window-size (commit dfa72fdb96befbd790f623bb2909a347176753c2) instead which saves much more memory: it is fetching against the new clone taking twice as long and consuming twice as much memory. Ugh, I've only tested git-svn: reload RA every log-window-size with file:// repos so far, so it looks like I'll need to setup remote repos on my test system to test. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: btw, git svn seems to disallow single word commit messages (or is it a svn config?). i found that i could not do git svn dcommit, when i had merely did git commit -m 'typos', for example, for an svn repo i have write access to. (I don't have them many such things, so it is difficult to tell whether it is a repo config, or a git svn strangeness). i just do rebase and do 'typo correction' or something before re-dcommit in the past. Probably an SVN hook preventing it. git-svn test cases such as t/t9118-git-svn-funky-branch-names.sh do single word commits. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-svn: disable _rev_list memoization
This memoization appears unneeded as the check_cherry_pick2 cache is
in front of it does enough.
With this change applied, importing from local svn+ssh and http copies
of the R repo[1] takes only 2:00 (2 hours) on my system and the git-svn
process never uses more than 60MB RSS on my x86-64 GNU/Linux system[2].
This 60M measurement is only for the git-svn Perl process itself and
does not include memory used by git subprocesses accessing large packs
(subprocess memory usage _is_ measured by my time(1) tool).
Before this change, an import took longer (2:20) on svn+ssh:// but
git-svn used around 240MB during the imports. Worse yet, git-svn
ballooned to over 400M when writing out the cache to the filesystem.
I also tried removing memoization for `has_no_changes', too, but a
local copy of the R repository(*) was not close to finishing within
10 hours on my system.
[1] http://svn.r-project.org/R
[2] file:// repos causes libsvn to use more memory internally
Signed-off-by: Eric Wong normalper...@yhbt.net
Cc: Hin-Tak Leung ht...@users.sourceforge.net
---
This patch is in my less-memo branch of bogomips.org/git-svn.git
Info here:
(not intended as a pull request for Junio, yet)
The following changes since commit fbecd99861ea5795aeba46faf2ac7a8c1b70d485:
Update draft release notes to 2.2 (2014-10-24 15:02:17 -0700)
are available in the git repository at:
git://bogomips.org/git-svn.git less-memo
for you to fetch changes up to 7676aff70973e617c3f58a8633db6d0e3ee99e45:
git-svn: disable _rev_list memoization (2014-10-27 01:39:39 +)
Eric Wong (7):
git-svn: reduce check_cherry_pick cache overhead
git-svn: cache only mergeinfo revisions
git-svn: remove mergeinfo rev caching
git-svn: reload RA every log-window-size
git-svn: remove unnecessary DESTROY override
git-svn: save a little memory as fetch progresses
git-svn: disable _rev_list memoization
Jakob Stoklund Olesen (2):
git-svn: only look at the new parts of svn:mergeinfo
git-svn: only look at the root path for svn:mergeinfo
Sveinung Kvilhaugsvik (1):
git-svn.txt: advertise pushurl with dcommit
Documentation/git-svn.txt | 4 ++
perl/Git/SVN.pm | 121
+-
perl/Git/SVN/Ra.pm| 27 ++-
3 files changed, 94 insertions(+), 58 deletions(-)
perl/Git/SVN.pm | 10 --
1 file changed, 10 deletions(-)
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index 4364506..5f9d469 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1600,7 +1600,6 @@ sub tie_for_persistent_memoization {
my %lookup_svn_merge_cache;
my %check_cherry_pick2_cache;
my %has_no_changes_cache;
- my %_rev_list_cache;
tie_for_persistent_memoization(\%lookup_svn_merge_cache,
$cache_path/lookup_svn_merge);
@@ -1622,14 +1621,6 @@ sub tie_for_persistent_memoization {
SCALAR_CACHE = ['HASH' = \%has_no_changes_cache],
LIST_CACHE = 'FAULT',
;
-
- tie_for_persistent_memoization(\%_rev_list_cache,
- $cache_path/_rev_list);
- memoize '_rev_list',
- SCALAR_CACHE = 'FAULT',
- LIST_CACHE = ['HASH' = \%_rev_list_cache],
- ;
-
}
sub unmemoize_svn_mergeinfo_functions {
@@ -1639,7 +1630,6 @@ sub tie_for_persistent_memoization {
Memoize::unmemoize 'lookup_svn_merge';
Memoize::unmemoize 'check_cherry_pick2';
Memoize::unmemoize 'has_no_changes';
- Memoize::unmemoize '_rev_list';
}
sub clear_memoized_mergeinfo_caches {
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Anomaly with the new code - Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: On Sat, Oct 25, 2014 00:34 BST Eric Wong wrote: 0006 is insufficient and incompatible with older SVN. I pushed git-svn: reload RA every log-window-size (commit dfa72fdb96befbd790f623bb2909a347176753c2) instead which saves much more memory: it is fetching against the new clone taking twice as long and consuming twice as much memory. Which SVN version are you using? I'm cloning (currently on r373xx) https://svn.r-project.org/R using --stdlayout and unable to see memory growth of the git-svn Perl process beyond 40M (on a 32-bit system). I also tried http:// (not https), svn+ssh:// on my local (64-bit) system and did not see memory growth, either: http://mid.gmane.org/20141027014033.ga4...@dcvr.yhbt.net I'm using svn 1.6.17 on Debian stable in all cases. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Anomaly with the new code - Re: git-svn performance
Eric Wong normalper...@yhbt.net wrote: Which SVN version are you using? I'm cloning (currently on r373xx) https://svn.r-project.org/R using --stdlayout and unable to see memory growth of the git-svn Perl process beyond 40M (on a 32-bit system). git-svn hit 45M and took 11:44 to finish. My ping times to svn.r-project.org is around 150ms (I'm running this from a server in Fremont, California). I'll keep the repo around and periodically fetch to see how it runs. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: differences between old clone and new Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: To compare the old clone with the new, I did: git branch -r | sort | xargs -n 1 git log --decorate=full -n 1 It turned out other than the empty vs 3 word commit messages about two years ago on trunk (which are inherited in all the newer branches), there are two other groups of differences. One branch on the old clone has an extra merge from trunk ( and some extra trunk commits) listed in 'git log', while another branch has the exact opposite - on the old clone has one fewer merge. I see the merge seem to be genuine - the subversion log often says so e.g. ported from rXXX from trunk, but the extra/missing pattern isn't consistent. So both merges are correct, but we lose one, and gain one? I'll try to check more closely tomorrow. Can you point out the exact revisions in the R repo? Thanks. So the histories are largely the same, except due to the extra merge, don't have the same sha1 sums. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Anomaly with the new code - Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: Eric Wong normalper...@yhbt.net wrote: Which SVN version are you using? I'm cloning (currently on r373xx) https://svn.r-project.org/R using --stdlayout and unable to see memory growth of the git-svn Perl process beyond 40M (on a 32-bit system). git-svn hit 45M and took 11:44 to finish. My ping times to svn.r-project.org is around 150ms (I'm running this from a server in Fremont, California). I'll keep the repo around and periodically fetch to see how it runs. I'll apply the 10 patches against 2.1.0 and see then. As I wrote in my last reply, my 3rd clone took about 8 hours to finish, and the max resident size is about 700MB (according to GNU time). The time command is not a good measurement since it includes child process memory use (which may be file-backed mmap for git repack or git cat-file --batch). My measurements are just the RSS of the git-svn Perl process (from ps aux or VmRSS in /proc/$PID/status on Linux) -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Regression and failure to clone/fetch with new code Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote:
Hi, I patched my system git with the recent git-svn improvements, and just use
it for general use; so theses are the patches, against 2.1.0.
0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch
0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patch
0003-git-svn-reduce-check_cherry_pick-cache-overhead.patch
0004-git-svn-cache-only-mergeinfo-revisions.patch
0005-git-svn-remove-mergeinfo-rev-caching.patch
0006-git-svn.txt-advertise-pushurl-with-dcommit.patch
0007-git-svn-reload-RA-every-log-window-size.patch
0008-git-svn-remove-unnecessary-DESTROY-override.patch
0009-git-svn-save-a-little-memory-as-fetch-progresses.patch
0010-git-svn-disable-_rev_list-memoization.patch
trying to do this:
git svn clone http://www.virtualbox.org/svn/vbox/trunk vbox
(there is no publicly visible branches, so it is just a straight-forward
single-branch clone).
aborts with
---
M src/VBox/Main/HostImpl.cpp
Incorrect parameters given: Could not convert '%ld' into a number at
/usr/share/perl5/vendor_perl/Git/SVN.pm line 1711.
$ git svn fetch --all
Index mismatch: d6c75bc195b1daad647322e2cc025bd31265c6b9 !=
3927d05f6ab037fcf2b4d964c9633efade037d1b
rereading a65b5fc0077c2fa80a344833b65ac19ff4ae88b6
M src/VBox/Main/HostImpl.cpp
Incorrect parameters given: Could not convert '%ld' into a number at
/usr/share/perl5/vendor_perl/Git/SVN.pm line 1711.
I have never seen such behavior before, and seeing as the lines indicated are
in
a routine called mergeinfo_changes, and recently added/changed by
quite a few of the patches, I started reverting from the back in this order:
#5, #4, #2, #1
and tried again between each revert. And it finally allows me to fetch again
after
reverting #1.
Me neither, this is new bug to me. I cannot reproduce it, either. Which
revision did you hit this on? I completed your vbox trunk clone without
any problems on my side (Debian i386, SVN 1.6.17).
Can you try the following to dump out the parameters passed to
mergeinfo_changes?
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1695,8 +1695,10 @@ sub parents_exclude {
}
# Compute what's new in svn:mergeinfo.
+use Data::Dumper;
sub mergeinfo_changes {
my ($self, $old_path, $old_rev, $path, $rev, $mergeinfo_prop) = @_;
+ print STDERR Dumper(\@_);
my %minfo = map {split :, $_ } split \n, $mergeinfo_prop;
my $old_minfo = {};
Btw, I missed part of your other email, but no, I never maintained any
Chinese packages in Debian.
I don't see any %ld close by, but presumably this is enough information for
somebody else
to try. The platform is linux x86_64. (mostly fedora 20 but with a lot of
additional
changes like a newer gnome than shipped, etc so probably not really fc20)
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Git.pm: add specified name to tempfile template
This should help me track down errors in git-svn more easily:
write .git/Git_XX: Bad file descriptor
at /usr/lib/perl5/SVN/Ra.pm line 623
Signed-off-by: Eric Wong normalper...@yhbt.net
---
Not sure you want to take this separately or in a git-svn pull.
Still working on the error this patch is meant to help me find.
perl/Git.pm | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/perl/Git.pm b/perl/Git.pm
index 204fdc6..b5905ee 100644
--- a/perl/Git.pm
+++ b/perl/Git.pm
@@ -1294,8 +1294,11 @@ sub _temp_cache {
$tmpdir = $self-repo_path();
}
+ my $n = $name;
+ $n =~ s/\W/_/g; # no strange chars
+
($$temp_fd, $fname) = File::Temp::tempfile(
- 'Git_XX', UNLINK = 1, DIR = $tmpdir,
+ Git_${n}_XX, UNLINK = 1, DIR = $tmpdir,
) or throw Error::Simple(couldn't open new temp file);
$$temp_fd-autoflush;
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-svn: prepare SVN::Ra config pieces once
Memoizing these initialization functions saves some memory for
long fetches which require scanning many unwanted revisions
before any wanted revisions happen.
Signed-off-by: Eric Wong normalper...@yhbt.net
---
perl/Git/SVN/Ra.pm | 63 +++---
1 file changed, 36 insertions(+), 27 deletions(-)
diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm
index 5bc5b4e..75cdac9 100644
--- a/perl/Git/SVN/Ra.pm
+++ b/perl/Git/SVN/Ra.pm
@@ -2,6 +2,7 @@ package Git::SVN::Ra;
use vars qw/@ISA $config_dir $_ignore_refs_regex $_log_window_size/;
use strict;
use warnings;
+use Memoize;
use SVN::Client;
use Git::SVN::Utils qw(
canonicalize_url
@@ -76,6 +77,40 @@ sub _auth_providers () {
\@rv;
}
+sub prepare_config_once {
+ SVN::_Core::svn_config_ensure($config_dir, undef);
+ my ($baton, $callbacks) = SVN::Core::auth_open_helper(_auth_providers);
+ my $config = SVN::Core::config_get_config($config_dir);
+ my $dont_store_passwords = 1;
+ my $conf_t = $config-{'config'};
+
+ no warnings 'once';
+ # The usage of $SVN::_Core::SVN_CONFIG_* variables
+ # produces warnings that variables are used only once.
+ # I had not found the better way to shut them up, so
+ # the warnings of type 'once' are disabled in this block.
+ if (SVN::_Core::svn_config_get_bool($conf_t,
+ $SVN::_Core::SVN_CONFIG_SECTION_AUTH,
+ $SVN::_Core::SVN_CONFIG_OPTION_STORE_PASSWORDS,
+ 1) == 0) {
+ SVN::_Core::svn_auth_set_parameter($baton,
+ $SVN::_Core::SVN_AUTH_PARAM_DONT_STORE_PASSWORDS,
+ bless (\$dont_store_passwords, _p_void));
+ }
+ if (SVN::_Core::svn_config_get_bool($conf_t,
+ $SVN::_Core::SVN_CONFIG_SECTION_AUTH,
+ $SVN::_Core::SVN_CONFIG_OPTION_STORE_AUTH_CREDS,
+ 1) == 0) {
+ $Git::SVN::Prompt::_no_auth_cache = 1;
+ }
+
+ return ($config, $baton, $callbacks);
+} # no warnings 'once'
+
+INIT {
+ Memoize::memoize '_auth_providers';
+ Memoize::memoize 'prepare_config_once';
+}
sub new {
my ($class, $url) = @_;
@@ -84,34 +119,8 @@ sub new {
::_req_svn();
- SVN::_Core::svn_config_ensure($config_dir, undef);
- my ($baton, $callbacks) = SVN::Core::auth_open_helper(_auth_providers);
- my $config = SVN::Core::config_get_config($config_dir);
$RA = undef;
- my $dont_store_passwords = 1;
- my $conf_t = ${$config}{'config'};
- {
- no warnings 'once';
- # The usage of $SVN::_Core::SVN_CONFIG_* variables
- # produces warnings that variables are used only once.
- # I had not found the better way to shut them up, so
- # the warnings of type 'once' are disabled in this block.
- if (SVN::_Core::svn_config_get_bool($conf_t,
- $SVN::_Core::SVN_CONFIG_SECTION_AUTH,
- $SVN::_Core::SVN_CONFIG_OPTION_STORE_PASSWORDS,
- 1) == 0) {
- SVN::_Core::svn_auth_set_parameter($baton,
- $SVN::_Core::SVN_AUTH_PARAM_DONT_STORE_PASSWORDS,
- bless (\$dont_store_passwords, _p_void));
- }
- if (SVN::_Core::svn_config_get_bool($conf_t,
- $SVN::_Core::SVN_CONFIG_SECTION_AUTH,
- $SVN::_Core::SVN_CONFIG_OPTION_STORE_AUTH_CREDS,
- 1) == 0) {
- $Git::SVN::Prompt::_no_auth_cache = 1;
- }
- } # no warnings 'once'
-
+ my ($config, $baton, $callbacks) = prepare_config_once();
my $self = SVN::Ra-new(url = $url, auth = $baton,
config = $config,
pool = SVN::Pool-new,
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: differences between old clone and new Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: On Tue, 28/10/14, Eric Wong normalper...@yhbt.net wrote: So both merges are correct, but we lose one, and gain one? I'll try to check more closely tomorrow. Can you point out the exact revisions in the R repo? Thanks. The missing merge on branch R-2-14-branch is: commit 93af4d4cc3a5e0039944dd4e340d26995be8a252 Merge: 121990f 6ff1b87 Author: ripley ripley@00db46b3-68df-0310-9c12-caf00c1e9a41 Date: Wed Feb 22 13:45:34 2012 + port r58453 from trunk git-svn-id: https://svn.r-project.org/R/branches/R-2-14-branch@58454 00db46b3-68df-0310-9c12-caf00c1e9a41 I'm curious if you can tell me which version of git-svn you used to get that as a merge commit. git-svn mergeinfo handling has changed (hopefully improved) over the years, so some differences in history can be (unfortunately) expected, I think. I cannot reproduce your original merge on Junio's current master. Using Junio's master[1] without any recent git-svn changes, a partial clone doing: git svn clone -s -r52000:58600 svn+ssh://127.0.0.1/path/to/my/R-mirror ...causes the merges in r58454 to be ignored as cherry-picks, too. I suspect it's correct for git-svn to ignore those as cherry-picks nowadays. Here's a snippet of what I see from the above command: --8--- r58452 = ebf3a1ca312ca7cc03dc2387d86491a0cdc95bad (refs/remotes/origin/trunk) M src/library/base/man/Primitive.Rd M src/main/names.c M doc/NEWS.Rd r58453 = 05b55eee9e6bed628873d34261e54c70f87a3736 (refs/remotes/origin/trunk) M doc/NEWS.Rd M src/library/base/man/Primitive.Rd M src/main/names.c W:svn cherry-pick ignored (/branches/R-2-12-branch:52939,54476,55265) - missing 492 commit(s) (eg df9d875de507ac51932c0ed980392e8262f98b31) W:svn cherry-pick ignored (/branches/R-2-13-branch:55265,55432) - missing 231 commit(s) (eg cad052d416d9b8a9dfbfb2ae7bf85c39306c67bb) W:svn cherry-pick ignored (/trunk:57183,57204-57205,57242,57259,57314,57316,57321,57370,57411,57428,57430,57432,57438,57440,57484,57489-57490,57579,57589,57604,57614-57618,57625,57679,57681,57687,57738,57741,57744-57745,57747,57752,57758,57761,57763,57765,57767,57769,57771,57790,57793,57803,57812,57814,57816,57826-57827,57836,57840-57841,57844,57846,57851,57853,57856,57861-57862,57867,57880,57884,57890,57893,57895,57900,57904,57908,57913,57920,57936,57939-57941,57950,57952,57959,57964,57970,57975,57977,57981,57987,58006,58008,58037,58039,58042,58047,58052,58056,58058,58066-58067,58082,58084,58089,58094,58098,58100,58107,58126,58129,58135,58142,58161,58178,58182,58187,58195,58204,58213,58217,58221,58225,58228,58232,58234,58239,58248,58253,58265,58269,58272,58274,58276,58278,58282,58284,58288,58294,58296,58305,58312,58314,58318,58324,58326,58328,58332,58334,58340,58346,58348,58353,58355,58357,58359,58361,58373,58378,58381,58386,58388,58392,58395,58397,58405,58412,58415,58429,58435,58437,58439,58453) - missing 716 commit(s) (eg e9ccca5db27696ed8faa4427ec4110ddf230d141) r58454 = 96d6087a494bb7da6d90f02e8bd36833eaad2067 (refs/remotes/origin/R-2-14-branch) M doc/manual/R-exts.texi M doc/NEWS.Rd M src/library/tools/R/check.R r58455 = 742cbc791fa6760d5dfb4c4ea1e032d32e9e87c9 (refs/remotes/origin/trunk) --8--- [1] - fbecd99 Update draft release notes to 2.2 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Regression and failure to clone/fetch with new code Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: Argh, sorry. I thought I included the info but I didn't. Thanks. I'll try a different version of svn later. What do you think were missing in my e-mails? I was skimming and missed the part about Debian packages :) -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-svn: add space after W: prefix in warning
And minor reformatting while we're in the area.
Signed-off-by: Eric Wong normalper...@yhbt.net
---
perl/Git/SVN.pm | 6 ++
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index 893b9a8..d9a52a5 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1798,12 +1798,10 @@ sub find_extra_svn_parents {
);
if ($ninc) {
- warn W:svn cherry-pick ignored ($spec) - missing .
+ warn W: svn cherry-pick ignored ($spec) - missing .
$ninc commit(s) (eg $ifirst)\n;
} else {
- warn
- Found merge parent ($spec): ,
- $merge_tip, \n;
+ warn Found merge parent ($spec): , $merge_tip, \n;
push @new_parents, $merge_tip;
}
}
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-svn: (cleanup) remove editor param passing
Neither find_extra_svk_parents or find_extra_svn_parents ever
used the `$ed' parameter.
Signed-off-by: Eric Wong normalper...@yhbt.net
---
perl/Git/SVN.pm | 10 --
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index 5f9d469..893b9a8 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1433,7 +1433,7 @@ sub check_author {
}
sub find_extra_svk_parents {
- my ($self, $ed, $tickets, $parents) = @_;
+ my ($self, $tickets, $parents) = @_;
# aha! svk:merge property changed...
my @tickets = split \n, $tickets;
my @known_parents;
@@ -1736,7 +1736,7 @@ sub mergeinfo_changes {
# note: this function should only be called if the various dirprops
# have actually changed
sub find_extra_svn_parents {
- my ($self, $ed, $mergeinfo, $parents) = @_;
+ my ($self, $mergeinfo, $parents) = @_;
# aha! svk:merge property changed...
memoize_svn_mergeinfo_functions();
@@ -1835,16 +1835,14 @@ sub make_log_entry {
my @parents = @$parents;
my $props = $ed-{dir_prop}{$self-path};
if ( $props-{svk:merge} ) {
- $self-find_extra_svk_parents
- ($ed, $props-{svk:merge}, \@parents);
+ $self-find_extra_svk_parents($props-{svk:merge}, \@parents);
}
if ( $props-{svn:mergeinfo} ) {
my $mi_changes = $self-mergeinfo_changes
($parent_path, $parent_rev,
$self-path, $rev,
$props-{svn:mergeinfo});
- $self-find_extra_svn_parents
- ($ed, $mi_changes, \@parents);
+ $self-find_extra_svn_parents($mi_changes, \@parents);
}
open my $un, '', $self-{dir}/unhandled.log or croak $!;
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Regression and failure to clone/fetch with new code Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote:
Here is the data dumper info . I tried the dumper code on the R repo
as well, and saw that against the virtual box repo, there is one
curious difference - $self-{last_rev} is a string rather than a number.
I tried hacking around doing $x += 0; to coerce last_rev
to a number at various places but didn't get very far. There seems to be some
caching
code in RA-get_dir so presumably that's why the same code run
on one repo gives it as string while on another gives it a number. Hope
you can figure where the coersion to string happened.
Thanks, I'm not able to reproduce the issue, but can you try the
following?
diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm
index 75cdac9..82d6108 100644
--- a/perl/Git/SVN/Ra.pm
+++ b/perl/Git/SVN/Ra.pm
@@ -153,6 +153,7 @@ sub url {
sub check_path {
my ($self, $path, $r) = @_;
my $cache = $self-{cache}-{check_path};
+ $r = int($r);
if ($r == $cache-{r} exists $cache-{data}-{$path}) {
return $cache-{data}-{$path};
}
@@ -169,6 +170,7 @@ sub check_path {
sub get_dir {
my ($self, $dir, $r) = @_;
my $cache = $self-{cache}-{get_dir};
+ $r = int($r);
if ($r == $cache-{r}) {
if (my $x = $cache-{data}-{$dir}) {
return wantarray ? @$x : $x-[0];
---
The above should apply to my current master which has some
minor cleanups (which I hope to send to Junio tomorrow).
The following changes since commit fbecd99861ea5795aeba46faf2ac7a8c1b70d485:
Update draft release notes to 2.2 (2014-10-24 15:02:17 -0700)
are available in the git repository at:
git://bogomips.org/git-svn.git master
for you to fetch changes up to da0bc948ac2e01652a150fd4a57cebad6143242c:
git-svn: add space after W: prefix in warning (2014-10-30 08:31:28 +)
Eric Wong (11):
git-svn: reduce check_cherry_pick cache overhead
git-svn: cache only mergeinfo revisions
git-svn: remove mergeinfo rev caching
git-svn: reload RA every log-window-size
git-svn: remove unnecessary DESTROY override
git-svn: save a little memory as fetch progresses
git-svn: disable _rev_list memoization
Git.pm: add specified name to tempfile template
git-svn: prepare SVN::Ra config pieces once
git-svn: (cleanup) remove editor param passing
git-svn: add space after W: prefix in warning
Jakob Stoklund Olesen (2):
git-svn: only look at the new parts of svn:mergeinfo
git-svn: only look at the root path for svn:mergeinfo
Sveinung Kvilhaugsvik (1):
git-svn.txt: advertise pushurl with dcommit
Documentation/git-svn.txt | 4 ++
perl/Git.pm | 5 +-
perl/Git/SVN.pm | 125 --
perl/Git/SVN/Ra.pm| 90 ++---
4 files changed, 134 insertions(+), 90 deletions(-)
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC] Git.pm: stat the FD to ensure the file is really open
For a (currently) unknown reason, Git::SVN::Fetcher::close_file
sometimes triggers Bad file descriptor errors on syswrite when
writing symlink contents on the svn_hash tempfile.
The current IO::Handle::opened call in Git.pm is not a
sufficient check, as the underlying file descriptor is closed
without the PerlIO layer knowing about it. This is likely a bug
inside libsvn (1.6.17), as none of the Git.pm or Git::SVN
modules close IOs without the knowledge of the PerlIO layer.
Cc: Kyle J. McKay mack...@gmail.com
Signed-off-by: Eric Wong normalper...@yhbt.net
---
Kyle/Junio: thoughts? I'm running out of time to track this down
so it might be necessary for 2.2-rc0. What's even stranger is
I cannot always reproduce the problem even without this patch,
so it may be only triggered by network latency...
Thanks.
perl/Git.pm | 18 ++
1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/perl/Git.pm b/perl/Git.pm
index b5905ee..f1f210b 100644
--- a/perl/Git.pm
+++ b/perl/Git.pm
@@ -99,7 +99,7 @@ increase notwithstanding).
=cut
-use Carp qw(carp croak); # but croak is bad - throw instead
+use Carp qw(carp croak cluck); # but croak is bad - throw instead
use Error qw(:try);
use Cwd qw(abs_path cwd);
use IPC::Open2 qw(open2);
@@ -1206,6 +1206,16 @@ sub temp_acquire {
$temp_fd;
}
+sub _temp_really_open {
+ my ($fh) = @_;
+
+ if (defined($fh) $fh-opened) {
+ return 1 if stat($fh);
+ cluck($fh closed independently of PerlIO\n);
+ }
+ return 0;
+}
+
=item temp_is_locked ( NAME )
Returns true if the internal lock created by a previous Ctemp_acquire()
@@ -1232,7 +1242,7 @@ sub temp_is_locked {
my ($self, $name) = _maybe_self(@_);
my $temp_fd = \$TEMP_FILEMAP{$name};
- defined $$temp_fd $$temp_fd-opened
$TEMP_FILES{$$temp_fd}{locked};
+ _temp_really_open($$temp_fd) $TEMP_FILES{$$temp_fd}{locked};
}
=item temp_release ( NAME )
@@ -1264,7 +1274,7 @@ sub temp_release {
carp Attempt to release temp file ',
$temp_fd, ' that has not been locked;
}
- temp_reset($temp_fd) if $trunc and $temp_fd-opened;
+ temp_reset($temp_fd) if $trunc _temp_really_open($temp_fd);
$TEMP_FILES{$temp_fd}{locked} = 0;
undef;
@@ -1276,7 +1286,7 @@ sub _temp_cache {
_verify_require();
my $temp_fd = \$TEMP_FILEMAP{$name};
- if (defined $$temp_fd and $$temp_fd-opened) {
+ if (_temp_really_open($$temp_fd)) {
if ($TEMP_FILES{$$temp_fd}{locked}) {
throw Error::Simple(Temp file with moniker ' .
$name . ' already in use);
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: differences between old clone and new Re: git-svn performance
Hin-Tak Leung ht...@users.sourceforge.net wrote: That's quite straight-forward, I think - except for the recent burst (I am essentially adapting the git 2.1.0 release shipped by the upcoming fedora 21 scheduled for christmas) I tend to update to the latest fedora release about a week or two after release; fedora 17 was shipped in May 2012 and only just enter Alpha in 22 Feb 2012. and I tracked R at least as frequently as weekly around then; So I would be using what ever version of git was shipping with fedora 16 around late Feb 2012. On fedora's build farm, git-1.7.7.5 was bult in dec 2011 and git-1.7.7.6 was built on 2012-01-19 . Depending on how soon 1.7.7.6 filtered down to update, and when I update my git and also tracked R, (all three of these events probably happened around 22 Feb), I could be using either 1.7.7.5 or 1.7.7.6. I still have the system software update log around (the repo was cloned on a now-dead system, then moved over when it died), and presumably I can get git log to show me the fetch date (?), I might be able to tell whether it is 17.7.5 or 1.7.7.6 if you really want to know. I tried a full clone on 1.7.7.6 (no git-svn difference from 1.7.7.5). Even with that old git, I was able to reproduce the same merge behavior as current (Junio's) master as well as our recent patches. So I believe r58454, r46925, and r46906 in the R repo are all handled correctly and no mergeinfo-handling regressions are introduced in the latest round of git-svn changes. Thanks. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] Git.pm: stat the FD to ensure the file is really open
Kyle J. McKay mack...@gmail.com wrote:
On Oct 30, 2014, at 15:08, Eric Wong wrote:
For a (currently) unknown reason, Git::SVN::Fetcher::close_file
sometimes triggers Bad file descriptor errors on syswrite when
writing symlink contents on the svn_hash tempfile.
The current IO::Handle::opened call in Git.pm is not a
sufficient check, as the underlying file descriptor is closed
without the PerlIO layer knowing about it. This is likely a bug
inside libsvn (1.6.17), as none of the Git.pm or Git::SVN
modules close IOs without the knowledge of the PerlIO layer.
Cc: Kyle J. McKay mack...@gmail.com
Signed-off-by: Eric Wong normalper...@yhbt.net
---
Kyle/Junio: thoughts? I'm running out of time to track this down
so it might be necessary for 2.2-rc0. What's even stranger is
I cannot always reproduce the problem even without this patch,
so it may be only triggered by network latency...
Thanks.
With this patch added, do you then see the warning:
carp Temp file ', $name,
' was closed. Opening replacement.;
From _temp_cache? It would seem odd if you didn't unless there was
only one symlink involved.
Nope. $$temp_fd was defined so the message is not hit.
Is suppose we can also make the following change:
--- a/perl/Git.pm
+++ b/perl/Git.pm
@@ -1282,11 +1282,9 @@ sub _temp_cache {
$name . ' already in use);
}
} else {
- if (defined $$temp_fd) {
- # then we're here because of a closed handle.
- carp Temp file ', $name,
- ' was closed. Opening replacement.;
- }
+ # then we're here because of a closed handle.
+ carp Temp file ', $name,
+ ' was closed. Opening replacement.;
my $fname;
my $tmpdir;
I suspect that symlinks were rare in the repositories I was testing
against. I wonder if having a test svn repo that adds several new
symlinks for several revisions in a row might trigger this problem
more robustly.
The _repository-temp_acquire('svn_hash') call is assigned to a my
variable and then has Git::temp_release called on it later in the
same function and the only calls made on it in between are
Git::temp_path, so I don't see how the libsvn library could be
responsible for closing it since it looks to me like it never sees
it. But I'm looking at v2.1.2 of Fetcher.pm, so if some other calls
have been inserted there since then.
I stared at it for a long while and could not figure out what was going
on, either. The other temp_acquire users are far less straightforward
and did not get mysteriously closed.
The changes themselves look okay, but eee. I don't see how
libsvn can see the svn_hash fd to close it unless it's randomly
closing fds.
Right. Even worse, I can't even get it to trigger anymore...
Thanks for your response!
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PULL] git-svn updates for 2.2
Hi Junio, I haven't heard back from Hin-Tak about the last one (git-svn: coerce check_path and get_log args to int)[1], but I think it's a harmless defensive patch in case you want to tag 2.2-rc0 soon. I'm also leaving Git.pm: stat the FD to ensure the file is really open out for now... [1] http://mid.gmane.org/20141030084619.ga12...@dcvr.yhbt.net --- The following changes since commit fbecd99861ea5795aeba46faf2ac7a8c1b70d485: Update draft release notes to 2.2 (2014-10-24 15:02:17 -0700) are available in the git repository at: git://bogomips.org/git-svn.git master for you to fetch changes up to 9d536cf625cf42b8a8272070e3720325c68f3b48: git-svn: coerce check_path and get_log args to int (2014-10-30 23:10:57 +) Eric Wong (12): git-svn: reduce check_cherry_pick cache overhead git-svn: cache only mergeinfo revisions git-svn: remove mergeinfo rev caching git-svn: reload RA every log-window-size git-svn: remove unnecessary DESTROY override git-svn: save a little memory as fetch progresses git-svn: disable _rev_list memoization Git.pm: add specified name to tempfile template git-svn: prepare SVN::Ra config pieces once git-svn: (cleanup) remove editor param passing git-svn: add space after W: prefix in warning git-svn: coerce check_path and get_log args to int Jakob Stoklund Olesen (2): git-svn: only look at the new parts of svn:mergeinfo git-svn: only look at the root path for svn:mergeinfo Sveinung Kvilhaugsvik (1): git-svn.txt: advertise pushurl with dcommit Documentation/git-svn.txt | 4 ++ perl/Git.pm | 5 +- perl/Git/SVN.pm | 125 -- perl/Git/SVN/Ra.pm| 92 +++--- 4 files changed, 136 insertions(+), 90 deletions(-) -- EW -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-svn: use SVN::Ra::get_dir2 when possible
This avoids the following failure with normal get_dir on newer
versions of SVN (tested with SVN 1.8.8-1ubuntu3.1):
Incorrect parameters given: Could not convert '%ld' into a number
get_dir2 also has the potential to be more efficient by requesting
less data.
ref: 1414636504.45506.yahoomailba...@web172304.mail.ir2.yahoo.com
ref: 1414722617.89476.yahoomailba...@web172305.mail.ir2.yahoo.com
Signed-off-by: Eric Wong normalper...@yhbt.net
Cc: Hin-Tak Leung ht...@users.sourceforge.net
---
This should fix the vbox clone problem. SVN Perl binding
breakage (again :). I shall revert the int() changes.
I added those two lines to my git and there is no improvement. It
still won't svn fetch the next revision. I think it may be
important/interesting to find out when or how it becomes non-int, so
I have tar'gz'ed my wont-fetch virtual box .git and in the middle of
uploading here:
http://sourceforge.net/projects/outmodedbonsai/files/R/
I am also uploading my old R clone also - maybe you'd like to see
why its .git/svn/.caches is so big compared to a recent one,
Jakob's changes causes different access patterns, so it's expected the
sizes vary. I also changed the cherry pick cache and removed the
_rev_list caching entirely, so it should be much smaller now.
as well as how and why there were an extra merge and two missing
merges compared to a recent clone?
The different merges are fine, I think, as stated in
http://mid.gmane.org/20141030230831.ga14...@dcvr.yhbt.net
perl/Git/SVN/Ra.pm | 12 +++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm
index 82d6108..1e52709 100644
--- a/perl/Git/SVN/Ra.pm
+++ b/perl/Git/SVN/Ra.pm
@@ -177,7 +177,17 @@ sub get_dir {
}
}
my $pool = SVN::Pool-new;
- my ($d, undef, $props) = $self-SUPER::get_dir($dir, $r, $pool);
+ my ($d, undef, $props);
+
+ if (defined SVN::Ra::get_dir2) { # appeared in SVN 1.4
+ # n.b. in addition to being potentially more efficient,
+ # this works around what appears to be a bug in some
+ # SVN 1.8 versions
+ my $kind = 1; # SVN_DIRENT_KIND
+ ($d, undef, $props) = $self-get_dir2($dir, $r, $kind, $pool);
+ } else {
+ ($d, undef, $props) = $self-get_dir($dir, $r, $pool);
+ }
my %dirents = map { $_ = { kind = $d-{$_}-kind } } keys %$d;
$pool-clear;
if ($r != $cache-{r}) {
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2] git-svn: use SVN::Ra::get_dir2 when possible
This avoids the following failure with normal get_dir on newer
versions of SVN (tested with SVN 1.8.8-1ubuntu3.1):
Incorrect parameters given: Could not convert '%ld' into a number
get_dir2 also has the potential to be more efficient by requesting
less data.
ref: 1414636504.45506.yahoomailba...@web172304.mail.ir2.yahoo.com
ref: 1414722617.89476.yahoomailba...@web172305.mail.ir2.yahoo.com
Signed-off-by: Eric Wong normalper...@yhbt.net
Cc: Hin-Tak Leung ht...@users.sourceforge.net
---
Sorry, waaay past my bed time. This version doesn't infinite loop
on autoload or older SVN(*) (at least I hope it doesn't)
perl/Git/SVN/Ra.pm | 12 +++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm
index 82d6108..496f430 100644
--- a/perl/Git/SVN/Ra.pm
+++ b/perl/Git/SVN/Ra.pm
@@ -177,7 +177,17 @@ sub get_dir {
}
}
my $pool = SVN::Pool-new;
- my ($d, undef, $props) = $self-SUPER::get_dir($dir, $r, $pool);
+ my ($d, undef, $props);
+
+ if (::compare_svn_version('1.4.0') = 0) {
+ # n.b. in addition to being potentially more efficient,
+ # this works around what appears to be a bug in some
+ # SVN 1.8 versions
+ my $kind = 1; # SVN_DIRENT_KIND
+ ($d, undef, $props) = $self-get_dir2($dir, $r, $kind, $pool);
+ } else {
+ ($d, undef, $props) = $self-SUPER::get_dir($dir, $r, $pool);
+ }
my %dirents = map { $_ = { kind = $d-{$_}-kind } } keys %$d;
$pool-clear;
if ($r != $cache-{r}) {
--
EW
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PULL v2] git-svn updates for 2.2
Eric Wong normalper...@yhbt.net wrote: Hi Junio, I haven't heard back from Hin-Tak about the last one (git-svn: coerce check_path and get_log args to int)[1], but I think it's a harmless defensive patch in case you want to tag 2.2-rc0 soon. OK, that coercion patch was pointless and I worked around the problem differently with git-svn: use SVN::Ra::get_dir2 when possible So here's another try which I feel better about (except it's way past my bedtime right now...): The following changes since commit fbecd99861ea5795aeba46faf2ac7a8c1b70d485: Update draft release notes to 2.2 (2014-10-24 15:02:17 -0700) are available in the git repository at: git://bogomips.org/git-svn.git for-junio for you to fetch changes up to 7ffa35b0479dac547659f06b8a6ea7d31c57cc05: git-svn: use SVN::Ra::get_dir2 when possible (2014-10-31 10:34:54 +) Eric Wong (12): git-svn: reduce check_cherry_pick cache overhead git-svn: cache only mergeinfo revisions git-svn: remove mergeinfo rev caching git-svn: reload RA every log-window-size git-svn: remove unnecessary DESTROY override git-svn: save a little memory as fetch progresses git-svn: disable _rev_list memoization Git.pm: add specified name to tempfile template git-svn: prepare SVN::Ra config pieces once git-svn: (cleanup) remove editor param passing git-svn: add space after W: prefix in warning git-svn: use SVN::Ra::get_dir2 when possible Jakob Stoklund Olesen (2): git-svn: only look at the new parts of svn:mergeinfo git-svn: only look at the root path for svn:mergeinfo Sveinung Kvilhaugsvik (1): git-svn.txt: advertise pushurl with dcommit Documentation/git-svn.txt | 4 ++ perl/Git.pm | 5 +- perl/Git/SVN.pm | 125 -- perl/Git/SVN/Ra.pm| 102 ++--- 4 files changed, 145 insertions(+), 91 deletions(-) -- EW -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] git-svn: use SVN::Ra::get_dir2 when possible
Eric Wong normalper...@yhbt.net wrote: This avoids the following failure with normal get_dir on newer versions of SVN (tested with SVN 1.8.8-1ubuntu3.1): Incorrect parameters given: Could not convert '%ld' into a number Filed a bug in Debian since I hit it in sid, too: http://bugs.debian.org/767530 Thanks all. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] git-svn: use SVN::Ra::get_dir2 when possible
Hin-Tak Leung ht...@users.sourceforge.net wrote: On Fri, Oct 31, 2014 19:08 GMT Eric Wong wrote: Filed a bug in Debian since I hit it in sid, too: http://bugs.debian.org/767530 Thanks all. Hmm, but why are you filing at debian? I had the error when i applied the dev code patches on top of 2.1.0, and the error disappeared as soon as I reverted the patches. So it looks like a issue for the upstream developers, rather than distro. Are you saying debian is shipping a dev snapshot of git svn? I don't track SVN upstream development, so haven't checked if it affects newer SVN, yet. It is the distro maintainer's job to get this resolved. This bug affects existing cases of git svn show-ignore, we just happened to find it with git svn fetch, first. Debian's bug tracker also has the lowest barrier to entry: no login/registration is necessary (just like this git list). If you want to handle this with upstream directly, please do. I try to avoid logins/accounts as much as possible. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] git-svn: use SVN::Ra::get_dir2 when possible
Hin-Tak Leung ht...@users.sourceforge.net wrote: Tested-by: Hin-Tak Leung ht...@users.sourceforge.net Okay, this one on top of my git 2.1.0 + 10 recent git svn improvement patches allow me to fetch further. I suspect the problem must be elsewhere though, and this just band-aided over it. For me, reverting the additional patches and just use git 2.1.0 also allow me to fetch further, so the problem appear to me to be associated with the additional patches. Junio already pulled all of those git-svn patches into git 2.2.0-rc0: http://mid.gmane.org/xmqq7fzgt0q4@gitster.dls.corp.google.com Can you give that a try? I've been able to complete the vbox fetch without problems. But you are saying that git-svn on debian as shipped exhibit this problem? No, I'm saying unrelated, (old) functionality such as git svn show-ignore also got broken by this SVN bug. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] git-svn: use SVN::Ra::get_dir2 when possible
Hin-Tak Leung ht...@users.sourceforge.net wrote: While my 2.10 + 11 patches continue to fetch, where it was stuck, now it does Couldn't find revmap... - also, the single branch clone is doing the 'trunk/branches/... thing - are these supposed to happen? I'm afraid this is a problem with the vbox repo not publishing anything but trunk, but their svn:mergeinfo refers to non-public branches. This shows only changes to trunk: svn log -v http://www.virtualbox.org/svn/vbox I do not know what their development process is like. From the looks of it, every change is made by a vboxsync user. I suspect that's driven by an automated export script, and the real development happens in a different (private) repo... -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn: use SVN::Ra::get_dir2 when possible
Hin-Tak Leung ht...@users.sourceforge.net wrote: Hmm, I see you are filing the problem against subversion. FWIW, I am currently using subversion-perl-1.8.10-1.fc20.x86_64 package on fedora 20. I'll possibly think about filing one under redhat's bugzilla and let them take it upward too. This is another problem with the vbox repository: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=767530#10 And forwarded upstream: http://mid.gmane.org/20141101182722.gb20...@freya.jamessan.com -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 2/2] git-svn: allow git-svn fetching to work using serf
Jonathan Nieder jrnie...@gmail.com wrote: (cc-ing Eric Wong, who maintains git-svn and knows both it and the libsvn perl bindings much better than I do) I doubt that's the case anymore. I've hardly looked at SVN in many years, now. Anyways, if the patches: 1) do not introduce regressions 2) fixes real problems I'm inclined to think they're OK... I'm sorry I can't help more, I had access to better drugs back then. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn: Configure a prompt callback for gnome_keyring.
Matthijs Kooijman matth...@stdin.nl wrote: Hi folks, any chance this patch can be merged? It's probably fine. Does anybody else have testing/feedback? I haven't used git-svn/SVN in years, and I don't use GNOME (nor much GUI). -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn: Configure a prompt callback for gnome_keyring.
Edward Rudd ur...@outoforder.cc wrote: Where is a link to the latest patch? I can give it a quick once-over with one of my git-svn'ed game ports I'm working on. (due for launching on the 10th.. WOOHOO!) http://mid.gmane.org/1371573490-21973-1-git-send-email-matth...@stdin.nl Thanks! -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn: Fix termination issues for remote svn connections
Junio C Hamano gits...@pobox.com wrote:
Uli Heller uli.hel...@daemons-point.com writes:
Nevertheless, I think it makes sense to fix the issue within the
git perl module Ra.pm, too. The change frees the private copy of
the remote access object on termination which prevents the error
from happening.
Thanks. Please sign-off your patch.
I am Cc'ing Kyle McKay who apparently had some experience working
with git-svn with newer svn that can only use serf, hoping that we
can get an independent opinion/test just to be sure. Also Cc'ed is
Eric Wong who has been the official git-svn area expert, but I
understand that Eric hasn't needed to use git-svn for quite a while,
so it is perfectly fine if he does not have any comment on this one.
We may want to find a volunteer to move git svn forward as a new
area expert (aka subsystem maintainer), by the way.
Correct, git-svn has the effect of being self-obsoleting.
I agree with adding a workaround for broken things, however
I suggest a code comment explaining why it is necessary.
The commit message is important, too, but might get harder to track
down if there's code movement/refactoring in the future.
+END {
+ $RA = undef;
+ $ra_invalid = 1;
+}
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git svn: reset invalidates the memoized mergeinfo caches
Peter Baumann waste.mana...@gmx.de wrote:
On Tue, Aug 07, 2012 at 08:45:10PM +, Eric Wong wrote:
Peter Baumann waste.mana...@gmx.de wrote:
+ for my $suffix (qw(yaml db)) {
+ unlink($cache_file.$suffix);
Need to check for unlink() errors (and ignore ENOENT).
I'm not sure what you mean here: Aren't we screwed either way if unlinking
the file failed? There is nothhing we can do about it if e.g. the user doesn't
have the permissions to delete the file, besides terminating, e.g.
for my $cache_file (($cache_path/lookup_svn_merge,
$cache_path/check_cherry_pick,
$cache_path/has_no_changes)) {
for my $suffix (qw(yaml db)) {
next unless (-e $cache_file.$suffix);
unlink($cache_file.$suffix) or
die Failed to delete $cache_file.$suffix;
}
Yes we're screwed, but silent failure is the worst way to fail,
especially if it can lead us back to the problems your patch is meant to
address.
Perhaps something like this (with $! to show the error):
my $file = $cache_file.$suffix;
next unless -e $file;
unlink($file) or die unlink($file) failed: $!\n;
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC] git svn: handle errors and concurrent commits in dcommit
Robert Luberda rob...@debian.org wrote: Eric Wong wrote: + echo PATH=\$PATH\; export PATH $hook + echo svnconf=\$svnconf\ $hook + cat $hook - 'EOF2' + cd work-auto-commits.svn + svn up --config-dir $svnconf That doesn't seem to interact well with users who depend on svn_cmd pointing to something non-standard. Not sure what to do about it, though I have no idea how to change it either. I've tried to source the lib-git-svn.sh file inside the hook, but it sources test-lib.sh, and the latter script doesn't work well if it is sourced by non-test script. Anyway I the part of my original patch unchanged. Ah, so svn_cmd only cares about --config-dir and you already handled that :) I misremembered it also allowed for non-standard SVN installations :x I've pushed your updated patch to my maint branch on git://bogomips.org/git-svn since master has larger pending changes. Thanks! -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC] git svn: handle errors and concurrent commits in dcommit
Junio C Hamano gits...@pobox.com wrote: I should have asked this yesterday, but do you mean you want to have your maint in the upcoming 1.7.12? This does look like a useful thing to do, but does not seem like a regression fix to me. Yeah, I wasn't sure what to name it since my master is still carrying Michael's larger SVN 1.7 changes. Perhaps I should've named my maint for-git-master in this case... -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] git svn: reset invalidates the memoized mergeinfo caches
Peter Baumann waste.mana...@gmx.de wrote: First, let me thank you for your review and your detailed explanation. I really appreciate it. You're welcome, Peter. Thanks again for the patch. I've signed-off and pushed for Junio. The following changes since commit 034161a94e827ef05790b1c7ce5a6e3e740c864e: Merge git://github.com/git-l10n/git-po (2012-08-09 10:51:46 -0700) are available in the git repository at: git://bogomips.org/git-svn for-git-master for you to fetch changes up to 61b472ed8b090a3e9240590c85041120a54dd268: git svn: reset invalidates the memoized mergeinfo caches (2012-08-10 19:53:18 +) Peter Baumann (1): git svn: reset invalidates the memoized mergeinfo caches Robert Luberda (1): git svn: handle errors and concurrent commits in dcommit git-svn.perl | 74 --- perl/Git/SVN.pm| 27 - t/t9163-git-svn-reset-clears-caches.sh | 78 t/t9164-git-svn-dcommit-concrrent.sh | 216 + 4 files changed, 374 insertions(+), 21 deletions(-) create mode 100755 t/t9163-git-svn-reset-clears-caches.sh create mode 100755 t/t9164-git-svn-dcommit-concrrent.sh -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] git svn: reset invalidates the memoized mergeinfo caches
Junio C Hamano gits...@pobox.com wrote: Eric Wong normalper...@yhbt.net writes: Peter Baumann (1): git svn: reset invalidates the memoized mergeinfo caches Robert Luberda (1): git svn: handle errors and concurrent commits in dcommit OK, so these two are fit for 1.7.12-rc3 and later? Yes. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/2] git-svn.perl: consider all ranges for a given merge, instead of only tip-by-tip
Steven Walter stevenrwal...@gmail.com wrote:
Consider the case where you have trunk, branchA of trunk, and branchB of
branchA. trunk is merged back into branchB, and then branchB is
reintegrated into trunk. The merge of branchB into trunk will have
svn:mergeinfo property references to both branchA and branchB. When
performing the check_cherry_pick check on branchB, it is necessary to
eliminate the merged contents of branchA as well as branchB, or else the
merge will be incorrectly ignored as a cherry-pick.
Signed-off-by: Steven Walter stevenrwal...@gmail.com
---
I think this series is good, but would feel more comfortable if
I got a second opinion from Sam.
This doesn't apply against Junio's master (nor mine on
git://bogomips.org/git-svn.git), though it works fine on Junio's maint.
git-svn.perl|8 ++-
t/t9163-git-svn-fetch-merge-branch-of-branch.sh | 60
+++
2 files changed, 63 insertions(+), 5 deletions(-)
create mode 100755 t/t9163-git-svn-fetch-merge-branch-of-branch.sh
diff --git a/git-svn.perl b/git-svn.perl
index ca038ec..abcec11 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -3657,14 +3657,14 @@ sub find_extra_svn_parents {
my @merge_tips;
my $url = $self-{url};
my $uuid = $self-ra_uuid;
- my %ranges;
+ my @all_ranges;
for my $merge ( @merges ) {
my ($tip_commit, @ranges) =
lookup_svn_merge( $uuid, $url, $merge );
unless (!$tip_commit or
grep { $_ eq $tip_commit } @$parents ) {
push @merge_tips, $tip_commit;
- $ranges{$tip_commit} = \@ranges;
+ push @all_ranges, @ranges;
} else {
push @merge_tips, undef;
}
@@ -3679,8 +3679,6 @@ sub find_extra_svn_parents {
my $spec = shift @merges;
next unless $merge_tip and $excluded{$merge_tip};
- my $ranges = $ranges{$merge_tip};
-
# check out 'new' tips
my $merge_base;
eval {
@@ -3702,7 +3700,7 @@ sub find_extra_svn_parents {
my (@incomplete) = check_cherry_pick(
$merge_base, $merge_tip,
$parents,
- @$ranges,
+ @all_ranges,
);
if ( @incomplete ) {
diff --git a/t/t9163-git-svn-fetch-merge-branch-of-branch.sh
b/t/t9163-git-svn-fetch-merge-branch-of-branch.sh
new file mode 100755
index 000..13ae7e3
--- /dev/null
+++ b/t/t9163-git-svn-fetch-merge-branch-of-branch.sh
@@ -0,0 +1,60 @@
+#!/bin/sh
+#
+# Copyright (c) 2012 Steven Walter
+#
+
+test_description='git svn merge detection'
+. ./lib-git-svn.sh
+
+svn_ver=$(svn --version --quiet)
+case $svn_ver in
+0.* | 1.[0-4].*)
+ skip_all=skipping git-svn test - SVN too old ($svn_ver)
+ test_done
+ ;;
+esac
+
+test_expect_success 'initialize source svn repo' '
+ svn_cmd mkdir -m x $svnrepo/trunk
+ svn_cmd mkdir -m x $svnrepo/branches
+ svn_cmd co $svnrepo/trunk $SVN_TREE
+ (
+ cd $SVN_TREE
+ touch foo
+ svn_cmd add foo
+ svn_cmd commit -m initial commit
+ svn_cmd cp -m branch $svnrepo/trunk
$svnrepo/branches/branch1
+ svn_cmd switch $svnrepo/branches/branch1
+ touch bar
+ svn_cmd add bar
+ svn_cmd commit -m branch1
+ svn_cmd cp -m branch $svnrepo/branches/branch1
$svnrepo/branches/branch2
+ svn_cmd switch $svnrepo/branches/branch2
+ touch baz
+ svn_cmd add baz
+ svn_cmd commit -m branch2
+ svn_cmd switch $svnrepo/trunk
+ touch bar2
+ svn_cmd add bar2
+ svn_cmd commit -m trunk
+ svn_cmd switch $svnrepo/branches/branch2
+ svn_cmd merge $svnrepo/trunk
+ svn_cmd commit -m merge trunk
+ svn_cmd switch $svnrepo/trunk
+ svn_cmd merge --reintegrate $svnrepo/branches/branch2
+ svn_cmd commit -m merge branch2
+ )
+ rm -rf $SVN_TREE
+'
+
+test_expect_success 'clone svn repo' '
+ git svn init -s $svnrepo
+ git svn fetch
+'
+
+test_expect_success 'verify merge commit' 'x=$(git rev-parse HEAD^2)
+ y=$(git rev-parse branch2)
+ test x$x = x$y
+'
+
+test_done
--
1.7.9.5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 2/2] git-svn.perl: keep processing all commits in parents_exclude
Steven Walter stevenrwal...@gmail.com wrote: On Sat, Aug 18, 2012 at 3:51 PM, Sam Vilain s...@vilain.net wrote: On 08/11/2012 10:14 AM, Steven Walter wrote: --- git-svn.perl |1 - t/t9164-git-svn-fetch-merge-branch-of-branch2.sh | 53 ++ Thanks all. Steven: This doesn't apply cleanly to master since git-svn.perl is split into separate files (git-svn.perl - perl/Git/SVN*) Can you rebase against Junio's current master? Thanks again. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Fix git-svn for SVN 1.7
Junio C Hamano gits...@pobox.com wrote: Unless you found a reason not to in the meantime, that is. Is what I have on 'pu' still good, or do you (Eric and/or Michael) have any updates you'd rather have me pull instead? No updates, everything is still good. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC] git svn: handle errors and concurrent commits in dcommit
Robert Luberda rob...@debian.org wrote:
While working on my next patch, I've accidentally discovered that bash gives
the following errors in the test file introduced in my commit :
./t9164-git-svn-dcommit-concrrent.sh: line 65: $hook: ambiguous redirect
./t9164-git-svn-dcommit-concrrent.sh: line 66: $hook: ambiguous redirect
(The test succeeds, even though assignments of the PATH and svnconf
variables is missing; is seems svn treats an empty value of --config-dir
as the current dir.)
Sorry about not noticing this before, dash is used as default /bin/sh on
my system. A simple patch for this issue is included below.
There is also a typo in the test file name: `concrrent' instead of
`concurrent', but it most probably doesn't make any sense to make a
patch for it.
Oops, I'll push the following out since Junio already merged your
original:
(your original patch was also whitespace-mangled)
From 209b4ac46d70aa6f29d51c2fbefa53509513362c Mon Sep 17 00:00:00 2001
From: Robert Luberda rob...@debian.org
Date: Mon, 20 Aug 2012 00:43:19 +0200
Subject: [PATCH] t9164: Add missing quotes in test
This fixes `ambiguous redirect' error given by bash.
[ew: fix misspelled test name,
also eliminate space after to conform to guidelines]
Signed-off-by: Eric Wong normalper...@yhbt.net
---
...n-dcommit-concrrent.sh = t9164-git-svn-dcommit-concurrent.sh} | 8
1 file changed, 4 insertions(+), 4 deletions(-)
rename t/{t9164-git-svn-dcommit-concrrent.sh =
t9164-git-svn-dcommit-concurrent.sh} (97%)
diff --git a/t/t9164-git-svn-dcommit-concrrent.sh
b/t/t9164-git-svn-dcommit-concurrent.sh
similarity index 97%
rename from t/t9164-git-svn-dcommit-concrrent.sh
rename to t/t9164-git-svn-dcommit-concurrent.sh
index aac2dda..d8464d4 100755
--- a/t/t9164-git-svn-dcommit-concrrent.sh
+++ b/t/t9164-git-svn-dcommit-concurrent.sh
@@ -60,11 +60,11 @@ setup_hook()
[ $cnt = 0 ] || exit 0
EOF1
if [ $hook_type = pre-commit ]; then
- echo echo 'commit disallowed' 2; exit 1 $hook
+ echo echo 'commit disallowed' 2; exit 1 $hook
else
- echo PATH=\$PATH\; export PATH $hook
- echo svnconf=\$svnconf\ $hook
- cat $hook - 'EOF2'
+ echo PATH=\$PATH\; export PATH $hook
+ echo svnconf=\$svnconf\ $hook
+ cat $hook - 'EOF2'
cd work-auto-commits.svn
svn up --config-dir $svnconf
echo $$ auto_updated_file
--
Eric Wong
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/2] git-svn.perl: consider all ranges for a given merge, instead of only tip-by-tip
Steven Walter stevenrwal...@gmail.com wrote: Consider the case where you have trunk, branch1 of trunk, and branch2 of branch1. trunk is merged back into branch2, and then branch2 is reintegrated into trunk. The merge of branch2 into trunk will have svn:mergeinfo property references to both branch1 and branch2. When git-svn fetches the commit that merges branch2 (check_cherry_pick), it is necessary to eliminate the merged contents of branch1 as well as branch2, or else the merge will be incorrectly ignored as a cherry-pick. Signed-off-by: Steven Walter stevenrwal...@gmail.com Thanks, I've Signed-off (also added Sam's Reviewed-by) and pushed to for-git-master on git://bogomips.org/git-svn (Will wait on a few other patches before asking Junio to pull) -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC] git svn: handle errors and concurrent commits in dcommit
Robert Luberda rob...@debian.org wrote: Eric Wong wrote: Oops, I'll push the following out since Junio already merged your original: I can see that you haven't pushed the change yet. Maybe it would be a good idea to fix other style mistakes (extra spaces after redirections, lack of spaces after function names, `[' used instead of `test', etc) as well? I can prepare a patch if you think it is a good idea. Oops, I got distracted. Yes please on an updated patch. Thanks. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC] git svn: don't introduce new paragraph for git-svn-id
Robert Luberda rob...@debian.org wrote: Junio C Hamano wrote: Eric Wong normalper...@yhbt.net writes: I think having svn in svn.trimsvnlog twice is redundant and not ideal. Perhaps just --trim-log and svn.trimlog? Do we ever want to trim our log when relaying the Git commits back to subversion? Having svn in trimsvnlog makes it clear that the logs from subversion side is getting trimmed. `git commit' already trims the messages (except for removing the leading whitespaces from the first non-whitespace-only line) and git svn doesn't change that. The new option affects the way the messages are imported from svn to git, with one exception when the --add-author-from option of dcommit is used (in which case it may skip adding an extra new line character before the `From: ' line). For that reason --trim-svn-log might be a better name. OK. I now agree --trim-svn-log is a better name : -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Allow fancy globs in git-svn init branches
Ammon Riley ammon.ri...@gmail.com wrote:
Branches passed to 'git-svn init' via the -b/--branches flag
automatically had a /* appended to them. When the branch contained
a fancy glob with a {} pattern, this is incorrect behaviour, and
leads to odd branches being created in the git repository.
Signed-off-by: Ammon Riley ammon.ri...@gmail.com
---
git-svn.perl | 2 +-
t/t9141-git-svn-multiple-branches.sh | 12
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/git-svn.perl b/git-svn.perl
index 0d77ffb..f8e8558 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -1678,7 +1678,7 @@ sub complete_url_ls_init {
my $remote_path = join_paths( $gs-path, $repo_path );
$remote_path =~ s{%([0-9A-F]{2})}{chr hex($1)}ieg;
$remote_path =~ s#^/##g;
- $remote_path .= /* if $remote_path !~ /\*/;
+ $remote_path .= /* if $remote_path !~ m#\*|\{[^/]+\}#;
my ($n) = ($switch =~ /^--(\w+)/);
if (length $pfx $pfx !~ m#/$#) {
die --prefix='$pfx' must have a trailing slash '/'\n;
diff --git a/t/t9141-git-svn-multiple-branches.sh
b/t/t9141-git-svn-multiple-branches.sh
index 3cd0671..1b872a9 100755
--- a/t/t9141-git-svn-multiple-branches.sh
+++ b/t/t9141-git-svn-multiple-branches.sh
@@ -119,4 +119,16 @@ test_expect_success 'create new branches and tags' '
svn_cmd up test -e tags_B/Tag2/a.file )
'
+test_expect_success 'clone multiple branch paths using fancy glob' '
+ git svn clone -T trunk \
+ -b b_one/{first} --branches b_two \
I'm concerned encouraging this can cause confusion on the command-line
for bash users.
In bash, b_one/{first} will be passed as-is (and hardly anybody
will have a repo with '{word}' in the path)
However, unless quoted on the command-line, a likely case of:
b_one/{first,second} will expand to: b_one/first b_one/second
...which causes b_one/second to be interpreted as the destination
directory. A knowledgeable bash user can avoid this by using:
-b=b_one/{first,second} to avoid this situation.
But with the above invocation, no explicit support is needed
for command-line parsing in git-svn.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Allow fancy globs in git-svn init branches
Ammon Riley ammon.ri...@gmail.com wrote:
I confess that I'd completely forgot about the {} expansion in bash.
Perhaps a note in the CAVEATS section of the documentation would
be sufficient?
I think so, yes. Can you send a patch for that instead? Thanks.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/7] Extract, test and enhance the logic to collapse ../foo paths.
Jonathan Nieder jrnie...@gmail.com wrote:
Hi,
Michael G Schwern wrote:
On 2012.7.30 12:51 PM, Eric Wong wrote:
Michael G Schwern wrote:
_collapse_dotdot() works better than the existing regex did.
I don't dispute it's better, but it's worth explaining in the commit
message to reviewers why something is better.
Yeah. I figured the tests covered that.
Now I'm tripping up on the same thing. Eric, did you ever find out
what the motivation for this patch was? Is SVN 1.7 more persnickety
about runs of multiple slashes in a row or something, or is it more
of an aesthetic thing?
I'm not sure about this case specifically, but SVN has (and will likely
become) more persnickety over time. I haven't had a chance to check SVN
itself, but I think being defensive and giving it prettier paths will
be safer in the future.
That said, I'd favor an implementation that split on m{/+} and
collapsed as Michael mentioned.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/7] Extract, test and enhance the logic to collapse ../foo paths.
Jonathan Nieder jrnie...@gmail.com wrote:
Eric Wong wrote:
That said, I'd favor an implementation that split on m{/+} and
collapsed as Michael mentioned.
Sounds sensible. Is canonicalize_path responsible for collapsing
runs of slashes? What should _collapse_dotdot do to
c:/.. or http://www.example.com/..;?
It should probably just return the root path (c:/ and
http://www.example.com/; respectively).
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/7] Extract, test and enhance the logic to collapse ../foo paths.
Jonathan Nieder jrnie...@gmail.com wrote: Maybe we can use apr_filepath_merge() to avoid reinventing the wheel? Ideally, yes. Is there an easy way to access that from Perl? (and for the older versions of SVN folks people are running). Perhaps we can expose equivalent functionality in git via git-rev-parse instead? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] git-svn: keep leading slash when canonicalizing paths (fallback case)
Jonathan Nieder jrnie...@gmail.com wrote: Noticed by forcing the fallback on and running tests. Without this patch, t9101.4 fails: Bad URL passed to RA layer: Unable to open an ra_local session to \ URL: Local URL 'file://homejrnsrcgit-scratch/t/trash%20directory.\ t9101-git-svn-props/svnrepo' contains unsupported hostname at \ /home/jrn/src/git-scratch/perl/blib/lib/Git/SVN.pm line 148 With it, the git-svn tests pass again. Signed-off-by: Jonathan Nieder jrnie...@gmail.com Thanks for noticing this. Signed-off-by: Eric Wong normalper...@yhbt.net and pushed to my master at git://bogomips.org/git-svn -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC v2] git svn: work around SVN 1.7 mishandling of svn:special changes
Jonathan Nieder jrnie...@gmail.com wrote:
Jonathan Nieder wrote:
Revisions prepared with ordinary svn commands (svn add and not svn
propset) don't trip this because they represent filetype changes
using a replace operation [...]
Perhaps git
svn should mimic that,
... and here's what that looks like. I like this more than ignoring
the problem in tests, and I suppose something like this is necessary
regardless of how quickly issue 4091 is fixed for compatibility with
the broken versions of svn.
I prefer this v2 more than ignoring the problem in tests, too.
It would be nice if the patch could be more concise, though. What do
you think?
I think it's fine.
git-svn.perl | 25 -
I needed to filter the patch through:
s,git-svn\.perl,perl/Git/SVN/Editor.pm,g
though... Will push the edited version to my master on
git://bogomips.org/git-svn
From b8c78e2a9d6141589202e98b898f477861fcb111 Mon Sep 17 00:00:00 2001
From: Jonathan Nieder jrnie...@gmail.com
Date: Tue, 9 Oct 2012 03:12:39 -0700
Subject: [PATCH] git svn: work around SVN 1.7 mishandling of svn:special
changes
Subversion represents symlinks as ordinary files with content starting
with link and the svn:special property set to *. Thus a file can
switch between being a symlink and a non-symlink simply by toggling
its svn:special property, and new checkouts will automatically write a
file of the appropriate type. Likewise, in subversion 1.6 and older,
running svn update would notice changes in filetype and update the
working copy appropriately.
Starting in subversion 1.7 (issue 4091), changes to the svn:special
property trip an assertion instead:
$ svn up svn-tree
Updating 'svn-tree':
svn: E235000: In file 'subversion/libsvn_wc/update_editor.c' \
line 1583: assertion failed (action == svn_wc_conflict_action_edit \
|| action == svn_wc_conflict_action_delete || action == \
svn_wc_conflict_action_replace)
Revisions prepared with ordinary svn commands (svn add and not svn
propset) don't trip this because they represent these filetype
changes using a replace operation, which is approximately equivalent
to removal followed by adding a new file and works fine. Follow suit.
Noticed using t9100. After this change, git-svn's file-to-symlink
changes are sent in a format that modern svn update can handle and
tests t9100.11-13 pass again.
[ew: s,git-svn\.perl,perl/Git/SVN/Editor.pm,g]
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
Signed-off-by: Eric Wong normalper...@yhbt.net
---
perl/Git/SVN/Editor.pm | 25 -
1 file changed, 24 insertions(+), 1 deletion(-)
diff --git a/perl/Git/SVN/Editor.pm b/perl/Git/SVN/Editor.pm
index 755092f..3bbc20a 100644
--- a/perl/Git/SVN/Editor.pm
+++ b/perl/Git/SVN/Editor.pm
@@ -345,7 +345,30 @@ sub M {
$self-close_file($fbat,undef,$self-{pool});
}
-sub T { shift-M(@_) }
+sub T {
+ my ($self, $m, $deletions) = @_;
+
+ # Work around subversion issue 4091: toggling the is a
+ # symlink property requires removing and re-adding a
+ # file or else svn up on affected clients trips an
+ # assertion and aborts.
+ if (($m-{mode_b} =~ /^120/ $m-{mode_a} !~ /^120/) ||
+ ($m-{mode_b} !~ /^120/ $m-{mode_a} =~ /^120/)) {
+ $self-D({
+ mode_a = $m-{mode_a}, mode_b = '00',
+ sha1_a = $m-{sha1_a}, sha1_b = '0' x 40,
+ chg = 'D', file_b = $m-{file_b}
+ });
+ $self-A({
+ mode_a = '00', mode_b = $m-{mode_b},
+ sha1_a = '0' x 40, sha1_b = $m-{sha1_b},
+ chg = 'A', file_b = $m-{file_b}
+ });
+ return;
+ }
+
+ $self-M($m, $deletions);
+}
sub change_file_prop {
my ($self, $fbat, $pname, $pval) = @_;
--
1.8.0.rc0.42.gb8c78e2
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC] svn test: escape peg revision separator using empty peg rev
Jonathan Nieder jrnie...@gmail.com wrote: Michael J Gruber wrote: Jonathan Nieder venit, vidit, dixit 09.10.2012 10:41: Signed-off-by: Jonathan Nieder jrnie...@gmail.com Tested with Subversion 1.6.18. Thanks both. Also pushed to master on git://bogomips.org/git-svn.git (commit 44bc5ac71fd99f195bf1a3bea63c11139d2d535f) Jonathan Nieder (2): git svn: work around SVN 1.7 mishandling of svn:special changes svn test: escape peg revision separator using empty peg rev -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC] svn test: escape peg revision separator using empty peg rev
Jonathan Nieder jrnie...@gmail.com wrote: Eric Wong wrote: Thanks both. Also pushed to master on git://bogomips.org/git-svn.git (commit 44bc5ac71fd99f195bf1a3bea63c11139d2d535f) Jonathan Nieder (2): git svn: work around SVN 1.7 mishandling of svn:special changes svn test: escape peg revision separator using empty peg rev Thanks. Here's the $deletions nit as a patch on top. -- 8 -- Subject: Git::SVN::Editor::T: pass $deletions to -A and -D For future reference, it'd be slightly easier for me to apply if you included the From: (and Date:) headers so I don't have to yank+paste them myself : This shouldn't make a difference because the $deletions hash is only used when adding a directory (see 379862ec, 2012-02-20) but it's nice to be consistent to make reading smoother anyway. No functional change intended. Signed-off-by: Jonathan Nieder jrnie...@gmail.com Signed-off-by: Eric Wong normalper...@yhbt.net And pushed to master on git://bogomips.org/git-svn.git (commit a9608896587718549e82c5bae11740f2c0eac4c6) Jonathan Nieder (3): git svn: work around SVN 1.7 mishandling of svn:special changes svn test: escape peg revision separator using empty peg rev Git::SVN::Editor::T: pass $deletions to -A and -D -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC] svn test: escape peg revision separator using empty peg rev
Jonathan Nieder jrnie...@gmail.com wrote: Eric Wong wrote: Jonathan Nieder jrnie...@gmail.com wrote: -- 8 -- Subject: Git::SVN::Editor::T: pass $deletions to -A and -D For future reference, it'd be slightly easier for me to apply if you included the From: (and Date:) headers so I don't have to yank+paste them myself : Ah, I assumed you were using git am --scissors. Will do next time. I missed the addition of --scissors. Will use it in the future : -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
