Re: Cannot push anything via export transport helper after push fails.
Felipe Contreras wrote: On Sat, May 11, 2013 at 7:29 AM, Andrey Borzenkov arvidj...@gmail.com wrote: I noticed that using git-remote-bzr, but as far as I can tell this is generic for all transport helpers using fast-export. What happened was git push failed due to merge conflict. So far so good - but from now on git assumes everything is up to date. bor@opensuse:/tmp/test/git git push origin master To bzr::bzr+ssh://bor@localhost/tmp/test/bzr ! [rejected]master - master (non-fast-forward) error: failed to push some refs to 'bzr::bzr+ssh://bor@localhost/tmp/test/bzr' hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. Merge the remote changes (e.g. 'git pull') hint: before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. bor@opensuse:/tmp/test/git git push origin master Everything up-to-date bor@opensuse:/tmp/test/git The problem seems to be that git fast-export updates marks unconditionally, whether export actually applied or not. So next time it assumes everything is already exported and does nothing. Is it expected behavior? Indeed, this is the way it currently works, and it's not easy to fix. We would need some way to make fast-export wait until we know the exit status of the remote helper, and then tell it when it failed, so the marks are not updated. However, the way remote-bzr/hg work is that the commits are still there anyway. So if you merge the next time you push those commits are already converted, so it's not a problem if fast-export is not exporting them again. So even though it's not ideal, it should work. The problem is when the remote-helper crashes and the marks of fast-export and the remote-helper are out of sync, and then the user is really screwed. I sent patches that should fix this problem: http://article.gmane.org/gmane.comp.version-control.git/246187 -- Felipe Contreras -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Cannot push anything via export transport helper after push fails.
Andrey Borzenkov wrote: В Sat, 11 May 2013 08:57:14 -0500 Felipe Contreras felipe.contre...@gmail.com пишет: On Sat, May 11, 2013 at 7:29 AM, Andrey Borzenkov arvidj...@gmail.com wrote: I noticed that using git-remote-bzr, but as far as I can tell this is generic for all transport helpers using fast-export. What happened was git push failed due to merge conflict. So far so good - but from now on git assumes everything is up to date. bor@opensuse:/tmp/test/git git push origin master To bzr::bzr+ssh://bor@localhost/tmp/test/bzr ! [rejected]master - master (non-fast-forward) error: failed to push some refs to 'bzr::bzr+ssh://bor@localhost/tmp/test/bzr' hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. Merge the remote changes (e.g. 'git pull') hint: before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. bor@opensuse:/tmp/test/git git push origin master Everything up-to-date bor@opensuse:/tmp/test/git The problem seems to be that git fast-export updates marks unconditionally, whether export actually applied or not. So next time it assumes everything is already exported and does nothing. Is it expected behavior? Indeed, this is the way it currently works, and it's not easy to fix. We would need some way to make fast-export wait until we know the exit status of the remote helper, and then tell it when it failed, so the marks are not updated. Hmm ... actually as far as I understand transport-helper keeps track of which revisions to push in remote helper ref (for the lack of better word). This makes use of marks as tracking means rather redundant. What about the idea below? This relies on transport helper to provide correct revisions and uses marks exclusively as cross-reference between GIT and remote SCM. It is on top of next branch. This is one way of using it, but not ideal, and I think the patch series I sent should work for all remote helpers. -- Felipe Contreras-- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Cannot push anything via export transport helper after push fails.
В Sat, 11 May 2013 08:57:14 -0500
Felipe Contreras felipe.contre...@gmail.com пишет:
On Sat, May 11, 2013 at 7:29 AM, Andrey Borzenkov arvidj...@gmail.com wrote:
I noticed that using git-remote-bzr, but as far as I can tell this is
generic for all transport helpers using fast-export.
What happened was git push failed due to merge conflict. So far so
good - but from now on git assumes everything is up to date.
bor@opensuse:/tmp/test/git git push origin master
To bzr::bzr+ssh://bor@localhost/tmp/test/bzr
! [rejected]master - master (non-fast-forward)
error: failed to push some refs to
'bzr::bzr+ssh://bor@localhost/tmp/test/bzr'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Merge the remote changes (e.g. 'git pull')
hint: before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
bor@opensuse:/tmp/test/git git push origin master
Everything up-to-date
bor@opensuse:/tmp/test/git
The problem seems to be that git fast-export updates marks
unconditionally, whether export actually applied or not. So next time
it assumes everything is already exported and does nothing.
Is it expected behavior?
Indeed, this is the way it currently works, and it's not easy to fix.
We would need some way to make fast-export wait until we know the exit
status of the remote helper, and then tell it when it failed, so the
marks are not updated.
Hmm ... actually as far as I understand transport-helper keeps track of
which revisions to push in remote helper ref (for the lack of better
word). This makes use of marks as tracking means rather redundant.
What about the idea below? This relies on transport helper to provide
correct revisions and uses marks exclusively as cross-reference between
GIT and remote SCM. It is on top of next branch.
---
builtin/fast-export.c | 15 ++-
contrib/remote-helpers/git-remote-bzr | 11 ---
transport-helper.c| 7 ++-
3 files changed, 24 insertions(+), 9 deletions(-)
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index d60d675..5bc4b3c 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -29,6 +29,7 @@ static enum { ERROR, DROP, REWRITE } tag_of_filtered_mode =
ERROR;
static int fake_missing_tagger;
static int use_done_feature;
static int no_data;
+static int do_not_skip_marked_commits;
static int full_tree;
static int parse_opt_signed_tag_mode(const struct option *opt,
@@ -95,7 +96,8 @@ static inline void mark_object(struct object *object,
uint32_t mark)
static inline void mark_next_object(struct object *object)
{
- mark_object(object, ++last_idnum);
+ if (!(do_not_skip_marked_commits lookup_decoration(idnums, object)))
+ mark_object(object, ++last_idnum);
}
static int get_object_mark(struct object *object)
@@ -144,7 +146,7 @@ static void export_blob(const unsigned char *sha1)
mark_next_object(object);
- printf(blob\nmark :%PRIu32\ndata %lu\n, last_idnum, size);
+ printf(blob\nmark :%PRIu32\ndata %lu\n, get_object_mark(object),
size);
if (size fwrite(buf, size, 1, stdout) != 1)
die_errno (Could not write blob '%s', sha1_to_hex(sha1));
printf(\n);
@@ -326,7 +328,7 @@ static void handle_commit(struct commit *commit, struct
rev_info *rev)
if (!commit-parents)
printf(reset %s\n, (const char*)commit-util);
printf(commit %s\nmark :%PRIu32\n%.*s\n%.*s\ndata %u\n%s,
- (const char *)commit-util, last_idnum,
+ (const char *)commit-util, get_object_mark(commit-object),
(int)(author_end - author), author,
(int)(committer_end - committer), committer,
(unsigned)(reencoded
@@ -631,7 +633,7 @@ static void import_marks(char *input_file)
if (!object)
continue;
- if (object-flags SHOWN)
+ if (get_object_mark(object))
error(Object %s already has a mark,
sha1_to_hex(sha1));
if (object-type != OBJ_COMMIT)
@@ -640,7 +642,8 @@ static void import_marks(char *input_file)
mark_object(object, mark);
- object-flags |= SHOWN;
+ if (!do_not_skip_marked_commits)
+ object-flags |= SHOWN;
}
fclose(f);
}
@@ -673,6 +676,8 @@ int cmd_fast_export(int argc, const char **argv, const char
*prefix)
OPT_BOOLEAN(0, use-done-feature, use_done_feature,
N_(Use the done feature to terminate the
stream)),
OPT_BOOL(0, no-data, no_data, N_(Skip output of blob
data)),
+ OPT_BOOL(0, do-not-skip-marked-commits,
do_not_skip_marked_commits,
+N_(Do not skip marked commits)),
OPT_END()
Cannot push anything via export transport helper after push fails.
I noticed that using git-remote-bzr, but as far as I can tell this is generic for all transport helpers using fast-export. What happened was git push failed due to merge conflict. So far so good - but from now on git assumes everything is up to date. bor@opensuse:/tmp/test/git git push origin master To bzr::bzr+ssh://bor@localhost/tmp/test/bzr ! [rejected]master - master (non-fast-forward) error: failed to push some refs to 'bzr::bzr+ssh://bor@localhost/tmp/test/bzr' hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. Merge the remote changes (e.g. 'git pull') hint: before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. bor@opensuse:/tmp/test/git git push origin master Everything up-to-date bor@opensuse:/tmp/test/git The problem seems to be that git fast-export updates marks unconditionally, whether export actually applied or not. So next time it assumes everything is already exported and does nothing. Is it expected behavior? TIA -andrey -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Cannot push anything via export transport helper after push fails.
On Sat, May 11, 2013 at 04:29:36PM +0400, Andrey Borzenkov wrote: I noticed that using git-remote-bzr, but as far as I can tell this is generic for all transport helpers using fast-export. What happened was git push failed due to merge conflict. So far so good - but from now on git assumes everything is up to date. bor@opensuse:/tmp/test/git git push origin master To bzr::bzr+ssh://bor@localhost/tmp/test/bzr ! [rejected]master - master (non-fast-forward) error: failed to push some refs to 'bzr::bzr+ssh://bor@localhost/tmp/test/bzr' hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. Merge the remote changes (e.g. 'git pull') hint: before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. bor@opensuse:/tmp/test/git git push origin master Everything up-to-date bor@opensuse:/tmp/test/git The problem seems to be that git fast-export updates marks unconditionally, whether export actually applied or not. So next time it assumes everything is already exported and does nothing. Is it expected behavior? What version of Git are you using? This sounds similar to the regression fixed by commit 126aac5 (transport-helper: fix remote helper namespace regression, 2013-05-10) but that was only introduced in commit 664059f (transport-helper: update remote helper namespace, 2013-04-17) which isn't in any released versions of Git. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Cannot push anything via export transport helper after push fails.
On Sat, May 11, 2013 at 7:29 AM, Andrey Borzenkov arvidj...@gmail.com wrote: I noticed that using git-remote-bzr, but as far as I can tell this is generic for all transport helpers using fast-export. What happened was git push failed due to merge conflict. So far so good - but from now on git assumes everything is up to date. bor@opensuse:/tmp/test/git git push origin master To bzr::bzr+ssh://bor@localhost/tmp/test/bzr ! [rejected]master - master (non-fast-forward) error: failed to push some refs to 'bzr::bzr+ssh://bor@localhost/tmp/test/bzr' hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. Merge the remote changes (e.g. 'git pull') hint: before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. bor@opensuse:/tmp/test/git git push origin master Everything up-to-date bor@opensuse:/tmp/test/git The problem seems to be that git fast-export updates marks unconditionally, whether export actually applied or not. So next time it assumes everything is already exported and does nothing. Is it expected behavior? Indeed, this is the way it currently works, and it's not easy to fix. We would need some way to make fast-export wait until we know the exit status of the remote helper, and then tell it when it failed, so the marks are not updated. However, the way remote-bzr/hg work is that the commits are still there anyway. So if you merge the next time you push those commits are already converted, so it's not a problem if fast-export is not exporting them again. So even though it's not ideal, it should work. The problem is when the remote-helper crashes and the marks of fast-export and the remote-helper are out of sync, and then the user is really screwed. Cheers. -- Felipe Contreras -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Cannot push anything via export transport helper after push fails.
В Sat, 11 May 2013 08:57:14 -0500 Felipe Contreras felipe.contre...@gmail.com пишет: The problem seems to be that git fast-export updates marks unconditionally, whether export actually applied or not. So next time it assumes everything is already exported and does nothing. Is it expected behavior? Indeed, this is the way it currently works, and it's not easy to fix. We would need some way to make fast-export wait until we know the exit status of the remote helper, and then tell it when it failed, so the marks are not updated. One possibility would be to omit *export-marks and manage GIT marks in remote helper as well. Helper would then update synchronously both GIT and BZR marks if no errors were detected. Or even better, it could update just those commits that had been successful. However, the way remote-bzr/hg work is that the commits are still there anyway. So if you merge the next time you push those commits are already converted, so it's not a problem if fast-export is not exporting them again. As I understand bzr commit ID is stable. What happens if we try to commit the same ID second time? So even though it's not ideal, it should work. I'm more concerned about transport errors. Any network glitch during push renders you repository unusable (at least, without much efforts). The problem is when the remote-helper crashes and the marks of fast-export and the remote-helper are out of sync, and then the user is really screwed. This case would benefit from moving processing of GIT marks into remote helper as well. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Cannot push anything via export transport helper after push fails.
On Sat, May 11, 2013 at 1:48 PM, Andrey Borzenkov arvidj...@gmail.com wrote: В Sat, 11 May 2013 08:57:14 -0500 Felipe Contreras felipe.contre...@gmail.com пишет: The problem seems to be that git fast-export updates marks unconditionally, whether export actually applied or not. So next time it assumes everything is already exported and does nothing. Is it expected behavior? Indeed, this is the way it currently works, and it's not easy to fix. We would need some way to make fast-export wait until we know the exit status of the remote helper, and then tell it when it failed, so the marks are not updated. One possibility would be to omit *export-marks and manage GIT marks in remote helper as well. Helper would then update synchronously both GIT and BZR marks if no errors were detected. Or even better, it could update just those commits that had been successful. That would need to change the whole architecture, because right now the remote helpers are agnostic of Git SHA-1s. However, the way remote-bzr/hg work is that the commits are still there anyway. So if you merge the next time you push those commits are already converted, so it's not a problem if fast-export is not exporting them again. As I understand bzr commit ID is stable. What happens if we try to commit the same ID second time? It's skipped, because it's already converted. So even though it's not ideal, it should work. I'm more concerned about transport errors. Any network glitch during push renders you repository unusable (at least, without much efforts). No, it doesn't. If the remote-helper fails gracefully, the bzr revisions are converted and stored in the bzr repo, even if they were not pushed to the remote. So it's OK if fast-export never exports them again; we already have them. Cheers. -- Felipe Contreras -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
