RE: Porting git to HP NonStop

2012-08-14 Thread Joachim Schmitz
 From: Joachim Schmitz [mailto:j...@schmitz-digital.de]
 Sent: Friday, August 10, 2012 10:09 PM
 To: 'Shawn Pearce'
 Cc: 'git@vger.kernel.org'; 'rsbec...@nexbridge.com'
 Subject: RE: Porting git to HP NonStop
 
  From: Joachim Schmitz [mailto:j...@schmitz-digital.de]
  Sent: Friday, August 10, 2012 7:33 PM
  To: 'Shawn Pearce'
  Cc: 'git@vger.kernel.org'; 'rsbec...@nexbridge.com'
  Subject: RE: Porting git to HP NonStop
 
   From: Shawn Pearce [mailto:spea...@spearce.org]
   Sent: Friday, August 10, 2012 6:28 PM
   To: Joachim Schmitz
   Cc: git@vger.kernel.org; rsbec...@nexbridge.com
   Subject: Re: Porting git to HP NonStop
  
   On Fri, Aug 10, 2012 at 8:04 AM, Joachim Schmitz
   j...@schmitz-digital.de
   wrote:
 snip
- HP NonStop doesn't have stat.st_?time.nsec, there are several
places
what an
#ifdef USE_NSEC is missing, I can provide a diff if needed
(offending
files: builtin/fetch-pack.c and read-cache.c).
  
   I think this would be appreciated by anyone else that has a similar
   problem where the platform lacks nsec.
 
  Will do.
 
 OK, here we go:
 
 /usr/local/bin/diff -EBbu ./builtin/fetch-pack.c.orig
./builtin/fetch-pack.c
snip

Sorry, this is not needed if I just set NO_NSEC, so just forget about it
(and thanks to Junio for telling be)

 /usr/local/bin/diff -EBbu ./git-compat-util.h.orig ./git-compat-util.h
 --- ./git-compat-util.h.orig2012-07-30 15:50:38 -0500
 +++ ./git-compat-util.h 2012-08-10 09:59:56 -0500
 @@ -74,7 +74,8 @@
  # define _XOPEN_SOURCE 500
  # endif
  #elif !defined(__APPLE__)  !defined(__FreeBSD__)  !defined(__USLC__)
  \
 -  !defined(_M_UNIX)  !defined(__sgi)  !defined(__DragonFly__)
 +  !defined(_M_UNIX)  !defined(__sgi)  !defined(__DragonFly__) 
\
 +  !defined(__TANDEM)
  #define _XOPEN_SOURCE 600 /* glibc2 and AIX 5.3L need 500, OpenBSD
 needs 600 for S_ISLNK() */  #define _XOPEN_SOURCE_EXTENDED 1 /* AIX 5.3L
 needs this */  #endif @@ -98,6 +99,11 @@  #include stdlib.h  #include
 stdarg.h  #include string.h
 +#ifdef __TANDEM
 +# include strings.h /* for strcasecmp() */
 +  typedef long int intptr_t;
 +  typedef unsigned long int uintptr_t;
 +#endif
  #include errno.h
  #include limits.h
  #include sys/param.h

This one still stands though, unless someone can come up with a better idea?

Bye, Jojo

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


What should mergetool do with --no-prompt?

2012-08-14 Thread David Aguilar
Right now there are two code paths, resolving deletion conflicts
and resolving symlink conflicts, in git-mergetool that do not
honor --no-prompt.  They force user-interaction with the shell
even though the caller (such as a program) said that they do
not want to be prompted.

This was an oversight from when this option was first added.

I think a simple and sensible thing to do would be for mergetool
to skip over these entries when --no-prompt is supplied.

Does this sound like a good idea?
-- 
David
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: What should mergetool do with --no-prompt?

2012-08-14 Thread Charles Bailey
On Tue, Aug 14, 2012 at 12:07:26AM -0700, David Aguilar wrote:
 Right now there are two code paths, resolving deletion conflicts
 and resolving symlink conflicts, in git-mergetool that do not
 honor --no-prompt.  They force user-interaction with the shell
 even though the caller (such as a program) said that they do
 not want to be prompted.
 
 This was an oversight from when this option was first added.
 
 I think a simple and sensible thing to do would be for mergetool
 to skip over these entries when --no-prompt is supplied.
 
 Does this sound like a good idea?

--no-prompt is designed to remove the prompt before launching a
mergetool. This is because it is mostly pointless but does provide a
convenient point to interrupt (C-c) a large multifile conflict
resolution.

It was never supposed to be a batch mode switch. By it's very nature
mergetool is interactive so I don't see any advantage to pretending
otherwise.

If the documentation indicates otherwise then it's my opinion that
this is what needs to be fixed.

Charles.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: git diff vs git diff-files

2012-08-14 Thread Thomas Rast
Bernd Jendrissek bernd.jendris...@gmail.com writes:

 On Mon, Aug 13, 2012 at 5:02 PM, Thomas Rast tr...@student.ethz.ch wrote:
 Can you share this repository?

 This weird behaviour doesn't even survive making a copy (cp -a) of the
 whole repository, so I very much doubt making it available would be
 illuminative. My disk's SMART data seems okay. The weird-quotient just
 rose a bit.

That sounds like an issue with stat() data, and then it doesn't.  cp -a
would change all the inode numbers, triggering a full refresh of the
index in 'git diff'.  But I'm not sure exactly how this can lead to
disagreement, since both commands read *both* index and worktree version
of the file.  Very confusing.

Can you try the following:

  git ls-files --debug 
gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
  cp .git/index .git/index.orig
  touch gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
  # note, it is important that you run diff first
  git diff gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
  git diff-files -p gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
  git ls-files --debug 
gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net

 Or at least the pre- and post-change
 files, transferred in such a way that there won't be any whitespace
 damage (your snippets above show obvious damage).  You can use

 http://www.bpj-code.co.za/downloads.php/bugs/TwoStageAmp-output.net?text
 contains the output from git show a5ee1e7. Leave off the ?text for an
 application/octet-stream download.

Umm, that's only one side of the diff, isn't it?  The hunk header claims
that the hunk goes up to line 47, so there must be more changes beyond
what you showed.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread Thomas Rast
Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 # On branch master
 # Your branch and 'origin/master' have diverged,
 # and have 250 and 19 different commit(s) each, respectively.
 #
 nothing to commit (working directory clean)

 He asked me what to do and I told him to do what has always worked for
 me in the past when something like this happened: gitk, reset master
 branch to here (to a commit before the divergence and using --hard),
 git pull origin master. Problem solved.

There are several layers of pitfalls and misunderstandings here.

* Is your work origin/master..master (that is, anything in master but
  not origin/master) really so worthless as to make scrap it all! the
  normal course of resolution?

  Or perhaps the real reason for the divergence is that upstream rewrote
  its master (k!), in which case you should get them acquainted with
  the clue bat... and probably rebase instead of merge.

* pull = fetch + merge!  Repeat this a few times until it sinks in.
  Then print it on A0 and stick it up in your office or something.

  For your case this means that the pull command is roughly equivalent
  to

git fetch origin master
git merge FETCH_HEAD

  The two-arg form of fetch does *not* update origin/master.  Assuming
  you got the reset right, the merge will fast-forward to whatever
  origin's master points to -- but origin/master is still the old state!

* Resetting to something that you think will fast-forward, only to then
  fast-forward it to the newest state, is silly.  You can just reset to
  the newest state instead.

Taking all of this together, I think you should stop using two-arg
pull[*] or fetch, and replace your error-prone recipe with simply

  git fetch
  git reset --hard origin/master

Assuming, as before, that your local work is worthless.  Is it?
Otherwise it would be better to run something like

  git fetch
  git rebase origin/master


[*] it's ok if you use it with an URL instead of a remote nickname

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] rev-list docs: clarify --topo-order description

2012-08-14 Thread Michael Haggerty

On 08/14/2012 12:21 AM, Junio C Hamano wrote:

We said --date-order still does not violate the topology, but it
was still not clear enough.

Reword the description for both --date-order and --topo-order,
and add an illustration to it.

Signed-off-by: Junio C Hamano gits...@pobox.com


Thanks for this change.  I was recently trying to figure out the meaning 
of these ordering options myself, and found the old text confusing.



---

  * Let's do this before I forget...; came up in discussion $gmane/203370

  Documentation/rev-list-options.txt | 29 +++--
  1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/Documentation/rev-list-options.txt 
b/Documentation/rev-list-options.txt
index 6a4b635..dc501ee 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -579,15 +579,32 @@ Commit Ordering
  By default, the commits are shown in reverse chronological order.

  --topo-order::
-
-   This option makes them appear in topological order (i.e.
-   descendant commits are shown before their parents).
+   This option makes them appear in topological order.  Even
+   without this option, descendant commits are shown before
+   their parents, but this tries to avoid showing commits on
+   multiple lines of history intermixed.

  --date-order::

-   This option is similar to '--topo-order' in the sense that no
-   parent comes before all of its children, but otherwise things
-   are still ordered in the commit timestamp order.
+   Show no parents before all of its children, but otherwise
+   show commits in the commit timestamp order.
++
+For example, in a commit history like this:
++
+
+
+---1247
+   \  \
+3568---
+
+
++
+where the numbers denote the order of commit timestamps, `git
+rev-list` and friends with `--date-order` show the commits in the
+timestamp order: 8 7 6 5 4 3 2 1.
++
+With `--topo-order`, they would show 8 6 5 3 7 4 2 1 (or 8 7 4 2 6 5
+3 1), to avoid commits from two branches mixed together.


Is it possible to predict which of the two orders would be taken here? 
It would be nice for the results to be deterministic.  For example, 
topology ties could be broken by choosing the commit with the most 
recent timestamp.



  --reverse::




Michael

--
Michael Haggerty
mhag...@alum.mit.edu
http://softwareswirl.blogspot.com/

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] rev-list docs: clarify --topo-order description

2012-08-14 Thread Thomas Rast
Junio C Hamano gits...@pobox.com writes:

  --topo-order::
 -
 - This option makes them appear in topological order (i.e.
 - descendant commits are shown before their parents).
 + This option makes them appear in topological order.  Even
 + without this option, descendant commits are shown before
 + their parents, but this tries to avoid showing commits on
 + multiple lines of history intermixed.

I don't think that is true in general.  Without any -order options, we
process commits in date order, which *usually* means topological order,
but not always.  You can easily verify this:

  $ git init
  $ date
  Tue Aug 14 10:39:49 CEST 2012
  $ echo initial file
  $ git add file
  $ GIT_COMMITTER_DATE=Tue Aug 14 11:39:49 2012 git commit
  $ echo foo file
  $ git commit -mfoo file
  $ git checkout -bside HEAD^
  $ echo bar file
  $ git commit -mbar file
  $ git log --all --oneline
  8c71325 bar
  e5072d7 initial
  1be702c foo

So the --topo-order switch *ensures* that we process commits in
topological order even in the face of skewed clocks.

I suspect that

 + their parents, but this tries to avoid showing commits on
 + multiple lines of history intermixed.

is just a fortunate side effect of the topological sort.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: git diff vs git diff-files

2012-08-14 Thread Bernd Jendrissek
On Tue, Aug 14, 2012 at 10:18 AM, Thomas Rast tr...@student.ethz.ch wrote:
 That sounds like an issue with stat() data, and then it doesn't.  cp -a
 would change all the inode numbers, triggering a full refresh of the
 index in 'git diff'.  But I'm not sure exactly how this can lead to
 disagreement, since both commands read *both* index and worktree version
 of the file.  Very confusing.

 Can you try the following:

   git ls-files --debug 
 gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
   cp .git/index .git/index.orig
   touch gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
   # note, it is important that you run diff first
   git diff gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
   git diff-files -p 
 gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
   git ls-files --debug 
 gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net

I'm can't tell what this was supposed to prove, but I think it's
proven that I'm in the wrong mailing list, and that I should be taking
this up with gnome-terminal. The Q1 part deletion line seems to be
appearing (in the terminal) only after I've copypasted it into
another terminal.

Running the same commands in xterm (correctly) shows the Q1 part
deletion line without having to copy  paste.

At this point I doubt there's much value in publishing the output, but
just in case (and it may be relevant when i take it to
gnome-terminal), it's here:

http://www.bpj-code.co.za/downloads.php/bugs/git-output-87vcgl999v.fsf?text

Note that line 104 of that output appears in the same terminal from
which I copypaste it only after I switch back to it, after having
pasted the output. Oi vey.

 http://www.bpj-code.co.za/downloads.php/bugs/TwoStageAmp-output.net?text
 contains the output from git show a5ee1e7. Leave off the ?text for an
 application/octet-stream download.

 Umm, that's only one side of the diff, isn't it?  The hunk header claims
 that the hunk goes up to line 47, so there must be more changes beyond
 what you showed.

You're right - the hunk isn't even self-consistent!
http://www.bpj-code.co.za/downloads.php/bugs/git-add--p-TwoStageAmp-output.net?text
is the bad diff, and I count only 46 lines of (context + deletions),
while the hunk header claims the (correct) 47 lines corresponding to
the whole file covered in the hunk.

Thanks for your time; I'm fairly convinced now that git is just an
innocent accessory to another program's bug.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: git diff vs git diff-files

2012-08-14 Thread Thomas Rast
Bernd Jendrissek bernd.jendris...@gmail.com writes:

 On Tue, Aug 14, 2012 at 10:18 AM, Thomas Rast tr...@student.ethz.ch wrote:
 Can you try the following:

   git ls-files --debug 
 gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
   cp .git/index .git/index.orig
   touch gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
   # note, it is important that you run diff first
   git diff gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
   git diff-files -p 
 gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net
   git ls-files --debug 
 gnetlist/tests/common/outputs/osmond/TwoStageAmp-output.net

 I'm can't tell what this was supposed to prove, but I think it's
 proven that I'm in the wrong mailing list, and that I should be taking
 this up with gnome-terminal. The Q1 part deletion line seems to be
 appearing (in the terminal) only after I've copypasted it into
 another terminal.

 Running the same commands in xterm (correctly) shows the Q1 part
 deletion line without having to copy  paste.

Fun stuff.  This will be very interesting to debug.  It's still
possible, though perhaps not extremely likely, that git is writing
garbage to the terminal and it just happens to work for xterm.

Whatever you do next, *please* post the *exact* file contents *and*
output in a format that does not suffer any transport damage, neither to
whitespace nor to binary data (such as terminal escapes).  Piping
through xxd comes to mind, perhaps by using

  GIT_PAGER=xxd git -p diff   diff-hexdump

etc. to ensure that git takes the same code paths as when writing to
less.

Cutpaste from a terminal will at best give you exactly what the
terminal figures it is *currently displaying*, which is completely
different from what programs write to it.  At worst, it will also suffer
whitespace damage.  So pretty much everything you showed so far, except
for the raw data of one (why only one?!) side of the diff, is useless.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] Implement ACL module architecture and sample MySQL ACL module

2012-08-14 Thread Michal Novotny
Hi,
this is the patch to introduce the ACL module architecture into git
versioning system. The variable of GIT_BASE_DIR is being used to seek
for the modules if available. If variable is unset then daemon looks
for the /etc/git-daemon.conf file existence and reads the
'base_path' key if it exists to determine the root path for
the git-daemon. This will be referred to as to $GIT_BASE_DIR
directory and the ACL modules subdirectory should reside in path of
$GIT_BASE_DIR/modules/acl. For MySQL connection you have to
alter the modules/config/modgitacl_mysql.cfg file to be able
to connect to the MySQL server using your credentials.

For MySQL database connection 2 new tables will get created
on your server and in the desired database. One holds the
ACLs for IP addresses and repositories and the second is
optional logging to the history table. This is by default
disabled however it could be enabled by setting the log_history
column of the git_access_acl entry to 1.

The patch also introduces the global_access_rule key to the config
which defines the default access policy if address and repository
key combination doesn't exist in the database. This automatically
creates the default access rule for the repository (which is
basically an entry with addr field set to '%' as all possible
entries in this column) based on the key settings.

The search algorithm for the entries is as follows:
1) Try to find the default entry for the repository
   - If default entry doesn't exist then create it based on the
 global_access_rule key settings (if set and valid)
2) Try to find entry for repository and IP address to override
   default settings
3) Log entries access if appropriate

If the new default access rule is being created then logging is
enabled for 'deny' global_access_rule and disabled for 'allow'
global_access_rule settings.

The MySQL module is just the working example of how to use the
ACL architecture so it's not being compiled by default and user
has to compile it manually. Basically this module is showing how
git ACL infrastructure works and it's good point to start writing
such modules.

The patch has been tested using following command line on one terminal:

$ make  (cd modules/acl  make  cd -)  GIT_BASE_DIR=`pwd` ./git-daemon 
--export-all

and cloning the existing repository on the second terminal with testing
all possible options in the database (using the sample MySQL module)
and everything was working fine.

Thanks,
Michal

Signed-off-by: Michal Novotny minov...@redhat.com
---
 Makefile   |   1 +
 daemon.c   | 123 +
 modules/acl/Makefile   |   6 ++
 modules/acl/modgitacl_mysql.c  | 211 +
 modules/config/modgitacl_mysql.cfg |  12 +++
 modules/modules.h  |   9 ++
 6 files changed, 362 insertions(+)
 create mode 100644 modules/acl/Makefile
 create mode 100644 modules/acl/modgitacl_mysql.c
 create mode 100644 modules/config/modgitacl_mysql.cfg
 create mode 100644 modules/modules.h

diff --git a/Makefile b/Makefile
index 6b0c961..6fe32a7 100644
--- a/Makefile
+++ b/Makefile
@@ -932,6 +932,7 @@ ifeq ($(uname_S),OSF1)
NO_NSEC = YesPlease
 endif
 ifeq ($(uname_S),Linux)
+   BASIC_CFLAGS += -ldl -rdynamic
NO_STRLCPY = YesPlease
NO_MKSTEMPS = YesPlease
HAVE_PATHS_H = YesPlease
diff --git a/daemon.c b/daemon.c
index ab21e66..8a06c05 100644
--- a/daemon.c
+++ b/daemon.c
@@ -1,9 +1,14 @@
+#defineGIT_DAEMON_CONFIG_FILE  /etc/git-daemon.conf
+#defineGIT_DAEMON_CONFIG_FILE_PATH_KEY base_path
+#defineBUFSIZE 1  12
+
 #include cache.h
 #include pkt-line.h
 #include exec_cmd.h
 #include run-command.h
 #include strbuf.h
 #include string-list.h
+#include modules/modules.h
 
 #ifndef HOST_NAME_MAX
 #define HOST_NAME_MAX 256
@@ -256,6 +261,117 @@ static int daemon_error(const char *dir, const char *msg)
return -1;
 }
 
+static char* daemon_read_config(const char *filename, char *key)
+{
+   FILE *fp;
+   char line[BUFSIZE];
+
+   fp = fopen(filename, r);
+   if (fp == NULL)
+   return NULL;
+
+   while (!feof(fp)) {
+   fgets(line, sizeof(line), fp);
+
+   if (strncmp(line, key, strlen(key)) == 0) {
+   return strdup( line + strlen(key) + 3 );
+   }
+   }
+   fclose(fp);
+
+   return NULL;
+}
+
+static int check_access_addrdir(char *libname, char *base_path, char *addr, 
const char *dir)
+{
+   int ret = -EPERM;
+   void *lib = NULL;
+   void *pCheck = NULL;
+   typedef int (*tCheckFunc) (char *base_path, char *addr, const char 
*dir);
+
+   lib = dlopen(libname, RTLD_LAZY);
+   if (lib == NULL) {
+   logerror(%s: Cannot load ACL library '%s', __FUNCTION__, 
libname);
+   goto cleanup;
+   }
+
+   pCheck = 

Re: git diff vs git diff-files

2012-08-14 Thread Andreas Schwab
Thomas Rast tr...@student.ethz.ch writes:

 Whatever you do next, *please* post the *exact* file contents *and*
 output in a format that does not suffer any transport damage, neither to
 whitespace nor to binary data (such as terminal escapes).  Piping
 through xxd comes to mind, perhaps by using

   GIT_PAGER=xxd git -p diff   diff-hexdump

 etc. to ensure that git takes the same code paths as when writing to
 less.

Or use script(1).

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
And now for something completely different.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: git diff vs git diff-files

2012-08-14 Thread Bernd Jendrissek
On Tue, Aug 14, 2012 at 11:55 AM, Thomas Rast tr...@student.ethz.ch wrote:
 Fun stuff.  This will be very interesting to debug.  It's still
 possible, though perhaps not extremely likely, that git is writing
 garbage to the terminal and it just happens to work for xterm.

 Whatever you do next, *please* post the *exact* file contents *and*

Hmm, this really isn't about git anymore, as I've reduced this problem
to the following file:

000: 200a 200a 200a 200a 200a 200a 200a 200a   . . . . . . . .
010: 200a 200a 200a 200a 200a 200a 200a 200a   . . . . . . . .
020: 200a 200a 200a 200a 200a 200a 200a 200a   . . . . . . . .
030: 200a 200a 200a 200a 200a 200a 200a 200a   . . . . . . . .
040: 200a 200a 200a 200a 200a 200a 200a 200a   . . . . . . . .
050: 200a 200a 200a 200a 200a 200a 200a 200a   . . . . . . . .
060: 200a 200a 200a 200a 200a 200a 200a 200a   . . . . . . . .
070: 200a 200a 200a 200a 200a 200a . . . . . .

berndj@capybara:~/gaf.git$ nl /tmp/g
 2  
 3  
 4  
 5  
 6  
 7
[snip]
60  
61  
62  
berndj@capybara:~/gaf.git$

Anything shorter than that causes first the command line (what you
might call line 0, not the one you might call line 63, and yes, I do
not that this is close to a binary round number), and after that,
nothing to disappear.

strace confirms that gnome-terminal reads, but neglects to write, line 1.

I notice now that a second gnome-terminal tab in the same directory
running the same commands reliably displays the correct output. I must
have dumped an ELF file to the terminal at some point, or something
similarly daft like that, and forgotten to reset the terminal. Not
that I recall having done such a thing recently.

In any case, it's too late now; I ran reset(1) and the terminal is
back behaving itself.

Thanks for helping me get (close to) the bottom of this. If git were
to be guilty (which I very much doubt), the evidence is long gone now.
I'll chalk this up to cosmic rays or something.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] rev-list docs: clarify --topo-order description

2012-08-14 Thread Junio C Hamano
Thomas Rast tr...@student.ethz.ch writes:

 So the --topo-order switch *ensures* that we process commits in
 topological order even in the face of skewed clocks.

Yes, I *think* that I attempted to show with the illustration.

 I suspect that

 +their parents, but this tries to avoid showing commits on
 +multiple lines of history intermixed.

 is just a fortunate side effect of the topological sort.

I am not sure if it is side effect; I *think* it was the primary
objective we added topo-order in the first place.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] rev-list docs: clarify --topo-order description

2012-08-14 Thread Thomas Rast
Junio C Hamano gits...@pobox.com writes:

 Thomas Rast tr...@student.ethz.ch writes:

 So the --topo-order switch *ensures* that we process commits in
 topological order even in the face of skewed clocks.

 Yes, I *think* that I attempted to show with the illustration.

But then the new description is wrong.  It claims that children are
always before parents, which is not true in the face of clock skew.  Or
am I missing something?

 I suspect that

 +   their parents, but this tries to avoid showing commits on
 +   multiple lines of history intermixed.

 is just a fortunate side effect of the topological sort.

 I am not sure if it is side effect; I *think* it was the primary
 objective we added topo-order in the first place.

I won't judge that, since it's wy before my time :-)

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] rev-list docs: clarify --topo-order description

2012-08-14 Thread Junio C Hamano
Martin von Zweigbergk martin.von.zweigbe...@gmail.com writes:

 Still, the Even without this option strongly suggests to me that
 what follows (descendant commits are shown before parents) applies
 to the By default case. Would it be correct to say something like
 By default, the commits are shown in reverse chronological order.
 When commit limiting is in effect, descendant commits are shown before
 parents.?

I wonder if spelling out that level of detail is unnecessarily
overspecifying the behaviour.

In general, I'd prefer to keep the insn to end users to the minimum:
i.e. use the default when you do not care too deeply about the order
but you want to get commits in the range in reverse time order in
general and let the default take care of the detail, and use one of
the --foo-order options if a script wants to be more specific about
the order.  That way we do not have to get our hands tied to an
unnecessary degree and keep the door open for us to improve the
implementation.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Porting git to HP NonStop

2012-08-14 Thread Junio C Hamano
Joachim Schmitz j...@schmitz-digital.de writes:

 /usr/local/bin/diff -EBbu ./git-compat-util.h.orig ./git-compat-util.h
 --- ./git-compat-util.h.orig2012-07-30 15:50:38 -0500
 +++ ./git-compat-util.h 2012-08-10 09:59:56 -0500
 @@ -74,7 +74,8 @@
  # define _XOPEN_SOURCE 500
  # endif
  #elif !defined(__APPLE__)  !defined(__FreeBSD__)  !defined(__USLC__)
  \
 -  !defined(_M_UNIX)  !defined(__sgi)  !defined(__DragonFly__)
 +  !defined(_M_UNIX)  !defined(__sgi)  !defined(__DragonFly__) 
 \
 +  !defined(__TANDEM)
  #define _XOPEN_SOURCE 600 /* glibc2 and AIX 5.3L need 500, OpenBSD
 needs 600 for S_ISLNK() */  #define _XOPEN_SOURCE_EXTENDED 1 /* AIX 5.3L
 needs this */  #endif @@ -98,6 +99,11 @@  #include stdlib.h  #include
 stdarg.h  #include string.h
 +#ifdef __TANDEM
 +# include strings.h /* for strcasecmp() */
 +  typedef long int intptr_t;
 +  typedef unsigned long int uintptr_t;
 +#endif
  #include errno.h
  #include limits.h
  #include sys/param.h

 This one still stands though, unless someone can come up with a better idea?

This hunk looks unobtrusive and obviously will not impact other
platforms, which is good.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: What should mergetool do with --no-prompt?

2012-08-14 Thread Junio C Hamano
Charles Bailey char...@hashpling.org writes:

 On Tue, Aug 14, 2012 at 12:07:26AM -0700, David Aguilar wrote:
 Right now there are two code paths, resolving deletion conflicts
 and resolving symlink conflicts, in git-mergetool that do not
 honor --no-prompt.  They force user-interaction with the shell
 even though the caller (such as a program) said that they do
 not want to be prompted.
 
 This was an oversight from when this option was first added.
 
 I think a simple and sensible thing to do would be for mergetool
 to skip over these entries when --no-prompt is supplied.
 
 Does this sound like a good idea?

 --no-prompt is designed to remove the prompt before launching a
 mergetool. This is because it is mostly pointless but does provide a
 convenient point to interrupt (C-c) a large multifile conflict
 resolution.

 It was never supposed to be a batch mode switch. By it's very nature
 mergetool is interactive so I don't see any advantage to pretending
 otherwise.

Could it be that the calling user or script does not even have a
terminal but still can spawn the chosen mergetool backend and
interact with the user via its GUI?  Or it may have a terminal that
is hard for the user to interact with, and the prompt and read ans
may get stuck.

In such an environment, the ideal behaviour for the git mergetool
frontend may be not to interact via the terminal at all and instead
run its interaction to choose the resolution using a matching GUI
interface.  I see when read ans fails (e.g. the standard input to
the mergetool is closed), resolve_{symlink,deleted}_merge will not
get stuck but instead fail, so perhaps David's issue could be solved
by running git mergetool --tool=... /dev/null or something?

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] rev-list docs: clarify --topo-order description

2012-08-14 Thread Junio C Hamano
Thomas Rast tr...@student.ethz.ch writes:

 Junio C Hamano gits...@pobox.com writes:

 Thomas Rast tr...@student.ethz.ch writes:

 So the --topo-order switch *ensures* that we process commits in
 topological order even in the face of skewed clocks.

 Yes, I *think* that I attempted to show with the illustration.

 But then the new description is wrong.  It claims that children are
 always before parents, which is not true in the face of clock skew.

Ah, you are talking about the even without this option part.
Yeah, that does not hold true.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


RE: git on HP NonStop

2012-08-14 Thread Joachim Schmitz
 From: Junio C Hamano [mailto:gits...@pobox.com]
 Sent: Tuesday, August 14, 2012 4:44 PM
 To: Joachim Schmitz
 Subject: Re: git on HP NonStop
 
 Joachim Schmitz j...@schmitz-digital.de writes:
 
  Interesting, I never mentioned Tandem did I, But still you recognized
  HP NonStop as that.
 
 No, *you* did in your patch #ifdef.

Ah, I see.

  Well, I do care about that platform,  but if you don't, there's not
  much point in me trying to get Tandem specific patches applied, is it?
 
 As long as the change is isolated (i.e. compilation without #define
TANDEM

__TANDEM actually

 for other people will produce byte-for-byte identical result as before),
and
 cleanly made (i.e. the resulting source code is not littered with #ifdef
 TANDEM in many places), I do not think there is a reason not to have such
a
 patchset.

It isn't in many places, only 2 places in git-compat-util.h so far:
/usr/local/bin/diff -EBbu ./git-compat-util.h.orig ./git-compat-util.h
--- ./git-compat-util.h.orig2012-07-30 15:50:38 -0500
+++ ./git-compat-util.h 2012-08-12 11:26:46 -0500
@@ -74,7 +74,8 @@
 # define _XOPEN_SOURCE 500
 # endif
 #elif !defined(__APPLE__)  !defined(__FreeBSD__)  !defined(__USLC__) 
\
-  !defined(_M_UNIX)  !defined(__sgi)  !defined(__DragonFly__)
+  !defined(_M_UNIX)  !defined(__sgi)  !defined(__DragonFly__)  \
+  !defined(__TANDEM)
 #define _XOPEN_SOURCE 600 /* glibc2 and AIX 5.3L need 500, OpenBSD needs
600 for S_ISLNK() */
 #define _XOPEN_SOURCE_EXTENDED 1 /* AIX 5.3L needs this */
 #endif
 @@ -98,6 +99,11 @@
 #include stdlib.h
 #include stdarg.h
 #include string.h
+#ifdef __TANDEM
+# include strings.h /* for strcasecmp() */
+  typedef int intptr_t; /* not int * ?!? */
+  typedef unsigned int uintptr_t; /* not unsigned int * ?!? */
+#endif
 #include errno.h
 #include limits.h
 #include sys/param.h

Too much? The 2nd part is not necessary NonStop specific, any idea for a
better way?

There there's Makefile:
/usr/local/bin/diff -EBbu ./Makefile.orig ./Makefile
--- ./Makefile.orig 2012-07-30 15:50:38 -0500
+++ ./Makefile  2012-08-14 06:07:16 -0500
@@ -1297,6 +1297,45 @@
NO_CURL =
NO_EXPAT =
 endif
+ifeq ($(uname_S),NONSTOP_KERNEL)
+   CC = cc -c99 # needs some C99 features, inline is just one of them
+   CFLAGS= -g -O
+   prefix = /usr/local
+
+   # as detected by './configure'
+   #NO_CURL = YesPlease # missdetected, disabled, see below
+   NEEDS_SSL_WITH_CURL = YesPlease # added manually, see above
+   HAVE_LIBCHARSET_H=YesPlease
+   NEEDS_LIBICONV = YesPlease # needs libiconv first, changed further
down
+   NO_SYS_SELECT_H=UnfortunatelyYes
+   NO_D_TYPE_IN_DIRENT = YesPlease
+   NO_HSTRERROR=YesPlease
+   NO_STRCASESTR=YesPlease
+   NO_FNMATCH_CASEFOLD = YesPlease
+   NO_MEMMEM = YesPlease
+   NO_STRLCPY = YesPlease
+   NO_SETENV = YesPlease
+   NO_UNSETENV = YesPlease
+   NO_MKDTEMP = YesPlease
+   NO_MKSTEMPS = YesPlease
+   OLD_ICONV=UnfortunatelyYes # currently libiconv-1.9.1
+   NO_REGEX=YesPlease # Why? ToDo?
+   NO_PTHREADS=UnfortunatelyYes # ToDo? Using PUT, maybe?
+
+   # our's are in ${prefix}/bin
+   PERL_PATH = ${prefix}/bin/perl
+   PYTHON_PATH = ${prefix}/bin/python
+
+   # not detected (nor checked for) by './configure'
+   COMPAT_CFLAGS += -DSA_RESTART=0 # we don't have SA_RESTART on
NonStop
+   COMPAT_CFLAGS += -DHAVE_STRING_H=1 # needed in
compat/fnmatch/fnmatch.c
+   NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease # we don't have that on
NonStop
+   NO_NSEC = YesPlease # we don't have that on NonStop
+   NO_PREAD = YesPlease # we could use floss_pread though?
+   NO_MMAP = YesPlease # we could use floss_mmap though?
+   # newly implemented further down
+   NO_POLL = YesPlease # we could use floss_poll though?
+endif
 ifneq (,$(findstring MINGW,$(uname_S)))
pathsep = ;
NO_PREAD = YesPlease
@@ -1526,6 +1565,11 @@
LIB_4_CRYPTO = $(OPENSSL_LINK) -lcrypto
 endif
 endif
+ifndef NO_GETTEXT
+ifndef LIBC_CONTAINS_LIBINTL
+   EXTLIBS += -lintl
+endif
+endif
 ifdef NEEDS_LIBICONV
ifdef ICONVDIR
BASIC_CFLAGS += -I$(ICONVDIR)/include
@@ -1538,11 +1582,6 @@
 ifdef NEEDS_LIBGEN
EXTLIBS += -lgen
 Endif
-ifndef NO_GETTEXT
-ifndef LIBC_CONTAINS_LIBINTL
-   EXTLIBS += -lintl
-endif
-endif
 ifdef NEEDS_SOCKET
EXTLIBS += -lsocket
 endif
@@ -1591,6 +1630,11 @@
BASIC_CFLAGS += -DNO_GETTEXT
USE_GETTEXT_SCHEME ?= fallthrough
 endif
+ifdef NO_POLL
+   NO_SYS_POLL_H = YesPlease
+   COMPAT_CFLAGS += -DNO_POLL -Icompat/win32 # so it finds poll.h
+   COMPAT_OBJS += compat/win32/poll.o
+endif
 ifdef NO_STRCASESTR
COMPAT_CFLAGS += -DNO_STRCASESTR
COMPAT_OBJS += compat/strcasestr.o

How das that look to you?

  Finally, I would prefer to see any message that is addressed to me as
  the project maintainer to be Cc'ed to the list.  I feel 80% of my
  

Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread PJ Weisberg
On Mon, Aug 13, 2012 at 12:58 PM, Hilco Wijbenga
hilco.wijbe...@gmail.com wrote:
 Hi all,

 A colleague of mine (after a relatively long absence) noticed the
 following when running git status:

 # On branch master
 # Your branch and 'origin/master' have diverged,
 # and have 250 and 19 different commit(s) each, respectively.
 #
 nothing to commit (working directory clean)

 He asked me what to do and I told him to do what has always worked for
 me in the past when something like this happened: gitk, reset master
 branch to here (to a commit before the divergence and using --hard),
 git pull origin master. Problem solved.

 Well, not this one. This one is persistent. :-) I am at a loss what to
 do. master and origin/master do *not* point at the same commit.
 Even after the git reset --hard ... and git pull. Running my
 silver bullet solution gets us in the same situation every time.

I assume that the commit you reset to wasn't actually before the
divergence, then.  It sounds like what you're trying to do is just
long-hand for 'git reset --hard origin/master'.  As mentioned before,
that *does* assume that you want to throw out everything you've
committed locally.  If that's *not* the case, try 'git rebase
origin/master' or 'git pull --rebase'.  And then go slap the person
who rewrote the history of origin/master.

-PJ

Gehm's Corollary to Clark's Law: Any technology distinguishable from
magic is insufficiently advanced.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] gitweb: Add option to limit repositories to be shown by ServerName

2012-08-14 Thread Junio C Hamano
Michal Novotny minov...@redhat.com writes:

 this is the patch to limit repositories to be shown by the
 ServerName in the gitweb.cgi script. This is useful for cases
 you're hosting multiple websites on a single machine and you don't
 want all the repos to be shown in all of them.
 ...
 Use case scenario: Imagine you have one server running HTTPd
for 3 domains, let's call them domain1,
domain2 and domain3, and you want all of
them to have a git server accessible via
gitweb at these URLs:
1) http://domain1/git
2) http://domain2/git
3) http://domain3/git


I do not run gitweb myself, but isn't the problem you are describing
merely a symptom caused by your VirtualHost / sections that are
not configured correctly, and instead having a single instance of
gitweb cgi enabled for all the virtual hosts?  Why does such a
physical host want to have git repositories for different domains in
a single place that is covered by a single instance of gitweb (hence
a single $projectroot) in the first place?  After all, domain1's
notion of kernel git repository http://domain1/git/kernel.git
might be totally different from that of domain2's, so wouldn't it be
far more natural to have one $projectroot (hence one instance of
gitweb) per such domains, configured in their own VirtualHost /
sections?

Also, there may even be http://domain3/git that does not want to be
served by gitweb but something else, e.g. cgit, on the same physical
host, and in such a case, the approach taken by this patch still uses
gitweb only to fail the request without letting cgit have its
chance, no?
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] gitweb: Add option to limit repositories to be shown by ServerName

2012-08-14 Thread J.H.
On 08/14/2012 10:04 AM, Junio C Hamano wrote:
 Michal Novotny minov...@redhat.com writes:
 
 this is the patch to limit repositories to be shown by the
 ServerName in the gitweb.cgi script. This is useful for cases
 you're hosting multiple websites on a single machine and you don't
 want all the repos to be shown in all of them.
 ...
 Use case scenario: Imagine you have one server running HTTPd
for 3 domains, let's call them domain1,
domain2 and domain3, and you want all of
them to have a git server accessible via
gitweb at these URLs:
1) http://domain1/git
2) http://domain2/git
3) http://domain3/git
 
 
 I do not run gitweb myself, but isn't the problem you are describing
 merely a symptom caused by your VirtualHost / sections that are
 not configured correctly, and instead having a single instance of
 gitweb cgi enabled for all the virtual hosts?  Why does such a
 physical host want to have git repositories for different domains in
 a single place that is covered by a single instance of gitweb (hence
 a single $projectroot) in the first place?  After all, domain1's
 notion of kernel git repository http://domain1/git/kernel.git
 might be totally different from that of domain2's, so wouldn't it be
 far more natural to have one $projectroot (hence one instance of
 gitweb) per such domains, configured in their own VirtualHost /
 sections?
 
 Also, there may even be http://domain3/git that does not want to be
 served by gitweb but something else, e.g. cgit, on the same physical
 host, and in such a case, the approach taken by this patch still uses
 gitweb only to fail the request without letting cgit have its
 chance, no?

The use case scenario is doable with Apache's vhost directives, and is
likely a much better way to accomplish this then trying to teach gitweb
about vhosting.

http://httpd.apache.org/docs/2.2/vhosts/mass.html

should give some hints on how this can be done, but the long and the
short is, single install of gitweb with the config file using, say %0,
to dynamically figure out which domain to work from.

- John


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Implement ACL module architecture and sample MySQL ACL module

2012-08-14 Thread Junio C Hamano
Michal Novotny minov...@redhat.com writes:

 Hi,
 this is the patch to introduce the ACL module architecture into git
 versioning system.

No, it doesn't.  It adds something only to git daemon, but does
not affect any other uses of Git.

Side note: I am not saying other uses of Git must be ACL
controlled by MySQL database.  They shouldn't be.  I am only
saying that the proposed commit log message must match what the
change does.

Please familiarize yourself with Documentation/SubmittingPatches
first, and then imitate the style in existing commits in the history
and posted patches by the good developers (you can tell who they
are by observing the list traffic for a few weeks), by the way.

As git daemon already has a mechanism to specify what repositories
are served with whitelist or blacklist, I am not sure if this patch
adds enough value to the system to make us want to add further
complexity only to carry more code to be audited for security.

Opinions?
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Implement ACL module architecture and sample MySQL ACL module

2012-08-14 Thread Shawn Pearce
On Tue, Aug 14, 2012 at 9:12 AM, Junio C Hamano gits...@pobox.com wrote:
 Michal Novotny minov...@redhat.com writes:

 Hi,
 this is the patch to introduce the ACL module architecture into git
 versioning system.

 No, it doesn't.  It adds something only to git daemon, but does
 not affect any other uses of Git.

Yes, this part of the commit message also confused me until I read
through the patch further. :-(

 Side note: I am not saying other uses of Git must be ACL
 controlled by MySQL database.  They shouldn't be.  I am only
 saying that the proposed commit log message must match what the
 change does.

 Please familiarize yourself with Documentation/SubmittingPatches
 first, and then imitate the style in existing commits in the history
 and posted patches by the good developers (you can tell who they
 are by observing the list traffic for a few weeks), by the way.

 As git daemon already has a mechanism to specify what repositories
 are served with whitelist or blacklist, I am not sure if this patch
 adds enough value to the system to make us want to add further
 complexity only to carry more code to be audited for security.

 Opinions?

Traditionally Git has been about providing the plumbing to handle the
protocol and storage, and other tools that wrap git manage access
controls, e.g. UNIX filesystem or gitolite. I would strongly prefer to
keep that arrangement.

Parsing the request line of git-daemon is easy. But we could make it
easier. An alternative arrangement would be to add a new command line
flag to git daemon like --command-filter that names an executable
git-daemon will invoke after parsing the request line. It can pass
along the client IP address, command request, repository name, and
resolved repository path, and tie stdin/stdout to the client. This
binary can decide to exec the proper git binary for the named command,
or just exit to disconnect the client and refuse service. This makes
it simple for a tool like gitolite to plug into the git-daemon
authorization path, without needing to be the network daemon itself,
worry about number of active connection slots, etc.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread Hilco Wijbenga
On 14 August 2012 01:27, Thomas Rast tr...@student.ethz.ch wrote:
 Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 # On branch master
 # Your branch and 'origin/master' have diverged,
 # and have 250 and 19 different commit(s) each, respectively.
 #
 nothing to commit (working directory clean)

 He asked me what to do and I told him to do what has always worked for
 me in the past when something like this happened: gitk, reset master
 branch to here (to a commit before the divergence and using --hard),
 git pull origin master. Problem solved.

 There are several layers of pitfalls and misunderstandings here.

 * Is your work origin/master..master (that is, anything in master but
   not origin/master) really so worthless as to make scrap it all! the
   normal course of resolution?

Of course, it's master. Nobody should be working on master directly.

   Or perhaps the real reason for the divergence is that upstream rewrote
   its master (k!), in which case you should get them acquainted with
   the clue bat... and probably rebase instead of merge.

Upstream is fine. Nobody else is having any problems.

 * pull = fetch + merge!  Repeat this a few times until it sinks in.
   Then print it on A0 and stick it up in your office or something.

Yes, I know.

   For your case this means that the pull command is roughly equivalent
   to

 git fetch origin master
 git merge FETCH_HEAD

   The two-arg form of fetch does *not* update origin/master.  Assuming
   you got the reset right, the merge will fast-forward to whatever
   origin's master points to -- but origin/master is still the old state!

Ah, now we're getting to something I did *not* know. :-) So FETCH_HEAD
!= origin/master? I tried to find out more information about
FETCH_HEAD but there doesn't seem to be much. I have seen FETCH_HEAD
show up in the terminal but always just ignored it as a Git
implementation detail. When/how does origin/master get set then? I
always assumed that was part of git fetch and then git merge would
actually move master to origin/master.

 * Resetting to something that you think will fast-forward, only to then
   fast-forward it to the newest state, is silly.  You can just reset to
   the newest state instead.

:-) Well, yeah, now that you point it out... :-)

Still, just resetting ignores all the problems that led to the current
situation. Normally, when I reset and then FF I can be sure (if it
works) that things were not completely screwed up. At least, that has
always been my theory.

 Taking all of this together, I think you should stop using two-arg
 pull[*] or fetch, and replace your error-prone recipe with simply

   git fetch
   git reset --hard origin/master

 Assuming, as before, that your local work is worthless.  Is it?
 Otherwise it would be better to run something like

   git fetch
   git rebase origin/master


 [*] it's ok if you use it with an URL instead of a remote nickname

Why would that be okay? What is the difference? Isn't the nickname
just an alias for a URL?
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Implement ACL module architecture and sample MySQL ACL module

2012-08-14 Thread Junio C Hamano
Shawn Pearce spea...@spearce.org writes:

 Parsing the request line of git-daemon is easy. But we could make it
 easier. An alternative arrangement would be to add a new command line
 flag to git daemon like --command-filter that names an executable
 git-daemon will invoke after parsing the request line. It can pass
 along the client IP address, command request, repository name, and
 resolved repository path, and tie stdin/stdout to the client. This
 binary can decide to exec the proper git binary for the named command,
 or just exit to disconnect the client and refuse service. This makes
 it simple for a tool like gitolite to plug into the git-daemon
 authorization path, without needing to be the network daemon itself,
 worry about number of active connection slots, etc.

I think that is a good direction to go in, except that I am unsure
what kind of conversation do you want to allow between the command
filter helper and the client by exposing standard input and output
stream to to the helper.  If the client side has a matching pre
negotiate command helper support, then presumably the helpers can
discuss what Git protocol proper does not care about before deciding
to allow the connection go through, but until that happens, opening
the stdio streams up to the helper sounds like an accident waiting
to happen to me (e.g. fetch-pack connects, the server side helper
reads the first pkt-line from the client, says OK, you may proceed
to the daemon, then the daemon spawns the upload-pack, which will
obviously see a corrupt request stream from fetch-pack).
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread Hilco Wijbenga
On 14 August 2012 09:02, PJ Weisberg p...@irregularexpressions.net wrote:
 On Mon, Aug 13, 2012 at 12:58 PM, Hilco Wijbenga
 hilco.wijbe...@gmail.com wrote:
 Hi all,

 A colleague of mine (after a relatively long absence) noticed the
 following when running git status:

 # On branch master
 # Your branch and 'origin/master' have diverged,
 # and have 250 and 19 different commit(s) each, respectively.
 #
 nothing to commit (working directory clean)

 He asked me what to do and I told him to do what has always worked for
 me in the past when something like this happened: gitk, reset master
 branch to here (to a commit before the divergence and using --hard),
 git pull origin master. Problem solved.

 Well, not this one. This one is persistent. :-) I am at a loss what to
 do. master and origin/master do *not* point at the same commit.
 Even after the git reset --hard ... and git pull. Running my
 silver bullet solution gets us in the same situation every time.

 I assume that the commit you reset to wasn't actually before the
 divergence, then.

It was according to gitk.

 It sounds like what you're trying to do is just
 long-hand for 'git reset --hard origin/master'.  As mentioned before,
 that *does* assume that you want to throw out everything you've
 committed locally.  If that's *not* the case, try 'git rebase
 origin/master' or 'git pull --rebase'.  And then go slap the person
 who rewrote the history of origin/master.

I'm not convinced anything is wrong with origin/master. This
particular colleague is the only one with a problem. And not for the
first time. :-)
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: What should mergetool do with --no-prompt?

2012-08-14 Thread Charles Bailey
On Tue, Aug 14, 2012 at 08:06:56AM -0700, Junio C Hamano wrote:
 
 Could it be that the calling user or script does not even have a
 terminal but still can spawn the chosen mergetool backend and
 interact with the user via its GUI?  Or it may have a terminal that
 is hard for the user to interact with, and the prompt and read ans
 may get stuck.

It could be, although this certainly wasn't considered in the original
design. I know that we removed explicit references to /dev/tty and
replaced them with exec nm juggling which made things generally more
robust and allowed some basic shell tests to work more reliably. I
don't object to handling non-interactive mode better but it feels
unsatisfactory to only be able to resolve some types of conflict and
have to skip others.

 In such an environment, the ideal behaviour for the git mergetool
 frontend may be not to interact via the terminal at all and instead
 run its interaction to choose the resolution using a matching GUI
 interface.  I see when read ans fails (e.g. the standard input to
 the mergetool is closed), resolve_{symlink,deleted}_merge will not
 get stuck but instead fail, so perhaps David's issue could be solved
 by running git mergetool --tool=... /dev/null or something?

To be honest, I wasn't sure what David's issue was, other than I
spotted this could/should it be fixed?. Is it a real world issue?

Charles.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread Junio C Hamano
Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 On 14 August 2012 01:27, Thomas Rast tr...@student.ethz.ch wrote:
 Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 # On branch master
 # Your branch and 'origin/master' have diverged,
 # and have 250 and 19 different commit(s) each, respectively.
 #
 nothing to commit (working directory clean)

 He asked me what to do and I told him to do what has always worked for
 me in the past when something like this happened: gitk, reset master
 branch to here (to a commit before the divergence and using --hard),
 git pull origin master. Problem solved.

 There are several layers of pitfalls and misunderstandings here.

 * Is your work origin/master..master (that is, anything in master but
   not origin/master) really so worthless as to make scrap it all! the
   normal course of resolution?

 Of course, it's master. Nobody should be working on master directly.

What a strange thing to say.  When will 'master' ever be updated
then and how?

If you mean 'master' as the integration branch for everybody to meet
and make progress, it would be more common for everybody to be
working on his own topic branch until perfection of the topic,
concluded by merging the completed topic to master and pushing the
master out to update it, no?

 * pull = fetch + merge!  Repeat this a few times until it sinks in.
   Then print it on A0 and stick it up in your office or something.

 Yes, I know.

   For your case this means that the pull command is roughly equivalent
   to

 git fetch origin master
 git merge FETCH_HEAD

   The two-arg form of fetch does *not* update origin/master.  Assuming
   you got the reset right, the merge will fast-forward to whatever
   origin's master points to -- but origin/master is still the old state!

 Ah, now we're getting to something I did *not* know. :-) So FETCH_HEAD
 != origin/master?

 I tried to find out more information about
 FETCH_HEAD but there doesn't seem to be much. I have seen FETCH_HEAD
 show up in the terminal but always just ignored it as a Git
 implementation detail. When/how does origin/master get set then? I
 always assumed that was part of git fetch and then git merge would
 actually move master to origin/master.

It could be git fetch --help is failing for you, but I am
reasonably sure most if not all of the above are answered there;
another thing something you may not have known :-).

 Taking all of this together, I think you should stop using two-arg
 pull[*] or fetch, and replace your error-prone recipe with simply

   git fetch
   git reset --hard origin/master

 Assuming, as before, that your local work is worthless.  Is it?
 Otherwise it would be better to run something like

   git fetch
   git rebase origin/master

Yeah, the latter makes sense, and I think it is a safer superset of
the former.  If there is nothing of value on 'master', the rebase
might stop and give control back to the user, but the user can tell
it to skip the cruft that came from the old 'master'.

 [*] it's ok if you use it with an URL instead of a remote nickname

 Why would that be okay? What is the difference? Isn't the nickname
 just an alias for a URL?

As long as you tell what refspecs to use on the command line, the
remote nickname behaves as just an alias for a URL, so yes,
because Thomas is discussing two-arg pull or fetch, one arg being
either nickname or URL and the other is an explicit refspec on the
command line, it would be okay because there is no difference in
that case.

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: What should mergetool do with --no-prompt?

2012-08-14 Thread Junio C Hamano
Charles Bailey char...@hashpling.org writes:

 On Tue, Aug 14, 2012 at 08:06:56AM -0700, Junio C Hamano wrote:
 
 Could it be that the calling user or script does not even have a
 terminal but still can spawn the chosen mergetool backend and
 interact with the user via its GUI?  Or it may have a terminal that
 is hard for the user to interact with, and the prompt and read ans
 may get stuck.

 It could be, although this certainly wasn't considered in the original
 design. I know that we removed explicit references to /dev/tty and
 replaced them with exec nm juggling which made things generally more
 robust and allowed some basic shell tests to work more reliably. I
 don't object to handling non-interactive mode better but it feels
 unsatisfactory to only be able to resolve some types of conflict and
 have to skip others.

Exactly.  The mention of a matching GUI below you quoted was a
suggestion to improve that only resolve some but not others
problem.  The usual mergetool backend, e.g. meld, may not be capable
of resolving symlink conflicts, but git mergetool could run a GUI
dialog that gives the user ours/theirs/fail choice (or better yet
merge result value textbox in addition) for such a path.  The same
for delete/modify conflicts.

 In such an environment, the ideal behaviour for the git mergetool
 frontend may be not to interact via the terminal at all and instead
 run its interaction to choose the resolution using a matching GUI
 interface.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Implement ACL module architecture and sample MySQL ACL module

2012-08-14 Thread Shawn Pearce
On Tue, Aug 14, 2012 at 10:06 AM, Junio C Hamano gits...@pobox.com wrote:
 Shawn Pearce spea...@spearce.org writes:

 Parsing the request line of git-daemon is easy. But we could make it
 easier. An alternative arrangement would be to add a new command line
 flag to git daemon like --command-filter that names an executable
 git-daemon will invoke after parsing the request line. It can pass
 along the client IP address, command request, repository name, and
 resolved repository path, and tie stdin/stdout to the client. This
 binary can decide to exec the proper git binary for the named command,
 or just exit to disconnect the client and refuse service. This makes
 it simple for a tool like gitolite to plug into the git-daemon
 authorization path, without needing to be the network daemon itself,
 worry about number of active connection slots, etc.

 I think that is a good direction to go in, except that I am unsure
 what kind of conversation do you want to allow between the command
 filter helper and the client by exposing standard input and output
 stream to to the helper.

Sorry, I was thinking the helper would exec the git command, and thus
pass along the stdin/stdout socket.

  If the client side has a matching pre
 negotiate command helper support, then presumably the helpers can
 discuss what Git protocol proper does not care about before deciding
 to allow the connection go through, but until that happens, opening
 the stdio streams up to the helper sounds like an accident waiting
 to happen to me (e.g. fetch-pack connects, the server side helper
 reads the first pkt-line from the client, says OK, you may proceed
 to the daemon, then the daemon spawns the upload-pack, which will
 obviously see a corrupt request stream from fetch-pack).

But seeing this, yes, that is a bad idea. Better to treat that like a
hook, where exit status 0 allows the connection to continue, and exit
status non-zero causes the connection to be closed. Maybe with an
error printed to stderr (if any) being echoed first to the client if
possible using the ERR formatting notation.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread Hilco Wijbenga
On 14 August 2012 10:19, Junio C Hamano gits...@pobox.com wrote:
 Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 On 14 August 2012 01:27, Thomas Rast tr...@student.ethz.ch wrote:
 Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 # On branch master
 # Your branch and 'origin/master' have diverged,
 # and have 250 and 19 different commit(s) each, respectively.
 #
 nothing to commit (working directory clean)

 He asked me what to do and I told him to do what has always worked for
 me in the past when something like this happened: gitk, reset master
 branch to here (to a commit before the divergence and using --hard),
 git pull origin master. Problem solved.

 There are several layers of pitfalls and misunderstandings here.

 * Is your work origin/master..master (that is, anything in master but
   not origin/master) really so worthless as to make scrap it all! the
   normal course of resolution?

 Of course, it's master. Nobody should be working on master directly.

 What a strange thing to say.  When will 'master' ever be updated
 then and how?

Well, yes, just before pushing, you'd work on master for a few
seconds. That doesn't really count. :-)

 If you mean 'master' as the integration branch for everybody to meet
 and make progress, it would be more common for everybody to be
 working on his own topic branch until perfection of the topic,
 concluded by merging the completed topic to master and pushing the
 master out to update it, no?

That's what I should have said. I assumed way too much context. I
don't think all neurons are firing yet. :-(

 * pull = fetch + merge!  Repeat this a few times until it sinks in.
   Then print it on A0 and stick it up in your office or something.

 Yes, I know.

   For your case this means that the pull command is roughly equivalent
   to

 git fetch origin master
 git merge FETCH_HEAD

   The two-arg form of fetch does *not* update origin/master.  Assuming
   you got the reset right, the merge will fast-forward to whatever
   origin's master points to -- but origin/master is still the old state!

 Ah, now we're getting to something I did *not* know. :-) So FETCH_HEAD
 != origin/master?

  I tried to find out more information about
 FETCH_HEAD but there doesn't seem to be much. I have seen FETCH_HEAD
 show up in the terminal but always just ignored it as a Git
 implementation detail. When/how does origin/master get set then? I
 always assumed that was part of git fetch and then git merge would
 actually move master to origin/master.

 It could be git fetch --help is failing for you, but I am
 reasonably sure most if not all of the above are answered there;
 another thing something you may not have known :-).

I was actually looking at git help merge since git help fetch
would be a far too logical place for FETCH_HEAD information. ;-) As I
said, not all neurons seem to be firing yet.

Apparently, my understanding is mostly correct, though. FETCH_HEAD is
indeed origin/master (I mean the SHA1 in master's HEAD on origin) and
the git merge part of git pull eventually sets my origin/master.

 Taking all of this together, I think you should stop using two-arg
 pull[*] or fetch, and replace your error-prone recipe with simply

   git fetch
   git reset --hard origin/master

 Assuming, as before, that your local work is worthless.  Is it?
 Otherwise it would be better to run something like

   git fetch
   git rebase origin/master

 Yeah, the latter makes sense, and I think it is a safer superset of
 the former.  If there is nothing of value on 'master', the rebase
 might stop and give control back to the user, but the user can tell
 it to skip the cruft that came from the old 'master'.

 [*] it's ok if you use it with an URL instead of a remote nickname

 Why would that be okay? What is the difference? Isn't the nickname
 just an alias for a URL?

 As long as you tell what refspecs to use on the command line, the
 remote nickname behaves as just an alias for a URL, so yes,
 because Thomas is discussing two-arg pull or fetch, one arg being
 either nickname or URL and the other is an explicit refspec on the
 command line, it would be okay because there is no difference in
 that case.

I suppose I'm not entirely clear on how this two step process is
safer. Doing git fetch would seem to be harmless, right? So the
problem is with git merge but master should always be behind
origin/master so that git merge should just FF to origin/master
which *should* be completely safe. Does that make sense? Especially
given our use of master as an integration branch?

[Given the trouble I have with getting people to use Git properly, I
prefer things as simple as possible. :-) ]
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Implement ACL module architecture and sample MySQL ACL module

2012-08-14 Thread Junio C Hamano
Shawn Pearce spea...@spearce.org writes:

 But seeing this, yes, that is a bad idea. Better to treat that like a
 hook, where exit status 0 allows the connection to continue, and exit
 status non-zero causes the connection to be closed. Maybe with an
 error printed to stderr (if any) being echoed first to the client if
 possible using the ERR formatting notation.

Yeah, note that we can only give a single ERR  line, though.

Something like this?  Totally untested, of course ;-)

 daemon.c | 79 
 1 file changed, 79 insertions(+)

diff --git a/daemon.c b/daemon.c
index ab21e66..41a9679 100644
--- a/daemon.c
+++ b/daemon.c
@@ -30,6 +30,7 @@ static const char daemon_usage[] =
[--interpolated-path=path]\n
[--reuseaddr] [--pid-file=file]\n
[--(enable|disable|allow-override|forbid-override)=service]\n
+   [--access-hook=path]\n
[--inetd | [--listen=host_or_ipaddr] [--port=n]\n
   [--detach] [--user=user [--group=group]]\n
[directory...];
@@ -256,6 +257,73 @@ static int daemon_error(const char *dir, const char *msg)
return -1;
 }
 
+static char *access_hook;
+
+static int run_access_hook(struct daemon_service *service, const char *dir, 
const char *path)
+{
+   struct child_process child;
+   struct strbuf buf = STRBUF_INIT;
+   const char *argv[8];
+   const char **arg = argv;
+   char *eol;
+   int seen_errors = 0;
+
+#define STRARG(x) ((x) ? (x) : )
+   *arg++ = access_hook;
+   *arg++ = service-name;
+   *arg++ = path;
+   *arg++ = STRARG(hostname);
+   *arg++ = STRARG(canon_hostname);
+   *arg++ = STRARG(ip_address);
+   *arg++ = STRARG(tcp_port);
+   *arg = NULL;
+#undef STRARG
+
+   memset(child, 0, sizeof(child));
+   child.use_shell = 1;
+   child.argv = argv;
+   child.no_stdin = 1;
+   child.no_stderr = 1;
+   child.out = -1;
+   if (start_command(child)) {
+   logerror(daemon access hook '%s' failed to start,
+access_hook);
+   goto error_return;
+   }
+   if (strbuf_read(buf, child.out, 0)  0) {
+   logerror(failed to read from pipe to daemon access hook '%s',
+access_hook);
+   strbuf_reset(buf);
+   seen_errors = 1;
+   }
+   if (close(child.out)  0) {
+   logerror(failed to close pipe to daemon access hook '%s',
+access_hook);
+   seen_errors = 1;
+   }
+   if (finish_command(child)  0) {
+   logerror(failed to finish-command daemon access hook '%s',
+access_hook);
+   seen_errors = 1;
+   }
+   if (!seen_errors) {
+   strbuf_release(buf);
+   return 0;
+   }
+
+error_return:
+   strbuf_ltrim(buf);
+   if (!buf.len)
+   strbuf_addstr(buf, service rejected);
+   eol = strchr(buf.buf, '\n');
+   if (eol)
+   *eol = '\0';
+   errno = EACCES;
+   daemon_error(dir, buf.buf);
+   strbuf_release(buf);
+   return -1;
+}
+
 static int run_service(char *dir, struct daemon_service *service)
 {
const char *path;
@@ -304,6 +372,13 @@ static int run_service(char *dir, struct daemon_service 
*service)
}
 
/*
+* Optionally, a hook can choose to deny access to the
+* repository depending on the phase of the moon.
+*/
+   if (access_hook  run_access_hook(service, dir, path))
+   return -1;
+
+   /*
 * We'll ignore SIGTERM from now on, we have a
 * good client.
 */
@@ -1142,6 +1217,10 @@ int main(int argc, char **argv)
export_all_trees = 1;
continue;
}
+   if (!prefixcmp(arg, --access-hook=)) {
+   access_hook = arg + 14;
+   continue;
+   }
if (!prefixcmp(arg, --timeout=)) {
timeout = atoi(arg+10);
continue;
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread Junio C Hamano
Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 I suppose I'm not entirely clear on how this two step process is
 safer. Doing git fetch would seem to be harmless, right? So the
 problem is with git merge but master should always be behind
 origin/master so that git merge should just FF to origin/master
 which *should* be completely safe. Does that make sense? Especially
 given our use of master as an integration branch?

 [Given the trouble I have with getting people to use Git properly, I
 prefer things as simple as possible. :-) ]

Between the two procedures Thomas gave you, fetch  rebase is
safer than fetch  reset --hard, exactly because it does not have
to rely on the validity of your which should always be behind
claim.

If it is behind, there won't be any difference, but if it is *not*,
the user will notice and won't lose his work on 'master' (which you
may argue that he shouldn't have done).  rebase will notice it.

The key for a procedure to be safe is not having to rely on the
claim of users such as my history *should* always and already be
behind, and not silently lose information when these *should*s are
violated for whatever reason.  After all, if all these *should*s
were true, the user wouldn't have been having problems in the first
place and posting to the list asking for help in the first place ;-)

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: What should mergetool do with --no-prompt?

2012-08-14 Thread David Aguilar
On Tue, Aug 14, 2012 at 10:23 AM, Junio C Hamano gits...@pobox.com wrote:
 Charles Bailey char...@hashpling.org writes:

 On Tue, Aug 14, 2012 at 08:06:56AM -0700, Junio C Hamano wrote:

 Could it be that the calling user or script does not even have a
 terminal but still can spawn the chosen mergetool backend and
 interact with the user via its GUI?  Or it may have a terminal that
 is hard for the user to interact with, and the prompt and read ans
 may get stuck.

 It could be, although this certainly wasn't considered in the original
 design. I know that we removed explicit references to /dev/tty and
 replaced them with exec nm juggling which made things generally more
 robust and allowed some basic shell tests to work more reliably. I
 don't object to handling non-interactive mode better but it feels
 unsatisfactory to only be able to resolve some types of conflict and
 have to skip others.

 Exactly.  The mention of a matching GUI below you quoted was a
 suggestion to improve that only resolve some but not others
 problem.  The usual mergetool backend, e.g. meld, may not be capable
 of resolving symlink conflicts, but git mergetool could run a GUI
 dialog that gives the user ours/theirs/fail choice (or better yet
 merge result value textbox in addition) for such a path.  The same
 for delete/modify conflicts.

 In such an environment, the ideal behaviour for the git mergetool
 frontend may be not to interact via the terminal at all and instead
 run its interaction to choose the resolution using a matching GUI
 interface.

That makes sense.  Perhaps a tcl/tk dialog could be used for these
when a special flag, e.g. --interactive-gui or something, is
supplied. I think that would be nice, and I can look into this when
I have some more tcl/tk hacking time.

I did have a real-world example -- a GUI that runs git-mergetool
on the user's behalf that (wrongly) assumed that git mergetool -y
would not block waiting for input.  This is not a problem with the
documentation or with the implementation -- it was simply an oversight
on my part.

Due to backwards-compatibility concerns, the only way to do
this (and work across as many git versions as possible) is to
detect these special cases -- submodules, symlinks, and deletions
-- and handle it in the calling app instead of delegating to mergetool.


There is a part of me that thinks that --no-prompt should
really not prompt, and that the various choices could be supplied
on the command-line, e.g. git mergetool --theirs --no-prompt path.

Flags like --ours and --theirs would be heavy hammers when run
without pathspecs. Nonetheless, would it would be helpful to have
these? I seems like it would be helpful when resolving these
special-case merge scenarios.

So I think I'll take the advice that the calling app should
special-case these for existing git versions, and then later
allow them to fall through to mergetool once we've had a chance
to add a matching GUI interface.

Thanks for the sanity check,
-- 
David
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH/RFC v3 00/16] GSOC remote-svn

2012-08-14 Thread Florian Achleitner
Hi.

Version 3 of this series adds the 'bidi-import' capability, as suggested
Jonathan. 
Diff details are attached to the patches.
04 and 05 are completely new.

[PATCH/RFC v3 01/16] Implement a remote helper for svn in C.
[PATCH/RFC v3 02/16] Integrate remote-svn into svn-fe/Makefile.
[PATCH/RFC v3 03/16] Add svndump_init_fd to allow reading dumps from
[PATCH/RFC v3 04/16] Connect fast-import to the remote-helper via
[PATCH/RFC v3 05/16] Add documentation for the 'bidi-import'
[PATCH/RFC v3 06/16] remote-svn, vcs-svn: Enable fetching to private
[PATCH/RFC v3 07/16] Add a symlink 'git-remote-svn' in base dir.
[PATCH/RFC v3 08/16] Allow reading svn dumps from files via file://
[PATCH/RFC v3 09/16] vcs-svn: add fast_export_note to create notes
[PATCH/RFC v3 10/16] Create a note for every imported commit
[PATCH/RFC v3 11/16] When debug==1, start fast-import with --stats
[PATCH/RFC v3 12/16] remote-svn: add incremental import.
[PATCH/RFC v3 13/16] Add a svnrdump-simulator replaying a dump file
[PATCH/RFC v3 14/16] transport-helper: add import|export-marks to
[PATCH/RFC v3 15/16] remote-svn: add marks-file regeneration.
[PATCH/RFC v3 16/16] Add a test script for remote-svn.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH/RFC v3 01/16] Implement a remote helper for svn in C.

2012-08-14 Thread Florian Achleitner
Enable basic fetching from subversion repositories. When processing remote URLs
starting with svn::, git invokes this remote-helper.
It starts svnrdump to extract revisions from the subversion repository in the
'dump file format', and converts them to a git-fast-import stream using
the functions of vcs-svn/.

Imported refs are created in a private namespace at 
refs/svn/remote-name/master.
The revision history is imported linearly (no branch detection) and completely,
i.e. from revision 0 to HEAD.

The 'bidi-import' capability is used. The remote-helper expects data from
fast-import on its stdin. It buffers a batch of 'import' command lines
in a string_list before starting to process them.

Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
---
diff:
- incorporate review
- remove redundant strbuf_init
- add 'bidi-import' to capabilities
- buffer all lines of a command batch in string_list

 contrib/svn-fe/remote-svn.c |  183 +++
 1 file changed, 183 insertions(+)
 create mode 100644 contrib/svn-fe/remote-svn.c

diff --git a/contrib/svn-fe/remote-svn.c b/contrib/svn-fe/remote-svn.c
new file mode 100644
index 000..ce59344
--- /dev/null
+++ b/contrib/svn-fe/remote-svn.c
@@ -0,0 +1,183 @@
+
+#include cache.h
+#include remote.h
+#include strbuf.h
+#include url.h
+#include exec_cmd.h
+#include run-command.h
+#include svndump.h
+#include notes.h
+#include argv-array.h
+
+static const char *url;
+static const char *private_ref;
+static const char *remote_ref = refs/heads/master;
+
+static int cmd_capabilities(const char *line);
+static int cmd_import(const char *line);
+static int cmd_list(const char *line);
+
+typedef int (*input_command_handler)(const char *);
+struct input_command_entry {
+   const char *name;
+   input_command_handler fct;
+   unsigned char batchable;/* whether the command starts or is 
part of a batch */
+};
+
+static const struct input_command_entry input_command_list[] = {
+   { capabilities, cmd_capabilities, 0 },
+   { import, cmd_import, 1 },
+   { list, cmd_list, 0 },
+   { NULL, NULL }
+};
+
+static int cmd_capabilities(const char *line) {
+   printf(import\n);
+   printf(bidi-import\n);
+   printf(refspec %s:%s\n\n, remote_ref, private_ref);
+   fflush(stdout);
+   return 0;
+}
+
+static void terminate_batch(void)
+{
+   /* terminate a current batch's fast-import stream */
+   printf(done\n);
+   fflush(stdout);
+}
+
+static int cmd_import(const char *line)
+{
+   int code;
+   int dumpin_fd;
+   unsigned int startrev = 0;
+   struct argv_array svndump_argv = ARGV_ARRAY_INIT;
+   struct child_process svndump_proc;
+
+   memset(svndump_proc, 0, sizeof (struct child_process));
+   svndump_proc.out = -1;
+   argv_array_push(svndump_argv, svnrdump);
+   argv_array_push(svndump_argv, dump);
+   argv_array_push(svndump_argv, url);
+   argv_array_pushf(svndump_argv, -r%u:HEAD, startrev);
+   svndump_proc.argv = svndump_argv.argv;
+
+   code = start_command(svndump_proc);
+   if (code)
+   die(Unable to start %s, code %d, svndump_proc.argv[0], code);
+   dumpin_fd = svndump_proc.out;
+
+   code = start_command(svndump_proc);
+   if (code)
+   die(Unable to start %s, code %d, svndump_proc.argv[0], code);
+   dumpin_fd = svndump_proc.out;
+
+   svndump_init_fd(dumpin_fd, STDIN_FILENO);
+   svndump_read(url, private_ref);
+   svndump_deinit();
+   svndump_reset();
+
+   close(dumpin_fd);
+   code = finish_command(svndump_proc);
+   if (code)
+   warning(%s, returned %d, svndump_proc.argv[0], code);
+   argv_array_clear(svndump_argv);
+
+   return 0;
+}
+
+static int cmd_list(const char *line)
+{
+   printf(? %s\n\n, remote_ref);
+   fflush(stdout);
+   return 0;
+}
+
+static int do_command(struct strbuf *line)
+{
+   const struct input_command_entry *p = input_command_list;
+   static struct string_list batchlines = STRING_LIST_INIT_DUP;
+   static const struct input_command_entry *batch_cmd;
+   /*
+* commands can be grouped together in a batch.
+* Batches are ended by \n. If no batch is active the program ends.
+* During a batch all lines are buffered and passed to the handler 
function
+* when the batch is terminated.
+*/
+   if (line-len == 0) {
+   if (batch_cmd) {
+   struct string_list_item *item;
+   for_each_string_list_item(item, batchlines)
+   batch_cmd-fct(item-string);
+   terminate_batch();
+   batch_cmd = NULL;
+   string_list_clear(batchlines, 0);
+   return 0;   /* end of the batch, continue reading 
other commands. */
+ 

[PATCH/RFC v3 03/16] Add svndump_init_fd to allow reading dumps from arbitrary FDs.

2012-08-14 Thread Florian Achleitner
The existing function only allows reading from a filename or
from stdin. Allow passing of a FD and an additional FD for
the back report pipe. This allows us to retrieve the name of
the pipe in the caller.

Fixes the filename could be NULL bug.

Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
---
- dup input file descriptor, because buffer_deinit closes the fd.
 vcs-svn/svndump.c |   22 ++
 vcs-svn/svndump.h |1 +
 2 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c
index 2b168ae..d81a078 100644
--- a/vcs-svn/svndump.c
+++ b/vcs-svn/svndump.c
@@ -468,11 +468,9 @@ void svndump_read(const char *url)
end_revision();
 }
 
-int svndump_init(const char *filename)
+static void init(int report_fd)
 {
-   if (buffer_init(input, filename))
-   return error(cannot open %s: %s, filename, strerror(errno));
-   fast_export_init(REPORT_FILENO);
+   fast_export_init(report_fd);
strbuf_init(dump_ctx.uuid, 4096);
strbuf_init(dump_ctx.url, 4096);
strbuf_init(rev_ctx.log, 4096);
@@ -482,6 +480,22 @@ int svndump_init(const char *filename)
reset_dump_ctx(NULL);
reset_rev_ctx(0);
reset_node_ctx(NULL);
+   return;
+}
+
+int svndump_init(const char *filename)
+{
+   if (buffer_init(input, filename))
+   return error(cannot open %s: %s, filename ? filename : 
NULL, strerror(errno));
+   init(REPORT_FILENO);
+   return 0;
+}
+
+int svndump_init_fd(int in_fd, int back_fd)
+{
+   if(buffer_fdinit(input, xdup(in_fd)))
+   return error(cannot open fd %d: %s, in_fd, strerror(errno));
+   init(xdup(back_fd));
return 0;
 }
 
diff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h
index df9ceb0..acb5b47 100644
--- a/vcs-svn/svndump.h
+++ b/vcs-svn/svndump.h
@@ -2,6 +2,7 @@
 #define SVNDUMP_H_
 
 int svndump_init(const char *filename);
+int svndump_init_fd(int in_fd, int back_fd);
 void svndump_read(const char *url);
 void svndump_deinit(void);
 void svndump_reset(void);
-- 
1.7.9.5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH/RFC v3 05/16] Add documentation for the 'bidi-import' capability of remote-helpers.

2012-08-14 Thread Florian Achleitner
Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
---
 Documentation/git-remote-helpers.txt |   21 -
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-remote-helpers.txt 
b/Documentation/git-remote-helpers.txt
index f5836e4..5faa48e 100644
--- a/Documentation/git-remote-helpers.txt
+++ b/Documentation/git-remote-helpers.txt
@@ -98,6 +98,20 @@ advertised with this capability must cover all refs reported 
by
 the list command.  If no 'refspec' capability is advertised,
 there is an implied `refspec *:*`.
 
+'bidi-import'::
+   The fast-import commands 'cat-blob' and 'ls' can be used by 
remote-helpers
+to retrieve information about blobs and trees that already exist in
+fast-import's memory. This requires a channel from fast-import to the
+remote-helper.
+If it is advertised in addition to import, git establishes a pipe from
+   fast-import to the remote-helper's stdin.
+   It follows that git and fast-import are both connected to the
+   remote-helper's stdin. Because git can send multiple commands to
+   the remote-helper it is required that helpers that use 'bidi-import'
+   buffer all 'import' commands of a batch before sending data to 
fast-import.
+This is to prevent mixing commands and fast-import responses on the
+helper's stdin.
+
 Capabilities for Pushing
 
 'connect'::
@@ -286,7 +300,12 @@ terminated with a blank line. For each batch of 'import', 
the remote
 helper should produce a fast-import stream terminated by a 'done'
 command.
 +
-Supported if the helper has the import capability.
+Note that if the 'bidi-import' capability is used the complete batch
+sequence has to be buffered before starting to send data to fast-import
+to prevent mixing of commands and fast-import responses on the helper's
+stdin.
++
+Supported if the helper has the 'import' capability.
 
 'connect' service::
Connects to given service. Standard input and standard output
-- 
1.7.9.5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH/RFC v3 06/16] remote-svn, vcs-svn: Enable fetching to private refs.

2012-08-14 Thread Florian Achleitner
The reference to update by the fast-import stream is hard-coded.
When fetching from a remote the remote-helper shall update refs
in a private namespace, i.e. a private subdir of refs/.
This namespace is defined by the 'refspec' capability, that the
remote-helper advertises as a reply to the 'capablilities' command.

Extend svndump and fast-export to allow passing the target ref.
Update svn-fe to be compatible.

Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
---
- fix hard-coded ref in test-svn-fe.c. Broke a testcase.

 contrib/svn-fe/svn-fe.c |2 +-
 test-svn-fe.c   |2 +-
 vcs-svn/fast_export.c   |4 ++--
 vcs-svn/fast_export.h   |2 +-
 vcs-svn/svndump.c   |   14 +++---
 vcs-svn/svndump.h   |2 +-
 6 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c
index 35db24f..c796cc0 100644
--- a/contrib/svn-fe/svn-fe.c
+++ b/contrib/svn-fe/svn-fe.c
@@ -10,7 +10,7 @@ int main(int argc, char **argv)
 {
if (svndump_init(NULL))
return 1;
-   svndump_read((argc  1) ? argv[1] : NULL);
+   svndump_read((argc  1) ? argv[1] : NULL, refs/heads/master);
svndump_deinit();
svndump_reset();
return 0;
diff --git a/test-svn-fe.c b/test-svn-fe.c
index 83633a2..cb0d80f 100644
--- a/test-svn-fe.c
+++ b/test-svn-fe.c
@@ -40,7 +40,7 @@ int main(int argc, char *argv[])
if (argc == 2) {
if (svndump_init(argv[1]))
return 1;
-   svndump_read(NULL);
+   svndump_read(NULL, refs/heads/master);
svndump_deinit();
svndump_reset();
return 0;
diff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c
index 1f04697..11f8f94 100644
--- a/vcs-svn/fast_export.c
+++ b/vcs-svn/fast_export.c
@@ -72,7 +72,7 @@ static char gitsvnline[MAX_GITSVN_LINE_LEN];
 void fast_export_begin_commit(uint32_t revision, const char *author,
const struct strbuf *log,
const char *uuid, const char *url,
-   unsigned long timestamp)
+   unsigned long timestamp, const char *local_ref)
 {
static const struct strbuf empty = STRBUF_INIT;
if (!log)
@@ -84,7 +84,7 @@ void fast_export_begin_commit(uint32_t revision, const char 
*author,
} else {
*gitsvnline = '\0';
}
-   printf(commit refs/heads/master\n);
+   printf(commit %s\n, local_ref);
printf(mark :%PRIu32\n, revision);
printf(committer %s %s@%s %ld +\n,
   *author ? author : nobody,
diff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h
index 8823aca..17eb13b 100644
--- a/vcs-svn/fast_export.h
+++ b/vcs-svn/fast_export.h
@@ -11,7 +11,7 @@ void fast_export_delete(const char *path);
 void fast_export_modify(const char *path, uint32_t mode, const char *dataref);
 void fast_export_begin_commit(uint32_t revision, const char *author,
const struct strbuf *log, const char *uuid,
-   const char *url, unsigned long timestamp);
+   const char *url, unsigned long timestamp, const char 
*local_ref);
 void fast_export_end_commit(uint32_t revision);
 void fast_export_data(uint32_t mode, off_t len, struct line_buffer *input);
 void fast_export_blob_delta(uint32_t mode,
diff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c
index d81a078..288bb42 100644
--- a/vcs-svn/svndump.c
+++ b/vcs-svn/svndump.c
@@ -299,22 +299,22 @@ static void handle_node(void)
node_ctx.text_length, input);
 }
 
-static void begin_revision(void)
+static void begin_revision(const char *remote_ref)
 {
if (!rev_ctx.revision)  /* revision 0 gets no git commit. */
return;
fast_export_begin_commit(rev_ctx.revision, rev_ctx.author.buf,
rev_ctx.log, dump_ctx.uuid.buf, dump_ctx.url.buf,
-   rev_ctx.timestamp);
+   rev_ctx.timestamp, remote_ref);
 }
 
-static void end_revision(void)
+static void end_revision()
 {
if (rev_ctx.revision)
fast_export_end_commit(rev_ctx.revision);
 }
 
-void svndump_read(const char *url)
+void svndump_read(const char *url, const char *local_ref)
 {
char *val;
char *t;
@@ -353,7 +353,7 @@ void svndump_read(const char *url)
if (active_ctx == NODE_CTX)
handle_node();
if (active_ctx == REV_CTX)
-   begin_revision();
+   begin_revision(local_ref);
if (active_ctx != DUMP_CTX)
end_revision();
active_ctx = REV_CTX;
@@ -366,7 +366,7 @@ void svndump_read(const char *url)
if (active_ctx == NODE_CTX)

[PATCH/RFC v3 08/16] Allow reading svn dumps from files via file:// urls.

2012-08-14 Thread Florian Achleitner
For testing as well as for importing large, already
available dumps, it's useful to bypass svnrdump and
replay the svndump from a file directly.

Add support for file:// urls in the remote url.
e.g. svn::file:///path/to/dump
When the remote helper finds an url starting with
file:// it tries to open that file instead of invoking svnrdump.

Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
---
 contrib/svn-fe/remote-svn.c |   59 ++-
 1 file changed, 36 insertions(+), 23 deletions(-)

diff --git a/contrib/svn-fe/remote-svn.c b/contrib/svn-fe/remote-svn.c
index ce59344..df1babc 100644
--- a/contrib/svn-fe/remote-svn.c
+++ b/contrib/svn-fe/remote-svn.c
@@ -10,6 +10,7 @@
 #include argv-array.h
 
 static const char *url;
+static int dump_from_file;
 static const char *private_ref;
 static const char *remote_ref = refs/heads/master;
 
@@ -54,34 +55,39 @@ static int cmd_import(const char *line)
struct argv_array svndump_argv = ARGV_ARRAY_INIT;
struct child_process svndump_proc;
 
-   memset(svndump_proc, 0, sizeof (struct child_process));
-   svndump_proc.out = -1;
-   argv_array_push(svndump_argv, svnrdump);
-   argv_array_push(svndump_argv, dump);
-   argv_array_push(svndump_argv, url);
-   argv_array_pushf(svndump_argv, -r%u:HEAD, startrev);
-   svndump_proc.argv = svndump_argv.argv;
-
-   code = start_command(svndump_proc);
-   if (code)
-   die(Unable to start %s, code %d, svndump_proc.argv[0], code);
-   dumpin_fd = svndump_proc.out;
-
-   code = start_command(svndump_proc);
-   if (code)
-   die(Unable to start %s, code %d, svndump_proc.argv[0], code);
-   dumpin_fd = svndump_proc.out;
+   if(dump_from_file) {
+   dumpin_fd = open(url, O_RDONLY);
+   if(dumpin_fd  0) {
+   die_errno(Couldn't open svn dump file %s., url);
+   }
+   }
+   else {
+   memset(svndump_proc, 0, sizeof (struct child_process));
+   svndump_proc.out = -1;
+   argv_array_push(svndump_argv, svnrdump);
+   argv_array_push(svndump_argv, dump);
+   argv_array_push(svndump_argv, url);
+   argv_array_pushf(svndump_argv, -r%u:HEAD, startrev);
+   svndump_proc.argv = svndump_argv.argv;
+
+   code = start_command(svndump_proc);
+   if (code)
+   die(Unable to start %s, code %d, 
svndump_proc.argv[0], code);
+   dumpin_fd = svndump_proc.out;
 
+   }
svndump_init_fd(dumpin_fd, STDIN_FILENO);
svndump_read(url, private_ref);
svndump_deinit();
svndump_reset();
 
close(dumpin_fd);
-   code = finish_command(svndump_proc);
-   if (code)
-   warning(%s, returned %d, svndump_proc.argv[0], code);
-   argv_array_clear(svndump_argv);
+   if(!dump_from_file) {
+   code = finish_command(svndump_proc);
+   if (code)
+   warning(%s, returned %d, svndump_proc.argv[0], code);
+   argv_array_clear(svndump_argv);
+   }
 
return 0;
 }
@@ -158,8 +164,15 @@ int main(int argc, const char **argv)
if (argc == 3)
url_in = argv[2];
 
-   end_url_with_slash(buf, url_in);
-   url = strbuf_detach(buf, NULL);
+   if (!prefixcmp(url_in, file://)) {
+   dump_from_file = 1;
+   url = url_decode(url_in + sizeof(file://)-1);
+   }
+   else {
+   dump_from_file = 0;
+   end_url_with_slash(buf, url_in);
+   url = strbuf_detach(buf, NULL);
+   }
 
strbuf_addf(buf, refs/svn/%s/master, remote-name);
private_ref = strbuf_detach(buf, NULL);
-- 
1.7.9.5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH/RFC v3 10/16] Create a note for every imported commit containing svn metadata.

2012-08-14 Thread Florian Achleitner
To provide metadata from svn dumps for further processing, e.g.
branch detection, attach a note to each imported commit that
stores additional information.
The notes are currently hard-coded in refs/notes/svn/revs.
Currently the following lines from the svn dump are directly
accumulated in the note. This can be refined on purpose, of course.
- Revision-number
- Node-path
- Node-kind
- Node-action
- Node-copyfrom-path
- Node-copyfrom-rev

Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
---
 vcs-svn/fast_export.c |   13 +
 vcs-svn/fast_export.h |2 ++
 vcs-svn/svndump.c |   21 +++--
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/vcs-svn/fast_export.c b/vcs-svn/fast_export.c
index 1ecae4b..796dd1a 100644
--- a/vcs-svn/fast_export.c
+++ b/vcs-svn/fast_export.c
@@ -12,6 +12,7 @@
 #include svndiff.h
 #include sliding_window.h
 #include line_buffer.h
+#include cache.h
 
 #define MAX_GITSVN_LINE_LEN 4096
 
@@ -68,6 +69,18 @@ void fast_export_modify(const char *path, uint32_t mode, 
const char *dataref)
putchar('\n');
 }
 
+void fast_export_begin_note(uint32_t revision, const char *author,
+   const char *log, unsigned long timestamp)
+{
+   timestamp = 1341914616;
+   size_t loglen = strlen(log);
+   printf(commit refs/notes/svn/revs\n);
+   printf(committer %s %s@%s %ld +\n, author, author, local, 
timestamp);
+   printf(data %PRIuMAX\n, loglen);
+   fwrite(log, loglen, 1, stdout);
+   fputc('\n', stdout);
+}
+
 void fast_export_note(const char *committish, const char *dataref)
 {
printf(N %s %s\n, dataref, committish);
diff --git a/vcs-svn/fast_export.h b/vcs-svn/fast_export.h
index 9b32f1e..c2f6f11 100644
--- a/vcs-svn/fast_export.h
+++ b/vcs-svn/fast_export.h
@@ -10,6 +10,8 @@ void fast_export_deinit(void);
 void fast_export_delete(const char *path);
 void fast_export_modify(const char *path, uint32_t mode, const char *dataref);
 void fast_export_note(const char *committish, const char *dataref);
+void fast_export_begin_note(uint32_t revision, const char *author,
+   const char *log, unsigned long timestamp);
 void fast_export_begin_commit(uint32_t revision, const char *author,
const struct strbuf *log, const char *uuid,
const char *url, unsigned long timestamp, const char 
*local_ref);
diff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c
index 288bb42..cd65b51 100644
--- a/vcs-svn/svndump.c
+++ b/vcs-svn/svndump.c
@@ -48,7 +48,7 @@ static struct {
 static struct {
uint32_t revision;
unsigned long timestamp;
-   struct strbuf log, author;
+   struct strbuf log, author, note;
 } rev_ctx;
 
 static struct {
@@ -77,6 +77,7 @@ static void reset_rev_ctx(uint32_t revision)
rev_ctx.timestamp = 0;
strbuf_reset(rev_ctx.log);
strbuf_reset(rev_ctx.author);
+   strbuf_reset(rev_ctx.note);
 }
 
 static void reset_dump_ctx(const char *url)
@@ -310,8 +311,15 @@ static void begin_revision(const char *remote_ref)
 
 static void end_revision()
 {
-   if (rev_ctx.revision)
+   struct strbuf mark = STRBUF_INIT;
+   if (rev_ctx.revision) {
fast_export_end_commit(rev_ctx.revision);
+   fast_export_begin_note(rev_ctx.revision, remote-svn,
+   Note created by remote-svn., 
rev_ctx.timestamp);
+   strbuf_addf(mark, :%PRIu32, rev_ctx.revision);
+   fast_export_note(mark.buf, inline);
+   fast_export_buf_to_data(rev_ctx.note);
+   }
 }
 
 void svndump_read(const char *url, const char *local_ref)
@@ -358,6 +366,7 @@ void svndump_read(const char *url, const char *local_ref)
end_revision();
active_ctx = REV_CTX;
reset_rev_ctx(atoi(val));
+   strbuf_addf(rev_ctx.note, %s\n, t);
break;
case sizeof(Node-path):
if (constcmp(t, Node-))
@@ -369,10 +378,12 @@ void svndump_read(const char *url, const char *local_ref)
begin_revision(local_ref);
active_ctx = NODE_CTX;
reset_node_ctx(val);
+   strbuf_addf(rev_ctx.note, %s\n, t);
break;
}
if (constcmp(t + strlen(Node-), kind))
continue;
+   strbuf_addf(rev_ctx.note, %s\n, t);
if (!strcmp(val, dir))
node_ctx.type = REPO_MODE_DIR;
else if (!strcmp(val, file))
@@ -383,6 +394,7 @@ void svndump_read(const char *url, const char *local_ref)
case sizeof(Node-action):
if (constcmp(t, Node-action))
 

[PATCH/RFC v3 12/16] remote-svn: add incremental import.

2012-08-14 Thread Florian Achleitner
Search for a note attached to the ref to update and read it's
'Revision-number:'-line. Start import from the next svn revision.

If there is no next revision in the svn repo, svnrdump terminates
with a message on stderr an non-zero return value. This looks a
little weird, but there is no other way to know whether there is
a new revision in the svn repo.

On the start of an incremental import, the parent of the first commit
in the fast-import stream is set to the branch name to update. All
following commits specify their parent by a mark number. Previous
mark files are currently not reused.

Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
---
 contrib/svn-fe/remote-svn.c |   66 +--
 contrib/svn-fe/svn-fe.c |3 +-
 test-svn-fe.c   |2 +-
 vcs-svn/fast_export.c   |   16 ---
 vcs-svn/fast_export.h   |6 ++--
 vcs-svn/svndump.c   |   12 
 vcs-svn/svndump.h   |2 +-
 7 files changed, 89 insertions(+), 18 deletions(-)

diff --git a/contrib/svn-fe/remote-svn.c b/contrib/svn-fe/remote-svn.c
index df1babc..d659a0e 100644
--- a/contrib/svn-fe/remote-svn.c
+++ b/contrib/svn-fe/remote-svn.c
@@ -13,6 +13,8 @@ static const char *url;
 static int dump_from_file;
 static const char *private_ref;
 static const char *remote_ref = refs/heads/master;
+static const char *notes_ref;
+struct rev_note { unsigned int rev_nr; };
 
 static int cmd_capabilities(const char *line);
 static int cmd_import(const char *line);
@@ -47,14 +49,70 @@ static void terminate_batch(void)
fflush(stdout);
 }
 
+/* NOTE: 'ref' refers to a git reference, while 'rev' refers to a svn 
revision. */
+static char *read_ref_note(const unsigned char sha1[20]) {
+   const unsigned char *note_sha1;
+   char *msg = NULL;
+   unsigned long msglen;
+   enum object_type type;
+   init_notes(NULL, notes_ref, NULL, 0);
+   if( (note_sha1 = get_note(NULL, sha1)) == NULL ||
+   !(msg = read_sha1_file(note_sha1, type, msglen)) ||
+   !msglen || type != OBJ_BLOB) {
+   free(msg);
+   return NULL;
+   }
+   free_notes(NULL);
+   return msg;
+}
+
+static int parse_rev_note(const char *msg, struct rev_note *res) {
+   const char *key, *value, *end;
+   size_t len;
+   while(*msg) {
+   end = strchr(msg, '\n');
+   len = end ? end - msg : strlen(msg);
+
+   key = Revision-number: ;
+   if(!prefixcmp(msg, key)) {
+   long i;
+   value = msg + strlen(key);
+   i = atol(value);
+   if(i  0 || i  UINT32_MAX)
+   return 1;
+   res-rev_nr = i;
+   }
+   msg += len + 1;
+   }
+   return 0;
+}
+
 static int cmd_import(const char *line)
 {
int code;
int dumpin_fd;
-   unsigned int startrev = 0;
+   char *note_msg;
+   unsigned char head_sha1[20];
+   unsigned int startrev;
struct argv_array svndump_argv = ARGV_ARRAY_INIT;
struct child_process svndump_proc;
 
+   if(read_ref(private_ref, head_sha1))
+   startrev = 0;
+   else {
+   note_msg = read_ref_note(head_sha1);
+   if(note_msg == NULL) {
+   warning(No note found for %s., private_ref);
+   startrev = 0;
+   }
+   else {
+   struct rev_note note = { 0 };
+   parse_rev_note(note_msg, note);
+   startrev = note.rev_nr + 1;
+   free(note_msg);
+   }
+   }
+
if(dump_from_file) {
dumpin_fd = open(url, O_RDONLY);
if(dumpin_fd  0) {
@@ -77,7 +135,7 @@ static int cmd_import(const char *line)
 
}
svndump_init_fd(dumpin_fd, STDIN_FILENO);
-   svndump_read(url, private_ref);
+   svndump_read(url, private_ref, notes_ref);
svndump_deinit();
svndump_reset();
 
@@ -177,6 +235,9 @@ int main(int argc, const char **argv)
strbuf_addf(buf, refs/svn/%s/master, remote-name);
private_ref = strbuf_detach(buf, NULL);
 
+   strbuf_addf(buf, refs/notes/%s/revs, remote-name);
+   notes_ref = strbuf_detach(buf, NULL);
+
while(1) {
if (strbuf_getline(buf, stdin, '\n') == EOF) {
if (ferror(stdin))
@@ -192,5 +253,6 @@ int main(int argc, const char **argv)
strbuf_release(buf);
free((void*)url);
free((void*)private_ref);
+   free((void*)notes_ref);
return 0;
 }
diff --git a/contrib/svn-fe/svn-fe.c b/contrib/svn-fe/svn-fe.c
index c796cc0..f363505 100644
--- a/contrib/svn-fe/svn-fe.c
+++ b/contrib/svn-fe/svn-fe.c
@@ -10,7 +10,8 @@ int main(int argc, char **argv)
 {
  

[PATCH/RFC v3 16/16] Add a test script for remote-svn.

2012-08-14 Thread Florian Achleitner
Use svnrdump_sim.py to emulate svnrdump without an svn server.
Tests fetching, incremental fetching, fetching from file://,
and the regeneration of fast-import's marks file.

Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
---
 t/t9020-remote-svn.sh |   69 +
 transport-helper.c|   15 ++-
 2 files changed, 77 insertions(+), 7 deletions(-)
 create mode 100755 t/t9020-remote-svn.sh

diff --git a/t/t9020-remote-svn.sh b/t/t9020-remote-svn.sh
new file mode 100755
index 000..a0c6a21
--- /dev/null
+++ b/t/t9020-remote-svn.sh
@@ -0,0 +1,69 @@
+#!/bin/sh
+
+test_description='tests remote-svn'
+
+. ./test-lib.sh
+
+# We override svnrdump by placing a symlink to the svnrdump-emulator in .
+export PATH=$HOME:$PATH
+ln -sf $GIT_BUILD_DIR/contrib/svn-fe/svnrdump_sim.py $HOME/svnrdump
+
+init_git () {
+   rm -fr .git 
+   git init 
+   #git remote add svnsim 
svn::sim:///$TEST_DIRECTORY/t9020/example.svnrdump
+   # let's reuse an exisiting dump file!?
+   git remote add svnsim svn::sim:///$TEST_DIRECTORY/t9154/svn.dump
+   git remote add svnfile svn::file:///$TEST_DIRECTORY/t9154/svn.dump
+}
+
+test_debug '
+   git --version
+   which git
+   which svnrdump
+'
+
+test_expect_success 'simple fetch' '
+   init_git 
+   git fetch svnsim 
+   test_cmp .git/refs/svn/svnsim/master .git/refs/remotes/svnsim/master  
+   cp .git/refs/remotes/svnsim/master master.good
+'
+
+test_debug '
+   cat .git/refs/svn/svnsim/master
+   cat .git/refs/remotes/svnsim/master
+'
+
+test_expect_success 'repeated fetch, nothing shall change' '
+   git fetch svnsim 
+   test_cmp master.good .git/refs/remotes/svnsim/master
+'
+
+test_expect_success 'fetch from a file:// url gives the same result' '
+   git fetch svnfile 
+'
+
+test_expect_failure 'the sha1 differ because the git-svn-id line in the commit 
msg contains the url' '
+   test_cmp .git/refs/remotes/svnfile/master 
.git/refs/remotes/svnsim/master
+'
+
+test_expect_success 'mark-file regeneration' '
+   mv .git/info/fast-import/marks/svnsim 
.git/info/fast-import/marks/svnsim.old 
+   git fetch svnsim 
+   test_cmp .git/info/fast-import/marks/svnsim.old 
.git/info/fast-import/marks/svnsim
+'
+
+test_expect_success 'incremental imports must lead to the same head' '
+   export SVNRMAX=3 
+   init_git 
+   git fetch svnsim 
+   test_cmp .git/refs/svn/svnsim/master .git/refs/remotes/svnsim/master  
+   unset SVNRMAX 
+   git fetch svnsim 
+   test_cmp master.good .git/refs/remotes/svnsim/master
+'
+
+test_debug 'git branch -a' 
+
+test_done
diff --git a/transport-helper.c b/transport-helper.c
index 47db055..a363f2c 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -17,6 +17,7 @@ static int debug;
 struct helper_data {
const char *name;
struct child_process *helper;
+   struct argv_array argv;
FILE *out;
unsigned fetch : 1,
import : 1,
@@ -103,7 +104,6 @@ static void do_take_over(struct transport *transport)
 static struct child_process *get_helper(struct transport *transport)
 {
struct helper_data *data = transport-data;
-   struct argv_array argv = ARGV_ARRAY_INIT;
struct strbuf buf = STRBUF_INIT;
struct child_process *helper;
const char **refspecs = NULL;
@@ -125,10 +125,11 @@ static struct child_process *get_helper(struct transport 
*transport)
helper-in = -1;
helper-out = -1;
helper-err = 0;
-   argv_array_pushf(argv, git-remote-%s, data-name);
-   argv_array_push(argv, transport-remote-name);
-   argv_array_push(argv, remove_ext_force(transport-url));
-   helper-argv = argv.argv;
+   argv_array_init(data-argv);
+   argv_array_pushf(data-argv, git-remote-%s, data-name);
+   argv_array_push(data-argv, transport-remote-name);
+   argv_array_push(data-argv, remove_ext_force(transport-url));
+   helper-argv = data-argv.argv;
helper-git_cmd = 0;
helper-silent_exec_failure = 1;
 
@@ -143,8 +144,6 @@ static struct child_process *get_helper(struct transport 
*transport)
 
data-helper = helper;
data-no_disconnect_req = 0;
-   free((void*) helper_env[1]);
-   argv_array_clear(argv);
 
/*
 * Open the output as FILE* so strbuf_getline() can be used.
@@ -247,6 +246,8 @@ static int disconnect_helper(struct transport *transport)
close(data-helper-out);
fclose(data-out);
res = finish_command(data-helper);
+   free((void*) data-helper-env[1]);
+   argv_array_clear(data-argv);
free(data-helper);
data-helper = NULL;
}
-- 
1.7.9.5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  

Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread Thomas Rast
Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 On 14 August 2012 10:19, Junio C Hamano gits...@pobox.com wrote:
 Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 On 14 August 2012 01:27, Thomas Rast tr...@student.ethz.ch wrote:
 [git pull with two args] it's ok if you use it with an URL instead
 of a remote nickname

 Why would that be okay? What is the difference? Isn't the nickname
 just an alias for a URL?

 As long as you tell what refspecs to use on the command line, the
 remote nickname behaves as just an alias for a URL, so yes,
 because Thomas is discussing two-arg pull or fetch, one arg being
 either nickname or URL and the other is an explicit refspec on the
 command line, it would be okay because there is no difference in
 that case.

 I suppose I'm not entirely clear on how this two step process is
 safer. Doing git fetch would seem to be harmless, right? So the
 problem is with git merge but master should always be behind
 origin/master so that git merge should just FF to origin/master
 which *should* be completely safe. Does that make sense? Especially
 given our use of master as an integration branch?

 [Given the trouble I have with getting people to use Git properly, I
 prefer things as simple as possible. :-) ]

I meant something else than Junio hinted at.  Saying

  git fetch origin master
  # or by extension
  git pull origin master

does not update the origin/* namespace, not even origin/master.  All
fetching happens only into FETCH_HEAD.  This leads to confusion such as
yours because origin/master and thus the upstream tracking displays will
not know about the change.

If you use it with an URL, such as one that might be sent with a pull
request:

} The following changes since commit 62bc83349d52be49b037d2c800a7f4064cfbc5b5:
} 
}   The sixth batch of topics graduated to 'master' (2012-04-27 14:12:56 -0700)
} 
} are available in the git repository at:
} 
}   https://github.com/git-l10n/git-po/ master

(I picked a random pull request from the l10n coordinator, Jiang Xin)
you would say

  git pull https://github.com/git-l10n/git-po/ master

and you would not have a reasonable expectation of git updating your
remotes/*, even if you had a remote 'l10n' that points at that exact
URL.  So you would not be confused if you pulled from there, but
l10n/master didn't move.


In some sense this is a really bad case of wrong UI design, because we
(this happens on #git a lot) have to teach users not to use the command
so they won't trip over this problem.  It would be better to fix the
real issue instead.  IIRC it was even on the 1.8.0 wishlist...

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH/RFC v3 02/16] Integrate remote-svn into svn-fe/Makefile.

2012-08-14 Thread Junio C Hamano
Florian Achleitner florian.achleitner.2.6...@gmail.com writes:

 Requires some sha.h to be used and the libraries
 to be linked, this is currently hardcoded.

 Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
 ---
  contrib/svn-fe/Makefile |   16 ++--
  1 file changed, 10 insertions(+), 6 deletions(-)

 diff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile
 index 360d8da..8f0eec2 100644
 --- a/contrib/svn-fe/Makefile
 +++ b/contrib/svn-fe/Makefile
 @@ -1,14 +1,14 @@
 -all:: svn-fe$X
 +all:: svn-fe$X remote-svn$X
  
  CC = gcc
  RM = rm -f
  MV = mv
  
 -CFLAGS = -g -O2 -Wall
 +CFLAGS = -g -O2 -Wall -DSHA1_HEADER='openssl/sha.h' 
 -Wdeclaration-after-statement
  LDFLAGS =
  ALL_CFLAGS = $(CFLAGS)
  ALL_LDFLAGS = $(LDFLAGS)
 -EXTLIBS =
 +EXTLIBS = -lssl -lcrypto -lpthread ../../xdiff/lib.a

I haven't looked carefully, but didn't we have to do a bit more
elaborate when linking with ssl/crypto in our main Makefile to be
portable across various vintages of OpenSSL libraries?

Does contrib/svn-fe/ already depend on OpenSSL by the way?  It needs
to be documented somewhere in the same directory.

If one builds the main Git binary with NO_OPENSSL, can this still be
built and linked?

What does this use xdiff/lib.a for?

The above are just mental notes; I didn't read the later patches in
the series that may already address these issues.

  GIT_LIB = ../../libgit.a
  VCSSVN_LIB = ../../vcs-svn/lib.a
 @@ -37,8 +37,12 @@ svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)
   $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ svn-fe.o \
   $(ALL_LDFLAGS) $(LIBS)
  
 -svn-fe.o: svn-fe.c ../../vcs-svn/svndump.h
 - $(QUIET_CC)$(CC) -I../../vcs-svn -o $*.o -c $(ALL_CFLAGS) $
 +remote-svn$X: remote-svn.o $(VCSSVN_LIB) $(GIT_LIB)
 + $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ remote-svn.o \
 + $(ALL_LDFLAGS) $(LIBS)
 + 
 +%.o: %.c ../../vcs-svn/svndump.h
 + $(QUIET_CC)$(CC) -I../../vcs-svn -I../../ -o $*.o -c $(ALL_CFLAGS) $
  
  svn-fe.html: svn-fe.txt
   $(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \
 @@ -58,6 +62,6 @@ svn-fe.1: svn-fe.txt
   $(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) libgit.a
  
  clean:
 - $(RM) svn-fe$X svn-fe.o svn-fe.html svn-fe.xml svn-fe.1
 + $(RM) svn-fe$X svn-fe.o svn-fe.html svn-fe.xml svn-fe.1 remote-svn.o
  
  .PHONY: all clean FORCE
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH/RFC v3 03/16] Add svndump_init_fd to allow reading dumps from arbitrary FDs.

2012-08-14 Thread Junio C Hamano
Florian Achleitner florian.achleitner.2.6...@gmail.com writes:

 The existing function only allows reading from a filename or
 from stdin. Allow passing of a FD and an additional FD for
 the back report pipe. This allows us to retrieve the name of
 the pipe in the caller.

 Fixes the filename could be NULL bug.

What bug?  Was this line meant to go in the log message?

 Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
 ---
 - dup input file descriptor, because buffer_deinit closes the fd.
  vcs-svn/svndump.c |   22 ++
  vcs-svn/svndump.h |1 +
  2 files changed, 19 insertions(+), 4 deletions(-)

 diff --git a/vcs-svn/svndump.c b/vcs-svn/svndump.c
 index 2b168ae..d81a078 100644
 --- a/vcs-svn/svndump.c
 +++ b/vcs-svn/svndump.c
 @@ -468,11 +468,9 @@ void svndump_read(const char *url)
   end_revision();
  }
  
 -int svndump_init(const char *filename)
 +static void init(int report_fd)
  {
 - if (buffer_init(input, filename))
 - return error(cannot open %s: %s, filename, strerror(errno));
 - fast_export_init(REPORT_FILENO);
 + fast_export_init(report_fd);
   strbuf_init(dump_ctx.uuid, 4096);
   strbuf_init(dump_ctx.url, 4096);
   strbuf_init(rev_ctx.log, 4096);
 @@ -482,6 +480,22 @@ int svndump_init(const char *filename)
   reset_dump_ctx(NULL);
   reset_rev_ctx(0);
   reset_node_ctx(NULL);
 + return;
 +}
 +
 +int svndump_init(const char *filename)
 +{
 + if (buffer_init(input, filename))
 + return error(cannot open %s: %s, filename ? filename : 
 NULL, strerror(errno));
 + init(REPORT_FILENO);
 + return 0;
 +}
 +
 +int svndump_init_fd(int in_fd, int back_fd)
 +{
 + if(buffer_fdinit(input, xdup(in_fd)))
 + return error(cannot open fd %d: %s, in_fd, strerror(errno));
 + init(xdup(back_fd));
   return 0;
  }
  
 diff --git a/vcs-svn/svndump.h b/vcs-svn/svndump.h
 index df9ceb0..acb5b47 100644
 --- a/vcs-svn/svndump.h
 +++ b/vcs-svn/svndump.h
 @@ -2,6 +2,7 @@
  #define SVNDUMP_H_
  
  int svndump_init(const char *filename);
 +int svndump_init_fd(int in_fd, int back_fd);
  void svndump_read(const char *url);
  void svndump_deinit(void);
  void svndump_reset(void);
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH/RFC v3 07/16] Add a symlink 'git-remote-svn' in base dir.

2012-08-14 Thread Junio C Hamano
Florian Achleitner florian.achleitner.2.6...@gmail.com writes:

 Allow execution of git-remote-svn even if the binary
 currently is located in contrib/svn-fe/.

 Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
 ---
  git-remote-svn |1 +
  1 file changed, 1 insertion(+)
  create mode 12 git-remote-svn

 diff --git a/git-remote-svn b/git-remote-svn
 new file mode 12
 index 000..d3b1c07
 --- /dev/null
 +++ b/git-remote-svn
 @@ -0,0 +1 @@
 +contrib/svn-fe/remote-svn
 \ No newline at end of file

Not paying enough attention to the patch you are sending?
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH/RFC v3 07/16] Add a symlink 'git-remote-svn' in base dir.

2012-08-14 Thread Junio C Hamano
Florian Achleitner florian.achleitner.2.6...@gmail.com writes:

 Allow execution of git-remote-svn even if the binary
 currently is located in contrib/svn-fe/.

 Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
 ---
  git-remote-svn |1 +
  1 file changed, 1 insertion(+)
  create mode 12 git-remote-svn

 diff --git a/git-remote-svn b/git-remote-svn
 new file mode 12
 index 000..d3b1c07
 --- /dev/null
 +++ b/git-remote-svn
 @@ -0,0 +1 @@
 +contrib/svn-fe/remote-svn
 \ No newline at end of file

Please scratch my previous comment.  I thought you were adding an
entry to .gitignore or something.

I'd rather not to see such a symbolic link that points at a build
product in the source tree.  Making a symlink from the toplevel
Makefile _after_ we built it in contrib/svn-fe/ (and removing it
upon make clean) is OK, though.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread Junio C Hamano
Thomas Rast tr...@student.ethz.ch writes:

 In some sense this is a really bad case of wrong UI design, because we
 (this happens on #git a lot) have to teach users not to use the command
 so they won't trip over this problem.  It would be better to fix the
 real issue instead.  IIRC it was even on the 1.8.0 wishlist...

Is it?

There already is a way to ask it to update the single tracking
branch while fetching; git fetch origin master that
unconditionally updates refs/remotes/origin/master without a way to
tell it not to do so will be a grave usability regression.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: send-email and in-reply-to = n

2012-08-14 Thread Martin von Zweigbergk
On Mon, Aug 13, 2012 at 4:53 PM, Junio C Hamano gits...@pobox.com wrote:
 Stephen Boyd bebar...@gmail.com writes:

 Can we throw up a big warning or just outright fail if someone types
 'n' or 'y' and hits enter for the in-reply-to question in
 git-send-email? I saw a git-send-email sent patch with an In-Reply-To
 header containing n on lkml today and it makes threading in my mail
 client get confused.

 Yeah, I think it is a good idea to minimally sanity check the answer
 to in-reply-to (and possibly other fields); perhaps does it have @
 and dot would be a good enough heuristics.

 Please make it so ;-)

And if you do, please include the check for the value for the From:
header in the and possibly other fields. I made the same mistake
when asked about that value just a few days ago.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v3] Let submodule command exit with error status if path does not exist

2012-08-14 Thread Junio C Hamano
Heiko Voigt hvo...@hvoigt.net writes:

 Previously the exit status of git submodule was zero for various
 subcommands even though the user specified an unknown path.

As any patch that fixes behaviour deals with Previously, I'd
prefer to omit it and describe the current problem in present tense
instead.

Will queue with minor tweaks.  Thanks.

 this is an updated version with your proposal incorporated. I changed
 the name of check_unmatched to die_if_unmatched because IMO it describes
 more clearly what the function is doing.

In general, I would actually prefer to call this kind of function
check.  That way, all the call sites only need to be aware that
there is a check done there, without having to know what happens
when the check triggers, and the implementation of check could
decide that dying is too much and weaken the behaviour to only warn
in later updates.

Such an update would not easily apply for this particular case
because you would need to spit out all of @out from the module_list
before giving the #unmatched warning token, and find a way to
buffer the error message from ls-files so that it can be given when
the warning is issued at the end if we wanted to weaken this to
warn.  So in this particular case, I do not mind renaming it to
die-if.

Thanks.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Document bash style we prefer for if and functions

2012-08-14 Thread Junio C Hamano
Heiko Voigt hvo...@hvoigt.net writes:

 During discussion of other patches these preferences have been revealed.
 Lets add them to the guidelines.

 Signed-off-by: Heiko Voigt hvo...@hvoigt.net
 ---
  Documentation/CodingGuidelines | 6 ++
  1 file changed, 6 insertions(+)

 diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines
 index 4557711..0a07990 100644
 --- a/Documentation/CodingGuidelines
 +++ b/Documentation/CodingGuidelines
 @@ -76,11 +76,16 @@ For shell scripts specifically (not exhaustive):
  
   - We do not use Process Substitution (list) or (list).
  
 + - We prefer then on the next line for if statements.
 +

This is not limited to if.  An easy rule of thumb is to write the
control structure _without_ using semicolon.

That is:

if test ...
then
...
else
...
fi

while test ...
do
...
done

not

if test ...; then
...
else ...;
fi

while test ...; do
...
done

   - We prefer test over [ ... ].
  
   - We do not write the noiseword function in front of shell
 functions.
  
 + - We prefer a space between the function name and the parentheses.
 +   E.g.: my_function ()
 +

Also opening { on the same line, i.e.

myfunc () {

   - As to use of grep, stick to a subset of BRE (namely, no \{m,n\},
 [::], [==], nor [..]) for portability.
  


 @@ -97,6 +102,7 @@ For shell scripts specifically (not exhaustive):
 interface translatable. See Marking strings for translation in
 po/README.
  
 +
  For C programs:

Probably not needed, as there is no such double space between C
and Documentation sections.

  
   - We use tabs to indent, and interpret tabs as taking up to
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


What's cooking in git.git (Aug 2012, #04; Tue, 14)

2012-08-14 Thread Junio C Hamano
Here are the topics that have been cooking.  Commits prefixed with '-' are
only in 'pu' (proposed updates) while commits prefixed with '+' are in 'next'.

Hopefully tomorrow's 1.7.12-rc3 will be the final rc before the real
thing.

You can find the changes described here in the integration branches of the
repositories listed at

http://git-blame.blogspot.com/p/git-public-repositories.html

--
[New Topics]

* hv/submodule-path-unmatch (2012-08-14) 1 commit
 - Let submodule command exit with error status if path does not exist

As the title says.

* jc/add-delete-default (2012-08-13) 1 commit
 - git add: notice removal of tracked paths by default

git add dir/ updated modified files and added new files, but does
not notice removed files, which may be Huh? to some users.  They
can of course use git add -A dir/, but why should they?

* jc/maint-rev-list-topo-doc (2012-08-13) 1 commit
 - rev-list docs: clarify --topo-order description

Further rewording suggestions by Martin and Thomas were good, but
haven't been incorporated into this patch.

--
[Stalled]

* mz/rebase-range (2012-07-18) 7 commits
 . rebase (without -p): correctly calculate patches to rebase
 . rebase -p: don't request --left-right only to ignore left side
 . rebase -p: use --cherry-mark for todo file
 . git-rebase--interactive.sh: look up subject in add_pick_line
 . git-rebase--interactive: group all $preserve_merges code
 . git-rebase--interactive.sh: extract function for adding pick line
 . git-rebase--am.sh: avoid special-casing --keep-empty

Expecting a reroll.

Performance concerns from Windows folks.  Also the series lacks
proper sign-offs.

* jl/submodule-rm (2012-07-05) 2 commits
 - rm: remove submodules from the index and the .gitmodules file
 - rm: don't fail when removing populated submodules

Expecting a reroll.

* ph/stash-rerere (2012-07-08) 2 commits
 - stash: invoke rerere in case of conflict
 - test: git-stash conflict sets up rerere

Will be rerolled but is going in the right direction.

* lt/commit-tree-guess-utf-8 (2012-06-28) 1 commit
 - commit/commit-tree: correct latin1 to utf-8

Teaches git commit and git commit-tree the we are told to use
utf-8 in log message, but this does not look like utf-8---attempt to
pass it through convert-from-latin1-to-utf8 and see if it makes
sense heuristics git mailinfo already uses.

A draft from Linus received privately without a log message.
Expecting a reroll.

* mb/remote-default-nn-origin (2012-07-11) 6 commits
 - Teach get_default_remote to respect remote.default.
 - Test that plain git fetch uses remote.default when on a detached HEAD.
 - Teach clone to set remote.default.
 - Teach git remote about remote.default.
 - Teach remote.c about the remote.default configuration setting.
 - Rename remote.c's default_remote_name static variables.

When the user does not specify what remote to interact with, we
often attempt to use 'origin'.  This can now be customized via a
configuration variable.

Expecting a reroll.
The first remote becomes the default bit is better done as a
separate step.

* jc/split-blob (2012-04-03) 6 commits
 - chunked-object: streaming checkout
 - chunked-object: fallback checkout codepaths
 - bulk-checkin: support chunked-object encoding
 - bulk-checkin: allow the same data to be multiply hashed
 - new representation types in the packstream
 - packfile: use varint functions

Not ready.

I finished the streaming checkout codepath, but as explained in
127b177 (bulk-checkin: support chunked-object encoding, 2011-11-30),
these are still early steps of a long and painful journey. At least
pack-objects and fsck need to learn the new encoding for the series
to be usable locally, and then index-pack/unpack-objects needs to
learn it to be used remotely.

Given that I heard a lot of noise that people want large files, and
that I was asked by somebody at GitTogether'11 privately for an
advice on how to pay developers (not me) to help adding necessary
support, I am somewhat dissapointed that the original patch series
that was sent long time ago still remains here without much comments
and updates from the developer community. I even made the interface
to the logic that decides where to split chunks easily replaceable,
and I deliberately made the logic in the original patch extremely
stupid to entice others, especially the bup fanbois, to come up
with a better logic, thinking that giving people an easy target to
shoot for, they may be encouraged to help out. The plan is not
working :-.

--
[Cooking]

* bc/receive-pack-stdout-protection (2012-08-06) 2 commits
  (merged to 'next' on 2012-08-07 at d7aa316)
 + receive-pack: do not leak output from auto-gc to standard output
 + t/t5400: demonstrate breakage caused by informational message from prune

When git push triggered the automatic gc on the receiving end, a
message from git prune 

Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread Hilco Wijbenga
On 14 August 2012 13:12, Thomas Rast tr...@student.ethz.ch wrote:
 Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 On 14 August 2012 10:19, Junio C Hamano gits...@pobox.com wrote:
 Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 On 14 August 2012 01:27, Thomas Rast tr...@student.ethz.ch wrote:
 [git pull with two args] it's ok if you use it with an URL instead
 of a remote nickname

 Why would that be okay? What is the difference? Isn't the nickname
 just an alias for a URL?

 As long as you tell what refspecs to use on the command line, the
 remote nickname behaves as just an alias for a URL, so yes,
 because Thomas is discussing two-arg pull or fetch, one arg being
 either nickname or URL and the other is an explicit refspec on the
 command line, it would be okay because there is no difference in
 that case.

 I suppose I'm not entirely clear on how this two step process is
 safer. Doing git fetch would seem to be harmless, right? So the
 problem is with git merge but master should always be behind
 origin/master so that git merge should just FF to origin/master
 which *should* be completely safe. Does that make sense? Especially
 given our use of master as an integration branch?

 [Given the trouble I have with getting people to use Git properly, I
 prefer things as simple as possible. :-) ]

 I meant something else than Junio hinted at.  Saying

   git fetch origin master
   # or by extension
   git pull origin master

 does not update the origin/* namespace, not even origin/master.  All
 fetching happens only into FETCH_HEAD.  This leads to confusion such as
 yours because origin/master and thus the upstream tracking displays will
 not know about the change.

I'll say. Now I'm really confused.

If what you say is true then what is updating origin/master? I've been
using git pull daily for over a year and origin/master is definitely
getting updated (at least according to gitk).

Mmm, just to make sure we are all talking about the same
origin/master: I mean my local reference to the SHA1 of the commit
that is master's HEAD on origin. After I have run git pull,  *my*
master and *my* origin/master point to the same commit. Or I'm
*really* confused. Or I've confused you by using incorrect
terminology. :-) Or by using the right terminology incorrectly. ;-)
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Your branch and 'origin/master' have diverged

2012-08-14 Thread Junio C Hamano
Hilco Wijbenga hilco.wijbe...@gmail.com writes:

 I meant something else than Junio hinted at.  Saying

   git fetch origin master
   # or by extension
   git pull origin master

 does not update the origin/* namespace, not even origin/master.  All
 fetching happens only into FETCH_HEAD.  This leads to confusion such as
 yours because origin/master and thus the upstream tracking displays will
 not know about the change.

 I'll say. Now I'm really confused.

 If what you say is true then what is updating origin/master? I've been
 using git pull daily for over a year and origin/master is definitely
 getting updated (at least according to gitk).

Now it is really the time for you to go back to git fetch --help
and read up on refspecs.

With

$ git fetch origin

you are not telling fetch what to fetch, so it goes to your .git/config
and finds remote.origin section to find what refspec to use.  They
would say something like

[remote origin]
url = ...
fetch = refs/heads/*:refs/remotes/origin/*

meaning (see the manual) fetch all the branches there, store them
with the corresponding name under  refs/remotes/origin.

With

$ git fetch origin master

you are overiding the refspec in .git/config and explicitly saying
I want to fetch the master branch, but do not want to update
anything with it.  It is a short-hand for

$ git fetch origin refs/heads/master

which in turn is a short-hand for

$ git fetch origin refs/heads/master:

If you wanted to update the tracking ref, you would use a refspec
with non-empty strings on the both sides of colon, i.e.

$ git fetch origin master:refs/remotes/origin/master

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] send-email: validate reconfirm interactive responses

2012-08-14 Thread Junio C Hamano
Martin von Zweigbergk martin.von.zweigbe...@gmail.com writes:

 On Tue, Aug 14, 2012 at 3:25 PM, Junio C Hamano gits...@pobox.com wrote:
 People answer 'y' to Who should the emails appear to be from?  and
 'n' to Message-ID to be used as In-Reply-To for the first email?
 for some unknown reason.

 Yeah, I know :-(. I did feel stupid already. Thanks for improving.

Actually, it is a very understandable mistake and I do not think it
is a user stupidity.  It is a UI bug in the prompter that gives:

  Who should the emails appear to be from? [Junio C Hamano gits...@pobox.com]

and does *not* tell the user that the way to accept the default is
to just press RETURN.  It makes it look as if it is asking Is it OK
to use this?, and it is a natural response to say Yes to the
prompt.

We would want to do something like the following pseudo-patch, I
think, but I do not know what is the best way to show both $prompt
and the press return suggestion to the user, so I am not going to
do this myself.

A tested patch to improve this is very much welcomed.

Thanks.

diff --git a/git-send-email.perl b/git-send-email.perl
index 607137b..2ec0ce8 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -688,6 +688,9 @@ sub ask {
unless defined $term-IN and defined fileno($term-IN) and
   defined $term-OUT and defined fileno($term-OUT);
while ($i++  10) {
+   if (defined $default) {
+   SAY (press RETURN to accept the default);
+   }
$resp = $term-readline($prompt);
if (!defined $resp) { # EOF
print \n;



--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH/RFC v3 00/16] GSOC remote-svn

2012-08-14 Thread David Michael Barr
On Wed, Aug 15, 2012 at 5:13 AM, Florian Achleitner
florian.achleitner.2.6...@gmail.com wrote:
 Hi.

 Version 3 of this series adds the 'bidi-import' capability, as suggested
 Jonathan.
 Diff details are attached to the patches.
 04 and 05 are completely new.

 [PATCH/RFC v3 01/16] Implement a remote helper for svn in C.
 [PATCH/RFC v3 02/16] Integrate remote-svn into svn-fe/Makefile.
 [PATCH/RFC v3 03/16] Add svndump_init_fd to allow reading dumps from
 [PATCH/RFC v3 04/16] Connect fast-import to the remote-helper via
 [PATCH/RFC v3 05/16] Add documentation for the 'bidi-import'
 [PATCH/RFC v3 06/16] remote-svn, vcs-svn: Enable fetching to private
 [PATCH/RFC v3 07/16] Add a symlink 'git-remote-svn' in base dir.
 [PATCH/RFC v3 08/16] Allow reading svn dumps from files via file://
 [PATCH/RFC v3 09/16] vcs-svn: add fast_export_note to create notes
 [PATCH/RFC v3 10/16] Create a note for every imported commit
 [PATCH/RFC v3 11/16] When debug==1, start fast-import with --stats
 [PATCH/RFC v3 12/16] remote-svn: add incremental import.
 [PATCH/RFC v3 13/16] Add a svnrdump-simulator replaying a dump file
 [PATCH/RFC v3 14/16] transport-helper: add import|export-marks to
 [PATCH/RFC v3 15/16] remote-svn: add marks-file regeneration.
 [PATCH/RFC v3 16/16] Add a test script for remote-svn.

Thank you Florian, this series was a great read. My apologies for the
limited interaction over the course of summer. You have done well and
engaged with the community to produce this result.

Thank you Jonathan for the persistent reviews. No doubt they have
contributed to the quality of the series.

Thank you Junio for your dedication to reviewing the traffic on this
mailing list.

I will no longer be reachable on this address after Friday.

I hope to make future contributions with the identity:
David Michael Barr b...@rr-dav.id.au
This will be my persistent address.

--
David Barr
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] daemon: --access-hook option

2012-08-14 Thread Junio C Hamano
The --access-hook option to git daemon specifies an external
command to be run every time a client connects, with

 - service name (e.g. upload-pack, etc.),
 - path to the repository,
 - hostname (%H),
 - canonical hostname (%CH),
 - ip address (%IP),
 - tcp port (%P)

as its command line arguments.  The external command can decide to
decline the service by exiting with a non-zero status (or to allow it
by exiting with a zero status).  It can also look at the $REMOTE_ADDR
and $REMOTE_PORT environment variables to learn about the requestor
when making this decision.

The external command can optionally write a single line to its
standard output to be sent to the requestor as an error message when
it declines the service.

Signed-off-by: Junio C Hamano gits...@pobox.com
---

 * This time, minimally tested, with a documentation update.

 Documentation/git-daemon.txt | 16 +
 daemon.c | 77 
 2 files changed, 93 insertions(+)

diff --git a/Documentation/git-daemon.txt b/Documentation/git-daemon.txt
index 31b28fc..c3ba4d7 100644
--- a/Documentation/git-daemon.txt
+++ b/Documentation/git-daemon.txt
@@ -16,6 +16,7 @@ SYNOPSIS
 [--reuseaddr] [--detach] [--pid-file=file]
 [--enable=service] [--disable=service]
 [--allow-override=service] [--forbid-override=service]
+[--access-hook=path]
 [--inetd | [--listen=host_or_ipaddr] [--port=n] [--user=user 
[--group=group]]
 [directory...]
 
@@ -171,6 +172,21 @@ the facility of inet daemon to achieve the same before 
spawning
errors are not enabled, all errors report access denied to the
client. The default is --no-informative-errors.
 
+--access-hook=path::
+   Every time a client connects, first run an external command
+   specified by the path with service name (e.g. upload-pack),
+   path to the repository, hostname (%H), canonical hostname
+   (%CH), ip address (%IP), and tcp port (%P) as its command line
+   arguments. The external command can decide to decline the
+   service by exiting with a non-zero status (or to allow it by
+   exiting with a zero status).  It can also look at the $REMOTE_ADDR
+   and $REMOTE_PORT environment variables to learn about the
+   requestor when making this decision.
++
+The external command can optionally write a single line to its
+standard output to be sent to the requestor as an error message when
+it declines the service.
+
 directory::
A directory to add to the whitelist of allowed directories. Unless
--strict-paths is specified this will also include subdirectories
diff --git a/daemon.c b/daemon.c
index ab21e66..4602b46 100644
--- a/daemon.c
+++ b/daemon.c
@@ -30,6 +30,7 @@ static const char daemon_usage[] =
[--interpolated-path=path]\n
[--reuseaddr] [--pid-file=file]\n
[--(enable|disable|allow-override|forbid-override)=service]\n
+   [--access-hook=path]\n
[--inetd | [--listen=host_or_ipaddr] [--port=n]\n
   [--detach] [--user=user [--group=group]]\n
[directory...];
@@ -256,6 +257,71 @@ static int daemon_error(const char *dir, const char *msg)
return -1;
 }
 
+static char *access_hook;
+
+static int run_access_hook(struct daemon_service *service, const char *dir, 
const char *path)
+{
+   struct child_process child;
+   struct strbuf buf = STRBUF_INIT;
+   const char *argv[8];
+   const char **arg = argv;
+   char *eol;
+   int seen_errors = 0;
+
+#define STRARG(x) ((x) ? (x) : )
+   *arg++ = access_hook;
+   *arg++ = service-name;
+   *arg++ = path;
+   *arg++ = STRARG(hostname);
+   *arg++ = STRARG(canon_hostname);
+   *arg++ = STRARG(ip_address);
+   *arg++ = STRARG(tcp_port);
+   *arg = NULL;
+#undef STRARG
+
+   memset(child, 0, sizeof(child));
+   child.use_shell = 1;
+   child.argv = argv;
+   child.no_stdin = 1;
+   child.no_stderr = 1;
+   child.out = -1;
+   if (start_command(child)) {
+   logerror(daemon access hook '%s' failed to start,
+access_hook);
+   goto error_return;
+   }
+   if (strbuf_read(buf, child.out, 0)  0) {
+   logerror(failed to read from pipe to daemon access hook '%s',
+access_hook);
+   strbuf_reset(buf);
+   seen_errors = 1;
+   }
+   if (close(child.out)  0) {
+   logerror(failed to close pipe to daemon access hook '%s',
+access_hook);
+   seen_errors = 1;
+   }
+   if (finish_command(child))
+   seen_errors = 1;
+
+   if (!seen_errors) {
+   strbuf_release(buf);
+   return 0;
+   }
+
+error_return:
+   strbuf_ltrim(buf);
+   if (!buf.len)
+   strbuf_addstr(buf, service