[PATCH v3 1/2] for-each-repo: new command used for multi-repo operations

2013-01-23 Thread Lars Hjemli
When working with multiple, unrelated (or loosly related) git repos,
there is often a need to locate all repos with uncommitted work and
perform some action on them (say, commit and push). Before this patch,
such tasks would require manually visiting all repositories, running
`git status` within each one and then decide what to do next.

This mundane task can now be automated by e.g. `git for-each-repo --dirty
status`, which will find all git repositories below the current directory
(even nested ones), check if they are dirty (as defined by `git diff --quiet
 git diff --cached --quiet`), and for each dirty repo print the path to
the repo and then execute `git status` within the repo.

The command also honours the option '--clean' which restricts the set of
repos to those which '--dirty' would skip.

Finally, the command to execute within each repo is optional. If none is
given, git-for-each-repo will just print the path to each repo found.

Signed-off-by: Lars Hjemli hje...@gmail.com
---
 .gitignore  |   1 +
 Documentation/git-for-each-repo.txt |  62 +++
 Makefile|   1 +
 builtin.h   |   1 +
 builtin/for-each-repo.c | 119 
 git.c   |   1 +
 t/t6400-for-each-repo.sh|  48 +++
 7 files changed, 233 insertions(+)
 create mode 100644 Documentation/git-for-each-repo.txt
 create mode 100644 builtin/for-each-repo.c
 create mode 100755 t/t6400-for-each-repo.sh

diff --git a/.gitignore b/.gitignore
index 63d4904..5036b84 100644
--- a/.gitignore
+++ b/.gitignore
@@ -56,6 +56,7 @@
 /git-filter-branch
 /git-fmt-merge-msg
 /git-for-each-ref
+/git-for-each-repo
 /git-format-patch
 /git-fsck
 /git-fsck-objects
diff --git a/Documentation/git-for-each-repo.txt 
b/Documentation/git-for-each-repo.txt
new file mode 100644
index 000..be49e96
--- /dev/null
+++ b/Documentation/git-for-each-repo.txt
@@ -0,0 +1,62 @@
+git-for-each-repo(1)
+
+
+NAME
+
+git-for-each-repo - Execute a git command in multiple repositories
+
+SYNOPSIS
+
+[verse]
+'git for-each-repo' [--all|--clean|--dirty] [command]
+
+DESCRIPTION
+---
+The git-for-each-repo command is used to locate all git repositoris
+within the current directory tree, and optionally execute a git command
+in each of the found repos.
+
+OPTIONS
+---
+-a::
+--all::
+   Include both clean and dirty repositories (this is the default
+   behaviour of `git-for-each-repo`).
+
+-c::
+--clean::
+   Only include repositories with a clean worktree.
+
+-d::
+--dirty::
+   Only include repositories with a dirty worktree.
+
+EXAMPLES
+
+
+Various ways to exploit this command::
++
+
+$ git for-each-repo1
+$ git for-each-repo fetch  2
+$ git for-each-repo -d gui 3
+$ git for-each-repo -c push4
+
++
+1 Print the path to all repos found below the current directory.
+
+2 Fetch updates from default remote in all repos.
+
+3 Start linkgit:git-gui[1] in each repo containing uncommitted changes.
+
+4 Push the current branch in each repo with no uncommited changes.
+
+NOTES
+-
+
+For the purpose of `git-for-each-repo`, a dirty worktree is defined as a
+worktree with uncommitted changes.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index a786d4c..8c42c17 100644
--- a/Makefile
+++ b/Makefile
@@ -870,6 +870,7 @@ BUILTIN_OBJS += builtin/fetch-pack.o
 BUILTIN_OBJS += builtin/fetch.o
 BUILTIN_OBJS += builtin/fmt-merge-msg.o
 BUILTIN_OBJS += builtin/for-each-ref.o
+BUILTIN_OBJS += builtin/for-each-repo.o
 BUILTIN_OBJS += builtin/fsck.o
 BUILTIN_OBJS += builtin/gc.o
 BUILTIN_OBJS += builtin/grep.o
diff --git a/builtin.h b/builtin.h
index 7e7bbd6..02fc712 100644
--- a/builtin.h
+++ b/builtin.h
@@ -73,6 +73,7 @@ extern int cmd_fetch(int argc, const char **argv, const char 
*prefix);
 extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
 extern int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix);
 extern int cmd_for_each_ref(int argc, const char **argv, const char *prefix);
+extern int cmd_for_each_repo(int argc, const char **argv, const char *prefix);
 extern int cmd_format_patch(int argc, const char **argv, const char *prefix);
 extern int cmd_fsck(int argc, const char **argv, const char *prefix);
 extern int cmd_gc(int argc, const char **argv, const char *prefix);
diff --git a/builtin/for-each-repo.c b/builtin/for-each-repo.c
new file mode 100644
index 000..9bdeb4a
--- /dev/null
+++ b/builtin/for-each-repo.c
@@ -0,0 +1,119 @@
+/*
+ * git for-each-repo builtin command.
+ *
+ * Copyright (c) 2013 Lars Hjemli hje...@gmail.com
+ */
+#include cache.h
+#include color.h
+#include builtin.h
+#include run-command.h
+#include parse-options.h
+
+#define ALL 0
+#define DIRTY 1
+#define CLEAN 2
+
+static int match;
+
+static const char * 

Re: [PATCH v3 1/2] for-each-repo: new command used for multi-repo operations

2013-01-23 Thread Junio C Hamano
Lars Hjemli hje...@gmail.com writes:

 diff --git a/Documentation/git-for-each-repo.txt 
 b/Documentation/git-for-each-repo.txt
 new file mode 100644
 index 000..be49e96
 --- /dev/null
 +++ b/Documentation/git-for-each-repo.txt
 @@ -0,0 +1,62 @@
 +git-for-each-repo(1)
 +
 +
 +NAME
 +
 +git-for-each-repo - Execute a git command in multiple repositories

multiple non-bare repositories, I think.

 +
 +SYNOPSIS
 +
 +[verse]
 +'git for-each-repo' [--all|--clean|--dirty] [command]
 +
 +DESCRIPTION
 +---
 +The git-for-each-repo command is used to locate all git repositoris

Likewise; all non-bare Git repositories.

 diff --git a/t/t6400-for-each-repo.sh b/t/t6400-for-each-repo.sh
 new file mode 100755
 index 000..4797629
 --- /dev/null
 +++ b/t/t6400-for-each-repo.sh
 @@ -0,0 +1,48 @@
 +#!/bin/sh
 +#
 +# Copyright (c) 2013 Lars Hjemli
 +#
 +
 +test_description='Test the git-for-each-repo command'
 +
 +. ./test-lib.sh
 +
 +test_expect_success setup '
 + test_create_repo clean 
 + (cd clean  test_commit foo) 
 + git init --separate-git-dir=.cleansub clean/gitfile 
 + (cd clean/gitfile  test_commit foo  echo bar foo.t) 
 + test_create_repo dirty-wt 
 + (cd dirty-wt  mv .git .linkedgit  ln -s .linkedgit .git 
 +   test_commit foo  rm foo.t) 
 + test_create_repo dirty-idx 
 + (cd dirty-idx  test_commit foo  git rm foo.t) 
 + mkdir fakedir  mkdir fakedir/.git
 +'
 +
 +test_expect_success without flags, all repos are included '
 + echo . expect 
 + echo clean expect 
 + echo clean/gitfile expect 
 + echo dirty-idx expect 
 + echo dirty-wt expect 
 + git for-each-repo | sort actual 
 + test_cmp expect actual
 +'
 +
 +test_expect_success --dirty only includes dirty repos '
 + echo clean/gitfile expect 
 + echo dirty-idx expect 
 + echo dirty-wt expect 
 + git for-each-repo --dirty | sort actual 
 + test_cmp expect actual
 +'
 +
 +test_expect_success --clean only includes clean repos '
 + echo . expect 
 + echo clean expect 
 + git for-each-repo --clean | sort actual 
 + test_cmp expect actual
 +'

Please add tests to show some command executions (e.g. test output
from git ls-files, or something).

 +static void handle_repo(char *path, const char **argv)
 +{
 + if (path[0] == '.'  path[1] == '/')
 + path += 2;
 + if (match != ALL  match != get_repo_state())
 + return;
 + if (*argv) {
 + color_fprintf_ln(stdout, GIT_COLOR_YELLOW, [%s], path);
 + run_command_v_opt(argv, RUN_GIT_CMD);

This seems to allow people to run only a single Git subcommand,
which is probably not what most people want to see.  Don't we want
to support something as simple as this?

git for-each-repository sh -c ls *.c

 + } else
 + printf(%s\n, path);

Assuming that the non *argv case is for consumption by programs and
scripts (similar to the way ls-files output is piped to downstream),
we prefer to (1) support -z so that xargs -0 can read paths with
funny characters, and (2) use quote_c_style() from quote.c when -z
is not in effect.

 +}
 + ...
 + setenv(GIT_DIR_ENVIRONMENT, gitdir, 1);
 + strbuf_setlen(path, len - 1);
 + setenv(GIT_WORK_TREE_ENVIRONMENT, path-buf, 1);
 + handle_repo(path-buf, argv);

When you are only showing the path to a repository, I do not think
you want setenv() or chdir() at all. Shouldn't these be done inside
handle_repo() function?  As you are only dealing with non-bare
repositories (and that is what you print in listing only mode
anyway), handle_repo() can borrow path (not path-buf) and append
and strip /.git as needed.

Also, while it is a good idea to protect this program from stray
GIT_DIR/GIT_WORK_TREE the user may have in the environment when this
program is started, I think this is not enough, if you allow the
*argv commands to run worktree related operations in each repository
you discover.  You would need to chdir() to the top of the working
tree.

The run-command API lets you specify custom environment only for the
child process without affecting yourself by setting .env member of
the child_process structure, so we may want to use that instead of
doing setenv() on ourselves (and letting it inherited by the child).

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html