Re: [PATCH/WIP v2 03/14] read-cache: connect to file watcher

2014-01-17 Thread Duy Nguyen
On Fri, Jan 17, 2014 at 10:24 PM, Torsten Bögershausen  wrote:
> On 2014-01-17 10.47, Nguyễn Thái Ngọc Duy wrote:
> [snip[
>> diff --git a/file-watcher-lib.c b/file-watcher-lib.c
>
>
>> +int connect_watcher(const char *path)
> Could it be worth to check if we can use some code from unix-socket.c ?
>
> Especially important could be that unix_sockaddr_init() wotks around a problem
> when "long" path names are used.
>

Thanks! I did not even know about unix-socket.c. Well, I never paid
attention to credential-cache.c :(
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH/WIP v2 03/14] read-cache: connect to file watcher

2014-01-17 Thread Torsten Bögershausen
On 2014-01-17 10.47, Nguyễn Thái Ngọc Duy wrote:
[snip[
> diff --git a/file-watcher-lib.c b/file-watcher-lib.c


> +int connect_watcher(const char *path)
Could it be worth to check if we can use some code from unix-socket.c ?

Especially important could be that unix_sockaddr_init() wotks around a problem
when "long" path names are used. 

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH/WIP v2 03/14] read-cache: connect to file watcher

2014-01-17 Thread Nguyễn Thái Ngọc Duy
This patch establishes a connection between a new file watcher daemon
and git. Each index file may have at most one file watcher attached to
it. The file watcher maintains a UNIX socket at
$GIT_DIR/index.watcher. Any process that has write access to $GIT_DIR
can talk to the file watcher.

A validation is performed after git connects to the file watcher to
make sure both sides have the same view. This is done by exchanging
the index signature (*) The file watcher keeps a copy of the signature
locally while git computes the signature from the index. If the
signatures do not match, something has gone wrong so both sides
reinitialize wrt. to file watching: the file watcher clears all
watches while git clears CE_WATCHED flags.

If the signatures match, we can trust the file watcher and git can
start asking questions that are not important to this patch.

This file watching thing is all about speed. So if the daemon is not
responding within 20ms (or even hanging), git moves on without it.

A note about per-repo vs global (or per-user) daemon approach. While I
implement per-repo daemon, this is actually implementation
details. Nothing can stop you from writing a global daemon that opens
unix sockets to many repos, e.g. to avoid hitting inotify's 128 user
instances limit.

If env variable GIT_NO_FILE_WATCHER is set, the file watcher is
ignored. 'WATC' extension is kept, but if the index is updated
(likely), it'll become invalid at the next connection.

(*) for current index versions, the signature is the index SHA-1
trailer. But it could be something else (e.g. v5 does not have SHA-1
trailer)

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 .gitignore   |   1 +
 Makefile |   2 +
 cache.h  |   3 +
 file-watcher-lib.c (new) |  97 
 file-watcher-lib.h (new) |   9 +++
 file-watcher.c (new) | 142 +++
 read-cache.c |  37 
 trace.c  |   3 +-
 8 files changed, 292 insertions(+), 2 deletions(-)
 create mode 100644 file-watcher-lib.c
 create mode 100644 file-watcher-lib.h
 create mode 100644 file-watcher.c

diff --git a/.gitignore b/.gitignore
index dc600f9..dc870cc 100644
--- a/.gitignore
+++ b/.gitignore
@@ -56,6 +56,7 @@
 /git-fast-import
 /git-fetch
 /git-fetch-pack
+/git-file-watcher
 /git-filter-branch
 /git-fmt-merge-msg
 /git-for-each-ref
diff --git a/Makefile b/Makefile
index 287e6f8..4369b3b 100644
--- a/Makefile
+++ b/Makefile
@@ -536,6 +536,7 @@ PROGRAMS += $(EXTRA_PROGRAMS)
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
+PROGRAM_OBJS += file-watcher.o
 PROGRAM_OBJS += http-backend.o
 PROGRAM_OBJS += imap-send.o
 PROGRAM_OBJS += sh-i18n--envsubst.o
@@ -798,6 +799,7 @@ LIB_OBJS += entry.o
 LIB_OBJS += environment.o
 LIB_OBJS += exec_cmd.o
 LIB_OBJS += fetch-pack.o
+LIB_OBJS += file-watcher-lib.o
 LIB_OBJS += fsck.o
 LIB_OBJS += gettext.o
 LIB_OBJS += gpg-interface.o
diff --git a/cache.h b/cache.h
index 069dce7..0d1 100644
--- a/cache.h
+++ b/cache.h
@@ -282,6 +282,7 @@ struct index_state {
struct hashmap name_hash;
struct hashmap dir_hash;
unsigned char sha1[20];
+   int watcher;
 };
 
 extern struct index_state the_index;
@@ -1241,6 +1242,8 @@ extern void alloc_report(void);
 __attribute__((format (printf, 1, 2)))
 extern void trace_printf(const char *format, ...);
 __attribute__((format (printf, 2, 3)))
+extern void trace_printf_key(const char *key, const char *fmt, ...);
+__attribute__((format (printf, 2, 3)))
 extern void trace_argv_printf(const char **argv, const char *format, ...);
 extern void trace_repo_setup(const char *prefix);
 extern int trace_want(const char *key);
diff --git a/file-watcher-lib.c b/file-watcher-lib.c
new file mode 100644
index 000..ed14ef9
--- /dev/null
+++ b/file-watcher-lib.c
@@ -0,0 +1,97 @@
+#include "cache.h"
+
+#define WAIT_TIME 20   /* in ms */
+#define TRACE_KEY "GIT_TRACE_WATCHER"
+
+int connect_watcher(const char *path)
+{
+   struct strbuf sb = STRBUF_INIT;
+   struct stat st;
+   int fd = -1;
+
+   strbuf_addf(&sb, "%s.watcher", path);
+   if (!stat(sb.buf, &st) && S_ISSOCK(st.st_mode)) {
+   struct sockaddr_un sun;
+   fd = socket(AF_UNIX, SOCK_DGRAM, 0);
+   sun.sun_family = AF_UNIX;
+   strlcpy(sun.sun_path, sb.buf, sizeof(sun.sun_path));
+   if (connect(fd, (struct sockaddr *)&sun, sizeof(sun))) {
+   error(_("unable to connect to file watcher: %s"),
+ strerror(errno));
+   close(fd);
+   fd = -1;
+   } else {
+   sprintf(sun.sun_path, "%c%"PRIuMAX, 0, 
(uintmax_t)getpid());
+   if (bind(fd, (struct sockaddr *)&sun, sizeof(sun))) {
+   error(_("unable to bind socket: %s"),
+