Re: Re: Re: Relative submodule URLs

2014-08-26 Thread Heiko Voigt
On Mon, Aug 25, 2014 at 09:29:07AM -0500, Robert Dailey wrote:
 On Sun, Aug 24, 2014 at 8:34 AM, Heiko Voigt hvo...@hvoigt.net wrote:
  New --with--remote parameter for 'git submodule'
  
 
  While having said all that about submodule settings I think a much
  much simpler start is to go ahead with a commandline setting, like
  Robert proposed here[2].
 
  For that we do not have to worry about how it can be stored,
  transported, defined per submodule or on a branch, since answers to this
  are given at the commandline (and current repository state).
 
  There are still open questions about this though:
 
* Should the name in the submodule be 'origin' even though you
  specified --with-remote=somewhere? For me its always confusing to
  have the same/similar remotes named differently in different
  repositories. That why I try to keep the names the same in all my
  clones of repositories (i.e. for my private, github, upstream
  remotes).
 
* When you do a 'git submodule sync --with-remote=somewhere' should
  the remote be added or replaced.
 
  My opinion on these are:
 
  The remote should be named as in the superproject so
  --with-remote=somewhere adds/replaces the remote 'somewhere' in the
  submodules named on the commandline (or all in case no submodule is
  specified). In case of a fresh clone of the submodule, there would be no
  origin but only a remote under the new name.
 
  Would the --with-remote feature I describe be a feasible start for you
  Robert? What do others think? Is the naming of the parameter
  '--with-remote' alright?
 
  Cheers Heiko
 
  [1] http://article.gmane.org/gmane.comp.version-control.git/255512
  [2] http://article.gmane.org/gmane.comp.version-control.git/255512
  [3] 
  https://github.com/jlehmann/git-submod-enhancements/wiki#special-ref-overriding-gitmodules-values
 
 Hi Heiko,
 
 My last email response was in violation of your request to keep the
 two topics separate, sorry about that. I started typing it this
 weekend and completed the draft this morning, without having read this
 response from you first.

Thats fine, no problem.

 Here is what I think would make the feature most usable. I think you
 went over some of these ideas but I just want to clarify, to make sure
 we're on the same page. Please correct me as needed.
 
 1. Running `git submodule update --with-remote name` shall fail the
 command unconditionally.

I am not sure but I think you mean

git submodule update --with-remote=name

With the equals sign, without it you would name the submodule paths to
update. No I think that should just add the remote name to all
submodules that would be updated and do the normal update operation on
them (with the new remote of course).

 2. Using the `--with-remote` option on submodule `update` or `sync`
 will fail if it detects absolute submodule URLs in .gitmodule

Yes, almost. Since you can have a mixture I suggest to only fail if the
submodules that would be processed have an absolute url in them. If
processed submodules are all relative it can go ahead.

 3. Running `git submodule update --init --with-remote name` shall
 fail the command ONLY if a submodule is being processed that is NOT
 also being initialized.

No since the --init flag just tells update to initialize submodules
on-demand. It should just go ahead the same way as without
--with-remote.

 4. The behavior of git submodule's `update` or `sync` commands
 combined with `--with-remote` will REPLACE or CREATE the 'origin'
 remote in each submodule it is run in. We will not allow the user to
 configure what the submodule remote name will end up being (I think
 this is current behavior and forces good practice; I consider `origin`
 an adopted standard for git, and actually wish it was more enforced
 for super projects as well!)

No please carefully read my email again. I specifically was describing
the opposite. --with-remote=name creates/replaces the remote name in
the submodule. I do not see a benefit in restricting the user from
creating different remote names in the submodule. I think it would be
more confusing if the remote 'origin' in the superproject does not point
to the same location as 'origin' in the submodule.

 Let me know if I've missed anything. Once we clarify requirements I'll
 attempt to start work on this during my free time. I'll start by
 testing this through msysgit, since I do not have linux installed, but
 I have Linux Mint running in a Virtual Machine so I can test on both
 platforms as needed (I don't have a lot of experience on Linux
 though).

I think it does not matter which development environment you use. In my
experience though Linux is around 30x faster when it comes to the
typical operations you do when developing git. Especially for running
the testsuite that makes a difference between a few hours and minutes.

 I hope you won't mind me reaching out for questions as needed, however
 I 

Bug report: Author/Commit date in ISO 8601 format

2014-08-26 Thread Oliver Busch

Dear git community,

I encountered the following problem: When using the date formatting 
option ISO (either by setting --date=iso or using format:%ci for the 
committer date), the output is formatted like this:


2014-08-25 17:49:43 +0200

But according to ISO 8601, should be formatted like this (see 
http://www.w3.org/TR/NOTE-datetime):


2014-08-25T17:49:43+02:00

This difference makes it impossible to use automated string conversion 
(in my case, I tried to use Delphi's XSBuildIns 
(http://wiert.me/2011/07/19/iso-8601-delphi-way-to-convert-xml-date-and-time-to-tdatetime-and-back-via-stack-overflow/), 
which converted git's output 2014-08-25 17:49:43 +0200 to a TDateTime 
of 2014-08-25 02:00:00; conversion worked correctly when using 
2014-08-25T17:49:43+02:00 as input).


I therefore suggest to adapt the output when using --date=iso or 
format:%ci to comply 100% with the ISO 8601 specs, or at least change 
documentation to say the output is only ISO-like.


Kind regards,

Oliver Busch

--

Oliver Busch, M.Eng.

Airport Research Center GmbH
Bismarckstraße 61
52066 Aachen
Germany

Phone: +49 241 16843-161
Fax: +49 241 16843-19
e-mail: oliver.bu...@arc-aachen.de
Website: http://www.airport-consultants.com

Register Court: Amtsgericht Aachen HRB 7313
Ust-Id-No.: DE196450052

Managing Directors:
Dipl.-Ing. Tom Alexander Heuer

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] bisect: save heap memory. allocate only the required amount

2014-08-26 Thread Arjun Sreedharan
On 26 August 2014 02:44, Junio C Hamano gits...@pobox.com wrote:
 Arjun Sreedharan arjun...@gmail.com writes:

 Find and allocate the required amount instead of allocating extra
 100 bytes

 Signed-off-by: Arjun Sreedharan arjun...@gmail.com
 ---

 Interesting.  How much memory do we typically waste (in other words,
 how big did cnt grew in your repository where you noticed this)?


I did not try to make an observation of that. I was thinking we're
anyway better off
not using a magic number to allot memory on the heap for each item in
the commit list.

  bisect.c | 7 +--
  1 file changed, 5 insertions(+), 2 deletions(-)

 diff --git a/bisect.c b/bisect.c
 index d6e851d..c96aab0 100644
 --- a/bisect.c
 +++ b/bisect.c
 @@ -215,10 +215,13 @@ static struct commit_list 
 *best_bisection_sorted(struct commit_list *list, int n
   }
   qsort(array, cnt, sizeof(*array), compare_commit_dist);
   for (p = list, i = 0; i  cnt; i++) {
 - struct name_decoration *r = xmalloc(sizeof(*r) + 100);
 + char name[100];
 + sprintf(name, dist=%d, array[i].distance);
 + int name_len = strlen(name);

 Decl-after-stmt.

 You do not have to run a separate strlen() for this.  The return
 value from sprintf() should tell you how much you need to allocate,
 I think.


Yes. yes.

 + struct name_decoration *r = xmalloc(sizeof(*r) + name_len);
   struct object *obj = (array[i].commit-object);

 - sprintf(r-name, dist=%d, array[i].distance);
 + memcpy(r-name, name, name_len + 1);
   r-next = add_decoration(name_decoration, obj, r);
   p-item = array[i].commit;
   p = p-next;
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Improving the git remote command

2014-08-26 Thread Rémy Hubscher
Hi,

I'd like to add a list parameter to the `git remote` command.

We already have:
 
 - `git remote add`
 - `git remote rename`
 - `git remote delete`

I often write `git remote list` before finaly using `git remote -v` but
it isn't intuitive.

I am proposing to add `git remote list` as a shortcut for `git remote -v`

What do you think?

Rémy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Improving the git remote command

2014-08-26 Thread Philippe Vaucher
 I often write `git remote list` before finaly using `git remote -v` but
 it isn't intuitive.

 I am proposing to add `git remote list` as a shortcut for `git remote -v`

I suffer from the same problem. I think your proposal is a logical and
nice idea.

Philippe
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 0/3] name_decoration cleanups

2014-08-26 Thread Jeff King
On Mon, Aug 25, 2014 at 02:11:09PM -0700, Junio C Hamano wrote:

 Jeff King p...@peff.net writes:
 
  On Mon, Aug 25, 2014 at 11:26:39AM -0700, Junio C Hamano wrote:
 
  Good digging, and I agree that it should use the FLEX_ARRAY for
  consistency.
 
  I can produce a patch, but I did not want to steal Arjun's thunder.
 
 Hmph, would it have to overlap?  I think we can queue Arjun's patch
 with +1 fix and FLEX_ARRAY thing separately, and they can go in in
 any order, no?

I more meant my suggestion to use add_name_decoration consistently. That
fixes the r-type thing _and_ replaces Arjun's patch. Fixing FLEX_ARRAY
on top is just gravy. :)

Here's the patch series I was thinking of:

  [1/3]: log-tree: make add_name_decoration a public function
  [2/3]: log-tree: make name_decoration hash static
  [3/3]: log-tree: use FLEX_ARRAY in name_decoration

I almost added a 4/3 to convert name_decoration to
commit_decoration, since that is how it is used (and name_decoration
is somewhat vague). But we actually do annotate other non-commit objects
that refs point to, as well. I'm not sure there is a way to _show_ them
currently, but I'd just as soon leave it as-is.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 3/3] log-tree: use FLEX_ARRAY in name_decoration

2014-08-26 Thread Jeff King
We are already using the flex-array technique; let's
annotate it with our usual FLEX_ARRAY macro. Besides being
more readable, this is slightly more efficient on compilers
that understand flex-arrays.

Note that we need to bump the allocation in add_name_decoration,
which did not explicitly add one byte for the NUL terminator
of the string we putting into the flex-array (it did not
need to before, because the struct itself was over-allocated
by one byte).

Signed-off-by: Jeff King p...@peff.net
---
This could come first in the series, but doing it last means we only
have to update one spot. :)

 commit.h   | 2 +-
 log-tree.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/commit.h b/commit.h
index 263b49e..1516fc9 100644
--- a/commit.h
+++ b/commit.h
@@ -29,7 +29,7 @@ extern const char *commit_type;
 struct name_decoration {
struct name_decoration *next;
int type;
-   char name[1];
+   char name[FLEX_ARRAY];
 };
 
 enum decoration_type {
diff --git a/log-tree.c b/log-tree.c
index 7cbc4ee..fb60018 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -77,7 +77,7 @@ int parse_decorate_color_config(const char *var, const int 
ofs, const char *valu
 void add_name_decoration(enum decoration_type type, const char *name, struct 
object *obj)
 {
int nlen = strlen(name);
-   struct name_decoration *res = xmalloc(sizeof(struct name_decoration) + 
nlen);
+   struct name_decoration *res = xmalloc(sizeof(*res) + nlen + 1);
memcpy(res-name, name, nlen + 1);
res-type = type;
res-next = add_decoration(name_decoration, obj, res);
-- 
2.1.0.346.ga0367b9
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/3] log-tree: make add_name_decoration a public function

2014-08-26 Thread Jeff King
The log-tree code keeps a struct decoration hash to show
text decorations for each commit during log traversals. It
makes this available to other files by providing global
access to the hash. This can result in other code adding
entries that do not conform to what log-tree expects.

For example, the bisect code adds its own dist
decorations to be shown. Originally the bisect code was
correct, but when the name_decoration code grew a new field
in eb3005e (commit.h: add 'type' to struct name_decoration,
2010-06-19), the bisect code was not updated. As a result,
the log-tree code can access uninitialized memory and even
segfault.

We can fix this by making name_decoration's adding function
public. If all callers use it, then any changes to structi
initialization only need to happen in one place (and because
the members come in as parameters, the compiler can notice a
caller who does not supply enough information).

As a bonus, this also means that the decoration hashes
created by the bisect code will use less memory (previously
we over-allocated space for the distance integer, but not we
format it into a temporary buffer and copy it to the final
flex-array).

Signed-off-by: Jeff King p...@peff.net
---
 bisect.c   |  7 ---
 commit.h   | 12 
 log-tree.c | 12 +---
 3 files changed, 17 insertions(+), 14 deletions(-)

diff --git a/bisect.c b/bisect.c
index d6e851d..df09cbc 100644
--- a/bisect.c
+++ b/bisect.c
@@ -215,11 +215,12 @@ static struct commit_list *best_bisection_sorted(struct 
commit_list *list, int n
}
qsort(array, cnt, sizeof(*array), compare_commit_dist);
for (p = list, i = 0; i  cnt; i++) {
-   struct name_decoration *r = xmalloc(sizeof(*r) + 100);
+   char buf[100]; /* enough for dist=%d */
struct object *obj = (array[i].commit-object);
 
-   sprintf(r-name, dist=%d, array[i].distance);
-   r-next = add_decoration(name_decoration, obj, r);
+   snprintf(buf, sizeof(buf), dist=%d, array[i].distance);
+   add_name_decoration(DECORATION_NONE, buf, obj);
+
p-item = array[i].commit;
p = p-next;
}
diff --git a/commit.h b/commit.h
index a8cbf52..4902f97 100644
--- a/commit.h
+++ b/commit.h
@@ -33,6 +33,18 @@ struct name_decoration {
char name[1];
 };
 
+enum decoration_type {
+   DECORATION_NONE = 0,
+   DECORATION_REF_LOCAL,
+   DECORATION_REF_REMOTE,
+   DECORATION_REF_TAG,
+   DECORATION_REF_STASH,
+   DECORATION_REF_HEAD,
+   DECORATION_GRAFTED,
+};
+
+void add_name_decoration(enum decoration_type type, const char *name, struct 
object *obj);
+
 struct commit *lookup_commit(const unsigned char *sha1);
 struct commit *lookup_commit_reference(const unsigned char *sha1);
 struct commit *lookup_commit_reference_gently(const unsigned char *sha1,
diff --git a/log-tree.c b/log-tree.c
index 0c53dc1..a821258 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -14,16 +14,6 @@
 
 struct decoration name_decoration = { object names };
 
-enum decoration_type {
-   DECORATION_NONE = 0,
-   DECORATION_REF_LOCAL,
-   DECORATION_REF_REMOTE,
-   DECORATION_REF_TAG,
-   DECORATION_REF_STASH,
-   DECORATION_REF_HEAD,
-   DECORATION_GRAFTED,
-};
-
 static char decoration_colors[][COLOR_MAXLEN] = {
GIT_COLOR_RESET,
GIT_COLOR_BOLD_GREEN,   /* REF_LOCAL */
@@ -84,7 +74,7 @@ int parse_decorate_color_config(const char *var, const int 
ofs, const char *valu
 #define decorate_get_color_opt(o, ix) \
decorate_get_color((o)-use_color, ix)
 
-static void add_name_decoration(enum decoration_type type, const char *name, 
struct object *obj)
+void add_name_decoration(enum decoration_type type, const char *name, struct 
object *obj)
 {
int nlen = strlen(name);
struct name_decoration *res = xmalloc(sizeof(struct name_decoration) + 
nlen);
-- 
2.1.0.346.ga0367b9

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/3] log-tree: make name_decoration hash static

2014-08-26 Thread Jeff King
In the previous commit, we made add_name_decoration global
so that adders would not have to access the hash directly.
We now make the hash itself static so that callers _have_ to
add through our function, making sure that all additions go
through a single point.  To do this, we have to add one more
accessor function: a way to lookup entries in the hash.

Since the only caller doesn't actually look at the returned
value, but rather only asks whether there is a decoration or
not, we could provide only a boolean has_name_decoration.
That would allow us to make struct name_decoration local
to log-tree, as well.

However, it's unlikely to cause any maintainability harm
making the actual data public, and this interface is more
flexible if we need to look at decorations from other parts
of the code in the future.

Signed-off-by: Jeff King p...@peff.net
---
 commit.h   |  2 +-
 log-tree.c | 11 ---
 revision.c |  2 +-
 3 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/commit.h b/commit.h
index 4902f97..263b49e 100644
--- a/commit.h
+++ b/commit.h
@@ -26,7 +26,6 @@ extern int save_commit_buffer;
 extern const char *commit_type;
 
 /* While we can decorate any object with a name, it's only used for commits.. 
*/
-extern struct decoration name_decoration;
 struct name_decoration {
struct name_decoration *next;
int type;
@@ -44,6 +43,7 @@ enum decoration_type {
 };
 
 void add_name_decoration(enum decoration_type type, const char *name, struct 
object *obj);
+const struct name_decoration *get_name_decoration(const struct object *obj);
 
 struct commit *lookup_commit(const unsigned char *sha1);
 struct commit *lookup_commit_reference(const unsigned char *sha1);
diff --git a/log-tree.c b/log-tree.c
index a821258..7cbc4ee 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -12,7 +12,7 @@
 #include sequencer.h
 #include line-log.h
 
-struct decoration name_decoration = { object names };
+static struct decoration name_decoration = { object names };
 
 static char decoration_colors[][COLOR_MAXLEN] = {
GIT_COLOR_RESET,
@@ -83,6 +83,11 @@ void add_name_decoration(enum decoration_type type, const 
char *name, struct obj
res-next = add_decoration(name_decoration, obj, res);
 }
 
+const struct name_decoration *get_name_decoration(const struct object *obj)
+{
+   return lookup_decoration(name_decoration, obj);
+}
+
 static int add_ref_decoration(const char *refname, const unsigned char *sha1, 
int flags, void *cb_data)
 {
struct object *obj;
@@ -177,13 +182,13 @@ void format_decorations(struct strbuf *sb,
int use_color)
 {
const char *prefix;
-   struct name_decoration *decoration;
+   const struct name_decoration *decoration;
const char *color_commit =
diff_get_color(use_color, DIFF_COMMIT);
const char *color_reset =
decorate_get_color(use_color, DECORATION_NONE);
 
-   decoration = lookup_decoration(name_decoration, commit-object);
+   decoration = get_name_decoration(commit-object);
if (!decoration)
return;
prefix =  (;
diff --git a/revision.c b/revision.c
index 2571ada..5aff2b4 100644
--- a/revision.c
+++ b/revision.c
@@ -473,7 +473,7 @@ static int rev_compare_tree(struct rev_info *revs,
 * If we are simplifying by decoration, then the commit
 * is worth showing if it has a tag pointing at it.
 */
-   if (lookup_decoration(name_decoration, commit-object))
+   if (get_name_decoration(commit-object))
return REV_TREE_DIFFERENT;
/*
 * A commit that is not pointed by a tag is uninteresting
-- 
2.1.0.346.ga0367b9

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] bisect: save heap memory. allocate only the required amount

2014-08-26 Thread Jeff King
On Mon, Aug 25, 2014 at 02:36:02PM -0700, Junio C Hamano wrote:

  I think you are right, and the patch is the right direction (assuming we
  want to do this; I question whether there are enough elements in the
  list for us to care about the size, and if there are, we are probably
  better off storing the int and formatting the strings on the fly).
 
 ;-)

Having now dug into this much further, the answers to those questions
are:

  1. Yes, we might actually have quite a lot of these, if you are
 bisecting a large range of commits. However, the code only runs
 when you are doing --bisect-all or skipping a commit, so probably
 nobody actually cares that much.

  2. It would be nicer to hold just an int, but we are hooking into
 the log-tree decorate machinery here, which expects a string. We'd
 need some kind of decorate-like callback machinery from log-tree to
 do this right. It's probably not worth the effort.

 Among the flex arrays we use, some are arrays of bools, some others
 are arrays of object names, and there are many arrays of even more
 esoteric types.  Only a small number of them are We want a struct
 with a constant string, and we do not want to do two allocations and
 to pay an extra dereference cost by using 'const char *'.

Yeah, I was working under the assumption that most of them were holding
a string. I just did a quick skim of the grep results for FLEX_ARRAY. Of
the 36 instances, 26 hold strings. 9 hold something else entirely, and 1
holds a char buffer that does not need to be NUL-terminated (so it
_could_ be handled by a similar helper, but using %s would be wrong).

So it's definitely the majority, but certainly not all. I decided to
look into this a little further, and my results are below. The tl;dr is
that no, we probably shouldn't do it. So you can stop reading if you
don't find this interesting. :)

 For them, by the time we allocate a struct, by definition we should
 have sufficient information to compute how large to make that
 structure and a printf-format plus its args would be the preferred
 form of that sufficient information, I would think.

I was tempted to also suggest a pure-string form (i.e., just take a
string, run strlen on it, and use that as the final value). That would
make the variadic macro problem go away. But besides name_decoration,
there are other cases that really do want formatting.  For instance,
alloc_ref basically wants to do (%s%s, prefix, name).

 The name fmt_flex_array(), which stresses too much on the
 formatting side without implying that it is the way to allocate
 the thing, may be horrible, and I agree with you that without
 variadic macros the end result may not read very well.  Unless we
 have great many number of places we can use the helper to make the
 code to create these objects look nicer, I am afraid that the
 pros-and-cons may not be very favourable.

Yeah, reading my suggestion again, it should clearly be
alloc_flex_struct or something.

Here's a fully-converted sample spot, where I think there's a slight
benefit:

diff --git a/remote.c b/remote.c
index 3d6c86a..ba32d40 100644
--- a/remote.c
+++ b/remote.c
@@ -928,14 +928,30 @@ int remote_find_tracking(struct remote *remote, struct 
refspec *refspec)
return query_refspecs(remote-fetch, remote-fetch_refspec_nr, refspec);
 }
 
+static void *alloc_flex_struct(size_t base, size_t offset, const char *fmt, 
...)
+{
+   va_list ap;
+   size_t extra;
+   char *ret;
+
+   va_start(ap, fmt);
+   extra = vsnprintf(NULL, 0, fmt, ap);
+   extra++; /* for NUL terminator */
+   va_end(ap);
+
+   ret = xcalloc(1, base + extra);
+   va_start(ap, fmt);
+   vsnprintf(ret + offset, extra, fmt, ap);
+   va_end(ap);
+
+   return ret;
+}
+
 static struct ref *alloc_ref_with_prefix(const char *prefix, size_t prefixlen,
const char *name)
 {
-   size_t len = strlen(name);
-   struct ref *ref = xcalloc(1, sizeof(struct ref) + prefixlen + len + 1);
-   memcpy(ref-name, prefix, prefixlen);
-   memcpy(ref-name + prefixlen, name, len);
-   return ref;
+   return alloc_flex_struct(sizeof(struct ref), offsetof(struct ref, name),
+%.*s%s, prefixlen, prefix, name);
 }
 
 struct ref *alloc_ref(const char *name)

Obviously the helper is much longer than the code it is replacing, but
it would be used in multiple spots. The main thing I like here is that
we are dropping the manual length computations, which are easy to get
wrong (it's easy to forget a +1 for a NUL terminator, etc).

The offsetof is a little ugly. And the fact that we have a pre-computed
length for prefixlen makes the format string a little ugly.

Here's a another example:

diff --git a/attr.c b/attr.c
index 734222d..100c423 100644
--- a/attr.c
+++ b/attr.c
@@ -89,8 +89,8 @@ static struct git_attr *git_attr_internal(const char *name, 
int len)
if (invalid_attr_name(name, len))

[no subject]

2014-08-26 Thread mail
Hi everyone.

Normally if I am tracking a file in a Directory which satisfies one of my
.gitignore rules, I can just add them via git add foo, and I have no
issues.

For example, let me create a new repo.  I create a directory lib
containing file foo.txt, and a Directory lib2 with a file bar.txt. 
I add and commit both files.

Now I create a .gitignore file containing the rule lib*/.  I modify
lib/foo.txt and lib2/bar.txt.  I do git status and I'm informed that
foo.txt and bar.txt are modified.  So far so good.

The following commands work as expected:
 git add lib/foo.txt
 git add lib2/bar.txt

But if I instead do:
  git add lib/foo.txt lib2/bar.txt

I get the following response:
  The following paths are ignored by one of your .gitignore files:
  lib
  lib2

I have verified this behavior in git 1.9.3 and 2.1.0.  Is this desired
behavior or is it a bug?  If it is desired behavior, could someone please
describe the utility to me?

Many thanks for your excellent work

Glen Stark


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/3] log-tree: make add_name_decoration a public function

2014-08-26 Thread Ramsay Jones
On 26/08/14 11:23, Jeff King wrote:
 The log-tree code keeps a struct decoration hash to show
 text decorations for each commit during log traversals. It
 makes this available to other files by providing global
 access to the hash. This can result in other code adding
 entries that do not conform to what log-tree expects.
 
 For example, the bisect code adds its own dist
 decorations to be shown. Originally the bisect code was
 correct, but when the name_decoration code grew a new field
 in eb3005e (commit.h: add 'type' to struct name_decoration,
 2010-06-19), the bisect code was not updated. As a result,
 the log-tree code can access uninitialized memory and even
 segfault.
 
 We can fix this by making name_decoration's adding function
 public. If all callers use it, then any changes to structi

s/structi/struct/

 initialization only need to happen in one place (and because
 the members come in as parameters, the compiler can notice a
 caller who does not supply enough information).
 
 As a bonus, this also means that the decoration hashes
 created by the bisect code will use less memory (previously
 we over-allocated space for the distance integer, but not we

s/not/now/

 format it into a temporary buffer and copy it to the final
 flex-array).
 
 Signed-off-by: Jeff King p...@peff.net

ATB,
Ramsay Jones



--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] bisect: save heap memory. allocate only the required amount

2014-08-26 Thread Ramsay Jones
On 26/08/14 12:03, Jeff King wrote:
[snip]
 
 Yeah, reading my suggestion again, it should clearly be
 alloc_flex_struct or something.
 
 Here's a fully-converted sample spot, where I think there's a slight
 benefit:
 
 diff --git a/remote.c b/remote.c
 index 3d6c86a..ba32d40 100644
 --- a/remote.c
 +++ b/remote.c
 @@ -928,14 +928,30 @@ int remote_find_tracking(struct remote *remote, struct 
 refspec *refspec)
   return query_refspecs(remote-fetch, remote-fetch_refspec_nr, refspec);
  }
  
 +static void *alloc_flex_struct(size_t base, size_t offset, const char *fmt, 
 ...)
 +{
 + va_list ap;
 + size_t extra;
 + char *ret;
 +
 + va_start(ap, fmt);
 + extra = vsnprintf(NULL, 0, fmt, ap);
 + extra++; /* for NUL terminator */
 + va_end(ap);
 +
 + ret = xcalloc(1, base + extra);
 + va_start(ap, fmt);
 + vsnprintf(ret + offset, extra, fmt, ap);

What is the relationship between 'base' and 'offset'?

Let me assume that base is always, depending on your compiler, either
equal to offset or offset+1. Yes? (I'm assuming base is always the
sizeof(struct whatever)). Do you need both base and offset?

 + va_end(ap);
 +
 + return ret;
 +}
 +
  static struct ref *alloc_ref_with_prefix(const char *prefix, size_t 
 prefixlen,
   const char *name)
  {
 - size_t len = strlen(name);
 - struct ref *ref = xcalloc(1, sizeof(struct ref) + prefixlen + len + 1);
 - memcpy(ref-name, prefix, prefixlen);
 - memcpy(ref-name + prefixlen, name, len);
 - return ref;
 + return alloc_flex_struct(sizeof(struct ref), offsetof(struct ref, name),
 +  %.*s%s, prefixlen, prefix, name);
  }
  
  struct ref *alloc_ref(const char *name)
 
 Obviously the helper is much longer than the code it is replacing, but
 it would be used in multiple spots. The main thing I like here is that
 we are dropping the manual length computations, which are easy to get
 wrong (it's easy to forget a +1 for a NUL terminator, etc).
 
 The offsetof is a little ugly. And the fact that we have a pre-computed
 length for prefixlen makes the format string a little ugly.
 
 Here's a another example:
 
 diff --git a/attr.c b/attr.c
 index 734222d..100c423 100644
 --- a/attr.c
 +++ b/attr.c
 @@ -89,8 +89,8 @@ static struct git_attr *git_attr_internal(const char *name, 
 int len)
   if (invalid_attr_name(name, len))
   return NULL;
  
 - a = xmalloc(sizeof(*a) + len + 1);
 - memcpy(a-name, name, len);
 + a = alloc_flex_array(sizeof(*a), offsetof(struct git_attr, name),
 +  %.*s, len, name);
   a-name[len] = 0;
   a-h = hval;
   a-next = git_attr_hash[pos];
 
 I think this is strictly worse for reading. The original computation was
 pretty easy in the first place, so we are not getting much benefit
 there. And again we have the precomputed len passed into the function,
 so we have to use the less readable %.*s. And note that offsetof()
 requires us to pass a real typename instead of just *a, as sizeof()
 allows (I suspect passing a-name - a would work on many systems, but
 I also suspect that is a gross violation of the C standard when a has
 not yet been initialized).
 
 So given that the benefit ranges from a little to negative in these
 two examples, I'm inclined to give up this line of inquiry.

Indeed. :-D

ATB,
Ramsay Jones



--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/2] Check order when reading index

2014-08-26 Thread Jaime Soriano Pastor
On Mon, Aug 25, 2014 at 10:52 PM, Junio C Hamano gits...@pobox.com wrote:
 On Mon, Aug 25, 2014 at 12:44 PM, Jeff King p...@peff.net wrote:
 For my own curiosity, how do you get into this situation, and what does
 it mean to have multiple stage#1 entries for the same path? What would
 git cat-file :1:path output?

 That is how we natively (read: not with the funky virtual stuff
 merge-recursive does) express a merge with multiple merge bases.
 You also should be able to read this in the way how git merge invokes
 merge strategies (one or more bases, double-dash and then current
 HEAD and the other branches). I think there are some tests in 3way
 merge tests that checks what should happen when our HEAD matches
 one of the stage #1 while their branch matches a different one of the
 stage #1, too.

I'm a bit lost with this, conceptually it doesn't seem to be any
problem with having multiple merge bases, but I don't manage to
reproduce it.
With natively do you mean some internal state that is never written
into the index? If this were the case then there wouldn't be any
problem with the restriction when reading the index file.

I have also tried to reproduce it by directly calling
git-merge-recursive and the most I have got is what it seemed to be
like a conflict in the stage #1:

$ git ls-files -s
100644 1036ba5101378f06aa10c5fa249b67e14f83166b 1 conflict
100644 2638c45f8e7bc5b56f70e92390153572811782c3 2 conflict
100644 178481050188cf00d7d9cd5a11e43ab8fab9294f 3 conflict

$ git cat-file blob 1036ba5101378f06aa10c5fa249b67e14f83166b
 Temporary merge branch 1
G
===
E
F
 Temporary merge branch 2

And after thinking a bit more about it, I don't see a way to have two
stage #1 entries for the same path with git commands only. It seems
that all entries are added through the add_index_entry_with_check()
function (except maybe the added to the cached tree extension), and
this function replaces existing entries if they have the same name and
stage.
Also, all tests pass with the patch, without allowing two entries for
the same stage.

So I'm afraid that I don't fully understand this case :/
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] bisect: save heap memory. allocate only the required amount

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 12:57:21PM +0100, Ramsay Jones wrote:

  +   ret = xcalloc(1, base + extra);
  +   va_start(ap, fmt);
  +   vsnprintf(ret + offset, extra, fmt, ap);
 
 What is the relationship between 'base' and 'offset'?
 
 Let me assume that base is always, depending on your compiler, either
 equal to offset or offset+1. Yes? (I'm assuming base is always the
 sizeof(struct whatever)). Do you need both base and offset?

It's much more complicated than that. Take struct name_decoration, for
instance, which looks like this:

  struct name_decoration {
struct name_decoration *next;
int type;
char name[FLEX_ARRAY];
  };

On my 64-bit system using gcc, sizeof() returns 16; it has to pad the
whole thing to 64-bit alignment in case I put two of them in an array.
But offsetof(name) is 12, since the array of char does not need the same
alignment; it can go right after the type and make use of the padding
bits.

As a side note, that means that the original char name[1] (before it
became FLEX_ARRAY) was not any less efficient on 64-bit machines (the
1-byte went into the padding, and sizeof() was the same). It did matter
on 32-bit systems, though where it bumped the empty struct size from 12
to 16.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/3] log-tree: make add_name_decoration a public function

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 12:48:24PM +0100, Ramsay Jones wrote:

  We can fix this by making name_decoration's adding function
  public. If all callers use it, then any changes to structi
 
 s/structi/struct/

I blame vi finger-cruft.

  initialization only need to happen in one place (and because
  the members come in as parameters, the compiler can notice a
  caller who does not supply enough information).
  
  As a bonus, this also means that the decoration hashes
  created by the bisect code will use less memory (previously
  we over-allocated space for the distance integer, but not we
 
 s/not/now/

Er, I blame vi again. Yeah, that's my story and I'm sticking to it.

Thanks for proofreading. :)

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/2] Check order when reading index

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 02:08:35PM +0200, Jaime Soriano Pastor wrote:

  That is how we natively (read: not with the funky virtual stuff
  merge-recursive does) express a merge with multiple merge bases.
  You also should be able to read this in the way how git merge invokes
  merge strategies (one or more bases, double-dash and then current
  HEAD and the other branches). I think there are some tests in 3way
  merge tests that checks what should happen when our HEAD matches
  one of the stage #1 while their branch matches a different one of the
  stage #1, too.
 
 I'm a bit lost with this, conceptually it doesn't seem to be any
 problem with having multiple merge bases, but I don't manage to
 reproduce it.
 With natively do you mean some internal state that is never written
 into the index? If this were the case then there wouldn't be any
 problem with the restriction when reading the index file.

FWIW, that was my question on reading Junio's response, too.

 I have also tried to reproduce it by directly calling
 git-merge-recursive and the most I have got is what it seemed to be
 like a conflict in the stage #1:
 
 $ git ls-files -s
 100644 1036ba5101378f06aa10c5fa249b67e14f83166b 1 conflict
 100644 2638c45f8e7bc5b56f70e92390153572811782c3 2 conflict
 100644 178481050188cf00d7d9cd5a11e43ab8fab9294f 3 conflict
 
 $ git cat-file blob 1036ba5101378f06aa10c5fa249b67e14f83166b
  Temporary merge branch 1
 G
 ===
 E
 F
  Temporary merge branch 2

Yes, I think merge-recursive resolves the earlier merges and then feeds
the result into the main merge. And that's how you end up with the
temporary merge branch name instead of something useful.

It might work to have a recursive merge that causes a conflict on path
X, and then we further need to resolve that conflict. I'm not sure if we
try to represent that in the index somehow, or if merge-recursive just
bails in this case (I didn't try it).

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 2/3] Makefile: use `find` to determine static header dependencies

2014-08-26 Thread Jeff King
On Mon, Aug 25, 2014 at 03:08:50PM -0700, Junio C Hamano wrote:

 Jonathan Nieder jrnie...@gmail.com writes:
 
  Wouldn't it be sufficient to start digging not from * but from
  ??*?
 
  Gah, the * was supposed to be . in my examples (though it doesn't
  hurt).
 
 find ??* \( -name Documentation -o -name .\?\* \) -prune -o -name \*.h
 
  Heh.  Yeah, that would work. ;-)
 
 Continuing useless discussion...
 
 Actually as you are not excluding CVS, RCS, etc., and using ??* as
 the starting point will exclude .git, .hg, etc. at the top, I think
 we can shorten it even further and say
 
   find ??* -name Documentation -prune -o -name \*.h
 
 or something.

I had originally considered starting with find *, but I was worried
about shell globbing overflowing command-line limits here. echo * on a
built tree is about 12K. That's laughably small for Linux, but would
other systems (which, after all, are the main targets) be more picky?

POSIX lists 4K as the minimum, and that has to fit the environment, too.

I'd also be fine to try it and see if anybody on an antique system
complains.

 Don't we want to exclude contrib/ by the way?

Probably. For calculating dependencies, it is OK to be overly
conservative (the worst case is that we trigger a recompile if somebody
touched contrib/.../foo.h, which is rather unlikely).

For the .pot file, being conservative is a little annoying.  In theory
we might want to translate stuff in contrib/, but it probably is just
extra work for translators for not much benefit (though I have not
really used gettext; I assume it only pulls in strings marked with _()
and friends, so being conservative is maybe not that big a deal...).

In that sense, maybe we should just hit the whole tree to be on the
conservative side. The two reasons I did not in my initial attempt were:

  1. Performance. But with the final form, we only the run the `find` at
 all very rarely, so shaving off a few readdirs() is not that big a
 deal.

  2. There are a few problematic areas. t/perf may contain build trees
 which are copies of git, which I expect would confuse gettext.

So I dunno.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] bisect: save heap memory. allocate only the required amount

2014-08-26 Thread Ramsay Jones
On 26/08/14 13:14, Jeff King wrote:
 On Tue, Aug 26, 2014 at 12:57:21PM +0100, Ramsay Jones wrote:
 
 +   ret = xcalloc(1, base + extra);
 +   va_start(ap, fmt);
 +   vsnprintf(ret + offset, extra, fmt, ap);

 What is the relationship between 'base' and 'offset'?

 Let me assume that base is always, depending on your compiler, either
 equal to offset or offset+1. Yes? (I'm assuming base is always the
 sizeof(struct whatever)). Do you need both base and offset?
 
 It's much more complicated than that. Take struct name_decoration, for
 instance, which looks like this:
 
   struct name_decoration {
   struct name_decoration *next;
   int type;
   char name[FLEX_ARRAY];
   };
 
 On my 64-bit system using gcc, sizeof() returns 16; it has to pad the
 whole thing to 64-bit alignment in case I put two of them in an array.
 But offsetof(name) is 12, since the array of char does not need the same
 alignment; it can go right after the type and make use of the padding
 bits.

Hmm, interesting. I must re-read the standard. I was convinced that the
standard *requires* any alignment padding to come *before* the name field.
(how would you put a, non-trivial, variable data structure into an array?)

ATB,
Ramsay Jones


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Improving the git remote command

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 11:29:32AM +0200, Rémy Hubscher wrote:

 I'd like to add a list parameter to the `git remote` command.
 
 We already have:
  
  - `git remote add`
  - `git remote rename`
  - `git remote delete`
 
 I often write `git remote list` before finaly using `git remote -v` but
 it isn't intuitive.

Right now the list operation is done by giving no arguments at all. This
is a bit unlike other parts of git, which would usually define git
remote list and then say that if no command is given, list is the
default.

But...

 I am proposing to add `git remote list` as a shortcut for `git remote -v`

This is somewhat different. I would have expected git remote list to
do the same thing as git remote (i.e., list without -v). I guess it
does not have to, though.

Perhaps -v should have been the default all along.  I do not use git
remote myself, so I don't know if -v is what most people use. But
changing the output of git remote now is probably a bad thing (I
expect some people may depend on parsing it to get the list of remotes;
they should probably use the git-config plumbing to do the same thing,
but it's actually rather tricky to do it that way).

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] bisect: save heap memory. allocate only the required amount

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 01:37:44PM +0100, Ramsay Jones wrote:

  On my 64-bit system using gcc, sizeof() returns 16; it has to pad the
  whole thing to 64-bit alignment in case I put two of them in an array.
  But offsetof(name) is 12, since the array of char does not need the same
  alignment; it can go right after the type and make use of the padding
  bits.
 
 Hmm, interesting. I must re-read the standard. I was convinced that the
 standard *requires* any alignment padding to come *before* the name field.
 (how would you put a, non-trivial, variable data structure into an array?)

I think you don't. How would you compute a[1] if a[0] has a variable
size?

You can put a flex-array structure into an array, but then each element
has the flex member as zero-size (and you should not access it, of
course).

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] bisect: save heap memory. allocate only the required amount

2014-08-26 Thread Ramsay Jones
On 26/08/14 13:43, Jeff King wrote:
 On Tue, Aug 26, 2014 at 01:37:44PM +0100, Ramsay Jones wrote:
 
 On my 64-bit system using gcc, sizeof() returns 16; it has to pad the
 whole thing to 64-bit alignment in case I put two of them in an array.
 But offsetof(name) is 12, since the array of char does not need the same
 alignment; it can go right after the type and make use of the padding
 bits.

 Hmm, interesting. I must re-read the standard. I was convinced that the
 standard *requires* any alignment padding to come *before* the name field.
 (how would you put a, non-trivial, variable data structure into an array?)
 
 I think you don't. How would you compute a[1] if a[0] has a variable
 size?
 
 You can put a flex-array structure into an array, but then each element
 has the flex member as zero-size (and you should not access it, of
 course).

Exactly. ;-)

ATB,
Ramsay Jones



--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Bug report: Author/Commit date in ISO 8601 format

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 09:29:44AM +0200, Oliver Busch wrote:

 I encountered the following problem: When using the date formatting option
 ISO (either by setting --date=iso or using format:%ci for the committer
 date), the output is formatted like this:
 
 2014-08-25 17:49:43 +0200
 
 But according to ISO 8601, should be formatted like this (see
 http://www.w3.org/TR/NOTE-datetime):
 
 2014-08-25T17:49:43+02:00

Yeah, it is not strictly ISO but more ISO-like (to further add
confusion, it is mostly RFC3339, which claims to be a profile of
ISO8601. But we don't follow the timezone conventions there. Yeesh).

Interestingly, this actually came up when the feature was added:

  http://thread.gmane.org/gmane.comp.version-control.git/52414/focus=52585

but there was some discussion of ISO8601's weird phrasing of T being
optional.

 I therefore suggest to adapt the output when using --date=iso or
 format:%ci to comply 100% with the ISO 8601 specs, or at least change
 documentation to say the output is only ISO-like.

I think changing the output at this point would cause backwards
compatibility problems (not to mention that it's a lot less readable for
humans).

Patches welcome for a documentation update. I also think something like
--date=iso8601-strict might make sense for the case of feeding the
result to another parser.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Bug report: Author/Commit date in ISO 8601 format

2014-08-26 Thread Oliver Busch
Peff is right, I did not think of backwards compatibility issues. I 
believe a new option like iso-strict as he suggested will do the trick 
(and I'm probably not the only one to appreciate its implementation...).


Regards,

Oliver

Am 26.08.2014 um 15:06 schrieb Jeff King:

On Tue, Aug 26, 2014 at 09:29:44AM +0200, Oliver Busch wrote:


I encountered the following problem: When using the date formatting option
ISO (either by setting --date=iso or using format:%ci for the committer
date), the output is formatted like this:

2014-08-25 17:49:43 +0200

But according to ISO 8601, should be formatted like this (see
http://www.w3.org/TR/NOTE-datetime):

2014-08-25T17:49:43+02:00

Yeah, it is not strictly ISO but more ISO-like (to further add
confusion, it is mostly RFC3339, which claims to be a profile of
ISO8601. But we don't follow the timezone conventions there. Yeesh).

Interestingly, this actually came up when the feature was added:

   http://thread.gmane.org/gmane.comp.version-control.git/52414/focus=52585

but there was some discussion of ISO8601's weird phrasing of T being
optional.


I therefore suggest to adapt the output when using --date=iso or
format:%ci to comply 100% with the ISO 8601 specs, or at least change
documentation to say the output is only ISO-like.

I think changing the output at this point would cause backwards
compatibility problems (not to mention that it's a lot less readable for
humans).

Patches welcome for a documentation update. I also think something like
--date=iso8601-strict might make sense for the case of feeding the
result to another parser.

-Peff




--

Oliver Busch, M.Eng.

Airport Research Center GmbH
Bismarckstraße 61
52066 Aachen
Germany

Phone: +49 241 16843-161
Fax: +49 241 16843-19
e-mail: oliver.bu...@arc-aachen.de
Website: http://www.airport-consultants.com

Register Court: Amtsgericht Aachen HRB 7313
Ust-Id-No.: DE196450052

Managing Directors:
Dipl.-Ing. Tom Alexander Heuer

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Bug report: Author/Commit date in ISO 8601 format

2014-08-26 Thread Oliver Busch
PS: As far as I understand it, there is no optionality of the T as 
an indicator for the start of the time part.


Am 26.08.2014 um 15:06 schrieb Jeff King:

On Tue, Aug 26, 2014 at 09:29:44AM +0200, Oliver Busch wrote:


I encountered the following problem: When using the date formatting option
ISO (either by setting --date=iso or using format:%ci for the committer
date), the output is formatted like this:

2014-08-25 17:49:43 +0200

But according to ISO 8601, should be formatted like this (see
http://www.w3.org/TR/NOTE-datetime):

2014-08-25T17:49:43+02:00

Yeah, it is not strictly ISO but more ISO-like (to further add
confusion, it is mostly RFC3339, which claims to be a profile of
ISO8601. But we don't follow the timezone conventions there. Yeesh).

Interestingly, this actually came up when the feature was added:

   http://thread.gmane.org/gmane.comp.version-control.git/52414/focus=52585

but there was some discussion of ISO8601's weird phrasing of T being
optional.


I therefore suggest to adapt the output when using --date=iso or
format:%ci to comply 100% with the ISO 8601 specs, or at least change
documentation to say the output is only ISO-like.

I think changing the output at this point would cause backwards
compatibility problems (not to mention that it's a lot less readable for
humans).

Patches welcome for a documentation update. I also think something like
--date=iso8601-strict might make sense for the case of feeding the
result to another parser.

-Peff




--

Oliver Busch, M.Eng.

Airport Research Center GmbH
Bismarckstraße 61
52066 Aachen
Germany

Phone: +49 241 16843-161
Fax: +49 241 16843-19
e-mail: oliver.bu...@arc-aachen.de
Website: http://www.airport-consultants.com

Register Court: Amtsgericht Aachen HRB 7313
Ust-Id-No.: DE196450052

Managing Directors:
Dipl.-Ing. Tom Alexander Heuer

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Bug report: Author/Commit date in ISO 8601 format

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 03:19:11PM +0200, Oliver Busch wrote:

 PS: As far as I understand it, there is no optionality of the T as an
 indicator for the start of the time part.

The standard says (and I am quoting from Wikipedia here, as I do not
have it myself):

  4.3.2 NOTE: By mutual agreement of the partners in information
  interchange, the character [T] may be omitted in applications where
  there is no risk of confusing a date and time of day representation
  with others defined in this International Standard.

But I am not sure that omitted means can be replaced with a space.
And while you can define by mutual agreement as git defines the
format, so any consumers agree to it that is not necessarily useful to
somebody who wants to feed the result to an iso8601 parser that does not
know or care about git (i.e., it shoves the conversion work onto the
person in the middle).

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


RE: Bug report: Author/Commit date in ISO 8601 format

2014-08-26 Thread Jason Pyeron
 -Original Message-
 From: Jeff King
 Sent: Tuesday, August 26, 2014 9:33
 
 On Tue, Aug 26, 2014 at 03:19:11PM +0200, Oliver Busch wrote:
 
  PS: As far as I understand it, there is no optionality of 
 the T as an
  indicator for the start of the time part.
 
 The standard says (and I am quoting from Wikipedia here, as I do not
 have it myself):
 
   4.3.2 NOTE: By mutual agreement of the partners in information
   interchange, the character [T] may be omitted in applications where
   there is no risk of confusing a date and time of day representation
   with others defined in this International Standard.

From ISO 8601:2004

4.3.2 Complete representations
The time elements of a date and time of day expression shall be written in the 
following sequence.
a) For calendar dates:
year - month - day of the month - time designator - hour - minute - second - 
zone designator
b) For ordinal dates:
year - day of the year - time designator - hour - minute - second - zone 
designator
c) For week dates:
year - week designator - week - day of the week - time designator - hour - 
minute - second - zone
designator

The zone designator is empty if use is made of local time in accordance with 
4.2.2.2 through 4.2.2.4, it is the
UTC designator [Z] if use is made of UTC of day in accordance with 4.2.4 and it 
is the difference-component if
use is made of local time and the difference from UTC in accordance with 
4.2.5.2.
The character [T] shall be used as time designator to indicate the start of the 
representation of the time of day
component in these expressions. The hyphen [-] and the colon [:] shall be used, 
in accordance with 4.4.4, as
separators within the date and time of day expressions, respectively, when 
required.

NOTE By mutual agreement of the partners in information interchange, the 
character [T] may be omitted in
applications where there is no risk of confusing a date and time of day 
representation with others defined in this
International Standard.

 
 But I am not sure that omitted means can be replaced with a space.
 And while you can define by mutual agreement as git defines the
 format, so any consumers agree to it that is not necessarily 
 useful to
 somebody who wants to feed the result to an iso8601 parser 
 that does not
 know or care about git (i.e., it shoves the conversion work onto the
 person in the middle).

Omitted /T?/ does not mean replaced with another character.

--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-   -
- Jason Pyeron  PD Inc. http://www.pdinc.us -
- Principal Consultant  10 West 24th Street #100-
- +1 (443) 269-1555 x333Baltimore, Maryland 21218   -
-   -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] contrib/svn-fe: fix Makefile

2014-08-26 Thread Maxim Bublis
Fixes several problems:
  * include config.mak.uname, config.mak.autogen and config.mak
in order to use settings for prefix and other such things;
  * link xdiff/lib.a as it is a requirement for libgit.a;
  * fix CFLAGS and EXTLIBS for Linux and Mac OS X.
---
 contrib/svn-fe/Makefile | 47 ++-
 1 file changed, 38 insertions(+), 9 deletions(-)

diff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile
index 360d8da..8e1d622 100644
--- a/contrib/svn-fe/Makefile
+++ b/contrib/svn-fe/Makefile
@@ -1,18 +1,45 @@
 all:: svn-fe$X
 
-CC = gcc
+CC = cc
 RM = rm -f
 MV = mv
 
 CFLAGS = -g -O2 -Wall
 LDFLAGS =
-ALL_CFLAGS = $(CFLAGS)
-ALL_LDFLAGS = $(LDFLAGS)
-EXTLIBS =
+EXTLIBS = -lz
+
+include ../../config.mak.uname
+-include ../../config.mak.autogen
+-include ../../config.mak
+
+ifeq ($(uname_S),Darwin)
+   CFLAGS += -I/opt/local/include
+   LDFLAGS += -L/opt/local/lib
+endif
+
+ifndef NO_OPENSSL
+   EXTLIBS += -lssl -lcrypto
+endif
+
+ifndef NO_PTHREADS
+   CFLAGS += $(PTHREADS_CFLAGS)
+   EXTLIBS += $(PTHREAD_LIBS)
+endif
+
+ifdef HAVE_CLOCK_GETTIME
+   CFLAGS += -DHAVE_CLOCK_GETTIME
+   EXTLIBS += -lrt
+endif
+
+ifdef NEEDS_LIBICONV
+   EXTLIBS += -liconv
+endif
 
 GIT_LIB = ../../libgit.a
 VCSSVN_LIB = ../../vcs-svn/lib.a
-LIBS = $(VCSSVN_LIB) $(GIT_LIB) $(EXTLIBS)
+XDIFF_LIB = ../../xdiff/lib.a
+
+LIBS = $(VCSSVN_LIB) $(GIT_LIB) $(XDIFF_LIB)
 
 QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir
 QUIET_SUBDIR1 =
@@ -33,12 +60,11 @@ ifndef V
 endif
 endif
 
-svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB)
-   $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ svn-fe.o \
-   $(ALL_LDFLAGS) $(LIBS)
+svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(XDIFF_LIB) $(GIT_LIB)
+   $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) $(EXTLIBS) -o $@ svn-fe.o 
$(LIBS)
 
 svn-fe.o: svn-fe.c ../../vcs-svn/svndump.h
-   $(QUIET_CC)$(CC) -I../../vcs-svn -o $*.o -c $(ALL_CFLAGS) $
+   $(QUIET_CC)$(CC) $(CFLAGS) -I../../vcs-svn -o $*.o -c $
 
 svn-fe.html: svn-fe.txt
$(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \
@@ -54,6 +80,9 @@ svn-fe.1: svn-fe.txt
 ../../vcs-svn/lib.a: FORCE
$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) vcs-svn/lib.a
 
+../../xdiff/lib.a: FORCE
+   $(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) xdiff/lib.a
+
 ../../libgit.a: FORCE
$(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) libgit.a
 
-- 
1.8.5.2 (Apple Git-48)

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Bug report: Author/Commit date in ISO 8601 format

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 10:10:33AM -0400, Jason Pyeron wrote:

  But I am not sure that omitted means can be replaced with a space.
  And while you can define by mutual agreement as git defines the
  format, so any consumers agree to it that is not necessarily 
  useful to
  somebody who wants to feed the result to an iso8601 parser 
  that does not
  know or care about git (i.e., it shoves the conversion work onto the
  person in the middle).
 
 Omitted /T?/ does not mean replaced with another character.

I would agree. But that is the argument made in the thread I linked
earlier.

I do not think there is much point in re-opening the argument, though.
Whatever git generates, changing the output would probably cause a lot
of pain.  We are likely better off adding a new, real iso8601 format
option (we can even deprecate the old one, or slate it for switching,
but we would need a notification period).

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Re: Re: Relative submodule URLs

2014-08-26 Thread Robert Dailey
On Tue, Aug 26, 2014 at 1:28 AM, Heiko Voigt hvo...@hvoigt.net wrote:
 Hi Heiko,

 My last email response was in violation of your request to keep the
 two topics separate, sorry about that. I started typing it this
 weekend and completed the draft this morning, without having read this
 response from you first.

 Thats fine, no problem.

 Here is what I think would make the feature most usable. I think you
 went over some of these ideas but I just want to clarify, to make sure
 we're on the same page. Please correct me as needed.

 1. Running `git submodule update --with-remote name` shall fail the
 command unconditionally.

 I am not sure but I think you mean

 git submodule update --with-remote=name

 With the equals sign, without it you would name the submodule paths to
 update. No I think that should just add the remote name to all
 submodules that would be updated and do the normal update operation on
 them (with the new remote of course).

I'm not sure about Linux but at least with msysgit on Windows, typing
a two-dash option (such as --with-remote) forces command line
evaluation to use the next placement parameter as the parameter for
it. I've seen this work the same way with argparse in python too. In
my experience, command line has worked that way, I'm not sure if that
is by design or not though. I never use equal signs with git commands,
never had a problem for some reason.

For example:

git rebase --onto release/1.0 head~3 head

The `--onto` option knows to use `release/1.0` as its parameter.

 2. Using the `--with-remote` option on submodule `update` or `sync`
 will fail if it detects absolute submodule URLs in .gitmodule

 Yes, almost. Since you can have a mixture I suggest to only fail if the
 submodules that would be processed have an absolute url in them. If
 processed submodules are all relative it can go ahead.

For example if it processes 3 submodules in the following order:

1. relative
2. absolute
3. relative

Should it fail before or after processing the 3rd relative submodule?
I was thinking it would fail while trying to sync/update the 2nd one
(which is absolute) and stop before processing the 3rd.

 3. Running `git submodule update --init --with-remote name` shall
 fail the command ONLY if a submodule is being processed that is NOT
 also being initialized.

 No since the --init flag just tells update to initialize submodules
 on-demand. It should just go ahead the same way as without
 --with-remote.

But doesn't the on-demand initialization need to evaluate relative
URLs and convert them to absolute based on the .gitmodules
configuration? I thought the idea was to make `--with-remote` invalid
for initialization/sync of absolute URLs.

In other words if I did:

git submodule init --with-remote fork my-submodule-dir

and if my-submodule-dir was not relative in .gitmodules, then the
`--with-remote` flag becomes useless. We could fail silently but for
educational purposes to the user I thought we were failing in these
scenarios. Maybe I misunderstood your original intent with the
failures? Is init not doing the relative to absolute evaluation like
I'm thinking? Please correct me where I'm wrong.

 4. The behavior of git submodule's `update` or `sync` commands
 combined with `--with-remote` will REPLACE or CREATE the 'origin'
 remote in each submodule it is run in. We will not allow the user to
 configure what the submodule remote name will end up being (I think
 this is current behavior and forces good practice; I consider `origin`
 an adopted standard for git, and actually wish it was more enforced
 for super projects as well!)

 No please carefully read my email again. I specifically was describing
 the opposite. --with-remote=name creates/replaces the remote name in
 the submodule. I do not see a benefit in restricting the user from
 creating different remote names in the submodule. I think it would be
 more confusing if the remote 'origin' in the superproject does not point
 to the same location as 'origin' in the submodule.

Well the reason why I said it would be 'origin' is so that the
submodule knows which remote to use internally during an update. I'm
assuming 'update' uses 'origin' internally in the submodule to know
which remote to pull from. My understanding of how `git submodule
update` knows which URL to pull from is probably incorrect. I'm not
familiar on the internal mechanics of how this works. Perhaps you
could explain or send me to some reading material on it?
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v6 2/6] Add git_env_ulong() to parse environment variable

2014-08-26 Thread Steffen Prohaska
The new function will be used to parse GIT_MMAP_LIMIT and
GIT_ALLOC_LIMIT.

Signed-off-by: Steffen Prohaska proha...@zib.de
---
 cache.h  |  1 +
 config.c | 11 +++
 2 files changed, 12 insertions(+)

diff --git a/cache.h b/cache.h
index fcb511d..b820b6a 100644
--- a/cache.h
+++ b/cache.h
@@ -1318,6 +1318,7 @@ extern int git_config_rename_section_in_file(const char 
*, const char *, const c
 extern const char *git_etc_gitconfig(void);
 extern int check_repository_format_version(const char *var, const char *value, 
void *cb);
 extern int git_env_bool(const char *, int);
+extern unsigned long git_env_ulong(const char *, unsigned long);
 extern int git_config_system(void);
 extern int config_error_nonbool(const char *);
 #if defined(__GNUC__)
diff --git a/config.c b/config.c
index 058505c..178721f 100644
--- a/config.c
+++ b/config.c
@@ -1122,6 +1122,17 @@ int git_env_bool(const char *k, int def)
return v ? git_config_bool(k, v) : def;
 }
 
+/*
+ * Use default if environment variable is unset or empty string.
+ */
+unsigned long git_env_ulong(const char *k, unsigned long val)
+{
+   const char *v = getenv(k);
+   if (v  *v  !git_parse_ulong(v, val))
+   die(failed to parse %s, k);
+   return val;
+}
+
 int git_config_system(void)
 {
return !git_env_bool(GIT_CONFIG_NOSYSTEM, 0);
-- 
2.1.0.8.gf3a29c8

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v6 5/6] Change copy_fd() to not close input fd

2014-08-26 Thread Steffen Prohaska
The caller opened the fd, so it should be responsible for closing it.

Signed-off-by: Steffen Prohaska proha...@zib.de
---
 copy.c | 5 +
 lockfile.c | 3 +++
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/copy.c b/copy.c
index a7f58fd..d0a1d82 100644
--- a/copy.c
+++ b/copy.c
@@ -10,7 +10,6 @@ int copy_fd(int ifd, int ofd)
break;
if (len  0) {
int read_error = errno;
-   close(ifd);
return error(copy-fd: read returned %s,
 strerror(read_error));
}
@@ -21,17 +20,14 @@ int copy_fd(int ifd, int ofd)
len -= written;
}
else if (!written) {
-   close(ifd);
return error(copy-fd: write returned 0);
} else {
int write_error = errno;
-   close(ifd);
return error(copy-fd: write returned %s,
 strerror(write_error));
}
}
}
-   close(ifd);
return 0;
 }
 
@@ -60,6 +56,7 @@ int copy_file(const char *dst, const char *src, int mode)
return fdo;
}
status = copy_fd(fdi, fdo);
+   close(fdi);
if (close(fdo) != 0)
return error(%s: close error: %s, dst, strerror(errno));
 
diff --git a/lockfile.c b/lockfile.c
index 2564a7f..2448d30 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -224,8 +224,11 @@ int hold_lock_file_for_append(struct lock_file *lk, const 
char *path, int flags)
} else if (copy_fd(orig_fd, fd)) {
if (flags  LOCK_DIE_ON_ERROR)
exit(128);
+   close(orig_fd);
close(fd);
return -1;
+   } else {
+   close(orig_fd);
}
return fd;
 }
-- 
2.1.0.8.gf3a29c8

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v6 1/6] convert: drop arguments other than 'path' from would_convert_to_git()

2014-08-26 Thread Steffen Prohaska
It is only the path that matters in the decision whether to filter or
not.  Clarify this by making path the single argument of
would_convert_to_git().

Signed-off-by: Steffen Prohaska proha...@zib.de
---
 convert.h   | 5 ++---
 sha1_file.c | 2 +-
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/convert.h b/convert.h
index 0c2143c..c638b33 100644
--- a/convert.h
+++ b/convert.h
@@ -40,10 +40,9 @@ extern int convert_to_working_tree(const char *path, const 
char *src,
   size_t len, struct strbuf *dst);
 extern int renormalize_buffer(const char *path, const char *src, size_t len,
  struct strbuf *dst);
-static inline int would_convert_to_git(const char *path, const char *src,
-  size_t len, enum safe_crlf checksafe)
+static inline int would_convert_to_git(const char *path)
 {
-   return convert_to_git(path, src, len, NULL, checksafe);
+   return convert_to_git(path, NULL, 0, NULL, 0);
 }
 
 /*
diff --git a/sha1_file.c b/sha1_file.c
index 3f70b1d..00c07f2 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -3144,7 +3144,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,
if (!S_ISREG(st-st_mode))
ret = index_pipe(sha1, fd, type, path, flags);
else if (size = big_file_threshold || type != OBJ_BLOB ||
-(path  would_convert_to_git(path, NULL, 0, 0)))
+(path  would_convert_to_git(path)))
ret = index_core(sha1, fd, size, type, path, flags);
else
ret = index_stream(sha1, fd, size, type, path, flags);
-- 
2.1.0.8.gf3a29c8

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v6 4/6] Introduce GIT_MMAP_LIMIT to allow testing expected mmap size

2014-08-26 Thread Steffen Prohaska
Similar to testing expectations about malloc with GIT_ALLOC_LIMIT (see
commit d41489 and previous commit), it can be useful to test
expectations about mmap.

This introduces a new environment variable GIT_MMAP_LIMIT to limit the
largest allowed mmap length.  xmmap() is modified to check the limit.
Together with GIT_ALLOC_LIMIT tests can now easily confirm expectations
about memory consumption.

GIT_MMAP_LIMIT will be used in the next commit to test that data will be
streamed to an external filter without mmaping the entire file.

[commit d41489]: d41489a6424308dc9a0409bc2f6845aa08bd4f7d Add more large
blob test cases

Signed-off-by: Steffen Prohaska proha...@zib.de
---
 sha1_file.c | 18 +-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/sha1_file.c b/sha1_file.c
index 00c07f2..d9b5157 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -663,10 +663,26 @@ void release_pack_memory(size_t need)
; /* nothing */
 }
 
+static void mmap_limit_check(size_t length)
+{
+   static size_t limit = 0;
+   if (!limit) {
+   limit = git_env_ulong(GIT_MMAP_LIMIT, 0);
+   if (!limit)
+   limit = SIZE_MAX;
+   }
+   if (length  limit)
+   die(attempting to mmap %PRIuMAX over limit %PRIuMAX,
+   (uintmax_t)length, (uintmax_t)limit);
+}
+
 void *xmmap(void *start, size_t length,
int prot, int flags, int fd, off_t offset)
 {
-   void *ret = mmap(start, length, prot, flags, fd, offset);
+   void *ret;
+
+   mmap_limit_check(length);
+   ret = mmap(start, length, prot, flags, fd, offset);
if (ret == MAP_FAILED) {
if (!length)
return NULL;
-- 
2.1.0.8.gf3a29c8

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v6 3/6] Change GIT_ALLOC_LIMIT check to use git_env_ulong()

2014-08-26 Thread Steffen Prohaska
GIT_ALLOC_LIMIT limits xmalloc()'s size, which is of type size_t.
Better use git_env_ulong() to parse the environment variable, so that
the postfixes 'k', 'm', and 'g' can be used; and use size_t to store the
limit for consistency.  The change to size_t has no direct practical
impact, because we use GIT_ALLOC_LIMIT to test small sizes.

The cast of size in the call to die() is changed to uintmax_t to match
the format string PRIuMAX.

Signed-off-by: Steffen Prohaska proha...@zib.de
---
 t/t1050-large.sh |  2 +-
 wrapper.c| 15 ---
 2 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/t/t1050-large.sh b/t/t1050-large.sh
index aea4936..e7657ab 100755
--- a/t/t1050-large.sh
+++ b/t/t1050-large.sh
@@ -13,7 +13,7 @@ test_expect_success setup '
echo X | dd of=large2 bs=1k seek=2000 
echo X | dd of=large3 bs=1k seek=2000 
echo Y | dd of=huge bs=1k seek=2500 
-   GIT_ALLOC_LIMIT=1500 
+   GIT_ALLOC_LIMIT=1500k 
export GIT_ALLOC_LIMIT
 '
 
diff --git a/wrapper.c b/wrapper.c
index bc1bfb8..c5204f7 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -11,14 +11,15 @@ static void (*try_to_free_routine)(size_t size) = 
do_nothing;
 
 static void memory_limit_check(size_t size)
 {
-   static int limit = -1;
-   if (limit == -1) {
-   const char *env = getenv(GIT_ALLOC_LIMIT);
-   limit = env ? atoi(env) * 1024 : 0;
+   static size_t limit = 0;
+   if (!limit) {
+   limit = git_env_ulong(GIT_ALLOC_LIMIT, 0);
+   if (!limit)
+   limit = SIZE_MAX;
}
-   if (limit  size  limit)
-   die(attempting to allocate %PRIuMAX over limit %d,
-   (intmax_t)size, limit);
+   if (size  limit)
+   die(attempting to allocate %PRIuMAX over limit %PRIuMAX,
+   (uintmax_t)size, (uintmax_t)limit);
 }
 
 try_to_free_t set_try_to_free_routine(try_to_free_t routine)
-- 
2.1.0.8.gf3a29c8

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v6 6/6] convert: stream from fd to required clean filter to reduce used address space

2014-08-26 Thread Steffen Prohaska
The data is streamed to the filter process anyway.  Better avoid mapping
the file if possible.  This is especially useful if a clean filter
reduces the size, for example if it computes a sha1 for binary data,
like git media.  The file size that the previous implementation could
handle was limited by the available address space; large files for
example could not be handled with (32-bit) msysgit.  The new
implementation can filter files of any size as long as the filter output
is small enough.

The new code path is only taken if the filter is required.  The filter
consumes data directly from the fd.  If it fails, the original data is
not immediately available.  The condition can easily be handled as
a fatal error, which is expected for a required filter anyway.

If the filter was not required, the condition would need to be handled
in a different way, like seeking to 0 and reading the data.  But this
would require more restructuring of the code and is probably not worth
it.  The obvious approach of falling back to reading all data would not
help achieving the main purpose of this patch, which is to handle large
files with limited address space.  If reading all data is an option, we
can simply take the old code path right away and mmap the entire file.

The environment variable GIT_MMAP_LIMIT, which has been introduced in
a previous commit is used to test that the expected code path is taken.
A related test that exercises required filters is modified to verify
that the data actually has been modified on its way from the file system
to the object store.

Signed-off-by: Steffen Prohaska proha...@zib.de
---
 convert.c | 55 +--
 convert.h |  5 +
 sha1_file.c   | 27 -
 t/t0021-conversion.sh | 24 +-
 4 files changed, 99 insertions(+), 12 deletions(-)

diff --git a/convert.c b/convert.c
index cb5fbb4..677d339 100644
--- a/convert.c
+++ b/convert.c
@@ -312,11 +312,12 @@ static int crlf_to_worktree(const char *path, const char 
*src, size_t len,
 struct filter_params {
const char *src;
unsigned long size;
+   int fd;
const char *cmd;
const char *path;
 };
 
-static int filter_buffer(int in, int out, void *data)
+static int filter_buffer_or_fd(int in, int out, void *data)
 {
/*
 * Spawn cmd and feed the buffer contents through its stdin.
@@ -355,7 +356,12 @@ static int filter_buffer(int in, int out, void *data)
 
sigchain_push(SIGPIPE, SIG_IGN);
 
-   write_err = (write_in_full(child_process.in, params-src, params-size) 
 0);
+   if (params-src) {
+   write_err = (write_in_full(child_process.in, params-src, 
params-size)  0);
+   } else {
+   write_err = copy_fd(params-fd, child_process.in);
+   }
+
if (close(child_process.in))
write_err = 1;
if (write_err)
@@ -371,7 +377,7 @@ static int filter_buffer(int in, int out, void *data)
return (write_err || status);
 }
 
-static int apply_filter(const char *path, const char *src, size_t len,
+static int apply_filter(const char *path, const char *src, size_t len, int fd,
 struct strbuf *dst, const char *cmd)
 {
/*
@@ -392,11 +398,12 @@ static int apply_filter(const char *path, const char 
*src, size_t len,
return 1;
 
memset(async, 0, sizeof(async));
-   async.proc = filter_buffer;
+   async.proc = filter_buffer_or_fd;
async.data = params;
async.out = -1;
params.src = src;
params.size = len;
+   params.fd = fd;
params.cmd = cmd;
params.path = path;
 
@@ -747,6 +754,25 @@ static void convert_attrs(struct conv_attrs *ca, const 
char *path)
}
 }
 
+int would_convert_to_git_filter_fd(const char *path)
+{
+   struct conv_attrs ca;
+
+   convert_attrs(ca, path);
+   if (!ca.drv)
+   return 0;
+
+   /*
+* Apply a filter to an fd only if the filter is required to succeed.
+* We must die if the filter fails, because the original data before
+* filtering is not available.
+*/
+   if (!ca.drv-required)
+   return 0;
+
+   return apply_filter(path, NULL, 0, -1, NULL, ca.drv-clean);
+}
+
 int convert_to_git(const char *path, const char *src, size_t len,
struct strbuf *dst, enum safe_crlf checksafe)
 {
@@ -761,7 +787,7 @@ int convert_to_git(const char *path, const char *src, 
size_t len,
required = ca.drv-required;
}
 
-   ret |= apply_filter(path, src, len, dst, filter);
+   ret |= apply_filter(path, src, len, -1, dst, filter);
if (!ret  required)
die(%s: clean filter '%s' failed, path, ca.drv-name);
 
@@ -778,6 +804,23 @@ int convert_to_git(const char *path, const char *src, 
size_t len,
return ret | ident_to_git(path, src, len, dst, 

[PATCH v6 0/6] Stream fd to clean filter; GIT_MMAP_LIMIT, GIT_ALLOC_LIMIT with git_env_ulong()

2014-08-26 Thread Steffen Prohaska
Changes since v5:

 - Introduce and use git_env_ulong().
 - Change copy_fd() to not close input fd, which simplified changes to convert.
 - More detailed explanation why filter must be marked required in commit
   message of 6/6.
 - Style fixes.

Steffen Prohaska (6):
  convert: drop arguments other than 'path' from would_convert_to_git()
  Add git_env_ulong() to parse environment variable
  Change GIT_ALLOC_LIMIT check to use git_env_ulong()
  Introduce GIT_MMAP_LIMIT to allow testing expected mmap size
  Change copy_fd() to not close input fd
  convert: stream from fd to required clean filter to reduce used
address space

 cache.h   |  1 +
 config.c  | 11 +++
 convert.c | 55 +--
 convert.h | 10 +++---
 copy.c|  5 +
 lockfile.c|  3 +++
 sha1_file.c   | 47 ---
 t/t0021-conversion.sh | 24 +-
 t/t1050-large.sh  |  2 +-
 wrapper.c | 15 +++---
 10 files changed, 144 insertions(+), 29 deletions(-)

-- 
2.1.0.8.gf3a29c8

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Improving the git remote command

2014-08-26 Thread Philippe Vaucher
 Perhaps -v should have been the default all along.  I do not use git
 remote myself, so I don't know if -v is what most people use. But
 changing the output of git remote now is probably a bad thing (I
 expect some people may depend on parsing it to get the list of remotes;
 they should probably use the git-config plumbing to do the same thing,
 but it's actually rather tricky to do it that way).

Just to be clear, the proposal is not about changing the output of git remote.

Anyway, it got me curious about other git commands reguarding list,
and I was very surprised because I couldn't find another one. I mean
git remote actually behaves like git branch and git tag. I have
no clue why I expect list to work with git remote.

It's probably because git branch and git tag expect a name, and
there list can only be expressed by no name or with some flags. On
the other hand, git remote expects a subcommand (add, delete, etc)
and there what logically maps to list is the subcommand list, no
name being more expected to produce a list of the subcommands.

Philippe
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Improving the git remote command

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 06:19:20PM +0200, Philippe Vaucher wrote:

  Perhaps -v should have been the default all along.  I do not use git
  remote myself, so I don't know if -v is what most people use. But
  changing the output of git remote now is probably a bad thing (I
  expect some people may depend on parsing it to get the list of remotes;
  they should probably use the git-config plumbing to do the same thing,
  but it's actually rather tricky to do it that way).
 
 Just to be clear, the proposal is not about changing the output of
 git remote.

I know. But we are left with three options:

  1. Add git remote list with verbose output. This is bad because it
 differs gratuitously from git remote.

  2. Add git remote list with non-verbose output. This is good because
 it means git remote is just a shortcut for git remote list,
 which is consistent with other parts of git. But it is potentially
 bad if -v is a better output format.

  3. Add git remote list with verbose output, and tweak git remote
 to match. This is bad because it breaks backwards compatibility.

The proposal is for (1). I think we agree that (3) is out. The question
is whether (1) or (2) is the least bad.

 Anyway, it got me curious about other git commands reguarding list,
 and I was very surprised because I couldn't find another one. I mean
 git remote actually behaves like git branch and git tag. I have
 no clue why I expect list to work with git remote.

Branch and tag take --list. Remote is the odd one out in that its
subcommands do not have dashes. git-stash also takes commands without
dashes (and has a list command), but its default mode is to create a
stash, not to list.

 It's probably because git branch and git tag expect a name, and
 there list can only be expressed by no name or with some flags. On
 the other hand, git remote expects a subcommand (add, delete, etc)
 and there what logically maps to list is the subcommand list, no
 name being more expected to produce a list of the subcommands.

Yeah. Branch and tag need dashed subcommands because otherwise it is
ambiguous with creating tag called list, functionality that existed
before --list was added. Git-remote was defined with subcommands from
day one, so it can get away with it. Git-stash is sort of in the
category as git-remote there, except that save can actually take an
argument. So to provide it you can't say git stash foobar, but instead
have to say git stash save foobar (it actually used to allow the
former, but you can imagine the annoyance when you typo git stash
lsit).

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/2] Check order when reading index

2014-08-26 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 With natively do you mean some internal state that is never written
 into the index? If this were the case then there wouldn't be any
 problem with the restriction when reading the index file.

 FWIW, that was my question on reading Junio's response, too.

The current code may not put two stage #1 entries for the same path
but allowing such entries was a part of the design from very early
days; iow it is a valid index file, unlike the one that has both
stage #0 and stage #1 for the same path.  It is a no-no to reject
such an index as long as we are discussing to add a new code to
tighten the sanity check on the file content.

 I have also tried to reproduce it by directly calling
 git-merge-recursive and the most I have got is what it seemed to be
 like a conflict in the stage #1:
 
 $ git ls-files -s
 100644 1036ba5101378f06aa10c5fa249b67e14f83166b 1 conflict
 100644 2638c45f8e7bc5b56f70e92390153572811782c3 2 conflict
 100644 178481050188cf00d7d9cd5a11e43ab8fab9294f 3 conflict
 
 $ git cat-file blob 1036ba5101378f06aa10c5fa249b67e14f83166b
  Temporary merge branch 1
 G
 ===
 E
 F
  Temporary merge branch 2

 Yes, I think merge-recursive resolves the earlier merges and then feeds
 the result into the main merge. And that's how you end up with the
 temporary merge branch name instead of something useful.

Yes---that is what I meant by the virtual stuff.  Unlike resolve
work by Daniel (around Sep 2005 $gmane/8088) that tried to use
multiple ancestors directly in a single merge, recursive limits
itself to repeated use of pairwise merges.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 2/3] Makefile: use `find` to determine static header dependencies

2014-08-26 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 Actually as you are not excluding CVS, RCS, etc., and using ??* as
 the starting point will exclude .git, .hg, etc. at the top, I think
 we can shorten it even further and say
 
  find ??* -name Documentation -prune -o -name \*.h
 
 or something.

 I had originally considered starting with find *, but I was worried
 about shell globbing overflowing command-line limits here. echo * on a
 built tree is about 12K.

OK.  What I queued is still your original which is the most
conservative among various fun alternatives we have seen so far on
this thread, so we should be good ;-)

Thanks.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] po/TEAMS: add new members to German translation team

2014-08-26 Thread Ralf Thielow
Signed-off-by: Ralf Thielow ralf.thie...@gmail.com
---
 po/TEAMS | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/po/TEAMS b/po/TEAMS
index 52c3d07..461cc14 100644
--- a/po/TEAMS
+++ b/po/TEAMS
@@ -15,6 +15,8 @@ Leader:   Ralf Thielow 
ralf.thie...@googlemail.com
 Members:   Thomas Rast t...@thomasrast.ch
Jan Krüger j...@jk.gs
Christian Stimming stimm...@tuhh.de
+   Phillip Szelat phillip.sze...@gmail.com
+   Matthias Rüster matthias.rues...@gmail.com
 
 Language:  fr (French)
 Repository:https://github.com/jnavila/git
-- 
2.1.0.240.g9cce596

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v6 5/6] Change copy_fd() to not close input fd

2014-08-26 Thread Junio C Hamano
Steffen Prohaska proha...@zib.de writes:

 The caller opened the fd, so it should be responsible for closing it.

H, I am not sure what the benefit of such a dogmatism.  This
function consumes all that is useful in the fd, and there is nothing
interesting that can be do further on it, no?

Ah, wait.  The caller could choose to rewind and reuse the contents,
and it is very selfish of this function to unilaterally declare that
once you give an fd to it you cannot do anything with it laster.

So I think this is a good change, but the justification is not
quite.  It is not the caller should be responsible; it is more
about the callee did not open it, does not own it, and should allow
the caller, if it chooses, reuse it by seeking after the callee is
done.

 Subject: Re: [PATCH v6 5/6] Change copy_fd() to not close input fd

Let's follow the area: description convention here, too, e.g.

copy_fd(): allow callers to work on input fd after it returns

or something.

Thanks.

  copy.c | 5 +
  lockfile.c | 3 +++
  2 files changed, 4 insertions(+), 4 deletions(-)

 diff --git a/copy.c b/copy.c
 index a7f58fd..d0a1d82 100644
 --- a/copy.c
 +++ b/copy.c
 @@ -10,7 +10,6 @@ int copy_fd(int ifd, int ofd)
   break;
   if (len  0) {
   int read_error = errno;
 - close(ifd);
   return error(copy-fd: read returned %s,
strerror(read_error));
   }
 @@ -21,17 +20,14 @@ int copy_fd(int ifd, int ofd)
   len -= written;
   }
   else if (!written) {
 - close(ifd);
   return error(copy-fd: write returned 0);
   } else {
   int write_error = errno;
 - close(ifd);
   return error(copy-fd: write returned %s,
strerror(write_error));
   }
   }
   }
 - close(ifd);
   return 0;
  }
  
 @@ -60,6 +56,7 @@ int copy_file(const char *dst, const char *src, int mode)
   return fdo;
   }
   status = copy_fd(fdi, fdo);
 + close(fdi);
   if (close(fdo) != 0)
   return error(%s: close error: %s, dst, strerror(errno));
  
 diff --git a/lockfile.c b/lockfile.c
 index 2564a7f..2448d30 100644
 --- a/lockfile.c
 +++ b/lockfile.c
 @@ -224,8 +224,11 @@ int hold_lock_file_for_append(struct lock_file *lk, 
 const char *path, int flags)
   } else if (copy_fd(orig_fd, fd)) {
   if (flags  LOCK_DIE_ON_ERROR)
   exit(128);
 + close(orig_fd);
   close(fd);
   return -1;
 + } else {
 + close(orig_fd);
   }
   return fd;
  }
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/2] Check order when reading index

2014-08-26 Thread Jaime Soriano Pastor
On Tue, Aug 26, 2014 at 6:53 PM, Junio C Hamano gits...@pobox.com wrote:

 Yes---that is what I meant by the virtual stuff.  Unlike resolve
 work by Daniel (around Sep 2005 $gmane/8088) that tried to use
 multiple ancestors directly in a single merge, recursive limits
 itself to repeated use of pairwise merges.

Ok, I see now. Then what about checking also in check_ce_order() that
only stage #1 can be repeated?
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Improving the git remote command

2014-08-26 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 ... But we are left with three options:

   1. Add git remote list with verbose output. This is bad because it
  differs gratuitously from git remote.

   2. Add git remote list with non-verbose output. This is good because
  it means git remote is just a shortcut for git remote list,
  which is consistent with other parts of git. But it is potentially
  bad if -v is a better output format.

   3. Add git remote list with verbose output, and tweak git remote
  to match. This is bad because it breaks backwards compatibility.

 The proposal is for (1). I think we agree that (3) is out. The question
 is whether (1) or (2) is the least bad.

I would imagine that those who want list of remotes programatically
would read from git config output and it would be with less
friction to change the output from git remote, a command that is
solely to cater to end-user humans, to suit people's needs, so I am
not sure if (3) is immediately out.

Having said that, my preference is 

0. Do nothing, but document the default to listing better if
   needed.

and then 2. above, and then 1.

 Yeah. Branch and tag need dashed subcommands because otherwise it is
 ambiguous with creating tag called list, functionality that existed
 before --list was added. Git-remote was defined with subcommands from
 day one, so it can get away with it. Git-stash is sort of in the
 category as git-remote there, except that save can actually take an
 argument. So to provide it you can't say git stash foobar, but instead
 have to say git stash save foobar (it actually used to allow the
 former, but you can imagine the annoyance when you typo git stash
 lsit).

Yeah, and there also is this one:

  http://thread.gmane.org/gmane.comp.version-control.git/231376/focus=231478

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 2/3] Makefile: use `find` to determine static header dependencies

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 09:54:19AM -0700, Junio C Hamano wrote:

 Jeff King p...@peff.net writes:
 
  Actually as you are not excluding CVS, RCS, etc., and using ??* as
  the starting point will exclude .git, .hg, etc. at the top, I think
  we can shorten it even further and say
  
 find ??* -name Documentation -prune -o -name \*.h
  
  or something.
 
  I had originally considered starting with find *, but I was worried
  about shell globbing overflowing command-line limits here. echo * on a
  built tree is about 12K.
 
 OK.  What I queued is still your original which is the most
 conservative among various fun alternatives we have seen so far on
 this thread, so we should be good ;-)

The only thing I think mine does not do that Jonathan suggested is
dropping .hg, etc. I do not know why anyone would track git in hg, but
it might make sense to s/.git/.?/ in what I sent.

(I noticed also that you did not queue the third patch to drop
CHECK_HEADER_DEPENDENCIES; I'm OK if we keep it, but I wanted to make
sure it was not an oversight).

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] l10n: de.po: translate 38 new messages

2014-08-26 Thread Ralf Thielow
Translate 38 new messages came from git.pot update in fe05e19
(l10n: git.pot: v2.1.0 round 1 (38 new, 9 removed)).

Signed-off-by: Ralf Thielow ralf.thie...@gmail.com
---
This is a resend of an earlier patch [1] with an extended
recipient list.

[1]
http://article.gmane.org/gmane.comp.version-control.git/254815

 po/de.po | 131 ++-
 1 file changed, 62 insertions(+), 69 deletions(-)

diff --git a/po/de.po b/po/de.po
index b8fb23f..ffc556c 100644
--- a/po/de.po
+++ b/po/de.po
@@ -23,16 +23,14 @@ msgid hint: %.*s\n
 msgstr Hinweis: %.*s\n
 
 #: advice.c:88
-#, fuzzy
 msgid 
 Fix them up in the work tree, and then use 'git add/rm file'\n
 as appropriate to mark resolution and make a commit, or use\n
 'git commit -a'.
 msgstr 
-Korrigieren Sie dies im Arbeitsverzeichnis,\n
-und benutzen Sie dann 'git add/rm Datei'\n
-um die Auflösung entsprechend zu markieren und zu committen,\n
-oder benutzen Sie 'git commit -a'.
+Korrigieren Sie dies im Arbeitsverzeichnis, und benutzen Sie\n
+dann 'git add/rm Datei' um die Auflösung entsprechend zu markieren\n
+und zu committen, oder benutzen Sie 'git commit -a'.
 
 #: archive.c:10
 msgid git archive [options] tree-ish [path...]
@@ -469,11 +467,11 @@ msgstr 
 #: diff.c:2934
 #, c-format
 msgid external diff died, stopping at %s
-msgstr 
+msgstr externes Diff-Programm unerwartet beendet, angehalten bei %s
 
 #: diff.c:3329
 msgid --follow requires exactly one pathspec
-msgstr 
+msgstr --follow erfordert genau eine Pfadspezifikation
 
 #: diff.c:3492
 #, c-format
@@ -1141,9 +1139,8 @@ msgstr 
 Tragen Sie Ihre Änderungen ein oder benutzen Sie \stash\ um fortzufahren.
 
 #: sequencer.c:250
-#, fuzzy
 msgid Failed to lock HEAD during fast_forward_to
-msgstr Fehler beim Sperren der Referenz zur Aktualisierung.
+msgstr Fehler beim Sperren von HEAD während fast_forward_to
 
 #. TRANSLATORS: %s will be revert or cherry-pick
 #: sequencer.c:293
@@ -2739,9 +2736,8 @@ msgstr Verarbeitet nur Zeilen im Bereich n,m, gezählt 
von 1
 #. relative timestamps, but your language may need more or
 #. fewer display columns.
 #: builtin/blame.c:2599
-#, fuzzy
 msgid 4 years, 11 months ago
-msgstr %s, und %lu Monat
+msgstr vor 4 Jahren, und 11 Monaten
 
 #: builtin/branch.c:24
 msgid git branch [options] [-r | -a] [--merged | --no-merged]
@@ -4249,9 +4245,9 @@ msgid malformed --author parameter
 msgstr Fehlerhafter --author Parameter
 
 #: builtin/commit.c:592
-#, fuzzy, c-format
+#, c-format
 msgid invalid date format: %s
-msgstr Ungültiger Commit: %s
+msgstr Ungültiges Datumsformat: %s
 
 #: builtin/commit.c:609
 #, c-format
@@ -4263,6 +4259,8 @@ msgid 
 unable to select a comment character that is not used\n
 in the current commit message
 msgstr 
+Konnte kein Kommentar-Zeichen auswählen, das nicht in\n
+der aktuellen Commit-Beschreibung verwendet wird.
 
 #: builtin/commit.c:679 builtin/commit.c:712 builtin/commit.c:1086
 #, c-format
@@ -4354,19 +4352,19 @@ msgstr 
 Eine leere Beschreibung bricht den Commit ab.\n
 
 #: builtin/commit.c:855
-#, fuzzy, c-format
+#, c-format
 msgid %sAuthor:%.*s %.*s
-msgstr %sAutor:%s
+msgstr  %sAutor:   %.*s %.*s
 
 #: builtin/commit.c:863
-#, fuzzy, c-format
+#, c-format
 msgid %sDate:  %s
-msgstr %sAutor:%s
+msgstr %sDatum:%s
 
 #: builtin/commit.c:870
-#, fuzzy, c-format
+#, c-format
 msgid %sCommitter: %.*s %.*s
-msgstr %sCommit-Ersteller: %s
+msgstr %sCommit-Ersteller: %.*s %.*s
 
 #: builtin/commit.c:888
 msgid Cannot read index
@@ -5081,11 +5079,11 @@ msgstr Überspringt Ausgabe von Blob-Daten
 
 #: builtin/fast-export.c:720
 msgid refspec
-msgstr 
+msgstr Refspec
 
 #: builtin/fast-export.c:721
 msgid Apply refspec to exported refs
-msgstr 
+msgstr Wendet Refspec auf exportierte Referenzen an
 
 #: builtin/fetch.c:20
 msgid git fetch [options] [repository [refspec...]]
@@ -5180,11 +5178,11 @@ msgstr akzeptiert Referenzen die .git/shallow 
aktualisieren
 
 #: builtin/fetch.c:125
 msgid refmap
-msgstr 
+msgstr Refmap
 
 #: builtin/fetch.c:126
 msgid specify fetch refmap
-msgstr 
+msgstr Angabe der Refmap für 'fetch'
 
 #: builtin/fetch.c:376
 msgid Couldn't find remote ref HEAD
@@ -5952,9 +5950,9 @@ msgid `git %s' is aliased to `%s'
 msgstr für `git %s' wurde der Alias `%s' angelegt
 
 #: builtin/index-pack.c:145
-#, fuzzy, c-format
+#, c-format
 msgid unable to open %s
-msgstr kann %s nicht lesen
+msgstr kann %s nicht öffnen
 
 #: builtin/index-pack.c:191
 #, c-format
@@ -5962,14 +5960,14 @@ msgid object type mismatch at %s
 msgstr Objekt-Typen passen bei %s nicht zusammen
 
 #: builtin/index-pack.c:211
-#, fuzzy, c-format
+#, c-format
 msgid did not receive expected object %s
-msgstr Kann Objekt %s nicht lesen.
+msgstr erwartetes Objekt %s nicht empfangen
 
 #: builtin/index-pack.c:214
-#, fuzzy, c-format
+#, c-format
 msgid object %s: expected type %s, found %s
-msgstr Objekt hat unerwarteten Typ
+msgstr Objekt %s: erwarteter Typ %s, %s gefunden
 
 #: 

Re: Improving the git remote command

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 10:24:35AM -0700, Junio C Hamano wrote:

 Jeff King p...@peff.net writes:
 
  ... But we are left with three options:
 
1. Add git remote list with verbose output. This is bad because it
   differs gratuitously from git remote.
 
2. Add git remote list with non-verbose output. This is good because
   it means git remote is just a shortcut for git remote list,
   which is consistent with other parts of git. But it is potentially
   bad if -v is a better output format.
 
3. Add git remote list with verbose output, and tweak git remote
   to match. This is bad because it breaks backwards compatibility.
 
  The proposal is for (1). I think we agree that (3) is out. The question
  is whether (1) or (2) is the least bad.
 
 I would imagine that those who want list of remotes programatically
 would read from git config output and it would be with less
 friction to change the output from git remote, a command that is
 solely to cater to end-user humans, to suit people's needs, so I am
 not sure if (3) is immediately out.

Yeah, I touched on that earlier. I would personally consider git
remote to be a porcelain, and git config to be the appropriate
plumbing for accessing those values. However, it's a little tricky to
robustly get the list of remotes with git config. So I would not be
surprised if scripts have used git remote to do the same thing (I know
for a fact that some internal scripts at GitHub did this, though I
recently cleaned them up so I do not have a vested interest either way
at this point).

That does not mean those scripts are right and we cannot change things,
but it may be a matter of practicality.

 Having said that, my preference is 
 
 0. Do nothing, but document the default to listing better if
needed.
 
 and then 2. above, and then 1.

Yeah, I'd agree with that.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 18/18] signed push: final protocol update

2014-08-26 Thread Shawn Pearce
On Mon, Aug 25, 2014 at 10:59 AM, Junio C Hamano gits...@pobox.com wrote:
 Shawn Pearce spea...@spearce.org writes:

 A stateless nonce could look like:

   nonce = HMAC_SHA1( SHA1(site+path) + '.' + now, site_key )

 where site_key is a private key known to the server. It doesn't have
 to be per-repo.

 receive-pack would then be willing to accept any nonce whose timestamp
 is within a window, e.g. 10 minutes of the current time, and whose
 signature verifies in the HMAC. The 10 minute window is important to
 allow clients time to generate the object list, perform delta
 compression, and begin transmitting to the server.

 Hmph, don't you send the finally tell the other end the sequence
 of update this ref from old to new and the packdata separately?

No. The command list (triples of old, new, ref) is sent in the same
HTTP request as the pack data, ahead of the pack data. So its one
request.

Push on smart HTTP is 3 HTTP requests:

  1)  get advertisement
  2)  POST empty flush packet to tickle auth (literally just ).
  3)  POST command list + pack

The nonce can be sent server-client in 1, and client-server in 3.

  I
 think we have a FLUSH in between, and the push certificate is given
 before the FLUSH, which you do not have to wait for 10 minutes.

Nope I think you need to wait for the pack to generate enough to start
sending the pack data stream. Nothing forces the smart HTTP client to
push its pending buffer out. We wait for the pack data to either
finish, or overflow the in-memory buffer, and then start transmitting.
If your client needs a lot of time for counting and delta compression,
we aren't likely to overflow and transmit for a while.

If you send a _lot_ of refs you can overflow, which will cause us to
transmit early. But we are talking about megabytes worth of (old, new,
ref) triplets to reach that overflow point.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 2/3] log-tree: make name_decoration hash static

2014-08-26 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 In the previous commit, we made add_name_decoration global
 so that adders would not have to access the hash directly.
 We now make the hash itself static so that callers _have_ to
 add through our function, making sure that all additions go
 through a single point.  To do this, we have to add one more
 accessor function: a way to lookup entries in the hash.

 Since the only caller doesn't actually look at the returned
 value, but rather only asks whether there is a decoration or
 not, we could provide only a boolean has_name_decoration.
 That would allow us to make struct name_decoration local
 to log-tree, as well.

 However, it's unlikely to cause any maintainability harm
 making the actual data public, and this interface is more
 flexible if we need to look at decorations from other parts
 of the code in the future.

I didn't think we want only-bool version, but it is nice to see it
explained well.

I may have called it lookup_name_decoration() to match, though, if I
were doing this patch ;-)

Thanks.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/2] Check order when reading index

2014-08-26 Thread Junio C Hamano
Jaime Soriano Pastor jsorianopas...@gmail.com writes:

 On Tue, Aug 26, 2014 at 6:53 PM, Junio C Hamano gits...@pobox.com wrote:

 Yes---that is what I meant by the virtual stuff.  Unlike resolve
 work by Daniel (around Sep 2005 $gmane/8088) that tried to use
 multiple ancestors directly in a single merge, recursive limits
 itself to repeated use of pairwise merges.

 Ok, I see now. Then what about checking also in check_ce_order() that
 only stage #1 can be repeated?

We could use multiple stage #3 entries to natively represent an
octopus merge in progress if we wanted to.  I do not think we want
to close the door for doing so, unless there is some compelling
reason.

Does the current codebase choke with such entries in the index file,
like you saw in your index file with both stage #0 and stage #1
entries?
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 2/3] log-tree: make name_decoration hash static

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 10:40:10AM -0700, Junio C Hamano wrote:

 I may have called it lookup_name_decoration() to match, though, if I
 were doing this patch ;-)

Hmph. I called it get because that was the opposite of add to me,
and I was matching add_name_decoration. Of course, in the regular
decoration code, the add function is also add and its opposite is
lookup. So mine is gratuitously different. I do not mind if you adjust
while applying.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v5 4/4] convert: Stream from fd to required clean filter instead of mmap

2014-08-26 Thread Jeff King
On Mon, Aug 25, 2014 at 06:55:51PM +0200, Steffen Prohaska wrote:

 It could be handled that way, but we would be back to the original problem
 that 32-bit git fails for large files.  The convert code path currently
 assumes that all data is available in a single buffer at some point to apply
 crlf and ident filters.
 
 If the initial filter, which is assumed to reduce the file size, fails, we
 could seek to 0 and read the entire file.  But git would then fail for large
 files with out-of-memory.  We would not gain anything for the use case that
 I describe in the commit message's first paragraph.

Ah. So the real problem is that we cannot handle _other_ conversions for
large files, and we must try to intercept the data before it gets to
them. So this is really just helping reduction filters. Even if our
streaming filter succeeds, it does not help the situation if it did not
reduce the large file to a smaller one.

It would be nice in the long run to let the other filters stream, too,
but that is not a problem we need to solve immediately. Your patch is a
strict improvement.

Thanks for the explanation; your approach makes a lot more sense to me
now.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v5 4/4] convert: Stream from fd to required clean filter instead of mmap

2014-08-26 Thread Jeff King
On Mon, Aug 25, 2014 at 11:35:45AM -0700, Junio C Hamano wrote:

 Steffen Prohaska proha...@zib.de writes:
 
  Couldn't we do that with an lseek (or even an mmap with offset 0)? That
  obviously would not work for non-file inputs, but I think we address
  that already in index_fd: we push non-seekable things off to index_pipe,
  where we spool them to memory.
 
  It could be handled that way, but we would be back to the original problem
  that 32-bit git fails for large files.
 
 Correct, and you are making an incremental improvement so that such
 a large blob can be handled _when_ the filters can successfully
 munge it back and forth.  If we fail due to out of memory when the
 filters cannot, that would be the same as without your improvement,
 so you are still making progress.

I do not think my proposal makes anything worse than Steffen's patch.
_If_ you have a non-required filter, and _if_ we can run it, then we
stream the filter and hopefully end up with a small enough result to fit
into memory. If we cannot run the filter, we are screwed anyway (we
follow the regular code path and dump the whole thing into memory; i.e.,
the same as without this patch series).

I think the main argument against going further is just that it is not
worth the complexity. Tell people doing reduction filters they need to
use required, and that accomplishes the same thing.

  So it seems like the ideal strategy would be:
  
   1. If it's seekable, try streaming. If not, fall back to lseek/mmap.
  
   2. If it's not seekable and the filter is required, try streaming. We
  die anyway if we fail.
 
 Puzzled...  Is it assumed that any content the filters tell us to
 use the contents from the db as-is by exiting with non-zero status
 will always be large not to fit in-core?  For small contents, isn't
 this ideal strategy a regression?

I am not sure what you mean by regression here. We will try to stream
more often, but I do not see that as a bad thing.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 0/2] describe: support the syntax --abbrev=+

2014-08-26 Thread Jonh Wendell
hi there!
just a ping here, these are my first patches to git.
any comment, feedback?

2014-08-23 14:13 GMT-03:00 Jonh Wendell jonh.wend...@gmail.com:
 Sometimes it's interesting to have a simple output that answers the question:
 Are there commits after the latest tag?

 One possible solution is to just print a + (plus) signal after the tag. 
 Example:

 git describe --abbrev=1 5261ec5d5
 v2.1.0-rc1-2-g5261ec

 git describe --abbrev=+ 5261ec5d5
 v2.1.0-rc1+


 Jonh Wendell (2):
   describe: support the syntax --abbrev=+
   describe: Add documentation for --abbrev=+

  Documentation/git-describe.txt |  6 ++
  builtin/describe.c | 26 +-
  2 files changed, 27 insertions(+), 5 deletions(-)

 --
 1.9.3




-- 
Jonh Wendell
http://www.bani.com.br
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v6 2/6] Add git_env_ulong() to parse environment variable

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 05:23:21PM +0200, Steffen Prohaska wrote:

 +/*
 + * Use default if environment variable is unset or empty string.
 + */
 +unsigned long git_env_ulong(const char *k, unsigned long val)
 +{
 + const char *v = getenv(k);
 + if (v  *v  !git_parse_ulong(v, val))
 + die(failed to parse %s, k);
 + return val;
 +}

I think the empty string behavior here is sensible. I notice that
git_env_bool is not so careful. I think we should probably do this
(independent of your series):

-- 8 --
Subject: git_env_bool: treat empty string as not set

If an environment variable we treat as a boolean is not set,
we use some default value provided by the caller. If it is
set but is the empty string, however, we treat it as
false. This can be rather confusing, as it is easy to set
the variable to the empty string in the shell (e.g., by
calling GIT_SMART_HTTP= instead of unset).

Instead, let's treat unset and empty variables the same.
This should not have any negative backwards-compatibility
consequences, because:

  1. The existing behavior was confusing and undocumented in
 the first place.

  2. For most variables, the default _is_ false, and so this
 change is a noop. The only affected variables are
 GIT_IMPLICIT_WORK_TREE (which is undocumented and
 internally always set to 0 or 1) and GIT_SMART_HTTP
 (which is also undocumented, and we use only for
 testing).

Since there won't be any fallout with the current variables,
this is a good time to make the switch (before any other
variables are added).

Signed-off-by: Jeff King p...@peff.net
---
 config.c| 2 +-
 t/t5551-http-fetch-smart.sh | 7 +++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/config.c b/config.c
index 058505c..7bf0704 100644
--- a/config.c
+++ b/config.c
@@ -1119,7 +1119,7 @@ const char *git_etc_gitconfig(void)
 int git_env_bool(const char *k, int def)
 {
const char *v = getenv(k);
-   return v ? git_config_bool(k, v) : def;
+   return v  *v ? git_config_bool(k, v) : def;
 }
 
 int git_config_system(void)
diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh
index 6cbc12d..831f9e4 100755
--- a/t/t5551-http-fetch-smart.sh
+++ b/t/t5551-http-fetch-smart.sh
@@ -168,6 +168,13 @@ test_expect_success 'GIT_SMART_HTTP can disable smart 
http' '
 test_must_fail git fetch)
 '
 
+test_expect_success 'empty GIT_SMART_HTTP leaves smart http enabled' '
+   (GIT_SMART_HTTP= 
+export GIT_SMART_HTTP 
+cd clone 
+git fetch)
+'
+
 test_expect_success 'invalid Content-Type rejected' '
test_must_fail git clone $HTTPD_URL/broken_smart/repo.git 2actual
grep not valid: actual
-- 
2.1.0.346.ga0367b9

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v6 5/6] Change copy_fd() to not close input fd

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 05:23:24PM +0200, Steffen Prohaska wrote:

 The caller opened the fd, so it should be responsible for closing it.
 
 Signed-off-by: Steffen Prohaska proha...@zib.de
 ---
  copy.c | 5 +
  lockfile.c | 3 +++
  2 files changed, 4 insertions(+), 4 deletions(-)
 
 diff --git a/copy.c b/copy.c
 index a7f58fd..d0a1d82 100644
 --- a/copy.c
 +++ b/copy.c
 @@ -10,7 +10,6 @@ int copy_fd(int ifd, int ofd)
   break;
   if (len  0) {
   int read_error = errno;
 - close(ifd);
   return error(copy-fd: read returned %s,
strerror(read_error));
   }

This saved errno is not necessary anymore (the problem was that close()
clobbered the error in the original code). It can go away, and we can
even drop the curly braces.

 @@ -21,17 +20,14 @@ int copy_fd(int ifd, int ofd)
   len -= written;
   }
   else if (!written) {
 - close(ifd);
   return error(copy-fd: write returned 0);
   } else {
   int write_error = errno;
 - close(ifd);
   return error(copy-fd: write returned %s,
strerror(write_error));
   }
   }

Ditto here. Actually, isn't this whole write just a reimplementation of
write_in_full? The latter treats a return of 0 as ENOSPC rather than
using a custom message, but I think that is sane.

All together:

---
 copy.c | 28 +---
 1 file changed, 5 insertions(+), 23 deletions(-)

diff --git a/copy.c b/copy.c
index a7f58fd..53a9ece 100644
--- a/copy.c
+++ b/copy.c
@@ -4,34 +4,16 @@ int copy_fd(int ifd, int ofd)
 {
while (1) {
char buffer[8192];
-   char *buf = buffer;
ssize_t len = xread(ifd, buffer, sizeof(buffer));
if (!len)
break;
-   if (len  0) {
-   int read_error = errno;
-   close(ifd);
+   if (len  0)
return error(copy-fd: read returned %s,
-strerror(read_error));
-   }
-   while (len) {
-   int written = xwrite(ofd, buf, len);
-   if (written  0) {
-   buf += written;
-   len -= written;
-   }
-   else if (!written) {
-   close(ifd);
-   return error(copy-fd: write returned 0);
-   } else {
-   int write_error = errno;
-   close(ifd);
-   return error(copy-fd: write returned %s,
-strerror(write_error));
-   }
-   }
+strerror(errno));
+   if (write_in_full(ofd, buffer, len)  0)
+   return error(copy-fd: write returned %s,
+strerror(errno));
}
-   close(ifd);
return 0;
 }
 
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 2/3] log-tree: make name_decoration hash static

2014-08-26 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 On Tue, Aug 26, 2014 at 10:40:10AM -0700, Junio C Hamano wrote:

 I may have called it lookup_name_decoration() to match, though, if I
 were doing this patch ;-)

 Hmph. I called it get because that was the opposite of add to me,
 and I was matching add_name_decoration. Of course, in the regular
 decoration code, the add function is also add and its opposite is
 lookup. So mine is gratuitously different. I do not mind if you adjust
 while applying.

I do not care too deeply either way, either ;-)  I just thought that
sharing the verb with the underlying function being wrapped would be
more consistent.

I wish we used lookup vs get more consistently, though.  One should
mean give us if we already have one otherwise fail while the other
should mean give us one, or create one if there isn't yet.

Unfortunately lookup_commit() and remote_get() both do auto-vivify X-.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v5 4/4] convert: Stream from fd to required clean filter instead of mmap

2014-08-26 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 On Mon, Aug 25, 2014 at 11:35:45AM -0700, Junio C Hamano wrote:

 Steffen Prohaska proha...@zib.de writes:
 
  Couldn't we do that with an lseek (or even an mmap with offset 0)? That
  obviously would not work for non-file inputs, but I think we address
  that already in index_fd: we push non-seekable things off to index_pipe,
  where we spool them to memory.
 
  It could be handled that way, but we would be back to the original problem
  that 32-bit git fails for large files.
 
 Correct, and you are making an incremental improvement so that such
 a large blob can be handled _when_ the filters can successfully
 munge it back and forth.  If we fail due to out of memory when the
 filters cannot, that would be the same as without your improvement,
 so you are still making progress.

 I do not think my proposal makes anything worse than Steffen's patch.

I think we are saying the same thing, but perhaps I didn't phrase it
well.

 I think the main argument against going further is just that it is not
 worth the complexity. Tell people doing reduction filters they need to
 use required, and that accomplishes the same thing.

  So it seems like the ideal strategy would be:
  
   1. If it's seekable, try streaming. If not, fall back to lseek/mmap.
  
   2. If it's not seekable and the filter is required, try streaming. We
  die anyway if we fail.
 
 Puzzled...  Is it assumed that any content the filters tell us to
 use the contents from the db as-is by exiting with non-zero status
 will always be large not to fit in-core?  For small contents, isn't
 this ideal strategy a regression?

 I am not sure what you mean by regression here. We will try to stream
 more often, but I do not see that as a bad thing.

I thought the proposed flow I was commenting on was

- try streaming and die if the filter fails

For an optional filter working on contents that would fit in core,
we currently do

- slurp in memory, filter it, use the original if the filter fails

If we switched to 2., then... ahh, ok, I misread is required part.
The regression does not apply to that case at all.


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 18/18] signed push: final protocol update

2014-08-26 Thread Junio C Hamano
Shawn Pearce spea...@spearce.org writes:

 On Mon, Aug 25, 2014 at 10:59 AM, Junio C Hamano gits...@pobox.com wrote:
 Shawn Pearce spea...@spearce.org writes:

 A stateless nonce could look like:

   nonce = HMAC_SHA1( SHA1(site+path) + '.' + now, site_key )

 where site_key is a private key known to the server. It doesn't have
 to be per-repo.

 receive-pack would then be willing to accept any nonce whose timestamp
 is within a window, e.g. 10 minutes of the current time, and whose
 signature verifies in the HMAC. The 10 minute window is important to
 allow clients time to generate the object list, perform delta
 compression, and begin transmitting to the server.

 Hmph, don't you send the finally tell the other end the sequence
 of update this ref from old to new and the packdata separately?

 No. The command list (triples of old, new, ref) is sent in the same
 HTTP request as the pack data, ahead of the pack data. So its one
 request.

That is unfortunate.  Would it be a major surgery to update the
protocol not to do that, perhaps by moving the command list from 3
to 2 (the latter of which is not currently doing anything useful
payload-wise, other than flushing a HTTP request early)?

 Push on smart HTTP is 3 HTTP requests:

   1)  get advertisement
   2)  POST empty flush packet to tickle auth (literally just ).
   3)  POST command list + pack

 The nonce can be sent server-client in 1, and client-server in 3.

  I
 think we have a FLUSH in between, and the push certificate is given
 before the FLUSH, which you do not have to wait for 10 minutes.

 Nope I think you need to wait for the pack to generate enough to start
 sending the pack data stream. Nothing forces the smart HTTP client to
 push its pending buffer out. We wait for the pack data to either
 finish, or overflow the in-memory buffer, and then start transmitting.
 If your client needs a lot of time for counting and delta compression,
 we aren't likely to overflow and transmit for a while.

 If you send a _lot_ of refs you can overflow, which will cause us to
 transmit early. But we are talking about megabytes worth of (old, new,
 ref) triplets to reach that overflow point.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 2/3] Makefile: use `find` to determine static header dependencies

2014-08-26 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 On Tue, Aug 26, 2014 at 09:54:19AM -0700, Junio C Hamano wrote:

 Jeff King p...@peff.net writes:
 
  Actually as you are not excluding CVS, RCS, etc., and using ??* as
  the starting point will exclude .git, .hg, etc. at the top, I think
  we can shorten it even further and say
  
find ??* -name Documentation -prune -o -name \*.h
  
  or something.
 
  I had originally considered starting with find *, but I was worried
  about shell globbing overflowing command-line limits here. echo * on a
  built tree is about 12K.
 
 OK.  What I queued is still your original which is the most
 conservative among various fun alternatives we have seen so far on
 this thread, so we should be good ;-)

 The only thing I think mine does not do that Jonathan suggested is
 dropping .hg, etc. I do not know why anyone would track git in hg, but
 it might make sense to s/.git/.?/ in what I sent.

 (I noticed also that you did not queue the third patch to drop
 CHECK_HEADER_DEPENDENCIES; I'm OK if we keep it, but I wanted to make
 sure it was not an oversight).

It started as I just ran out of time to really think about it and
transitioned to Ahh, I forgot that I postponed deciding ;-)

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 18/18] signed push: final protocol update

2014-08-26 Thread Junio C Hamano
Junio C Hamano gits...@pobox.com writes:

 That is unfortunate.  Would it be a major surgery to update the
 protocol not to do that, perhaps by moving the command list from 3
 to 2 (the latter of which is not currently doing anything useful
 payload-wise, other than flushing a HTTP request early)?

Nah, that was one of the most stupid thing I ever said here X.
There is nothing that ties #2 and #3 unless the server side keeps
some state, so that would not work very well X-.


 Push on smart HTTP is 3 HTTP requests:

   1)  get advertisement
   2)  POST empty flush packet to tickle auth (literally just ).
   3)  POST command list + pack
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v6 2/6] Add git_env_ulong() to parse environment variable

2014-08-26 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 On Tue, Aug 26, 2014 at 05:23:21PM +0200, Steffen Prohaska wrote:

 +/*
 + * Use default if environment variable is unset or empty string.
 + */
 +unsigned long git_env_ulong(const char *k, unsigned long val)
 +{
 +const char *v = getenv(k);
 +if (v  *v  !git_parse_ulong(v, val))
 +die(failed to parse %s, k);
 +return val;
 +}

 I think the empty string behavior here is sensible. I notice that
 git_env_bool is not so careful. I think we should probably do this
 (independent of your series):

 -- 8 --
 Subject: git_env_bool: treat empty string as not set

 If an environment variable we treat as a boolean is not set,
 we use some default value provided by the caller. If it is
 set but is the empty string, however, we treat it as
 false. This can be rather confusing, as it is easy to set
 the variable to the empty string in the shell (e.g., by
 calling GIT_SMART_HTTP= instead of unset).

I think different people have different confusion criteria.
To me, these two are very different operations:

$ VAR=
$ unset VAR

I think it boils down to that I see that the distance between unset
vs set to empty is far larger than the distance between empty vs
false.  You probably see these two distances the other way,
i.e. set to empty is almost like unset and empty is not a valid
way to say false.

Due to this difference, the new test confused me and had me read it
three times.

 +test_expect_success 'empty GIT_SMART_HTTP leaves smart http enabled' '
 + (GIT_SMART_HTTP= 
 +  export GIT_SMART_HTTP 
 +  cd clone 
 +  git fetch)
 +'

The test before this one explicitly sets GIT_SMART_HTTP to 0 and
expects the fetch to fail.  It is sensible to you because 0 is a
lot more explicit false than an empty string to you, and you
equate an empty and unset, hence the new one should succeed.

But it looks nonsensical to me that the new one expects to succeed,
because 0 and an empty string are both valid way to say false
to me, and it should behave the same way as the 0 one.

I view the *v check before git_parse_ulong() being unnecessarily
defensive to avoid triggering die().  An empty string is obviously
not a number (somebody could argue that it is the same as zero,
though), but nevertheless the user _is_ telling us to use that value
by setting and exporting the variable.  If we cannot parse it,
barfing is what the user would appreciate.

So, I am not sure the patch in the message I am responding to, and I
am not sure about that *v check in Steffen's patch, either.

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] contrib/svn-fe: fix Makefile

2014-08-26 Thread Torsten Bögershausen

On 08/26/2014 02:44 PM, Maxim Bublis wrote:

Fixes several problems:
   * include config.mak.uname, config.mak.autogen and config.mak
 in order to use settings for prefix and other such things;
   * link xdiff/lib.a as it is a requirement for libgit.a;
   * fix CFLAGS and EXTLIBS for Linux and Mac OS X.
---
  contrib/svn-fe/Makefile | 47 ++-
  1 file changed, 38 insertions(+), 9 deletions(-)

diff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile
index 360d8da..8e1d622 100644
--- a/contrib/svn-fe/Makefile
+++ b/contrib/svn-fe/Makefile
@@ -1,18 +1,45 @@
  all:: svn-fe$X
  
-CC = gcc

+CC = cc
  RM = rm -f
  MV = mv
  
  CFLAGS = -g -O2 -Wall

  LDFLAGS =
-ALL_CFLAGS = $(CFLAGS)
-ALL_LDFLAGS = $(LDFLAGS)
-EXTLIBS =
+EXTLIBS = -lz
+
+include ../../config.mak.uname
+-include ../../config.mak.autogen
+-include ../../config.mak
+
+ifeq ($(uname_S),Darwin)
+   CFLAGS += -I/opt/local/include
+   LDFLAGS += -L/opt/local/lib
+endif
+

Should it be possible to disable this by using NO_DARWIN_PORTS
like we do in the main Makefile ?

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v6 2/6] Add git_env_ulong() to parse environment variable

2014-08-26 Thread Jeff King
On Tue, Aug 26, 2014 at 01:20:53PM -0700, Junio C Hamano wrote:

 I think different people have different confusion criteria.
 To me, these two are very different operations:
 
 $ VAR=
 $ unset VAR
 
 I think it boils down to that I see that the distance between unset
 vs set to empty is far larger than the distance between empty vs
 false.  You probably see these two distances the other way,
 i.e. set to empty is almost like unset and empty is not a valid
 way to say false.
 
 Due to this difference, the new test confused me and had me read it
 three times.

I agree that it is rather a subjective decision.

 So, I am not sure the patch in the message I am responding to, and I
 am not sure about that *v check in Steffen's patch, either.

If it is truly some people prefer it one way and some the other, I am
not sure if we should leave it as-is (that is preferring one way). The
middle ground would be to die(). That does not seem super-friendly, but
then we would also die with GIT_SMART_HTTP=foobar, so perhaps it is not
unreasonable to just consider it a syntax error.

I dunno. I can live with leaving it as-is. Certainly the existing
behavior is not what I expected, but it is not like it came up in the
real world (and I would not expect it to do so often). And it is
consistent with the config, which treats:

  [foo]
  bar =

as boolean false. That _also_ seems weird to me, but that is not
something I think we can easily change or outlaw at this point anyway.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Re: Re: Re: Relative submodule URLs

2014-08-26 Thread Heiko Voigt
On Tue, Aug 26, 2014 at 10:18:48AM -0500, Robert Dailey wrote:
 On Tue, Aug 26, 2014 at 1:28 AM, Heiko Voigt hvo...@hvoigt.net wrote:
  My last email response was in violation of your request to keep the
  two topics separate, sorry about that. I started typing it this
  weekend and completed the draft this morning, without having read this
  response from you first.
 
  Thats fine, no problem.
 
  Here is what I think would make the feature most usable. I think you
  went over some of these ideas but I just want to clarify, to make sure
  we're on the same page. Please correct me as needed.
 
  1. Running `git submodule update --with-remote name` shall fail the
  command unconditionally.
 
  I am not sure but I think you mean
 
  git submodule update --with-remote=name
 
  With the equals sign, without it you would name the submodule paths to
  update. No I think that should just add the remote name to all
  submodules that would be updated and do the normal update operation on
  them (with the new remote of course).
 
 I'm not sure about Linux but at least with msysgit on Windows, typing
 a two-dash option (such as --with-remote) forces command line
 evaluation to use the next placement parameter as the parameter for
 it. I've seen this work the same way with argparse in python too. In
 my experience, command line has worked that way, I'm not sure if that
 is by design or not though. I never use equal signs with git commands,
 never had a problem for some reason.

 For example:
 
 git rebase --onto release/1.0 head~3 head
 
 The `--onto` option knows to use `release/1.0` as its parameter.

If you are on Window or Linux does not make a difference here. I just
realized we are quite inconsistent:

$ git grep -E -e --\w+=\w+ -- Documentation/ | wc -l
 226

$ git grep -E -e --\w+ \w+ -- Documentation/ | wc -l
  75

I would prefer the first though since that one is used more often. But
we can leave that for later, once we have some code to talk about.

  2. Using the `--with-remote` option on submodule `update` or `sync`
  will fail if it detects absolute submodule URLs in .gitmodule
 
  Yes, almost. Since you can have a mixture I suggest to only fail if the
  submodules that would be processed have an absolute url in them. If
  processed submodules are all relative it can go ahead.
 
 For example if it processes 3 submodules in the following order:
 
 1. relative
 2. absolute
 3. relative
 
 Should it fail before or after processing the 3rd relative submodule?
 I was thinking it would fail while trying to sync/update the 2nd one
 (which is absolute) and stop before processing the 3rd.

For consistency I would prefer if it fails right from the beginning in
this situation since the command can not be completed.

  3. Running `git submodule update --init --with-remote name` shall
  fail the command ONLY if a submodule is being processed that is NOT
  also being initialized.
 
  No since the --init flag just tells update to initialize submodules
  on-demand. It should just go ahead the same way as without
  --with-remote.
 
 But doesn't the on-demand initialization need to evaluate relative
 URLs and convert them to absolute based on the .gitmodules
 configuration? I thought the idea was to make `--with-remote` invalid
 for initialization/sync of absolute URLs.
 
 In other words if I did:
 
 git submodule init --with-remote fork my-submodule-dir
 
 and if my-submodule-dir was not relative in .gitmodules, then the
 `--with-remote` flag becomes useless. We could fail silently but for
 educational purposes to the user I thought we were failing in these
 scenarios. Maybe I misunderstood your original intent with the
 failures? Is init not doing the relative to absolute evaluation like
 I'm thinking? Please correct me where I'm wrong.

Yes it does the relative to absolute evaluation. But that is a different
topic. For absolute urls in .gitmodules it should fail, but you were
talking about --init in general and in general that should not fail IMO.
So e.g.

git submodule update --init --with-remote=name

when all submodule urls are relative in .gitmodules and some submodules
have already been initialized should succeed.

  4. The behavior of git submodule's `update` or `sync` commands
  combined with `--with-remote` will REPLACE or CREATE the 'origin'
  remote in each submodule it is run in. We will not allow the user to
  configure what the submodule remote name will end up being (I think
  this is current behavior and forces good practice; I consider `origin`
  an adopted standard for git, and actually wish it was more enforced
  for super projects as well!)
 
  No please carefully read my email again. I specifically was describing
  the opposite. --with-remote=name creates/replaces the remote name in
  the submodule. I do not see a benefit in restricting the user from
  creating different remote names in the submodule. I think it would be
  more confusing if the remote 'origin' in the 

Re: Transaction patch series overview

2014-08-26 Thread Junio C Hamano
Jonathan Nieder jrnie...@gmail.com writes:

  https://code-review.googlesource.com/#/q/topic:ref-transaction-1

 Outcome of the experiment: patches gained some minor changes and then

  1-12
   Reviewed-by: Jonathan Nieder jrnie...@gmail.com

  13
   Reviewed-by: Michael Haggerty mhag...@alum.mit.edu
   Reviewed-by: Jonathan Nieder jrnie...@gmail.com

  14
   Reviewed-by: Jonathan Nieder jrnie...@gmail.com

  15-16
   Reviewed-by: Michael Haggerty mhag...@alum.mit.edu
   Reviewed-by: Jonathan Nieder jrnie...@gmail.com

  17
   Reviewed-by: Michael Haggerty mhag...@alum.mit.edu

  18
   Reviewed-by: Michael Haggerty mhag...@alum.mit.edu
   Reviewed-by: Jonathan Nieder jrnie...@gmail.com

  19
   Reviewed-by: Jonathan Nieder jrnie...@gmail.com

  20
   Reviewed-by: Michael Haggerty mhag...@alum.mit.edu
   Reviewed-by: Jonathan Nieder jrnie...@gmail.com

 I found the web UI helpful in seeing how each patch evolved since I
 last looked at it.  Interdiff relative to the version in pu is below.

Thanks for the interdiff, it really helps to be able to take a
glance without having to click around.

It seems that I can hold the topic in 'pu' a bit longer and expect a
reroll from this effort before merging it to 'next'?

 The next series from Ronnie's collection is available at
 https://code-review.googlesource.com/#/q/topic:ref-transaction in case
 someone wants a fresh series to look at.

Thanks.

 diff --git a/builtin/commit.c b/builtin/commit.c
 index 668fa6a..9bf1003 100644
 --- a/builtin/commit.c
 +++ b/builtin/commit.c
 @@ -1765,8 +1765,8 @@ int cmd_commit(int argc, const char **argv, const char 
 *prefix)
   transaction = ref_transaction_begin(err);
   if (!transaction ||
   ref_transaction_update(transaction, HEAD, sha1,
 -current_head ?
 -current_head-object.sha1 : NULL,
 +current_head
 +? current_head-object.sha1 : NULL,
  0, !!current_head, err) ||
   ref_transaction_commit(transaction, sb.buf, err)) {
   rollback_index_files();

Perhaps this is nicer, but probably most people would not care.

 diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
 index 91099ad..224fadc 100644
 --- a/builtin/receive-pack.c
 +++ b/builtin/receive-pack.c
 @@ -580,18 +580,18 @@ static char *update(struct command *cmd, struct 
 shallow_info *si)
  
   if (shallow_update  si-shallow_ref[cmd-index] 
   update_shallow_ref(cmd, si))
 - return xstrdup(shallow error);
 + return shallow error;
  
   transaction = ref_transaction_begin(err);
   if (!transaction ||
   ref_transaction_update(transaction, namespaced_name,
  new_sha1, old_sha1, 0, 1, err) ||
   ref_transaction_commit(transaction, push, err)) {
 - char *str = strbuf_detach(err, NULL);
   ref_transaction_free(transaction);
  
 - rp_error(%s, str);
 - return str;
 + rp_error(%s, err.buf);
 + strbuf_release(err);
 + return failed to update ref;
   }

I am guessing that the purpose of this change is to make sure that
cmd-error_string is taken from a fixed set of strings, not an
arbitrary collection of errors from the transaction subsystem, but
what is the significance of doing so?  Do the other side i18n while
running print_ref_status() or something?

 diff --git a/refs.h b/refs.h
 index aad846c..69ef28c 100644
 --- a/refs.h
 +++ b/refs.h
 @@ -180,8 +180,7 @@ extern int peel_ref(const char *refname, unsigned char 
 *sha1);
   */
  #define REF_NODEREF  0x01
  /*
 - * Locks any ref (for 'HEAD' type refs) and sets errno to something
 - * meaningful on failure.
 + * This function sets errno to something meaningful on failure.
   */
  extern struct ref_lock *lock_any_ref_for_update(const char *refname,
   const unsigned char *old_sha1,

To contrast this function with lock_ref_sha1() that only allowed
refs under refs/ hierarchy, the comment may have made sense, but now
that other function is gone, losing the comment makes sense.

I removed from the above the interdiff hunks I did not have any
comments or questions on.

Thanks again.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2 18/23] refs.c: add a backend method structure with transaction functions

2014-08-26 Thread Junio C Hamano
Ronnie Sahlberg sahlb...@google.com writes:

 +struct ref_be {
 + transaction_begin_fn transaction_begin;
 + transaction_update_sha1_fn transaction_update_sha1;
 + transaction_create_sha1_fn transaction_create_sha1;
 + transaction_delete_sha1_fn transaction_delete_sha1;
 + transaction_update_reflog_fn transaction_update_reflog;
 + transaction_commit_fn transaction_commit;
 + transaction_free_fn transaction_free;
 +};
 +
 +extern struct ref_be *refs;

The overall organization is nice, but please don't use such a short
name for the systemwide default singleton instance, which should not
be accessed by normal code other than via helpers that implicitly
use that singleton (e.g. resolve_ref_unsafe() which invokes the
method of the same name on the singleton, passing the parameters it
received[*1*]).  The name will be used for other things (e.g. a
local variable for a collection of refs) by code that do not care
about the underlying implementation of the helpers and will cause
confusion later.

Perhaps the_refs_backend or something?

Also does the singleton have to be extern, not a static inside refs.c,
perhaps with a setter function to switch it or something?


[Reference]

*1* A typical helper that uses the singleton looks like this:

+const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1,
+  int reading, int *flag)
+{
+   return refs-resolve_ref_unsafe(ref, sha1, reading, flag);
+}
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v3 03/23] refs.c: add a new refs.c file to hold all common refs code

2014-08-26 Thread Junio C Hamano
Ronnie Sahlberg sahlb...@google.com writes:

 Create a new erfs.c file that will be used to hold all the refs
 code that is backend agnostic and will be shared across all backends.

 The reason we renamed everything to refs-be-files.c in the previous patch
 and now start moving the common code back to the new refs.c file
 instead of the other way around is the etive volumes of code.

Huh?  Why not create refs-be-files.c and move whatever need to be
there over there, instead of rename the file and move things that
shouldn't have been moved back like this?

Puzzled.

I do not see 02/23 here, but I am assuming that is is just

git mv refs.c refs-be-files.c

which may have been a seven-line patch with format-patch -M ;-)


 With the ref_cache, packed refs and loose ref handling that are all
 part of the files based implementation the backend specific part
 of the old refs.c file is several times larger than the backend agnostic
 part. Therefore it makes more sense to first rename everything to be
 part of the files based backend and then move the parts that can be used
 as common code back to refs.c.

 Signed-off-by: Ronnie Sahlberg sahlb...@google.com
 ---
  Makefile | 1 +
  refs.c   | 3 +++
  2 files changed, 4 insertions(+)
  create mode 100644 refs.c

 diff --git a/Makefile b/Makefile
 index e010ad1..937d22a 100644
 --- a/Makefile
 +++ b/Makefile
 @@ -857,6 +857,7 @@ LIB_OBJS += quote.o
  LIB_OBJS += reachable.o
  LIB_OBJS += read-cache.o
  LIB_OBJS += reflog-walk.o
 +LIB_OBJS += refs.o
  LIB_OBJS += refs-be-files.o
  LIB_OBJS += remote.o
  LIB_OBJS += replace_object.o
 diff --git a/refs.c b/refs.c
 new file mode 100644
 index 000..77492ff
 --- /dev/null
 +++ b/refs.c
 @@ -0,0 +1,3 @@
 +/*
 + * Common refs code for all backends.
 + */
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v3 19/23] refs-be-files.c: add a backend method structure with transaction functions

2014-08-26 Thread Junio C Hamano
Ronnie Sahlberg sahlb...@google.com writes:

 diff --git a/refs-be-files.c b/refs-be-files.c
 index e58a7e1..27eafd0 100644
 --- a/refs-be-files.c
 +++ b/refs-be-files.c
 ...
 +struct ref_be refs_files = {
 + files_transaction_begin,
 + files_transaction_update_sha1,
 + files_transaction_create_sha1,
 + files_transaction_delete_sha1,
 + files_transaction_update_reflog,
 + files_transaction_commit,
 + files_transaction_free,
 +};
 +
 +struct ref_be *refs = refs_files;

 diff --git a/refs.c b/refs.c
 index 6b434ad..b8c942f 100644
 --- a/refs.c
 +++ b/refs.c
 ...
 +void transaction_free(struct ref_transaction *transaction)
 +{
 + return refs-transaction_free(transaction);
 +}
 diff --git a/refs.h b/refs.h
 index a14fc5d..4b669f5 100644
 --- a/refs.h
 +++ b/refs.h
 ...
 +struct ref_be {
 + transaction_begin_fn transaction_begin;
 + transaction_update_sha1_fn transaction_update_sha1;
 + transaction_create_sha1_fn transaction_create_sha1;
 + transaction_delete_sha1_fn transaction_delete_sha1;
 + transaction_update_reflog_fn transaction_update_reflog;
 + transaction_commit_fn transaction_commit;
 + transaction_free_fn transaction_free;
 +};
 +
 +extern struct ref_be *refs;
 +
  #endif /* REFS_H */

The overall structure is certainly nice, but this means you only can
LINK with one backend.  Is that what we really want?

I would have expected something like this:

  * In refs.c, there is a static struct ref_be *the_refs_backend
that points at the chosen singleton backend;

  * Upon start-up, set_refs_backend() function that is exported from
refs.c can be used to set the_refs_backend;

  * Each refs-be-frotz.c will export struct ref_be refs_frotz (or
perhaps struct refs_be refs_be_frotz) to the outside world, so
that the start-up code can call set_refs_backend() with it.

  * It is probably sensible to keep the_refs_backend default to
refs_be_files.

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v6 2/6] Add git_env_ulong() to parse environment variable

2014-08-26 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 If it is truly some people prefer it one way and some the other, I am
 not sure if we should leave it as-is (that is preferring one way). 

A worse position is to have git_env_bool() that says empty is
false and add a new git_env_ulong() that says empty is unset.

We should pick one or the other and use it for both.

As you pointed out in the later part of the message, an empty string
is a valid way to spell false in the config subsystem since the
beginning at 17712991 (Add .git/config file parser, 2005-10-10);
consistency with it probably is sensible.

But perhaps my brain is rotten with too much Perl and Python X-.
I do not know where Linus picked up that, though ;-)

 The middle ground would be to die(). That does not seem super-friendly, but
 then we would also die with GIT_SMART_HTTP=foobar, so perhaps it is not
 unreasonable to just consider it a syntax error.

Hmm, I am not sure if dying is better.  Unless we decide to make
empty string no longer false everywhere and warn now and then later
die as part of a 3.0 transition plan or something, that is.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v3 03/23] refs.c: add a new refs.c file to hold all common refs code

2014-08-26 Thread Ronnie Sahlberg
On Tue, Aug 26, 2014 at 2:31 PM, Junio C Hamano gits...@pobox.com wrote:
 Ronnie Sahlberg sahlb...@google.com writes:

 Create a new erfs.c file that will be used to hold all the refs
 code that is backend agnostic and will be shared across all backends.

 The reason we renamed everything to refs-be-files.c in the previous patch
 and now start moving the common code back to the new refs.c file
 instead of the other way around is the etive volumes of code.

 Huh?  Why not create refs-be-files.c and move whatever need to be
 there over there, instead of rename the file and move things that
 shouldn't have been moved back like this?


The reason is the relative size of the code. I could do it the other
way but then
the changes that is moving the code would be much bigger.

Moving it like this, by first renaming it to refs-be-files.c and then
moving the backend agnostic parts back
is that the backend agnostic parts are mostly helper functions that
are independent of eachother.
This makes it possible to move just a few functions at a time making
the individual changes smaller and easier to manage when there are
merge conflicts.

A lot of the code that implements the actual files implementation for
refs storage implements the ref-cache/ref-dir/packed-refs/loose refs
etc.
This is all code that is intertwined and is difficult to split up.
Thus almost forcing me to move the whole 3000 lines of implementation
in one single monolithic patch.

I think first rename, then move the agnostic parts a small section at
a time was the least bad solution.




 Puzzled.

 I do not see 02/23 here, but I am assuming that is is just

 git mv refs.c refs-be-files.c

 which may have been a seven-line patch with format-patch -M ;-)


 With the ref_cache, packed refs and loose ref handling that are all
 part of the files based implementation the backend specific part
 of the old refs.c file is several times larger than the backend agnostic
 part. Therefore it makes more sense to first rename everything to be
 part of the files based backend and then move the parts that can be used
 as common code back to refs.c.

 Signed-off-by: Ronnie Sahlberg sahlb...@google.com
 ---
  Makefile | 1 +
  refs.c   | 3 +++
  2 files changed, 4 insertions(+)
  create mode 100644 refs.c

 diff --git a/Makefile b/Makefile
 index e010ad1..937d22a 100644
 --- a/Makefile
 +++ b/Makefile
 @@ -857,6 +857,7 @@ LIB_OBJS += quote.o
  LIB_OBJS += reachable.o
  LIB_OBJS += read-cache.o
  LIB_OBJS += reflog-walk.o
 +LIB_OBJS += refs.o
  LIB_OBJS += refs-be-files.o
  LIB_OBJS += remote.o
  LIB_OBJS += replace_object.o
 diff --git a/refs.c b/refs.c
 new file mode 100644
 index 000..77492ff
 --- /dev/null
 +++ b/refs.c
 @@ -0,0 +1,3 @@
 +/*
 + * Common refs code for all backends.
 + */
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


What's cooking in git.git (Aug 2014, #04; Tue, 26)

2014-08-26 Thread Junio C Hamano
Here are the topics that have been cooking.  Commits prefixed with
'-' are only in 'pu' (proposed updates) while commits prefixed with
'+' are in 'next'.

The 'maint' branch now is for the 2.1.x maintenance track, and a few
fixes for recent regressions have been merged to 'maint' and 'master'.
The 'next' has been rewound, while kicking a couple of topics back
to 'pu' per topic owners' requests.  I still haven't caught up with
new topics and rerolls that came late in the last cycle yet but
hopefully I can in the coming days.

You can find the changes described here in the integration branches
of the repositories listed at

http://git-blame.blogspot.com/p/git-public-repositories.html

--
[Graduated to master]

* jk/diff-tree-t-fix (2014-08-20) 1 commit
  (merged to 'next' on 2014-08-21 at 0f652d6)
 + intersect_paths: respect mode in git's tree-sort

 Fix (rarely used) git diff-tree -t regression in 2.0.


* jk/fix-profile-feedback-build (2014-08-19) 1 commit
  (merged to 'next' on 2014-08-21 at b282021)
 + Makefile: make perf tests optional for profile build

 Fix profile-feedback build broken in 2.1 for tarball releases.


* jk/pack-shallow-always-without-bitmap (2014-08-12) 1 commit
  (merged to 'next' on 2014-08-21 at e04c935)
 + pack-objects: turn off bitmaps when we see --shallow lines

 Reachability bitmaps do not work with shallow operations.
 Fixes regression in 2.0.

--
[New Topics]

* et/spell-poll-infinite-with-minus-one-only (2014-08-22) 1 commit
 - upload-pack: keep poll(2)'s timeout to -1

 We used to pass -1000 to poll(2), expecting it to also mean no
 timeout, which should be spelled as -1.

 Will merge to 'next'.


* jk/make-simplify-dependencies (2014-08-25) 2 commits
 - Makefile: use `find` to determine static header dependencies
 - i18n: treat make pot as an explicitly-invoked target

 Admit that keeping LIB_H up-to-date, only for those that do not use
 the automatically generated dependencies, is a losing battle, and
 make it conservative by making everything depend on anything.

 Will merge to 'next'.


* jk/prompt-stash-could-be-packed (2014-08-25) 1 commit
 - git-prompt: do not look for refs/stash in $GIT_DIR

 The prompt script checked $GIT_DIR/ref/stash file to see if there
 is a stash, which was a no-no.

 Will merge to 'next'.


* jk/prune-top-level-refs-after-packing (2014-08-25) 1 commit
 - pack-refs: prune top-level refs like refs/foo

 After pack-refs --prune packed refs at the top-level, it failed
 to prune them.

 Will merge to 'next'.


* jk/fast-import-fixes (2014-08-25) 2 commits
 - fast-import: fix buffer overflow in dump_tags
 - fast-import: clean up pack_data pointer in end_packfile

 With sufficiently long refnames, fast-import could have overflown
 an on-stack buffer.

 Will merge to 'next'.


* nd/fetch-pass-quiet-to-gc-child-process (2014-08-18) 2 commits
 - fetch: silence git-gc if --quiet is given
 - fetch: convert argv_gc_auto to struct argv_array

 Progress output from git gc --auto was visible in git fetch -q.

 Will merge to 'next'.


* rs/list-optim (2014-08-25) 2 commits
 - walker: avoid quadratic list insertion in mark_complete
 - sha1_name: avoid quadratic list insertion in handle_one_ref

 Fix a couple of accumulate into a sorted list to accumulate and
 then sort the list.

 Will merge to 'next'.


* sb/mailsplit-dead-code-removal (2014-08-13) 1 commit
 - mailsplit.c: remove dead code

 Will merge to 'next'.


* tb/pretty-format-cd-date-format (2014-08-21) 1 commit
 - pretty: note that %cd respects the --date= option

 Documentation update.

 Will merge to 'next'.


* jk/name-decoration-alloc (2014-08-26) 3 commits
 - log-tree: use FLEX_ARRAY in name_decoration
 - log-tree: make name_decoration hash static
 - log-tree: make add_name_decoration a public function

 The API to allocate the structure to keep track of commit
 decoration was cumbersome to use, inviting lazy code to
 overallocate memory.

 Will merge to 'next'.

--
[Stalled]

* cb/mergetool-difftool (2014-07-21) 2 commits
 - difftool: don't assume that default sh is sane
 - mergetool: don't require a work tree for --tool-help

 Update the way the difftool --help shows the help message that is
 shared with the mergetool to reduce one shell dependency.

 Will be rerolled.


* ta/config-add-to-empty-or-true-fix (2014-08-18) 1 commit
 - make config --add behave correctly for empty and NULL values

 Will be rerolled.


* rr/mergetool-temporary-filename-tweak (2014-08-21) 1 commit
 - Allow the user to change the temporary file name for mergetool

 Needs rerolling (new paragraph in doc seems to be in a wrong place)


* jk/tag-contains (2014-06-30) 8 commits
 . perf: add tests for tag --contains
 . tag: use commit_contains
 . commit: provide a fast multi-tip contains function
 . string-list: add pos to iterator callback
 . add functions for 

Re: [PATCH v2 18/23] refs.c: add a backend method structure with transaction functions

2014-08-26 Thread Ronnie Sahlberg
On Tue, Aug 26, 2014 at 2:25 PM, Junio C Hamano gits...@pobox.com wrote:
 Ronnie Sahlberg sahlb...@google.com writes:

 +struct ref_be {
 + transaction_begin_fn transaction_begin;
 + transaction_update_sha1_fn transaction_update_sha1;
 + transaction_create_sha1_fn transaction_create_sha1;
 + transaction_delete_sha1_fn transaction_delete_sha1;
 + transaction_update_reflog_fn transaction_update_reflog;
 + transaction_commit_fn transaction_commit;
 + transaction_free_fn transaction_free;
 +};
 +
 +extern struct ref_be *refs;

 The overall organization is nice, but please don't use such a short
 name for the systemwide default singleton instance, which should not
 be accessed by normal code other than via helpers that implicitly
 use that singleton (e.g. resolve_ref_unsafe() which invokes the
 method of the same name on the singleton, passing the parameters it
 received[*1*]).  The name will be used for other things (e.g. a
 local variable for a collection of refs) by code that do not care
 about the underlying implementation of the helpers and will cause
 confusion later.

 Perhaps the_refs_backend or something?

 Also does the singleton have to be extern, not a static inside refs.c,
 perhaps with a setter function to switch it or something?



Thanks!

I did these changes :
1, rename to the_refs_backend
2, add a function set_refs_backend()

To this series and update it at
https://github.com/rsahlberg/git/tree/backend-struct-db



 [Reference]

 *1* A typical helper that uses the singleton looks like this:

 +const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1,
 +  int reading, int *flag)
 +{
 +   return refs-resolve_ref_unsafe(ref, sha1, reading, flag);
 +}
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Transaction patch series overview

2014-08-26 Thread Jonathan Nieder
Hi again,

Junio C Hamano wrote:

 It seems that I can hold the topic in 'pu' a bit longer and expect a
 reroll from this effort before merging it to 'next'?

Sorry for the delay.  The following changes on top of master
(refs.c: change ref_transaction_update() to do error checking and
return status, 2014-07-14) are available in the git repository at

  git://repo.or.cz/git/jrn.git tags/rs/ref-transaction-1

for you to fetch changes up to 432ad1f39fd773b37757dd8f10b3075dc3d0d0a2:

  refs.c: make delete_ref use a transaction (2014-08-26 14:22:53 -0700)


Use ref transactions, early part

Ronnie explains:

This patch series expands on the transaction API. It converts
ref updates, inside refs.c as well as external, to use the
transaction API for updates. This makes most ref updates
atomic when there are failures locking or writing to a ref.

This series teaches the tag, replace, commit, cherry-pick,
fast-import, checkout -b, branch, receive-pack, and http-fetch
commands and all update_ref and delete_ref callers to use the ref
transaction API instead of lock_ref_sha1.

The main user-visible change should be some cosmetic changes to error
messages.  The series also combines multiple ref updates into a single
transaction in 'git http-fetch -w' and when writing tags in
fast-import but otherwise doesn't change the granularity of
transactions.

Reviewed at https://code-review.googlesource.com/#/q/topic:ref-transaction-1


Ronnie Sahlberg (20):
  refs.c: change ref_transaction_create to do error checking and return 
status
  refs.c: update ref_transaction_delete to check for error and return status
  refs.c: make ref_transaction_begin take an err argument
  refs.c: add transaction.status and track OPEN/CLOSED
  tag.c: use ref transactions when doing updates
  replace.c: use the ref transaction functions for updates
  commit.c: use ref transactions for updates
  sequencer.c: use ref transactions for all ref updates
  fast-import.c: change update_branch to use ref transactions
  branch.c: use ref transaction for all ref updates
  refs.c: change update_ref to use a transaction
  receive-pack.c: use a reference transaction for updating the refs
  fast-import.c: use a ref transaction when dumping tags
  walker.c: use ref transaction for ref updates
  refs.c: make lock_ref_sha1 static
  refs.c: remove the update_ref_lock function
  refs.c: remove the update_ref_write function
  refs.c: remove lock_ref_sha1
  refs.c: make prune_ref use a transaction to delete the ref
  refs.c: make delete_ref use a transaction

 branch.c   |  31 ---
 builtin/commit.c   |  25 +++--
 builtin/receive-pack.c |  25 +++--
 builtin/replace.c  |  14 +--
 builtin/tag.c  |  16 ++--
 builtin/update-ref.c   |  14 ++-
 fast-import.c  |  54 +++
 refs.c | 244 -
 refs.h |  77 
 sequencer.c|  26 --
 walker.c   |  70 --
 11 files changed, 367 insertions(+), 229 deletions(-)

[...]
 Jonathan Nieder jrnie...@gmail.com writes:

 --- a/builtin/receive-pack.c
 +++ b/builtin/receive-pack.c
 @@ -580,18 +580,18 @@ static char *update(struct command *cmd, struct 
 shallow_info *si)
[...]
  if (!transaction ||
  ref_transaction_update(transaction, namespaced_name,
 new_sha1, old_sha1, 0, 1, err) ||
  ref_transaction_commit(transaction, push, err)) {
 -char *str = strbuf_detach(err, NULL);
  ref_transaction_free(transaction);
  
 -rp_error(%s, str);
 -return str;
 +rp_error(%s, err.buf);
 +strbuf_release(err);
 +return failed to update ref;
  }

 I am guessing that the purpose of this change is to make sure that
 cmd-error_string is taken from a fixed set of strings, not an
 arbitrary collection of errors from the transaction subsystem, but
 what is the significance of doing so?  Do the other side i18n while
 running print_ref_status() or something?

It's about keeping the message in parentheses on the ! rejected
master - master line short and simple, since the more detailed
diagnosis is redundant next to the full message that is sent over
sideband earlier and gets a line to itself.

Side effects:

 * a less invasive patch --- no more need to change the calling
   convention to return a dynamically allocated string, with
   assertions throughout the file that check for leaks

 * using the same all lowercase and brief convention makes the
   message less jarring next to other statuses for ! rejected lines,
   like 

Re: [PATCH v3 19/23] refs-be-files.c: add a backend method structure with transaction functions

2014-08-26 Thread Ronnie Sahlberg
On Tue, Aug 26, 2014 at 2:38 PM, Junio C Hamano gits...@pobox.com wrote:
 Ronnie Sahlberg sahlb...@google.com writes:

 diff --git a/refs-be-files.c b/refs-be-files.c
 index e58a7e1..27eafd0 100644
 --- a/refs-be-files.c
 +++ b/refs-be-files.c
 ...
 +struct ref_be refs_files = {
 + files_transaction_begin,
 + files_transaction_update_sha1,
 + files_transaction_create_sha1,
 + files_transaction_delete_sha1,
 + files_transaction_update_reflog,
 + files_transaction_commit,
 + files_transaction_free,
 +};
 +
 +struct ref_be *refs = refs_files;

 diff --git a/refs.c b/refs.c
 index 6b434ad..b8c942f 100644
 --- a/refs.c
 +++ b/refs.c
 ...
 +void transaction_free(struct ref_transaction *transaction)
 +{
 + return refs-transaction_free(transaction);
 +}
 diff --git a/refs.h b/refs.h
 index a14fc5d..4b669f5 100644
 --- a/refs.h
 +++ b/refs.h
 ...
 +struct ref_be {
 + transaction_begin_fn transaction_begin;
 + transaction_update_sha1_fn transaction_update_sha1;
 + transaction_create_sha1_fn transaction_create_sha1;
 + transaction_delete_sha1_fn transaction_delete_sha1;
 + transaction_update_reflog_fn transaction_update_reflog;
 + transaction_commit_fn transaction_commit;
 + transaction_free_fn transaction_free;
 +};
 +
 +extern struct ref_be *refs;
 +
  #endif /* REFS_H */

 The overall structure is certainly nice, but this means you only can
 LINK with one backend.  Is that what we really want?

 I would have expected something like this:

   * In refs.c, there is a static struct ref_be *the_refs_backend
 that points at the chosen singleton backend;

Done.
It is also initialized to default to the files backend :
refs.c:

  /* We always have a files backend and it is the default */
  struct ref_be *the_refs_backend = refs_be_files;


This does make refs_be_files and later refs_be_db public symbols instead of
the singleton. But we probably want/need these structures to be public anyway
if we at some stage want to be able to switch between backends at runtime.
refs.h:

   extern struct ref_be refs_be_files;
   void set_refs_backend(struct ref_be *ref_be);

Thus allowing us to
   set_refs_backend(refs_be_files) to switch back to the files backend.





   * Upon start-up, set_refs_backend() function that is exported from
 refs.c can be used to set the_refs_backend;

   * Each refs-be-frotz.c will export struct ref_be refs_frotz (or
 perhaps struct refs_be refs_be_frotz) to the outside world, so
 that the start-up code can call set_refs_backend() with it.

Yepp.
refs-be-db.c: does this.


   * It is probably sensible to keep the_refs_backend default to
 refs_be_files.


Yepp.

https://github.com/rsahlberg/git/tree/backend-struct-db
https://github.com/rsahlberg/git/tree/backend-struct-db-2 (adds a db
backend and daemon)


Thanks. Good suggestions.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] git-imap-send: use libcurl for implementation

2014-08-26 Thread Bernhard Reiter
Use libcurl's high-level API functions to implement git-imap-send
instead of the previous low-level OpenSSL-based functions.

Since version 7.30.0, libcurl's API has been able to communicate with
IMAP servers. Using those high-level functions instead of the current
ones would reduce imap-send.c by some 1200 lines of code. For now,
the old ones are wrapped in #ifdefs, and the new functions are enabled
by make if curl's version is = 7.35.0, from which version on curl's
CURLOPT_LOGIN_OPTIONS (enabling IMAP authentication) parameter has been
available.

As I don't have access to that many IMAP servers, I haven't been able to
test the new code with a wide variety of parameter combinations. I did
test both secure and insecure (imaps:// and imap://) connections and
values of PLAIN and LOGIN for the authMethod.

Signed-off-by: Bernhard Reiter ock...@raz.or.at
---
I rebased the patch on the pu branch, hope that was the right thing to do.

Bernhard

 Documentation/git-imap-send.txt |   3 +-
 INSTALL |  15 ++--
 Makefile|  16 +++-
 git.spec.in |   5 +-
 imap-send.c | 165 +---
 5 files changed, 167 insertions(+), 37 deletions(-)

diff --git a/Documentation/git-imap-send.txt b/Documentation/git-imap-send.txt
index 7d991d9..9d244c4 100644
--- a/Documentation/git-imap-send.txt
+++ b/Documentation/git-imap-send.txt
@@ -75,7 +75,8 @@ imap.preformattedHTML::
 
 imap.authMethod::
Specify authenticate method for authentication with IMAP server.
-   Current supported method is 'CRAM-MD5' only. If this is not set
+   If you compiled git with the NO_CURL option or if your curl version is
+7.35.0, the only supported method is 'CRAM-MD5'. If this is not set
then 'git imap-send' uses the basic IMAP plaintext LOGIN command.
 
 Examples
diff --git a/INSTALL b/INSTALL
index 6ec7a24..e2770a0 100644
--- a/INSTALL
+++ b/INSTALL
@@ -108,18 +108,21 @@ Issues of note:
  so you might need to install additional packages other than Perl
  itself, e.g. Time::HiRes.
 
-   - openssl library is used by git-imap-send to use IMAP over SSL.
- If you don't need it, use NO_OPENSSL.
+   - openssl library is used by git-imap-send to use IMAP over SSL,
+ unless you're using curl = 7.35.0, in which case that will be
+ used. If you don't need git-imap-send, you can use NO_OPENSSL.
 
  By default, git uses OpenSSL for SHA1 but it will use its own
  library (inspired by Mozilla's) with either NO_OPENSSL or
  BLK_SHA1.  Also included is a version optimized for PowerPC
  (PPC_SHA1).
 
-   - libcurl library is used by git-http-fetch and git-fetch.  You
- might also want the curl executable for debugging purposes.
- If you do not use http:// or https:// repositories, you do not
- have to have them (use NO_CURL).
+   - libcurl library is used by git-http-fetch, git-fetch, and, if
+ the curl version = 7.35.0, for git-imap-send.  You might also
+ want the curl executable for debugging purposes. If you do not
+ use http:// or https:// repositories, and do not want to put
+ patches into an IMAP mailbox, you do not have to have them
+ (use NO_CURL).
 
- expat library; git-http-push uses it for remote lock
  management over DAV.  Similar to curl above, this is optional
diff --git a/Makefile b/Makefile
index 237bc05..c08963c 100644
--- a/Makefile
+++ b/Makefile
@@ -1133,6 +1133,9 @@ ifdef HAVE_ALLOCA_H
BASIC_CFLAGS += -DHAVE_ALLOCA_H
 endif
 
+IMAP_SEND_BUILDDEPS =
+IMAP_SEND_LDFLAGS = $(OPENSSL_LINK) $(OPENSSL_LIBSSL) $(LIB_4_CRYPTO)
+
 ifdef NO_CURL
BASIC_CFLAGS += -DNO_CURL
REMOTE_CURL_PRIMARY =
@@ -1167,6 +1170,15 @@ else
PROGRAM_OBJS += http-push.o
endif
endif
+   curl_check := $(shell (echo 072300; curl-config --vernum) 2/dev/null | 
sort -r | sed -ne 2p)
+   ifeq $(curl_check) 072300
+   USE_CURL_FOR_IMAP_SEND = YesPlease
+   endif
+   ifdef USE_CURL_FOR_IMAP_SEND
+   BASIC_CFLAGS += -DUSE_CURL_FOR_IMAP_SEND
+   IMAP_SEND_BUILDDEPS = http.o
+   IMAP_SEND_LDFLAGS += $(CURL_LIBCURL)
+   endif
ifndef NO_EXPAT
ifdef EXPATDIR
BASIC_CFLAGS += -I$(EXPATDIR)/include
@@ -2084,9 +2096,9 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) 
$(LIBS)
 
-git-imap-send$X: imap-send.o GIT-LDFLAGS $(GITLIBS)
+git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
-   $(LIBS) $(OPENSSL_LINK) $(OPENSSL_LIBSSL) $(LIB_4_CRYPTO)
+   $(LIBS) $(IMAP_SEND_LDFLAGS)
 
 git-http-fetch$X: http.o 

[PATCH 0/20] rs/ref-transaction-1 (Re: Transaction patch series overview)

2014-08-26 Thread Jonathan Nieder
Jonathan Nieder wrote:

 This series teaches the tag, replace, commit, cherry-pick,
 fast-import, checkout -b, branch, receive-pack, and http-fetch
 commands and all update_ref and delete_ref callers to use the ref
 transaction API instead of lock_ref_sha1.

 The main user-visible change should be some cosmetic changes to error
 messages.  The series also combines multiple ref updates into a single
 transaction in 'git http-fetch -w' and when writing tags in
 fast-import but otherwise doesn't change the granularity of
 transactions.

 Reviewed at https://code-review.googlesource.com/#/q/topic:ref-transaction-1

 
 Ronnie Sahlberg (20):
   refs.c: change ref_transaction_create to do error checking and return 
 status
   refs.c: update ref_transaction_delete to check for error and return 
 status
   refs.c: make ref_transaction_begin take an err argument
   refs.c: add transaction.status and track OPEN/CLOSED
   tag.c: use ref transactions when doing updates
   replace.c: use the ref transaction functions for updates
   commit.c: use ref transactions for updates
   sequencer.c: use ref transactions for all ref updates
   fast-import.c: change update_branch to use ref transactions
   branch.c: use ref transaction for all ref updates
   refs.c: change update_ref to use a transaction
   receive-pack.c: use a reference transaction for updating the refs
   fast-import.c: use a ref transaction when dumping tags
   walker.c: use ref transaction for ref updates
   refs.c: make lock_ref_sha1 static
   refs.c: remove the update_ref_lock function
   refs.c: remove the update_ref_write function
   refs.c: remove lock_ref_sha1
   refs.c: make prune_ref use a transaction to delete the ref
   refs.c: make delete_ref use a transaction

And here are the patches.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 02/20] refs.c: update ref_transaction_delete to check for error and return status

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Wed, 16 Apr 2014 15:27:45 -0700

Change ref_transaction_delete() to do basic error checking and return
non-zero of error. Update all callers to check the return for
ref_transaction_delete(). There are currently no conditions in _delete that
will return error but there will be in the future. Add an err argument that
will be updated on failure.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 builtin/update-ref.c |  5 +++--
 refs.c   | 16 +++-
 refs.h   | 12 
 3 files changed, 22 insertions(+), 11 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 41121fa..7c9c248 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -258,8 +258,9 @@ static const char *parse_cmd_delete(struct strbuf *input, 
const char *next)
if (*next != line_termination)
die(delete %s: extra input: %s, refname, next);
 
-   ref_transaction_delete(transaction, refname, old_sha1,
-  update_flags, have_old);
+   if (ref_transaction_delete(transaction, refname, old_sha1,
+  update_flags, have_old, err))
+   die(%s, err.buf);
 
update_flags = 0;
free(refname);
diff --git a/refs.c b/refs.c
index c49f1c6..40f04f4 100644
--- a/refs.c
+++ b/refs.c
@@ -3469,19 +3469,25 @@ int ref_transaction_create(struct ref_transaction 
*transaction,
return 0;
 }
 
-void ref_transaction_delete(struct ref_transaction *transaction,
-   const char *refname,
-   const unsigned char *old_sha1,
-   int flags, int have_old)
+int ref_transaction_delete(struct ref_transaction *transaction,
+  const char *refname,
+  const unsigned char *old_sha1,
+  int flags, int have_old,
+  struct strbuf *err)
 {
-   struct ref_update *update = add_update(transaction, refname);
+   struct ref_update *update;
 
+   if (have_old  !old_sha1)
+   die(BUG: have_old is true but old_sha1 is NULL);
+
+   update = add_update(transaction, refname);
update-flags = flags;
update-have_old = have_old;
if (have_old) {
assert(!is_null_sha1(old_sha1));
hashcpy(update-old_sha1, old_sha1);
}
+   return 0;
 }
 
 int update_ref(const char *action, const char *refname,
diff --git a/refs.h b/refs.h
index b648819..71389a1 100644
--- a/refs.h
+++ b/refs.h
@@ -308,11 +308,15 @@ int ref_transaction_create(struct ref_transaction 
*transaction,
  * Add a reference deletion to transaction.  If have_old is true, then
  * old_sha1 holds the value that the reference should have had before
  * the update (which must not be the null SHA-1).
+ * Function returns 0 on success and non-zero on failure. A failure to delete
+ * means that the transaction as a whole has failed and will need to be
+ * rolled back.
  */
-void ref_transaction_delete(struct ref_transaction *transaction,
-   const char *refname,
-   const unsigned char *old_sha1,
-   int flags, int have_old);
+int ref_transaction_delete(struct ref_transaction *transaction,
+  const char *refname,
+  const unsigned char *old_sha1,
+  int flags, int have_old,
+  struct strbuf *err);
 
 /*
  * Commit all of the changes that have been queued in transaction, as
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 01/20] refs.c: change ref_transaction_create to do error checking and return status

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Wed, 16 Apr 2014 15:26:44 -0700

Do basic error checking in ref_transaction_create() and make it return
non-zero on error. Update all callers to check the result of
ref_transaction_create(). There are currently no conditions in _create that
will return error but there will be in the future. Add an err argument that
will be updated on failure.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 builtin/update-ref.c |  4 +++-
 refs.c   | 18 --
 refs.h   | 48 +---
 3 files changed, 56 insertions(+), 14 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 3067b11..41121fa 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -226,7 +226,9 @@ static const char *parse_cmd_create(struct strbuf *input, 
const char *next)
if (*next != line_termination)
die(create %s: extra input: %s, refname, next);
 
-   ref_transaction_create(transaction, refname, new_sha1, update_flags);
+   if (ref_transaction_create(transaction, refname, new_sha1,
+  update_flags, err))
+   die(%s, err.buf);
 
update_flags = 0;
free(refname);
diff --git a/refs.c b/refs.c
index 3f05e88..c49f1c6 100644
--- a/refs.c
+++ b/refs.c
@@ -3449,18 +3449,24 @@ int ref_transaction_update(struct ref_transaction 
*transaction,
return 0;
 }
 
-void ref_transaction_create(struct ref_transaction *transaction,
-   const char *refname,
-   const unsigned char *new_sha1,
-   int flags)
+int ref_transaction_create(struct ref_transaction *transaction,
+  const char *refname,
+  const unsigned char *new_sha1,
+  int flags,
+  struct strbuf *err)
 {
-   struct ref_update *update = add_update(transaction, refname);
+   struct ref_update *update;
+
+   if (!new_sha1 || is_null_sha1(new_sha1))
+   die(BUG: create ref with null new_sha1);
+
+   update = add_update(transaction, refname);
 
-   assert(!is_null_sha1(new_sha1));
hashcpy(update-new_sha1, new_sha1);
hashclr(update-old_sha1);
update-flags = flags;
update-have_old = 1;
+   return 0;
 }
 
 void ref_transaction_delete(struct ref_transaction *transaction,
diff --git a/refs.h b/refs.h
index c5376ce..b648819 100644
--- a/refs.h
+++ b/refs.h
@@ -10,6 +10,38 @@ struct ref_lock {
int force_write;
 };
 
+/*
+ * A ref_transaction represents a collection of ref updates
+ * that should succeed or fail together.
+ *
+ * Calling sequence
+ * 
+ * - Allocate and initialize a `struct ref_transaction` by calling
+ *   `ref_transaction_begin()`.
+ *
+ * - List intended ref updates by calling functions like
+ *   `ref_transaction_update()` and `ref_transaction_create()`.
+ *
+ * - Call `ref_transaction_commit()` to execute the transaction.
+ *   If this succeeds, the ref updates will have taken place and
+ *   the transaction cannot be rolled back.
+ *
+ * - At any time call `ref_transaction_free()` to discard the
+ *   transaction and free associated resources.  In particular,
+ *   this rolls back the transaction if it has not been
+ *   successfully committed.
+ *
+ * Error handling
+ * --
+ *
+ * On error, transaction functions append a message about what
+ * went wrong to the 'err' argument.  The message mentions what
+ * ref was being updated (if any) when the error occurred so it
+ * can be passed to 'die' or 'error' as-is.
+ *
+ * The message is appended to err without first clearing err.
+ * err will not be '\n' terminated.
+ */
 struct ref_transaction;
 
 /*
@@ -248,7 +280,7 @@ struct ref_transaction *ref_transaction_begin(void);
  * it must not have existed beforehand.
  * Function returns 0 on success and non-zero on failure. A failure to update
  * means that the transaction as a whole has failed and will need to be
- * rolled back. On failure the err buffer will be updated.
+ * rolled back.
  */
 int ref_transaction_update(struct ref_transaction *transaction,
   const char *refname,
@@ -262,11 +294,15 @@ int ref_transaction_update(struct ref_transaction 
*transaction,
  * that the reference should have after the update; it must not be the
  * null SHA-1.  It is verified that the reference does not exist
  * already.
+ * Function returns 0 on success and non-zero on failure. A failure to create
+ * means that the transaction as a whole has failed and will need to be
+ * rolled back.
  */
-void ref_transaction_create(struct ref_transaction *transaction,
-   const char *refname,
-   const unsigned char *new_sha1,
-   int flags);
+int 

Re: [PATCH 3/3] log-tree: use FLEX_ARRAY in name_decoration

2014-08-26 Thread Eric Sunshine
On Tue, Aug 26, 2014 at 6:24 AM, Jeff King p...@peff.net wrote:
 We are already using the flex-array technique; let's
 annotate it with our usual FLEX_ARRAY macro. Besides being
 more readable, this is slightly more efficient on compilers
 that understand flex-arrays.

 Note that we need to bump the allocation in add_name_decoration,
 which did not explicitly add one byte for the NUL terminator
 of the string we putting into the flex-array (it did not

s/we/we are/

 need to before, because the struct itself was over-allocated
 by one byte).

 Signed-off-by: Jeff King p...@peff.net
 ---
 This could come first in the series, but doing it last means we only
 have to update one spot. :)

  commit.h   | 2 +-
  log-tree.c | 2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)

 diff --git a/commit.h b/commit.h
 index 263b49e..1516fc9 100644
 --- a/commit.h
 +++ b/commit.h
 @@ -29,7 +29,7 @@ extern const char *commit_type;
  struct name_decoration {
 struct name_decoration *next;
 int type;
 -   char name[1];
 +   char name[FLEX_ARRAY];
  };

  enum decoration_type {
 diff --git a/log-tree.c b/log-tree.c
 index 7cbc4ee..fb60018 100644
 --- a/log-tree.c
 +++ b/log-tree.c
 @@ -77,7 +77,7 @@ int parse_decorate_color_config(const char *var, const int 
 ofs, const char *valu
  void add_name_decoration(enum decoration_type type, const char *name, struct 
 object *obj)
  {
 int nlen = strlen(name);
 -   struct name_decoration *res = xmalloc(sizeof(struct name_decoration) 
 + nlen);
 +   struct name_decoration *res = xmalloc(sizeof(*res) + nlen + 1);
 memcpy(res-name, name, nlen + 1);
 res-type = type;
 res-next = add_decoration(name_decoration, obj, res);
 --
 2.1.0.346.ga0367b9
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 03/20] refs.c: make ref_transaction_begin take an err argument

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Mon, 19 May 2014 10:42:34 -0700

Add an err argument to _begin so that on non-fatal failures in future ref
backends we can report a nice error back to the caller.
While _begin can currently never fail for other reasons than OOM, in which
case we die() anyway, we may add other types of backends in the future.
For example, a hypothetical MySQL backend could fail in _begin with
Can not connect to MySQL server. No route to host.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 builtin/update-ref.c | 5 -
 refs.c   | 2 +-
 refs.h   | 2 +-
 3 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 7c9c248..96a53b9 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -365,7 +365,9 @@ int cmd_update_ref(int argc, const char **argv, const char 
*prefix)
die(Refusing to perform update with empty message.);
 
if (read_stdin) {
-   transaction = ref_transaction_begin();
+   transaction = ref_transaction_begin(err);
+   if (!transaction)
+   die(%s, err.buf);
if (delete || no_deref || argc  0)
usage_with_options(git_update_ref_usage, options);
if (end_null)
@@ -374,6 +376,7 @@ int cmd_update_ref(int argc, const char **argv, const char 
*prefix)
if (ref_transaction_commit(transaction, msg, err))
die(%s, err.buf);
ref_transaction_free(transaction);
+   strbuf_release(err);
return 0;
}
 
diff --git a/refs.c b/refs.c
index 40f04f4..9cb7908 100644
--- a/refs.c
+++ b/refs.c
@@ -3397,7 +3397,7 @@ struct ref_transaction {
size_t nr;
 };
 
-struct ref_transaction *ref_transaction_begin(void)
+struct ref_transaction *ref_transaction_begin(struct strbuf *err)
 {
return xcalloc(1, sizeof(struct ref_transaction));
 }
diff --git a/refs.h b/refs.h
index 71389a1..3f37c65 100644
--- a/refs.h
+++ b/refs.h
@@ -262,7 +262,7 @@ enum action_on_err {
  * Begin a reference transaction.  The reference transaction must
  * be freed by calling ref_transaction_free().
  */
-struct ref_transaction *ref_transaction_begin(void);
+struct ref_transaction *ref_transaction_begin(struct strbuf *err);
 
 /*
  * The following functions add a reference check or update to a
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 04/20] refs.c: add transaction.status and track OPEN/CLOSED

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Tue, 29 Apr 2014 12:06:19 -0700

Track the state of a transaction in a new state field. Check the field for
sanity, i.e. that state must be OPEN when _commit/_create/_delete or
_update is called or else die(BUG:...)

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 refs.c | 34 +-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 9cb7908..cc63056 100644
--- a/refs.c
+++ b/refs.c
@@ -3387,6 +3387,21 @@ struct ref_update {
 };
 
 /*
+ * Transaction states.
+ * OPEN:   The transaction is in a valid state and can accept new updates.
+ * An OPEN transaction can be committed.
+ * CLOSED: A closed transaction is no longer active and no other operations
+ * than free can be used on it in this state.
+ * A transaction can either become closed by successfully committing
+ * an active transaction or if there is a failure while building
+ * the transaction thus rendering it failed/inactive.
+ */
+enum ref_transaction_state {
+   REF_TRANSACTION_OPEN   = 0,
+   REF_TRANSACTION_CLOSED = 1
+};
+
+/*
  * Data structure for holding a reference transaction, which can
  * consist of checks and updates to multiple references, carried out
  * as atomically as possible.  This structure is opaque to callers.
@@ -3395,6 +3410,7 @@ struct ref_transaction {
struct ref_update **updates;
size_t alloc;
size_t nr;
+   enum ref_transaction_state state;
 };
 
 struct ref_transaction *ref_transaction_begin(struct strbuf *err)
@@ -3437,6 +3453,9 @@ int ref_transaction_update(struct ref_transaction 
*transaction,
 {
struct ref_update *update;
 
+   if (transaction-state != REF_TRANSACTION_OPEN)
+   die(BUG: update called for transaction that is not open);
+
if (have_old  !old_sha1)
die(BUG: have_old is true but old_sha1 is NULL);
 
@@ -3457,6 +3476,9 @@ int ref_transaction_create(struct ref_transaction 
*transaction,
 {
struct ref_update *update;
 
+   if (transaction-state != REF_TRANSACTION_OPEN)
+   die(BUG: create called for transaction that is not open);
+
if (!new_sha1 || is_null_sha1(new_sha1))
die(BUG: create ref with null new_sha1);
 
@@ -3477,6 +3499,9 @@ int ref_transaction_delete(struct ref_transaction 
*transaction,
 {
struct ref_update *update;
 
+   if (transaction-state != REF_TRANSACTION_OPEN)
+   die(BUG: delete called for transaction that is not open);
+
if (have_old  !old_sha1)
die(BUG: have_old is true but old_sha1 is NULL);
 
@@ -3532,8 +3557,13 @@ int ref_transaction_commit(struct ref_transaction 
*transaction,
int n = transaction-nr;
struct ref_update **updates = transaction-updates;
 
-   if (!n)
+   if (transaction-state != REF_TRANSACTION_OPEN)
+   die(BUG: commit called for transaction that is not open);
+
+   if (!n) {
+   transaction-state = REF_TRANSACTION_CLOSED;
return 0;
+   }
 
/* Allocate work space */
delnames = xmalloc(sizeof(*delnames) * n);
@@ -3595,6 +3625,8 @@ int ref_transaction_commit(struct ref_transaction 
*transaction,
clear_loose_ref_cache(ref_cache);
 
 cleanup:
+   transaction-state = REF_TRANSACTION_CLOSED;
+
for (i = 0; i  n; i++)
if (updates[i]-lock)
unlock_ref(updates[i]-lock);
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 05/20] tag.c: use ref transactions when doing updates

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Wed, 16 Apr 2014 15:30:41 -0700

Change tag.c to use ref transactions for all ref updates.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 builtin/tag.c | 16 ++--
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/builtin/tag.c b/builtin/tag.c
index c6e8a71..f3f172f 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -548,7 +548,6 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
struct strbuf ref = STRBUF_INIT;
unsigned char object[20], prev[20];
const char *object_ref, *tag;
-   struct ref_lock *lock;
struct create_tag_options opt;
char *cleanup_arg = NULL;
int annotate = 0, force = 0, lines = -1;
@@ -556,6 +555,8 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
const char *msgfile = NULL, *keyid = NULL;
struct msg_arg msg = { 0, STRBUF_INIT };
struct commit_list *with_commit = NULL;
+   struct ref_transaction *transaction;
+   struct strbuf err = STRBUF_INIT;
struct option options[] = {
OPT_CMDMODE('l', list, cmdmode, N_(list tag names), 'l'),
{ OPTION_INTEGER, 'n', NULL, lines, N_(n),
@@ -701,14 +702,17 @@ int cmd_tag(int argc, const char **argv, const char 
*prefix)
if (annotate)
create_tag(object, tag, buf, opt, prev, object);
 
-   lock = lock_any_ref_for_update(ref.buf, prev, 0, NULL);
-   if (!lock)
-   die(_(%s: cannot lock the ref), ref.buf);
-   if (write_ref_sha1(lock, object, NULL)  0)
-   die(_(%s: cannot update the ref), ref.buf);
+   transaction = ref_transaction_begin(err);
+   if (!transaction ||
+   ref_transaction_update(transaction, ref.buf, object, prev,
+  0, 1, err) ||
+   ref_transaction_commit(transaction, NULL, err))
+   die(%s, err.buf);
+   ref_transaction_free(transaction);
if (force  !is_null_sha1(prev)  hashcmp(prev, object))
printf(_(Updated tag '%s' (was %s)\n), tag, 
find_unique_abbrev(prev, DEFAULT_ABBREV));
 
+   strbuf_release(err);
strbuf_release(buf);
strbuf_release(ref);
return 0;
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 06/20] replace.c: use the ref transaction functions for updates

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Wed, 16 Apr 2014 15:32:29 -0700

Update replace.c to use ref transactions for updates.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 builtin/replace.c | 14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/builtin/replace.c b/builtin/replace.c
index 1bb491d..1fcd06d 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -153,7 +153,8 @@ static int replace_object_sha1(const char *object_ref,
unsigned char prev[20];
enum object_type obj_type, repl_type;
char ref[PATH_MAX];
-   struct ref_lock *lock;
+   struct ref_transaction *transaction;
+   struct strbuf err = STRBUF_INIT;
 
obj_type = sha1_object_info(object, NULL);
repl_type = sha1_object_info(repl, NULL);
@@ -166,12 +167,13 @@ static int replace_object_sha1(const char *object_ref,
 
check_ref_valid(object, prev, ref, sizeof(ref), force);
 
-   lock = lock_any_ref_for_update(ref, prev, 0, NULL);
-   if (!lock)
-   die(%s: cannot lock the ref, ref);
-   if (write_ref_sha1(lock, repl, NULL)  0)
-   die(%s: cannot update the ref, ref);
+   transaction = ref_transaction_begin(err);
+   if (!transaction ||
+   ref_transaction_update(transaction, ref, repl, prev, 0, 1, err) ||
+   ref_transaction_commit(transaction, NULL, err))
+   die(%s, err.buf);
 
+   ref_transaction_free(transaction);
return 0;
 }
 
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 07/20] commit.c: use ref transactions for updates

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Wed, 16 Apr 2014 15:34:19 -0700

Change commit.c to use ref transactions for all ref updates.
Make sure we pass a NULL pointer to ref_transaction_update if have_old
is false.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 builtin/commit.c | 25 -
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 5e2221c..9bf1003 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1627,11 +1627,12 @@ int cmd_commit(int argc, const char **argv, const char 
*prefix)
const char *index_file, *reflog_msg;
char *nl;
unsigned char sha1[20];
-   struct ref_lock *ref_lock;
struct commit_list *parents = NULL, **pptr = parents;
struct stat statbuf;
struct commit *current_head = NULL;
struct commit_extra_header *extra = NULL;
+   struct ref_transaction *transaction;
+   struct strbuf err = STRBUF_INIT;
 
if (argc == 2  !strcmp(argv[1], -h))
usage_with_options(builtin_commit_usage, 
builtin_commit_options);
@@ -1753,16 +1754,6 @@ int cmd_commit(int argc, const char **argv, const char 
*prefix)
strbuf_release(author_ident);
free_commit_extra_headers(extra);
 
-   ref_lock = lock_any_ref_for_update(HEAD,
-  !current_head
-  ? NULL
-  : current_head-object.sha1,
-  0, NULL);
-   if (!ref_lock) {
-   rollback_index_files();
-   die(_(cannot lock HEAD ref));
-   }
-
nl = strchr(sb.buf, '\n');
if (nl)
strbuf_setlen(sb, nl + 1 - sb.buf);
@@ -1771,10 +1762,17 @@ int cmd_commit(int argc, const char **argv, const char 
*prefix)
strbuf_insert(sb, 0, reflog_msg, strlen(reflog_msg));
strbuf_insert(sb, strlen(reflog_msg), : , 2);
 
-   if (write_ref_sha1(ref_lock, sha1, sb.buf)  0) {
+   transaction = ref_transaction_begin(err);
+   if (!transaction ||
+   ref_transaction_update(transaction, HEAD, sha1,
+  current_head
+  ? current_head-object.sha1 : NULL,
+  0, !!current_head, err) ||
+   ref_transaction_commit(transaction, sb.buf, err)) {
rollback_index_files();
-   die(_(cannot update HEAD ref));
+   die(%s, err.buf);
}
+   ref_transaction_free(transaction);
 
unlink(git_path(CHERRY_PICK_HEAD));
unlink(git_path(REVERT_HEAD));
@@ -1803,5 +1801,6 @@ int cmd_commit(int argc, const char **argv, const char 
*prefix)
if (!quiet)
print_summary(prefix, sha1, !current_head);
 
+   strbuf_release(err);
return 0;
 }
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 08/20] sequencer.c: use ref transactions for all ref updates

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Wed, 16 Apr 2014 15:37:45 -0700

Change to use ref transactions for all updates to refs.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 sequencer.c | 26 ++
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 0a80c58..5e93b6a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -272,23 +272,33 @@ static int error_dirty_index(struct replay_opts *opts)
 static int fast_forward_to(const unsigned char *to, const unsigned char *from,
int unborn, struct replay_opts *opts)
 {
-   struct ref_lock *ref_lock;
+   struct ref_transaction *transaction;
struct strbuf sb = STRBUF_INIT;
-   int ret;
+   struct strbuf err = STRBUF_INIT;
 
read_cache();
if (checkout_fast_forward(from, to, 1))
exit(1); /* the callee should have complained already */
-   ref_lock = lock_any_ref_for_update(HEAD, unborn ? null_sha1 : from,
-  0, NULL);
-   if (!ref_lock)
-   return error(_(Failed to lock HEAD during fast_forward_to));
 
strbuf_addf(sb, %s: fast-forward, action_name(opts));
-   ret = write_ref_sha1(ref_lock, to, sb.buf);
+
+   transaction = ref_transaction_begin(err);
+   if (!transaction ||
+   ref_transaction_update(transaction, HEAD,
+  to, unborn ? null_sha1 : from,
+  0, 1, err) ||
+   ref_transaction_commit(transaction, sb.buf, err)) {
+   ref_transaction_free(transaction);
+   error(%s, err.buf);
+   strbuf_release(sb);
+   strbuf_release(err);
+   return -1;
+   }
 
strbuf_release(sb);
-   return ret;
+   strbuf_release(err);
+   ref_transaction_free(transaction);
+   return 0;
 }
 
 static int do_recursive_merge(struct commit *base, struct commit *next,
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 10/20] branch.c: use ref transaction for all ref updates

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Wed, 16 Apr 2014 16:21:53 -0700

Change create_branch to use a ref transaction when creating the new branch.

This also fixes a race condition in the old code where two concurrent
create_branch could race since the lock_any_ref_for_update/write_ref_sha1
did not protect against the ref already existing. I.e. one thread could end up
overwriting a branch even if the forcing flag is false.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 branch.c | 31 +--
 1 file changed, 17 insertions(+), 14 deletions(-)

diff --git a/branch.c b/branch.c
index 660097b..37ac555 100644
--- a/branch.c
+++ b/branch.c
@@ -226,7 +226,6 @@ void create_branch(const char *head,
   int force, int reflog, int clobber_head,
   int quiet, enum branch_track track)
 {
-   struct ref_lock *lock = NULL;
struct commit *commit;
unsigned char sha1[20];
char *real_ref, msg[PATH_MAX + 20];
@@ -285,15 +284,6 @@ void create_branch(const char *head,
die(_(Not a valid branch point: '%s'.), start_name);
hashcpy(sha1, commit-object.sha1);
 
-   if (!dont_change_ref) {
-   lock = lock_any_ref_for_update(ref.buf, NULL, 0, NULL);
-   if (!lock)
-   die_errno(_(Failed to lock ref for update));
-   }
-
-   if (reflog)
-   log_all_ref_updates = 1;
-
if (forcing)
snprintf(msg, sizeof msg, branch: Reset to %s,
 start_name);
@@ -301,13 +291,26 @@ void create_branch(const char *head,
snprintf(msg, sizeof msg, branch: Created from %s,
 start_name);
 
+   if (reflog)
+   log_all_ref_updates = 1;
+
+   if (!dont_change_ref) {
+   struct ref_transaction *transaction;
+   struct strbuf err = STRBUF_INIT;
+
+   transaction = ref_transaction_begin(err);
+   if (!transaction ||
+   ref_transaction_update(transaction, ref.buf, sha1,
+  null_sha1, 0, !forcing, err) ||
+   ref_transaction_commit(transaction, msg, err))
+   die(%s, err.buf);
+   ref_transaction_free(transaction);
+   strbuf_release(err);
+   }
+
if (real_ref  track)
setup_tracking(ref.buf + 11, real_ref, track, quiet);
 
-   if (!dont_change_ref)
-   if (write_ref_sha1(lock, sha1, msg)  0)
-   die_errno(_(Failed to write ref));
-
strbuf_release(ref);
free(real_ref);
 }
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 09/20] fast-import.c: change update_branch to use ref transactions

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Wed, 16 Apr 2014 16:21:13 -0700

Change update_branch() to use ref transactions for updates.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 fast-import.c | 25 +++--
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/fast-import.c b/fast-import.c
index 6707a66..79160d5 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1679,8 +1679,9 @@ found_entry:
 static int update_branch(struct branch *b)
 {
static const char *msg = fast-import;
-   struct ref_lock *lock;
+   struct ref_transaction *transaction;
unsigned char old_sha1[20];
+   struct strbuf err = STRBUF_INIT;
 
if (read_ref(b-name, old_sha1))
hashclr(old_sha1);
@@ -1689,29 +1690,33 @@ static int update_branch(struct branch *b)
delete_ref(b-name, old_sha1, 0);
return 0;
}
-   lock = lock_any_ref_for_update(b-name, old_sha1, 0, NULL);
-   if (!lock)
-   return error(Unable to lock %s, b-name);
if (!force_update  !is_null_sha1(old_sha1)) {
struct commit *old_cmit, *new_cmit;
 
old_cmit = lookup_commit_reference_gently(old_sha1, 0);
new_cmit = lookup_commit_reference_gently(b-sha1, 0);
-   if (!old_cmit || !new_cmit) {
-   unlock_ref(lock);
+   if (!old_cmit || !new_cmit)
return error(Branch %s is missing commits., b-name);
-   }
 
if (!in_merge_bases(old_cmit, new_cmit)) {
-   unlock_ref(lock);
warning(Not updating %s
 (new tip %s does not contain %s),
b-name, sha1_to_hex(b-sha1), 
sha1_to_hex(old_sha1));
return -1;
}
}
-   if (write_ref_sha1(lock, b-sha1, msg)  0)
-   return error(Unable to update %s, b-name);
+   transaction = ref_transaction_begin(err);
+   if (!transaction ||
+   ref_transaction_update(transaction, b-name, b-sha1, old_sha1,
+  0, 1, err) ||
+   ref_transaction_commit(transaction, msg, err)) {
+   ref_transaction_free(transaction);
+   error(%s, err.buf);
+   strbuf_release(err);
+   return -1;
+   }
+   ref_transaction_free(transaction);
+   strbuf_release(err);
return 0;
 }
 
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 11/20] refs.c: change update_ref to use a transaction

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Thu, 24 Apr 2014 16:36:55 -0700

Change the update_ref helper function to use a ref transaction internally.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 refs.c | 29 +
 1 file changed, 25 insertions(+), 4 deletions(-)

diff --git a/refs.c b/refs.c
index cc63056..dcc877b 100644
--- a/refs.c
+++ b/refs.c
@@ -3519,11 +3519,32 @@ int update_ref(const char *action, const char *refname,
   const unsigned char *sha1, const unsigned char *oldval,
   int flags, enum action_on_err onerr)
 {
-   struct ref_lock *lock;
-   lock = update_ref_lock(refname, oldval, flags, NULL, onerr);
-   if (!lock)
+   struct ref_transaction *t;
+   struct strbuf err = STRBUF_INIT;
+
+   t = ref_transaction_begin(err);
+   if (!t ||
+   ref_transaction_update(t, refname, sha1, oldval, flags,
+  !!oldval, err) ||
+   ref_transaction_commit(t, action, err)) {
+   const char *str = update_ref failed for ref '%s': %s;
+
+   ref_transaction_free(t);
+   switch (onerr) {
+   case UPDATE_REFS_MSG_ON_ERR:
+   error(str, refname, err.buf);
+   break;
+   case UPDATE_REFS_DIE_ON_ERR:
+   die(str, refname, err.buf);
+   break;
+   case UPDATE_REFS_QUIET_ON_ERR:
+   break;
+   }
+   strbuf_release(err);
return 1;
-   return update_ref_write(action, refname, sha1, lock, NULL, onerr);
+   }
+   strbuf_release(err);
+   return 0;
 }
 
 static int ref_update_compare(const void *r1, const void *r2)
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 12/20] receive-pack.c: use a reference transaction for updating the refs

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Mon, 28 Apr 2014 14:36:15 -0700

Wrap all the ref updates inside a transaction.

In the new API there is no distinction between failure to lock and
failure to write a ref.  Both can be permanent (e.g., a ref
refs/heads/topic is blocking creation of the lock file
refs/heads/topic/1.lock) or transient (e.g., file system full) and
there's no clear difference in how the client should respond, so
replace the two statuses failed to lock and failed to write with
a single status failed to update ref.  In both cases a more
detailed message is sent by sideband to diagnose the problem.

Example, before:

 error: there are still refs under 'refs/heads/topic'
 remote: error: failed to lock refs/heads/topic
 To foo
  ! [remote rejected] HEAD - topic (failed to lock)

After:

 error: there are still refs under 'refs/heads/topic'
 remote: error: Cannot lock the ref 'refs/heads/topic'.
 To foo
  ! [remote rejected] HEAD - topic (failed to update ref)

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 builtin/receive-pack.c | 25 -
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index c323081..224fadc 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -475,7 +475,6 @@ static const char *update(struct command *cmd, struct 
shallow_info *si)
const char *namespaced_name;
unsigned char *old_sha1 = cmd-old_sha1;
unsigned char *new_sha1 = cmd-new_sha1;
-   struct ref_lock *lock;
 
/* only refs/... are allowed */
if (!starts_with(name, refs/) || check_refname_format(name + 5, 0)) {
@@ -576,19 +575,27 @@ static const char *update(struct command *cmd, struct 
shallow_info *si)
return NULL; /* good */
}
else {
+   struct strbuf err = STRBUF_INIT;
+   struct ref_transaction *transaction;
+
if (shallow_update  si-shallow_ref[cmd-index] 
update_shallow_ref(cmd, si))
return shallow error;
 
-   lock = lock_any_ref_for_update(namespaced_name, old_sha1,
-  0, NULL);
-   if (!lock) {
-   rp_error(failed to lock %s, name);
-   return failed to lock;
-   }
-   if (write_ref_sha1(lock, new_sha1, push)) {
-   return failed to write; /* error() already called */
+   transaction = ref_transaction_begin(err);
+   if (!transaction ||
+   ref_transaction_update(transaction, namespaced_name,
+  new_sha1, old_sha1, 0, 1, err) ||
+   ref_transaction_commit(transaction, push, err)) {
+   ref_transaction_free(transaction);
+
+   rp_error(%s, err.buf);
+   strbuf_release(err);
+   return failed to update ref;
}
+
+   ref_transaction_free(transaction);
+   strbuf_release(err);
return NULL; /* good */
}
 }
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 13/20] fast-import.c: use a ref transaction when dumping tags

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Mon, 28 Apr 2014 15:23:58 -0700

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Reviewed-by: Michael Haggerty mhag...@alum.mit.edu
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 fast-import.c | 29 +++--
 1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/fast-import.c b/fast-import.c
index 79160d5..e7f6e37 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1735,15 +1735,32 @@ static void dump_tags(void)
 {
static const char *msg = fast-import;
struct tag *t;
-   struct ref_lock *lock;
-   char ref_name[PATH_MAX];
+   struct strbuf ref_name = STRBUF_INIT;
+   struct strbuf err = STRBUF_INIT;
+   struct ref_transaction *transaction;
 
+   transaction = ref_transaction_begin(err);
+   if (!transaction) {
+   failure |= error(%s, err.buf);
+   goto cleanup;
+   }
for (t = first_tag; t; t = t-next_tag) {
-   sprintf(ref_name, tags/%s, t-name);
-   lock = lock_ref_sha1(ref_name, NULL);
-   if (!lock || write_ref_sha1(lock, t-sha1, msg)  0)
-   failure |= error(Unable to update %s, ref_name);
+   strbuf_reset(ref_name);
+   strbuf_addf(ref_name, refs/tags/%s, t-name);
+
+   if (ref_transaction_update(transaction, ref_name.buf, t-sha1,
+  NULL, 0, 0, err)) {
+   failure |= error(%s, err.buf);
+   goto cleanup;
+   }
}
+   if (ref_transaction_commit(transaction, msg, err))
+   failure |= error(%s, err.buf);
+
+ cleanup:
+   ref_transaction_free(transaction);
+   strbuf_release(ref_name);
+   strbuf_release(err);
 }
 
 static void dump_marks_helper(FILE *f,
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 15/20] refs.c: make lock_ref_sha1 static

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Mon, 28 Apr 2014 15:38:47 -0700

No external callers reference lock_ref_sha1 any more so let's declare it
static.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Reviewed-by: Michael Haggerty mhag...@alum.mit.edu
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 refs.c | 7 +--
 refs.h | 6 --
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/refs.c b/refs.c
index dcc877b..0dc093c 100644
--- a/refs.c
+++ b/refs.c
@@ -2069,7 +2069,10 @@ int dwim_log(const char *str, int len, unsigned char 
*sha1, char **log)
return logs_found;
 }
 
-/* This function should make sure errno is meaningful on error */
+/*
+ * Locks a refs/ ref returning the lock on success and NULL on failure.
+ * On failure errno is set to something meaningful.
+ */
 static struct ref_lock *lock_ref_sha1_basic(const char *refname,
const unsigned char *old_sha1,
int flags, int *type_p)
@@ -2170,7 +2173,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char 
*refname,
return NULL;
 }
 
-struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char 
*old_sha1)
+static struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char 
*old_sha1)
 {
char refpath[PATH_MAX];
if (check_refname_format(refname, 0))
diff --git a/refs.h b/refs.h
index 3f37c65..65dd593 100644
--- a/refs.h
+++ b/refs.h
@@ -170,12 +170,6 @@ extern int ref_exists(const char *);
  */
 extern int peel_ref(const char *refname, unsigned char *sha1);
 
-/*
- * Locks a refs/ ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
- */
-extern struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char 
*old_sha1);
-
 /** Locks any ref (for 'HEAD' type refs). */
 #define REF_NODEREF0x01
 /* errno is set to something meaningful on failure */
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 14/20] walker.c: use ref transaction for ref updates

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Thu, 17 Apr 2014 11:31:06 -0700

Switch to using ref transactions in walker_fetch(). As part of the refactoring
to use ref transactions we also fix a potential memory leak where in the
original code if write_ref_sha1() would fail we would end up returning from
the function without free()ing the msg string.

Note that this function is only called when fetching from a remote HTTP
repository onto the local (most of the time single-user) repository which
likely means that the type of collisions that the previous locking would
protect against and cause the fetch to fail for are even more rare.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 walker.c | 72 
 1 file changed, 41 insertions(+), 31 deletions(-)

diff --git a/walker.c b/walker.c
index 1dd86b8..b8a5441 100644
--- a/walker.c
+++ b/walker.c
@@ -251,40 +251,40 @@ void walker_targets_free(int targets, char **target, 
const char **write_ref)
 int walker_fetch(struct walker *walker, int targets, char **target,
 const char **write_ref, const char *write_ref_log_details)
 {
-   struct ref_lock **lock = xcalloc(targets, sizeof(struct ref_lock *));
+   struct strbuf refname = STRBUF_INIT;
+   struct strbuf err = STRBUF_INIT;
+   struct ref_transaction *transaction = NULL;
unsigned char *sha1 = xmalloc(targets * 20);
-   char *msg;
-   int ret;
-   int i;
+   char *msg = NULL;
+   int i, ret = -1;
 
save_commit_buffer = 0;
 
-   for (i = 0; i  targets; i++) {
-   if (!write_ref || !write_ref[i])
-   continue;
-
-   lock[i] = lock_ref_sha1(write_ref[i], NULL);
-   if (!lock[i]) {
-   error(Can't lock ref %s, write_ref[i]);
-   goto unlock_and_fail;
+   if (write_ref) {
+   transaction = ref_transaction_begin(err);
+   if (!transaction) {
+   error(%s, err.buf);
+   goto done;
}
}
-
if (!walker-get_recover)
for_each_ref(mark_complete, NULL);
 
for (i = 0; i  targets; i++) {
if (interpret_target(walker, target[i], sha1[20 * i])) {
error(Could not interpret response from server '%s' as 
something to pull, target[i]);
-   goto unlock_and_fail;
+   goto done;
}
if (process(walker, lookup_unknown_object(sha1[20 * i])))
-   goto unlock_and_fail;
+   goto done;
}
 
if (loop(walker))
-   goto unlock_and_fail;
-
+   goto done;
+   if (!write_ref) {
+   ret = 0;
+   goto done;
+   }
if (write_ref_log_details) {
msg = xmalloc(strlen(write_ref_log_details) + 12);
sprintf(msg, fetch from %s, write_ref_log_details);
@@ -292,23 +292,33 @@ int walker_fetch(struct walker *walker, int targets, char 
**target,
msg = NULL;
}
for (i = 0; i  targets; i++) {
-   if (!write_ref || !write_ref[i])
+   if (!write_ref[i])
continue;
-   ret = write_ref_sha1(lock[i], sha1[20 * i], msg ? msg : fetch 
(unknown));
-   lock[i] = NULL;
-   if (ret)
-   goto unlock_and_fail;
+   strbuf_reset(refname);
+   strbuf_addf(refname, refs/%s, write_ref[i]);
+   if (ref_transaction_update(transaction, refname.buf,
+  sha1[20 * i], NULL, 0, 0,
+  err)) {
+   error(%s, err.buf);
+   goto done;
+   }
}
+   if (ref_transaction_commit(transaction,
+  msg ? msg : fetch (unknown),
+  err)) {
+   error(%s, err.buf);
+   goto done;
+   }
+
+   ret = 0;
+
+done:
+   ref_transaction_free(transaction);
free(msg);
-
-   return 0;
-
-unlock_and_fail:
-   for (i = 0; i  targets; i++)
-   if (lock[i])
-   unlock_ref(lock[i]);
-
-   return -1;
+   free(sha1);
+   strbuf_release(err);
+   strbuf_release(refname);
+   return ret;
 }
 
 void walker_free(struct walker *walker)
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 17/20] refs.c: remove the update_ref_write function

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Tue, 29 Apr 2014 13:42:07 -0700

Since we only call update_ref_write from a single place and we only call it
with onerr==QUIET_ON_ERR we can just as well get rid of it and just call
write_ref_sha1 directly. This changes the return status for _commit from
1 to -1 on failures when writing to the ref. Eventually we will want
_commit to start returning more detailed error conditions than the current
simple success/failure.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 refs.c | 34 --
 1 file changed, 8 insertions(+), 26 deletions(-)

diff --git a/refs.c b/refs.c
index de07791..ef7660a 100644
--- a/refs.c
+++ b/refs.c
@@ -3336,25 +3336,6 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
return retval;
 }
 
-static int update_ref_write(const char *action, const char *refname,
-   const unsigned char *sha1, struct ref_lock *lock,
-   struct strbuf *err, enum action_on_err onerr)
-{
-   if (write_ref_sha1(lock, sha1, action)  0) {
-   const char *str = Cannot update the ref '%s'.;
-   if (err)
-   strbuf_addf(err, str, refname);
-
-   switch (onerr) {
-   case UPDATE_REFS_MSG_ON_ERR: error(str, refname); break;
-   case UPDATE_REFS_DIE_ON_ERR: die(str, refname); break;
-   case UPDATE_REFS_QUIET_ON_ERR: break;
-   }
-   return 1;
-   }
-   return 0;
-}
-
 /**
  * Information needed for a single ref update.  Set new_sha1 to the
  * new value or to zero to delete the ref.  To check the old value
@@ -3604,14 +3585,15 @@ int ref_transaction_commit(struct ref_transaction 
*transaction,
struct ref_update *update = updates[i];
 
if (!is_null_sha1(update-new_sha1)) {
-   ret = update_ref_write(msg,
-  update-refname,
-  update-new_sha1,
-  update-lock, err,
-  UPDATE_REFS_QUIET_ON_ERR);
-   update-lock = NULL; /* freed by update_ref_write */
-   if (ret)
+   ret = write_ref_sha1(update-lock, update-new_sha1,
+msg);
+   update-lock = NULL; /* freed by write_ref_sha1 */
+   if (ret) {
+   if (err)
+   strbuf_addf(err, Cannot update the ref 
'%s'.,
+   update-refname);
goto cleanup;
+   }
}
}
 
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 16/20] refs.c: remove the update_ref_lock function

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Tue, 29 Apr 2014 12:14:47 -0700

Since we now only call update_ref_lock with onerr==QUIET_ON_ERR we no longer
need this function and can replace it with just calling lock_any_ref_for_update
directly.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Reviewed-by: Michael Haggerty mhag...@alum.mit.edu
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 refs.c | 30 ++
 1 file changed, 6 insertions(+), 24 deletions(-)

diff --git a/refs.c b/refs.c
index 0dc093c..de07791 100644
--- a/refs.c
+++ b/refs.c
@@ -3336,24 +3336,6 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
return retval;
 }
 
-static struct ref_lock *update_ref_lock(const char *refname,
-   const unsigned char *oldval,
-   int flags, int *type_p,
-   enum action_on_err onerr)
-{
-   struct ref_lock *lock;
-   lock = lock_any_ref_for_update(refname, oldval, flags, type_p);
-   if (!lock) {
-   const char *str = Cannot lock the ref '%s'.;
-   switch (onerr) {
-   case UPDATE_REFS_MSG_ON_ERR: error(str, refname); break;
-   case UPDATE_REFS_DIE_ON_ERR: die(str, refname); break;
-   case UPDATE_REFS_QUIET_ON_ERR: break;
-   }
-   }
-   return lock;
-}
-
 static int update_ref_write(const char *action, const char *refname,
const unsigned char *sha1, struct ref_lock *lock,
struct strbuf *err, enum action_on_err onerr)
@@ -3602,12 +3584,12 @@ int ref_transaction_commit(struct ref_transaction 
*transaction,
for (i = 0; i  n; i++) {
struct ref_update *update = updates[i];
 
-   update-lock = update_ref_lock(update-refname,
-  (update-have_old ?
-   update-old_sha1 : NULL),
-  update-flags,
-  update-type,
-  UPDATE_REFS_QUIET_ON_ERR);
+   update-lock = lock_any_ref_for_update(update-refname,
+  (update-have_old ?
+   update-old_sha1 :
+   NULL),
+  update-flags,
+  update-type);
if (!update-lock) {
if (err)
strbuf_addf(err, Cannot lock the ref '%s'.,
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 18/20] refs.c: remove lock_ref_sha1

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Tue, 29 Apr 2014 15:45:52 -0700

lock_ref_sha1 was only called from one place in refs.c and only provided
a check that the refname was sane before adding back the initial refs/
part of the ref path name, the initial refs/ that this caller had already
stripped off before calling lock_ref_sha1.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Reviewed-by: Michael Haggerty mhag...@alum.mit.edu
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 refs.c | 15 +--
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/refs.c b/refs.c
index ef7660a..f0883d0 100644
--- a/refs.c
+++ b/refs.c
@@ -2173,15 +2173,6 @@ static struct ref_lock *lock_ref_sha1_basic(const char 
*refname,
return NULL;
 }
 
-static struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char 
*old_sha1)
-{
-   char refpath[PATH_MAX];
-   if (check_refname_format(refname, 0))
-   return NULL;
-   strcpy(refpath, mkpath(refs/%s, refname));
-   return lock_ref_sha1_basic(refpath, old_sha1, 0, NULL);
-}
-
 struct ref_lock *lock_any_ref_for_update(const char *refname,
 const unsigned char *old_sha1,
 int flags, int *type_p)
@@ -2391,8 +2382,12 @@ static void try_remove_empty_parents(char *name)
 /* make sure nobody touched the ref, and unlink */
 static void prune_ref(struct ref_to_prune *r)
 {
-   struct ref_lock *lock = lock_ref_sha1(r-name + 5, r-sha1);
+   struct ref_lock *lock;
 
+   if (check_refname_format(r-name + 5, 0))
+   return;
+
+   lock = lock_ref_sha1_basic(r-name, r-sha1, 0, NULL);
if (lock) {
unlink_or_warn(git_path(%s, r-name));
unlock_ref(lock);
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 19/20] refs.c: make prune_ref use a transaction to delete the ref

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Wed, 30 Apr 2014 09:03:36 -0700

Change prune_ref to delete the ref using a ref transaction. To do this we also
need to add a new flag REF_ISPRUNING that will tell the transaction that we
do not want to delete this ref from the packed refs. This flag is private to
refs.c and not exposed to external callers.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
 refs.c | 28 +---
 refs.h | 13 +++--
 2 files changed, 32 insertions(+), 9 deletions(-)

diff --git a/refs.c b/refs.c
index f0883d0..5b2d335 100644
--- a/refs.c
+++ b/refs.c
@@ -25,6 +25,11 @@ static unsigned char refname_disposition[256] = {
 };
 
 /*
+ * Used as a flag to ref_transaction_delete when a loose ref is being
+ * pruned.
+ */
+#define REF_ISPRUNING  0x0100
+/*
  * Try to read one refname component from the front of refname.
  * Return the length of the component found, or -1 if the component is
  * not legal.  It is legal if it is something reasonable to have under
@@ -2382,17 +2387,25 @@ static void try_remove_empty_parents(char *name)
 /* make sure nobody touched the ref, and unlink */
 static void prune_ref(struct ref_to_prune *r)
 {
-   struct ref_lock *lock;
+   struct ref_transaction *transaction;
+   struct strbuf err = STRBUF_INIT;
 
if (check_refname_format(r-name + 5, 0))
return;
 
-   lock = lock_ref_sha1_basic(r-name, r-sha1, 0, NULL);
-   if (lock) {
-   unlink_or_warn(git_path(%s, r-name));
-   unlock_ref(lock);
-   try_remove_empty_parents(r-name);
+   transaction = ref_transaction_begin(err);
+   if (!transaction ||
+   ref_transaction_delete(transaction, r-name, r-sha1,
+  REF_ISPRUNING, 1, err) ||
+   ref_transaction_commit(transaction, NULL, err)) {
+   ref_transaction_free(transaction);
+   error(%s, err.buf);
+   strbuf_release(err);
+   return;
}
+   ref_transaction_free(transaction);
+   strbuf_release(err);
+   try_remove_empty_parents(r-name);
 }
 
 static void prune_refs(struct ref_to_prune *r)
@@ -3597,8 +3610,9 @@ int ref_transaction_commit(struct ref_transaction 
*transaction,
struct ref_update *update = updates[i];
 
if (update-lock) {
-   delnames[delnum++] = update-lock-ref_name;
ret |= delete_ref_loose(update-lock, update-type);
+   if (!(update-flags  REF_ISPRUNING))
+   delnames[delnum++] = update-lock-ref_name;
}
}
 
diff --git a/refs.h b/refs.h
index 65dd593..69ef28c 100644
--- a/refs.h
+++ b/refs.h
@@ -170,9 +170,18 @@ extern int ref_exists(const char *);
  */
 extern int peel_ref(const char *refname, unsigned char *sha1);
 
-/** Locks any ref (for 'HEAD' type refs). */
+/*
+ * Flags controlling lock_any_ref_for_update(), ref_transaction_update(),
+ * ref_transaction_create(), etc.
+ * REF_NODEREF: act on the ref directly, instead of dereferencing
+ *  symbolic references.
+ *
+ * Flags = 0x100 are reserved for internal use.
+ */
 #define REF_NODEREF0x01
-/* errno is set to something meaningful on failure */
+/*
+ * This function sets errno to something meaningful on failure.
+ */
 extern struct ref_lock *lock_any_ref_for_update(const char *refname,
const unsigned char *old_sha1,
int flags, int *type_p);
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 20/20] refs.c: make delete_ref use a transaction

2014-08-26 Thread Jonathan Nieder
From: Ronnie Sahlberg sahlb...@google.com
Date: Wed, 30 Apr 2014 09:22:45 -0700

Change delete_ref to use a ref transaction for the deletion. At the same time
since we no longer have any callers of repack_without_ref we can now delete
this function.

Change delete_ref to return 0 on success and 1 on failure instead of the
previous 0 on success either 1 or -1 on failure.

Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Reviewed-by: Michael Haggerty mhag...@alum.mit.edu
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
Thanks for reading.

 refs.c | 35 ++-
 1 file changed, 14 insertions(+), 21 deletions(-)

diff --git a/refs.c b/refs.c
index 5b2d335..7996be9 100644
--- a/refs.c
+++ b/refs.c
@@ -2548,11 +2548,6 @@ int repack_without_refs(const char **refnames, int n, 
struct strbuf *err)
return ret;
 }
 
-static int repack_without_ref(const char *refname)
-{
-   return repack_without_refs(refname, 1, NULL);
-}
-
 static int delete_ref_loose(struct ref_lock *lock, int flag)
 {
if (!(flag  REF_ISPACKED) || flag  REF_ISSYMREF) {
@@ -2570,24 +2565,22 @@ static int delete_ref_loose(struct ref_lock *lock, int 
flag)
 
 int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 {
-   struct ref_lock *lock;
-   int ret = 0, flag = 0;
+   struct ref_transaction *transaction;
+   struct strbuf err = STRBUF_INIT;
 
-   lock = lock_ref_sha1_basic(refname, sha1, delopt, flag);
-   if (!lock)
+   transaction = ref_transaction_begin(err);
+   if (!transaction ||
+   ref_transaction_delete(transaction, refname, sha1, delopt,
+  sha1  !is_null_sha1(sha1), err) ||
+   ref_transaction_commit(transaction, NULL, err)) {
+   error(%s, err.buf);
+   ref_transaction_free(transaction);
+   strbuf_release(err);
return 1;
-   ret |= delete_ref_loose(lock, flag);
-
-   /* removing the loose one could have resurrected an earlier
-* packed one.  Also, if it was not loose we need to repack
-* without it.
-*/
-   ret |= repack_without_ref(lock-ref_name);
-
-   unlink_or_warn(git_path(logs/%s, lock-ref_name));
-   clear_loose_ref_cache(ref_cache);
-   unlock_ref(lock);
-   return ret;
+   }
+   ref_transaction_free(transaction);
+   strbuf_release(err);
+   return 0;
 }
 
 /*
-- 
2.1.0.rc2.206.gedb03e5

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


mktree: multiple same-named objects

2014-08-26 Thread David Turner
git mktree seems to allow the creation of a tree object with multiple
objects of the same name but different SHAs.  This leads to weird
behavior later, unsurprisingly.  For instance, if there are two tree
objects with the same name but different SHAs, the checked out tree will
be the union of them (reasonably), but if you do git add $name, some or
all unmodified files under $name will show up in git status as modified
-- since they differ from one of the parent trees, presumably.

And if different git implementations treat this case differently, then
it might be possible to make a repo that appears to contain one thing
when viewed with one implementation, but contains a different thing for
a different implementation.

Summary: git mktree ought to forbid this, and possibly there ought to be
other checks (for instance, when unpacking) to prevent this.


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


  1   2   >