Re: [PATCH] gitmodules: clarify what history depth a shallow clone has

[Stefan Beller]

On Wed, Apr 19, 2017 at 12:56 AM, Sebastian Schuberth
<sschube...@gmail.com> wrote:
> Signed-off-by: Sebastian Schuberth <sschube...@gmail.com>

Thanks,
Stefan

> ---
>  Documentation/gitmodules.txt | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
> index 8f7c50f..6f39f24 100644
> --- a/Documentation/gitmodules.txt
> +++ b/Documentation/gitmodules.txt
> @@ -84,8 +84,8 @@ submodule..ignore::
>
>  submodule..shallow::
> When set to true, a clone of this submodule will be performed as a
> -   shallow clone unless the user explicitly asks for a non-shallow
> -   clone.
> +   shallow clone (with a history depth of 1) unless the user explicitly
> +   asks for a non-shallow clone.
>
>
>  EXAMPLES
>
> --
> https://github.com/git/git/pull/347

[PATCH] gitmodules: clarify what history depth a shallow clone has

[Sebastian Schuberth]

Signed-off-by: Sebastian Schuberth <sschube...@gmail.com>
---
 Documentation/gitmodules.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index 8f7c50f..6f39f24 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -84,8 +84,8 @@ submodule..ignore::
 
 submodule..shallow::
When set to true, a clone of this submodule will be performed as a
-   shallow clone unless the user explicitly asks for a non-shallow
-   clone.
+   shallow clone (with a history depth of 1) unless the user explicitly
+   asks for a non-shallow clone.
 
 
 EXAMPLES

--
https://github.com/git/git/pull/347

Re: [PATCH] clone,fetch: explain the shallow-clone option a little more clearly

[Duy Nguyen]

On Wed, Dec 7, 2016 at 1:29 AM, Junio C Hamano <gits...@pobox.com> wrote:
> Duy Nguyen <pclo...@gmail.com> writes:
>
>> On Tue, Dec 6, 2016 at 1:28 AM, Junio C Hamano <gits...@pobox.com> wrote:
>>> I however offhand do not think the feature can be used to make the
>>> repository shallower
>>
>> I'm pretty sure it can,...
>
> I wrote my message after a short local testing, but it is very
> possible I botched it and reached a wrong conclusion above.
>
> If we can use the command to make it shallower, then the phrase
> "deepen" would probably be what we need to be fixing in this patch:
>
>>   OPT_STRING_LIST(0, "shallow-exclude", _not, N_("revision"),
>> -         N_("deepen history of shallow clone by excluding 
>> rev")),
>> +     N_("deepen history of shallow clone, excluding rev")),
>
> Perhaps a shorter version of:
>
> Adjust the depth of shallow clone so that commits that are
> decendants of the named rev are made available, while commits
> that are ancestors of the named rev are made beyond reach.
>
> or something like that.  Here is my (somewhat botched) attempt:
>
> Adjust shallow clone's history to be cut at the rev

OK mine is "exclude the given tag from local history".

BTW the clone's string could be rephrased better because we know
there's no local history to begin with, if we stick to any verbs that
involves deepening or shortening.
-- 
Duy

Re: [PATCH] clone,fetch: explain the shallow-clone option a little more clearly

[Junio C Hamano]

Duy Nguyen <pclo...@gmail.com> writes:

> On Tue, Dec 6, 2016 at 1:28 AM, Junio C Hamano <gits...@pobox.com> wrote:
>> I however offhand do not think the feature can be used to make the
>> repository shallower
>
> I'm pretty sure it can,...

I wrote my message after a short local testing, but it is very
possible I botched it and reached a wrong conclusion above.

If we can use the command to make it shallower, then the phrase
"deepen" would probably be what we need to be fixing in this patch:

>   OPT_STRING_LIST(0, "shallow-exclude", _not, N_("revision"),
> - N_("deepen history of shallow clone by excluding rev")),
> + N_("deepen history of shallow clone, excluding rev")),

Perhaps a shorter version of:

Adjust the depth of shallow clone so that commits that are
decendants of the named rev are made available, while commits
that are ancestors of the named rev are made beyond reach.

or something like that.  Here is my (somewhat botched) attempt:

Adjust shallow clone's history to be cut at the rev

Re: [PATCH] clone,fetch: explain the shallow-clone option a little more clearly

[Duy Nguyen]

On Tue, Dec 6, 2016 at 1:28 AM, Junio C Hamano  wrote:
> I however offhand do not think the feature can be used to make the
> repository shallower

I'm pretty sure it can, though it's a waste because you should be able
to shorten your history without talking to a remote server. But that
no-remote shortening is not implemented yet. And you probably want an
option to check with remote anyway to make sure the part you cut out
is available there, no history will be lost.
-- 
Duy

Re: [PATCH] clone,fetch: explain the shallow-clone option a little more clearly

[Junio C Hamano]

Alex Henrie <alexhenri...@gmail.com> writes:

> "deepen by excluding" does not make sense because excluding a revision
> does not deepen a repository; it makes the repository more shallow.

I think that an intuitive way the feature should work may be:

 - You started with "git fetch --depth=20" and then later say "I
   have only a very short segment of the recent history, but I want
   a history that dates back to v1.0" with "--shallow-exclude=v1.0".
   In this case, you would be deepening.

 - You instead started with "git fetch --depth=2" that dated
   back to v0.5.  "--shallow-exclude=v1.0" you say today would mean
   "I have very old cruft I no longer look at.  I just want my
   history lead back to v1.0 and no earlier".  In such a case, you
   indeed would be making the repository shallower.

I however offhand do not think the feature can be used to make the
repository shallower, and I agree your changes to the usage string
probably describe what the option does more correctly.

I however suspect that the feature is simply buggy and it would
eventually want to allow to shorten the history as well.  At that
point we may want to work on the verb 'deepen' there, too.

>
> Signed-off-by: Alex Henrie <alexhenri...@gmail.com>
> ---
>  builtin/clone.c | 2 +-
>  builtin/fetch.c | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/builtin/clone.c b/builtin/clone.c
> index 6c76a6e..e3cb808 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -99,7 +99,7 @@ static struct option builtin_clone_options[] = {
>   OPT_STRING(0, "shallow-since", _since, N_("time"),
>   N_("create a shallow clone since a specific time")),
>   OPT_STRING_LIST(0, "shallow-exclude", _not, N_("revision"),
> - N_("deepen history of shallow clone by excluding rev")),
> + N_("deepen history of shallow clone, excluding rev")),
>   OPT_BOOL(0, "single-branch", _single_branch,
>   N_("clone only one branch, HEAD or --branch")),
>   OPT_BOOL(0, "shallow-submodules", _shallow_submodules,
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index b6a5597..fc74c84 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -122,7 +122,7 @@ static struct option builtin_fetch_options[] = {
>   OPT_STRING(0, "shallow-since", _since, N_("time"),
>  N_("deepen history of shallow repository based on time")),
>   OPT_STRING_LIST(0, "shallow-exclude", _not, N_("revision"),
> - N_("deepen history of shallow clone by excluding rev")),
> + N_("deepen history of shallow clone, excluding rev")),
>   OPT_INTEGER(0, "deepen", _relative,
>   N_("deepen history of shallow clone")),
>   { OPTION_SET_INT, 0, "unshallow", , NULL,

[PATCH] clone,fetch: explain the shallow-clone option a little more clearly

[Alex Henrie]

"deepen by excluding" does not make sense because excluding a revision
does not deepen a repository; it makes the repository more shallow.

Signed-off-by: Alex Henrie <alexhenri...@gmail.com>
---
 builtin/clone.c | 2 +-
 builtin/fetch.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/clone.c b/builtin/clone.c
index 6c76a6e..e3cb808 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -99,7 +99,7 @@ static struct option builtin_clone_options[] = {
OPT_STRING(0, "shallow-since", _since, N_("time"),
    N_("create a shallow clone since a specific time")),
OPT_STRING_LIST(0, "shallow-exclude", _not, N_("revision"),
-   N_("deepen history of shallow clone by excluding rev")),
+   N_("deepen history of shallow clone, excluding rev")),
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_BOOL(0, "shallow-submodules", _shallow_submodules,
diff --git a/builtin/fetch.c b/builtin/fetch.c
index b6a5597..fc74c84 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -122,7 +122,7 @@ static struct option builtin_fetch_options[] = {
OPT_STRING(0, "shallow-since", _since, N_("time"),
   N_("deepen history of shallow repository based on time")),
OPT_STRING_LIST(0, "shallow-exclude", _not, N_("revision"),
-   N_("deepen history of shallow clone by excluding rev")),
+   N_("deepen history of shallow clone, excluding rev")),
OPT_INTEGER(0, "deepen", _relative,
N_("deepen history of shallow clone")),
{ OPTION_SET_INT, 0, "unshallow", , NULL,
-- 
2.10.2

Re: Git shallow clone branch doesn't work with recursive submodules cloning

[Jeff King]

On Mon, Aug 15, 2016 at 03:53:48PM +0300, Arkady Shapkin wrote:

> So it will work only if github update their server configuration
> (boringssl submodule on github)?

Correct.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Git shallow clone branch doesn't work with recursive submodules cloning

[Arkady Shapkin]

So it will work only if github update their server configuration
(boringssl submodule on github)?

2016-08-15 15:47 GMT+03:00 Jeff King :
> On Mon, Aug 15, 2016 at 03:29:14PM +0300, Arkady Shapkin wrote:
>
>> Thank you, after updating to "2.9.3.windows.1" options "--recursive
>> --depth 1" now works.
>>
>> But "--recursive --shallow-submodules" and "--recursive
>> --shallow-submodules --depth 1" still doesn't work.
>
> It does "work", but the server hosting your submodule may not be
> configured to allow you to access the reachable-but-non-tip sha1
> directly. So it's not a bug, but rather a configuration issue (and the
> "fix" in v2.9.1 is to be less aggressive about enabling
> shallow-submodules, since the default server configuration does not
> allow it to work well).
>
> More discussion is in:
>
>   
> http://public-inbox.org/git/ofe09d48f2.d1d14f49-onc2257fd7.00280736-c2257fd7.00282...@notes.na.collabserv.com/t/#u
>
> -Peff



-- 
WBR,
Arkady Shapkin aka Dragon
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Git shallow clone branch doesn't work with recursive submodules cloning

[Jeff King]

On Mon, Aug 15, 2016 at 03:29:14PM +0300, Arkady Shapkin wrote:

> Thank you, after updating to "2.9.3.windows.1" options "--recursive
> --depth 1" now works.
> 
> But "--recursive --shallow-submodules" and "--recursive
> --shallow-submodules --depth 1" still doesn't work.

It does "work", but the server hosting your submodule may not be
configured to allow you to access the reachable-but-non-tip sha1
directly. So it's not a bug, but rather a configuration issue (and the
"fix" in v2.9.1 is to be less aggressive about enabling
shallow-submodules, since the default server configuration does not
allow it to work well).

More discussion is in:

  
http://public-inbox.org/git/ofe09d48f2.d1d14f49-onc2257fd7.00280736-c2257fd7.00282...@notes.na.collabserv.com/t/#u

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Git shallow clone branch doesn't work with recursive submodules cloning

[Arkady Shapkin]

Thank you, after updating to "2.9.3.windows.1" options "--recursive
--depth 1" now works.

But "--recursive --shallow-submodules" and "--recursive
--shallow-submodules --depth 1" still doesn't work.

2016-08-15 15:04 GMT+03:00 Jeff King :
> On Mon, Aug 15, 2016 at 02:20:27PM +0300, Arkady Shapkin wrote:
>
>> I am trying clone repository by tag with recursive submodules init,
>> but for one submodule it doesn't work.
>> What I'm doing wrong?
>
> Nothing. See 18a74a0 (clone: do not let --depth imply
> --shallow-submodules, 2016-06-19).
>
>> >git --version
>> git version 2.9.0.windows.1
>
> The fix is in v2.9.1.
>
> -Peff



-- 
WBR,
Arkady Shapkin aka Dragon
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Git shallow clone branch doesn't work with recursive submodules cloning

[Jeff King]

On Mon, Aug 15, 2016 at 02:20:27PM +0300, Arkady Shapkin wrote:

> I am trying clone repository by tag with recursive submodules init,
> but for one submodule it doesn't work.
> What I'm doing wrong?

Nothing. See 18a74a0 (clone: do not let --depth imply
--shallow-submodules, 2016-06-19).

> >git --version
> git version 2.9.0.windows.1

The fix is in v2.9.1.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Git shallow clone branch doesn't work with recursive submodules cloning

[Arkady Shapkin]

Hi,

I am trying clone repository by tag with recursive submodules init,
but for one submodule it doesn't work.
What I'm doing wrong?

>git clone https://github.com/grpc/grpc.git --recursive --depth 1 dsad5

Cloning into 'dsad5'...
remote: Counting objects: 7475, done.
remote: Compressing objects: 100% (4695/4695), done.
remote: Total 7475 (delta 2593), reused 5610 (delta 1265), pack-reused 0
Receiving objects: 100% (7475/7475), 4.85 MiB | 2.55 MiB/s, done.
Resolving deltas: 100% (2593/2593), done.
Checking connectivity... done.
Checking out files: 100% (6820/6820), done.
Submodule 'third_party/boringssl'
(https://github.com/google/boringssl.git) registered for path
'third_party/boringssl'
Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git)
registered for path 'third_party/gflags'
Submodule 'third_party/googletest'
(https://github.com/google/googletest.git) registered for path
'third_party/googletest'
Submodule 'third_party/nanopb' (https://github.com/nanopb/nanopb.git)
registered for path 'third_party/nanopb'
Submodule 'third_party/protobuf'
(https://github.com/google/protobuf.git) registered for path
'third_party/protobuf'
Submodule 'third_party/zlib' (https://github.com/madler/zlib)
registered for path 'third_party/zlib'
Cloning into 'D:/Work/conan-packages/dsad5/third_party/boringssl'...
Cloning into 'D:/Work/conan-packages/dsad5/third_party/gflags'...
Cloning into 'D:/Work/conan-packages/dsad5/third_party/googletest'...
Cloning into 'D:/Work/conan-packages/dsad5/third_party/nanopb'...
Cloning into 'D:/Work/conan-packages/dsad5/third_party/protobuf'...
Cloning into 'D:/Work/conan-packages/dsad5/third_party/zlib'...
error: no such remote ref c880e42ba1c8032d4cdde2aba0541d8a9d9fa2e9
Fetched in submodule path 'third_party/boringssl', but it did not
contain c880e42ba1c8032d4cdde2aba0541d8a9d9fa2e9. Direct fetching of
that commit failed.

>git --version
git version 2.9.0.windows.1

-- 
WBR,
Arkady Shapkin
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: http-backend fatal error message on shallow clone

[Duy Nguyen]

On Tue, Jun 21, 2016 at 2:10 PM, Jeff King <p...@peff.net> wrote:
> ...
> So this request actually takes _two_ upload-pack instances to serve
> (which is not uncommon when we need multiple rounds of
> get_common_commits(), though I am a little surprised that would be the
> case for a clone). And the first one seems to be missing a closing
> "" flush packet from the client to end it.
>
> If that analysis is correct, this isn't affecting operation in any way;
> it's just giving a useless message from upload-pack (and as you note,
> that could trigger http-backend to exit(1), which may make the webserver
> unhappy).
>
> If we further instrument upload-pack to set GIT_TRACE_PACKET on the
> server side, we can see the two requests:
>
> packet:  upload-pack< want 379518c0c94e3b1a0710129d03d5560814a0ba6f 
> multi_ack_detailed no-done side-band-64k thin-pack include-tag ofs-delta 
> agent=git/2.9.0.37.gb3ad8ab.dirty
> packet:  upload-pack< deepen 1
> packet:  upload-pack< 
> packet:  upload-pack> shallow 379518c0c94e3b1a0710129d03d5560814a0ba6f
> packet:  upload-pack> 
>
> packet:  upload-pack< want 379518c0c94e3b1a0710129d03d5560814a0ba6f 
> multi_ack_detailed no-done side-band-64k thin-pack include-tag ofs-delta 
> agent=git/2.9.0.37.gb3ad8ab.dirty
> packet:  upload-pack< deepen 1
> packet:  upload-pack< 
> packet:  upload-pack> shallow 379518c0c94e3b1a0710129d03d5560814a0ba6f
> packet:  upload-pack> 
> packet:  upload-pack< done
> packet:  upload-pack> NAK
> packet:  upload-pack> 
>
> I think in the first one we would get "deepen 1" from the client in
> receive_needs(), and similarly write out our "shallow" line. But then we
> go into get_common_commits() and the client has hung up, which causes
> the message. Whereas in the second line it gives us a "done", which
> completes the negotiation.
>
> So my not-very-educated thoughts are:
>
>   1. The client should probably be sending an extra flush in the first
>  request. Alternatively, in the stateless-rpc case we should just
>  accept the lack of flush as an acceptable end to the request.

Our pkt-line.c can't deal with eof if I remember correctly (I tried to
use pkt-line for the parallel checkout stuff, where workers can also
exit early...) so sending extra flush may be easier. Old upload-pack
will not have a problem with this extra flush right? I haven't checked
upload-pack.c yet...

>   2. Presumably the shallowness is what causes the double-request, as
>  fetch-pack wants to see the shallow list before proceeding. But
>  since it has no actual commits to negotiate, the negotiation is a
>  noop. So probably this case could actually happen in a single
>  request.

I seem to recall our discussion a few months(?) ago about quickfetch()
where shallow clone would force another fetch initiated from
backfill_tags(). I guess that's why you see two fetches.

>
>  I suspect that other fetches could not, though, so I'm not sure how
>  much effort is worth putting into optimizing.
>
> -Peff
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: http-backend fatal error message on shallow clone

[Jeff King]

On Tue, Jun 21, 2016 at 11:23:03AM +, Eric Wong wrote:

> I noticed "fatal: The remote end hung up unexpectedly" in server
> logs from shallow clones.  Totally reproducible in the test
> cases, too.  The following change shows it:
> [...]
> [Tue Jun 21 11:07:41.391269 2016] [cgi:error] [pid 21589] [client 
> 127.0.0.1:37518] AH01215: fatal: The remote end hung up unexpectedly
> 
> It doesn't show above, but I think http-backend exits
> with a non-zero status, too, which might cause some CGI
> implementations to complain or break.
> 
> Not sure if it's just a corner case that wasn't tested
> or something else, but the clone itself seems successful...

The dying process is actually upload-pack. If we instrument it like
this:

diff --git a/upload-pack.c b/upload-pack.c
index 9e03c27..a1da676 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -820,6 +820,14 @@ static int upload_pack_config(const char *var, const char 
*value, void *unused)
return parse_hide_refs_config(var, value, "uploadpack");
 }
 
+NORETURN
+static void custom_die(const char *err, va_list params)
+{
+   vreportf("fatal: ", err, params);
+   warning("aborting");
+   abort();
+}
+
 int main(int argc, const char **argv)
 {
const char *dir;
@@ -836,6 +844,9 @@ int main(int argc, const char **argv)
OPT_END()
};
 
+   warning("running upload-pack");
+   set_die_routine(custom_die);
+
git_setup_gettext();
 
packet_trace_identity("upload-pack");

we can see two things. One is we can get a backtrace from the core file:

#0  0x7f04aef51458 in __GI_raise (sig=sig@entry=6) at 
../sysdeps/unix/sysv/linux/raise.c:55
#1  0x7f04aef528da in __GI_abort () at abort.c:89
#2  0x00406009 in custom_die (err=0x4ede60 "The remote end hung up 
unexpectedly", 
params=0x7ffe15858758) at upload-pack.c:828
#3  0x0045ec63 in die (err=0x4ede60 "The remote end hung up 
unexpectedly") at usage.c:108
#4  0x0041e016 in get_packet_data (fd=0, src_buf=0x0, src_size=0x0, 
dst=0x7ffe158588b0, 
size=4, options=2) at pkt-line.c:167
#5  0x0041e0ea in packet_read (fd=0, src_buf=0x0, src_len=0x0, 
buffer=0x73cc40  "deepen 1", size=65520, options=2) at 
pkt-line.c:204
#6  0x0041e22b in packet_read_line_generic (fd=0, src=0x0, src_len=0x0, 
dst_len=0x0)
at pkt-line.c:234
#7  0x0041e27c in packet_read_line (fd=0, len_p=0x0) at pkt-line.c:244
#8  0x00404dc9 in get_common_commits () at upload-pack.c:384
#9  0x00405eb4 in upload_pack () at upload-pack.c:798
#10 0x00406229 in main (argc=1, argv=0x7ffe15858c28) at 
upload-pack.c:872

So we expected to read a packet but didn't get one. Not surprising. The
interesting thing is that we are in get_common_commits(), and if we were
to get a flush packet in stateless-rpc mode, we would simply exit(0).
But we get EOF instead, which provokes packet_read_line() to die.

The other interesting thing is that we can see in httpd's error.log that
it is the penultimate upload-pack that dies (I trimmed the log lines for
readability):

AH01215: warning: running upload-pack
AH01215: fatal: The remote end hung up unexpectedly
AH01215: warning: aborting
AH01215: error: git-upload-pack died of signal 6
AH01215: warning: running upload-pack

So this request actually takes _two_ upload-pack instances to serve
(which is not uncommon when we need multiple rounds of
get_common_commits(), though I am a little surprised that would be the
case for a clone). And the first one seems to be missing a closing
"" flush packet from the client to end it.

If that analysis is correct, this isn't affecting operation in any way;
it's just giving a useless message from upload-pack (and as you note,
that could trigger http-backend to exit(1), which may make the webserver
unhappy).

If we further instrument upload-pack to set GIT_TRACE_PACKET on the
server side, we can see the two requests:

packet:  upload-pack< want 379518c0c94e3b1a0710129d03d5560814a0ba6f 
multi_ack_detailed no-done side-band-64k thin-pack include-tag ofs-delta 
agent=git/2.9.0.37.gb3ad8ab.dirty
packet:  upload-pack< deepen 1
packet:  upload-pack< 
packet:  upload-pack> shallow 379518c0c94e3b1a0710129d03d5560814a0ba6f
packet:  upload-pack> 

packet:  upload-pack< want 379518c0c94e3b1a0710129d03d5560814a0ba6f 
multi_ack_detailed no-done side-band-64k thin-pack include-tag ofs-delta 
agent=git/2.9.0.37.gb3ad8ab.dirty
packet:  upload-pack< deepen 1
packet:  upload-pack< 
packet:  upload-pack> shallow 379518c0c94e3b1a0710129d03d5560814a0ba6f
packet:  upload-pack> 
packet:  upload-pack< done
packet:  upload-pack> NAK
packet:  upload-pack> 

I think in the first one we would get "deepen 1" from the client in
receive_needs(), and similarly write out our "shallow" line. But then we
go into get_common_commits() and the client has hung up, which causes
the message. Whereas in the second line it gives us a "done", which
completes 

http-backend fatal error message on shallow clone

[Eric Wong]

I noticed "fatal: The remote end hung up unexpectedly" in server
logs from shallow clones.  Totally reproducible in the test
cases, too.  The following change shows it:

diff --git a/t/t5561-http-backend.sh b/t/t5561-http-backend.sh
index 90e0d6f..cfa55ce 100755
--- a/t/t5561-http-backend.sh
+++ b/t/t5561-http-backend.sh
@@ -132,5 +132,11 @@ test_expect_success 'server request log matches test 
results' '
test_cmp exp act
 '
 
+test_expect_success 'shallow clone' '
+   config http.uploadpack true &&
+   git clone --depth=1 "$HTTPD_URL/smart/repo.git" shallow &&
+   tail "$HTTPD_ROOT_PATH"/error.log | grep fatal
+'
+
 stop_httpd
 test_done


And the last test ends like this:

expecting success: 
config http.uploadpack true &&
git clone --depth=1 "$HTTPD_URL/smart/repo.git" shallow &&
tail "$HTTPD_ROOT_PATH"/error.log | grep fatal

Cloning into 'shallow'...
[Tue Jun 21 11:07:41.391269 2016] [cgi:error] [pid 21589] [client 
127.0.0.1:37518] AH01215: fatal: The remote end hung up unexpectedly
ok 15 - shallow clone

It doesn't show above, but I think http-backend exits
with a non-zero status, too, which might cause some CGI
implementations to complain or break.

Not sure if it's just a corner case that wasn't tested
or something else, but the clone itself seems successful...
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] shallow clone to not imply shallow submodules

[Jeff King]

On Mon, Jun 20, 2016 at 09:59:58AM -0700, Stefan Beller wrote:

> Signed-off-by: Stefan Beller <sbel...@google.com>
> ---
> 
> Hi Junio, Peff,
> 
> I thought about this patch squashed into  
> "clone: do not let --depth imply --shallow-submodules" will actually test
> for the regression.

Yep, it looks good to me.

> +test_expect_success 'shallow clone does not imply shallow submodule' '
> + test_when_finished "rm -rf super_clone" &&
> + git clone --recurse-submodules --depth 2 "file://$pwd/." super_clone &&
> + (
> + cd super_clone &&
> + git log --oneline >lines &&
> + test_line_count = 2 lines
> + ) &&
> + (
> + cd super_clone/sub &&
> + git log --oneline >lines &&
> + test_line_count = 3 lines
> + )
> +'

This follows the style of the other tests, so it's the right thing here.
But as a style suggestion, I think:

  git -C super_clone/sub log --oneline >lines &&
  test_line_count = 3 lines

is nicer than the subshell. It's more succinct, and it saves a process.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] shallow clone to not imply shallow submodules

[Jeff King]

On Mon, Jun 20, 2016 at 10:14:47AM -0700, Stefan Beller wrote:

> > This follows the style of the other tests, so it's the right thing here.
> > But as a style suggestion, I think:
> >
> >   git -C super_clone/sub log --oneline >lines &&
> >   test_line_count = 3 lines
> >
> > is nicer than the subshell. It's more succinct, and it saves a process.
> 
> which we would want to refactor to in a follow up, but not merge it
> through to 2.9.1.

Yeah, exactly. That was what I meant by "here".

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] shallow clone to not imply shallow submodules

[Stefan Beller]

On Mon, Jun 20, 2016 at 10:13 AM, Jeff King <p...@peff.net> wrote:
> On Mon, Jun 20, 2016 at 09:59:58AM -0700, Stefan Beller wrote:
>
>> Signed-off-by: Stefan Beller <sbel...@google.com>
>> ---
>>
>> Hi Junio, Peff,
>>
>> I thought about this patch squashed into
>> "clone: do not let --depth imply --shallow-submodules" will actually test
>> for the regression.
>
> Yep, it looks good to me.
>
>> +test_expect_success 'shallow clone does not imply shallow submodule' '
>> + test_when_finished "rm -rf super_clone" &&
>> + git clone --recurse-submodules --depth 2 "file://$pwd/." super_clone &&
>> + (
>> + cd super_clone &&
>> + git log --oneline >lines &&
>> + test_line_count = 2 lines
>> + ) &&
>> + (
>> + cd super_clone/sub &&
>> + git log --oneline >lines &&
>> + test_line_count = 3 lines
>> + )
>> +'
>
> This follows the style of the other tests, so it's the right thing here.
> But as a style suggestion, I think:
>
>   git -C super_clone/sub log --oneline >lines &&
>   test_line_count = 3 lines
>
> is nicer than the subshell. It's more succinct, and it saves a process.

which we would want to refactor to in a follow up, but not merge it
through to 2.9.1.

Thanks,
Stefan

>
> -Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] shallow clone to not imply shallow submodules

[Stefan Beller]

Signed-off-by: Stefan Beller <sbel...@google.com>
---

Hi Junio, Peff,

I thought about this patch squashed into  
"clone: do not let --depth imply --shallow-submodules" will actually test
for the regression.

Thanks,
Stefan

 t/t5614-clone-submodules.sh | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/t/t5614-clone-submodules.sh b/t/t5614-clone-submodules.sh
index f7c630b..a9aaa01 100755
--- a/t/t5614-clone-submodules.sh
+++ b/t/t5614-clone-submodules.sh
@@ -37,7 +37,7 @@ test_expect_success 'nonshallow clone implies nonshallow 
submodule' '
)
 '
 
-test_expect_success 'shallow clone does not imply shallow submodule' '
+test_expect_success 'shallow clone with shallow submodule' '
test_when_finished "rm -rf super_clone" &&
git clone --recurse-submodules --depth 2 --shallow-submodules 
"file://$pwd/." super_clone &&
    (
@@ -52,6 +52,21 @@ test_expect_success 'shallow clone does not imply shallow 
submodule' '
)
 '
 
+test_expect_success 'shallow clone does not imply shallow submodule' '
+   test_when_finished "rm -rf super_clone" &&
+   git clone --recurse-submodules --depth 2 "file://$pwd/." super_clone &&
+   (
+   cd super_clone &&
+   git log --oneline >lines &&
+   test_line_count = 2 lines
+   ) &&
+   (
+   cd super_clone/sub &&
+       git log --oneline >lines &&
+   test_line_count = 3 lines
+   )
+'
+
 test_expect_success 'shallow clone with non shallow submodule' '
test_when_finished "rm -rf super_clone" &&
git clone --recurse-submodules --depth 2 --no-shallow-submodules 
"file://$pwd/." super_clone &&
-- 
2.7.0.rc0.40.g5328432.dirty

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 23/27] clone: define shallow clone boundary with --shallow-exclude

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
Signed-off-by: Junio C Hamano <gits...@pobox.com>
---
 Documentation/git-clone.txt |  5 +
 builtin/clone.c | 10 +-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index a410409..5049663 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -196,6 +196,11 @@ objects from the source repository into a pack in the 
cloned repository.
 --shallow-since=::
    Create a shallow clone with a history after the specified time.
 
+--shallow-exclude=::
+   Create a shallow clone with a history, excluding commits
+   reachable from a specified remote branch or tag.  This option
+   can be specified multiple times.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..3849231 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
 static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
 static const char *real_git_dir;
 static char *option_upload_pack = "git-upload-pack";
 static int option_verbosity;
@@ -89,6 +90,8 @@ static struct option builtin_clone_options[] = {
        N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", _since, N_("time"),
N_("create a shallow clone since a specific time")),
+   OPT_STRING_LIST(0, "shallow-exclude", _not, N_("revision"),
+   N_("deepen history of shallow clone by excluding rev")),
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -852,7 +855,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
-   if (option_depth || option_since)
+   if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +986,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones; 
use file:// instead."));
+   if (option_not.nr)
+   warning(_("--shallow-exclude is ignored in local 
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -1004,6 +1009,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
 option_since);
+   if (option_not.nr)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
-- 
2.8.2.524.g6ff3d78

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 18/27] clone: define shallow clone boundary based on time with --shallow-since

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
Signed-off-by: Junio C Hamano <gits...@pobox.com>
---
 Documentation/git-clone.txt |  3 +++
 builtin/clone.c | 16 +---
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index b7c467a..a410409 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -193,6 +193,9 @@ objects from the source repository into a pack in the 
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
 
+--shallow-since=::
+   Create a shallow clone with a history after the specified time.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
 
 static int option_no_checkout, option_bare, option_mirror, 
option_single_branch = -1;
 static int option_local = -1, option_no_hardlinks, option_shared, 
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
 static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
   N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", _depth, N_("depth"),
N_("create a shallow clone of that depth")),
+   OPT_STRING(0, "shallow-since", _since, N_("time"),
+   N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
+   if (option_depth || option_since)
+   deepen = 1;
if (option_single_branch == -1)
-   option_single_branch = option_depth ? 1 : 0;
+   option_single_branch = deepen ? 1 : 0;
 
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
+   if (option_since)
+   warning(_("--shallow-since is ignored in local clones; 
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
 option_depth);
+   if (option_since)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
 option_upload_pack);
 
-   if (transport->smart_options && !option_depth)
+   if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected = 
1;
 
refs = transport_get_remote_refs(transport);
-- 
2.8.2.524.g6ff3d78

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 23/27] clone: define shallow clone boundary with --shallow-exclude

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
---
 Documentation/git-clone.txt |  5 +
 builtin/clone.c | 10 +-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index a410409..5049663 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -196,6 +196,11 @@ objects from the source repository into a pack in the 
cloned repository.
 --shallow-since=::
Create a shallow clone with a history after the specified time.
 
+--shallow-exclude=::
+   Create a shallow clone with a history, excluding commits
+   reachable from a specified remote branch or tag.  This option
+   can be specified multiple times.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..3849231 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
 static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
 static const char *real_git_dir;
 static char *option_upload_pack = "git-upload-pack";
 static int option_verbosity;
@@ -89,6 +90,8 @@ static struct option builtin_clone_options[] = {
    N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", _since, N_("time"),
N_("create a shallow clone since a specific time")),
+   OPT_STRING_LIST(0, "shallow-exclude", _not, N_("revision"),
+   N_("deepen history of shallow clone by excluding rev")),
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -852,7 +855,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
-   if (option_depth || option_since)
+   if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +986,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones; 
use file:// instead."));
+   if (option_not.nr)
+   warning(_("--shallow-exclude is ignored in local 
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -1004,6 +1009,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
 option_since);
+   if (option_not.nr)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
-- 
2.8.2.524.g6ff3d78

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 18/27] clone: define shallow clone boundary based on time with --shallow-since

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
Signed-off-by: Junio C Hamano <gits...@pobox.com>
---
 Documentation/git-clone.txt |  3 +++
 builtin/clone.c | 16 +---
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index b7c467a..a410409 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -193,6 +193,9 @@ objects from the source repository into a pack in the 
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
 
+--shallow-since=::
+   Create a shallow clone with a history after the specified time.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
 
 static int option_no_checkout, option_bare, option_mirror, 
option_single_branch = -1;
 static int option_local = -1, option_no_hardlinks, option_shared, 
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
 static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
   N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", _depth, N_("depth"),
N_("create a shallow clone of that depth")),
+   OPT_STRING(0, "shallow-since", _since, N_("time"),
+   N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
+   if (option_depth || option_since)
+   deepen = 1;
if (option_single_branch == -1)
-   option_single_branch = option_depth ? 1 : 0;
+   option_single_branch = deepen ? 1 : 0;
 
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
+   if (option_since)
+   warning(_("--shallow-since is ignored in local clones; 
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
 option_depth);
+   if (option_since)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
 option_upload_pack);
 
-   if (transport->smart_options && !option_depth)
+   if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected = 
1;
 
refs = transport_get_remote_refs(transport);
-- 
2.8.2.524.g6ff3d78

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 22/26] clone: define shallow clone boundary with --shallow-exclude

[Eric Sunshine]

On Wed, Apr 13, 2016 at 8:55 AM, Nguyễn Thái Ngọc Duy <pclo...@gmail.com> wrote:
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
> ---
> diff --git a/builtin/clone.c b/builtin/clone.c
> @@ -44,6 +44,7 @@ static int deepen;
> +static struct string_list option_not = STRING_LIST_INIT_NODUP;
> @@ -52,6 +53,13 @@ static struct string_list option_config;
> +static int option_parse_deepen_not(const struct option *opt,
> +  const char *arg, int unset)
> +{
> +   string_list_append(_not, arg);
> +   return 0;
> +}
> +
> @@ -89,6 +97,9 @@ static struct option builtin_clone_options[] = {
> N_("create a shallow clone of that depth")),
> OPT_STRING(0, "shallow-since", _since, N_("time"),
> N_("create a shallow clone since a specific time")),
> +   { OPTION_CALLBACK, 0, "shallow-exclude", NULL, N_("revision"),
> +   N_("deepen history of shallow clone by excluding rev"),
> +   PARSE_OPT_NONEG, option_parse_deepen_not },

OPT_STRING_LIST()?

> OPT_BOOL(0, "single-branch", _single_branch,
> N_("clone only one branch, HEAD or --branch")),
> OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 17/26] clone: define shallow clone boundary based on time with --shallow-since

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
---
 Documentation/git-clone.txt |  3 +++
 builtin/clone.c | 16 +---
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index b7c467a..a410409 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -193,6 +193,9 @@ objects from the source repository into a pack in the 
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
 
+--shallow-since=::
+   Create a shallow clone with a history after the specified time.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
 
 static int option_no_checkout, option_bare, option_mirror, 
option_single_branch = -1;
 static int option_local = -1, option_no_hardlinks, option_shared, 
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
 static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
   N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", _depth, N_("depth"),
    N_("create a shallow clone of that depth")),
+   OPT_STRING(0, "shallow-since", _since, N_("time"),
+   N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
+   if (option_depth || option_since)
+   deepen = 1;
if (option_single_branch == -1)
-   option_single_branch = option_depth ? 1 : 0;
+   option_single_branch = deepen ? 1 : 0;
 
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
+   if (option_since)
+   warning(_("--shallow-since is ignored in local clones; 
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
 option_depth);
+   if (option_since)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
 option_upload_pack);
 
-   if (transport->smart_options && !option_depth)
+   if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected = 
1;
 
refs = transport_get_remote_refs(transport);
-- 
2.8.0.rc0.210.gd302cd2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 22/26] clone: define shallow clone boundary with --shallow-exclude

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
---
 Documentation/git-clone.txt |  5 +
 builtin/clone.c | 18 +-
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index a410409..5049663 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -196,6 +196,11 @@ objects from the source repository into a pack in the 
cloned repository.
 --shallow-since=::
Create a shallow clone with a history after the specified time.
 
+--shallow-exclude=::
+   Create a shallow clone with a history, excluding commits
+   reachable from a specified remote branch or tag.  This option
+   can be specified multiple times.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..5ccf6b7 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
 static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
 static const char *real_git_dir;
 static char *option_upload_pack = "git-upload-pack";
 static int option_verbosity;
@@ -52,6 +53,13 @@ static struct string_list option_config;
 static struct string_list option_reference;
 static int option_dissociate;
 
+static int option_parse_deepen_not(const struct option *opt,
+  const char *arg, int unset)
+{
+   string_list_append(_not, arg);
+   return 0;
+}
+
 static struct option builtin_clone_options[] = {
OPT__VERBOSITY(_verbosity),
OPT_BOOL(0, "progress", _progress,
@@ -89,6 +97,9 @@ static struct option builtin_clone_options[] = {
        N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", _since, N_("time"),
N_("create a shallow clone since a specific time")),
+   { OPTION_CALLBACK, 0, "shallow-exclude", NULL, N_("revision"),
+   N_("deepen history of shallow clone by excluding rev"),
+   PARSE_OPT_NONEG, option_parse_deepen_not },
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -852,7 +863,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
-   if (option_depth || option_since)
+   if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +994,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones; 
use file:// instead."));
+   if (option_not.nr)
+   warning(_("--shallow-exclude is ignored in local 
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -1004,6 +1017,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
 option_since);
+   if (option_not.nr)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
-- 
2.8.0.rc0.210.gd302cd2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 16/25] clone: define shallow clone boundary based on time with --shallow-since

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
---
 Documentation/git-clone.txt |  3 +++
 builtin/clone.c | 16 +---
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index b7c467a..a410409 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -193,6 +193,9 @@ objects from the source repository into a pack in the 
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
 
+--shallow-since=::
+   Create a shallow clone with a history after the specified time.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
 
 static int option_no_checkout, option_bare, option_mirror, 
option_single_branch = -1;
 static int option_local = -1, option_no_hardlinks, option_shared, 
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
 static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
   N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", _depth, N_("depth"),
    N_("create a shallow clone of that depth")),
+   OPT_STRING(0, "shallow-since", _since, N_("time"),
+   N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
+   if (option_depth || option_since)
+   deepen = 1;
if (option_single_branch == -1)
-   option_single_branch = option_depth ? 1 : 0;
+   option_single_branch = deepen ? 1 : 0;
 
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
+   if (option_since)
+   warning(_("--shallow-since is ignored in local clones; 
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
 option_depth);
+   if (option_since)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
 option_upload_pack);
 
-   if (transport->smart_options && !option_depth)
+   if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected = 
1;
 
refs = transport_get_remote_refs(transport);
-- 
2.7.1.532.gd9e3aaa

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 21/25] clone: define shallow clone boundary with --shallow-exclude

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
---
 Documentation/git-clone.txt |  5 +
 builtin/clone.c | 18 +-
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index a410409..5049663 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -196,6 +196,11 @@ objects from the source repository into a pack in the 
cloned repository.
 --shallow-since=::
Create a shallow clone with a history after the specified time.
 
+--shallow-exclude=::
+   Create a shallow clone with a history, excluding commits
+   reachable from a specified remote branch or tag.  This option
+   can be specified multiple times.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..5ccf6b7 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
 static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
 static const char *real_git_dir;
 static char *option_upload_pack = "git-upload-pack";
 static int option_verbosity;
@@ -52,6 +53,13 @@ static struct string_list option_config;
 static struct string_list option_reference;
 static int option_dissociate;
 
+static int option_parse_deepen_not(const struct option *opt,
+  const char *arg, int unset)
+{
+   string_list_append(_not, arg);
+   return 0;
+}
+
 static struct option builtin_clone_options[] = {
OPT__VERBOSITY(_verbosity),
OPT_BOOL(0, "progress", _progress,
@@ -89,6 +97,9 @@ static struct option builtin_clone_options[] = {
        N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", _since, N_("time"),
N_("create a shallow clone since a specific time")),
+   { OPTION_CALLBACK, 0, "shallow-exclude", NULL, N_("revision"),
+   N_("deepen history of shallow clone by excluding rev"),
+   PARSE_OPT_NONEG, option_parse_deepen_not },
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -852,7 +863,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
-   if (option_depth || option_since)
+   if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +994,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones; 
use file:// instead."));
+   if (option_not.nr)
+   warning(_("--shallow-exclude is ignored in local 
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -1004,6 +1017,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
 option_since);
+   if (option_not.nr)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
-- 
2.7.1.532.gd9e3aaa

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 17/25] clone: define shallow clone boundary based on time with --shallow-since

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclo...@gmail.com> writes:

> Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
> ---

It is kind of surprising that 16 & 17 can be so simple and does not
have to update the way the cut-off points at the client side are
computed or recorded.  We must have done something right when we
designed the initial "--depth" support ;-).

On the other hand, that probably means we have the same "we clone
once, wait for a while and then do a shallow fetch with too short a
history span--the objects in the original clone all go too stale
that they become invisible in the resulting history" property (I
would not call that "an issue") as before.  It is just the way the
shallow boundary is specified got more user friendly (from a number
of parent-child hops to timespan).

Let's keep reading...

>  Documentation/git-clone.txt |  3 +++
>  builtin/clone.c | 16 +---
>  2 files changed, 16 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
> index 789b668..1b6b639 100644
> --- a/Documentation/git-clone.txt
> +++ b/Documentation/git-clone.txt
> @@ -194,6 +194,9 @@ objects from the source repository into a pack in the 
> cloned repository.
>   `--no-single-branch` is given to fetch the histories near the
>   tips of all branches.
>  
> +--shallow-since=::
> + Create a shallow clone with a history after the specified time.
> +
>  --[no-]single-branch::
>   Clone only the history leading to the tip of a single branch,
>   either specified by the `--branch` option or the primary
> diff --git a/builtin/clone.c b/builtin/clone.c
> index bcba080..dc2ef4f 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
>  
>  static int option_no_checkout, option_bare, option_mirror, 
> option_single_branch = -1;
>  static int option_local = -1, option_no_hardlinks, option_shared, 
> option_recursive;
> -static char *option_template, *option_depth;
> +static int deepen;
> +static char *option_template, *option_depth, *option_since;
>  static char *option_origin = NULL;
>  static char *option_branch = NULL;
>  static const char *real_git_dir;
> @@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
>  N_("path to git-upload-pack on the remote")),
>   OPT_STRING(0, "depth", _depth, N_("depth"),
>   N_("create a shallow clone of that depth")),
> + OPT_STRING(0, "shallow-since", _since, N_("time"),
> + N_("create a shallow clone since a specific time")),
>   OPT_BOOL(0, "single-branch", _single_branch,
>   N_("clone only one branch, HEAD or --branch")),
>   OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
> @@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char 
> *prefix)
>   usage_msg_opt(_("You must specify a repository to clone."),
>   builtin_clone_usage, builtin_clone_options);
>  
> + if (option_depth || option_since)
> + deepen = 1;
>   if (option_single_branch == -1)
> - option_single_branch = option_depth ? 1 : 0;
> + option_single_branch = deepen ? 1 : 0;
>  
>   if (option_mirror)
>   option_bare = 1;
> @@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char 
> *prefix)
>   if (is_local) {
>   if (option_depth)
>   warning(_("--depth is ignored in local clones; use 
> file:// instead."));
> + if (option_since)
> + warning(_("--shallow-since is ignored in local clones; 
> use file:// instead."));
>   if (!access(mkpath("%s/shallow", path), F_OK)) {
>   if (option_local > 0)
>   warning(_("source repository is shallow, 
> ignoring --local"));
> @@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char 
> *prefix)
>   if (option_depth)
>   transport_set_option(transport, TRANS_OPT_DEPTH,
>option_depth);
> + if (option_since)
> + transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
> +  option_since);
>   if (option_single_branch)
>   transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
>  
> @@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char 
> *prefix)
>

[PATCH v2 17/25] clone: define shallow clone boundary based on time with --shallow-since

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
---
 Documentation/git-clone.txt |  3 +++
 builtin/clone.c | 16 +---
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 789b668..1b6b639 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -194,6 +194,9 @@ objects from the source repository into a pack in the 
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
 
+--shallow-since=::
+   Create a shallow clone with a history after the specified time.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
 
 static int option_no_checkout, option_bare, option_mirror, 
option_single_branch = -1;
 static int option_local = -1, option_no_hardlinks, option_shared, 
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
 static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
   N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", _depth, N_("depth"),
    N_("create a shallow clone of that depth")),
+   OPT_STRING(0, "shallow-since", _since, N_("time"),
+   N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
+   if (option_depth || option_since)
+   deepen = 1;
if (option_single_branch == -1)
-   option_single_branch = option_depth ? 1 : 0;
+   option_single_branch = deepen ? 1 : 0;
 
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
+   if (option_since)
+   warning(_("--shallow-since is ignored in local clones; 
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
 option_depth);
+   if (option_since)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
 option_upload_pack);
 
-   if (transport->smart_options && !option_depth)
+   if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected = 
1;
 
refs = transport_get_remote_refs(transport);
-- 
2.7.0.377.g4cd97dd

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 22/25] clone: define shallow clone boundary with --shallow-exclude

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
---
 Documentation/git-clone.txt |  5 +
 builtin/clone.c | 18 +-
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 1b6b639..31e1610 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -197,6 +197,11 @@ objects from the source repository into a pack in the 
cloned repository.
 --shallow-since=::
Create a shallow clone with a history after the specified time.
 
+--shallow-exclude=::
+   Create a shallow clone with a history, excluding commits
+   reachable from a specified remote branch or tag.  This option
+   can be specified multiple times.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..5ccf6b7 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
 static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
 static const char *real_git_dir;
 static char *option_upload_pack = "git-upload-pack";
 static int option_verbosity;
@@ -52,6 +53,13 @@ static struct string_list option_config;
 static struct string_list option_reference;
 static int option_dissociate;
 
+static int option_parse_deepen_not(const struct option *opt,
+  const char *arg, int unset)
+{
+   string_list_append(_not, arg);
+   return 0;
+}
+
 static struct option builtin_clone_options[] = {
OPT__VERBOSITY(_verbosity),
OPT_BOOL(0, "progress", _progress,
@@ -89,6 +97,9 @@ static struct option builtin_clone_options[] = {
        N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", _since, N_("time"),
N_("create a shallow clone since a specific time")),
+   { OPTION_CALLBACK, 0, "shallow-exclude", NULL, N_("revision"),
+   N_("deepen history of shallow clone by excluding rev"),
+   PARSE_OPT_NONEG, option_parse_deepen_not },
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -852,7 +863,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
-   if (option_depth || option_since)
+   if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +994,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones; 
use file:// instead."));
+   if (option_not.nr)
+   warning(_("--shallow-exclude is ignored in local 
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -1004,6 +1017,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
 option_since);
+   if (option_not.nr)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
-- 
2.7.0.377.g4cd97dd

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V4 1/2] glossary: define the term shallow clone

[Stephen P. Smith]

There are several places in the documentation that
the term shallow clone is used. Defining the term
enables its use elsewhere with a known definition.

Signed-off-by: Stephen P. Smith <isch...@cox.net>
---

Notes:
The review comments for the user guide update[1] suggested a change
in the definition of a shallow clone to:
   - note differences between a shallow clone and a shallow repository and
   - define it as a noun (which is the way the user guide update
 patch uses the term)

[1] http://article.gmane.org/gmane.comp.version-control.git/283052

 Documentation/glossary-content.txt | 5 +
 1 file changed, 5 insertions(+)

diff --git a/Documentation/glossary-content.txt 
b/Documentation/glossary-content.txt
index e225974..cafc284 100644
--- a/Documentation/glossary-content.txt
+++ b/Documentation/glossary-content.txt
@@ -531,6 +531,11 @@ The most notable example is `HEAD`.
"Secure Hash Algorithm 1"; a cryptographic hash function.
In the context of Git used as a synonym for <<def_object_name,object 
name>>.
 
+[[def_shallow_clone]]shallow clone::
+   Mostly a synonym to <<def_shallow_repository,shallow repository>>
+   but the phrase makes it more explicit that it was created by
+   running `git clone --depth=...` command.
+
 [[def_shallow_repository]]shallow repository::
A shallow <<def_repository,repository>> has an incomplete
history some of whose <<def_commit,commits>> have 
<<def_parent,parents>> cauterized away (in other
-- 
2.7.0-rc2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 13/20] clone: define shallow clone boundary based on time with --since

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
---
 Documentation/git-clone.txt |  3 +++
 builtin/clone.c | 16 +---
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 6bf000d..28993c6 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -192,6 +192,9 @@ objects from the source repository into a pack in the 
cloned repository.
Create a 'shallow' clone with a history truncated to the
specified number of revisions.
 
+--since=::
+   Create a 'shallow' clone with a history after the specified time.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index a0b3cd9..94bbfef 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
 
 static int option_no_checkout, option_bare, option_mirror, 
option_single_branch = -1;
 static int option_local = -1, option_no_hardlinks, option_shared, 
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
 static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
   N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", _depth, N_("depth"),
    N_("create a shallow clone of that depth")),
+   OPT_STRING(0, "since", _since, N_("time"),
+   N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -846,8 +849,10 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
+   if (option_depth || option_since)
+   deepen = 1;
if (option_single_branch == -1)
-   option_single_branch = option_depth ? 1 : 0;
+   option_single_branch = deepen ? 1 : 0;
 
if (option_mirror)
option_bare = 1;
@@ -973,6 +978,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
+   if (option_since)
+   warning(_("--since is ignored in local clones; use 
file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -991,6 +998,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
 option_depth);
+   if (option_since)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
@@ -998,7 +1008,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
 option_upload_pack);
 
-   if (transport->smart_options && !option_depth)
+   if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected = 
1;
 
refs = transport_get_remote_refs(transport);
-- 
2.3.0.rc1.137.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 18/20] clone: define shallow clone boundary with --not

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com>
---
 Documentation/git-clone.txt |  5 +
 builtin/clone.c | 18 +-
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 28993c6..3589e57 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -195,6 +195,11 @@ objects from the source repository into a pack in the 
cloned repository.
 --since=::
Create a 'shallow' clone with a history after the specified time.
 
+--not=::
+   Create a 'shallow' clone with a history, excluding commits
+   reachable from a specified remote branch or tag.  This option
+   can be specified multiple times.
+
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index 94bbfef..0e99354 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
 static char *option_template, *option_depth, *option_since;
 static char *option_origin = NULL;
 static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
 static const char *real_git_dir;
 static char *option_upload_pack = "git-upload-pack";
 static int option_verbosity;
@@ -52,6 +53,13 @@ static struct string_list option_config;
 static struct string_list option_reference;
 static int option_dissociate;
 
+static int option_parse_deepen_not(const struct option *opt,
+  const char *arg, int unset)
+{
+   string_list_append(_not, arg);
+   return 0;
+}
+
 static struct option builtin_clone_options[] = {
OPT__VERBOSITY(_verbosity),
OPT_BOOL(0, "progress", _progress,
@@ -89,6 +97,9 @@ static struct option builtin_clone_options[] = {
        N_("create a shallow clone of that depth")),
OPT_STRING(0, "since", _since, N_("time"),
N_("create a shallow clone since a specific time")),
+   { OPTION_CALLBACK, 0, "not", NULL, N_("revision"),
+   N_("deepen history of shallow clone by excluding rev"),
+   PARSE_OPT_NONEG, option_parse_deepen_not },
OPT_BOOL(0, "single-branch", _single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", _git_dir, N_("gitdir"),
@@ -849,7 +860,7 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
 
-   if (option_depth || option_since)
+   if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -980,6 +991,8 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
warning(_("--depth is ignored in local clones; use 
file:// instead."));
if (option_since)
warning(_("--since is ignored in local clones; use 
file:// instead."));
+   if (option_not.nr)
+   warning(_("--not is ignored in local clones; use 
file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow, 
ignoring --local"));
@@ -1001,6 +1014,9 @@ int cmd_clone(int argc, const char **argv, const char 
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
 option_since);
+   if (option_not.nr)
+   transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 
-- 
2.3.0.rc1.137.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2 1/2] glossary: define the term shallow clone

[Stephen P. Smith]

There are several places in the documentation that
the term shallow clone is used. Defining the term
enables its use elsewhere with a known definition.
---
 Documentation/glossary-content.txt | 4 
 1 file changed, 4 insertions(+)

diff --git a/Documentation/glossary-content.txt 
b/Documentation/glossary-content.txt
index e225974..cecc83d 100644
--- a/Documentation/glossary-content.txt
+++ b/Documentation/glossary-content.txt
@@ -531,6 +531,10 @@ The most notable example is `HEAD`.
"Secure Hash Algorithm 1"; a cryptographic hash function.
In the context of Git used as a synonym for <<def_object_name,object 
name>>.
 
+[[def_shallow_clone]]shallow clone::
+   A clone of a <<def_repository,repository>> which creates a
+   <<def_shallow_repository,shallow_repository>>.
+
 [[def_shallow_repository]]shallow repository::
A shallow <<def_repository,repository>> has an incomplete
history some of whose <<def_commit,commits>> have 
<<def_parent,parents>> cauterized away (in other
-- 
2.6.3.368.gf34be46

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V3 1/2] glossary: define the term shallow clone

[Stephen P. Smith]

There are several places in the documentation that
the term shallow clone is used. Defining the term
enables its use elsewhere with a known definition.

Signed-off-by: Stephen P. Smith <isch...@cox.net>
---
 Documentation/glossary-content.txt | 4 
 1 file changed, 4 insertions(+)

diff --git a/Documentation/glossary-content.txt 
b/Documentation/glossary-content.txt
index e225974..cecc83d 100644
--- a/Documentation/glossary-content.txt
+++ b/Documentation/glossary-content.txt
@@ -531,6 +531,10 @@ The most notable example is `HEAD`.
"Secure Hash Algorithm 1"; a cryptographic hash function.
In the context of Git used as a synonym for <<def_object_name,object 
name>>.
 
+[[def_shallow_clone]]shallow clone::
+   A clone of a <<def_repository,repository>> which creates a
+   <<def_shallow_repository,shallow_repository>>.
+
 [[def_shallow_repository]]shallow repository::
A shallow <<def_repository,repository>> has an incomplete
history some of whose <<def_commit,commits>> have 
<<def_parent,parents>> cauterized away (in other
-- 
2.6.3.368.gf34be46

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] Define the term shallow clone.

[Stephen P. Smith]

There are several places in the documentation that
the term shallow clone is used. Defining the term
enables its use elsewhere with a known definition.

Signed-off-by: Stephen P. Smith <isch...@cox.net>
---
 Documentation/glossary-content.txt | 4 
 1 file changed, 4 insertions(+)

diff --git a/Documentation/glossary-content.txt 
b/Documentation/glossary-content.txt
index e225974..d384aed 100644
--- a/Documentation/glossary-content.txt
+++ b/Documentation/glossary-content.txt
@@ -531,6 +531,10 @@ The most notable example is `HEAD`.
"Secure Hash Algorithm 1"; a cryptographic hash function.
In the context of Git used as a synonym for <<def_object_name,object 
name>>.
 
+[[def_shallow_clone]]shallow clone::
+   A clone of a <<def_repository,repository>> which creates a
+<<def_shallow_repository,shallow_repository>>.
+
 [[def_shallow_repository]]shallow repository::
A shallow <<def_repository,repository>> has an incomplete
history some of whose <<def_commit,commits>> have 
<<def_parent,parents>> cauterized away (in other
-- 
2.6.3.368.gf34be46

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] Define the term shallow clone.

[Eric Sunshine]

On Mon, Dec 21, 2015 at 9:09 PM, Stephen P. Smith <isch...@cox.net> wrote:
> There are several places in the documentation that
> the term shallow clone is used. Defining the term
> enables its use elsewhere with a known definition.
>
> Signed-off-by: Stephen P. Smith <isch...@cox.net>
> ---
> diff --git a/Documentation/glossary-content.txt 
> b/Documentation/glossary-content.txt
> @@ -531,6 +531,10 @@ The most notable example is `HEAD`.
> "Secure Hash Algorithm 1"; a cryptographic hash function.
> In the context of Git used as a synonym for <<def_object_name,object 
> name>>.
>
> +[[def_shallow_clone]]shallow clone::
> +   A clone of a <<def_repository,repository>> which creates a
> +<<def_shallow_repository,shallow_repository>>.

Botched indentation on second line of definition. Use tab rather than spaces.

>  [[def_shallow_repository]]shallow repository::
> A shallow <<def_repository,repository>> has an incomplete
> history some of whose <<def_commit,commits>> have 
> <<def_parent,parents>> cauterized away (in other
> --
> 2.6.3.368.gf34be46
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Jason Paller-Rzepka]

I should point out that I never saw a "deepen 0" line.  Rather, I saw
git send "option depth 0" to the remote helper.
Duy, you mentioned that depth=0 means "do not change depth".  I assume
that means the server should use exactly the shallows that the client
sent, and it does not need to traverse the tree or modify the shallow
or unshallow sets at all.  Right?
Duy, you also mentioned that "those lines should be rejected any way".
You just mean that a "deepen 0" line should be rejected, right? And
that's because the right way to tell git-upload-pack not to change the
depth is to omit the "deepen" line after the "shallow" lines, so
there's never a need to send "deepen 0"?

Thanks!

On Sun, Dec 6, 2015 at 5:46 AM, Duy Nguyen  wrote:
> On Sun, Dec 6, 2015 at 8:01 AM, Jeff King  wrote:
>> On Sun, Dec 06, 2015 at 01:37:18AM -0500, Jeff King wrote:
>>
>>> And indeed, replacing the logic with what I wrote does make the backfill
>>> go away in my test case. But it's so far from what is there that I feel
>>> like I must be missing something.
>>
>> I think one thing I was missing is that we need to just grab the
>> _object_, but we need to realize that the ref needs updating[1]. So we
>> cannot skip backfill of any tag that we do not already have, even if we
>> already have the tag object.
>
> It's probably worth adding a few comment lines about this. I did
> search back commit history but did not get this.
>
>> Which made me wonder why this:
>>
>>   git init parent &&
>>   git -C parent commit --allow-empty -m one &&
>>   git clone parent child &&
>>   git -C parent commit --allow-empty -m two &&
>>   git -C parent tag -m mytag foo &&
>>   git -C parent commit --allow-empty -m three &&
>>   git -C child fetch
>>
>> does not appear to need to backfill to pick up refs/tags/foo. But it
>> does. It's just that it hits the quickfetch() code path and does not
>> have to ask the other side for a pack. And that explains why it does hit
>> in the --shallow case: we explicitly disable quickfetch in such cases.
>>
>> For the unshallow case, of course we could use it (but only for the
>> second, backfill fetch). Something like this seems to work for me:
>>
>> diff --git a/builtin/fetch.c b/builtin/fetch.c
>> index ed84963..b33b90f 100644
>> --- a/builtin/fetch.c
>> +++ b/builtin/fetch.c
>> @@ -881,6 +881,8 @@ static void backfill_tags(struct transport *transport, 
>> struct ref *ref_map)
>>
>> transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, NULL);
>> transport_set_option(transport, TRANS_OPT_DEPTH, "0");
>> +   if (unshallow)
>> +   depth = NULL;
>> fetch_refs(transport, ref_map);
>>
>> if (gsecondary) {
>>
>> But I admit I am not at all confident that it doesn't cause other
>> problems, or that it covers all cases. Even in a shallow repo, we should
>> be able to quickfetch individual tags, shouldn't we?
>
> Yes. depth is only non-NULL when you pass --depth (or --unshallow).
> quickfetch should happen when you fetch without those options.
>
>> I wonder if we could just always set "depth = NULL" here.
>
> --unshallow is essentially --depth=, so I don't see why
> --unshallow should be singled out here. We probably want to restore
> depth back (or pass a flag to explicitly ignore the "depth" exception
> in quickfetch). For multiple fetches, we spawn new commands so depth
> being NULL does not harm. Just in case somebody tries to fetch a
> couple more times in the same process in future.
> --
> Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Duy Nguyen]

On Mon, Dec 7, 2015 at 8:57 PM, Jason Paller-Rzepka  wrote:
> Duy, you mentioned that depth=0 means "do not change depth".  I assume that
> means the server should use exactly the shallows that the client sent, and
> it does not need to traverse the tree or modify the shallow or unshallow
> sets at all.  Right?

Correct. The server might send new shallow lines anyway though, if the
server repo is also shallow and the new fetched ref needs to be cut.
But I don't know if Dulwich supports that yet.

> Duy, you also mentioned that "those lines should be rejected any way".  You
> just mean that a "deepen 0" line should be rejected, right? And that's
> because the right way to tell git-upload-pack not to change the depth is to
> omit the "deepen" line after the "shallow" lines, so there's never a need to
> send "deepen 0"?

Also correct. I didn't check the code when I wrote that. But I have
checked and upload-pack does reject "deepen 0"

if (starts_with(line, "deepen ")) {
   char *end;
   depth = strtol(line + 7, , 0);
   if (end == line + 7 || depth <= 0)
  die("Invalid deepen: %s", line);
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Junio C Hamano]

Jeff King  writes:

> I think one thing I was missing is that we need to just grab the
> _object_, but we need to realize that the ref needs updating[1]. So we
> cannot skip backfill of any tag that we do not already have, even if we
> already have the tag object.
> ...
> [1] I'm still puzzled why find_non_local_tags uses has_sha1_file() on
> the tag object at all, then.

The designed semantics of auto-following tags (not necessarily as
implemented or documented, i.e. there may be implementation or
documentation bugs), I think, is to arrive at the same state as
doing a fetch (or a push) without the auto-following and then doing
a separate fetch (or a push) of tags that point at the objects that
are reachable from the tips of refs after finishing the first
(i.e. without auto-follow) fetch (or a push).  In a scenario where
we already have a commit reachable from existing remote-tracking
branch and the current transfer (be it a fetch or a push, with or
without auto-follow) does not update any remote-tracking branch
(because the source side did not have any changes), if the source
side added a tag that refers to that commit that the receiving end
lacks, that tag needs to be transferred and then stored.

So has_sha1_file() is not the right test---if anything, it needs to
be checking if the object being checked is reachable from a tip of
some ref.

But of course, that test is rather expensive, so perhaps the
implementation cheated and uses has_sha1_file() instead?  The only
case it would misidentify would be after an aborted fetch (or push)
left unconnected island of objects and some of these objects that
are not reachable are pointed at by tags the receiving end does not
have.





 
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Jeff King]

On Mon, Dec 07, 2015 at 01:27:51PM -0800, Junio C Hamano wrote:

> Jeff King  writes:
> 
> > I think one thing I was missing is that we need to just grab the
> > _object_, but we need to realize that the ref needs updating[1]. So we
> > cannot skip backfill of any tag that we do not already have, even if we
> > already have the tag object.
> > ...
> > [1] I'm still puzzled why find_non_local_tags uses has_sha1_file() on
> > the tag object at all, then.
> 
> The designed semantics of auto-following tags (not necessarily as
> implemented or documented, i.e. there may be implementation or
> documentation bugs), I think, is to arrive at the same state as
> doing a fetch (or a push) without the auto-following and then doing
> a separate fetch (or a push) of tags that point at the objects that
> are reachable from the tips of refs after finishing the first
> (i.e. without auto-follow) fetch (or a push).  In a scenario where
> we already have a commit reachable from existing remote-tracking
> branch and the current transfer (be it a fetch or a push, with or
> without auto-follow) does not update any remote-tracking branch
> (because the source side did not have any changes), if the source
> side added a tag that refers to that commit that the receiving end
> lacks, that tag needs to be transferred and then stored.
> 
> So has_sha1_file() is not the right test---if anything, it needs to
> be checking if the object being checked is reachable from a tip of
> some ref.
> 
> But of course, that test is rather expensive, so perhaps the
> implementation cheated and uses has_sha1_file() instead?  The only
> case it would misidentify would be after an aborted fetch (or push)
> left unconnected island of objects and some of these objects that
> are not reachable are pointed at by tags the receiving end does not
> have.

I may have confused myself. There are actually two has_sha1_file() calls
in find_non_local_tags.

I agree it is the only sensible test for "do we have the commit this tag
peels to, and if so, we want to grab the tag".  Reachability is too
expensive to compute.

But for the other one ("do we have the tag object itself"), I initially
claimed "if we have the tag object already, we do not have to do the
backfill fetch". Which is not quite true. We have to update the ref even
if we have the tag object. But then, what if we have the tag object for
other reasons (e.g., because another tag points at it?).

E.g. in this sequence:

  git -C parent commit --allow-empty -m base
  git -C parent tag -m mytag foo
  git clone parent child
  git -C parent update-ref refs/tags/bar foo
  git -C child fetch

we must backfill refs/tags/bar during the fetch, even though we already
have the object. I don't see any point in checking has_sha1_file() for
the tagged object at all. If we don't have it, we obviously must fetch.
And if we do have it, we must fetch the ref, even if that results in no
objects transferred.

It's entirely possible I'm just confused, and AFAICT nobody has noticed
any breakage here, so please don't feel you need to spend a lot of time
humoring me. I'm just writing up my confusion for posterity. :)

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Duy Nguyen]

On Sun, Dec 6, 2015 at 8:01 AM, Jeff King  wrote:
> On Sun, Dec 06, 2015 at 01:37:18AM -0500, Jeff King wrote:
>
>> And indeed, replacing the logic with what I wrote does make the backfill
>> go away in my test case. But it's so far from what is there that I feel
>> like I must be missing something.
>
> I think one thing I was missing is that we need to just grab the
> _object_, but we need to realize that the ref needs updating[1]. So we
> cannot skip backfill of any tag that we do not already have, even if we
> already have the tag object.

It's probably worth adding a few comment lines about this. I did
search back commit history but did not get this.

> Which made me wonder why this:
>
>   git init parent &&
>   git -C parent commit --allow-empty -m one &&
>   git clone parent child &&
>   git -C parent commit --allow-empty -m two &&
>   git -C parent tag -m mytag foo &&
>   git -C parent commit --allow-empty -m three &&
>   git -C child fetch
>
> does not appear to need to backfill to pick up refs/tags/foo. But it
> does. It's just that it hits the quickfetch() code path and does not
> have to ask the other side for a pack. And that explains why it does hit
> in the --shallow case: we explicitly disable quickfetch in such cases.
>
> For the unshallow case, of course we could use it (but only for the
> second, backfill fetch). Something like this seems to work for me:
>
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index ed84963..b33b90f 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -881,6 +881,8 @@ static void backfill_tags(struct transport *transport, 
> struct ref *ref_map)
>
> transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, NULL);
> transport_set_option(transport, TRANS_OPT_DEPTH, "0");
> +   if (unshallow)
> +   depth = NULL;
> fetch_refs(transport, ref_map);
>
> if (gsecondary) {
>
> But I admit I am not at all confident that it doesn't cause other
> problems, or that it covers all cases. Even in a shallow repo, we should
> be able to quickfetch individual tags, shouldn't we?

Yes. depth is only non-NULL when you pass --depth (or --unshallow).
quickfetch should happen when you fetch without those options.

> I wonder if we could just always set "depth = NULL" here.

--unshallow is essentially --depth=, so I don't see why
--unshallow should be singled out here. We probably want to restore
depth back (or pass a flag to explicitly ignore the "depth" exception
in quickfetch). For multiple fetches, we spawn new commands so depth
being NULL does not harm. Just in case somebody tries to fetch a
couple more times in the same process in future.
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Jeff King]

On Sat, Dec 05, 2015 at 08:00:28PM -0800, Junio C Hamano wrote:

> > I suppose followtags feature has been around long enough that we can
> > simply trust that and skip the second fetch?
> 
> H, I wonder why the code needs the backfill fetch while talking
> to a server that has the include-tag capability, then?

The logic in find_non_local_tags makes no sense to me.

After we do the first fetch, we call this function to find out if there
are any tags we need to pick up. We iterate through the list of refs
given to us by the remote (including peeled lines), and do:

if (ends_with(ref->name, "^{}")) {
if (item && !has_sha1_file(ref->old_sha1) &&
!will_fetch(head, ref->old_sha1) &&
!has_sha1_file(item->util) &&
!will_fetch(head, item->util))
item->util = NULL;
...
}

where "ref" is the peeled line (i.e., the pointed-to commit), and "item"
is the preceding line (i.e., the tag itself) with util set to its sha1.
Any such item whose util survives as non-NULL is fetched in the backfill
step.

So I'd think the logic for backfilling a given tag should be:

  1. We have the peeled object, and...

  2. We don't currently have the tag pointing to it, and...

  3. We are not already going to fetch it.

You could write that as:

  if (has_sha1_file(ref->old_sha1) &&
  !has_sha1_file(item->util) &&
  !will_fetch(head, item->util))

Of course the conditional in the code is "should we skip the backfill",
the negation. So using De Morgan's laws, we'd write:

  if (!has_sha1_file(ref->old_sha1) ||
  has_sha1_file(item->util) ||
  will_fetch(head, item->util))

Which is quite a bit different than what is there. I'm not sure at all
what the "will_fetch(head, ref->old_sha1)" is doing. In fact, for the
backfill step, it looks like we feed an empty "head" list, so the
!will_fetch() is always true.

And indeed, replacing the logic with what I wrote does make the backfill
go away in my test case. But it's so far from what is there that I feel
like I must be missing something.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Jeff King]

On Sun, Dec 06, 2015 at 01:37:18AM -0500, Jeff King wrote:

> And indeed, replacing the logic with what I wrote does make the backfill
> go away in my test case. But it's so far from what is there that I feel
> like I must be missing something.

I think one thing I was missing is that we need to just grab the
_object_, but we need to realize that the ref needs updating[1]. So we
cannot skip backfill of any tag that we do not already have, even if we
already have the tag object.

Which made me wonder why this:

  git init parent &&
  git -C parent commit --allow-empty -m one &&
  git clone parent child &&
  git -C parent commit --allow-empty -m two &&
  git -C parent tag -m mytag foo &&
  git -C parent commit --allow-empty -m three &&
  git -C child fetch

does not appear to need to backfill to pick up refs/tags/foo. But it
does. It's just that it hits the quickfetch() code path and does not
have to ask the other side for a pack. And that explains why it does hit
in the --shallow case: we explicitly disable quickfetch in such cases.

For the unshallow case, of course we could use it (but only for the
second, backfill fetch). Something like this seems to work for me:

diff --git a/builtin/fetch.c b/builtin/fetch.c
index ed84963..b33b90f 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -881,6 +881,8 @@ static void backfill_tags(struct transport *transport, 
struct ref *ref_map)
 
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, NULL);
transport_set_option(transport, TRANS_OPT_DEPTH, "0");
+   if (unshallow)
+   depth = NULL;
fetch_refs(transport, ref_map);
 
if (gsecondary) {

But I admit I am not at all confident that it doesn't cause other
problems, or that it covers all cases. Even in a shallow repo, we should
be able to quickfetch individual tags, shouldn't we? I wonder if we
could just always set "depth = NULL" here.

-Peff

[1] I'm still puzzled why find_non_local_tags uses has_sha1_file() on
the tag object at all, then.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Junio C Hamano]

Duy Nguyen  writes:

> On Fri, Dec 4, 2015 at 11:45 PM, Junio C Hamano  wrote:
> ...
>> just like a regular fetch that auto-follows tags, where it has to
>> make a second fetch if the primary fetch fails to include everything
>> that is needed for propagating the tag for whatever reason.
>>
>> Having said that, IIRC, these days a depth limited clone is created
>> implicitly with --single-branch option, and I am not sure what the
>> right behaviour for the auto-following of tags in such a repository.
>
> I suppose followtags feature has been around long enough that we can
> simply trust that and skip the second fetch?

H, I wonder why the code needs the backfill fetch while talking
to a server that has the include-tag capability, then?
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Stefan Beller]

On Fri, Dec 4, 2015 at 1:27 PM, Jeff King  wrote:
>>
>> and could not see a second fetch, but only a
>> fetch-pack with --depth=2147483647
>
> This seems to reproduce consistently for me:
>
>   $ git clone --depth=1 git://github.com/git/git

I used the http protocol, so I guess that's the difference.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Jeff King]

On Fri, Dec 04, 2015 at 01:57:30PM -0800, Junio C Hamano wrote:

> Jeff King <p...@peff.net> writes:
> 
> > This seems to reproduce consistently for me:
> >
> >   $ git clone --depth=1 git://github.com/git/git
> >   Cloning into 'git'...
> >   remote: Counting objects: 2925, done.
> >   remote: Compressing objects: 100% (2602/2602), done.
> >   remote: Total 2925 (delta 230), reused 2329 (delta 206), pack-reused 0
> >   Receiving objects: 100% (2925/2925), 6.17 MiB | 10.80 MiB/s, done.
> >   Resolving deltas: 100% (230/230), done.
> >
> >   $ cd git
> >   $ git fetch --unshallow
> >   remote: Counting objects: 185430, done.
> >   remote: Compressing objects: 100% (46933/46933), done.
> >   remote: Total 185430 (delta 140505), reused 181589 (delta 136694), 
> > pack-reused 0
> >   Receiving objects: 100% (185430/185430), 52.80 MiB | 10.84 MiB/s, done.
> >   Resolving deltas: 100% (140505/140505), completed with 1784 local objects.
> >   remote: Counting objects: 579, done.
> >   remote: Compressing objects: 100% (579/579), done.
> >   remote: Total 579 (delta 0), reused 579 (delta 0), pack-reused 0
> >   Receiving objects: 100% (579/579), 266.85 KiB | 0 bytes/s, done.
> >   [... fetch output ...]
> >
> > That looks like two packs being received for the --unshallow case.
> 
> What is puzzling is that I do not seem to see this "two fetches"
> with the local transport.  I only see "deepen 2147483647" in the
> protocol log.

Yeah, I do not ever see "deepen 0" from GIT_TRACE_PACKET output. FWIW,
here's the output I am using to reproduce this locally:

  # do this once
  git clone --bare git://github.com/git/git src.git

  # do this for each test run
  rm -rf repo
  git clone --no-local --depth=1 src.git repo
  cd repo
  echo "==> unshallow" &&
  git fetch --progress --unshallow 2>&1 | grep remote

And you can see that there are two separate sections (and I traced this
to backfill_tags() with gdb).

Note that the issue goes away if the shallow clone is done with "--bare"
(I guess because we pick up tags differently in the initial clone?).

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Junio C Hamano]

Jeff King  writes:

>> But why would fetching a tag (or set of tags) merit a depth of zero?
>> Doesn't depth 1 mean "give me the the objects, and none of their
>> descendants"?  Why use 0?
>
> That comes from this line:
>
>   transport_set_option(transport, TRANS_OPT_DEPTH, "0");
>
> That line blame back to b888d61 (Make fetch a builtin, 2007-09-10),
> which isn't incredibly helpful.

Hmm, "0" means "no depth limitations", which is exactly what we want
in this "unshallow" case, I would think.  The behaviour observed is
just like a regular fetch that auto-follows tags, where it has to
make a second fetch if the primary fetch fails to include everything
that is needed for propagating the tag for whatever reason.

Having said that, IIRC, these days a depth limited clone is created
implicitly with --single-branch option, and I am not sure what the
right behaviour for the auto-following of tags in such a repository.

> I think that comes from the original git-fetch.sh, which had:
>
>   ?*)
>   # do not deepen a shallow tree when following tags
>   shallow_depth=
>
> Which makes sense. I think the code at that point is not aware that we
> just "unshallowed" and can therefore drop the depth parameter
> altogether. But I admit I am not all that familiar with the shallow
> code.
>
> +cc Duy, who can probably say something way more intelligent about this
> off the top of his head. :)
>
> -Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Jason Paller-Rzepka]

It appears that it happens when the shallow history grows to include a
commit that's pointed to by a previously unseen tag.  For example,
when I deepen a checkout of git to depth 8, I hit v2.5.2, and a second
fetch takes place.

```
$ git clone --depth=1 http://github.com/git/git
Cloning into 'git'...
remote: Counting objects: 2925, done.
remote: Compressing objects: 100% (2602/2602), done.
remote: Total 2925 (delta 230), reused 2329 (delta 206), pack-reused 0
Receiving objects: 100% (2925/2925), 6.17 MiB | 0 bytes/s, done.
Resolving deltas: 100% (230/230), done.
Checking connectivity... done.
$ git -C git fetch --depth=8
remote: Counting objects: 858, done.
remote: Compressing objects: 100% (774/774), done.
remote: Total 858 (delta 793), reused 138 (delta 80), pack-reused 0
Receiving objects: 100% (858/858), 364.53 KiB | 0 bytes/s, done.
Resolving deltas: 100% (793/793), completed with 476 local objects.
remote: Counting objects: 1, done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (1/1), done.
>From http://github.com/git/git
 * [new tag] v2.5.2 -> v2.5.2
$
```

But why would fetching a tag (or set of tags) merit a depth of zero?
Doesn't depth 1 mean "give me the the objects, and none of their
descendants"?  Why use 0?

Thanks!
Jason

On Fri, Dec 4, 2015 at 4:27 PM, Jeff King <p...@peff.net> wrote:
> On Fri, Dec 04, 2015 at 12:46:59PM -0800, Stefan Beller wrote:
>
>> On Mon, Nov 30, 2015 at 11:35 AM, Jason Paller-Rzepka
>> <jaso...@google.com> wrote:
>> > Hi all,
>> >
>> > Would anyone be willing to help me understand some shallow-clone
>> > behavior?  (I found a bug in Dulwich, and I'm looking for some context
>> > so I can determine how to fix it.)
>> >
>> > I learned that cgit sometimes performs two fetches for a `git fetch
>> > --unshallow`: one with depth 'infinity', and a subsequent one with
>> > depth zero.
>>
>> Is there a condition to trigger this 'sometimes' ?
>>
>> I just tried reproducing via
>> $ GIT_TRACE=1 git fetch --unshallow
>>
>> and could not see a second fetch, but only a
>> fetch-pack with --depth=2147483647
>
> This seems to reproduce consistently for me:
>
>   $ git clone --depth=1 git://github.com/git/git
>   Cloning into 'git'...
>   remote: Counting objects: 2925, done.
>   remote: Compressing objects: 100% (2602/2602), done.
>   remote: Total 2925 (delta 230), reused 2329 (delta 206), pack-reused 0
>   Receiving objects: 100% (2925/2925), 6.17 MiB | 10.80 MiB/s, done.
>   Resolving deltas: 100% (230/230), done.
>
>   $ cd git
>   $ git fetch --unshallow
>   remote: Counting objects: 185430, done.
>   remote: Compressing objects: 100% (46933/46933), done.
>   remote: Total 185430 (delta 140505), reused 181589 (delta 136694), 
> pack-reused 0
>   Receiving objects: 100% (185430/185430), 52.80 MiB | 10.84 MiB/s, done.
>   Resolving deltas: 100% (140505/140505), completed with 1784 local objects.
>   remote: Counting objects: 579, done.
>   remote: Compressing objects: 100% (579/579), done.
>   remote: Total 579 (delta 0), reused 579 (delta 0), pack-reused 0
>   Receiving objects: 100% (579/579), 266.85 KiB | 0 bytes/s, done.
>   [... fetch output ...]
>
> That looks like two packs being received for the --unshallow case.
>
> -Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Jeff King]

On Fri, Dec 04, 2015 at 04:38:16PM -0500, Jason Paller-Rzepka wrote:

> It appears that it happens when the shallow history grows to include a
> commit that's pointed to by a previously unseen tag.  For example,
> when I deepen a checkout of git to depth 8, I hit v2.5.2, and a second
> fetch takes place.

Yeah. The code is in builtin/fetch.c:backfill_tags.

> But why would fetching a tag (or set of tags) merit a depth of zero?
> Doesn't depth 1 mean "give me the the objects, and none of their
> descendants"?  Why use 0?

That comes from this line:

  transport_set_option(transport, TRANS_OPT_DEPTH, "0");

That line blame back to b888d61 (Make fetch a builtin, 2007-09-10),
which isn't incredibly helpful.

I think that comes from the original git-fetch.sh, which had:

  ?*)
  # do not deepen a shallow tree when following tags
  shallow_depth=

Which makes sense. I think the code at that point is not aware that we
just "unshallowed" and can therefore drop the depth parameter
altogether. But I admit I am not all that familiar with the shallow
code.

+cc Duy, who can probably say something way more intelligent about this
off the top of his head. :)

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Stefan Beller]

On Mon, Nov 30, 2015 at 11:35 AM, Jason Paller-Rzepka
<jaso...@google.com> wrote:
> Hi all,
>
> Would anyone be willing to help me understand some shallow-clone
> behavior?  (I found a bug in Dulwich, and I'm looking for some context
> so I can determine how to fix it.)
>
> I learned that cgit sometimes performs two fetches for a `git fetch
> --unshallow`: one with depth 'infinity', and a subsequent one with
> depth zero.

Is there a condition to trigger this 'sometimes' ?

I just tried reproducing via
$ GIT_TRACE=1 git fetch --unshallow

and could not see a second fetch, but only a
fetch-pack with --depth=2147483647
>
> Could anyone answer:
> 1) What is the purpose of the second fetch?
> 2) What does this depth of zero mean? Is it the same as a depth of
> infinity?  (I assume not... but, since I thought the smallest
> meaningful depth was 1, I don't know what else it might mean.)
>
> Thank you!
> Jason
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Jeff King]

On Fri, Dec 04, 2015 at 12:46:59PM -0800, Stefan Beller wrote:

> On Mon, Nov 30, 2015 at 11:35 AM, Jason Paller-Rzepka
> <jaso...@google.com> wrote:
> > Hi all,
> >
> > Would anyone be willing to help me understand some shallow-clone
> > behavior?  (I found a bug in Dulwich, and I'm looking for some context
> > so I can determine how to fix it.)
> >
> > I learned that cgit sometimes performs two fetches for a `git fetch
> > --unshallow`: one with depth 'infinity', and a subsequent one with
> > depth zero.
> 
> Is there a condition to trigger this 'sometimes' ?
> 
> I just tried reproducing via
> $ GIT_TRACE=1 git fetch --unshallow
> 
> and could not see a second fetch, but only a
> fetch-pack with --depth=2147483647

This seems to reproduce consistently for me:

  $ git clone --depth=1 git://github.com/git/git
  Cloning into 'git'...
  remote: Counting objects: 2925, done.
  remote: Compressing objects: 100% (2602/2602), done.
  remote: Total 2925 (delta 230), reused 2329 (delta 206), pack-reused 0
  Receiving objects: 100% (2925/2925), 6.17 MiB | 10.80 MiB/s, done.
  Resolving deltas: 100% (230/230), done.

  $ cd git
  $ git fetch --unshallow
  remote: Counting objects: 185430, done.
  remote: Compressing objects: 100% (46933/46933), done.
  remote: Total 185430 (delta 140505), reused 181589 (delta 136694), 
pack-reused 0
  Receiving objects: 100% (185430/185430), 52.80 MiB | 10.84 MiB/s, done.
  Resolving deltas: 100% (140505/140505), completed with 1784 local objects.
  remote: Counting objects: 579, done.
  remote: Compressing objects: 100% (579/579), done.
  remote: Total 579 (delta 0), reused 579 (delta 0), pack-reused 0
  Receiving objects: 100% (579/579), 266.85 KiB | 0 bytes/s, done.
  [... fetch output ...]

That looks like two packs being received for the --unshallow case.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Stefan Beller]

I can reproduce it now. Instead of using my $random version, I just
needed origin/master
to reproduce.

The second fetch is invoked via
(as outputted via GIT_TRACE=1 git -C git fetch --depth=8)

13:44:56.863841 run-command.c:343   trace: run_command:
'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin'
'https://github.com/git/git/'

so it seems like there is no explicit depth given, so I think the 0
comes from the initialization step and nobody touched it to fill with
meaningful values.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Junio C Hamano]

Jeff King  writes:

> This seems to reproduce consistently for me:
>
>   $ git clone --depth=1 git://github.com/git/git
>   Cloning into 'git'...
>   remote: Counting objects: 2925, done.
>   remote: Compressing objects: 100% (2602/2602), done.
>   remote: Total 2925 (delta 230), reused 2329 (delta 206), pack-reused 0
>   Receiving objects: 100% (2925/2925), 6.17 MiB | 10.80 MiB/s, done.
>   Resolving deltas: 100% (230/230), done.
>
>   $ cd git
>   $ git fetch --unshallow
>   remote: Counting objects: 185430, done.
>   remote: Compressing objects: 100% (46933/46933), done.
>   remote: Total 185430 (delta 140505), reused 181589 (delta 136694), 
> pack-reused 0
>   Receiving objects: 100% (185430/185430), 52.80 MiB | 10.84 MiB/s, done.
>   Resolving deltas: 100% (140505/140505), completed with 1784 local objects.
>   remote: Counting objects: 579, done.
>   remote: Compressing objects: 100% (579/579), done.
>   remote: Total 579 (delta 0), reused 579 (delta 0), pack-reused 0
>   Receiving objects: 100% (579/579), 266.85 KiB | 0 bytes/s, done.
>   [... fetch output ...]
>
> That looks like two packs being received for the --unshallow case.

What is puzzling is that I do not seem to see this "two fetches"
with the local transport.  I only see "deepen 2147483647" in the
protocol log.

Moreover, the only interesting lines in the output from

$ git grep -B1 'deepen ' \*.[ch]

are

fetch-pack.c-   if (args->depth > 0)
fetch-pack.c:   packet_buf_write(_buf, "deepen %d", args->depth);

so I do not see how anybody would be sending "deepen 0" as Jason
saw.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Multiple fetches when unshallowing a shallow clone

[Duy Nguyen]

On Fri, Dec 4, 2015 at 11:45 PM, Junio C Hamano  wrote:
> Jeff King  writes:
>
>>> But why would fetching a tag (or set of tags) merit a depth of zero?
>>> Doesn't depth 1 mean "give me the the objects, and none of their
>>> descendants"?  Why use 0?
>>
>> That comes from this line:
>>
>>   transport_set_option(transport, TRANS_OPT_DEPTH, "0");
>>
>> That line blame back to b888d61 (Make fetch a builtin, 2007-09-10),
>> which isn't incredibly helpful.
>
> Hmm, "0" means "no depth limitations", which is exactly what we want
> in this "unshallow" case, I would think.  The behaviour observed is

No depth 0 means "do not change depth", which is why Jeff saw no
'deepen' lines (and those lines should be rejected any way). It's
equivalent of doing "git fetch" without --depth.

> just like a regular fetch that auto-follows tags, where it has to
> make a second fetch if the primary fetch fails to include everything
> that is needed for propagating the tag for whatever reason.
>
> Having said that, IIRC, these days a depth limited clone is created
> implicitly with --single-branch option, and I am not sure what the
> right behaviour for the auto-following of tags in such a repository.

I suppose followtags feature has been around long enough that we can
simply trust that and skip the second fetch? But it's not that easy
for subsequent fetches after the initial fetch in git-clone, because
we no longer know if --single-branch was used (of if there is any new
branch fetched since).
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Multiple fetches when unshallowing a shallow clone

[Jason Paller-Rzepka]

Hi all,

Would anyone be willing to help me understand some shallow-clone
behavior?  (I found a bug in Dulwich, and I'm looking for some context
so I can determine how to fix it.)

I learned that cgit sometimes performs two fetches for a `git fetch
--unshallow`: one with depth 'infinity', and a subsequent one with
depth zero.

Could anyone answer:
1) What is the purpose of the second fetch?
2) What does this depth of zero mean? Is it the same as a depth of
infinity?  (I assume not... but, since I thought the smallest
meaningful depth was 1, I don't know what else it might mean.)

Thank you!
Jason
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git clone --depth: shallow clone problems

[Duy Nguyen]

On Tue, May 26, 2015 at 9:54 PM, Junio C Hamano gits...@pobox.com wrote:
 Duy Nguyen pclo...@gmail.com writes:

  --unshallow::
 - If the source repository is complete, convert a shallow
 - repository to a complete one, removing all the limitations
 + If the source repository is complete, convert all shallow
 + refs to complete ones, removing all the limitations

 Define shallow ref, or better yet explain without introducing such
 a new term (I do not think shallow-ness is a property of a ref, by
 the way---I think you meant all refs that can reach the points the
 history is cauterised by .git/shallow, but we shouldn't assume that
 the target audience of this paragraph to know Git as well as I do).

OK maybe

Make sure all existing refs have the same history as the ones from the
source repository. If the source repository is complete, this removes
all limitations imposed by shallow repositories.

?

   imposed by shallow repositories.
  +
  If the source repository is shallow, fetch as much as possible so that
 -the current repository has the same history as the source repository.
 +the all existing refs of current repository have the same history as in
 +the source repository.

 Makes sense, modulo so that the all existing refs sounds strange
 to my ears (all the existing refs perhaps).

Will fix. Although I think the new paragraph above already covers this case too.

 ++
 +Note that if the repository is created with --single-branch, which is
 +default for a shallow clone, only one ref is completed. `--unshallow`
 +does not fetch all remaining refs from source repository.

 I do not think this Note is being as helpful as it could be.

  - If the repository was created with --single-branch but if the
user later fetched and started tracking other branches, the
statement only one ref is completed is untrue; the true version
is only the existing refs are completed, which leads to a more
important point.  It says the same thing as all existing refs
above and does not add any useful information.

  - The last sentence is less than useful but merely frustrating---it
says what you cannot do with this option, strongly hints that the
writer of the sentence knows what the reader wants to achieve,
but without saying what other way the reader go to achieve it.

 Perhaps replace that Note paragraph with something along this line?

 +
 By fetching and tracking refs that you haven't been tracking,
 you can add these new refs to all refs to be unshallowed.

It was meant to emphasize --unshallow would not create a true clone,
if people miss the keywords all existing refs.. Maybe..

You may need to fetch and track other refs from the source repository
if you want to make full clone.

?


 2) git fetch --unshallow should convert the clone into an actual
 equivalent of a normal, not shallow clone.

 I was thinking of some way to undo --single-branch too. I don't think
 it should be the job of --unshallow, maybe a new option, but I can't
 think of a name better than --really-unshallow :P

 Isn't that just the matter of updating remote.origin.fetch?  I do
 not think it belongs to clone (or fetch).  Perhaps it belongs to
 remote, where it already shows with git remote -v what is
 fetched and pushed.

Yeah, perhaps showing an advice about git-remote is enough if the
current ref spec does not fetch everything.

  --depth depth::
   Create a 'shallow' clone with a history truncated to the
 - specified number of revisions.
 + specified number of revisions. --single-branch is
 + automatically specified unless the user overrides it with
 + --no-single-branch

 Yeah, something like that would be a definite improvement.

 By the way, while composing this message, I scratched my head after
 typing

 $ git clone --shallow=4 --no-local ./git.git ./playpen

 Is it just me or do we want to add such a synonym?

I have a series to define shallow boundaries by tags or dates, in
addition to history depth. It may take a few more months until I'm
finished with my ongoing topics. Then you could think about what
--shallow means exactly ;-)
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git clone --depth: shallow clone problems

[Junio C Hamano]

Duy Nguyen pclo...@gmail.com writes:

 On Tue, May 26, 2015 at 9:54 PM, Junio C Hamano gits...@pobox.com wrote:
 Duy Nguyen pclo...@gmail.com writes:

  --unshallow::
 - If the source repository is complete, convert a shallow
 - repository to a complete one, removing all the limitations
 + If the source repository is complete, convert all shallow
 + refs to complete ones, removing all the limitations

 Define shallow ref, or better yet explain without introducing such
 a new term (I do not think shallow-ness is a property of a ref, by
 the way---I think you meant all refs that can reach the points the
 history is cauterised by .git/shallow, but we shouldn't assume that
 the target audience of this paragraph to know Git as well as I do).

 OK maybe

 Make sure all existing refs have the same history as the ones from the
 source repository. If the source repository is complete, this removes
 all limitations imposed by shallow repositories.

OK, I can understand that description a lot easier than the original
one.

 ++
 +Note that if the repository is created with --single-branch, which is
 +default for a shallow clone, only one ref is completed. `--unshallow`
 +does not fetch all remaining refs from source repository.

 I do not think this Note is being as helpful as it could be.

  - If the repository was created with --single-branch but if the
user later fetched and started tracking other branches, the
statement only one ref is completed is untrue; the true version
is only the existing refs are completed, which leads to a more
important point.  It says the same thing as all existing refs
above and does not add any useful information.

  - The last sentence is less than useful but merely frustrating---it
says what you cannot do with this option, strongly hints that the
writer of the sentence knows what the reader wants to achieve,
but without saying what other way the reader go to achieve it.

 Perhaps replace that Note paragraph with something along this line?

 +
 By fetching and tracking refs that you haven't been tracking,
 you can add these new refs to all refs to be unshallowed.

 It was meant to emphasize --unshallow would not create a true clone,
 if people miss the keywords all existing refs.. Maybe..

I know; it is not helpful to describe what it does *NOT* do, without
saying what the reader is likely trying to find out.

 You may need to fetch and track other refs from the source repository
 if you want to make full clone.

That is much better.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git clone --depth: shallow clone problems

[Duy Nguyen]

On Thu, Apr 23, 2015 at 04:53:04PM +0200, Michal Pomorski wrote:
 tl: skip to the second paragraph
 
 So here is what I just experienced:
 We had an emergency error in an application at work and as the
 responsible developer was unavailable, I was asked to check it out and
 look into it.
 We are a small branch of a bigger company and our connection to the
 company's source servers is really slow, so just to quickly start it
 up, I decided to take a shallow clone (that's what it is for, right?).
 After a while we realized, the clone I have made was not sufficient
 and was missing some newest work done on the project.
 I did git fetch --unshallow which finished surprisingly quickly, and
 it did not bring any newer commits.
 Unaware of the fine print hiding in the documentation of git clone, I
 blamed the repo (and in extension the person who provided me the
 address to it). After coming to a realization, a process which cost me
 time and embarrassment, I understood what is supposed to be the
 correct behaviour:
 --single-branch
 Clone only the history leading to the tip of a single branch, either
 specified by the --branch option or the primary branch remote's HEAD
 points at. When creating a shallow clone with the --depth option, this
 is the default, unless --no-single-branch is given to fetch the
 histories near the tips of all branches.
 Of course, the new commits were put on a custom branch, and I knew
 that all the time, but I expected the branch to show up eventually, at
 least after git fetch --unshallow.
 

 I hope you can see the faults in the usability of the commands I was
 exposed to:
 1) git clone --depth  should:
-warn about only fetching the current HEAD (name it by a real name
 if applicable)
 and/or
-require the --branch option so that it is not left to chance
 (current HEAD could be anything; is it really meaningful to talk about
 the current HEAD on a server?)
 and/or
-make the --no-single-branch the default...
 and maybe
-have the option to clone the most recent commits.

--single-branch being the default was because we (at least I, stilll)
believed it was the common case, so I don't think we should revert
it. But I can see --unshallow documentation is misleading. How about
this? I think it's better than what we have.

-- 8 --
Subject: [PATCH] fetch-options.txt: clarify what --unshallow does

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 Documentation/fetch-options.txt | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 45583d8..63d3452 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -14,12 +14,17 @@
branch history. Tags for the deepened commits are not fetched.
 
 --unshallow::
-   If the source repository is complete, convert a shallow
-   repository to a complete one, removing all the limitations
+   If the source repository is complete, convert all shallow
+   refs to complete ones, removing all the limitations
imposed by shallow repositories.
 +
 If the source repository is shallow, fetch as much as possible so that
-the current repository has the same history as the source repository.
+the all existing refs of current repository have the same history as in
+the source repository.
++
+Note that if the repository is created with --single-branch, which is
+default for a shallow clone, only one ref is completed. `--unshallow`
+does not fetch all remaining refs from source repository.
 
 --update-shallow::
By default when fetching from a shallow repository,
-- 
2.3.0.rc1.137.g477eb31
-- 8 --

 2) git fetch --unshallow should convert the clone into an actual
 equivalent of a normal, not shallow clone.

I was thinking of some way to undo --single-branch too. I don't think
it should be the job of --unshallow, maybe a new option, but I can't
think of a name better than --really-unshallow :P

It's been a while and no one responds to this, perhaps people are not
interested in such an option?

 3) The documentation should be improved. The behaviour of --shallow is
 described partly in the description of --no-single-branch. This is the
 documentation equivalent of the infamous come from control flow
 structure.

Yes. Like this?

-- 8 --
Subject: [PATCH] clone.txt: mention --single-branch in --depth

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 Documentation/git-clone.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index f1f2a3f..9c320da 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -185,7 +185,9 @@ objects from the source repository into a pack in the 
cloned repository.
 
 --depth depth::
Create a 'shallow' clone with a history truncated to the
-   specified number of revisions.
+   specified number of revisions. --single-branch

Re: git clone --depth: shallow clone problems

[Junio C Hamano]

Duy Nguyen pclo...@gmail.com writes:

  --unshallow::
 - If the source repository is complete, convert a shallow
 - repository to a complete one, removing all the limitations
 + If the source repository is complete, convert all shallow
 + refs to complete ones, removing all the limitations

Define shallow ref, or better yet explain without introducing such
a new term (I do not think shallow-ness is a property of a ref, by
the way---I think you meant all refs that can reach the points the
history is cauterised by .git/shallow, but we shouldn't assume that
the target audience of this paragraph to know Git as well as I do).

   imposed by shallow repositories.
  +
  If the source repository is shallow, fetch as much as possible so that
 -the current repository has the same history as the source repository.
 +the all existing refs of current repository have the same history as in
 +the source repository.

Makes sense, modulo so that the all existing refs sounds strange
to my ears (all the existing refs perhaps).

 ++
 +Note that if the repository is created with --single-branch, which is
 +default for a shallow clone, only one ref is completed. `--unshallow`
 +does not fetch all remaining refs from source repository.

I do not think this Note is being as helpful as it could be.

 - If the repository was created with --single-branch but if the
   user later fetched and started tracking other branches, the
   statement only one ref is completed is untrue; the true version
   is only the existing refs are completed, which leads to a more
   important point.  It says the same thing as all existing refs
   above and does not add any useful information.

 - The last sentence is less than useful but merely frustrating---it
   says what you cannot do with this option, strongly hints that the
   writer of the sentence knows what the reader wants to achieve,
   but without saying what other way the reader go to achieve it.

Perhaps replace that Note paragraph with something along this line?

+
By fetching and tracking refs that you haven't been tracking,
you can add these new refs to all refs to be unshallowed.

 2) git fetch --unshallow should convert the clone into an actual
 equivalent of a normal, not shallow clone.

 I was thinking of some way to undo --single-branch too. I don't think
 it should be the job of --unshallow, maybe a new option, but I can't
 think of a name better than --really-unshallow :P

Isn't that just the matter of updating remote.origin.fetch?  I do
not think it belongs to clone (or fetch).  Perhaps it belongs to
remote, where it already shows with git remote -v what is
fetched and pushed.

  --depth depth::
   Create a 'shallow' clone with a history truncated to the
 - specified number of revisions.
 + specified number of revisions. --single-branch is
 + automatically specified unless the user overrides it with
 + --no-single-branch

Yeah, something like that would be a definite improvement.

By the way, while composing this message, I scratched my head after
typing

$ git clone --shallow=4 --no-local ./git.git ./playpen

Is it just me or do we want to add such a synonym?

Thanks.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

git clone --depth: shallow clone problems

[Michal Pomorski]

tl: skip to the second paragraph

So here is what I just experienced:
We had an emergency error in an application at work and as the
responsible developer was unavailable, I was asked to check it out and
look into it.
We are a small branch of a bigger company and our connection to the
company's source servers is really slow, so just to quickly start it
up, I decided to take a shallow clone (that's what it is for, right?).
After a while we realized, the clone I have made was not sufficient
and was missing some newest work done on the project.
I did git fetch --unshallow which finished surprisingly quickly, and
it did not bring any newer commits.
Unaware of the fine print hiding in the documentation of git clone, I
blamed the repo (and in extension the person who provided me the
address to it). After coming to a realization, a process which cost me
time and embarrassment, I understood what is supposed to be the
correct behaviour:
--single-branch
Clone only the history leading to the tip of a single branch, either
specified by the --branch option or the primary branch remote's HEAD
points at. When creating a shallow clone with the --depth option, this
is the default, unless --no-single-branch is given to fetch the
histories near the tips of all branches.
Of course, the new commits were put on a custom branch, and I knew
that all the time, but I expected the branch to show up eventually, at
least after git fetch --unshallow.


I hope you can see the faults in the usability of the commands I was exposed to:
1) git clone --depth  should:
   -warn about only fetching the current HEAD (name it by a real name
if applicable)
and/or
   -require the --branch option so that it is not left to chance
(current HEAD could be anything; is it really meaningful to talk about
the current HEAD on a server?)
and/or
   -make the --no-single-branch the default...
and maybe
   -have the option to clone the most recent commits.

2) git fetch --unshallow should convert the clone into an actual
equivalent of a normal, not shallow clone.

3) The documentation should be improved. The behaviour of --shallow is
described partly in the description of --no-single-branch. This is the
documentation equivalent of the infamous come from control flow
structure.

Regards,
Michal Pomorski
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to repair a shallow clone (?)

[Philip Oakley]

From: Trần Ngọc Quân vnwild...@gmail.com

On 06/12/2014 19:23, Torsten Bögershausen wrote:

I think I started to clone the repo in a shallow way
(SparkleShare asked if I want to clone the complete history,
and I probably answered no )

Is there a way to repair this situation ?
(Except doing a complete re-clone ?)


I think git don't accept push from shallow repo. I've ever encounter
this problem. I UNshallow it, then every thing will work:

$ git fetch --unshallow origin

This command will convert a shallow repository to a complete one.
See git-fetch(1) and git-clone(1).

Since v1.9.0 (14 Feb '14.) you can do various push/pull from a shallow 
clone (I'd asked this way back 
http://stackoverflow.com/questions/6900103/why-cant-i-push-from-a-shallow-clone 
and noted when it was corrected/improved)


That's not to say that you don't have to take care about your local 
depth being sufficiently inclusive.


I'm sure that sometime a --timedepth=time_t time will eventually be 
coded by someone sufficiently in need. ;-)


--
Philip 


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to repair a shallow clone (?)

[Duy Nguyen]

On Sat, Dec 6, 2014 at 7:23 PM, Torsten Bögershausen tbo...@web.de wrote:
 I share a bare repo with Sparkleshare which does an auto-synch.
 Now the synch had stopped, and trying to push to the central repo
 by hand gives this:



 git push  origin master
 fatal: protocol error: expected old/new/ref, got 'shallow 
 72fb4080921221293e28a97a0e8c78d6100c5186'
 fatal: The remote end hung up unexpectedly
 Counting objects: 4, done.
 Delta compression using up to 2 threads.
 Compressing objects: 100% (4/4), done.
 error: pack-objects died of signal 13
 error: failed to push some refs to x

 Both machines have Git 2.0.0

Please try again with $GIT_TRACE_PACKET=/some-log-file. receive-pack
2.0 should recognize this shallow line.

 Is this a known issue/problem ?

No.
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to repair a shallow clone (?)

[Torsten Bögershausen]

On 2014-12-07 12.44, Duy Nguyen wrote:
 Is this a known issue/problem ?
 
 No.
 
Thanks everybody for the support.

The machine was equipped with git version 1.7.10.4 in /usr/bin.

I installed 2.1 or so under /usr/local/bin, (and even /root/bin) 
thinking that this would help, but it didn't.

Because the login shell for the user storage which manages the push/pull
on the server side was /usr/bin/git-shell, not /usr/local/bin/git-shell.



--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

How to repair a shallow clone (?)

[Torsten Bögershausen]

I share a bare repo with Sparkleshare which does an auto-synch.
Now the synch had stopped, and trying to push to the central repo
by hand gives this:



git push  origin master
fatal: protocol error: expected old/new/ref, got 'shallow 
72fb4080921221293e28a97a0e8c78d6100c5186'
fatal: The remote end hung up unexpectedly
Counting objects: 4, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (4/4), done.
error: pack-objects died of signal 13
error: failed to push some refs to x

Both machines have Git 2.0.0

Is this a known issue/problem ?

I think I started to clone the repo in a shallow way
(SparkleShare asked if I want to clone the complete history,
and I probably answered no )

Is there a way to repair this situation ?
(Except doing a complete re-clone ?)

Thanks for help
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to repair a shallow clone (?)

[Trần Ngọc Quân]

On 06/12/2014 19:23, Torsten Bögershausen wrote:
 I think I started to clone the repo in a shallow way
 (SparkleShare asked if I want to clone the complete history,
 and I probably answered no )

 Is there a way to repair this situation ?
 (Except doing a complete re-clone ?)

I think git don't accept push from shallow repo. I've ever encounter
this problem. I UNshallow it, then every thing will work:

$ git fetch --unshallow origin

This command will convert a shallow repository to a complete one.
See git-fetch(1) and git-clone(1).

I hope it helpful!
Thanks,

-- 
Trần Ngọc Quân.

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH/RFC v2 2/2] submodule: modify clone command to recursively shallow clone submodules

[Cole Minnaar]

When cloning a repository that contains submodules and specifying the
`--depth` option to the 'git clone' command, the top level repository will be
cloned with the specified depth, but all submodules within the
repository will be cloned in their entirety.

Modified 'git clone' to pass the `--depth` option, if specified, to any
submodule clone commands.
Modified 'git clone' to pass the `--no-single-branch`, if specified, to any
submodule clone commands.

Signed-off-by: Cole Minnaar cole.minn...@gmail.com
---
 Documentation/git-clone.txt |  5 -
 builtin/clone.c | 15 +--
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 0363d00..7621251 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -178,7 +178,8 @@ objects from the source repository into a pack in the 
cloned repository.
 
 --depth depth::
Create a 'shallow' clone with a history truncated to the
-   specified number of revisions.
+   specified number of revisions. If `--recursive` was also specified
+   the depth value will be passed to all submodules within when cloning.
 
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
@@ -192,6 +193,8 @@ objects from the source repository into a pack in the 
cloned repository.
initial cloning.  If the HEAD at the remote did not point at any
branch when `--single-branch` clone was made, no remote-tracking
branch is created.
+   If `--recursive` was also specified, this option will be passed to
+   all submodules when cloning.
 
 --recursive::
 --recurse-submodules::
diff --git a/builtin/clone.c b/builtin/clone.c
index dd4092b..b27917c 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -48,6 +48,7 @@ static int option_verbosity;
 static int option_progress = -1;
 static struct string_list option_config;
 static struct string_list option_reference;
+static struct argv_array argv_submodule_cmd = ARGV_ARRAY_INIT;
 
 static int opt_parse_reference(const struct option *opt, const char *arg, int 
unset)
 {
@@ -100,10 +101,6 @@ static struct option builtin_clone_options[] = {
OPT_END()
 };
 
-static const char *argv_submodule[] = {
-   submodule, update, --init, --recursive, NULL
-};
-
 static char *get_repo_path(const char *repo, int *is_bundle)
 {
static char *suffix[] = { /.git, , .git/.git, .git };
@@ -663,8 +660,14 @@ static int checkout(void)
err |= run_hook_le(NULL, post-checkout, sha1_to_hex(null_sha1),
   sha1_to_hex(sha1), 1, NULL);
 
-   if (!err  option_recursive)
-   err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
+   if (!err  option_recursive) {
+   argv_array_pushl(argv_submodule_cmd, submodule, update, 
--init, --recursive, NULL);
+   if (option_depth)
+   argv_array_pushf(argv_submodule_cmd, --depth=%d, 
atoi(option_depth));
+   if (!option_single_branch)
+   argv_array_pushl(argv_submodule_cmd, 
--no-single-branch, NULL);
+   err = run_command_v_opt(argv_submodule_cmd.argv, RUN_GIT_CMD);
+   }
 
return err;
 }
-- 
2.1.0.240.g8a0e823

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH/RFC 2/2] submodule: modify clone command to recursively shallow clone submodules

[Cole Minnaar]

When cloning a repository that contains submodules and specifying the
`--depth` option to the 'git clone' command, the top level repository will be
cloned with the specified depth, but all submodules within the
repository will be cloned in their entirety.

Modified 'git clone' to pass the `--depth` option, if specified, to any
submodule clone commands.
Modified 'git clone' to pass the `--no-single-branch`, if specified, to any
submodule clone commands.

Signed-off-by: Cole Minnaar cole.minn...@gmail.com
---
 Documentation/git-clone.txt |  5 -
 builtin/clone.c | 15 +--
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 0363d00..f2fd8c8 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -178,7 +178,8 @@ objects from the source repository into a pack in the 
cloned repository.
 
 --depth depth::
Create a 'shallow' clone with a history truncated to the
-   specified number of revisions.
+   specified number of revisions. If `--recursive` was also specified,
+   the depth value will be passed to all submodules within when cloning.
 
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
@@ -192,6 +193,8 @@ objects from the source repository into a pack in the 
cloned repository.
initial cloning.  If the HEAD at the remote did not point at any
branch when `--single-branch` clone was made, no remote-tracking
branch is created.
+   If `--recursive` was also specified, this option will be passed
+   to all submodules when cloning.
 
 --recursive::
 --recurse-submodules::
diff --git a/builtin/clone.c b/builtin/clone.c
index dd4092b..c906d8e 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -48,6 +48,7 @@ static int option_verbosity;
 static int option_progress = -1;
 static struct string_list option_config;
 static struct string_list option_reference;
+statis struct argv_array argv_submodule_cmd = ARGV_ARRAY_INIT;
 
 static int opt_parse_reference(const struct option *opt, const char *arg, int 
unset)
 {
@@ -100,10 +101,6 @@ static struct option builtin_clone_options[] = {
OPT_END()
 };
 
-static const char *argv_submodule[] = {
-   submodule, update, --init, --recursive, NULL
-};
-
 static char *get_repo_path(const char *repo, int *is_bundle)
 {
static char *suffix[] = { /.git, , .git/.git, .git };
@@ -663,8 +660,14 @@ static int checkout(void)
err |= run_hook_le(NULL, post-checkout, sha1_to_hex(null_sha1),
   sha1_to_hex(sha1), 1, NULL);
 
-   if (!err  option_recursive)
-   err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
+   if (!err  option_recursive) {
+   argv_array_pushl(argv_submodule_cmd, submodule, update, 
--init, --recursive, NULL);
+   if (option_depth)
+   argv_array_pushf(argv_submodule_cmd, --depth=%d, 
atoi(option_depth));
+   if (!option_single_branch)
+   argv_array_pushl(argv_submodule_cmd, 
--no-single-branch, NULL);
+   err = run_command_v_opt(argv_submodule_cmd.argv, RUN_GIT_CMD);
+   }
 
return err;
 }
-- 
2.1.0.238.gce1d3a9.dirty

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Fwd: Shallow clone

[Steven Evergreen]

Hi, everyone. I'm trying to perform a shallow clone with visibility of
all remote branches.

git clone REPO --depth 1 --no-single-branch

is consistently giving me

Cloning into 'REPONAME'...
fatal: (null) is unknown object
remote: Total 0 (delta 0), reused 0 (delta 0)
fatal: recursion detected in die handler
remote: aborting due to possible repository corruption on the remote side.
fatal: error in sideband demultiplexer

with 2.0.3 .

Is this command not supported? Have I hit a bug?

Thanks!
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Shallow clone

[Duy Nguyen]

On Tue, Aug 19, 2014 at 6:11 PM, Steven Evergreen
i.stevenevergr...@gmail.com wrote:
 Hi, everyone. I'm trying to perform a shallow clone with visibility of
 all remote branches.

 git clone REPO --depth 1 --no-single-branch

 is consistently giving me

 Cloning into 'REPONAME'...
 fatal: (null) is unknown object
 remote: Total 0 (delta 0), reused 0 (delta 0)
 fatal: recursion detected in die handler
 remote: aborting due to possible repository corruption on the remote side.
 fatal: error in sideband demultiplexer

 with 2.0.3 .

 Is this command not supported? Have I hit a bug?

Yes it's supported and yes I think you hit a bug. Any chance you can
share this repository? If not, perhaps just SHA-1 so we know what the
commit DAG looks like?
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Bug in shallow clone?

[Thomas Kieffer]

Hi there Git developers,

I'm not sure if I found a bug in the command

git clone repo.git cloned_repo --depth 1

I follow these steps:

git init
echo First commit  test.txt
git add -A
git commit -am First commit

echo Second commit  test.txt
git commit -am Second commit

echo Third commit  test.txt
git commit -am Third commit

git clone --bare . ./bare.git

I then clone the bare repository with --depth 1.

git clone file:///path/to/bare.git ./clone --depth 1

It always returns the last two commits. If I specify --depth 2 it 
returns the last 3 commits.


If I use --depth 1 on a Github repository it works as expected.

Am I doing something wrong or is it really a bug?

Kind Regards,

Thomas Kieffer


BTW.: Git is amazing and I love it :)
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug in shallow clone?

[Duy Nguyen]

On Wed, May 28, 2014 at 9:02 PM, Thomas Kieffer thomaskief...@web.de wrote:
 I then clone the bare repository with --depth 1.

 git clone file:///path/to/bare.git ./clone --depth 1

 It always returns the last two commits. If I specify --depth 2 it returns
 the last 3 commits.

 If I use --depth 1 on a Github repository it works as expected.

 Am I doing something wrong or is it really a bug?

Depth calculation has been corrected lately. It depends on your
version, maybe it's older than 1.8.2? If it's the latest, we screwed
something up again..
-- 
Duy
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug in shallow clone?

[Dennis Kaarsemaker]

On wo, 2014-05-28 at 21:16 +0700, Duy Nguyen wrote:
 On Wed, May 28, 2014 at 9:02 PM, Thomas Kieffer thomaskief...@web.de wrote:
  I then clone the bare repository with --depth 1.
 
  git clone file:///path/to/bare.git ./clone --depth 1
 
  It always returns the last two commits. If I specify --depth 2 it returns
  the last 3 commits.
 
  If I use --depth 1 on a Github repository it works as expected.
 
  Am I doing something wrong or is it really a bug?
 
 Depth calculation has been corrected lately. It depends on your
 version, maybe it's older than 1.8.2? If it's the latest, we screwed
 something up again..

2.0.0-rc4 does this correctly.
-- 
Dennis Kaarsemaker
http://www.kaarsemaker.net

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 00/28] First class shallow clone

[Nguyễn Thái Ngọc Duy]

This reroll should fix all comments I have received in v3. I reordered
the shallow checks a bit so in common case it should be as cheap as
a normal fetch or push. See 08/28 and 20/28 for the big picture. I'm not
entirely happy with the hook issue in 20/28, but it looks good enough
for me. There are a few XXXes for further improvement, but I'll keep
them until this lands.

To recap, this series allows you to clone from a shallow repo, push to
or fetch from any shallow repo. Normally it will reject new refs
that introduce new shallow boundaries to your repository, so if you're
in a full clone, it will always stay a full clone. Use fetch
--update-shallow or set receive.shallowupdate to accept those refs.

Nguyễn Thái Ngọc Duy (28):
  transport.h: remove send_pack prototype, already defined in send-pack.h
  Replace struct extra_have_objects with struct sha1_array
  send-pack: forbid pushing from a shallow repository
  clone: prevent --reference to a shallow repository
  Make the sender advertise shallow commits to the receiver
  connect.c: teach get_remote_heads to parse shallow lines
  shallow.c: extend setup_*_shallow() to accept extra shallow commits
  shallow.c: the 8 steps to select new commits for .git/shallow
  shallow.c: steps 6 and 7 to select new commits for .git/shallow
  fetch-pack.c: move shallow update code out of fetch_pack()
  fetch-pack.h: one statement per bitfield declaration
  clone: support remote shallow repository
  fetch: support fetching from a shallow repository
  upload-pack: make sure deepening preserves shallow roots
  fetch: add --update-shallow to accept refs that update .git/shallow
  receive-pack: reorder some code in unpack()
  receive/send-pack: support pushing from a shallow clone
  New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses
  connected.c: add new variant that runs with --shallow-file
  receive-pack: allow pushes that update .git/shallow
  send-pack: support pushing to a shallow clone
  remote-curl: pass ref SHA-1 to fetch-pack as well
  Support shallow fetch/clone over smart-http
  receive-pack: support pushing to a shallow clone via http
  send-pack: support pushing from a shallow clone via http
  clone: use git protocol for cloning shallow repo locally
  prune: clean .git/shallow after pruning objects
  git-clone.txt: remove shallow clone limitations

 Documentation/config.txt  |   4 +
 Documentation/fetch-options.txt   |  14 +-
 Documentation/git-clone.txt   |   7 +-
 Documentation/git-prune.txt   |   2 +
 Documentation/gitremote-helpers.txt   |   7 +
 Documentation/technical/pack-protocol.txt |   7 +-
 builtin/clone.c   |  18 +-
 builtin/fetch-pack.c  |  23 +-
 builtin/fetch.c   |  15 +-
 builtin/gc.c  |   1 +
 builtin/prune.c   |   4 +
 builtin/receive-pack.c| 314 +++
 builtin/send-pack.c   |   9 +-
 cache.h   |   3 +
 commit.h  |  39 ++-
 connect.c |  22 +-
 connected.c   |  42 ++-
 connected.h   |   2 +
 environment.c |   6 +
 fetch-pack.c  | 132 +++-
 fetch-pack.h  |  29 +-
 git.c |   2 +-
 remote-curl.c |  34 ++-
 remote.h  |   9 +-
 send-pack.c   |  27 +-
 send-pack.h   |   2 +-
 shallow.c | 486 +-
 t/t5304-prune.sh  |  10 +
 t/t5536-fetch-shallow.sh (new +x) | 203 +
 t/t5537-push-shallow.sh (new +x)  | 183 +++
 t/t5601-clone.sh  |   7 +
 trace.c   |   2 +-
 transport-helper.c|   6 +
 transport.c   |  25 +-
 transport.h   |  16 +-
 upload-pack.c |   8 +-
 36 files changed, 1555 insertions(+), 165 deletions(-)
 create mode 100755 t/t5536-fetch-shallow.sh
 create mode 100755 t/t5537-push-shallow.sh

-- 
1.8.5.1.25.g8667982

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 17/28] receive/send-pack: support pushing from a shallow clone

[Nguyễn Thái Ngọc Duy]

() */
+   if (!si-nr_ours  !si-nr_theirs)
+   return;
+
+   for (cmd = commands; cmd; cmd = cmd-next) {
+   if (is_null_sha1(cmd-new_sha1))
+   continue;
+   sha1_array_append(ref, cmd-new_sha1);
+   cmd-index = ref-nr - 1;
+   }
+   si-ref = ref;
+
+   ref_status = xmalloc(sizeof(*ref_status) * ref-nr);
+   assign_shallow_commits_to_refs(si, NULL, ref_status);
+   for (cmd = commands; cmd; cmd = cmd-next) {
+   if (is_null_sha1(cmd-new_sha1))
+   continue;
+   if (ref_status[cmd-index]) {
+   cmd-error_string = shallow update not allowed;
+   cmd-skip_update = 1;
+   }
+   }
+   if (alt_shallow_file  *alt_shallow_file) {
+   unlink(alt_shallow_file);
+   alt_shallow_file = NULL;
+   }
+   free(ref_status);
+}
+
 static void report(struct command *commands, const char *unpack_status)
 {
struct command *cmd;
@@ -950,6 +1001,9 @@ int cmd_receive_pack(int argc, const char **argv, const 
char *prefix)
int i;
char *dir = NULL;
struct command *commands;
+   struct sha1_array shallow = SHA1_ARRAY_INIT;
+   struct sha1_array ref = SHA1_ARRAY_INIT;
+   struct shallow_info si;
 
packet_trace_identity(receive-pack);
 
@@ -1006,11 +1060,14 @@ int cmd_receive_pack(int argc, const char **argv, const 
char *prefix)
if (advertise_refs)
return 0;
 
-   if ((commands = read_head_info()) != NULL) {
+   if ((commands = read_head_info(shallow)) != NULL) {
const char *unpack_status = NULL;
 
-   if (!delete_only(commands))
-   unpack_status = unpack_with_sideband();
+   prepare_shallow_info(si, shallow);
+   if (!delete_only(commands)) {
+   unpack_status = unpack_with_sideband(si);
+   update_shallow_info(commands, si, ref);
+   }
execute_commands(commands, unpack_status);
if (pack_lockfile)
unlink_or_warn(pack_lockfile);
@@ -1027,8 +1084,11 @@ int cmd_receive_pack(int argc, const char **argv, const 
char *prefix)
}
if (auto_update_server_info)
update_server_info(0);
+   clear_shallow_info(si);
}
if (use_sideband)
packet_flush(1);
+   sha1_array_clear(shallow);
+   sha1_array_clear(ref);
return 0;
 }
diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index 62cc4d3..ea2ab28 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -208,7 +208,7 @@ int cmd_send_pack(int argc, const char **argv, const char 
*prefix)
(send_all  args.send_mirror))
usage(send_pack_usage);
 
-   if (is_repository_shallow())
+   if (is_repository_shallow()  args.stateless_rpc)
die(attempt to push from a shallow repository);
 
if (remote_name) {
diff --git a/send-pack.c b/send-pack.c
index 14005fa..cd536b4 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -214,6 +214,9 @@ int send_pack(struct send_pack_args *args,
return 0;
}
 
+   if (!args-dry_run)
+   advertise_shallow_grafts(out);
+
/*
 * Finally, tell the other end!
 */
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
new file mode 100755
index 000..650c31a
--- /dev/null
+++ b/t/t5537-push-shallow.sh
@@ -0,0 +1,70 @@
+#!/bin/sh
+
+test_description='push from/to a shallow clone'
+
+. ./test-lib.sh
+
+commit() {
+   echo $1 tracked 
+   git add tracked 
+   git commit -m $1
+}
+
+test_expect_success 'setup' '
+   git config --global transfer.fsckObjects true 
+   commit 1 
+   commit 2 
+   commit 3 
+   commit 4 
+   (
+   git init full-abc 
+   cd full-abc 
+   commit a 
+   commit b 
+   commit c
+   ) 
+   git clone --no-local --depth=2 .git shallow 
+   git --git-dir=shallow/.git log --format=%s actual 
+   cat EOF expect 
+4
+3
+EOF
+   test_cmp expect actual 
+   git clone --no-local --depth=2 full-abc/.git shallow2 
+   git --git-dir=shallow2/.git log --format=%s actual 
+   cat EOF expect 
+c
+b
+EOF
+   test_cmp expect actual
+'
+
+test_expect_success 'push from shallow clone' '
+   (
+   cd shallow 
+   commit 5 
+   git push ../.git +master:refs/remotes/shallow/master
+   ) 
+   git log --format=%s shallow/master actual 
+   git fsck 
+   cat EOF expect 
+5
+4
+3
+2
+1
+EOF
+   test_cmp expect actual
+'
+
+test_expect_success 'push from shallow clone, with grafted roots' '
+   (
+   cd shallow2 
+   test_must_fail git push ../.git +master:refs/remotes/shallow2/master 
2err 
+   grep shallow2/master.*shallow

[PATCH v4 21/28] send-pack: support pushing to a shallow clone

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 builtin/send-pack.c |  4 +++-
 t/t5537-push-shallow.sh | 38 ++
 transport.c |  5 ++---
 3 files changed, 43 insertions(+), 4 deletions(-)

diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index ea2ab28..664dd20 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -101,6 +101,7 @@ int cmd_send_pack(int argc, const char **argv, const char 
*prefix)
int fd[2];
struct child_process *conn;
struct sha1_array extra_have = SHA1_ARRAY_INIT;
+   struct sha1_array shallow = SHA1_ARRAY_INIT;
struct ref *remote_refs, *local_refs;
int ret;
int helper_status = 0;
@@ -232,7 +233,8 @@ int cmd_send_pack(int argc, const char **argv, const char 
*prefix)
args.verbose ? CONNECT_VERBOSE : 0);
}
 
-   get_remote_heads(fd[0], NULL, 0, remote_refs, REF_NORMAL, extra_have, 
NULL);
+   get_remote_heads(fd[0], NULL, 0, remote_refs, REF_NORMAL,
+extra_have, shallow);
 
transport_verify_remote_names(nr_refspecs, refspecs);
 
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
index ff5eb5b..f5c74e6 100755
--- a/t/t5537-push-shallow.sh
+++ b/t/t5537-push-shallow.sh
@@ -82,4 +82,42 @@ EOF
test_cmp expect actual
 '
 
+test_expect_success 'push from shallow to shallow' '
+   (
+   cd shallow 
+   git --git-dir=../shallow2/.git config receive.shallowupdate true 
+   git push ../shallow2/.git +master:refs/remotes/shallow/master 
+   git --git-dir=../shallow2/.git config receive.shallowupdate false
+   ) 
+   (
+   cd shallow2 
+   git log --format=%s shallow/master actual 
+   git fsck 
+   cat EOF expect 
+5
+4
+3
+EOF
+   test_cmp expect actual
+   )
+'
+
+test_expect_success 'push from full to shallow' '
+   ! git --git-dir=shallow2/.git cat-file blob `echo 1|git hash-object 
--stdin` 
+   commit 1 
+   git push shallow2/.git +master:refs/remotes/top/master 
+   (
+   cd shallow2 
+   git log --format=%s top/master actual 
+   git fsck 
+   cat EOF expect 
+1
+4
+3
+EOF
+   test_cmp expect actual 
+   git cat-file blob `echo 1|git hash-object --stdin` /dev/null
+   )
+'
+
 test_done
diff --git a/transport.c b/transport.c
index a09fdb6..d596abb 100644
--- a/transport.c
+++ b/transport.c
@@ -819,11 +819,10 @@ static int git_transport_push(struct transport 
*transport, struct ref *remote_re
struct ref *tmp_refs;
connect_setup(transport, 1, 0);
 
-   get_remote_heads(data-fd[0], NULL, 0, tmp_refs, REF_NORMAL, 
NULL, NULL);
+   get_remote_heads(data-fd[0], NULL, 0, tmp_refs, REF_NORMAL,
+NULL, data-shallow);
data-got_remote_heads = 1;
}
-   if (data-shallow.nr)
-   die(pushing to a shallow repository is not supported);
 
memset(args, 0, sizeof(args));
args.send_mirror = !!(flags  TRANSPORT_PUSH_MIRROR);
-- 
1.8.5.1.25.g8667982

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 24/28] receive-pack: support pushing to a shallow clone via http

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 builtin/receive-pack.c  |  3 ---
 t/t5537-push-shallow.sh | 35 +++
 2 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 5c85bb4..78fe8ee 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1179,9 +1179,6 @@ int cmd_receive_pack(int argc, const char **argv, const 
char *prefix)
if (!enter_repo(dir, 0))
die('%s' does not appear to be a git repository, dir);
 
-   if (is_repository_shallow()  stateless_rpc)
-   die(attempt to push into a shallow repository);
-
git_config(receive_pack_config, NULL);
 
if (0 = transfer_unpack_limit)
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
index f5c74e6..866621a 100755
--- a/t/t5537-push-shallow.sh
+++ b/t/t5537-push-shallow.sh
@@ -16,6 +16,7 @@ test_expect_success 'setup' '
commit 2 
commit 3 
commit 4 
+   git clone . full 
(
git init full-abc 
cd full-abc 
@@ -120,4 +121,38 @@ EOF
)
 '
 
+if test -n $NO_CURL -o -z $GIT_TEST_HTTPD; then
+   say 'skipping remaining tests, git built without http support'
+   test_done
+fi
+
+LIB_HTTPD_PORT=${LIB_HTTPD_PORT-'5537'}
+. $TEST_DIRECTORY/lib-httpd.sh
+start_httpd
+
+test_expect_success 'push to shallow repo via http' '
+   git clone --bare --no-local shallow 
$HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   (
+   cd $HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   git config http.receivepack true
+   ) 
+   (
+   cd full 
+   commit 9 
+   git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master
+   ) 
+   (
+   cd $HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   git fsck 
+   git log --format=%s top/master actual 
+   cat EOF expect 
+9
+4
+3
+EOF
+   test_cmp expect actual
+   )
+'
+
+stop_httpd
 test_done
-- 
1.8.5.1.25.g8667982

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 25/28] send-pack: support pushing from a shallow clone via http

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 builtin/send-pack.c |  3 ---
 send-pack.c | 19 +--
 t/t5537-push-shallow.sh | 25 +
 3 files changed, 42 insertions(+), 5 deletions(-)

diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index 664dd20..cc25744 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -209,9 +209,6 @@ int cmd_send_pack(int argc, const char **argv, const char 
*prefix)
(send_all  args.send_mirror))
usage(send_pack_usage);
 
-   if (is_repository_shallow()  args.stateless_rpc)
-   die(attempt to push from a shallow repository);
-
if (remote_name) {
remote = remote_get(remote_name);
if (!remote_has_url(remote, dest)) {
diff --git a/send-pack.c b/send-pack.c
index cd536b4..848d15e 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -175,6 +175,21 @@ static int sideband_demux(int in, int out, void *data)
return ret;
 }
 
+static int advertise_shallow_grafts_cb(const struct commit_graft *graft, void 
*cb)
+{
+   struct strbuf *sb = cb;
+   if (graft-nr_parent == -1)
+   packet_buf_write(sb, shallow %s\n, sha1_to_hex(graft-sha1));
+   return 0;
+}
+
+void advertise_shallow_grafts_buf(struct strbuf *sb)
+{
+   if (!is_repository_shallow())
+   return;
+   for_each_commit_graft(advertise_shallow_grafts_cb, sb);
+}
+
 int send_pack(struct send_pack_args *args,
  int fd[], struct child_process *conn,
  struct ref *remote_refs,
@@ -215,7 +230,7 @@ int send_pack(struct send_pack_args *args,
}
 
if (!args-dry_run)
-   advertise_shallow_grafts(out);
+   advertise_shallow_grafts_buf(req_buf);
 
/*
 * Finally, tell the other end!
@@ -276,7 +291,7 @@ int send_pack(struct send_pack_args *args,
}
 
if (args-stateless_rpc) {
-   if (!args-dry_run  cmds_sent) {
+   if (!args-dry_run  (cmds_sent || is_repository_shallow())) {
packet_buf_flush(req_buf);
send_sideband(out, -1, req_buf.buf, req_buf.len, 
LARGE_PACKET_MAX);
}
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
index 866621a..0a6e40f 100755
--- a/t/t5537-push-shallow.sh
+++ b/t/t5537-push-shallow.sh
@@ -154,5 +154,30 @@ EOF
)
 '
 
+test_expect_success 'push from shallow repo via http' '
+   mv $HTTPD_DOCUMENT_ROOT_PATH/repo.git shallow-upstream.git 
+   git clone --bare --no-local full $HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   (
+   cd $HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   git config http.receivepack true
+   ) 
+   commit 10 
+   git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master 
+   (
+   cd $HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   git fsck 
+   git log --format=%s top/master actual 
+   cat EOF expect 
+10
+1
+4
+3
+2
+1
+EOF
+   test_cmp expect actual
+   )
+'
+
 stop_httpd
 test_done
-- 
1.8.5.1.25.g8667982

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 28/28] git-clone.txt: remove shallow clone limitations

[Nguyễn Thái Ngọc Duy]

Now that git supports data transfer from or to a shallow clone, these
limitations are not true anymore.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 Documentation/git-clone.txt | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 450f158..4987857 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -181,12 +181,7 @@ objects from the source repository into a pack in the 
cloned repository.
 
 --depth depth::
Create a 'shallow' clone with a history truncated to the
-   specified number of revisions.  A shallow repository has a
-   number of limitations (you cannot clone or fetch from
-   it, nor push from nor into it), but is adequate if you
-   are only interested in the recent history of a large project
-   with a long history, and would want to send in fixes
-   as patches.
+   specified number of revisions.
 
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
-- 
1.8.5.1.25.g8667982

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 17/28] Support pushing from a shallow clone

[Eric Sunshine]

On Sun, Nov 24, 2013 at 10:55 PM, Nguyễn Thái Ngọc Duy
pclo...@gmail.com wrote:
 Pushing from a shallow clone using today's send-pack and receive-pack
 may work, if the transferred pack does not end up at any graft
 points. If it does, recent receive-pack that does connectivity check
 will reject the push. If receive-pack is old and does not have the
 connectivity check, the upstream repo becomes corrupt.

 The pack protocol is updated and send-pack now sends all shallow
 grafts before it sends the commands, if the repo is shallow. This
 protocol extension will break current receive-pack, which is intended,
 mostly to stop corrupting the upstream repo.

 Changes on the receiver are similar to what has been done in
 fetch-pack, i.e. filter out refs that require new shallow roots then
 go along as usual.

 Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
 ---
 diff --git a/Documentation/technical/pack-protocol.txt 
 b/Documentation/technical/pack-protocol.txt
 index eb8edd1..c73b62f 100644
 --- a/Documentation/technical/pack-protocol.txt
 +++ b/Documentation/technical/pack-protocol.txt
 @@ -43,6 +43,9 @@ static int fix_thin = 1;
  static const char *head_name;
  static void *head_name_to_free;
  static int sent_capabilities;
 +static int shallow_push;
 +static const char* alternate_shallow_file;

s/char\* /char */

 +static struct extra_have_objects shallow;

  static enum deny_action parse_deny_action(const char *var, const char *value)
  {
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 00/28] First class shallow clone

[Nguyễn Thái Ngọc Duy]

Compared to v2 [1], v3 grows a bit. The biggest difference is
.git/shallow is not updated by default (except when you clone from
a shallow repository). When you send something, the safe refs that
do not need new shallow roots are accepted at the receiver end, the
others rejected.

To accept those other refs, either use fetch --update-shallow or
enable receive.shallowupdate on the receiver side of a push.

Filtering safe refs requires walking through some commits, so
it'll be more expensive than normal full clones. This is especially
true for the receiver of a push (see 07/28 and 17/28). I envision
shallow repos are used as upstream to archive old history, so this is
not a good news. Commit cache (or pack v4) might help. We might even
be able to move some work from receive-pack to send-pack to reduce
server load..

[1] http://mid.gmane.org/1374314290-5976-1-git-send-email-pclo...@gmail.com

Nguyễn Thái Ngọc Duy (28):
  transport.h: remove send_pack prototype, already defined in send-pack.h
  send-pack: forbid pushing from a shallow repository
  clone: prevent --reference to a shallow repository
  update-server-info: do not publish shallow clones

   This part is just cleanup.

  Advertise shallow graft information on the server end
  connect.c: teach get_remote_heads to parse shallow lines
  shallow.c: add remove_reachable_shallow_points()
  shallow.c: add mark_new_shallow_refs()
  shallow.c: extend setup_*_shallow() to accept extra shallow points
  fetch-pack.c: move shallow update code out of fetch_pack()
  fetch-pack.h: one statement per bitfield declaration
  clone: support remote shallow repository
  fetch: support fetching from a shallow repository
  upload-pack: make sure deepening preserves shallow roots
  fetch: add --update-shallow to get refs that require updating .git/shallow

   Basic shallow fetch/clone support on git protocol

  receive-pack: reorder some code in unpack()
  Support pushing from a shallow clone
  New var GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses
  connected.c: add new variant that runs with --shallow-file
  receive-pack: allow pushing with new shallow roots
  send-pack: support pushing to a shallow clone
  remote-curl: pass ref SHA-1 to fetch-pack as well
  
   Push support

  Support fetch/clone over http
  receive-pack: support pushing to a shallow clone via http
  send-pack: support pushing from a shallow clone via http

   smart-http support

  git-clone.txt: remove shallow clone limitations
  clone: use git protocol for cloning shallow repo locally
  prune: clean .git/shallow after pruning objects

   miscellaneous

 Documentation/config.txt  |   4 +
 Documentation/fetch-options.txt   |  14 +-
 Documentation/git-clone.txt   |   7 +-
 Documentation/gitremote-helpers.txt   |   7 +
 Documentation/technical/pack-protocol.txt |   7 +-
 builtin/clone.c   |  18 +-
 builtin/fetch-pack.c  |  23 +-
 builtin/fetch.c   |  15 +-
 builtin/gc.c  |   1 +
 builtin/prune.c   |   4 +
 builtin/receive-pack.c| 248 +
 builtin/send-pack.c   |   5 +-
 cache.h   |   1 +
 commit.h  |  19 +-
 connect.c |  14 +-
 connected.c   |  42 +++-
 connected.h   |   2 +
 environment.c |   6 +
 fetch-pack.c  | 132 ++--
 fetch-pack.h  |  29 +--
 git.c |   2 +-
 remote-curl.c |  33 ++-
 remote.h  |   5 +-
 send-pack.c   |  20 +-
 server-info.c |   9 +
 shallow.c | 348 +-
 t/t5536-fetch-shallow.sh (new +x) | 193 +
 t/t5537-push-shallow.sh (new +x)  | 184 
 t/t5601-clone.sh  |   7 +
 transport-helper.c|   6 +
 transport.c   |  22 +-
 transport.h   |  16 +-
 upload-pack.c |   8 +-
 33 files changed, 1323 insertions(+), 128 deletions(-)
 create mode 100755 t/t5536-fetch-shallow.sh
 create mode 100755 t/t5537-push-shallow.sh

-- 
1.8.2.83.gc99314b

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 17/28] Support pushing from a shallow clone

[Nguyễn Thái Ngọc Duy]

Pushing from a shallow clone using today's send-pack and receive-pack
may work, if the transferred pack does not end up at any graft
points. If it does, recent receive-pack that does connectivity check
will reject the push. If receive-pack is old and does not have the
connectivity check, the upstream repo becomes corrupt.

The pack protocol is updated and send-pack now sends all shallow
grafts before it sends the commands, if the repo is shallow. This
protocol extension will break current receive-pack, which is intended,
mostly to stop corrupting the upstream repo.

Changes on the receiver are similar to what has been done in
fetch-pack, i.e. filter out refs that require new shallow roots then
go along as usual.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 Documentation/technical/pack-protocol.txt |  4 +-
 builtin/receive-pack.c| 68 +-
 builtin/send-pack.c   |  2 +-
 send-pack.c   |  3 ++
 t/t5537-push-shallow.sh (new +x)  | 70 +++
 5 files changed, 143 insertions(+), 4 deletions(-)
 create mode 100755 t/t5537-push-shallow.sh

diff --git a/Documentation/technical/pack-protocol.txt 
b/Documentation/technical/pack-protocol.txt
index eb8edd1..c73b62f 100644
--- a/Documentation/technical/pack-protocol.txt
+++ b/Documentation/technical/pack-protocol.txt
@@ -464,7 +464,9 @@ contain all the objects that the server will need to 
complete the new
 references.
 
 
-  update-request=  command-list [pack-file]
+  update-request=  *shallow command-list [pack-file]
+
+  shallow   =  PKT-LINE(shallow SP obj-id)
 
   command-list  =  PKT-LINE(command NUL capability-list LF)
   *PKT-LINE(command LF)
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 22e162d..254feff 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -43,6 +43,9 @@ static int fix_thin = 1;
 static const char *head_name;
 static void *head_name_to_free;
 static int sent_capabilities;
+static int shallow_push;
+static const char* alternate_shallow_file;
+static struct extra_have_objects shallow;
 
 static enum deny_action parse_deny_action(const char *var, const char *value)
 {
@@ -189,6 +192,7 @@ struct command {
const char *error_string;
unsigned int skip_update:1,
 did_not_exist:1;
+   int index;
unsigned char old_sha1[20];
unsigned char new_sha1[20];
char ref_name[FLEX_ARRAY]; /* more */
@@ -687,7 +691,7 @@ static int iterate_receive_command_list(void *cb_data, 
unsigned char sha1[20])
struct command *cmd = *cmd_list;
 
while (cmd) {
-   if (!is_null_sha1(cmd-new_sha1)) {
+   if (!is_null_sha1(cmd-new_sha1)  !cmd-skip_update) {
hashcpy(sha1, cmd-new_sha1);
*cmd_list = cmd-next;
return 0;
@@ -712,8 +716,25 @@ static void reject_updates_to_hidden(struct command 
*commands)
}
 }
 
+static void filter_shallow_refs(struct command *commands,
+   const int *ref_status, int nr_ref)
+{
+   struct command *cmd;
+   for (cmd = commands; cmd; cmd = cmd-next)
+   if (!is_null_sha1(cmd-new_sha1)  ref_status[cmd-index]) {
+   cmd-error_string = shallow update not allowed;
+   cmd-skip_update = 1;
+   }
+   if (*alternate_shallow_file) {
+   unlink(alternate_shallow_file);
+   alternate_shallow_file = NULL;
+   }
+}
+
 static void execute_commands(struct command *commands, const char 
*unpacker_error)
 {
+   int *ref_status = NULL;
+   struct extra_have_objects ref;
struct command *cmd;
unsigned char sha1[20];
 
@@ -723,6 +744,20 @@ static void execute_commands(struct command *commands, 
const char *unpacker_erro
return;
}
 
+   memset(ref, 0, sizeof(ref));
+   if (shallow_push) {
+   int i;
+   for (i = 0, cmd = commands; cmd; i++, cmd = cmd-next)
+   if (!is_null_sha1(cmd-new_sha1)) {
+   add_extra_have(ref, cmd-new_sha1);
+   cmd-index = i;
+   }
+
+   ref_status = xmalloc(sizeof(*ref_status) * ref.nr);
+   if (mark_new_shallow_refs(ref, ref_status, NULL, shallow))
+   filter_shallow_refs(commands, ref_status, ref.nr);
+   }
+
cmd = commands;
if (check_everything_connected(iterate_receive_command_list,
   0, cmd))
@@ -752,6 +787,8 @@ static void execute_commands(struct command *commands, 
const char *unpacker_erro
 
cmd-error_string = update(cmd);
}
+   free(ref.array);
+   free(ref_status);
 }
 
 static struct command *read_head_info

[PATCH v3 21/28] send-pack: support pushing to a shallow clone

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 builtin/send-pack.c |  5 -
 t/t5537-push-shallow.sh | 38 ++
 transport.c |  5 ++---
 3 files changed, 44 insertions(+), 4 deletions(-)

diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index 5db1311..0031165 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -100,6 +100,7 @@ int cmd_send_pack(int argc, const char **argv, const char 
*prefix)
int fd[2];
struct child_process *conn;
struct extra_have_objects extra_have;
+   struct extra_have_objects shallow;
struct ref *remote_refs, *local_refs;
int ret;
int helper_status = 0;
@@ -232,8 +233,10 @@ int cmd_send_pack(int argc, const char **argv, const char 
*prefix)
}
 
memset(extra_have, 0, sizeof(extra_have));
+   memset(shallow, 0, sizeof(shallow));
 
-   get_remote_heads(fd[0], NULL, 0, remote_refs, REF_NORMAL, extra_have, 
NULL);
+   get_remote_heads(fd[0], NULL, 0, remote_refs, REF_NORMAL,
+extra_have, shallow);
 
transport_verify_remote_names(nr_refspecs, refspecs);
 
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
index 0084a31..ccb41b6 100755
--- a/t/t5537-push-shallow.sh
+++ b/t/t5537-push-shallow.sh
@@ -83,4 +83,42 @@ EOF
git config receive.shallowupdate false
 '
 
+test_expect_success 'push from shallow to shallow' '
+   (
+   cd shallow 
+   git --git-dir=../shallow2/.git config receive.shallowupdate true 
+   git push ../shallow2/.git +master:refs/remotes/shallow/master 
+   git --git-dir=../shallow2/.git config receive.shallowupdate false
+   ) 
+   (
+   cd shallow2 
+   git log --format=%s shallow/master actual 
+   git fsck 
+   cat EOF expect 
+5
+4
+3
+EOF
+   test_cmp expect actual
+   )
+'
+
+test_expect_success 'push from full to shallow' '
+   ! git --git-dir=shallow2/.git cat-file blob `echo 1|git hash-object 
--stdin` 
+   commit 1 
+   git push shallow2/.git +master:refs/remotes/top/master 
+   (
+   cd shallow2 
+   git log --format=%s top/master actual 
+   git fsck 
+   cat EOF expect 
+1
+4
+3
+EOF
+   test_cmp expect actual 
+   git cat-file blob `echo 1|git hash-object --stdin` /dev/null
+   )
+'
+
 test_done
diff --git a/transport.c b/transport.c
index c0be6b1..6a3fe9b 100644
--- a/transport.c
+++ b/transport.c
@@ -818,11 +818,10 @@ static int git_transport_push(struct transport 
*transport, struct ref *remote_re
struct ref *tmp_refs;
connect_setup(transport, 1, 0);
 
-   get_remote_heads(data-fd[0], NULL, 0, tmp_refs, REF_NORMAL, 
NULL, NULL);
+   get_remote_heads(data-fd[0], NULL, 0, tmp_refs, REF_NORMAL,
+NULL, data-shallow);
data-got_remote_heads = 1;
}
-   if (data-shallow.nr)
-   die(pushing to a shallow repository is not supported);
 
memset(args, 0, sizeof(args));
args.send_mirror = !!(flags  TRANSPORT_PUSH_MIRROR);
-- 
1.8.2.83.gc99314b

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 25/28] send-pack: support pushing from a shallow clone via http

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 builtin/send-pack.c |  3 ---
 send-pack.c | 19 +--
 t/t5537-push-shallow.sh | 25 +
 3 files changed, 42 insertions(+), 5 deletions(-)

diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index 0031165..966b45e 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -208,9 +208,6 @@ int cmd_send_pack(int argc, const char **argv, const char 
*prefix)
(send_all  args.send_mirror))
usage(send_pack_usage);
 
-   if (is_repository_shallow()  args.stateless_rpc)
-   die(attempt to push from a shallow repository);
-
if (remote_name) {
remote = remote_get(remote_name);
if (!remote_has_url(remote, dest)) {
diff --git a/send-pack.c b/send-pack.c
index 8b5571c..13167ea 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -174,6 +174,21 @@ static int sideband_demux(int in, int out, void *data)
return ret;
 }
 
+static int advertise_shallow_grafts_cb(const struct commit_graft *graft, void 
*cb)
+{
+   struct strbuf *sb = cb;
+   if (graft-nr_parent == -1)
+   packet_buf_write(sb, shallow %s\n, sha1_to_hex(graft-sha1));
+   return 0;
+}
+
+void advertise_shallow_grafts_buf(struct strbuf *sb)
+{
+   if (!is_repository_shallow())
+   return;
+   for_each_commit_graft(advertise_shallow_grafts_cb, sb);
+}
+
 int send_pack(struct send_pack_args *args,
  int fd[], struct child_process *conn,
  struct ref *remote_refs,
@@ -214,7 +229,7 @@ int send_pack(struct send_pack_args *args,
}
 
if (!args-dry_run)
-   advertise_shallow_grafts(out);
+   advertise_shallow_grafts_buf(req_buf);
 
/*
 * Finally, tell the other end!
@@ -275,7 +290,7 @@ int send_pack(struct send_pack_args *args,
}
 
if (args-stateless_rpc) {
-   if (!args-dry_run  cmds_sent) {
+   if (!args-dry_run  (cmds_sent || is_repository_shallow())) {
packet_buf_flush(req_buf);
send_sideband(out, -1, req_buf.buf, req_buf.len, 
LARGE_PACKET_MAX);
}
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
index d252a78..f8200f7 100755
--- a/t/t5537-push-shallow.sh
+++ b/t/t5537-push-shallow.sh
@@ -155,5 +155,30 @@ EOF
)
 '
 
+test_expect_success 'push from shallow repo via http' '
+   mv $HTTPD_DOCUMENT_ROOT_PATH/repo.git shallow-upstream.git 
+   git clone --bare --no-local full $HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   (
+   cd $HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   git config http.receivepack true
+   ) 
+   commit 10 
+   git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master 
+   (
+   cd $HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   git fsck 
+   git log --format=%s top/master actual 
+   cat EOF expect 
+10
+1
+4
+3
+2
+1
+EOF
+   test_cmp expect actual
+   )
+'
+
 stop_httpd
 test_done
-- 
1.8.2.83.gc99314b

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 24/28] receive-pack: support pushing to a shallow clone via http

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 builtin/receive-pack.c  |  3 ---
 t/t5537-push-shallow.sh | 35 +++
 2 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 366ecde..ec681ba 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1157,9 +1157,6 @@ int cmd_receive_pack(int argc, const char **argv, const 
char *prefix)
if (!enter_repo(dir, 0))
die('%s' does not appear to be a git repository, dir);
 
-   if (is_repository_shallow()  stateless_rpc)
-   die(attempt to push into a shallow repository);
-
git_config(receive_pack_config, NULL);
 
if (0 = transfer_unpack_limit)
diff --git a/t/t5537-push-shallow.sh b/t/t5537-push-shallow.sh
index ccb41b6..d252a78 100755
--- a/t/t5537-push-shallow.sh
+++ b/t/t5537-push-shallow.sh
@@ -16,6 +16,7 @@ test_expect_success 'setup' '
commit 2 
commit 3 
commit 4 
+   git clone . full 
(
git init full-abc 
cd full-abc 
@@ -121,4 +122,38 @@ EOF
)
 '
 
+if test -n $NO_CURL -o -z $GIT_TEST_HTTPD; then
+   say 'skipping remaining tests, git built without http support'
+   test_done
+fi
+
+LIB_HTTPD_PORT=${LIB_HTTPD_PORT-'5537'}
+. $TEST_DIRECTORY/lib-httpd.sh
+start_httpd
+
+test_expect_success 'push to shallow repo via http' '
+   git clone --bare --no-local shallow 
$HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   (
+   cd $HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   git config http.receivepack true
+   ) 
+   (
+   cd full 
+   commit 9 
+   git push $HTTPD_URL/smart/repo.git +master:refs/remotes/top/master
+   ) 
+   (
+   cd $HTTPD_DOCUMENT_ROOT_PATH/repo.git 
+   git fsck 
+   git log --format=%s top/master actual 
+   cat EOF expect 
+9
+4
+3
+EOF
+   test_cmp expect actual
+   )
+'
+
+stop_httpd
 test_done
-- 
1.8.2.83.gc99314b

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 26/28] git-clone.txt: remove shallow clone limitations

[Nguyễn Thái Ngọc Duy]

Now that git supports push/pull from/to a shallow clone, these
limitations are not true anymore.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 Documentation/git-clone.txt | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 450f158..4987857 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -181,12 +181,7 @@ objects from the source repository into a pack in the 
cloned repository.
 
 --depth depth::
Create a 'shallow' clone with a history truncated to the
-   specified number of revisions.  A shallow repository has a
-   number of limitations (you cannot clone or fetch from
-   it, nor push from nor into it), but is adequate if you
-   are only interested in the recent history of a large project
-   with a long history, and would want to send in fixes
-   as patches.
+   specified number of revisions.
 
 --[no-]single-branch::
Clone only the history leading to the tip of a single branch,
-- 
1.8.2.83.gc99314b

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Bug] git show crashes with deepened shallow clone

[Stefan Näwe]

Just a quick report since I don't have time to bisect now (will do later
if no other gitster is faster...)

When I execute the below script 'git show' crashes. 'git log --oneline -2' gives
for example:

  3808bade5b76c4663ac4a3f751dc9f1ed0b08f2e three
  error: Could not read 1e8777edeb2b7e757f74c789e679fc6c71073897
  fatal: Failed to traverse parents of commit 
0aa4ef86004f5bb29f67e971d7963f5cf430c668

gdb backtrace of one run is attached.
It happens on 32-bit Debian (5.0.10), 64-bit openSUSE (12.2), and Windows XP 
with git 1.8.4
on all systems.

The help of 'git fetch' says:

  --depth=depth

  Deepen or shorten the history of a shallow repository created by git 
clone with
  --depth=depth option (see git-clone(1)) to the specified number of 
commits from
  the tip of each remote branch history. Tags for the deepened commits are 
not fetched.
---^

But that's not true. The tag 'two' (from the script below) gets fetched when
deepening the repository.



---8---8---8---8---8---8---8---8---8---8---8---8---8---8---8
git init shallow-test 
(cd shallow-test 
echo foo  file 
git add file 
git commit -mone
git tag -mone one 
echo bar  file 
git commit -mtwo file
git tag -mtwo two 
echo baz  file 
git commit -mthree file
git tag -mthree three ) 
git clone --depth=1 -b three file://`pwd`/shallow-test shallow-test-clone 
(cd shallow-test-clone 
git fetch --depth=2 
git log --oneline -2 ;
git show two)
---8---8---8---8---8---8---8---8---8---8---8---8---8---8---8


Stefan
-- 

/dev/random says: The cost of feathers has risen... Now even DOWN is up!
python -c print 
'73746566616e2e6e616577654061746c61732d656c656b74726f6e696b2e636f6d'.decode('hex')
Core was generated by `/home/naewe/src/git/git show two'.
Program terminated with signal 11, Segmentation fault.
[New process 13783]
#0  read_object_with_reference (sha1=0x4 Address 0x4 out of bounds, 
required_type_name=0x817ca97 tree, size=0xbfdfe3d8, actual_sha1_return=0x0)
at cache.h:697
697 memcpy(sha_dst, sha_src, 20);
#0  read_object_with_reference (sha1=0x4 Address 0x4 out of bounds, 
required_type_name=0x817ca97 tree, size=0xbfdfe3d8, actual_sha1_return=0x0)
at cache.h:697
type = -1075846344
required_type = OBJ_TREE
buffer = (void *) 0x0
isize = value optimized out
actual_sha1 = \fãß¿\000\000\000\000¸âß¿±n\022\b¼`S\t
#1  0x081392bf in diff_tree_sha1 (old=0x4 Address 0x4 out of bounds, 
new=0x95429cc A1þM3Í\205Ú\200Zɦi|\QÉ\023\210\034, base=0x8155ab9 , 
opt=0xbfdfe7e4) at tree-diff.c:277
tree1 = (void *) 0x954a9d0
tree2 = value optimized out
t1 = {
  buffer = 0x1, 
  entry = {
sha1 = 0x0, 
path = 0x0, 
mode = 3219121144
  }, 
  size = 135386226
}
t2 = {
  buffer = 0xbfdfe3e8, 
  entry = {
sha1 = 0x80c44cc é1ÿÿÿë\r, '\220' repeats 13 times, 
U\211åW\211×V\211ÎS\203ì\034ÇEð, 
path = 0x816d329 Could not read %s, 
mode = 135991090
  }, 
  size = 3219121108
}
size1 = value optimized out
size2 = value optimized out
retval = value optimized out
#2  0x080f12d9 in log_tree_commit (opt=0xbfdfe530, commit=0x9536088) at 
log-tree.c:778
log = {
  commit = 0x9536088, 
  parent = 0x0
}
shown = 0
#3  0x08082dea in cmd_log_walk (rev=0xbfdfe530) at builtin/log.c:342
commit = (struct commit *) 0x9536088
saved_nrl = 0
saved_dcctc = 0
#4  0x080835cd in cmd_show (argc=2, argv=0xbfdfec28, prefix=0x0) at 
builtin/log.c:566
o = (struct object *) 0x9536088
name = 0x952c390 two
rev = {
  commits = 0x0, 
  pending = {
nr = 0, 
alloc = 0, 
objects = 0x0
  }, 
  boundary_commits = {
nr = 0, 
alloc = 0, 
objects = 0x0
  }, 
  cmdline = {
nr = 1, 
alloc = 24, 
rev = 0x952c9f8
  }, 
  prefix = 0x0, 
  def = 0x815364c HEAD, 
  prune_data = {
raw = 0x0, 
nr = 0, 
has_wildcard = 0, 
recursive = 0, 
max_depth = 0, 
items = 0x0
  }, 
  sort_order = REV_SORT_IN_GRAPH_ORDER, 
  early_output = 0, 
  ignore_missing = 0, 
  dense = 1, 
  prune = 0, 
  no_walk = 1, 
  show_all = 0, 
  remove_empty_trees = 0, 
  simplify_history = 1, 
  topo_order = 0, 
  simplify_merges = 0, 
  simplify_by_decoration = 0, 
  tag_objects = 0, 
  tree_objects = 0, 
  blob_objects = 0, 
  verify_objects = 0, 
  edge_hint = 0, 
  limited = 0, 
  unpacked = 0, 
  boundary = 0, 
  count = 0, 
  left_right = 0, 
  left_only = 0, 
  right_only = 0, 
  rewrite_parents = 0, 
  print_parents = 0, 
  show_source = 0, 
  show_decorations = 0, 
  reverse = 0, 
  reverse_output_stage = 0, 
  cherry_pick = 0, 
  cherry_mark = 0, 
  bisect = 0, 
  ancestry_path = 0, 
  first_parent_only = 0, 
  line_level_traverse = 0, 
  diff = 1, 
  full_diff = 0, 
  show_root_diff = 1, 
  no_commit_id = 0, 
  verbose_header = 1, 
  

Re: [Bug] git show crashes with deepened shallow clone

[Stefan Näwe]

Am 25.09.2013 16:47, schrieb Stefan Näwe:
 Just a quick report since I don't have time to bisect now (will do later
 if no other gitster is faster...)

Seems to be somewhere between v1.8.3.1 (OK) and v1.8.3.2 (not OK) !!


 
 When I execute the below script 'git show' crashes. 'git log --oneline -2' 
 gives
 for example:
 
   3808bade5b76c4663ac4a3f751dc9f1ed0b08f2e three
   error: Could not read 1e8777edeb2b7e757f74c789e679fc6c71073897
   fatal: Failed to traverse parents of commit 
 0aa4ef86004f5bb29f67e971d7963f5cf430c668
 
 gdb backtrace of one run is attached.
 It happens on 32-bit Debian (5.0.10), 64-bit openSUSE (12.2), and Windows XP 
 with git 1.8.4
 on all systems.
 
 The help of 'git fetch' says:
 
   --depth=depth
 
   Deepen or shorten the history of a shallow repository created by git 
 clone with
   --depth=depth option (see git-clone(1)) to the specified number of 
 commits from
   the tip of each remote branch history. Tags for the deepened commits 
 are not fetched.
 ---^
 
 But that's not true. The tag 'two' (from the script below) gets fetched when
 deepening the repository.

v1.8.3.1 fetches the tag also.


Stefan
-- 

/dev/random says: Pobody's Nerfect!
python -c print 
'73746566616e2e6e616577654061746c61732d656c656b74726f6e696b2e636f6d'.decode('hex')
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug] git show crashes with deepened shallow clone

[Stefan Näwe]

Am 25.09.2013 16:47, schrieb Stefan Näwe:
 Just a quick report since I don't have time to bisect now (will do later
 if no other gitster is faster...)

I'd marry 'git bisect' if I wasn't already... ;-)

This is what it gives me if I use my script (slightly modified to also run make)
with 'git bisect run':

6035d6aad8ca11954c0d7821f6f3e7c047039c8f is the first bad commit
commit 6035d6aad8ca11954c0d7821f6f3e7c047039c8f
Author: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Date:   Sun May 26 08:16:15 2013 +0700

fetch-pack: prepare updated shallow file before fetching the pack

index-pack --strict looks up and follows parent commits. If shallow
information is not ready by the time index-pack is run, index-pack may
be led to non-existent objects. Make fetch-pack save shallow file to
disk before invoking index-pack.

git learns new global option --shallow-file to pass on the alternate
shallow file path. Undocumented (and not even support --shallow-file=
syntax) because it's unlikely to be used again elsewhere.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com

:100644 100644 67bd5091be0b34bfea075cd60281d22cf5b34768 
6e9c7cd9d5da7d24d4810b36039681f184325932 M  commit.h
:100644 100644 f156dd4fac30cda4e09c508b7091cdb8d96917e2 
6b5467c6dec9645f53d83cdad2467a158db622c0 M  fetch-pack.c
:100644 100644 1ada169d5cff3051effee33c6f9ba5b9be15b2e6 
88eef5a7cc6d36f6e17f4855945116dd6f1b0681 M  git.c
:100644 100644 6be915f38f1fe8dbe0a22c4cd8ae2569331f483f 
cbe2526d8c2b2643957eea2729a16269a7a74fab M  shallow.c
:04 04 5333beeb4db3fc37c37e5a4d03816c4750ce6b28 
3b0fb999b8655155cba24e2d09ebff29058d29d7 M  t
bisect run success


Stefan
-- 

/dev/random says: Answers: $1 * Correct answers: $5 * Dumb looks: Free! *
python -c print 
'73746566616e2e6e616577654061746c61732d656c656b74726f6e696b2e636f6d'.decode('hex')
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug] git show crashes with deepened shallow clone

[Stefan Näwe]

Stefan Näwe stefan.naewe at atlas-elektronik.com writes:

 Am 25.09.2013 16:47, schrieb Stefan Näwe:
 This is what it gives me if I use my script (slightly modified 
 to also run make) with 'git bisect run':
 
 6035d6aad8ca11954c0d7821f6f3e7c047039c8f is the first bad commit

And to answer myself once more:

That's fixed in

   6da8bdc fetch-pack: do not remove .git/shallow file when --depth is not 
specified

Sorry for all the noise.

Stefan

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug] git show crashes with deepened shallow clone

[Jonathan Nieder]

Stefan Näwe wrote:
 Stefan Näwe stefan.naewe at atlas-elektronik.com writes:

 6035d6aad8ca11954c0d7821f6f3e7c047039c8f is the first bad commit

 And to answer myself once more:

 That's fixed in

6da8bdc fetch-pack: do not remove .git/shallow file when --depth is not 
 specified

 Sorry for all the noise.

No problem.

It's probably worth releasing 1.8.4.1 soon.  Thanks for a reminder.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Why can't I push from a shallow clone?

[Fredrik Gustafsson]

On Thu, Jul 25, 2013 at 07:33:16PM -0700, Gulshan Singh wrote:
 I've been trying to figure out why I can't push from a shallow clone
 (using --depth) to a repository. I've made simple examples where it
 works, but I've read that in doesn't work in every case. However, I
 can't come up with a case where it doesn't work. Googling gives this
 answer: 
 http://stackoverflow.com/questions/6900103/why-cant-i-push-from-a-shallow-clone,
 but I don't completely understand the explanation, so I was hoping
 someone could explain it.

I can't explain it better than what Junio did in the link you just
provide. However there's ongoing work to allow shallow clones to be able
to push. You can read about it here:
http://thread.gmane.org/gmane.comp.version-control.git/230612/focus=230878

-- 
Med vänliga hälsningar
Fredrik Gustafsson

tel: 0733-608274
e-post: iv...@iveqy.com
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Why can't I push from a shallow clone?

[Konstantin Khomoutov]

On Thu, 25 Jul 2013 19:33:16 -0700
Gulshan Singh gsingh2...@gmail.com wrote:

 I've been trying to figure out why I can't push from a shallow clone
 (using --depth) to a repository. I've made simple examples where it
 works, but I've read that in doesn't work in every case. However, I
 can't come up with a case where it doesn't work. Googling gives this
 answer:
 http://stackoverflow.com/questions/6900103/why-cant-i-push-from-a-shallow-clone,
 but I don't completely understand the explanation, so I was hoping
 someone could explain it.

See also this thread [1] which originated from this SO question [2].

1. http://thread.gmane.org/gmane.comp.version-control.git/221634
2. http://stackoverflow.com/q/16077691/720999
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Why can't I push from a shallow clone?

[Gulshan Singh]

Hey Fredrick,

It's good to know that work is being done on this. I still want to
understand it though, so I guess I'll ask specific questions about
Junio's response. He states,

If you cloned shallowly some time ago, worked without communicating
with the other side while the other side progressed, AND if the other
side's progress included a rewind  rebuild of the history, you would
see a similar topology.
The leftmost 'S' in the above picture might have been the tip of the
branch when you shallowly cloned with depth 1, and since then the
remote end may have discarded topmost three commits and have rebuilt
its history that leads to the rightmost 'R'. In such a case pushing to
the remote's HEAD will fail.

So as an example, if you shallow clone a branch, then someone squashes
the head commit and makes a new commit, you won't be able to push to
HEAD because the parent has changed. But if someone does that, I don't
think you would be able to push even if it was a full clone. That's
why it's usually not a good idea to rebase shared branches. So did I
misunderstand the scenario?


On Thu, Jul 25, 2013 at 11:55 PM, Fredrik Gustafsson iv...@iveqy.com wrote:
 On Thu, Jul 25, 2013 at 07:33:16PM -0700, Gulshan Singh wrote:
 I've been trying to figure out why I can't push from a shallow clone
 (using --depth) to a repository. I've made simple examples where it
 works, but I've read that in doesn't work in every case. However, I
 can't come up with a case where it doesn't work. Googling gives this
 answer: 
 http://stackoverflow.com/questions/6900103/why-cant-i-push-from-a-shallow-clone,
 but I don't completely understand the explanation, so I was hoping
 someone could explain it.

 I can't explain it better than what Junio did in the link you just
 provide. However there's ongoing work to allow shallow clones to be able
 to push. You can read about it here:
 http://thread.gmane.org/gmane.comp.version-control.git/230612/focus=230878

 --
 Med vänliga hälsningar
 Fredrik Gustafsson

 tel: 0733-608274
 e-post: iv...@iveqy.com



-- 
Gulshan Singh
University of Michigan, Class of 2015
College of Engineering, Computer Science Major
guls...@umich.edu | 248.961.6317
Alternate E-mail: gsingh2...@gmail.com
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Why can't I push from a shallow clone?

[Philip Oakley]

From: Gulshan Singh gsingh2...@gmail.com
Sent: Friday, July 26, 2013 8:12 PM

Hey Fredrick,

It's good to know that work is being done on this. I still want to
understand it though, so I guess I'll ask specific questions about
Junio's response. He states,

If you cloned shallowly some time ago, worked without communicating
with the other side while the other side progressed, AND if the other
side's progress included a rewind  rebuild of the history, you would
see a similar topology.
The leftmost 'S' in the above picture might have been the tip of the
branch when you shallowly cloned with depth 1, and since then the
remote end may have discarded topmost three commits and have rebuilt
its history that leads to the rightmost 'R'. In such a case pushing to
the remote's HEAD will fail.

So as an example, if you shallow clone a branch, then someone squashes
the head commit and makes a new commit, you won't be able to push to
HEAD because the parent has changed. But if someone does that, I don't
think you would be able to push even if it was a full clone. That's
why it's usually not a good idea to rebase shared branches. So did I
misunderstand the scenario?


In your example you have a sucessful failure, that is, the shallow issue 
hasn't got in the way of the normal non-ff failure on push. (asuming the 
squash is an extra commit, not a re-written top commit, so at least you 
still have your common commit of your depth1 shallow clone)


For it to be a proper shallow failure the remote repo needs to be 
re-written sufficiently far back that your shallow clone has nothing in 
common with it. In such a case your DAG can't join onto the remote's 
DAG.





On Thu, Jul 25, 2013 at 11:55 PM, Fredrik Gustafsson iv...@iveqy.com 
wrote:

On Thu, Jul 25, 2013 at 07:33:16PM -0700, Gulshan Singh wrote:

I've been trying to figure out why I can't push from a shallow clone
(using --depth) to a repository. I've made simple examples where it
works, but I've read that in doesn't work in every case. However, I
can't come up with a case where it doesn't work. Googling gives this
answer: 
http://stackoverflow.com/questions/6900103/why-cant-i-push-from-a-shallow-clone,

but I don't completely understand the explanation, so I was hoping
someone could explain it.


I can't explain it better than what Junio did in the link you just
provide. However there's ongoing work to allow shallow clones to be 
able

to push. You can read about it here:
http://thread.gmane.org/gmane.comp.version-control.git/230612/focus=230878
--
Med vänliga hälsningar
Fredrik Gustafsson
tel: 0733-608274
e-post: iv...@iveqy.com

--
Gulshan Singh
University of Michigan, Class of 2015
College of Engineering, Computer Science Major
guls...@umich.edu | 248.961.6317
Alternate E-mail: gsingh2...@gmail.com

--
Philip 


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Why can't I push from a shallow clone?

[Gulshan Singh]

I've been trying to figure out why I can't push from a shallow clone
(using --depth) to a repository. I've made simple examples where it
works, but I've read that in doesn't work in every case. However, I
can't come up with a case where it doesn't work. Googling gives this
answer: 
http://stackoverflow.com/questions/6900103/why-cant-i-push-from-a-shallow-clone,
but I don't completely understand the explanation, so I was hoping
someone could explain it.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 00/16] First class shallow clone

[Philip Oakley]

From: Duy Nguyen pclo...@gmail.com
Sent: Wednesday, July 24, 2013 2:57 AM
On Wed, Jul 24, 2013 at 5:33 AM, Philip Oakley philipoak...@iee.org 
wrote:
In some sense a project with a sub-module is a narrow clone, split at 
a

'commit' object.


Yes, except narrow clone is more flexible. You have to decide the
split boundary at commit time for sub-module, while you decide the
same at clone time for narrow clone.



True. It was the thought experiment part I was referring to.


There have been comments on the git-user list about the
problem of accidental adding of large files which then make the 
repo's foot
print pretty large as one use case [Git is consuming very much RAM]. 
The
bigFileThreshold being one way of spotting such files as separate 
objects,

and 'trimming' them.


I think rewriting history to remove those accidents is better than
working around it (the same for accidentally committing password). We
might be able to spot problems early, maybe warn user at commit time
that they have added an exceptionally large blob, maybe before push
time..


Again, it was a thought experiment which related to a recent git-user 
list comment.
I'd expect a real use case could be a team where one member who is 
preparing documentation adds a [large] video to his branch and others 
then get a bit concerned when they try to track it / pull it as they 
really don't want it yet. The guy may have many versions on the central 
repo before a final rebase has a single compressed version. Colleagues 
may want to review the text surrounding it but not pull the video 
itself. (remembering 50 % of 'idiots' are twice as dumb as the 
average... ;-)




The Git is consuming very much RAM part is not right. We try to keep
memory usage under a limit regardless of the size of a blob. There may
be some cases we haven't fixed yet. Reports are welcome.


I think this was a Windows user, but reports do pop up every now and 
again. Some times its disc pressure, or just perceived slowness (from 
others)




--
Duy


Philip


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 00/16] First class shallow clone

[Piotr Krukowiecki]

(resending, as my phone mail client decided to send it in html, sorry
about that)

On Wed, Jul 24, 2013 at 3:57 AM, Duy Nguyen pclo...@gmail.com wrote:
 On Wed, Jul 24, 2013 at 5:33 AM, Philip Oakley philipoak...@iee.org wrote:
 There have been comments on the git-user list about the
 problem of accidental adding of large files which then make the repo's foot
 print pretty large as one use case [Git is consuming very much RAM]. The
 bigFileThreshold being one way of spotting such files as separate objects,
 and 'trimming' them.

 I think rewriting history to remove those accidents is better than
 working around it (the same for accidentally committing password). We
 might be able to spot problems early, maybe warn user at commit time
 that they have added an exceptionally large blob, maybe before push
 time..

I can imagine a situation where large files were part of the project
at some point in history (they were required to build/use it) and
later were removed because build/project has changed.

It would be useful to have the history for log/blame/etc even if you
could not build/use old versions. A warning when checking
out/branching such incomplete tree would be needed.

-- 
Piotr Krukowiecki
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html