Re: [RFC 0/4] Implementation of fetch-blobs and fetch-refs

[Stefan Beller]

Sorry for not paying attention to this patch set earlier. I missed
(and might still
miss) the big picture(tm) here. This is because
http://public-inbox.org/git/ffd92ad9-39fe-c76b-178d-6e3d6a425...@google.com/
seemed to be specific for big binary files, but ...

On Mon, Apr 10, 2017 at 1:46 PM, Jonathan Tan  wrote:

> In particular, patch 1 demonstrates that a new server endpoint that
> serves refs without the initial ref advertisement can be done in 228
> lines of code (according to "git diff --stat") while accounting for the
> various special cases that upload-pack imposes (stateless RPC, inclusion
> of an additional response when depth is requested, handling of "done" in
> request, sending a packfile directly after a response containing "ACK
> ready" without waiting for "done", and so on).

... from looking at the patches, this actually implements the idea
that was thrown around on the mailing list as "client speaks first",
as this essentially omits the refs advertisement, and then continues the
conversation by running upload-pack as normal.

That seems very exciting to me!

> To that end, I'm sending these patches in the hope of showing that these
> features are useful (omitting ref advertisements help greatly when
> serving large repos, as described in the commit message of patch 1, and
> serving blobs is useful for any fetch-blob-on-demand repo scheme) and
> that my proposed way of implementing them can be done in a relatively
> uncomplicated manner (as seen in these patches).

Yes, if we encapsulate v1 protocol on (both?) sides, we seem to need
very little additional code, but get the benefit of using lots of well
tested code.

Thanks,
Stefan

[RFC 0/4] Implementation of fetch-blobs and fetch-refs

[Jonathan Tan]

My previous proposal about a possible new server endpoint that serves
blobs and refs [1] didn't seem to garner much interest, so I thought I'd
provide a possible implementation of that proposal as a work in
progress.

In particular, patch 1 demonstrates that a new server endpoint that
serves refs without the initial ref advertisement can be done in 228
lines of code (according to "git diff --stat") while accounting for the
various special cases that upload-pack imposes (stateless RPC, inclusion
of an additional response when depth is requested, handling of "done" in
request, sending a packfile directly after a response containing "ACK
ready" without waiting for "done", and so on).

I'm hoping that these features would be of more interest to more people.
To that end, I'm sending these patches in the hope of showing that these
features are useful (omitting ref advertisements help greatly when
serving large repos, as described in the commit message of patch 1, and
serving blobs is useful for any fetch-blob-on-demand repo scheme) and
that my proposed way of implementing them can be done in a relatively
uncomplicated manner (as seen in these patches).

Patch 1-3 show what serving refs without the advertisement would look
like from the server's and fetch-pack's points of view. Patch 4 is
similar to some of the other blob-serving patches except that this
contains reachability checks and that this bundles the resulting objects
in a packfile.

[1] 

Jonathan Tan (4):
  server-endpoint: serve refs without advertisement
  fetch-pack: refactor "want" pkt-line generation
  fetch-pack: support new server endpoint
  server-endpoint: serve blobs by hash

 .gitignore |   1 +
 Makefile   |   3 +
 builtin/fetch-pack.c   |  10 +-
 fetch-pack.c   | 129 +++--
 fetch-pack.h   |   1 +
 server-endpoint.c  | 347 +
 t/helper/.gitignore|   1 +
 t/helper/test-un-pkt.c |  40 ++
 t/t5573-server-endpoint.sh |  60 
 t/t-mytests.sh | 242 +++
 10 files changed, 790 insertions(+), 44 deletions(-)
 create mode 100644 server-endpoint.c
 create mode 100644 t/helper/test-un-pkt.c
 create mode 100644 t/t5573-server-endpoint.sh
 create mode 100644 t/t-mytests.sh

-- 
2.12.2.715.g7642488e1d-goog