Re: reftable [v7]: new ref storage format

On Tue, Aug 15, 2017 at 7:48 PM, Shawn Pearce <spea...@spearce.org> wrote: > 7th iteration of the reftable storage format. > > You can read a rendered version of this here: > https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md FYI, JGit h

Re: reftable [v7]: new ref storage format

On Tue, Aug 15, 2017 at 11:15 PM, Stefan Beller <sbel...@google.com> wrote: > On Tue, Aug 15, 2017 at 7:48 PM, Shawn Pearce <spea...@spearce.org> wrote: >> 7th iteration of the reftable storage format. >> >> You can read a rendered version of this here: >>

Re: reftable [v7]: new ref storage format

7th iteration of the reftable storage format. You can read a rendered version of this here: https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md Changes from v6: - Blocks are variable sized, and alignment is optional. - ref index is required on variable sized

Re: reftable [v6]: new ref storage format

On Mon, Aug 14, 2017 at 5:13 AM, Michael Haggerty <mhag...@alum.mit.edu> wrote: > On 08/07/2017 03:47 AM, Shawn Pearce wrote: >> 6th iteration of the reftable storage format. > > Thanks! > >> Changes from v5: >> - extensions.refStorage = reftable is used to sel

Re: reftable [v6]: new ref storage format

On Tue, Aug 8, 2017 at 4:34 PM, Junio C Hamano <gits...@pobox.com> wrote: > Shawn Pearce <spea...@spearce.org> writes: > >> Given that the index can now also be multi-level, I don't expect to >> see a 2G index. A 2G index forces the reader to load the entire 2G to &g

Re: reftable [v6]: new ref storage format

On Tue, Aug 8, 2017 at 12:25 PM, Junio C Hamano <gits...@pobox.com> wrote: > Shawn Pearce <spea...@spearce.org> writes: > >> For `log_type = 0x4..0x7` the `log_chained` section is used instead to >> compress information that already appeared in a prior log record

Re: reftable [v6]: new ref storage format

On Tue, Aug 8, 2017 at 12:01 PM, Junio C Hamano wrote: > I notice that you raised the location of restart table within a > block in this iteration (or maybe it happened in v5). > > This forces you to hold all contents in core before the first byte > is written out. You start

Re: reftable [v5]: new ref storage format

On Tue, Aug 8, 2017 at 12:52 AM, Jeff King <p...@peff.net> wrote: > On Mon, Aug 07, 2017 at 03:40:48PM +, David Turner wrote: > >> > -Original Message----- >> > From: Shawn Pearce [mailto:spea...@spearce.org] >> > In git-core, I'm worried about th

Re: reftable [v6]: new ref storage format

On Mon, Aug 7, 2017 at 11:27 AM, Stefan Beller <sbel...@google.com> wrote: > On Sun, Aug 6, 2017 at 6:47 PM, Shawn Pearce <spea...@spearce.org> wrote: >> 6th iteration of the reftable storage format. >> >> You can read a rendered version of this here: >> htt

Re: reftable [v5]: new ref storage format

On Sun, Aug 6, 2017 at 4:37 PM, Ben Alex <ben.a...@acegi.com.au> wrote: > Just on the LmdbJava specific pieces: > > On Mon, Aug 7, 2017 at 8:56 AM, Shawn Pearce <spea...@spearce.org> wrote: >> >> Looks pretty complete. Its a Java wrapper around the C imp

Re: reftable [v6]: new ref storage format

6th iteration of the reftable storage format. You can read a rendered version of this here: https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md Changes from v5: - extensions.refStorage = reftable is used to select this format. - Log records can be

Re: reftable [v5]: new ref storage format

On Sun, Aug 6, 2017 at 9:56 AM, Ævar Arnfjörð Bjarmason <ava...@gmail.com> wrote: > On Sun, Aug 06 2017, Shawn Pearce jotted: > >> 5th iteration of the reftable storage format. > > I haven't kept up with all of the discussion, sorry if these comments > repeat somethi

Re: reftable [v5]: new ref storage format

5th iteration of the reftable storage format. You can read a rendered version of this here: https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md Significant changes from v4: - Supported Michael Haggerty's multi-level index. - Restart table now appears at

Re: reftable [v4]: new ref storage format

On Tue, Aug 1, 2017 at 6:51 PM, Michael Haggerty <mhag...@alum.mit.edu> wrote: > On Tue, Aug 1, 2017 at 4:27 PM, Shawn Pearce <spea...@spearce.org> wrote: >> On Mon, Jul 31, 2017 at 11:41 PM, Michael Haggerty <mhag...@alum.mit.edu> >> wrote: >>> On Sun, J

Re: reftable [v4]: new ref storage format

On Thu, Aug 3, 2017 at 3:48 PM, Michael Haggerty wrote: > I've revised the blockless reftable proposal to address some feedback: I've been thinking more about your blockless proposal. I experimentally modified my reftable implementation to omit padding between blocks,

Re: reftable [v4]: new ref storage format

On Thu, Aug 3, 2017 at 11:38 AM, Michael Haggerty <mhag...@alum.mit.edu> wrote: > On Tue, Aug 1, 2017 at 7:38 PM, Shawn Pearce <spea...@spearce.org> wrote: >> On Tue, Aug 1, 2017 at 6:51 PM, Michael Haggerty <mhag...@alum.mit.edu> >> wrote: >>> On Tue,

Re: reftable [v4]: new ref storage format

On Wed, Aug 2, 2017 at 1:28 PM, Jeff King wrote: > On Wed, Aug 02, 2017 at 12:50:39PM -0700, Junio C Hamano wrote: > >> With the traditional "packed-refs plus loose" layout, no matter how >> many times a handful of selected busy refs are updated during the >> day, you'd need to

Re: reftable [v4]: new ref storage format

On Wed, Aug 2, 2017 at 6:50 PM, Junio C Hamano wrote: > Junio C Hamano writes: > >> I like the general idea, what the file format can represent and how >> it does so, but I am a bit uneasy about how well this "stacked" part >> would work for desktop clients.

Re: reftable [v4]: new ref storage format

On Wed, Aug 2, 2017 at 2:28 AM, Jeff King <p...@peff.net> wrote: > On Tue, Aug 01, 2017 at 07:38:37PM -0700, Shawn Pearce wrote: > >> > OBJS blocks can also be >> > unbounded in size if very many references point at the same object, >> > thought tha

Re: reftable [v4]: new ref storage format

On Tue, Aug 1, 2017 at 6:51 PM, Michael Haggerty <mhag...@alum.mit.edu> wrote: > On Tue, Aug 1, 2017 at 4:27 PM, Shawn Pearce <spea...@spearce.org> wrote: >> On Mon, Jul 31, 2017 at 11:41 PM, Michael Haggerty <mhag...@alum.mit.edu> >> wrote: >>> On Sun, J

Re: reftable [v4]: new ref storage format

On Tue, Aug 1, 2017 at 4:27 PM, Shawn Pearce <spea...@spearce.org> wrote: > On Mon, Jul 31, 2017 at 11:41 PM, Michael Haggerty <mhag...@alum.mit.edu> > wrote: >> On Sun, Jul 30, 2017 at 8:51 PM, Shawn Pearce <spea...@spearce.org> wrote: >>> 4th i

Re: reftable [v4]: new ref storage format

On Mon, Jul 31, 2017 at 11:41 PM, Michael Haggerty <mhag...@alum.mit.edu> wrote: > On Sun, Jul 30, 2017 at 8:51 PM, Shawn Pearce <spea...@spearce.org> wrote: >> 4th iteration of the reftable storage format. >> [...] > > Before we commit to Shawn's reftable pro

Re: reftable [v4]: new ref storage format

On Mon, Jul 31, 2017 at 11:41 PM, Michael Haggerty <mhag...@alum.mit.edu> wrote: > On Sun, Jul 30, 2017 at 8:51 PM, Shawn Pearce <spea...@spearce.org> wrote: >> 4th iteration of the reftable storage format. >> [...] > > Before we commit to Shawn's reftable pro

Re: reftable [v4]: new ref storage format

On Mon, Jul 31, 2017 at 4:43 PM, Shawn Pearce <spea...@spearce.org> wrote: > On Mon, Jul 31, 2017 at 12:42 PM, Junio C Hamano <gits...@pobox.com> wrote: >> >> As a block cannot be longer than 16MB, allocating uint32 to a >> restart offset may be a bit overki

Re: reftable [v4]: new ref storage format

On Tue, Aug 1, 2017 at 6:54 AM, Dave Borowitz <dborow...@google.com> wrote: > On Sun, Jul 30, 2017 at 11:51 PM, Shawn Pearce <spea...@spearce.org> wrote: >> - Ref-like files (FETCH_HEAD, MERGE_HEAD) also use type 0x3. > >> - Combine reflog storage with ref

Re: reftable [v4]: new ref storage format

On Mon, Jul 31, 2017 at 12:42 PM, Junio C Hamano <gits...@pobox.com> wrote: > Shawn Pearce <spea...@spearce.org> writes: > >> ### Peeling >> >> References in a reftable are always peeled. > > This hopefully means "a record for an annotate

Re: reftable [v4]: new ref storage format

On Mon, Jul 31, 2017 at 12:01 PM, Stefan Beller <sbel...@google.com> wrote: > On Sun, Jul 30, 2017 at 8:51 PM, Shawn Pearce <spea...@spearce.org> wrote: >> 4th iteration of the reftable storage format. >> >> You can read a rendered version of this here: >>

Re: reftable [v4]: new ref storage format

4th iteration of the reftable storage format. You can read a rendered version of this here: https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md Significant changes from v3: - Incorporated Michael Haggerty's update_index concept for reflog. - Explicitly

Re: reftable [v3]: new ref storage format

On Fri, Jul 28, 2017 at 7:18 PM, Michael Haggerty <mhag...@alum.mit.edu> wrote: > On Fri, Jul 28, 2017 at 3:12 PM, Shawn Pearce <spea...@spearce.org> wrote: >> I'm with you this far, and like the {min,max}_update_index in the >> header. I'm concerned about update_inde

Re: reftable [v3]: new ref storage format

On Thu, Jul 27, 2017 at 7:28 AM, Michael Haggerty <mhag...@alum.mit.edu> wrote: > On Sat, Jul 22, 2017 at 11:29 AM, Shawn Pearce <spea...@spearce.org> wrote: >> 3rd iteration of the reftable storage format. >> >> [...] >> ### Objectives >> >> -

Re: reftable [v3]: new ref storage format

On Mon, Jul 24, 2017 at 3:22 PM, Stefan Beller <sbel...@google.com> wrote: > On Sat, Jul 22, 2017 at 11:29 AM, Shawn Pearce <spea...@spearce.org> wrote: >> 3rd iteration of the reftable storage format. >> >> You can read a rendered version of this here: >>

Re: reftable: new ref storage format

On Sun, Jul 23, 2017 at 3:56 PM, Shawn Pearce <spea...@spearce.org> wrote: > On Mon, Jul 17, 2017 at 6:43 PM, Michael Haggerty <mhag...@alum.mit.edu> > wrote: >> On Sun, Jul 16, 2017 at 12:43 PM, Shawn Pearce <spea...@spearce.org> wrote: >>> On Sun, Jul

Re: reftable: new ref storage format

+git@vger.kernel.org. I originally sent the below reply privately by mistake. On Mon, Jul 17, 2017 at 6:43 PM, Michael Haggerty <mhag...@alum.mit.edu> wrote: > On Sun, Jul 16, 2017 at 12:43 PM, Shawn Pearce <spea...@spearce.org> wrote: >> On Sun, Jul 16, 2017 at 10:33 AM, M

Re: reftable [v2]: new ref storage format

My apologies for not responding to this piece of feedback earlier. On Wed, Jul 19, 2017 at 7:02 AM, Ævar Arnfjörð Bjarmason <ava...@gmail.com> wrote: > On Tue, Jul 18 2017, Shawn Pearce jotted: >> On Mon, Jul 17, 2017 at 12:51 PM, Junio C Hamano <gits...@pobox.com> wrote: &

Re: reftable [v3]: new ref storage format

3rd iteration of the reftable storage format. You can read a rendered version of this here: https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md Significant changes from v2: - efficient lookup by SHA-1 for allow-tip-sha1-in-want. - type 0x4 for FETCH_HEAD,

Re: reftable [v2]: new ref storage format

On Mon, Jul 17, 2017 at 12:51 PM, Junio C Hamano <gits...@pobox.com> wrote: > Shawn Pearce <spea...@spearce.org> writes: >> You can read a rendered version of this here: >> https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable

Re: reftable [v2]: new ref storage format

On Mon, Jul 17, 2017 at 11:53 AM, Stefan Beller <sbel...@google.com> wrote: > On Mon, Jul 17, 2017 at 8:01 AM, Shawn Pearce <spea...@spearce.org> wrote: > >> A ref block is written as: >> >> 'r' >> uint24 ( block_len ) >> ref_record

reftable [v2]: new ref storage format

This is an updated draft after discussion on list with Peff, Michael Haggerty, and Dave Borowitz. You can read a rendered version of this here: https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md Biggest changes from the first proposal are: - reflog is now

Re: reftable: new ref storage format

On Sun, Jul 16, 2017 at 2:13 PM, Dave Borowitz <dborow...@google.com> wrote: > On Sun, Jul 16, 2017 at 3:43 PM, Shawn Pearce <spea...@spearce.org> wrote: >> True... but... in my "android" example repository we have 866,456 live >> refs. A block size of 64k n

Re: reftable: new ref storage format

On Sun, Jul 16, 2017 at 12:43 PM, Shawn Pearce <spea...@spearce.org> wrote: > On Sun, Jul 16, 2017 at 10:33 AM, Michael Haggerty <mhag...@alum.mit.edu> > wrote: > >> * The tuning parameter number_of_restarts currently trades off space >> (for the full refnames

Re: reftable: new ref storage format

On Sun, Jul 16, 2017 at 10:33 AM, Michael Haggerty wrote: > Thanks for your reftable proposal. Thanks for your time reading the proposal. I really was looking forward to your insights on this project. > It would solve a lot of scalability > problems that we currently have,

Re: reftable: new ref storage format

On Fri, Jul 14, 2017 at 1:08 PM, Jeff King <p...@peff.net> wrote: > On Thu, Jul 13, 2017 at 05:11:52PM -0700, Shawn Pearce wrote: > > I think the "stack" implementation is what makes me most uncomfortable > with this proposal. Atomicity with filesystem operations and e

Re: reftable: new ref storage format

On Fri, Jul 14, 2017 at 7:27 AM, Dave Borowitz <dborow...@google.com> wrote: > On Thu, Jul 13, 2017 at 8:11 PM, Shawn Pearce <spea...@spearce.org> wrote: >> In another (Gerrit Code Review), we disable reflogs for >> the insane refs/changes/ namespace, as nearly every r

Re: reftable: new ref storage format

On Thu, Jul 13, 2017 at 1:35 PM, Jeff King wrote: > On Thu, Jul 13, 2017 at 12:56:54PM -0700, Stefan Beller wrote: > > I agree that a full binary search of a reftable is harder because of the > prefix compression (it may still be possible by scanning backwards, but > I think there

Re: reftable: new ref storage format

On Thu, Jul 13, 2017 at 12:32 PM, Jeff King <p...@peff.net> wrote: > On Wed, Jul 12, 2017 at 05:17:58PM -0700, Shawn Pearce wrote: > >> ### Problem statement >> >> Some repositories contain a lot of references (e.g. android at 866k, >> rails at 31k). The

reftable: new ref storage format

We've been having scaling problems with insane number of references (>866k), so I started thinking a lot about improving ref storage. I've written a simple approach, and implemented it in JGit. Performance is promising: - 62M packed-refs compresses to 27M - 42.3 usec lookup You can read a

Re: [PATCH] fetch-pack: always allow fetching of literal SHA1s

On Tue, May 9, 2017 at 9:33 PM, Jeff King <p...@peff.net> wrote: > On Tue, May 09, 2017 at 09:22:11PM -0700, Shawn Pearce wrote: > >> > Hmm. That makes sense generally, as the request should succeed. But it >> > seems like we're creating a client that will sometimes s

Re: [PATCH] fetch-pack: always allow fetching of literal SHA1s

On Tue, May 9, 2017 at 3:16 PM, Jeff King wrote: > On Tue, May 09, 2017 at 11:20:42AM -0700, Jonathan Tan wrote: > >> fetch-pack, when fetching a literal SHA-1 from a server that is not >> configured with uploadpack.allowtipsha1inwant (or similar), always >> returns an error

Re: [PATCH] http.postbuffer: make a size_t

On Thu, Mar 30, 2017 at 1:29 PM, David Turner wrote: > Unfortunately, in order to push some large repos, the http postbuffer > must sometimes exceed two gigabytes. On a 64-bit system, this is OK: > we just malloc a larger buffer. I'm slightly curious what server you are

Re: RFC v3: Another proposed hash function transition plan

On Mon, Mar 6, 2017 at 4:17 PM, Jonathan Nieder wrote: > Linus Torvalds wrote: >> On Fri, Mar 3, 2017 at 5:12 PM, Jonathan Nieder wrote: > >>> This document is still in flux but I thought it best to send it out >>> early to start getting feedback. >> >>

Re: SHA1 collisions found

On Tue, Feb 28, 2017 at 11:34 AM, Linus Torvalds wrote: > On Tue, Feb 28, 2017 at 11:07 AM, Junio C Hamano wrote: >> >> In a way similar to 8415558f55 ("sha1dc: avoid c99 >> declaration-after-statement", 2017-02-24), we would want this on >> top.

Re: Git clonebundles

On Mon, Jan 30, 2017 at 11:00 PM, Stefan Saasen wrote: > > Bitbucket recently added support for Mercurial’s clonebundle extension > (http://gregoryszorc.com/blog/2015/10/22/cloning-improvements-in-mercurial-3.6/). > Mercurial’s clone bundles allow the Mercurial client to

Re: [RFC] Add support for downloading blobs on demand

On Fri, Jan 13, 2017 at 7:52 AM, Ben Peart wrote: > > Goal > > > To be able to better handle repos with many files that any individual > developer doesn’t need it would be nice if clone/fetch only brought down > those files that were actually needed. > > To enable that,

Re: [PATCH 2/2] connect: know that zero-ID is not a ref

On Fri, Sep 2, 2016 at 1:13 PM, Jeff King wrote: > > Hmm. So since this is backwards-compatible, I'm not overly concerned > with changing the client. But I wonder if you considered that the > documentation is wrong, and that JGit should stop sending the extra > capabilities line?

Re: [PATCH 2/2] connect: know that zero-ID is not a ref

On Fri, Sep 2, 2016 at 12:56 PM, Stefan Beller <sbel...@google.com> wrote: > On Fri, Sep 2, 2016 at 12:39 PM, Shawn Pearce <spea...@spearce.org> wrote: >> On Fri, Sep 2, 2016 at 10:15 AM, Jonathan Tan <jonathanta...@google.com> >> wrote: >>>

Re: [PATCH 2/2] connect: know that zero-ID is not a ref

On Fri, Sep 2, 2016 at 10:15 AM, Jonathan Tan wrote: > > + if (is_null_oid(_oid)) { > + if (strcmp(name, "capabilities^{}")) Its not the zero ID that is special, its the "capabilities^{}" name that is special when its the first entry

Re: [PATCH] contrib/persistent-https: update ldflags syntax for Go 1.7+

On Mon, Jul 18, 2016 at 9:32 PM, Parker Moore wrote: >> The label does not even identify the version of the source in any way, so I >> am not sure how people are depending on that feature anyway ;-) > > Would it be a better solution simply to remove this build flag? >

Re: [PATCH 2/4] receive-pack: implement advertising and receiving push options

On Fri, Jul 8, 2016 at 5:31 PM, Stefan Beller wrote: > + > + /* NEEDSWORK: expose the limitations to be configurable. */ > + int max_options = 32; > + > + /* > +* NEEDSWORK: expose the limitations to be configurable; > +* Once the limit can be

Re: [PATCH] http: Support sending custom HTTP headers

On Mon, Apr 25, 2016 at 6:13 AM, Johannes Schindelin wrote: > To make communication for `git fetch`, `git ls-remote` and friends extra > secure, we introduce a way to send custom HTTP headers with all > requests. Hmm. Its not Apr 1 2016. So I guess you are serious. :)

Re: Is it possible to give specific PR number

On Wed, Mar 16, 2016 at 8:20 AM, Igor Korot wrote: > Hi, > Is it possible to tell Git to have a PR with a specific number? Git does not have PRs. Are you referring to a GitHub Pull request? If so you should ask GitHub support. GitHub is a commercial entity that is separate

Re: RFC: Resumable clone based on hybrid "smart" and "dumb" HTTP

On Sun, Feb 14, 2016 at 9:05 AM, Jeff King <p...@peff.net> wrote: > On Sat, Feb 13, 2016 at 06:14:31PM -0800, Shawn Pearce wrote: > >> > And with "resumable=", the client does not have to hit the server >> > to do a redirect; it can go straight to

Re: [PATCH] remote-curl: Include curl_errorstr on SSL setup failures

On Sun, Feb 14, 2016 at 8:50 AM, Jeff King <p...@peff.net> wrote: > On Sat, Feb 13, 2016 at 05:39:34PM -0800, Shawn Pearce wrote: > >> For curl error 35 (CURLE_SSL_CONNECT_ERROR) users need the >> additional text stored in CURLOPT_ERRORBUFFER to debug why >>

Re: RFC: Resumable clone based on hybrid "smart" and "dumb" HTTP

On Wed, Feb 10, 2016 at 1:49 PM, Jeff King <p...@peff.net> wrote: > On Wed, Feb 10, 2016 at 12:11:46PM -0800, Shawn Pearce wrote: > >> On Wed, Feb 10, 2016 at 10:59 AM, Shawn Pearce <spea...@spearce.org> wrote: >> > >> > ... Thoughts? >> >>

[PATCH] remote-curl: Include curl_errorstr on SSL setup failures

, such as when the SSL setup fails. Don't include HTTP 0 in the message. Signed-off-by: Shawn Pearce <spea...@spearce.org> --- remote-curl.c | 16 ++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/remote-curl.c b/remote-curl.c index c704857..f611432 100644 --- a/remote-

Re: RFC: Resumable clone based on hybrid "smart" and "dumb" HTTP

On Wed, Feb 10, 2016 at 7:43 PM, Junio C Hamano wrote: > Jonathan Nieder writes: > >> I really like this design. I'm tempted to implement it (since it >> lacks a bunch of the downsides of clone.bundle). > > Just to see people are not stepping on each

Re: RFC: Resumable clone based on hybrid "smart" and "dumb" HTTP

On Wed, Feb 10, 2016 at 10:59 AM, Shawn Pearce <spea...@spearce.org> wrote: > > ... Thoughts? Several of us at $DAY_JOB talked about this more today and thought a variation makes more sense: 1. Clients attempting clone ask for /info/refs?service=git-upload-pack like they do today.

RFC: Resumable clone based on hybrid "smart" and "dumb" HTTP

Sorry, no code, only words today. Previously people have proposed a few different options for resumable clone, like "clone.bundle" (currently used by Android as a hack) or "skip start of regenerated pack". Here is another option. We could implement resumable clone by making a bit of a hybrid of

Re: git objects

On Thu, Dec 24, 2015 at 2:17 PM, Thiago Farina wrote: > > When ever I make a commit (assume I'm changing a single file) and do a > 'git push origin master', git says 'Counting objects: 6, done.' > > Does git makes 6 objects everytime? What are those objects? 1 commit object;

Re: RefTree: Alternate ref backend

On Tue, Dec 22, 2015 at 9:17 AM, Michael Haggerty wrote: > > etc. But we store branches into the main "refs/remotes/origin/" > namespace, leaving no reserved space for the remote "HEAD" (not to > mention other namespaces that might appear on the remote, such as >

Re: RefTree: Alternate ref backend

On Tue, Dec 22, 2015 at 11:09 AM, Junio C Hamano <gits...@pobox.com> wrote: > Shawn Pearce <spea...@spearce.org> writes: > >>> But really, aside from slightly helping >>> disambiguate references from paths in the command line, what is it good >>>

Re: RefTree: Alternate ref backend

On Tue, Dec 22, 2015 at 7:41 AM, Michael Haggerty <mhag...@alum.mit.edu> wrote: > On 12/17/2015 10:02 PM, Shawn Pearce wrote: >> I started playing around with the idea of storing references directly >> in Git. Exploiting the GITLINK tree entry, we can associate a

Re: RefTree: Alternate ref backend

On Thu, Dec 17, 2015 at 2:10 PM, Jeff King <p...@peff.net> wrote: > On Thu, Dec 17, 2015 at 01:02:50PM -0800, Shawn Pearce wrote: > >> I started playing around with the idea of storing references directly >> in Git. Exploiting the GITLINK tree entry, we can associat

RefTree: Alternate ref backend

I started playing around with the idea of storing references directly in Git. Exploiting the GITLINK tree entry, we can associate a name to any SHA-1. By storing all references in a single tree, atomic transactions are possible. Its a simple compare-and-swap of a single 40 byte SHA-1. This of

Re: RefTree: Alternate ref backend

On Thu, Dec 17, 2015 at 1:57 PM, Junio C Hamano <gits...@pobox.com> wrote: > Shawn Pearce <spea...@spearce.org> writes: > >> For example, recent git.git has a structure like this: >> >> $ git ls-tree -r refs/txn/committed >> 12 blob 22e42

Re: [RFC/PATCH v1] Add Travis CI support

On Fri, Sep 25, 2015 at 11:52 AM, Jeff King wrote: > On Fri, Sep 25, 2015 at 11:29:31AM -0700, Junio C Hamano wrote: > >> > So I wonder if it would be >> > helpful to have a microformat that the client would use to look at this. >> > E.g., it would fetch the cert tree, then

Re: Bloom filters for have/want negotiation

On Sat, Sep 12, 2015 at 12:01 PM, Junio C Hamano <gits...@pobox.com> wrote: > Shawn Pearce <spea...@spearce.org> writes: > >> The worst case is due to a bug in the negotiation. With nothing >> common, the client just goes on forever until it reaches roots >>

Re: Bloom filters for have/want negotiation

On Fri, Sep 11, 2015 at 2:13 PM, Michael Haggerty wrote: > I have been thinking about Wilhelm Bierbaum's talk at the last GitMerge > conference [1] in which he describes a scheme for using Bloom filters to > make the initial reference advertisement less expensive. ... > But

Re: Git Smart HTTP with HTTP/2.0

On Sat, Jul 11, 2015 at 12:00 AM, Ilari Liusvaara ilari.liusva...@elisanet.fi wrote: On Sat, Jul 11, 2015 at 11:10:48AM +0800, ForceCharlie wrote: As we known, HTTP/2.0 has been released. All Git-Smart-HTTP are currently implemented using HTTP/1.1. Nit: It is HTTP/2. Frequently used Git

Re: Git Smart HTTP with HTTP/2.0

On Sat, Jul 11, 2015 at 11:26 AM, Ilari Liusvaara ilari.liusva...@elisanet.fi wrote: On Sat, Jul 11, 2015 at 10:23:09AM -0700, Shawn Pearce wrote: Websockets over HTTP/2 (a.k.a. websockets2) has not been defined yet. With Websockets(1), it would probably already be possible to tunnel

send-pack silently tries to recreate funny remote refs

Today I learned that git push will silently try to recreate a remote funny ref if you push to it. Hypothetically ... lets say I have a reimplementation of Git called JGit. Lets say its on a server somewhere on the network. Lets say its a bit more liberal in what it accepts... $ git push origin

Re: [PATCH 3/7] pack-protocol.txt: Mark all LFs in push-cert as required

On Wed, Jul 1, 2015 at 1:49 PM, Junio C Hamano gits...@pobox.com wrote: Dave Borowitz dborow...@google.com writes: I am moderately negative about this; wouldn't it make the end result cleaner to fix the implementation? I'm not sure I understand your suggestion. Are you saying, you would

Re: [PATCH 3/7] pack-protocol.txt: Mark all LFs in push-cert as required

On Fri, Jul 3, 2015 at 11:07 AM, Jeff King p...@peff.net wrote: I wondered briefly whether this impacted the keepalives we added to `upload-pack` in 05e9515; those are implemented as 0-byte data packets, which we send during the potentially long counting/delta-compression phase before we send

Re: RFC/Pull Request: Refs db backend

On Tue, Jun 23, 2015 at 4:47 AM, Jeff King p...@peff.net wrote: One of the problems we've had with large-ref repos is that the reflog storage is quite inefficient. Yup. We ran into this with Gerrit Code Review years ago. The refs/changes/... namespace created by Gerrit Code Review is 1 ref per

git push keeps writing after server failure

I did something stupid like trying to push a copy of WebKit[1] into my GitHub account. This is ~5.2 GiB of data, which GitHub prefers not to accept. Ok ... $ git push --all g...@github.com:spearce/wk.git Counting objects: 2752427, done. Delta compression using up to 12 threads. Compressing

Re: [PATCH] index-pack: avoid excessive re-reading of pack directory

On Wed, Jun 10, 2015 at 7:00 AM, Jeff King p...@peff.net wrote: On Tue, Jun 09, 2015 at 08:46:24PM -0700, Shawn Pearce wrote: This patch introduces a quick flag to has_sha1_file which callers can use when they would prefer high performance at the cost of false negatives during repacks

Re: [PATCH] index-pack: avoid excessive re-reading of pack directory

On Fri, Jun 5, 2015 at 5:29 AM, Jeff King p...@peff.net wrote: However, some code paths make a large number of has_sha1_file checks which are _not_ expected to return 1. The collision test in index-pack.c is such a case. On a local system, this can cause a performance slowdown of around 5%.

Re: git commit --amend safety check?

On Tue, Mar 10, 2015 at 10:11 PM, Junio C Hamano gits...@pobox.com wrote: Shawn Pearce spea...@spearce.org writes: We keep seeing reports of Gerrit Code Review users who incorrectly do something like: git clone URL foo cd foo git commit --amend -m My first change! -a git push URL

git commit --amend safety check?

We keep seeing reports of Gerrit Code Review users who incorrectly do something like: git clone URL foo cd foo git commit --amend -m My first change! -a git push URL HEAD:refs/for/master Step #3 is where they get into trouble. They just amended the published tip commit and pushed it back

Re: git-scm.com website (was: Promoting Git developers)

On Mon, Mar 9, 2015 at 6:57 AM, Michael J Gruber g...@drmicha.warpmail.net wrote: Since we're talking business: git-scm.com still looks a bit like a ProGit/Github promotion site. I don't have anything against either, and git-scm.com provides a lot of the information that users are looking

Re: git-scm.com website

On Mon, Mar 9, 2015 at 9:06 AM, David Kastrup d...@gnu.org wrote: Shawn Pearce spea...@spearce.org writes: On Mon, Mar 9, 2015 at 6:57 AM, Michael J Gruber g...@drmicha.warpmail.net wrote: Since we're talking business: git-scm.com still looks a bit like a ProGit/Github promotion site. I

Re: [RFC/PATCH 0/3] protocol v2

On Wed, Mar 4, 2015 at 4:05 AM, Duy Nguyen pclo...@gmail.com wrote: On Wed, Mar 4, 2015 at 11:27 AM, Shawn Pearce spea...@spearce.org wrote: Let me go on a different tangent a bit from the current protocol. http://www.grpc.io/ was recently released and is built on the HTTP/2 standard. It uses

Re: An interesting opinion on DVCS/git

On Sun, Mar 1, 2015 at 7:29 PM, Stefan Beller sbel...@google.com wrote: bitquabit.com/post/unorthodocs-abandon-your-dvcs-and-return-to-sanity Indeed, a DVCS like Git or Hg does not fit everyone. And neither do centralized systems like Subversion. Choice is good. However... I found some passages

Re: [RFC/PATCH 0/3] protocol v2

On Tue, Mar 3, 2015 at 5:54 PM, Duy Nguyen pclo...@gmail.com wrote: On Wed, Mar 4, 2015 at 12:13 AM, Junio C Hamano gits...@pobox.com wrote: My recollection is that the consensus from the last time we discussed protocol revamping was to list one capability per packet so that packet length

Re: does the git over ssh protocol tell the server the hostname?

On Thu, Feb 26, 2015 at 3:14 PM, Christoph Anton Mitterer cales...@scientia.net wrote: I saw that when plain git (i.e. git://) is used, the client tells the server the hostname specified on the client side. For http one has the same automatically via http's Host: header. But after watching

Re: [Discuss] soften warning message after cloning void

On Wed, Feb 18, 2015 at 4:43 PM, Shawn Pearce spea...@spearce.org wrote: On Wed, Feb 18, 2015 at 4:25 PM, Junio C Hamano gits...@pobox.com wrote: Checking out a random branch is absolutely the worst thing you can do. Personally, I would think that the best thing you can do is to educate your

Re: [Discuss] soften warning message after cloning void

On Wed, Feb 18, 2015 at 4:25 PM, Junio C Hamano gits...@pobox.com wrote: Checking out a random branch is absolutely the worst thing you can do. Personally, I would think that the best thing you can do is to educate your users not to clone from a void. Create some history that is worth sharing,

Re: Pack v4 again..

On Sun, Feb 15, 2015 at 10:45 PM, Jeff King p...@peff.net wrote: On Sun, Feb 15, 2015 at 11:59:02PM -0500, Nicolas Pitre wrote: Yet, I think the biggest problem with pack v4 at the moment is the packing algorithm for tree objects. We are piggy-backing on the pack v2 object delta compression

Re: Multi-threaded 'git clone'

On Mon, Feb 16, 2015 at 10:43 AM, Junio C Hamano gits...@pobox.com wrote: Jeff King p...@peff.net writes: ... And the whole output is checksummed by a single sha1 over the whole stream that comes at the end. I think the most feasible thing would be to quickly spool it to a server on the

Re: Poor push performance with large number of refs

On Tue, Dec 9, 2014 at 4:37 PM, brian m. carlson sand...@crustytoothpaste.net wrote: I have a repository that's just under 2 GiB in size and contains over 2 refs, with a copy of it on a server. Both sides are using Git 2.1.2. If I push a branch that contains a single commit, it takes

Re: [PATCH] fetch-pack: don't resend known-common refs in find_common

On Sun, Oct 26, 2014 at 8:39 AM, Dennis Kaarsemaker den...@kaarsemaker.net wrote: On Wed, Oct 22, 2014 at 10:11:40AM -0700, Junio C Hamano wrote: Dennis Kaarsemaker den...@kaarsemaker.net writes: On di, 2014-10-21 at 10:56 -0700, Junio C Hamano wrote: Dennis Kaarsemaker

  1   2   3   >