Re: Anomalous AUTHOR and DOCUMENTATION sections in manpages

2014-01-22 Thread Shawn Pearce
On Wed, Jan 22, 2014 at 3:22 AM, Michael Haggerty mhag...@alum.mit.edu wrote: I just noticed that there are exactly four Git manpages with an AUTHOR section and five with a DOCUMENTATION section: $ make doc $ grep -nIE -e '^\.SH DOCUMENTATION|AUTHOR' Documentation/*.[0-9]

Re: [PATCH v4 08/23] ewah: compressed bitmap implementation

2014-01-23 Thread Shawn Pearce
On Thu, Jan 23, 2014 at 10:33 AM, Jeff King p...@peff.net wrote: On Wed, Jan 22, 2014 at 06:05:36PM -0800, Jonathan Nieder wrote: Jeff King wrote: EWAH is a word-aligned compressed variant of a bitset (i.e. a data structure that acts as a 0-indexed boolean array for many entries). I

[PATCH] Introduce experimental remote object access mode

2014-02-11 Thread Shawn Pearce
Make it easy to experiment what remote access to objects would be like if the network ran at say 1 ms round trip latency to obtain any object not on the local repository. $ time git ls-tree -r HEAD real 0m0.059s $ time GIT_RTT=1 git ls-tree -r HEAD real 0m27.283s Yes kids, slowing down

Re: [PATCH] Introduce experimental remote object access mode

2014-02-12 Thread Shawn Pearce
On Tue, Feb 11, 2014 at 11:29 AM, Junio C Hamano gits...@pobox.com wrote: Shawn Pearce spea...@spearce.org writes: Why would you do this? Perhaps you need more time in your day to consume tea or coffee. Set GIT_RTT and enjoy a beverage. So the conclusion is that it is not practical to do

Re: Git GSoC 2014

2014-02-15 Thread Shawn Pearce
On Sat, Feb 15, 2014 at 4:17 AM, Thomas Rast t...@thomasrast.ch wrote: David Kastrup d...@gnu.org writes: Thomas Rast t...@thomasrast.ch writes: Motivation: I believe that migrating to libgit2 is the better approach, medium term, than rewriting everything ourselves to be nice, clean and

Re: [PATCH 0/6] inotify support

2014-02-19 Thread Shawn Pearce
On Sun, Jan 12, 2014 at 3:03 AM, Nguyễn Thái Ngọc Duy pclo...@gmail.com wrote: It's been 37 weeks since Robert Zeh's attempt to bring inotify support to Git [1] and unless I missed some mails, no updates since. So here's another attempt with my preferred approach (can't help it, playing with

Re: Git in GSoC 2014

2014-02-26 Thread Shawn Pearce
be interested on mentoring for the equivalent task (index v4 on JGit). I've CC'ed Shawn Pearce. A project that added to both libgit2 and JGit would be cool, but I don't know if that is asking too much of the student (multiple languages and projects is going to increase the time spent on non-code

Re: [RFC/WIP] Pluggable reference backends

2014-03-10 Thread Shawn Pearce
On Mon, Mar 10, 2014 at 4:00 AM, Michael Haggerty mhag...@alum.mit.edu wrote: I have started working on pluggable ref backends. In this email I would like to share my plans and solicit feedback. Yay! JGit already has pluggable ref backends, so it is good to see this starting in git-core.

Re: [RFC/WIP] Pluggable reference backends

2014-03-10 Thread Shawn Pearce
On Mon, Mar 10, 2014 at 2:07 PM, Michael Haggerty mhag...@alum.mit.edu wrote: On 03/10/2014 04:52 PM, Jeff King wrote: On Mon, Mar 10, 2014 at 07:30:45AM -0700, Shawn Pearce wrote: * Store references in a SQLite database, to get correct transaction handling. No to SQLLite in git-core

Re: egit vs. git behaviour (was: [RFC/WIP] Pluggable reference backends)

2014-03-12 Thread Shawn Pearce
On Wed, Mar 12, 2014 at 3:26 AM, Andreas Krey a.k...@gmx.de wrote: On Mon, 10 Mar 2014 19:39:00 +, Shawn Pearce wrote: Yes, this was my real concern. Eclipse users using EGit expect EGit to be compatible with git-core at the filesystem level so they can do something in EGit then switch

No progress from push when using bitmaps

2014-03-12 Thread Shawn Pearce
Today I tried pushing a copy of linux.git from a client that had bitmaps into a JGit server. The client stalled for a long time with no progress, because it reused the existing pack. No progress appeared while it was sending the existing file on the wire: $ git push git://localhost/linux.git

Re: No progress from push when using bitmaps

2014-03-13 Thread Shawn Pearce
On Thu, Mar 13, 2014 at 2:26 PM, Jeff King p...@peff.net wrote: On Wed, Mar 12, 2014 at 05:21:21PM -0700, Shawn Pearce wrote: Today I tried pushing a copy of linux.git from a client that had bitmaps into a JGit server. The client stalled for a long time with no progress, because it reused

Re: [PATCH v3 0/8] Hiding refs

2014-03-14 Thread Shawn Pearce
On Fri, Mar 14, 2014 at 5:37 AM, Duy Nguyen pclo...@gmail.com wrote: On Wed, Mar 12, 2014 at 3:36 AM, Jeff King p...@peff.net wrote: If the client is limited to setting a few flags, then something like http can get away with: GET

Re: [PATCH v3 0/8] Hiding refs

2014-03-14 Thread Shawn Pearce
On Fri, Mar 14, 2014 at 4:30 PM, Duy Nguyen pclo...@gmail.com wrote: On Fri, Mar 14, 2014 at 11:45 PM, Shawn Pearce spea...@spearce.org wrote: You missed the SSH case. It doesn't have this slot to hide the data into. Right now we run this for ssh case: ssh host git-upload-pack repo-path. New

Re: security vulnerability disclosure procedure?

2014-04-21 Thread Shawn Pearce
On Sun, Apr 20, 2014 at 1:42 PM, Richard Hansen rhan...@bbn.com wrote: I have discovered a minor security vulnerability. I could send the patch to the mailing list, but I wanted someone else to take a look first just to make sure it's minor enough that folks will think it's OK to publicly

Re: [PATCH 1/2] blame: large-scale performance rewrite

2014-04-25 Thread Shawn Pearce
On Fri, Apr 25, 2014 at 4:56 PM, David Kastrup d...@gnu.org wrote: The previous implementation used a single sorted linear list of blame entries for organizing all partial or completed work. Every subtask had to scan the whole list, with most entries not being relevant to the task. The

Re: [PATCH 1/2] blame: large-scale performance rewrite

2014-04-26 Thread Shawn Pearce
On Sat, Apr 26, 2014 at 12:48 AM, David Kastrup d...@gnu.org wrote: Shawn Pearce spea...@spearce.org writes: On Fri, Apr 25, 2014 at 4:56 PM, David Kastrup d...@gnu.org wrote: The previous implementation used a single sorted linear list of blame entries for organizing all partial or completed

Re: [PATCH 1/2] blame: large-scale performance rewrite

2014-04-26 Thread Shawn Pearce
On Sat, Apr 26, 2014 at 9:50 AM, David Kastrup d...@gnu.org wrote: Shawn Pearce spea...@spearce.org writes: On Sat, Apr 26, 2014 at 12:48 AM, David Kastrup d...@gnu.org wrote: Shawn Pearce spea...@spearce.org writes: And JGit was already usually slower than git-core. Now it will be even

Re: [PATCH 1/2] blame: large-scale performance rewrite

2014-04-26 Thread Shawn Pearce
On Sat, Apr 26, 2014 at 10:30 AM, David Kastrup d...@gnu.org wrote: David Kastrup d...@gnu.org writes: http://repo.or.cz/r/wortliste.git git blame [-M / -C] wortliste The latter one is _really_ taking a severe hit from the O(n^2) algorithms. If your benchmarks for that one still point

Re: [PATCH 1/2] blame: large-scale performance rewrite

2014-04-27 Thread Shawn Pearce
On Sat, Apr 26, 2014 at 2:39 PM, David Kastrup d...@gnu.org wrote: At least the stuff I fixed with regard to performance would seem to be done right in JGit to start with. Hah! Its Java. We have to do things right, otherwise its too slow. :-) Its still not as fast as I want it to be. :-)

Re: [PATCH 00/32] Split index mode for very large indexes

2014-04-28 Thread Shawn Pearce
On Mon, Apr 28, 2014 at 3:55 AM, Nguyễn Thái Ngọc Duy pclo...@gmail.com wrote: I hinted about it earlier [1]. It now passes the test suite and with a design that I'm happy with (thanks to Junio for a suggestion about the rename problem). From the user point of view, this reduces the writable

Re: smudge/clean filters and SHA1 hashes

2014-05-02 Thread Shawn Pearce
On Fri, May 2, 2014 at 2:05 PM, Leo Razoumov slonik...@gmail.com wrote: surprisingly, searching this list and by way of Google I cannot find an answer to a simple question: In presence of smudge/clean filters which SHA1 hash (clean content or smudged content) gets stored in the repository?

Re: [PATCH 18/18] signed push: final protocol update

2014-08-21 Thread Shawn Pearce
On Tue, Aug 19, 2014 at 3:06 PM, Junio C Hamano gits...@pobox.com wrote: + push-cert = PKT-LINE(push-cert NUL capability-list LF) Haha. NUL. I love our wire protocol. + PKT-LINE(certificate version 0.1 LF) + PKT-LINE(pusher ident LF) +

Re: [PATCH 18/18] signed push: final protocol update

2014-08-22 Thread Shawn Pearce
On Fri, Aug 22, 2014 at 10:59 AM, Junio C Hamano gits...@pobox.com wrote: Junio C Hamano gits...@pobox.com writes: There are a few gotchas I can certainly use help on, especially from a smart-http expert ;-). * pushed-to URL will identify the site and the repository, so you cannot MITM

Re: [BUG] resolved deltas

2014-08-25 Thread Shawn Pearce
On Sat, Aug 23, 2014 at 3:56 AM, Jeff King p...@peff.net wrote: [+cc spearce, as I think this is a bug in code.google.com's sending side, and he can probably get the attention of the right folks] ... My guess is a bug on the sending side. We have seen duplicate pack objects before, but never

Re: [PATCH 18/18] signed push: final protocol update

2014-08-26 Thread Shawn Pearce
On Mon, Aug 25, 2014 at 10:59 AM, Junio C Hamano gits...@pobox.com wrote: Shawn Pearce spea...@spearce.org writes: A stateless nonce could look like: nonce = HMAC_SHA1( SHA1(site+path) + '.' + now, site_key ) where site_key is a private key known to the server. It doesn't have to be per

[PATCH] Document LF appearing in shallow command during send-pack/receive-pack

2014-08-27 Thread Shawn Pearce
The implementation sends an LF, but the protocol documentation was missing this detail. Signed-off-by: Shawn Pearce spea...@spearce.org --- Documentation/technical/pack-protocol.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/technical/pack-protocol.txt b

Re: Using Gerrit to review Git patches (was: Re: Transaction patch series overview)

2014-08-27 Thread Shawn Pearce
On Wed, Aug 27, 2014 at 2:53 PM, Michael Haggerty mhag...@alum.mit.edu wrote: On 08/26/2014 02:03 AM, Jonathan Nieder wrote: Jonathan Nieder wrote: [...] I've having trouble keeping track of how patches change, which patches have been reviewed and which haven't, unaddressed comments, and so

Re: [PATCH 2/2] index-pack: handle duplicate base objects gracefully

2014-08-29 Thread Shawn Pearce
On Fri, Aug 29, 2014 at 3:08 PM, Jeff King p...@peff.net wrote: On Fri, Aug 29, 2014 at 02:56:18PM -0700, Junio C Hamano wrote: Jeff King p...@peff.net writes: If a pack contains duplicates of an object, and if that object has any deltas pointing at it with REF_DELTA, we will try to

Re: [PATCH v3 20/21] signed push: add pushee header to push certificate

2014-09-04 Thread Shawn Pearce
On Thu, Sep 4, 2014 at 1:04 PM, Junio C Hamano gits...@pobox.com wrote: Record the URL of the intended recipient for a push (after anonymizing it if it has authentication material) on a new pushee URL header. Because the networking configuration (SSH-tunnels, proxies, etc.) on the pushing

Re: [PATCH 18/18] signed push: final protocol update

2014-09-04 Thread Shawn Pearce
On Thu, Sep 4, 2014 at 4:57 PM, Junio C Hamano gits...@pobox.com wrote: Shawn Pearce spea...@spearce.org writes: As you know, the stateless HTTP thing doesn't allow the nonce on the server to be carried from the initial ref advertisement into the final receive-pack. We would either need

Re: Porting git to HP NonStop

2012-08-10 Thread Shawn Pearce
On Fri, Aug 10, 2012 at 8:04 AM, Joachim Schmitz j...@schmitz-digital.de wrote: - HP NonStop is lacking poll(), git is making quite some use of it. My Solution: I 'stole' the implementation from GNUlib, which implements poll() using select(). Git should either provide its own poll(), not use

Re: Porting git to HP NonStop

2012-08-10 Thread Shawn Pearce
On Fri, Aug 10, 2012 at 10:32 AM, Joachim Schmitz j...@schmitz-digital.de wrote: then use `git init --bare` in a new directory to copy in the templates, and see if its the template copying code that is making an incorrect copy. git init --bare gives the same error. It isn't copying any of the

Re: [PATCH 0/4] jk/version-string and google code

2012-08-10 Thread Shawn Pearce
On Fri, Aug 10, 2012 at 2:50 PM, Jeff King p...@peff.net wrote: On Fri, Aug 10, 2012 at 11:52:28AM -0700, Junio C Hamano wrote: When evaluating a change in the interoperability area, it does not add much more confidence to the correctness that the change has been in use for months with the

Re: [PATCH/RFC] index-pack: produce pack index version 3

2012-08-13 Thread Shawn Pearce
Let me start by echoing Junio's remark... lacks sufficient justification. You don't give enough evidence to support even why it is worth looking at this commit, let alone why it should be included and cause a format change in the idx file format. At some point you start to hand-wave about how it

Re: [PATCH] Implement ACL module architecture and sample MySQL ACL module

2012-08-14 Thread Shawn Pearce
On Tue, Aug 14, 2012 at 9:12 AM, Junio C Hamano gits...@pobox.com wrote: Michal Novotny minov...@redhat.com writes: Hi, this is the patch to introduce the ACL module architecture into git versioning system. No, it doesn't. It adds something only to git daemon, but does not affect any

Re: [PATCH] Implement ACL module architecture and sample MySQL ACL module

2012-08-14 Thread Shawn Pearce
On Tue, Aug 14, 2012 at 10:06 AM, Junio C Hamano gits...@pobox.com wrote: Shawn Pearce spea...@spearce.org writes: Parsing the request line of git-daemon is easy. But we could make it easier. An alternative arrangement would be to add a new command line flag to git daemon like --command

Re: [PATCH] daemon: --access-hook option

2012-08-15 Thread Shawn Pearce
On Tue, Aug 14, 2012 at 10:12 PM, Junio C Hamano gits...@pobox.com wrote: The --access-hook option to git daemon specifies an external command to be run every time a client connects, with - service name (e.g. upload-pack, etc.), - path to the repository, - hostname (%H), - canonical

Re: [PATCH/RFC] index-pack: produce pack index version 3

2012-08-16 Thread Shawn Pearce
On Wed, Aug 15, 2012 at 10:42 PM, Junio C Hamano gits...@pobox.com wrote: Shawn Pearce spea...@spearce.org writes: ... But I think its worth giving him a few weeks to finish getting the code ready, vs. rushing something in that someone else thinks might help. We have waited more than 6 years

Re: [PATCH] Enable info/refs gzip decompression in HTTP client

2012-09-19 Thread Shawn Pearce
On Wed, Sep 19, 2012 at 5:43 PM, Junio C Hamano gits...@pobox.com wrote: - Accept-Encoding: deflate, gzip + Accept-Encoding: gzip ... was loss of deflate intended? If so why? Could you explain it in the log message? Yes. I would add the following to the end of the commit message as a new

Re: [PATCH] Disable dumb HTTP fallback with GIT_CURL_FALLBACK=0

2012-09-19 Thread Shawn Pearce
On Wed, Sep 19, 2012 at 7:55 PM, Shawn O. Pearce spea...@spearce.org wrote: From: Shawn O. Pearce spea...@spearce.org I can't explain why git send-email did this. I obviously didn't need the extra From header here. format-patch didn't write it to the patch file, it was injected by send-email. My

Re: [PATCH] Disable dumb HTTP fallback with GIT_CURL_FALLBACK=0

2012-09-19 Thread Shawn Pearce
On Wed, Sep 19, 2012 at 8:48 PM, Jeff King p...@peff.net wrote: On Wed, Sep 19, 2012 at 07:55:53PM -0700, Shawn O. Pearce wrote: If the user doesn't want to use the dumb HTTP protocol, she may set GIT_CURL_FALLBACK=0 in the environment before invoking a Git protocol operation. This is mostly

Re: [PATCH] Disable dumb HTTP fallback with GIT_CURL_FALLBACK=0

2012-09-20 Thread Shawn Pearce
On Thu, Sep 20, 2012 at 10:24 AM, Jeff King p...@peff.net wrote: On Wed, Sep 19, 2012 at 10:57:35PM -0700, Shawn O. Pearce wrote: so transient errors on the initial smart contact can cause us to fall back to dumb, Transient errors is actually what is leading me down this path. We see

Re: The GitTogether

2012-09-21 Thread Shawn Pearce
On Fri, Sep 21, 2012 at 2:20 AM, Christian Couder christian.cou...@gmail.com wrote: On Thu, Sep 20, 2012 at 8:53 PM, Sebastian Schuberth sschube...@gmail.com wrote: On 19.09.2012 15:43, Michael Haggerty wrote: Is there any news about the proposed gatherings? I would be quite interested in

Re: [PATCH] Disable dumb HTTP fallback with GIT_CURL_FALLBACK=0

2012-09-21 Thread Shawn Pearce
On Thu, Sep 20, 2012 at 10:26 PM, Jeff King p...@peff.net wrote: On Thu, Sep 20, 2012 at 04:05:03PM -0700, Shawn O. Pearce wrote: But right now I am seeing failures in libcurl's SSL connection that may also be causing the smart connection failures. For example this trace, where libcurl was

Re: git clone over http with basic auth bug?

2012-09-23 Thread Shawn Pearce
On Sat, Sep 22, 2012 at 1:43 PM, Paul J R m...@pjr.cc wrote: It hadnt occured to me that git-http-backend behaves differently to the dumb http protocol on read (though that was from reading the git source so i obviously missed what was going on there). Ultimately im writing a little webapp

Using bitmaps to accelerate fetch and clone

2012-09-26 Thread Shawn Pearce
Google has published a series of patches (see links below) to JGit to improve fetch and clone performance by adding compressed bitmaps to the pack-*.idx structure. Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB)

Re: Using bitmaps to accelerate fetch and clone

2012-09-27 Thread Shawn Pearce
On Thu, Sep 27, 2012 at 5:17 AM, Nguyen Thai Ngoc Duy pclo...@gmail.com wrote: On Thu, Sep 27, 2012 at 7:47 AM, Shawn Pearce spea...@spearce.org wrote: Google has published a series of patches (see links below) to JGit to Should discussions about this series happen in here, jgit mailing

Re: Commit cache to speed up rev-list and merge

2012-09-27 Thread Shawn Pearce
On Thu, Sep 27, 2012 at 5:17 AM, Nguyen Thai Ngoc Duy pclo...@gmail.com wrote: I'd like to see some sort of extension mechanism like in $GIT_DIR/index, so that we don't have to increase pack index version often. What I have in mind is optional commit cache to speed up rev-list and merge, which

Re: Using bitmaps to accelerate fetch and clone

2012-09-27 Thread Shawn Pearce
On Thu, Sep 27, 2012 at 10:20 AM, Jeff King p...@peff.net wrote: On Thu, Sep 27, 2012 at 07:17:42PM +0700, Nguyen Thai Ngoc Duy wrote: Operation Index V2 Index VE003 Clone 37530ms (524.06 MiB) 82ms (524.06 MiB) Fetch (1 commit back)

Re: Commit cache to speed up rev-list and merge

2012-09-27 Thread Shawn Pearce
On Thu, Sep 27, 2012 at 10:39 AM, Jeff King p...@peff.net wrote: On Thu, Sep 27, 2012 at 08:51:51AM -0700, Shawn O. Pearce wrote: On Thu, Sep 27, 2012 at 5:17 AM, Nguyen Thai Ngoc Duy pclo...@gmail.com wrote: I'd like to see some sort of extension mechanism like in $GIT_DIR/index, so that

Re: Using bitmaps to accelerate fetch and clone

2012-09-27 Thread Shawn Pearce
On Thu, Sep 27, 2012 at 11:22 AM, Jeff King p...@peff.net wrote: I think clients will also want it. If we can make git rev-list --objects --all faster (which this should be able to do), we can speed up git prune, which in turn is by far the slowest part of git gc --auto, since in the typical

Re: git smart-http do not authent to allow git ls-remote to be called anonymously

2012-09-30 Thread Shawn Pearce
On Sun, Sep 30, 2012 at 7:35 AM, 乙酸鋰 ch3co...@gmail.com wrote: I use smart-http on Apache. If nothing to be pushed / pulled, I do not want password to be supplied. And allow git ls-remote to run without password *.git/info/refs?service=git-upload-pack *.git/info/refs?service=git-receive-pack

Re: Commit cache to speed up rev-list and merge

2012-09-30 Thread Shawn Pearce
On Thu, Sep 27, 2012 at 7:14 PM, Nguyen Thai Ngoc Duy pclo...@gmail.com wrote: On Thu, Sep 27, 2012 at 10:51 PM, Shawn Pearce spea...@spearce.org wrote: In Linus' Linux kernel tree there are currently about 323,178 commits. If we store just the pre-parsed commit time as an int32 field

Re: Using bitmaps to accelerate fetch and clone

2012-09-30 Thread Shawn Pearce
On Sun, Sep 30, 2012 at 6:59 PM, Nguyen Thai Ngoc Duy pclo...@gmail.com wrote: On Mon, Oct 1, 2012 at 8:07 AM, Shawn Pearce spea...@spearce.org wrote: You mentioned this before in your idea mail a while back. I wonder if it's worth storing bitmaps for all packs, not just the self contained

Re: Commit cache to speed up rev-list and merge

2012-09-30 Thread Shawn Pearce
On Sun, Sep 30, 2012 at 7:05 PM, Nguyen Thai Ngoc Duy pclo...@gmail.com wrote: On Mon, Oct 1, 2012 at 8:49 AM, Shawn Pearce spea...@spearce.org wrote: On Thu, Sep 27, 2012 at 7:14 PM, Nguyen Thai Ngoc Duy pclo...@gmail.com wrote: On Thu, Sep 27, 2012 at 10:51 PM, Shawn Pearce spea

Re: [PATCH] Retry HTTP requests on SSL connect failures

2012-10-01 Thread Shawn Pearce
On Mon, Oct 1, 2012 at 3:18 PM, Jeff King p...@peff.net wrote: On Mon, Oct 01, 2012 at 02:23:06PM -0700, Shawn O. Pearce wrote: When libcurl fails to connect to an SSL server always retry the request once. Since the connection failed before the HTTP headers can be sent, no data has exchanged

Re: Using bitmaps to accelerate fetch and clone

2012-10-02 Thread Shawn Pearce
On Mon, Oct 1, 2012 at 5:48 AM, Nguyen Thai Ngoc Duy pclo...@gmail.com wrote: Probably off topic. Does saving a list of missing bases in the pack index help storing thin packs directly? I may be missing some points because I don't see why thin packs cannot be stored on disk in the first place.

Re: upload-pack is slow with lots of refs

2012-10-03 Thread Shawn Pearce
On Wed, Oct 3, 2012 at 11:55 AM, Jeff King p...@peff.net wrote: On Wed, Oct 03, 2012 at 11:53:35AM -0700, Junio C Hamano wrote: Jeff King p...@peff.net writes: Has there been any work on extending the protocol so that the client tells the server what refs it's interested in? I don't

Re: upload-pack is slow with lots of refs

2012-10-05 Thread Shawn Pearce
On Thu, Oct 4, 2012 at 11:24 PM, Johannes Sixt j.s...@viscovery.net wrote: Am 10/3/2012 21:41, schrieb Shawn Pearce: On Wed, Oct 3, 2012 at 11:55 AM, Jeff King p...@peff.net wrote: On Wed, Oct 03, 2012 at 11:53:35AM -0700, Junio C Hamano wrote: Jeff King p...@peff.net writes: Has there been

Re: upload-pack is slow with lots of refs

2012-10-09 Thread Shawn Pearce
On Mon, Oct 8, 2012 at 8:05 AM, Johannes Sixt j...@kdbg.org wrote: Am 05.10.2012 18:57, schrieb Shawn Pearce: On Thu, Oct 4, 2012 at 11:24 PM, Johannes Sixt j.s...@viscovery.net wrote: Upload-pack can just start advertising refs in the v1 way and announce a v2 capability and listen

Re: Git ~unusable on slow lines :,'C

2012-10-09 Thread Shawn Pearce
On Tue, Oct 9, 2012 at 7:06 AM, Marcel Partap mpar...@gmx.net wrote: Bam, the server kicked me off after taking to long to sync my copy. This is unrelated to git. The HTTP server's configuration is too impatient. Yes. How does that mean it is unrelated to git? It means its out of our control,

Re: Is anyone working on a next-gen Git protocol?

2012-10-10 Thread Shawn Pearce
On Wed, Oct 10, 2012 at 6:44 PM, Nguyen Thai Ngoc Duy pclo...@gmail.com wrote: On Thu, Oct 11, 2012 at 3:46 AM, Junio C Hamano gits...@pobox.com wrote: Steffen Prohaska proha...@zib.de writes: I've recently discovered that the current protocol can be amazingly inefficient when it comes to

Re: push race

2012-10-15 Thread Shawn Pearce
On Mon, Oct 15, 2012 at 11:56 AM, Jeff King p...@peff.net wrote: Right. The only thing that needs locking is the refs, because the object database is add-only for normal operations, and by definition collisions mean you have the same content (or are astronomically unlucky, but your consolation

Re: Fix potential hang in https handshake.

2012-10-19 Thread Shawn Pearce
On Fri, Oct 19, 2012 at 3:36 AM, Jeff King p...@peff.net wrote: On Thu, Oct 18, 2012 at 03:59:41PM -0700, Junio C Hamano wrote: It will sometimes happen that curl_multi_fdset() doesn't return any file descriptors. In that case, it's recommended that the application sleep for a short time

Re: [PATCH v2] git-submodule add: Add -r/--record option.

2012-10-28 Thread Shawn Pearce
On Sun, Oct 28, 2012 at 1:48 PM, Jens Lehmann jens.lehm...@web.de wrote: Am 23.10.2012 22:55, schrieb W. Trevor King: As Phil pointed out, doing anything with this variable is ambiguous: On Mon, Oct 22, 2012 at 06:03:53PM -0400, Phil Hord wrote: Some projects now use the 'branch' config value

Re: Reading commit objects

2013-05-21 Thread Shawn Pearce
On Tue, May 21, 2013 at 3:33 PM, Chico Sokol chico.so...@gmail.com wrote: It was git who created that object. We're trying to build a improved java library focused in our needs (jgit has a really confusing api focused in solving egit needs). JGit code... is confusing because its fast. We

Re: java zlib woes (was: Reading commit objects)

2013-05-21 Thread Shawn Pearce
On Tue, May 21, 2013 at 9:51 PM, Andreas Krey a.k...@gmx.de wrote: On Tue, 21 May 2013 19:18:35 +, Chico Sokol wrote: Ok, we discovered that the commit object actually contains the tree object's sha1, by reading its contents with python zlib library. So the bug must be with our java code

Re: Reading commit objects

2013-05-22 Thread Shawn Pearce
On Tue, May 21, 2013 at 3:18 PM, Chico Sokol chico.so...@gmail.com wrote: Ok, we discovered that the commit object actually contains the tree object's sha1, by reading its contents with python zlib library. So the bug must be with our java code (we're building a java lib). Is there any

Re: Reading commit objects

2013-05-22 Thread Shawn Pearce
On Wed, May 22, 2013 at 7:25 AM, Chico Sokol chico.so...@gmail.com wrote: Your code is broken. IOUtils is probably corrupting what you get back. After inflating the stream you should see the object type (commit), space, its length in bytes as a base 10 string, and then a NUL ('\0'). Following

Re: Reading commit objects

2013-05-22 Thread Shawn Pearce
On Wed, May 22, 2013 at 7:20 AM, Chico Sokol chico.so...@gmail.com wrote: I'm not criticizing JGit, guys. It simply doesn't fit into our needs. We're not interested in mapping git commands in java and don't have the same RAM limitations. I guess you aren't trying to process the WebKit or Linux

Re: [PATCH 09/16] documentation: add documentation for the bitmap format

2013-06-24 Thread Shawn Pearce
On Mon, Jun 24, 2013 at 5:23 PM, Vicent Marti tan...@gmail.com wrote: This is the technical documentation and design rationale for the new Bitmap v2 on-disk format. --- Documentation/technical/bitmap-format.txt | 235 + 1 file changed, 235 insertions(+)

Re: [PATCH 09/16] documentation: add documentation for the bitmap format

2013-06-26 Thread Shawn Pearce
On Tue, Jun 25, 2013 at 4:08 PM, Vicent Martí tan...@gmail.com wrote: On Tue, Jun 25, 2013 at 11:17 PM, Junio C Hamano gits...@pobox.com wrote: What case are you talking about? The n-th object must be one of these four types and can never be of more than one type at the same time, so a

Re: [PATCH 09/16] documentation: add documentation for the bitmap format

2013-06-26 Thread Shawn Pearce
On Tue, Jun 25, 2013 at 11:11 PM, Jeff King p...@peff.net wrote: On Tue, Jun 25, 2013 at 09:33:11PM +0200, Vicent Martí wrote: One way we side-stepped the size inflation problem in JGit was to only use the bitmap index information when sending data on the wire to a client. Here delta reuse

Re: [PATCH 09/16] documentation: add documentation for the bitmap format

2013-06-26 Thread Shawn Pearce
On Wed, Jun 26, 2013 at 6:53 PM, Colby Ranger cran...@google.com wrote: + Generating this reverse index at runtime is **not** free (around 900ms + generation time for a repository like `torvalds/linux`), and once again, + this generation time needs to happen every time `pack-objects` is +

Re: [PATCH 09/16] documentation: add documentation for the bitmap format

2013-06-27 Thread Shawn Pearce
On Wed, Jun 26, 2013 at 7:45 PM, Jeff King p...@peff.net wrote: In particular, it seems like the slowness we saw with the v1 bitmap format is not what Shawn and Colby have experienced. So it's possible that our test setup is bad or different. Or maybe the C v1 reading implementation had some

Re: [PATCH 09/16] documentation: add documentation for the bitmap format

2013-07-01 Thread Shawn Pearce
On Mon, Jul 1, 2013 at 11:47 AM, Colby Ranger cran...@google.com wrote: But I think we are comparing apples to steaks here, Vincent is (rightfully) concerned about process startup performance, whereas our timings were assuming the process was already running. I did some timing on loading

Re: [PATCH 09/16] documentation: add documentation for the bitmap format

2013-07-07 Thread Shawn Pearce
On Sun, Jul 7, 2013 at 2:46 AM, Jeff King p...@peff.net wrote: On Mon, Jul 01, 2013 at 11:47:32AM -0700, Colby Ranger wrote: But I think we are comparing apples to steaks here, Vincent is (rightfully) concerned about process startup performance, whereas our timings were assuming the

Re: [PATCH 4/4] pack-revindex: radix-sort the revindex

2013-07-07 Thread Shawn Pearce
On Sun, Jul 7, 2013 at 3:14 AM, Jeff King p...@peff.net wrote: The pack revindex stores the offsets of the objects in the pack in sorted order, allowing us to easily find the on-disk size of each object. To compute it, we populate an array with the offsets from the sha1-sorted idx file, and

Re: [PATCH 4/4] pack-revindex: radix-sort the revindex

2013-07-08 Thread Shawn Pearce
On Mon, Jul 8, 2013 at 12:57 AM, Jeff King p...@peff.net wrote: On Sun, Jul 07, 2013 at 04:52:23PM -0700, Shawn O. Pearce wrote: On Sun, Jul 7, 2013 at 3:14 AM, Jeff King p...@peff.net wrote: The pack revindex stores the offsets of the objects in the pack in sorted order, allowing us to

expanding pack idx fanout table

2013-07-08 Thread Shawn Pearce
Has anyone studied the impact of converting the pack idx fanout table from 256 entries to 65536 entries? Back of the envelope estimates for 3.1M objects in linux.git suggests a 2^16 fanout table would decrease the number of binary search iterations from ~14 to ~6. The increased table costs an

Re: Remote hung up during `git fetch`

2012-11-22 Thread Shawn Pearce
On Thu, Nov 22, 2012 at 10:39 AM, Yichao Yu yyc1...@gmail.com wrote: I sent this email yesterday to the git mailing list but I cannot find it in any archive so I decide to send it again. If it was HTML formatted it would have been silently dropped by the list. Does anyone know what has

Re: Possible vulnerability to SHA-1 collisions

2012-11-24 Thread Shawn Pearce
I don't think there is an issue the way you have tried to describe this scenario. On Sat, Nov 24, 2012 at 3:12 AM, Michael Hirshleifer 111...@caltech.edu wrote: Evil Guy creates 2 files, 1 evil and 1 innocuous, with the same SHA-1 checksum (including Git header). Mr. Evil creates a local branch

Re: Know how much to download before clonig

2012-11-24 Thread Shawn Pearce
On Sat, Nov 24, 2012 at 4:02 AM, arif aft...@gmail.com wrote: Is there any way so that i can tell how much i have to download to obtain a full clone (not a shallow one) of a respository? Ask the repository owner to tell you? There is no information sent by the server about size of repository,

Re: Millisecond precision in timestamps?

2012-11-27 Thread Shawn Pearce
On Tue, Nov 27, 2012 at 12:48 PM, Eric S. Raymond e...@thyrsus.com wrote: Because I do a lot of work on repository conversion tools, I've had to learn a lot of detail about ontological mismatches between version-control systems - especially places where you lose metadata moving between them.

Re: Millisecond precision in timestamps?

2012-11-27 Thread Shawn Pearce
On Tue, Nov 27, 2012 at 3:04 PM, Eric S. Raymond e...@thyrsus.com wrote: Junio C Hamano gits...@pobox.com: fsck_ident() in fsck.c rejects anything but [1-9][0-9]* after the author and committer ident (i.e. the timestamp has to be integral number of seconds since the epoch, not before it, nor

Re: Millisecond precision in timestamps?

2012-11-27 Thread Shawn Pearce
On Tue, Nov 27, 2012 at 4:26 PM, Felipe Contreras felipe.contre...@gmail.com wrote: On Wed, Nov 28, 2012 at 1:12 AM, Eric S. Raymond e...@thyrsus.com wrote: Shawn Pearce spea...@spearce.org: Well... if we added a fractional seconds to a commit, older versions of Git will scream loudly

Re: git fetch pack freezes

2012-11-28 Thread Shawn Pearce
On Wed, Nov 28, 2012 at 6:12 AM, Ivan Kanis ivan.ka...@googlemail.com wrote: We are using git with the smart HTTP protocol. We are seeing the client freeze with Jenkins poll. It stops the continuous build which is a bit of a pain. It happens with the latest release 1.8.0 on the client side.

Re: git fetch pack freezes

2012-11-28 Thread Shawn Pearce
On Wed, Nov 28, 2012 at 9:25 AM, Ivan Kanis ivan.ka...@googlemail.com wrote: Shawn Pearce spea...@spearce.org wrote: On Wed, Nov 28, 2012 at 6:12 AM, Ivan Kanis ivan.ka...@googlemail.com wrote: On the server we are seeing the following error message: Upgrade your server. OK we'll look

Re: does a successful 'git gc' imply 'git fsck'

2012-12-01 Thread Shawn Pearce
On Sat, Dec 1, 2012 at 6:31 PM, Sitaram Chamarty sitar...@gmail.com wrote: Background: I have a situation where I have to fix up a few hundred repos in terms of 'git gc' (the auto gc seems to have failed in many cases; they have far more than 6700 loose objects). I also found some corrupted

Re: [PATCH] git(1): remove a defunct link to list of authors

2012-12-07 Thread Shawn Pearce
On Fri, Dec 7, 2012 at 9:59 AM, Junio C Hamano gits...@pobox.com wrote: Junio C Hamano gits...@pobox.com writes: The linked page has not been showing the promised more complete list for more than 6 months by now, and nobody has resurrected the list there nor elsewhere since then.

Re: [RFC/PATCH] ignoring a fetch that overwrites local symref

2012-12-12 Thread Shawn Pearce
On Tue, Dec 11, 2012 at 11:46 AM, Junio C Hamano gits...@pobox.com wrote: This is a companion to an ancient thread http://thread.gmane.org/gmane.comp.version-control.git/145311/focus=145337 in which an error was dealt with while pushing into a mirror repository that has a symbolic

Re: Pushing symbolic references to remote repositories?

2012-12-21 Thread Shawn Pearce
On Fri, Dec 21, 2012 at 11:53 AM, Dun Peal dunpea...@gmail.com wrote: I need to share a symbolic reference - essentially, a named pointer to another reference - among multiple repositories. As shown in the code below, I can successfully create a local symbolic-ref `foo_ptr` to branch `foo`,

Re: [RFC] pack-objects: compression level for non-blobs

2012-12-31 Thread Shawn Pearce
This thread is pretty interesting. Unfortunately the holidays have kept me busy. But I am excited by the work David and Peff are doing. :-) On Sun, Dec 30, 2012 at 1:31 PM, Jeff King p...@peff.net wrote: On Sun, Dec 30, 2012 at 07:53:48PM +0700, Nguyen Thai Ngoc Duy wrote: $ cd objects/pack

Re: [RFC] pack-objects: compression level for non-blobs

2013-01-01 Thread Shawn Pearce
On Tue, Jan 1, 2013 at 4:10 AM, Duy Nguyen pclo...@gmail.com wrote: On Tue, Jan 1, 2013 at 11:15 AM, Duy Nguyen pclo...@gmail.com wrote: Fix pack-objects to behave the way JGit does, cluster commits first in the pack stream. Now you have a dense space of commits. If I remember right this has a

Re: [PATCH 0/2] optimizing pack access on read only fetch repos

2013-01-29 Thread Shawn Pearce
On Sat, Jan 26, 2013 at 10:32 PM, Junio C Hamano gits...@pobox.com wrote: Jeff King p...@peff.net writes: This is a repost from here: http://thread.gmane.org/gmane.comp.version-control.git/211176 which got no response initially. Basically the issue is that read-only repos (e.g., a CI

Re: [PATCH 1/2] upload-pack: avoid parsing objects during ref advertisement

2013-01-29 Thread Shawn Pearce
On Fri, Jan 6, 2012 at 11:17 AM, Jeff King p...@peff.net wrote: When we advertise a ref, the first thing we do is parse the pointed-to object. This gives us two things: ... The downside is that we are no longer verifying objects that we advertise by fully parsing them (however, we do still

Re: [PATCH 0/2] optimizing pack access on read only fetch repos

2013-01-31 Thread Shawn Pearce
On Tue, Jan 29, 2013 at 1:19 PM, Jeff King p...@peff.net wrote: On Tue, Jan 29, 2013 at 07:58:01AM -0800, Junio C Hamano wrote: The point is not about space. Disk is cheap, and it is not making it any worse than what happens to your target audience, that is a fetch-only repository with only

Re: [PATCH 4/6] introduce a commit metapack

2013-01-31 Thread Shawn Pearce
On Wed, Jan 30, 2013 at 7:56 AM, Junio C Hamano gits...@pobox.com wrote: Jeff King p...@peff.net writes: From this: Then it will be very natural for the extension data that store the commit metainfo to name objects in the pack the .idx file describes by the offset in the SHA-1 table. I

Re: [PATCH/RFC 0/6] commit caching

2013-01-31 Thread Shawn Pearce
On Tue, Jan 29, 2013 at 1:14 AM, Jeff King p...@peff.net wrote: This is the cleaned-up version of the commit caching patches I mentioned here: http://article.gmane.org/gmane.comp.version-control.git/212329 ... The short of it is that for an extra 31M of disk space (~4%), I get a warm-cache

  1   2   3   >