Re: git notes show is orders of magnitude slower than doing it manually with ls-tree and cat-file

2014-11-26 Thread Johan Herland
(First of all, thanks to both for great investigation and analysis) On Wed, Nov 26, 2014 at 5:46 AM, Jeff King p...@peff.net wrote: On Wed, Nov 26, 2014 at 11:25:53AM +0900, Mike Hommey wrote: Now, looking at the notes tree reflog, I see that at some point, some notes were added at the

Re: git notes show is orders of magnitude slower than doing it manually with ls-tree and cat-file

2014-11-26 Thread Mike Hommey
On Wed, Nov 26, 2014 at 12:46:20PM +0100, Johan Herland wrote: (First of all, thanks to both for great investigation and analysis) On Wed, Nov 26, 2014 at 5:46 AM, Jeff King p...@peff.net wrote: On Wed, Nov 26, 2014 at 11:25:53AM +0900, Mike Hommey wrote: Now, looking at the notes tree

git notes show is orders of magnitude slower than doing it manually with ls-tree and cat-file

2014-11-25 Thread Mike Hommey
Hi, I have a note tree with a bit more than 200k notes. $ time git notes --ref foo show $sha1 /dev/null real0m0.147s user0m0.136s sys 0m0.008s That's a lot of time, especially when you have a script that does that on a fair amount of sha1s. Now, the interesting thing is this: $

Re: git notes show is orders of magnitude slower than doing it manually with ls-tree and cat-file

2014-11-25 Thread Jeff King
On Wed, Nov 26, 2014 at 09:42:42AM +0900, Mike Hommey wrote: I have a note tree with a bit more than 200k notes. $ time git notes --ref foo show $sha1 /dev/null real0m0.147s user0m0.136s sys 0m0.008s That's a lot of time, especially when you have a script that does that on

Re: git notes show is orders of magnitude slower than doing it manually with ls-tree and cat-file

2014-11-25 Thread Jeff King
On Tue, Nov 25, 2014 at 08:00:51PM -0500, Jeff King wrote: On Wed, Nov 26, 2014 at 09:42:42AM +0900, Mike Hommey wrote: I have a note tree with a bit more than 200k notes. $ time git notes --ref foo show $sha1 /dev/null real0m0.147s user0m0.136s sys 0m0.008s

Re: git notes show is orders of magnitude slower than doing it manually with ls-tree and cat-file

2014-11-25 Thread Jeff King
On Tue, Nov 25, 2014 at 08:24:48PM -0500, Jeff King wrote: However, this is not what trees created by git-notes look like. It shards the object sha1s into subtrees (1a/2b/{36}), and I think does so dynamically in a way that keeps each individual tree size low. The in-memory data structure

Re: git notes show is orders of magnitude slower than doing it manually with ls-tree and cat-file

2014-11-25 Thread Mike Hommey
On Tue, Nov 25, 2014 at 08:24:49PM -0500, Jeff King wrote: On Tue, Nov 25, 2014 at 08:00:51PM -0500, Jeff King wrote: On Wed, Nov 26, 2014 at 09:42:42AM +0900, Mike Hommey wrote: I have a note tree with a bit more than 200k notes. $ time git notes --ref foo show $sha1 /dev/null

Re: git notes show is orders of magnitude slower than doing it manually with ls-tree and cat-file

2014-11-25 Thread Mike Hommey
On Tue, Nov 25, 2014 at 08:34:57PM -0500, Jeff King wrote: On Tue, Nov 25, 2014 at 08:24:48PM -0500, Jeff King wrote: However, this is not what trees created by git-notes look like. It shards the object sha1s into subtrees (1a/2b/{36}), and I think does so dynamically in a way that keeps

Re: git notes show is orders of magnitude slower than doing it manually with ls-tree and cat-file

2014-11-25 Thread Jeff King
On Wed, Nov 26, 2014 at 11:25:53AM +0900, Mike Hommey wrote: Now, looking at the notes tree reflog, I see that at some point, some notes were added at the top-level of the tree, without being nested, which is strange. That's somewhat expected. The fanout is dynamic based on the number of

Re: git notes show is orders of magnitude slower than doing it manually with ls-tree and cat-file

2014-11-25 Thread Jeff King
On Wed, Nov 26, 2014 at 11:30:39AM +0900, Mike Hommey wrote: Hmph. Having just written all that, I looked at your example again, and you are running git ls-tree -r, which would read the whole tree anyway. So git notes should be _faster_ for a single lookup. The -r actually doesn't