Re: How git affects kernel.org performance

2007-01-12 Thread Nigel Cunningham
Hi. On Wed, 2007-01-10 at 22:07 +0800, Fengguang Wu wrote: > Thanks, Nigel. > But I'm very sorry that the calculation in the patch was wrong. > > Would you give this new patch a run? Sorry for my slowness. I just did time find /usr/src | wc -l again: Without patch: 35.137, 35.104, 35.351 seco

Re: How git affects kernel.org performance

2007-01-10 Thread Fengguang Wu
On Wed, Jan 10, 2007 at 02:20:49PM +1100, Nigel Cunningham wrote: > Hi. > > On Wed, 2007-01-10 at 09:57 +0800, Fengguang Wu wrote: > > On Tue, Jan 09, 2007 at 08:23:32AM -0800, Linus Torvalds wrote: > > > > > > > > > On Tue, 9 Jan 2007, Fengguang Wu wrote: > > > > > > > > > > The fastest and proba

Re: How git affects kernel.org performance

2007-01-09 Thread Nigel Cunningham
Hi. On Wed, 2007-01-10 at 09:57 +0800, Fengguang Wu wrote: > On Tue, Jan 09, 2007 at 08:23:32AM -0800, Linus Torvalds wrote: > > > > > > On Tue, 9 Jan 2007, Fengguang Wu wrote: > > > > > > > > The fastest and probably most important thing to add is some readahead > > > > smarts to directories ---

Re: How git affects kernel.org performance

2007-01-09 Thread Fengguang Wu
On Tue, Jan 09, 2007 at 08:23:32AM -0800, Linus Torvalds wrote: > > > On Tue, 9 Jan 2007, Fengguang Wu wrote: > > > > > > The fastest and probably most important thing to add is some readahead > > > smarts to directories --- both to the htree and non-htree cases. If > > > > Here's is a quick hack

Re: How git affects kernel.org performance

2007-01-09 Thread Linus Torvalds
On Tue, 9 Jan 2007, Fengguang Wu wrote: > > > > The fastest and probably most important thing to add is some readahead > > smarts to directories --- both to the htree and non-htree cases. If > > Here's is a quick hack to practice the directory readahead idea. > Comments are welcome, it's a fre

Re: How git affects kernel.org performance

2007-01-09 Thread Fengguang Wu
On Mon, Jan 08, 2007 at 07:58:19AM -0500, Theodore Tso wrote: > On Mon, Jan 08, 2007 at 08:35:55AM +0530, Suparna Bhattacharya wrote: > > > Yeah, slowly-growing directories will get splattered all over the disk. > > > > > > Possible short-term fixes would be to just allocate up to (say) eight > >

Re: How git affects kernel.org performance

2007-01-08 Thread Jeremy Higdon
On Mon, Jan 08, 2007 at 05:09:34PM -0800, Paul Jackson wrote: > Jeff wrote: > > Something I just thought of: ATA and SCSI hard disks do their own > > read-ahead. > > Probably this is wishful thinking on my part, but I would have hoped > that most of the read-ahead they did was for stuff that happ

Re: How git affects kernel.org performance

2007-01-08 Thread Paul Jackson
Jeff wrote: > Something I just thought of: ATA and SCSI hard disks do their own > read-ahead. Probably this is wishful thinking on my part, but I would have hoped that most of the read-ahead they did was for stuff that happened to be on the cylinder they were reading anyway. So long as their rea

Re: How git affects kernel.org performance

2007-01-08 Thread Theodore Tso
On Mon, Jan 08, 2007 at 02:59:52PM +0100, Pavel Machek wrote: > Hi! > > > > Would e2fsck -D help? What kind of optimization > > > does it perform? > > > > It will help a little; e2fsck -D compresses the logical view of the > > directory, but it doesn't optimize the physical layout on disk at all,

Re: How git affects kernel.org performance

2007-01-08 Thread Johannes Stezenbach
On Mon, Jan 08, 2007 at 07:58:19AM -0500, Theodore Tso wrote: > > The fastest and probably most important thing to add is some readahead > smarts to directories --- both to the htree and non-htree cases. If > you're using some kind of b-tree structure, such as XFS does for > directories, prealloc

Re: How git affects kernel.org performance

2007-01-08 Thread Pavel Machek
Hi! > > Would e2fsck -D help? What kind of optimization > > does it perform? > > It will help a little; e2fsck -D compresses the logical view of the > directory, but it doesn't optimize the physical layout on disk at all, > and of course, it won't help with the lack of readahead logic. It's > po

Re: How git affects kernel.org performance

2007-01-08 Thread Theodore Tso
On Mon, Jan 08, 2007 at 02:41:47PM +0100, Johannes Stezenbach wrote: > > Would e2fsck -D help? What kind of optimization > does it perform? It will help a little; e2fsck -D compresses the logical view of the directory, but it doesn't optimize the physical layout on disk at all, and of course, it

Re: How git affects kernel.org performance

2007-01-08 Thread Jeff Garzik
Theodore Tso wrote: The fastest and probably most important thing to add is some readahead smarts to directories --- both to the htree and non-htree cases. If you're using some kind of b-tree structure, such as XFS does for directories, preallocation doesn't help you much. Delayed allocation ca

Re: How git affects kernel.org performance

2007-01-08 Thread Theodore Tso
On Mon, Jan 08, 2007 at 08:35:55AM +0530, Suparna Bhattacharya wrote: > > Yeah, slowly-growing directories will get splattered all over the disk. > > > > Possible short-term fixes would be to just allocate up to (say) eight > > blocks when we grow a directory by one block. Or teach the > > direct

Re: How git affects kernel.org performance

2007-01-07 Thread Suparna Bhattacharya
On Sun, Jan 07, 2007 at 01:15:42AM -0800, Andrew Morton wrote: > On Sun, 7 Jan 2007 09:55:26 +0100 > Willy Tarreau <[EMAIL PROTECTED]> wrote: > > > On Sat, Jan 06, 2007 at 09:39:42PM -0800, Linus Torvalds wrote: > > > > > > > > > On Sat, 6 Jan 2007, H. Peter Anvin wrote: > > > > > > > > During ext

Re: How git affects kernel.org performance

2007-01-07 Thread Rene Herman
On 01/07/2007 10:15 AM, Andrew Morton wrote: Yeah, slowly-growing directories will get splattered all over the disk. Possible short-term fixes would be to just allocate up to (say) eight blocks when we grow a directory by one block. Or teach the directory-growth code to use ext3 reservations

Re: How git affects kernel.org performance

2007-01-07 Thread Andrew Morton
On Sun, 7 Jan 2007 09:55:26 +0100 Willy Tarreau <[EMAIL PROTECTED]> wrote: > On Sat, Jan 06, 2007 at 09:39:42PM -0800, Linus Torvalds wrote: > > > > > > On Sat, 6 Jan 2007, H. Peter Anvin wrote: > > > > > > During extremely high load, it appears that what slows kernel.org down > > > more > > >