Re: [Gnu-arch-users] Re: [GNU-arch-dev] [ANNOUNCEMENT] /Arch/ embraces `git'

2005-04-21 Thread Tom Lord

   Using your suggested indexing method that uses [0:4] as the 1st level key 
and
 [0:3]
   [4:8] as the 2nd level key, I obtain an indexed archive that occupies 159M,
   where the top level contains 18665 1st level keys, the largest first level 
dir
   contains 5 entries, and all 2nd level dirs contain exactly 1 entry.


That's just a mistake in the spec.  The format should probably be
multi-level but, yes, the fanout I suggested is currently quite bogus.
When I write that part of that code (today or tomorrow) I'll fix it.

A few people pointed that out.  Thanks.

-t

-
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Gnu-arch-users] Re: [GNU-arch-dev] [ANNOUNCEMENT] /Arch/ embraces `git'

2005-04-21 Thread Tom Lord


Yes, it really doesn't make much sense to have so big keys on the
directories.

It's official... i'm blushing wildly thank you for the various
replies that pointed out my thinko.

That part of my spec hasn't been coded yet --- i just wrote text.  It
really was the silly late-night error of sort: hmm...let's see, 4 hex
digits plus 4 hex digits  that's 16 bits sounds about right.

Really, I'll fix it.

-t
-
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Gnu-arch-users] Re: [GNU-arch-dev] [ANNOUNCEMENT] /Arch/ embraces `git'

2005-04-21 Thread Tom Lord

[your 0:3/4:7 directory hierarchy is horked]

Absolutely.  Made a dumb mistake the night I wrote that spec
and embarassed that I initially defended it.   I had an arithmetic
error.   Thanks, this time, for your persistence in pointing it out.

-t
-
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Gnu-arch-users] Re: [GNU-arch-dev] [ANNOUNCEMENT] /Arch/ embraces `git'

2005-04-21 Thread Tom Lord


Tom, please stop this ext* filesystem bashing ;-) 

For one thing... yes, i'm totally embarassed on this issue.
I made a late-night math error in a spec.  *hopefully* would
have noticed it on my own as I coded to that spec but y'all
have been wonderful at pointing out my mistake to me even 
though I initially defended it.

As for ext* bashing it's not bashing exactly.  I/O bandwidth
gets a little better, disks get a bunch cheaper --- then ext* 
doesn't look bad at all in this respect.  And we're awefully close
to that point.   

Meanwhile, for times measured in years, I've gotten complaints from
ext* users about software that is just fine on other filesystems
over issues like the allocation size of small files.

So ext*, from my perspective, was a little too far ahead of its time
and, yes, my complaints about it are just about reaching their
expiration date.

Anyway, thanks for all the sanity about directory layout.  Really,
it was just an I'm too sleepy to be doing this right now error.

-t

-
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[ANNOUNCEMENT] /Arch/ embraces `git'

2005-04-20 Thread Tom Lord

`git', by Linus Torvalds, contains some very good ideas and some
very entertaining source code -- recommended reading for hackers.

/GNU Arch/ will adopt `git':

From the /Arch/ perspective: `git' technology will form the
basis of a new archive/revlib/cache format and the basis
of new network transports.

From the `git' perspective, /Arch/ will replace the lame directory
cache component of `git' with a proper revision control system.

In my view, the core ideas in `git' are quite profound and deserve
an impeccable implementation.   This is practical because those ideas
are also pretty simple.

I started here:

   http://www.seyza.com/=clients/linus/tree/index.html

and for those interested in `git'-theory, a good place to start is

   http://www.seyza.com/=clients/linus/tree/src/liblob/index.html

(Linus is not literally a client of mine.  That's just the directory 
where this goes.)

-t
-
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[ANNOUNCEMENT] /Arch/ embraces `git'

2005-04-20 Thread Tom Lord

`git', by Linus Torvalds, contains some very good ideas and some
very entertaining source code -- recommended reading for hackers.

/GNU Arch/ will adopt `git':

From the /Arch/ perspective: `git' technology will form the
basis of a new archive/revlib/cache format and the basis
of new network transports.

From the `git' perspective, /Arch/ will replace the lame directory
cache component of `git' with a proper revision control system.

In my view, the core ideas in `git' are quite profound and deserve
an impeccable implementation.   This is practical because those ideas
are also pretty simple.

I started here:

   http://www.seyza.com/=clients/linus/tree/index.html

and for those interested in `git'-theory, a good place to start is

   http://www.seyza.com/=clients/linus/tree/src/liblob/index.html

(Linus is not literally a client of mine.  That's just the directory 
where this goes.)

-t
-
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: on when to checksum

2005-04-20 Thread Tom Lord

   From: Linus Torvalds [EMAIL PROTECTED]

   On Wed, 20 Apr 2005, Tom Lord wrote:

I think you have made a mistake by moving the sha1 checksum from the
zipped form to the inflated form.  Here is why:

   I'd have agreed with you (and I did, violently) if it wasn't for the
   performance issues. It makes a huge difference for write-tree, and to me,
   clearly performance _does_ matter.

   Fractions of seconds may not sound like a lot, but they add up. I work 
   with 200-patch series myself all the time, so I'm very sensitive to a 0.3 
   second difference in performance.


How many times per day do you invoke `write-tree' and why?

It takes a large multiple of `0.3s' to get me to take you seriously
on this point.

I have long harbored the suspician that your perceived bandwidth
implies that you process a lot of patches unread or barely read --
implying that your day-to-day bitslingling could/should largely be
handled by an Arch-style patch-queue-manager (a script).

-t
-
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [GNU-arch-dev] [ANNOUNCEMENT] /Arch/ embraces `git'

2005-04-20 Thread Tom Lord



   From: [EMAIL PROTECTED]

Thank you for your experiment.  I'm not surprised by the 
result but it is very nice to know that my expectations
are right.

I think that to a large extent you are seeing artifacts
of the questionable trade-offs that (reports tell me) the
ext* filesystems make.   With a different filesystem, the 
results would be very different.

I'm imagining a blob database containing may revisions of the linux
kernel.  It will contain millions of blobs.

It's fine that some filesystems and some blob operations work fine
on a directory with millions of files but what about other operations
on the database?   I pity the poor program that has to `readdir' through
millions of files.

That said: I may add an optional flat-directory format to my library,
just to avoid issues such as those you raise over the next couple 
years.

-t

-
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: on when to checksum

2005-04-20 Thread Tom Lord

(I'll have to study/think about that for a while before a proper
reply.  Tomorrow, probably.)

Thanks,
-t

-
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html