Re: [ZODB-Dev] Blob directory structure scalability limits

2008-06-21 Thread Christian Theune
On Fri, Jun 20, 2008 at 07:35:10PM +0200, Christian Theune wrote: > [...] and I'm working on an offline migration script that should be > completed by tomorrow. The offline migration script is completed. I'm currently trying to make the tests run on windows. The code probably does run, but I get

Re: [ZODB-Dev] Blob directory structure scalability limits

2008-06-20 Thread Christian Theune
Hi, On Fri, Jun 20, 2008 at 07:55:17PM +0200, Andreas Jung wrote: > > > --On 19. Juni 2008 14:31:47 +0200 Martijn Faassen > <[EMAIL PROTECTED]> wrote: > >> Hi there, >> >> On Thu, Jun 19, 2008 at 1:38 PM, Christian Theune <[EMAIL PROTECTED]> wrote: >> [snip] >>> We propose to keep both implementa

Re: [ZODB-Dev] Blob directory structure scalability limits

2008-06-20 Thread Andreas Jung
--On 19. Juni 2008 14:31:47 +0200 Martijn Faassen <[EMAIL PROTECTED]> wrote: Hi there, On Thu, Jun 19, 2008 at 1:38 PM, Christian Theune <[EMAIL PROTECTED]> wrote: [snip] We propose to keep both implementations around and allow to select which one to use. We would extend the FileSystemHelp

Re: [ZODB-Dev] Blob directory structure scalability limits

2008-06-20 Thread Christian Theune
Me again, On Thu, Jun 19, 2008 at 01:38:38PM +0200, Christian Theune wrote: > Hi, > > We propose to introduce a new mode for the blob storage which breaks the > directory structure into one level per byte of the oid. This would lead to > directories 0x00-0xFF nested in 8 levels. I implemented a

Re: [ZODB-Dev] Blob directory structure scalability limits

2008-06-19 Thread Christian Theune
Hi, On Thu, Jun 19, 2008 at 09:29:13AM -0400, Jim Fulton wrote: >> We propose to introduce a new mode for the blob storage which breaks >> the >> directory structure into one level per byte of the oid. This would >> lead to >> directories 0x00-0xFF nested in 8 levels. > > +1 > > ... > >> We pr

Re: [ZODB-Dev] Blob directory structure scalability limits

2008-06-19 Thread Jim Fulton
On Jun 19, 2008, at 7:38 AM, Christian Theune wrote: Hi, one of our installations hit a scalability limits with the current blob directory structure. The current structure looks like: \blobs\ \ .blob .blob ... \ ... ...\ We hit a limit with a database

Re: [ZODB-Dev] Blob directory structure scalability limits

2008-06-19 Thread Martijn Faassen
Hi there, On Thu, Jun 19, 2008 at 1:38 PM, Christian Theune <[EMAIL PROTECTED]> wrote: [snip] > We propose to keep both implementations around and allow to select which one > to use. We would extend the FileSystemHelper to abstract the two strategies. Could the default be to choose the more scala

Re: [ZODB-Dev] Blob directory structure scalability limits

2008-06-19 Thread Christian Theune
On Thu, Jun 19, 2008 at 08:00:27AM -0400, Benji York wrote: > On Thu, Jun 19, 2008 at 7:38 AM, Christian Theune <[EMAIL PROTECTED]> wrote: > > We propose to introduce a new mode for the blob storage which breaks the > > directory structure into one level per byte of the oid. This would lead to > >

Re: [ZODB-Dev] Blob directory structure scalability limits

2008-06-19 Thread Benji York
On Thu, Jun 19, 2008 at 7:38 AM, Christian Theune <[EMAIL PROTECTED]> wrote: > We propose to introduce a new mode for the blob storage which breaks the > directory structure into one level per byte of the oid. This would lead to > directories 0x00-0xFF nested in 8 levels. I suppose the directories