Roché Compaan wrote:
> On Fri, 2009-11-13 at 13:33 -0700, Shane Hathaway wrote:
>> Stephan Richter wrote:
>>> http://svn.zope.org/z3c.sharding/trunk
> 
> Great stuff! This approaches scaling a large data set at application
> level though. Don't you think a ZODB storage doing this for you would
> solve the problem more generally?

I think proper construction of horizontally scalable databases must be 
done partly at application level, since a lot of the issues to be solved 
are specific to the application.

> I think that the master index needs to be partitioned as well. In
> benchmarks I performed early last year (http://bit.ly/pSVmd), a BTree
> could only handle about 250 inserts / second when it approached 10
> million objects, so I'm guessing it will be almost unusable at a 100
> million.

Right.  The top level probably ought to be a dynamic hash, not a BTree. 
  I intended z3c.sharding more as a proof of concept.

Shane

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

Reply via email to