On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: > Roché Compaan wrote: > > On Mon, 2008-02-25 at 07:36 +0200, Roché Compaan wrote: > >> I'll update my blog post with the final stats and let you know when it > >> is ready. > >> > > > > I'll have to keep running these tests because the more I run them the > > faster the ZODB becomes ;-) Would you have guessed that the ZODB is > > faster at both insertion and lookups than Postgres? > > > > http://www.upfrontsystems.co.za/Members/roche/where-im-calling-from/zodb-benchmarks-revisited > > For some workloads, ZODB is definitely faster. However, I think your > analysis needs to provide more detail:
I posted some of the details earlier in this thread but I'll put them up on the post as well. > - How much RAM did ZODB and Postgres consume during the tests? Don't know, will check. > - How often are you committing a transaction? 100 inserts per transaction. > - Did you use optimal methods of insertion in Postgres, such as COPY? > Also note that a standard way to insert a lot of data into a relational > database is to temporarily drop indexes and re-create them after > insertion. Your original test may be more valid than you thought. I don't think that this describes the typical interaction between an application and a database. Usually records will be added to Postgres using INSERT. A goal of the benchmarks is to understand the limitations for applications that use the ZODB and to challenge the idea that an RDBMS should be used for applications that, in naive terms, require a "big" database that can write fast. > - Did you use optimal methods of retrieval in Postgres? It is > frequently not necessary to pull the data into the application. Copying > to another table could be faster than fetching rows. I realise this. The only thing the lookup stats tell me is that lookups in the ZODB don't need drastic optimisation, it is already very fast. > - What is the ZODB cache size? How much does the speed change as you > change the cache size? A lot! With the default cache size the insert rate is a mere 50 inserts/second at around 10 million objects. I used a cache size of 100000 in the benchmarks. > - How much disk space does each database consume when there are 10M objects? ZODB: 19GB How do you check the size of a Postgres database? > > > Lookups are even faster then what I originally reported. Lookup times > > averages at 2 milliseconds (0.002s) on 10 million objects. > > Maybe if you set up a ZODB cache that allows just over 10 million > objects, the lookup time will drop to microseconds. You might need a > lot of RAM to do that, though. Maybe, but somehow I think that disk IO will prevent this. I'll check. > > I think somebody else should run these tests as well and validate the > > methodology behind them, otherwise I'm spreading lies. > > You're not far from something interesting. > > On a related topic, you might be interested in the RelStorage > performance charts I just posted. Don't take them too seriously, but I > think the charts are useful. > > http://shane.willowrise.com/archives/relstorage-10-and-measurements/ Thanks for all your questions. I'll certainly post the missing detail on the web and investigate some of the things that might affect performance. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
