Re: [ZODB-Dev] Re: ZODB Benchmarks
Benji York wrote: If you're on Linux, you can tweak swappiness (/proc/sys/vm/swappiness; http://lwn.net/Articles/83588/) to affect how much RAM is used for the page cache and how much for your process. While we're on that subject. We recently had a box that would take strain for almost no reason. You'd copy a bigish file from one place to another and the load average would just soar as the various zope and zeo instances tried to get to the disk. Turns out this machine used the anticipatory io scheduler, which really messes things up. We changed it to deadline, like so: echo deadline > /sys/block/sda/queue/scheduler Performance is a lot better now. Our (not very scientific) tests shows that deadline is also a little better than cfq for running zope. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: ZODB Benchmarks
David Pratt wrote: Hi Benji. Have you any settings to recommend or use a default. Many thanks. For benchmarking? No. Too high and you'll spend a bunch of time swapping to free up space for disk cache; too low and you may not have a large enough disk cache to be effective. -- Benji York Senior Software Engineer Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: ZODB Benchmarks
Hi Benji. Have you any settings to recommend or use a default. Many thanks. Regards, David Benji York wrote: Roché Compaan wrote: On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: Maybe if you set up a ZODB cache that allows just over 10 million objects, the lookup time will drop to microseconds. You might need a lot of RAM to do that, though. Maybe, but somehow I think that disk IO will prevent this. I'll check. If you're on Linux, you can tweak swappiness (/proc/sys/vm/swappiness; http://lwn.net/Articles/83588/) to affect how much RAM is used for the page cache and how much for your process. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: ZODB Benchmarks
Roché Compaan wrote: On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: Maybe if you set up a ZODB cache that allows just over 10 million objects, the lookup time will drop to microseconds. You might need a lot of RAM to do that, though. Maybe, but somehow I think that disk IO will prevent this. I'll check. If you're on Linux, you can tweak swappiness (/proc/sys/vm/swappiness; http://lwn.net/Articles/83588/) to affect how much RAM is used for the page cache and how much for your process. -- Benji York Senior Software Engineer Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: ZODB Benchmarks
Roché Compaan wrote: On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: On a related topic, you might be interested in the RelStorage performance charts I just posted. Don't take them too seriously, but I think the charts are useful. http://shane.willowrise.com/archives/relstorage-10-and-measurements/ One question, if you run the test with concurrent threads, do each thread insert a 100 objects (or a 1 for the second test)? Yes. Is this test available in SVN somewhere? It's in the RelStorage 1.0 release as well as SVN. It's called relstorage/tests/speedtest.py. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: ZODB Benchmarks
On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: > On a related topic, you might be interested in the RelStorage > performance charts I just posted. Don't take them too seriously, but > I > think the charts are useful. > > http://shane.willowrise.com/archives/relstorage-10-and-measurements/ One question, if you run the test with concurrent threads, do each thread insert a 100 objects (or a 1 for the second test)? Is this test available in SVN somewhere? -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: ZODB Benchmarks
On Tue, Mar 4, 2008 at 10:16 PM, Shane Hathaway <[EMAIL PROTECTED]> wrote: > Not if you're only retrieving intermediate information. Sure. And my point is that in a typical websetting, you don't. You either retrieve data to be displayed, or you insert data from a HTTP post. Massaging data from one place of the database to another place in the database is usually nothing you do on a per-request basis. And if you do, then maybe you shouldn't. :) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: ZODB Benchmarks
Hi Roche. I figured this out once and it was included in PGStorage so it should be in relstorage also. Take a look at get_db_size method in postgres adapter. relstorage is in the zope repository. Regards, David Roché Compaan wrote: - How much disk space does each database consume when there are 10M objects? ZODB: 19GB How do you check the size of a Postgres database? ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: ZODB Benchmarks
On Tue, 2008-03-04 at 23:00 +0100, Bernd Dorn wrote: > On 04.03.2008, at 22:16, Shane Hathaway wrote: > > > Lennart Regebro wrote: > >> On Tue, Mar 4, 2008 at 9:27 PM, Shane Hathaway > >> <[EMAIL PROTECTED]> wrote: > >>> - Did you use optimal methods of retrieval in Postgres? It is > >>> frequently not necessary to pull the data into the application. > >>> Copying > >>> to another table could be faster than fetching rows. > >> But is that relevant in this case? Retrieval must reasonably really > >> retrieve the data, not just move it around. :) > > > > Not if you're only retrieving intermediate information. When you > > write an application against a relational database, a lot of the > > intermediate information does not need to be exposed to Python, > > helping performance significantly. > > yes this is the major benefit when using a relational database over > zodb, because zodb has no server side query language, This certainly makes some queries faster in an RDBMS. My first goal was to determine the speed of the most basic operation like insert and lookup. > so the whole > lookup insert comparison does not reflect real world issues. I disagree. It certainly tests some real world use cases. Most notably the use case where you have a very large user base inserting content at a very high rate into the ZODB. I think what is unknown at this stage is what the penalty would be when using ZEO. But you can only know this if you know how fast direct interaction with the ZODB is. Not all applications require ZEO either. > for example in one of our applications we have to calculate neighbours > of users, based on books they have in their bookshelfs. with about > 1 users and each of them having an average of 100-500 books out of > ca. 1 million, the calculation of the neighbours takes seconds when > you have to calculate this on the client, by getting all indexes etc. > we switched to sql and wrote a single sql statement that does exactly > the same comparison which now takes about 300ms. > > your comparisons would only be accurate if comparing relstorage with > filestorage over zeo, because in this case there is no server side > query possible on object attributes . it would be interesting to look > at performance when having 4-10 zodb clients and then compare zeo/ > filestorage against relstorage with postgres. Hopefully the tests are accurate in comparing the speed for basic operations like insertion and lookup. They might be more *relevant* if one performs the same tests using ZEO. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: ZODB Benchmarks
On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: > Roché Compaan wrote: > > On Mon, 2008-02-25 at 07:36 +0200, Roché Compaan wrote: > >> I'll update my blog post with the final stats and let you know when it > >> is ready. > >> > > > > I'll have to keep running these tests because the more I run them the > > faster the ZODB becomes ;-) Would you have guessed that the ZODB is > > faster at both insertion and lookups than Postgres? > > > > http://www.upfrontsystems.co.za/Members/roche/where-im-calling-from/zodb-benchmarks-revisited > > For some workloads, ZODB is definitely faster. However, I think your > analysis needs to provide more detail: I posted some of the details earlier in this thread but I'll put them up on the post as well. > - How much RAM did ZODB and Postgres consume during the tests? Don't know, will check. > - How often are you committing a transaction? 100 inserts per transaction. > - Did you use optimal methods of insertion in Postgres, such as COPY? > Also note that a standard way to insert a lot of data into a relational > database is to temporarily drop indexes and re-create them after > insertion. Your original test may be more valid than you thought. I don't think that this describes the typical interaction between an application and a database. Usually records will be added to Postgres using INSERT. A goal of the benchmarks is to understand the limitations for applications that use the ZODB and to challenge the idea that an RDBMS should be used for applications that, in naive terms, require a "big" database that can write fast. > - Did you use optimal methods of retrieval in Postgres? It is > frequently not necessary to pull the data into the application. Copying > to another table could be faster than fetching rows. I realise this. The only thing the lookup stats tell me is that lookups in the ZODB don't need drastic optimisation, it is already very fast. > - What is the ZODB cache size? How much does the speed change as you > change the cache size? A lot! With the default cache size the insert rate is a mere 50 inserts/second at around 10 million objects. I used a cache size of 10 in the benchmarks. > - How much disk space does each database consume when there are 10M objects? ZODB: 19GB How do you check the size of a Postgres database? > > > Lookups are even faster then what I originally reported. Lookup times > > averages at 2 milliseconds (0.002s) on 10 million objects. > > Maybe if you set up a ZODB cache that allows just over 10 million > objects, the lookup time will drop to microseconds. You might need a > lot of RAM to do that, though. Maybe, but somehow I think that disk IO will prevent this. I'll check. > > I think somebody else should run these tests as well and validate the > > methodology behind them, otherwise I'm spreading lies. > > You're not far from something interesting. > > On a related topic, you might be interested in the RelStorage > performance charts I just posted. Don't take them too seriously, but I > think the charts are useful. > > http://shane.willowrise.com/archives/relstorage-10-and-measurements/ Thanks for all your questions. I'll certainly post the missing detail on the web and investigate some of the things that might affect performance. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
