Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-05 Thread Roché Compaan
On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: Roché Compaan wrote: On Mon, 2008-02-25 at 07:36 +0200, Roché Compaan wrote: I'll update my blog post with the final stats and let you know when it is ready. I'll have to keep running these tests because the more I run them the

Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-05 Thread Roché Compaan
On Tue, 2008-03-04 at 23:00 +0100, Bernd Dorn wrote: On 04.03.2008, at 22:16, Shane Hathaway wrote: Lennart Regebro wrote: On Tue, Mar 4, 2008 at 9:27 PM, Shane Hathaway [EMAIL PROTECTED] wrote: - Did you use optimal methods of retrieval in Postgres? It is frequently not necessary

Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-05 Thread David Pratt
Hi Roche. I figured this out once and it was included in PGStorage so it should be in relstorage also. Take a look at get_db_size method in postgres adapter. relstorage is in the zope repository. Regards, David Roché Compaan wrote: - How much disk space does each database consume when

Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-05 Thread Lennart Regebro
On Tue, Mar 4, 2008 at 10:16 PM, Shane Hathaway [EMAIL PROTECTED] wrote: Not if you're only retrieving intermediate information. Sure. And my point is that in a typical websetting, you don't. You either retrieve data to be displayed, or you insert data from a HTTP post. Massaging data from one

Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-05 Thread Roché Compaan
On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: On a related topic, you might be interested in the RelStorage performance charts I just posted. Don't take them too seriously, but I think the charts are useful. http://shane.willowrise.com/archives/relstorage-10-and-measurements/

Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-05 Thread Shane Hathaway
Roché Compaan wrote: On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: On a related topic, you might be interested in the RelStorage performance charts I just posted. Don't take them too seriously, but I think the charts are useful.

Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-05 Thread Benji York
Roché Compaan wrote: On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: Maybe if you set up a ZODB cache that allows just over 10 million objects, the lookup time will drop to microseconds. You might need a lot of RAM to do that, though. Maybe, but somehow I think that disk IO will

Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-05 Thread David Pratt
Hi Benji. Have you any settings to recommend or use a default. Many thanks. Regards, David Benji York wrote: Roché Compaan wrote: On Tue, 2008-03-04 at 13:27 -0700, Shane Hathaway wrote: Maybe if you set up a ZODB cache that allows just over 10 million objects, the lookup time will drop to

Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-05 Thread Benji York
David Pratt wrote: Hi Benji. Have you any settings to recommend or use a default. Many thanks. For benchmarking? No. Too high and you'll spend a bunch of time swapping to free up space for disk cache; too low and you may not have a large enough disk cache to be effective. -- Benji York

Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-05 Thread Izak Burger
Benji York wrote: If you're on Linux, you can tweak swappiness (/proc/sys/vm/swappiness; http://lwn.net/Articles/83588/) to affect how much RAM is used for the page cache and how much for your process. While we're on that subject. We recently had a box that would take strain for almost no