Re: [PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-04-04 Thread Peter Schuller
Hello, > I'd always do benchmarks with a realistic value of shared_buffers (i.e. > much higher than that). > > Another thought that comes to mind is that the bitmap index scan does > depend on the size of work_mem. > > Try increasing your shared_buffers to a reasonable working value (say > 10%-1

Re: [PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-04-04 Thread Andrew - Supernews
On 2007-04-04, Peter Schuller <[EMAIL PROTECTED]> wrote: >> The next question then is whether anything in your postgres configuration >> is preventing it getting useful performance from the OS. What settings >> have you changed in postgresql.conf? > > The only options not commented out are the foll

Re: [PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-04-04 Thread Dave Cramer
On 4-Apr-07, at 2:01 AM, Peter Schuller wrote: Hello, The next question then is whether anything in your postgres configuration is preventing it getting useful performance from the OS. What settings have you changed in postgresql.conf? The only options not commented out are the followin

Re: [PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-04-03 Thread Peter Schuller
Hello, > The next question then is whether anything in your postgres configuration > is preventing it getting useful performance from the OS. What settings > have you changed in postgresql.conf? The only options not commented out are the following (it's not even tweaked for buffer sizes and such,

Re: [PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-04-02 Thread Andrew - Supernews
On 2007-04-02, Peter Schuller <[EMAIL PROTECTED]> wrote: > I have confirmed that I am seeing expected performance for random > short and highly concurrent reads in one large (> 200 GB) file. The > I/O is done using libaio however, so depending on implementation I > suppose the I/O scheduling behavi

Re: [PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-04-02 Thread Peter Schuller
Hello, > > SELECT * FROM test WHERE value = 'xxx' LIMIT 1000; > > I tested this on a 14-way software raid10 on freebsd, using pg 8.1.6, and > couldn't reproduce anything like it. With one client I get about 200 disk > requests per second, scaling almost exactly linearly for the first 5 or so > cl

Re: [PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-04-02 Thread Peter Schuller
Hello, > If you are dealing with timed data or similar, you may consider to > partition your table(s). Unfortunately this is not the case; the insertion is more or less random (not quite, but for the purpose of this problem it is). Thanks for the pointers though. That is sure to be useful in som

Re: [PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-03-31 Thread Andrew - Supernews
On 2007-03-30, Peter Schuller <[EMAIL PROTECTED]> wrote: [...] > Other than absolute performance, an important goal is to be able to > scale fairly linearly with the number of underlying disk drives. We > are fully willing to take a disk seek per item selected, as long as it > scales. > > To this e

Re: [PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-03-29 Thread Marc Mamin
ilto:[EMAIL PROTECTED] On Behalf Of Peter Schuller Sent: Friday, March 30, 2007 7:17 AM To: pgsql-performance@postgresql.org Subject: [PERFORM] Scaling SELECT:s with the number of disks on a stripe Hello, I am looking to use PostgreSQL for storing some very simple flat data mostly in a single table.

[PERFORM] Scaling SELECT:s with the number of disks on a stripe

2007-03-29 Thread Peter Schuller
Hello, I am looking to use PostgreSQL for storing some very simple flat data mostly in a single table. The amount of data will be in the hundreds of gigabytes range. Each row is on the order of 100-300 bytes in size; in other words, small enough that I am expecting disk I/O to be seek bound (even