Re: [PERFORM] RAID arrays and performance

2008-09-19 Thread Matthew Wakeling
On Thu, 18 Sep 2008, Greg Smith wrote: It's just that our system is doing a lot of bitmap index scans at the moment, and it'd help to be able to spread them across the 16 discs in the RAID array. It's the bottleneck in our system at the moment. If you have some specific bitmap index scan test

Re: [PERFORM] RAID arrays and performance

2008-09-19 Thread Tom Lane
Matthew Wakeling [EMAIL PROTECTED] writes: In order to improve the performance, I made the system look ahead in the source, in groups of a thousand entries, so instead of running: SELECT * FROM table WHERE field = 'something'; a thousand times, we now run: SELECT * FROM table WHERE field IN

Re: [PERFORM] RAID arrays and performance

2008-09-19 Thread Matthew Wakeling
On Fri, 19 Sep 2008, Tom Lane wrote: Your example shows the IN-list as being sorted, but I wonder whether you actually are sorting the items in practice? If not, you might try that to improve locality of access to the index. Well, like I said, we generally don't have the luxury of dictating

Re: [PERFORM] RAID arrays and performance

2008-09-19 Thread Mark Mielke
Matthew Wakeling [EMAIL PROTECTED] writes: In order to improve the performance, I made the system look ahead in the source, in groups of a thousand entries, so instead of running: SELECT * FROM table WHERE field = 'something'; a thousand times, we now run: SELECT * FROM table WHERE field IN

Re: [PERFORM] why does this use the wrong index?

2008-09-19 Thread Jeff Davis
So, What can I do to encourage Postgres to use the first index even when the date range is smaller. It looks like PostgreSQL is estimating the selectivity of your date ranges poorly. In the second (bad) plan it estimates that the index scan with the filter will return 1 row (and that's

Re: [PERFORM] why does this use the wrong index?

2008-09-19 Thread Jeff Davis
On Fri, 2008-09-19 at 11:25 -0700, Jeff Davis wrote: What's the n_distinct for start_time? Actually, I take that back. Apparently, PostgreSQL can't change x BETWEEN y AND y into x=y, so PostgreSQL can't use n_distinct at all. That's your problem. If it's one day only, change it to equality and

Re: [PERFORM] Intel's X25-M SSD

2008-09-19 Thread Greg Smith
On Mon, 8 Sep 2008, Merlin Moncure wrote: What's interesting about the X25 is that they managed to pull the numbers they got out of a MLC flash product. They managed this with a DRAM buffer and the custom controller. I finally found a good analysis of what's wrong with most of the cheap MLC