Re: [PERFORM] Problem with 11 M records table

2008-05-14 Thread Ramasubramanian G
Hi , Set this parameter in psotgresql.conf set enable_seqscan=off; And try: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Scott Marlowe Sent: Tuesday, May 13, 2008 11:32 PM To: idc danny Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM]

Re: [PERFORM] Regexps - never completing join.

2008-05-14 Thread Rusty Conover
On May 13, 2008, at 11:45 PM, Rusty Conover wrote: Hi Guys, I'm using postgresql 8.3.1 and I'm seeing weird behavior between what I expect and what's happening when the query is executed I'm trying to match a table that contains regexps against another table that is full of the text

Re: [PERFORM] can I move sort to first outer join ?

2008-05-14 Thread PFC
On Wed, 14 May 2008 06:40:40 +0200, fernando castano [EMAIL PROTECTED] wrote: Hi all, This sql is taking too long for the size of my tiny db. Any tips from this alias? I tried moving the sort to the first left outer join (between projects and features tables) using a nested subquery,

[PERFORM] postgres overall performance seems to degrade when large SELECT are requested

2008-05-14 Thread Valentin Bogdanov
HI, I have an application that maintains 150 open connections to a Postgres DB server. The application works fine without a problem for the most time. The problem seem to arise when a SELECT that returns a lot of rows is executed or the SELECT is run on a large object. These selects are run

Re: [PERFORM] postgres overall performance seems to degrade when large SELECT are requested

2008-05-14 Thread PFC
The problem seem to arise when a SELECT that returns a lot of rows is Does the SELECT return a lot of rows, or does it scan a lot of rows ? (for instance, if you use aggregates, it might scan lots of data but only return few rows). The problem is that when the SELECTs are run the main

Re: [PERFORM] Regexps - never completing join.

2008-05-14 Thread Rusty Conover
Returning to this problem this morning, I made some more insight. The regexp cache isn't getting very many hits because the executor is looping through all of the classification rows then looping through all of the regular expressions, causing each expression to be recompiled every time

[PERFORM] I/O on select count(*)

2008-05-14 Thread Doug Eck
I have a large table (~ 2B rows) that contains an indexed timestamp column. I am attempting to run a query to determine the number of rows for a given day using something like select count(*) from tbl1 where ts between '2008-05-12 00:00:00.000' and '2008-05-12 23:59:59.999'. Explain tells me

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Merlin Moncure
On Wed, May 14, 2008 at 4:09 PM, Doug Eck [EMAIL PROTECTED] wrote: I have a large table (~ 2B rows) that contains an indexed timestamp column. I am attempting to run a query to determine the number of rows for a given day using something like select count(*) from tbl1 where ts between

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Doug Eck
- Original Message From: Merlin Moncure [EMAIL PROTECTED] To: Doug Eck [EMAIL PROTECTED] Cc: pgsql-performance@postgresql.org Sent: Wednesday, May 14, 2008 3:38:23 PM Subject: Re: [PERFORM] I/O on select count(*) On Wed, May 14, 2008 at 4:09 PM, Doug Eck [EMAIL PROTECTED] wrote: I

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Kevin Grittner
Doug Eck [EMAIL PROTECTED] wrote: I am attempting to run a query to determine the number of rows for a given day using something like select count(*) from tbl1 where ts between '2008-05-12 00:00:00.000' and '2008-05-12 23:59:59.999'. Explain tells me that the query will be done using

[PERFORM] poor row estimates with multi-column joins

2008-05-14 Thread Robert Treat
The following query produces some fairly off estimates for the number of rows that should be returned (this is based on a much more complex query, but whittling it down to this which seems to be the heart of the problem) peii=# explain analyze select * from adv.peii_fast_lookup pfl1 join

[PERFORM] Update performance degrades over time

2008-05-14 Thread Subbiah Stalin-XCGF84
Hi All, We are doing some load tests with our application running postgres 8.2.4. At times we see updates on a table taking longer (around 11-16secs) than expected sub-second response time. The table in question is getting updated constantly through the load tests. In checking the table size

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Greg Smith
On Wed, 14 May 2008, Kevin Grittner wrote: If this is the first time that the rows are being read since they were inserted (or since the database was loaded, including from backup), it may be rewriting the rows to set hint bits, which can make subsequent access faster. This is the second time

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Alvaro Herrera
Greg Smith wrote: On Wed, 14 May 2008, Kevin Grittner wrote: If this is the first time that the rows are being read since they were inserted (or since the database was loaded, including from backup), it may be rewriting the rows to set hint bits, which can make subsequent access faster.

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Greg Smith
On Wed, 14 May 2008, Alvaro Herrera wrote: If neither of the bits is set, then the transaction is either in progress (which you can check by examining the list of running transactions in shared memory) or your process is the first one to check (in which case, you need to consult pg_clog to

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Jan de Visser
On 5/14/08, Greg Smith [EMAIL PROTECTED] wrote: On Wed, 14 May 2008, Alvaro Herrera wrote: If neither of the bits is set, then the transaction is either in progress (which you can check by examining the list of running transactions in shared memory) or your process is the first one to check

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Pavan Deolasee
On Thu, May 15, 2008 at 7:51 AM, Greg Smith [EMAIL PROTECTED] wrote: So is vacuum helpful here because it will force all that to happen in one batch? To put that another way: if I've run a manual vacuum, is it true that it will have updated all the hint bits to XMIN_COMMITTED for all the

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Luke Lonergan
BTW ­ we¹ve removed HINT bit checking in Greenplum DB and improved the visibility caching which was enough to provide performance at the same level as with the HINT bit optimization, but avoids this whole ³write the data, write it to the log also, then write it again just for good measure²

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Greg Smith
On Thu, 15 May 2008, Pavan Deolasee wrote: I had suggested in the past that whenever we set hint bits for a tuple, we should check all other tuples in the page and set their hint bits too to avoid multiple writes of the same page. I guess the idea got rejected because of lack of benchmarks to

Re: [PERFORM] I/O on select count(*)

2008-05-14 Thread Joshua D. Drake
On Thu, 15 May 2008 10:52:01 +0800 Luke Lonergan [EMAIL PROTECTED] wrote: BTW ­ we¹ve removed HINT bit checking in Greenplum DB and improved the visibility caching which was enough to provide performance at the same level as with the HINT bit optimization, but avoids this whole ³write the