Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-23 Thread Kris Jurka
On Thu, 23 Sep 2004, Stephen Crowley wrote: > Thanks for the explanation. So what sort of changes need to be made to > the client/server protocol to fix this problem? The problem is that there is no way to indicate why you are using a particular statement in the extended query protocol. For t

Re: [PERFORM] Large # of rows in query extremely slow, not using index

2004-09-23 Thread Stephen Crowley
Thanks for the explanation. So what sort of changes need to be made to the client/server protocol to fix this problem? On Thu, 23 Sep 2004 18:22:15 -0500 (EST), Kris Jurka <[EMAIL PROTECTED]> wrote: > > > On Tue, 14 Sep 2004, Stephen Crowley wrote: > > > Problem solved.. I set the fetchSize t

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-23 Thread Kris Jurka
On Tue, 14 Sep 2004, Stephen Crowley wrote: > Problem solved.. I set the fetchSize to a reasonable value instead of > the default of unlimited in the PreparedStatement and now the query > is . After some searching it seeems this is a common problem, would it > make sense to change the default v

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-20 Thread Manfred Koizar
On Fri, 17 Sep 2004 19:23:44 -0500, Stephen Crowley <[EMAIL PROTECTED]> wrote: >Seq Scan [...] rows=265632 > Filter: ((date = '2004-09-07'::date) AND ((stock)::text = 'MSFT'::text)) >Total runtime: 412703.000 ms > >random_page_cost and effective_cache_size are both default, 8 and 1000 Usually ran

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-19 Thread Josh Berkus
Stephen, > " -> Seq Scan on island_history (cost=0.00..417867.13 rows=292274 > width=83) (actual time=346759.000..346759.000 rows=10 loops=1)" Take a look at your row comparisons. When was the last time you ran ANALYZE? -- Josh Berkus Aglio Database Solutions San Francisco ---

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-17 Thread Stephen Crowley
Ok.. now I ran "VACUUM FULL' and things seem to be working as they should.. explain analyze select * from history where date='2004-09-07' and stock='MSFT'; Seq Scan on island_history (cost=0.00..275359.13 rows=292274 width=83) (actual time=50.000..411683.000 rows=265632 loops=1) Filter: ((dat

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-17 Thread Manfred Koizar
On Thu, 16 Sep 2004 20:51:11 -0500, Stephen Crowley <[EMAIL PROTECTED]> wrote: >explain analyze select * from history where date='2004-09-07' and >stock='ORCL' LIMIT 10; >" -> Index Scan using island_history_date_stock_time on >island_history (cost=0.00..183099.72 rows=102166 width=83) (actual

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-16 Thread Joshua D. Drake
When I set enable_seqscan to OFF and force everything to use the index every stock I query returns within 100ms, but turn seqscan back ON and its back up to taking several minutes for non-index using plans. Any ideas? --Stephen Try increasing your statistics target and re-running analyze. Try say 1

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-16 Thread Stephen Crowley
Here are some results of explain analyze, I've included the LIMIT 10 because otherwise the resultset would exhaust all available memory. explain analyze select * from history where date='2004-09-07' and stock='ORCL' LIMIT 10; "Limit (cost=0.00..17.92 rows=10 width=83) (actual time=1612.000..170

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-14 Thread Pierre-Frédéric Caillaud
I have a table with ~8 million rows and I am executing a query which should return about ~800,000 rows. The problem is that as soon as I execute the query it absolutely kills my machine and begins swapping for 5 or 6 minutes before it begins returning results. Is postgres trying to load the whole

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-14 Thread Markus Schaber
Hi, Stephen, On Mon, 13 Sep 2004 19:51:22 -0500 Stephen Crowley <[EMAIL PROTECTED]> wrote: > Does postgres cache the entire result set before it begins returning > data to the client? > > I have a table with ~8 million rows and I am executing a query which > should return about ~800,000 rows. Th

Re: [PERFORM] Large # of rows in query extremely slow, not using index

2004-09-13 Thread Stephen Crowley
Problem solved.. I set the fetchSize to a reasonable value instead of the default of unlimited in the PreparedStatement and now the query is . After some searching it seeems this is a common problem, would it make sense to change the default value to something other than 0 in the JDBC driver? If

Re: [PERFORM] Large # of rows in query extremely slow, not using index

2004-09-13 Thread Tom Lane
Stephen Crowley <[EMAIL PROTECTED]> writes: > On Mon, 13 Sep 2004 21:11:07 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >> Stephen Crowley <[EMAIL PROTECTED]> writes: >>> Does postgres cache the entire result set before it begins returning >>> data to the client? >> >> The backend doesn't, but libpq

Re: [PERFORM] Large # of rows in query extremely slow, not using index

2004-09-13 Thread Stephen Crowley
On Mon, 13 Sep 2004 21:11:07 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: > Stephen Crowley <[EMAIL PROTECTED]> writes: > > Does postgres cache the entire result set before it begins returning > > data to the client? > > The backend doesn't, but libpq does, and I think JDBC does too. > > I'd recomm

Re: [PERFORM] Large # of rows in query extremely slow, not using

2004-09-13 Thread Rod Taylor
On Mon, 2004-09-13 at 20:51, Stephen Crowley wrote: > Does postgres cache the entire result set before it begins returning > data to the client? Sometimes you need to be careful as to how the clients treat the data. For example psql will resize columns width on the length (width) of the data ret

Re: [PERFORM] Large # of rows in query extremely slow, not using index

2004-09-13 Thread Tom Lane
Stephen Crowley <[EMAIL PROTECTED]> writes: > Does postgres cache the entire result set before it begins returning > data to the client? The backend doesn't, but libpq does, and I think JDBC does too. I'd recommend using a cursor so you can FETCH a reasonable number of rows at a time. > Also, wh