Re: [PERFORM] NOT IN 2hrs vs EXCEPT 2 sec.

2009-01-29 Thread Kevin Traster
On Wed, Jan 28, 2009 at 11:37 PM, Scott Marlowe scott.marl...@gmail.comwrote: On Thu, Jan 29, 2009 at 12:01 AM, Kevin Traster ke...@mffais.com wrote: 2 questions: 1) Different costs for same actions. Doing an explain on 2 nearly identical queries both involving the same Index scan on

Re: [PERFORM] NOT IN 2hrs vs EXCEPT 2 sec.

2009-01-29 Thread Kevin Traster
On Wed, Jan 28, 2009 at 11:37 PM, Scott Marlowe scott.marl...@gmail.comwrote: On Thu, Jan 29, 2009 at 12:01 AM, Kevin Traster ke...@mffais.com wrote: 2 questions: 1) Different costs for same actions. Doing an explain on 2 nearly identical queries both involving the same Index scan on

Re: [PERFORM] NOT IN 2hrs vs EXCEPT 2 sec.

2009-01-29 Thread Gregory Stark
Kevin Traster ke...@mffais.com writes: Regarding the previous posts about the same issues of PERFORMENCE between NOT IN versus EXCEPT. There has not been any answer to explain it - just talk about the differenences between the two results. Yes, I can still get the results using EXCEPT but it

Re: [PERFORM] NOT IN 2hrs vs EXCEPT 2 sec.

2009-01-29 Thread Tom Lane
Kevin Traster ke...@mffais.com writes: Unique (cost=3506.21..303375872.86 rows=71946 width=8) - Index Scan using cik_ciknum_idx on cik (cost=3506.21..303375616.75 rows=102444 width=8) Filter: (NOT (subplan)) SubPlan - Materialize (cost=3506.21..6002.40

[PERFORM] Max on union

2009-01-29 Thread anders.blaagaard
Hi, If I have a view like: create view X as ( select x from A union all select x from B) and do select max(x) from X I get a plan like: Aggregate Append Seq Scan on A Seq Scan on B If A and B are indexed on x, I can get the result much faster as: select max(x) from (

Re: [PERFORM] LIKE Query performance

2009-01-29 Thread Hari, Balaji
Is there a patch to make Wildspeed work with postgresql version 8.3.1? P.S My bad, the version number was incorrect in my previous mail. -Original Message- From: Oleg Bartunov [mailto:o...@sai.msu.su] Sent: Wednesday, January 28, 2009 1:27 AM To: Hari, Balaji Cc:

Re: [PERFORM] LIKE Query performance

2009-01-29 Thread Hari, Balaji
We won't need full text searching capabilities as in documents as the data type is varchar. Wildspeed will exactly fit our needs. -Original Message- From: Oleg Bartunov [mailto:o...@sai.msu.su] Sent: Wednesday, January 28, 2009 1:27 AM To: Hari, Balaji Cc:

Re: [PERFORM] LIKE Query performance

2009-01-29 Thread Oleg Bartunov
On Thu, 29 Jan 2009, Hari, Balaji wrote: Is there a patch to make Wildspeed work with postgresql version 8.3.1? unfortunately, no. P.S My bad, the version number was incorrect in my previous mail. -Original Message- From: Oleg Bartunov [mailto:o...@sai.msu.su] Sent: Wednesday,

Re: [PERFORM] Max on union

2009-01-29 Thread Robert Haas
On Thu, Jan 29, 2009 at 10:58 AM, anders.blaaga...@nordea.com wrote: Hi, If I have a view like: create view X as ( select x from A union all select x from B) and do select max(x) from X I get a plan like: Aggregate Append Seq Scan on A Seq Scan on B If A and B are

[PERFORM] Using multiple cores for index creation?

2009-01-29 Thread henk de wit
Hi,When I try to restore a database dump on PostgreSQL 8.3 that's approximately 130GB in size and takes about 1 hour, I noticed index creation makes up the bulk of that time. I'm using a very fast I/O subsystem (16 Mtron Pro 7535 SSDs using a dual 1.2Ghz IOP/4GB cache RAID controller), fast

[PERFORM] Sort performance

2009-01-29 Thread Subbiah Stalin-XCGF84
Hi All, I'm in the process of tuning a query that does a sort on a huge dataset. With work_mem set to 2M, i see the sort operation spilling to disk writing upto 430MB and then return the first 500 rows. Our query is of the sort select co1, col2... from table where col1 like 'aa%' order col1

Re: [PERFORM] PG performance in high volume environment (many INSERTs and lots of aggregation reporting)

2009-01-29 Thread Chris Browne
phoenix.ki...@gmail.com (Phoenix Kiula) writes: [Ppsted similar note to PG General but I suppose it's more appropriate in this list. Apologies for cross-posting.] Hi. Further to my bafflement with the count(*) queries as described in this thread:

Re: [PERFORM] Using multiple cores for index creation?

2009-01-29 Thread Robert Haas
On Thu, Jan 29, 2009 at 3:21 PM, henk de wit henk53...@hotmail.com wrote: Hi, When I try to restore a database dump on PostgreSQL 8.3 that's approximately 130GB in size and takes about 1 hour, I noticed index creation makes up the bulk of that time. I'm using a very fast I/O subsystem (16

Re: [PERFORM] Using multiple cores for index creation?

2009-01-29 Thread Joshua D. Drake
On Thu, 2009-01-29 at 18:09 -0500, Robert Haas wrote: On Thu, Jan 29, 2009 at 3:21 PM, henk de wit henk53...@hotmail.com wrote: Hi, When I try to restore a database dump on PostgreSQL 8.3 that's approximately 130GB in size and takes about 1 hour, I noticed index creation makes up the bulk

Re: [PERFORM] Sort performance

2009-01-29 Thread Robert Haas
On Thu, Jan 29, 2009 at 3:15 PM, Subbiah Stalin-XCGF84 ssubb...@motorola.com wrote: I'm in the process of tuning a query that does a sort on a huge dataset. With work_mem set to 2M, i see the sort operation spilling to disk writing upto 430MB and then return the first 500 rows. Our query is of

Re: [PERFORM] Sort performance

2009-01-29 Thread Gregory Stark
Robert Haas robertmh...@gmail.com writes: On Thu, Jan 29, 2009 at 3:15 PM, Subbiah Stalin-XCGF84 ssubb...@motorola.com wrote: i see the sort operation spilling to disk writing upto 430MB and then return the first 500 rows. Our query is of the sort Now if set the work_mem to 500MB (i did

Re: [PERFORM] Sort performance

2009-01-29 Thread Subbiah Stalin-XCGF84
Here you go. Limit (cost=502843.44..502844.69 rows=501 width=618) (actual time=561397.940..561429.242 rows=501 loops=1) - Sort (cost=502843.44..503923.48 rows=432014 width=618) (actual time=561397.934..561429.062 rows=501 loops=1) Sort Key: name - Seq Scan on objects

Re: [PERFORM] Sort performance

2009-01-29 Thread Subbiah Stalin-XCGF84
Thanks Greg. You were right. If I set my sort_mem to 1G (yes I have loads of memory, only for testing purpose), then I don't see any thing written to disk. So in-memory require more memory than reported on-disk storage. Stalin -Original Message- From: Greg Stark

Re: [PERFORM] Using multiple cores for index creation?

2009-01-29 Thread henk de wit
Hi, You can then pull a TOC out with pg_restore and break that appart. Reading the TOC is pretty self evident. Once you get down to index creation you can create multiple files each with a group of indexes to create. Then call pg_restore multiple times in a script against the individual TOC

Re: [PERFORM] Using multiple cores for index creation?

2009-01-29 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: On Thu, 2009-01-29 at 18:09 -0500, Robert Haas wrote: Andrew Dunstan has been working on this problem. His latest parallel restore patch can be found here: http://archives.postgresql.org/message-id/4977e070.6070...@dunslane.net Yeah but that

Re: [PERFORM] PG performance in high volume environment (many INSERTs and lots of aggregation reporting)

2009-01-29 Thread Scott Marlowe
On Thu, Jan 29, 2009 at 1:56 PM, Chris Browne cbbro...@acm.org wrote: It is common for systems where it is necessary for aggregation reporting to be fast to do pre-computation of the aggregates, and that is in no way specific to PostgreSQL. If you need *really* fast aggregates, then it will