Re: [PERFORM] Greenplum MapReduce

2009-08-03 Thread Richard Huxton
Suvankar Roy wrote: Hi all, Has anybody worked on Greenplum MapReduce programming ? I am facing a problem while trying to execute the below Greenplum Mapreduce program written in YAML (in blue). The other poster suggested contacting Greenplum and I can only agree. The error is thrown in

Re: [PERFORM] Greenplum MapReduce

2009-08-03 Thread Richard Huxton
Suvankar Roy wrote: Hi Richard, I sincerely regret the inconvenience caused. No big inconvenience, but the lists can be very busy sometimes and the easier you make it for people to answer your questions the better the answers you will get. %YAML 1.1 --- VERSION: 1.0.0.1 DATABASE:

Re: [PERFORM] Why is PostgreSQL so slow on Windows ( Postgres 8.3.7) version

2009-08-03 Thread Grzegorz Jaśkiewicz
how about normalizing the schema for start ? by the looks of it, you have huge table,with plenty of varchars, that smells like bad design of db. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription:

Re: [PERFORM] Why is PostgreSQL so slow on Windows ( Postgres 8.3.7) version

2009-08-03 Thread Marc Cousin
The few 'obvious' things I see : ID and POLLID aren't of the same type (numeric vs bigint) TTIME isn't indexed. And as a general matter, you should stick to native datatypes if you don't need numeric. But as said in the other answer, maybe you should redo this schema and use more consistent

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread Scott Carey
On 7/31/09 4:01 PM, PFC li...@peufeu.com wrote: On Fri, 31 Jul 2009 19:04:52 +0200, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark gsst...@mit.edu writes: On Thu, Jul 30, 2009 at 11:30 PM, Tom Lanet...@sss.pgh.pa.us wrote: I did some tracing and verified that pg_dump passes data to

[PERFORM] Query help

2009-08-03 Thread Subbiah Stalin-XCGF84
All, Not sure what's wrong in below execution plan but at times the query runs for 5 minutes to complete and after a while it runs within a second or two. Here is explain analyze out of the query. SELECT OBJECTS.ID,OBJECTS.NAME,OBJECTS.TYPE,OBJECTS.STATUS,OBJECTS.ALTNAME,OBJE

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread Tom Lane
Scott Carey sc...@richrelevance.com writes: I get very different (contradictory) behavior. Server with fast RAID, 32GB RAM, 2 x 4 core 3.16Ghz Xeon 54xx CPUs. CentOS 5.2 8.3.6 No disk wait time during any test. One test beforehand was used to prime the disk cache. 100% CPU in the below

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread Scott Carey
On 8/3/09 11:56 AM, Tom Lane t...@sss.pgh.pa.us wrote: Scott Carey sc...@richrelevance.com writes: I get very different (contradictory) behavior. Server with fast RAID, 32GB RAM, 2 x 4 core 3.16Ghz Xeon 54xx CPUs. CentOS 5.2 8.3.6 No disk wait time during any test. One test beforehand

Re: [PERFORM] Query help

2009-08-03 Thread Kevin Grittner
Subbiah Stalin-XCGF84 ssubb...@motorola.com wrote: Not sure what's wrong in below execution plan but at times the query runs for 5 minutes to complete and after a while it runs within a second or two. The plan doesn't look entirely unreasonable for the given query, although it's hard to be

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread PFC
I get very different (contradictory) behavior. Server with fast RAID, 32GB RAM, 2 x 4 core 3.16Ghz Xeon 54xx CPUs. CentOS 5.2 8.3.6 That's a very different serup from my (much less powerful) box, so that would explain it... No disk wait time during any test. One test beforehand was

Re: [PERFORM] Query help

2009-08-03 Thread Subbiah Stalin-XCGF84
Sure I can provide those details. I have seen this query running 5+ minutes for different values for doaminID too. Its just that it happens at random and gets fixed within few mins. Shared buffer=8G, effective cache size=4G. Optimizer/autovaccum settings are defaults relname

Re: [PERFORM] Query help

2009-08-03 Thread Kevin Grittner
Subbiah Stalin-XCGF84 ssubb...@motorola.com wrote: Shared buffer=8G, effective cache size=4G. That is odd; if your shared buffers are at 8G, you must have more than 4G of cache. How much RAM is used for cache at the OS level? Normally you would add that to the shared buffers to get your

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread Merlin Moncure
On Mon, Aug 3, 2009 at 2:56 PM, Tom Lanet...@sss.pgh.pa.us wrote: I don't see anything very contradictory here.  What you're demonstrating is that it's nice to be able to throw a third CPU at the compression part of the problem.  That's likely to remain true if we shift to a different

Re: [PERFORM] Query help

2009-08-03 Thread Subbiah Stalin-XCGF84
Server has 32G memory and it's a dedicated to run PG and no other application is sharing this database. I have checked checkpoints and they don't occur during those slow query runtimes. Checkpoint_segments is set 128. here is quick snap from vmstat. # vmstat 5 5 kthr memorypage

Re: [PERFORM] PostgreSQL 8.4 performance tuning questions

2009-08-03 Thread PFC
lzo is much, much, (much) faster than zlib. Note, I've tried several decompression speed is even more awesome... times to contact the author to get clarification on licensing terms and have been unable to get a response. lzop and the LZO library are distributed under the terms of the GNU

Re: [PERFORM] Greenplum MapReduce

2009-08-03 Thread Suvankar Roy
Hi Robert, Thanks much for your valuable inputs This spaces and tabs problem is killing me in a way, it is pretty cumbersome to say the least Regards, Suvankar Roy Robert Mah r...@pobox.com Sent by: Robert Mah robert@gmail.com 08/02/2009 10:52 PM To 'Suvankar Roy'