Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Grzegorz Jaśkiewicz
The card is configured in 1+0 . with 128k stripe afaik (I'm a developer, we don't have hardware guys here). Are you's sure about the lack of cache by default on the card ? I thought the difference is that 5104 has 256, and 5105 has 512 ram already on it. -- Sent via pgsql-performance mailing

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Mark Kirkwood
On 26/05/11 20:11, Grzegorz Jaśkiewicz wrote: The card is configured in 1+0 . with 128k stripe afaik (I'm a developer, we don't have hardware guys here). Are you's sure about the lack of cache by default on the card ? I thought the difference is that 5104 has 256, and 5105 has 512 ram already on

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-26 Thread Merlin Moncure
On Wed, May 25, 2011 at 9:20 PM, Merlin Moncure mmonc...@gmail.com wrote: On Wed, May 25, 2011 at 8:03 PM, Steve Crawford scrawf...@pinpointresearch.com wrote: On 05/25/2011 11:45 AM, Reuven M. Lerner wrote: Hi, Alex.  You wrote: Have you tried something like: SELECT  

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread panam
Hi there, Kevin Grittner wrote: Is there a way to determine the values actually used? The pg_settings view. Try the query shown here: http://wiki.postgresql.org/wiki/Server_Configuration Thanks Kevin, very usful. Here is the output: version;PostgreSQL 9.0.4, compiled by Visual C++ build

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-26 Thread Merlin Moncure
On Thu, May 26, 2011 at 8:11 AM, Merlin Moncure mmonc...@gmail.com wrote: On Wed, May 25, 2011 at 9:20 PM, Merlin Moncure mmonc...@gmail.com wrote: On Wed, May 25, 2011 at 8:03 PM, Steve Crawford scrawf...@pinpointresearch.com wrote: On 05/25/2011 11:45 AM, Reuven M. Lerner wrote: Hi, Alex.  

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-26 Thread Reuven M. Lerner
Wow. Color me impressed and grateful. I've been working on a different project today, but I'll test these tonight. I'll never underestimate the regexp functionality in PostgreSQL again! Reuven -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread Cédric Villemain
2011/5/26 panam pa...@gmx.net: Hi there, Kevin Grittner wrote: Is there a way to determine the values actually used? The pg_settings view.  Try the query shown here: http://wiki.postgresql.org/wiki/Server_Configuration Thanks Kevin, very usful. Here is the output: version;PostgreSQL

[PERFORM] The shared buffers challenge

2011-05-26 Thread Merlin Moncure
Hello performers, I've long been unhappy with the standard advice given for setting shared buffers. This includes the stupendously vague comments in the standard documentation, which suggest certain settings in order to get 'good performance'. Performance of what? Connection negotiation speed?

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread Kevin Grittner
Cédric Villemaincedric.villemain.deb...@gmail.com wrote: 2011/5/26 panam pa...@gmx.net: max_connections;100 work_mem;1GB Each connection can allocate work_mem, potentially several times. On a machines without hundreds of GB of RAM, that pair of settings could cause severe swapping.

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Kevin Grittner
Merlin Moncure mmonc...@gmail.com wrote: So, the challenge is this: I'd like to see repeatable test cases that demonstrate regular performance gains 20%. Double bonus points for cases that show gains 50%. Are you talking throughput, maximum latency, or some other metric? In our shop

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-26 Thread Steve Crawford
On 05/26/2011 05:36 AM, Merlin Moncure wrote: ... got it: select decode(regexp_replace('141142143', '([0-9][0-9][0-9])', $q$\\\1$q$ , 'g'), 'escape'); decode abc (1 row) merlin Nice. A word of warning, in 9.0 this returns a hex string: select decode(regexp_replace('141142143',

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Merlin Moncure
On Thu, May 26, 2011 at 10:10 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Merlin Moncure mmonc...@gmail.com wrote: So, the challenge is this: I'd like to see repeatable test cases that demonstrate regular performance gains 20%.  Double bonus points for cases that show gains 50%.

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Claudio Freire
On Thu, May 26, 2011 at 5:36 PM, Merlin Moncure mmonc...@gmail.com wrote: Point being: cranking buffers may have been the bee's knees with, say, the 8.2 buffer manager, but present and future improvements may have render that change moot or even counter productive. I suggest you read the docs

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Merlin Moncure
On Thu, May 26, 2011 at 10:45 AM, Claudio Freire klaussfre...@gmail.com wrote: On Thu, May 26, 2011 at 5:36 PM, Merlin Moncure mmonc...@gmail.com wrote: Point being: cranking buffers may have been the bee's knees with, say, the 8.2 buffer manager, but present and future improvements may have

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Claudio Freire
On Thu, May 26, 2011 at 6:02 PM, Merlin Moncure mmonc...@gmail.com wrote: The point is what we can prove, because going through the motions of doing that is useful. Exactly, and whatever you can prove will be workload-dependant. So you can't prove anything generally, since no single setting is

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Merlin Moncure
On Thu, May 26, 2011 at 11:37 AM, Claudio Freire klaussfre...@gmail.com wrote: On Thu, May 26, 2011 at 6:02 PM, Merlin Moncure mmonc...@gmail.com wrote: The point is what we can prove, because going through the motions of doing that is useful. Exactly, and whatever you can prove will be

Re: [PERFORM] LIMIT and UNION ALL

2011-05-26 Thread Dave Johansen
On Wed, May 18, 2011 at 8:54 AM, Robert Klemme shortcut...@googlemail.comwrote: On Wed, May 18, 2011 at 5:26 PM, Dave Johansen davejohan...@gmail.com wrote: I am using Postgres 8.3.3 and I have a VIEW which is a UNION ALL of two tables but when I do a select on the view using a LIMIT, it

Re: [PERFORM] LIMIT and UNION ALL

2011-05-26 Thread Tom Lane
Dave Johansen davejohan...@gmail.com writes: ... So is there a way to make the planner perform the same sort of operation and push those same constraints into the sub-queries on its own? No. As was mentioned upthread, there is a solution for this in 9.1, although it doesn't work in exactly

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread Kevin Grittner
panam pa...@gmx.net wrote: I cannot use it because of the way that query is generated (by hibernate). The (simplyfied) base query is just SELECT b.id from box the subquery (SELECT m1.id FROM message m1 LEFT JOIN message m2 ON (m1.box_id = m2.box_id AND m1.id m2.id )

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Kevin Grittner
Merlin Moncure mmonc...@gmail.com wrote: Kevin Grittner kevin.gritt...@wicourts.gov wrote: Merlin Moncure mmonc...@gmail.com wrote: So, the challenge is this: I'd like to see repeatable test cases that demonstrate regular performance gains 20%. Double bonus points for cases that show gains

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread panam
Sorry, SELECT MAX(e.id) FROM event_message e WHERE e.box_id = id as posted previously should actually read SELECT max(m1.id) FROM message m1 WHERE m1.box_id = b.id) so I tried this already. Regards, panam -- View this message in context:

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread Cédric Villemain
2011/5/26 panam pa...@gmx.net: Hi all, Cédric Villemain-3 wrote: without explaining further why the antijoin has bad performance without cluster, I wonder why you don't use this query : SELECT  b.id,                   max(m.id) FROM box b, message m WHERE m.box_id = b.id GROUP BY

[PERFORM] Performance block size.

2011-05-26 Thread Tory M Blue
Working on some optimization as well as finally getting off my backside and moving us to 64bit (32gb+memory). I was reading and at some point it appears on freeBSD the Postgres block size was upped to 16kb, from 8kb. And on my fedora systems I believe the default build is 8kb. When we were

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Mark Kirkwood
On 26/05/11 20:31, Grzegorz Jaśkiewicz wrote: Would HT have any impact to the I/O performance (postgresql, and fs in general) ?. There have been previous discussions on this list about HT on vs off (I can't recall what the consensus, if any about what the cause of any performance difference

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Greg Smith
Merlin Moncure wrote: So, the challenge is this: I'd like to see repeatable test cases that demonstrate regular performance gains 20%. Double bonus points for cases that show gains 50%. Do I run around challenging your suggestions and giving you homework? You have no idea how much eye

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Greg Smith
Mark Kirkwood wrote: You can interrogate the config of the card and the raid 10 array using the megaraid cli package - you need to read the (frankly terrible) manual to discover which switches to use to determine battery and cache status etc. If you email me privately I'll get you a link to

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Mark Kirkwood
On 27/05/11 11:19, Greg Smith wrote: Mark Kirkwood wrote: You can interrogate the config of the card and the raid 10 array using the megaraid cli package - you need to read the (frankly terrible) manual to discover which switches to use to determine battery and cache status etc. If you email

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Mark Kirkwood
On 27/05/11 11:22, Mark Kirkwood wrote: On 27/05/11 11:19, Greg Smith wrote: Mark Kirkwood wrote: You can interrogate the config of the card and the raid 10 array using the megaraid cli package - you need to read the (frankly terrible) manual to discover which switches to use to determine

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Samuel Gendler
On Thu, May 26, 2011 at 4:10 PM, Greg Smith g...@2ndquadrant.com wrote: As for figuring out how this impacts more complicated cases, I hear somebody wrote a book or something that went into pages and pages of detail about all this. You might want to check it out. I was just going to

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread Craig Ringer
On 05/27/2011 02:13 AM, Cédric Villemain wrote: I am not an hibernate expert, but I'll surprised if you can not drive hibernate to do what you want. If nothing else, you can do a native query in hand-written SQL through Hibernate. ORMs are useful tools for some jobs, but it's good to be able

[PERFORM] Is any effect other process performance after vaccumdb finished ?

2011-05-26 Thread Junghwe Kim
Hi. First extremely thanks for your works about postgresql . I wonder that after executing 'vaccumdb -z' some other process can not read their own msg queue during 2 ~ 3 minuts. vaccum executed every hour. and The processes have not any relations between postgreql. Is it possible ?

Re: [PERFORM] Is any effect other process performance after vaccumdb finished ?

2011-05-26 Thread Craig Ringer
On 27/05/2011 9:58 AM, Junghwe Kim wrote: Hi. First extremely thanks for your works about postgresql . I wonder that after executing 'vaccumdb -z' some other process can not read their own msg queue during 2 ~ 3 minuts. The most likely cause is checkpoint activity. Enable checkpoint