Re: [PERFORM] Is any effect other process performance after vaccumdb finished ?

2011-05-26 Thread Craig Ringer
On 27/05/2011 9:58 AM, Junghwe Kim wrote: Hi. First extremely thanks for your works about postgresql . I wonder that after executing 'vaccumdb -z' some other process can not read their own msg queue during 2 ~ 3 minuts. The most likely cause is checkpoint activity. Enable checkpoint logging

[PERFORM] Is any effect other process performance after vaccumdb finished ?

2011-05-26 Thread Junghwe Kim
Hi. First extremely thanks for your works about postgresql . I wonder that after executing 'vaccumdb -z' some other process can not read their own msg queue during 2 ~ 3 minuts. vaccum executed every hour. and The processes have not any relations between postgreql. Is it possible ?

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread Craig Ringer
On 05/27/2011 02:13 AM, Cédric Villemain wrote: I am not an hibernate expert, but I'll surprised if you can not drive hibernate to do what you want. If nothing else, you can do a native query in hand-written SQL through Hibernate. ORMs are useful tools for some jobs, but it's good to be able

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Samuel Gendler
On Thu, May 26, 2011 at 4:10 PM, Greg Smith wrote: > > As for figuring out how this impacts more complicated cases, I hear > somebody wrote a book or something that went into pages and pages of detail > about all this. You might want to check it out. > > I was just going to suggest that there wa

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Mark Kirkwood
On 27/05/11 11:22, Mark Kirkwood wrote: On 27/05/11 11:19, Greg Smith wrote: Mark Kirkwood wrote: You can interrogate the config of the card and the raid 10 array using the megaraid cli package - you need to read the (frankly terrible) manual to discover which switches to use to determine bat

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Mark Kirkwood
On 27/05/11 11:19, Greg Smith wrote: Mark Kirkwood wrote: You can interrogate the config of the card and the raid 10 array using the megaraid cli package - you need to read the (frankly terrible) manual to discover which switches to use to determine battery and cache status etc. If you email m

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Greg Smith
Mark Kirkwood wrote: You can interrogate the config of the card and the raid 10 array using the megaraid cli package - you need to read the (frankly terrible) manual to discover which switches to use to determine battery and cache status etc. If you email me privately I'll get you a link to the

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Greg Smith
Merlin Moncure wrote: So, the challenge is this: I'd like to see repeatable test cases that demonstrate regular performance gains > 20%. Double bonus points for cases that show gains > 50%. Do I run around challenging your suggestions and giving you homework? You have no idea how much eye ro

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Mark Kirkwood
On 26/05/11 20:31, Grzegorz Jaśkiewicz wrote: Would HT have any impact to the I/O performance (postgresql, and fs in general) ?. There have been previous discussions on this list about HT on vs off (I can't recall what the consensus, if any about what the cause of any performance difference

[PERFORM] Performance block size.

2011-05-26 Thread Tory M Blue
Working on some optimization as well as finally getting off my backside and moving us to 64bit (32gb+memory). I was reading and at some point it appears on freeBSD the Postgres block size was upped to 16kb, from 8kb. And on my fedora systems I believe the default build is 8kb. When we were using

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread Cédric Villemain
2011/5/26 panam : > Hi all, > > > Cédric Villemain-3 wrote: >> >> without explaining further why the antijoin has bad performance >> without cluster, I wonder why you don't use this query : >> >> SELECT  b.id, >>                   max(m.id) >> FROM box b, message m >> WHERE m.box_id = b.id >> GROUP

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread panam
Sorry, SELECT MAX(e.id) FROM event_message e WHERE e.box_id = id as posted previously should actually read SELECT max(m1.id) FROM message m1 WHERE m1.box_id = b.id) so I tried this already. Regards, panam -- View this message in context: http://postgresql.1045698.n5.nabble.com/Hash-Anti-J

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Kevin Grittner
Merlin Moncure wrote: > Kevin Grittner wrote: >> Merlin Moncure wrote: >> >>> So, the challenge is this: I'd like to see repeatable test cases >>> that demonstrate regular performance gains > 20%. Double bonus >>> points for cases that show gains > 50%. >> >> Are you talking throughput, maximum

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread Kevin Grittner
panam wrote: > I cannot use it because of the way that query is generated > (by hibernate). > > The (simplyfied) base query is just > > SELECT b.id from box > > the subquery > > (SELECT m1.id FROM message m1 >LEFT JOIN message m2 > ON (m1.box_id = m2.box_id AND m1.id < m2.id )

Re: [PERFORM] LIMIT and UNION ALL

2011-05-26 Thread Tom Lane
Dave Johansen writes: > ... So is there a way to make the > planner perform the same sort of operation and push those same constraints > into the sub-queries on its own? No. As was mentioned upthread, there is a solution for this in 9.1, although it doesn't work in exactly the way you suggest.

Re: [PERFORM] LIMIT and UNION ALL

2011-05-26 Thread Dave Johansen
On Wed, May 18, 2011 at 8:54 AM, Robert Klemme wrote: > On Wed, May 18, 2011 at 5:26 PM, Dave Johansen > wrote: > > I am using Postgres 8.3.3 and I have a VIEW which is a UNION ALL of two > > tables but when I do a select on the view using a LIMIT, it scans the > entire > > tables and takes signi

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Merlin Moncure
On Thu, May 26, 2011 at 11:37 AM, Claudio Freire wrote: > On Thu, May 26, 2011 at 6:02 PM, Merlin Moncure wrote: >> The point is what we can prove, because going through the >> motions of doing that is useful. > > Exactly, and whatever you can "prove" will be workload-dependant. > So you can't pr

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Claudio Freire
On Thu, May 26, 2011 at 6:02 PM, Merlin Moncure wrote: > The point is what we can prove, because going through the > motions of doing that is useful. Exactly, and whatever you can "prove" will be workload-dependant. So you can't prove anything "generally", since no single setting is best for all.

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread panam
Hi all, Cédric Villemain-3 wrote: > > without explaining further why the antijoin has bad performance > without cluster, I wonder why you don't use this query : > > SELECT b.id, > max(m.id) > FROM box b, message m > WHERE m.box_id = b.id > GROUP BY b.id; > > looks similar an

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Merlin Moncure
On Thu, May 26, 2011 at 10:45 AM, Claudio Freire wrote: > On Thu, May 26, 2011 at 5:36 PM, Merlin Moncure wrote: >> Point being: cranking buffers >> may have been the bee's knees with, say, the 8.2 buffer manager, but >> present and future improvements may have render that change moot or >> even

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Claudio Freire
On Thu, May 26, 2011 at 5:36 PM, Merlin Moncure wrote: > Point being: cranking buffers > may have been the bee's knees with, say, the 8.2 buffer manager, but > present and future improvements may have render that change moot or > even counter productive. I suggest you read the docs on how shared

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Merlin Moncure
On Thu, May 26, 2011 at 10:10 AM, Kevin Grittner wrote: > Merlin Moncure wrote: > >> So, the challenge is this: I'd like to see repeatable test cases >> that demonstrate regular performance gains > 20%.  Double bonus >> points for cases that show gains > 50%. > > Are you talking throughput, maxim

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-26 Thread Steve Crawford
On 05/26/2011 05:36 AM, Merlin Moncure wrote: ... got it: select decode(regexp_replace('141142143', '([0-9][0-9][0-9])', $q$\\\1$q$ , 'g'), 'escape'); decode abc (1 row) merlin Nice. A word of warning, in 9.0 this returns a hex string: select decode(regexp_replace('141142143', '(

Re: [PERFORM] The shared buffers challenge

2011-05-26 Thread Kevin Grittner
Merlin Moncure wrote: > So, the challenge is this: I'd like to see repeatable test cases > that demonstrate regular performance gains > 20%. Double bonus > points for cases that show gains > 50%. Are you talking throughput, maximum latency, or some other metric? In our shop the metric we tu

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread Kevin Grittner
Cédric Villemain wrote: > 2011/5/26 panam : >> "max_connections";"100" >> "work_mem";"1GB" Each connection can allocate work_mem, potentially several times. On a machines without hundreds of GB of RAM, that pair of settings could cause severe swapping. >> "Patholgical" query: >> >> select

[PERFORM] The shared buffers challenge

2011-05-26 Thread Merlin Moncure
Hello performers, I've long been unhappy with the standard advice given for setting shared buffers. This includes the stupendously vague comments in the standard documentation, which suggest certain settings in order to get 'good performance'. Performance of what? Connection negotiation speed? N

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread Cédric Villemain
2011/5/26 panam : > Hi there, > > > Kevin Grittner wrote: >> >>> Is there a way to determine the values actually used? >> The pg_settings view.  Try the query shown here: >> http://wiki.postgresql.org/wiki/Server_Configuration >> > Thanks Kevin, very usful. Here is the output: > > "version";"Postgr

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-26 Thread Reuven M. Lerner
Wow. Color me impressed and grateful. I've been working on a different project today, but I'll test these tonight. I'll never underestimate the regexp functionality in PostgreSQL again! Reuven -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to y

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-26 Thread Merlin Moncure
On Thu, May 26, 2011 at 8:11 AM, Merlin Moncure wrote: > On Wed, May 25, 2011 at 9:20 PM, Merlin Moncure wrote: >> On Wed, May 25, 2011 at 8:03 PM, Steve Crawford >> wrote: >>> On 05/25/2011 11:45 AM, Reuven M. Lerner wrote: Hi, Alex.  You wrote: > > Have you tried something li

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-26 Thread panam
Hi there, Kevin Grittner wrote: > >> Is there a way to determine the values actually used? > The pg_settings view. Try the query shown here: > http://wiki.postgresql.org/wiki/Server_Configuration > Thanks Kevin, very usful. Here is the output: "version";"PostgreSQL 9.0.4, compiled by Visual C

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-26 Thread Merlin Moncure
On Wed, May 25, 2011 at 9:20 PM, Merlin Moncure wrote: > On Wed, May 25, 2011 at 8:03 PM, Steve Crawford > wrote: >> On 05/25/2011 11:45 AM, Reuven M. Lerner wrote: >>> >>> Hi, Alex.  You wrote: Have you tried something like: SELECT  encode(regexp_replace('141142143', '(\d{3})', '\

Re: FW: [PERFORM] KVP table vs. hstore - hstore performance (Was: Postgres NoSQL emulation)

2011-05-26 Thread Pierre C
My problem is, that in fact I don't know which tag to index since I'm running a web admin application where users can enter arbitrary queries. For a tag cloud, try this : - table tags ( tag_id, tag_name ) - table articles ( article_id ) - table articles_to_tags( article_id, tag_id ) now this

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-26 Thread Jochen Erwied
Thursday, May 26, 2011, 12:26:17 AM you wrote: > Here you go... it looked nicer before I started to make optimizations; > I've gotten it to run about 2x as fast as the previous version, but now > I'm sorta stuck, looking for further optimizations, including possible > use of builtin functions.

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Grzegorz Jaśkiewicz
Would HT have any impact to the I/O performance (postgresql, and fs in general) ?. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Mark Kirkwood
On 26/05/11 20:11, Grzegorz Jaśkiewicz wrote: The card is configured in 1+0 . with 128k stripe afaik (I'm a developer, we don't have hardware guys here). Are you's sure about the lack of cache by default on the card ? I thought the difference is that 5104 has 256, and 5105 has 512 ram already on

Re: [PERFORM] serveRAID M5014 SAS

2011-05-26 Thread Grzegorz Jaśkiewicz
The card is configured in 1+0 . with 128k stripe afaik (I'm a developer, we don't have hardware guys here). Are you's sure about the lack of cache by default on the card ? I thought the difference is that 5104 has 256, and 5105 has 512 ram already on it. -- Sent via pgsql-performance mailing list