Re: [HACKERS] remembering locks and dynahash.c

2012-05-31 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: I am working on making resource owner remember a limited number of locks, so it can reassign them more efficiently. Check, per previous discussion. Currently I'm having resowner.c remember the LOCALLOCKTAG, because I thought that that was the only

Re: [HACKERS] Uppercase tab completion keywords in psql?

2012-05-31 Thread Devrim GÜNDÜZ
Hi, On Wed, 2012-05-30 at 22:03 -0400, Bruce Momjian wrote: A control variable was added in this commit: commit db84ba65ab5c0ad0b34d68ab5a687bc5f4ca3ba6 Author: Peter Eisentraut pete...@gmx.net Thanks Bruce, apparently I missed it. Regards, -- Devrim GÜNDÜZ Principal

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-31 Thread Yeb Havinga
On 2012-05-30 21:26, Kohei KaiGai wrote: If we would have an ideal optimizer, I'd still like the optimizer to wipe out redundant clauses transparently, rather than RLSBYPASS permissions, because it just controls all-or-nothing stuff. For example, if tuples are categorized to unclassified,

Re: [HACKERS] FailedAssertion(!(PrivateRefCount[i] == 0), File: bufmgr.c, Line: 1741

2012-05-31 Thread Heikki Linnakangas
On 31.05.2012 08:06, Erik Rijkers wrote: On Thu, May 31, 2012 03:30, Robert Haas wrote: On Wed, May 30, 2012 at 6:00 PM, Erik Rijkerse...@xs4all.nl wrote: directory 2012-05-30 23:40:57.909 CEST 3909 CONTEXT: writing block 5152 of relation base/21268/26569 xlog redo multi-insert

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-31 Thread Kohei KaiGai
2012/5/31 Yeb Havinga yebhavi...@gmail.com: On 2012-05-30 21:26, Kohei KaiGai wrote: If we would have an ideal optimizer, I'd still like the optimizer to wipe out redundant clauses transparently, rather than RLSBYPASS permissions, because it just controls all-or-nothing stuff. For example,

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Tatsuo Ishii
We recently fixed a couple of O(N^2) loops in pg_dump, but those covered extremely specific cases that might or might not have anything to do with what you're seeing. The complainant was extremely helpful about tracking down the problems:

Re: [HACKERS] GiST buffering build, bug in levelStep calculation

2012-05-31 Thread Heikki Linnakangas
On 29.05.2012 23:46, Alexander Korotkov wrote: On Wed, May 30, 2012 at 12:25 AM, Tom Lanet...@sss.pgh.pa.us wrote: Alexander Korotkovaekorot...@gmail.com writes: On Tue, May 29, 2012 at 11:42 PM, Tom Lanet...@sss.pgh.pa.us wrote: While I'm looking at this, is the first test involving

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Simon Riggs
On 30 May 2012 17:19, Peter Geoghegan pe...@2ndquadrant.com wrote: On 30 May 2012 15:25, Simon Riggs si...@2ndquadrant.com wrote: 1. It seems wrong to do it in xact_redo_commit_internal().  It won't matter if commit_siblings0 since there won't be any other backends with transaction IDs anyway,

Re: [HACKERS] GiST buffering build, bug in levelStep calculation

2012-05-31 Thread Alexander Korotkov
On Thu, May 31, 2012 at 1:36 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I note that the calculations assume that leaf tuples and internal tuples have similar sizes. We calculate the average leaf tuple size, and use that to calculate the fan-out of internal pages. On

[HACKERS] heap_form_tuple crashing

2012-05-31 Thread Atri Sharma
Hi all, I am trying to call heap_form_tuple to create a tuple from a datum. My call to heap_form_tuple looks like: val1=0; tupledesc1=BlessTupleDesc(node-ss.ss_currentRelation-rd_att); tuple=heap_form_tuple(tupledesc1,p1,val1); p1 is a pointer to a Datum instance which is created from a char

Re: [HACKERS] patch: avoid heavyweight locking on hash metapage

2012-05-31 Thread Dickson S. Guedes
2012/5/30 Robert Haas robertmh...@gmail.com: I tested the effect of this by setting up a series of 5-minute read-only pgbench run at scale factor 300 with 8GB of shared buffers on the IBM POWER7 machine. I know it doesn't matter, but out of curiosity what OS you used? best regards, --

Re: [HACKERS] patch: avoid heavyweight locking on hash metapage

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 7:07 AM, Dickson S. Guedes lis...@guedesoft.net wrote: 2012/5/30 Robert Haas robertmh...@gmail.com: I tested the effect of this by setting up a series of 5-minute read-only pgbench run at scale factor 300 with 8GB of shared buffers on the IBM POWER7 machine. I know it

Re: [HACKERS] FailedAssertion(!(PrivateRefCount[i] == 0), File: bufmgr.c, Line: 1741

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 1:06 AM, Erik Rijkers e...@xs4all.nl wrote: In my test, I run the bash code (the bits that I posted earlier) in a while loop so that the table is CREATEd, COPYied into, and DROPped every few seconds -- perhaps that wasn't clear.  That loop is necessary; a few

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Peter Geoghegan
On 31 May 2012 11:19, Simon Riggs si...@2ndquadrant.com wrote: I've looked at this more closely now and I can see that the call to XLogFlush() that is made from xact_redo_commit_internal() doesn't ever actually flush WAL, so whether we delay or not is completely irrelevant. So un-agreed. No

Re: [HACKERS] heap_form_tuple crashing

2012-05-31 Thread Heikki Linnakangas
On 31.05.2012 13:42, Atri Sharma wrote: I am trying to call heap_form_tuple to create a tuple from a datum. My call to heap_form_tuple looks like: val1=0; tupledesc1=BlessTupleDesc(node-ss.ss_currentRelation-rd_att); tuple=heap_form_tuple(tupledesc1,p1,val1); p1 is a pointer to a Datum

Re: [HACKERS] heap_form_tuple crashing

2012-05-31 Thread Atri Sharma
On Thu, May 31, 2012 at 5:02 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 31.05.2012 13:42, Atri Sharma wrote: I am trying to call heap_form_tuple to create a tuple from a datum. My call to heap_form_tuple looks like: val1=0;

Re: [HACKERS] heap_form_tuple crashing

2012-05-31 Thread Heikki Linnakangas
On 31.05.2012 14:42, Atri Sharma wrote: Another thing I wanted to ask was that would you recommend building tuples from strings directly or converting them to Datum first and then build the tuples from Datum instances? It depends. If you have all the values in strings already, then

Re: [HACKERS] heap_form_tuple crashing

2012-05-31 Thread Atri Sharma
On Thu, May 31, 2012 at 5:14 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 31.05.2012 14:42, Atri Sharma wrote: Another thing I wanted to ask was that would you recommend building tuples from strings directly or converting them to Datum first and then build the tuples

Re: [HACKERS] FailedAssertion(!(PrivateRefCount[i] == 0), File: bufmgr.c, Line: 1741

2012-05-31 Thread Simon Riggs
On 31 May 2012 12:14, Robert Haas robertmh...@gmail.com wrote: On Thu, May 31, 2012 at 1:06 AM, Erik Rijkers e...@xs4all.nl wrote: In my test, I run the bash code (the bits that I posted earlier) in a while loop so that the table is CREATEd, COPYied into, and DROPped every few seconds --

Re: [HACKERS] FailedAssertion(!(PrivateRefCount[i] == 0), File: bufmgr.c, Line: 1741

2012-05-31 Thread Erik Rijkers
On Thu, May 31, 2012 13:14, Robert Haas wrote: On Thu, May 31, 2012 at 1:06 AM, Erik Rijkers e...@xs4all.nl wrote: In my test, I run the bash code (the bits that I posted earlier) in a while loop so that the table is CREATEd, COPYied into, and DROPped every few seconds -- perhaps that

Re: [HACKERS] slow dropping of tables, DropRelFileNodeBuffers, tas

2012-05-31 Thread Simon Riggs
On 30 May 2012 12:10, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Hmm, we do this in smgrDoPendingDeletes: for (i = 0; i = MAX_FORKNUM; i++) {        smgrdounlink(srel, i, false); } So we drop the buffers for each relation fork separately, which means that we scan the

Re: [HACKERS] FailedAssertion(!(PrivateRefCount[i] == 0), File: bufmgr.c, Line: 1741

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 7:49 AM, Erik Rijkers e...@xs4all.nl wrote: On Thu, May 31, 2012 13:14, Robert Haas wrote: On Thu, May 31, 2012 at 1:06 AM, Erik Rijkers e...@xs4all.nl wrote: In my test, I run the bash code (the bits that I posted earlier) in a while loop so that the table is

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 6:19 AM, Simon Riggs si...@2ndquadrant.com wrote: I've looked at this more closely now and I can see that the call to XLogFlush() that is made from xact_redo_commit_internal() doesn't ever actually flush WAL, so whether we delay or not is completely irrelevant. So

Re: [HACKERS] Issues with MinGW W64

2012-05-31 Thread Johann 'Myrkraverk' Oskarsson
Tom Lane t...@sss.pgh.pa.us writes: Robert Haas robertmh...@gmail.com writes: On Tue, May 29, 2012 at 9:04 AM, Johann 'Myrkraverk' Oskarsson joh...@2ndquadrant.com wrote: The header file crtdefs.h in MinGW typedefs errcode which conflicts with Postgres' elog.h. Eep. Maybe this is not

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Simon Riggs
On 31 May 2012 13:16, Robert Haas robertmh...@gmail.com wrote: On Thu, May 31, 2012 at 6:19 AM, Simon Riggs si...@2ndquadrant.com wrote: I've looked at this more closely now and I can see that the call to XLogFlush() that is made from xact_redo_commit_internal() doesn't ever actually flush

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Peter Geoghegan
On 31 May 2012 13:16, Robert Haas robertmh...@gmail.com wrote: On Thu, May 31, 2012 at 6:19 AM, Simon Riggs si...@2ndquadrant.com wrote: Frankly, I think this whole thing should be pushed to 9.3.  The commit_delay and commit_siblings knobs suck, but they've sucked for a long time, and it won't

Re: [HACKERS] heap_form_tuple crashing

2012-05-31 Thread Tom Lane
Atri Sharma atri.j...@gmail.com writes: My call to heap_form_tuple looks like: val1=0; tupledesc1=BlessTupleDesc(node-ss.ss_currentRelation-rd_att); tuple=heap_form_tuple(tupledesc1,p1,val1); p1 is a pointer to a Datum instance which is created from a char array. Does that actually match

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Robert Klemme
On Thu, May 31, 2012 at 10:45 AM, Tatsuo Ishii is...@postgresql.org wrote: Just for record, I rerun the test again with my single-LOCK patch, and now total runtime of pg_dump is 113 minutes. 188 minutes(9.0)-125 minutes(git master)-113 minutes(with my patch). So far, I'm glad to see 40% time

Re: [HACKERS] FailedAssertion(!(PrivateRefCount[i] == 0), File: bufmgr.c, Line: 1741

2012-05-31 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: The one thing that still seems a little odd to me is that this caused a pin count to get orphaned. It seems reasonable that ignoring the AccessExclusiveLock could result in not-found errors trying to open a missing relation, and even fsync requests on

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 8:38 AM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 31 May 2012 13:16, Robert Haas robertmh...@gmail.com wrote: On Thu, May 31, 2012 at 6:19 AM, Simon Riggs si...@2ndquadrant.com wrote: Frankly, I think this whole thing should be pushed to 9.3.  The commit_delay

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 31 May 2012 13:16, Robert Haas robertmh...@gmail.com wrote: Frankly, I think this whole thing should be pushed to 9.3. What matters is that we have a patch that provides a massive performance gain in write performance in just a few lines of code,

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Tatsuo Ishii
On Thu, May 31, 2012 at 10:45 AM, Tatsuo Ishii is...@postgresql.org wrote: Just for record, I rerun the test again with my single-LOCK patch, and now total runtime of pg_dump is 113 minutes. 188 minutes(9.0)-125 minutes(git master)-113 minutes(with my patch). So far, I'm glad to see 40% time

Re: [HACKERS] FailedAssertion(!(PrivateRefCount[i] == 0), File: bufmgr.c, Line: 1741

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 9:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The one thing that still seems a little odd to me is that this caused a pin count to get orphaned.  It seems reasonable that ignoring the AccessExclusiveLock could result in not-found

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Robert Klemme
On Thu, May 31, 2012 at 4:07 PM, Tatsuo Ishii is...@postgresql.org wrote: On Thu, May 31, 2012 at 10:45 AM, Tatsuo Ishii is...@postgresql.org wrote: Just for record, I rerun the test again with my single-LOCK patch, and now total runtime of pg_dump is 113 minutes. 188 minutes(9.0)-125

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 7:31 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Wed, 30 May 2012, Robert Haas wrote: I'd really like to find out exactly where all those s_lock calls are coming from.  Is there any way you can get oprofile to output a partial stack backtrace?  If you have perf

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Claudio Freire
On Thu, May 31, 2012 at 11:17 AM, Robert Klemme shortcut...@googlemail.com wrote: OK, my fault was to assume you wanted to measure only your part, while apparently you meant overall savings.  But Tom had asked for separate measurements if I understood him correctly.  Also, that measurement of

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Tom Lane
Claudio Freire klaussfre...@gmail.com writes: It's not clear whether Tom is already working on that O(N^2) fix in locking. I'm not; Jeff Janes is. But you shouldn't be holding your breath anyway, since it's 9.3 material at this point. regards, tom lane -- Sent via

Re: [HACKERS] Issues with MinGW W64

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 8:15 AM, Johann 'Myrkraverk' Oskarsson joh...@2ndquadrant.com wrote: MinGW W64's sys/stat.h #defines stat to be _stati64 and there is subsequently a compilation error in port.h: note: expected 'struct _stati64 *' but argument is of type 'struct stat *' error:

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 10:31 AM, Tom Lane t...@sss.pgh.pa.us wrote: Claudio Freire klaussfre...@gmail.com writes: It's not clear whether Tom is already working on that O(N^2) fix in locking. I'm not; Jeff Janes is.  But you shouldn't be holding your breath anyway, since it's 9.3 material at

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-31 Thread Robert Haas
On Wed, May 30, 2012 at 3:26 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: My preference is RLSBYPASS permission rather than the approach with functions that return policy clause at run-time, because it needs to invalidate prepared statement at random timing. In case of this function approach,

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, May 31, 2012 at 10:31 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not; Jeff Janes is.  But you shouldn't be holding your breath anyway, since it's 9.3 material at this point. I agree we can't back-patch that change, but then I think we ought to

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Simon Riggs
On 31 May 2012 15:00, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 31 May 2012 13:16, Robert Haas robertmh...@gmail.com wrote: Frankly, I think this whole thing should be pushed to 9.3. What matters is that we have a patch that provides a massive

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Bruce Momjian
On Thu, May 31, 2012 at 10:50:51AM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Thu, May 31, 2012 at 10:31 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not; Jeff Janes is. �But you shouldn't be holding your breath anyway, since it's 9.3 material at this point. I

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Claudio Freire
On Thu, May 31, 2012 at 11:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: The performance patches we applied to pg_dump over the past couple weeks were meant to relieve pain in situations where the big server-side lossage wasn't the dominant factor in runtime (ie, partial dumps). But this one is

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 10:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, May 31, 2012 at 10:31 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not; Jeff Janes is.  But you shouldn't be holding your breath anyway, since it's 9.3 material at this point.

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Bruce Momjian
On Thu, May 31, 2012 at 11:04:12AM -0400, Robert Haas wrote: On Thu, May 31, 2012 at 10:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, May 31, 2012 at 10:31 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not; Jeff Janes is.  But you shouldn't be

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 31 May 2012 15:00, Tom Lane t...@sss.pgh.pa.us wrote: If we want to finish the beta cycle in a reasonable time period and get back to actual development, we have to refrain from adding more possibly-destabilizing development work to 9.2.  And that

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Peter Geoghegan
On 31 May 2012 14:58, Robert Haas robertmh...@gmail.com wrote: Fixing regressions before release is essential; improving performance is not - especially when the improvement relates to a little-used feature that you were proposing to get rid of two weeks ago. Yes, the fact that I wanted to get

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Tom Lane
Claudio Freire klaussfre...@gmail.com writes: On Thu, May 31, 2012 at 11:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: The performance patches we applied to pg_dump over the past couple weeks were meant to relieve pain in situations where the big server-side lossage wasn't the dominant factor in

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Claudio Freire
On Thu, May 31, 2012 at 12:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: No, Tatsuo's patch attacks a phase dominated by latency in some setups. No, it does not.  The reason it's a win is that it avoids the O(N^2) behavior in the server.  Whether the bandwidth savings is worth worrying about

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Peter Geoghegan
On 31 May 2012 16:26, Peter Geoghegan pe...@2ndquadrant.com wrote: On 31 May 2012 16:23, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: In what way is it possibly destabilising? I'm prepared to believe that it only affects performance, but it could be

Re: [HACKERS] hash index concurrency

2012-05-31 Thread David Fetter
On Wed, May 30, 2012 at 12:21:33AM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Tue, May 29, 2012 at 11:21 PM, Jeff Janes jeff.ja...@gmail.com wrote: 2) Only support bitmap scans and not ordinary tid scans (the way gin indexes already do). -1 on losing amgettuple.

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-05-31 Thread Peter Geoghegan
On 31 May 2012 16:23, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: In what way is it possibly destabilising? I'm prepared to believe that it only affects performance, but it could be destabilizing to that.  It needs proper review and testing, and the next CF

Re: [HACKERS] Draft release notes complete

2012-05-31 Thread Bruce Momjian
On Tue, May 15, 2012 at 12:57:37PM -0400, Magnus Hagander wrote: On Fri, May 11, 2012 at 11:44 AM, Andrew Dunstan and...@dunslane.net wrote: On 05/11/2012 05:32 AM, Magnus Hagander wrote: But in the interest of actually being productive - what *is* the usecase for needing a 5 minute

Re: [HACKERS] Draft release notes complete

2012-05-31 Thread Magnus Hagander
On Thu, May 31, 2012 at 5:55 PM, Bruce Momjian br...@momjian.us wrote: On Tue, May 15, 2012 at 12:57:37PM -0400, Magnus Hagander wrote: On Fri, May 11, 2012 at 11:44 AM, Andrew Dunstan and...@dunslane.net wrote: On 05/11/2012 05:32 AM, Magnus Hagander wrote: But in the interest of

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 11:23 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 31 May 2012, Robert Haas wrote: Thanks.  How did you generate this perf report?  It's cool, because I haven't figured out how to make perf generate a report that is easily email-able, and it seems you have.

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Merlin Moncure
On Thu, May 31, 2012 at 10:23 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 31 May 2012, Robert Haas wrote: Thanks.  How did you generate this perf report?  It's cool, because I haven't figured out how to make perf generate a report that is easily email-able, and it seems you have.

Re: [HACKERS] hot standby PSQL 9.1 Windows 2008 Servers

2012-05-31 Thread Kevin Grittner
chinnaobi wrote: You mean when the primary which is going to switch its role to standby might not have sent all the WAL records to the standby and If it is switched to standby it has more WAL records than the standby which is now serves as primary. Is it ?? What happens when there is a

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Sergey Koposov
On Thu, 31 May 2012, Robert Haas wrote: Oh, ho. So from this we can see that the problem is that we're getting huge amounts of spinlock contention when pinning and unpinning index pages. It would be nice to have a self-contained reproducible test case for this, so that we could experiment

Re: [HACKERS] Figuring out shared buffer pressure

2012-05-31 Thread Bruce Momjian
On Wed, May 30, 2012 at 05:55:07PM -0400, Bruce Momjian wrote: Seems buffers_alloc is the number of calls to StrategyGetBuffer(), which tells how many time we have requested a buffer.  Not sure how that helps measure buffer pressure. Once the linked list is empty, every request for a

Re: [HACKERS] hot standby PSQL 9.1 Windows 2008 Servers

2012-05-31 Thread chinnaobi
Dear Kevin, Thank you for your reply. Yeah I am writing an application using powershell, it's true it is not trivial and especially a guy like me who has no idea on database. You raised all the cases which I am muddling with, But currently I am testing this setup: Always standby server is

Re: [HACKERS] hot standby PSQL 9.1 Windows 2008 Servers

2012-05-31 Thread chinnaobi
Sorry to mention, In my setup the primary and standby servers receive same traffic, so no issue with the network fault between the primary and the standby, but not between the primary and some of the clients updating it -- View this message in context:

Re: [HACKERS] heap_form_tuple crashing

2012-05-31 Thread Atri Sharma
On Thu, May 31, 2012 at 7:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Atri Sharma atri.j...@gmail.com writes: My call to heap_form_tuple looks like: val1=0; tupledesc1=BlessTupleDesc(node-ss.ss_currentRelation-rd_att); tuple=heap_form_tuple(tupledesc1,p1,val1); p1 is a pointer to a Datum

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Merlin Moncure
On Thu, May 31, 2012 at 11:54 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 31 May 2012, Robert Haas wrote: Oh, ho.  So from this we can see that the problem is that we're getting huge amounts of spinlock contention when pinning and unpinning index pages. It would be nice to have

[HACKERS] Create collation incorrect error code

2012-05-31 Thread Magnus Hagander
Strangeness: template1=# create collation nb_NO.utf8 (locale=nb_NO.utf8); ERROR: could not create locale nb_no.utf8: Success Clearly it's not successful... On some runs, I get: template1=# create collation nb_NO.utf8 (locale=nb_NO.utf8); ERROR: could not create locale nb_no.utf8: No such

Re: [HACKERS] slow dropping of tables, DropRelFileNodeBuffers, tas

2012-05-31 Thread Sergey Koposov
On Thu, 31 May 2012, Simon Riggs wrote: That struck me as a safe and easy optimisation. This was a problem I'd been trying to optimise for 9.2, so I've written a patch that appears simple and clean enough to be applied directly. Thanks! The patch indeed improved the timings, The dropping of

Re: [HACKERS] Issues with MinGW W64

2012-05-31 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, May 31, 2012 at 8:15 AM, Johann 'Myrkraverk' Oskarsson joh...@2ndquadrant.com wrote: Is this something to discuss with the MinGW W64 team? My viewpoint on this (which is different than Tom's) is that we're probably not entitled to assume

Re: [HACKERS] Create collation incorrect error code

2012-05-31 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: template1=# create collation nb_NO.utf8 (locale=nb_NO.utf8); ERROR: could not create locale nb_no.utf8: Success What platform? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Create collation incorrect error code

2012-05-31 Thread Magnus Hagander
On Thu, May 31, 2012 at 8:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: template1=# create collation nb_NO.utf8 (locale=nb_NO.utf8); ERROR:  could not create locale nb_no.utf8: Success What platform? D'uh, sorry. Ubuntu 12.04. --  Magnus Hagander  

Re: [HACKERS] Figuring out shared buffer pressure

2012-05-31 Thread Cédric Villemain
Le jeudi 31 mai 2012 19:11:07, Bruce Momjian a écrit : On Wed, May 30, 2012 at 05:55:07PM -0400, Bruce Momjian wrote: Seems buffers_alloc is the number of calls to StrategyGetBuffer(), which tells how many time we have requested a buffer. Not sure how that helps measure buffer

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 2:03 PM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, May 31, 2012 at 11:54 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 31 May 2012, Robert Haas wrote: Oh, ho.  So from this we can see that the problem is that we're getting huge amounts of spinlock

Re: [HACKERS] Issues with MinGW W64

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 2:13 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, May 31, 2012 at 8:15 AM, Johann 'Myrkraverk' Oskarsson joh...@2ndquadrant.com wrote: Is this something to discuss with the MinGW W64 team? My viewpoint on this (which is

[HACKERS] \conninfo and SSL

2012-05-31 Thread Bruce Momjian
On startup, psql shows the SSL information: $ psql 'sslmode=require host=localhost' psql (9.2beta1) SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256) However, \conninfo does not mention SSL: postgres= \conninfo You are connected to database postgres

Re: [HACKERS] \conninfo and SSL

2012-05-31 Thread Magnus Hagander
On Thu, May 31, 2012 at 9:04 PM, Bruce Momjian br...@momjian.us wrote: On startup, psql shows the SSL information:        $ psql 'sslmode=require host=localhost'        psql (9.2beta1)        SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256) However, \conninfo does not mention SSL:  

Re: [HACKERS] extending relations more efficiently

2012-05-31 Thread Jeff Davis
On Tue, 2012-05-01 at 10:08 -0400, Robert Haas wrote: We've previously discussed the possible desirability of extending relations in larger increments, rather than one block at a time, for performance reasons. I attempted to determine how much performance we could possibly buy this way, and

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Merlin Moncure
On Thu, May 31, 2012 at 1:50 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, May 31, 2012 at 2:03 PM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, May 31, 2012 at 11:54 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 31 May 2012, Robert Haas wrote: Oh, ho.  So from this we

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-05-31 Thread Robert Haas
On Wed, May 30, 2012 at 12:17 PM, Fujii Masao masao.fu...@gmail.com wrote: OTOH, I wonder whether we really need to send keepalive messages periodically to calculate a network latency. ISTM we don't unless a network latency varies from situation to situation so frequently and we'd like to

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-31 Thread Kohei KaiGai
2012/5/31 Robert Haas robertmh...@gmail.com: If we would have an ideal optimizer, I'd still like the optimizer to wipe out redundant clauses transparently, rather than RLSBYPASS permissions, because it just controls all-or-nothing stuff. For example, if tuples are categorized to unclassified,

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-31 Thread Kohei KaiGai
2012/5/31 Kohei KaiGai kai...@kaigai.gr.jp: 2012/5/31 Robert Haas robertmh...@gmail.com: If we would have an ideal optimizer, I'd still like the optimizer to wipe out redundant clauses transparently, rather than RLSBYPASS permissions, because it just controls all-or-nothing stuff. For

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Robert Haas
On Thu, May 31, 2012 at 3:25 PM, Merlin Moncure mmonc...@gmail.com wrote: Hm, couple questions: how do you determine if/when to un-nail a buffer, and who makes that decision (bgwriter?) Well, I think some experimentation might be required, but my first thought is to tie it into buffer eviction.

Re: [HACKERS] Figuring out shared buffer pressure

2012-05-31 Thread Daniel Farina
On Thu, May 31, 2012 at 10:11 AM, Bruce Momjian br...@momjian.us wrote: However, this doesn't help people configure shared buffers larger (e.g. 35%) if their working set is larger.  Right now, I don't see how a user would know this is happening.  On the flip side, they might have a smaller

Re: [HACKERS] pgbench--new transaction type

2012-05-31 Thread Jeff Janes
On Sun, Jun 19, 2011 at 3:30 PM, Greg Smith g...@2ndquadrant.com wrote: I applied Jeff's patch but changed this to address concerns about the program getting stuck running for too long in the function: #define plpgsql_loops   512 This would be better named as plpgsql_batch_size or something

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Jeff Janes
On Sun, May 27, 2012 at 11:45 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: Hi, I did another test using the same data and the same code, which I've provided before and the performance of the single thread seems to be degrading quadratically with the number of threads. Here are the

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Jeff Janes
On Thu, May 31, 2012 at 9:17 AM, Robert Haas robertmh...@gmail.com wrote: Oh, ho.  So from this we can see that the problem is that we're getting huge amounts of spinlock contention when pinning and unpinning index pages. It would be nice to have a self-contained reproducible test case for

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Jeff Janes
On Thu, May 31, 2012 at 11:50 AM, Robert Haas robertmh...@gmail.com wrote: This test case is unusual because it hits a whole series of buffers very hard.  However, there are other cases where this happens on a single buffer that is just very, very hot, like the root block of a btree index,

Re: [HACKERS] slow dropping of tables, DropRelFileNodeBuffers, tas

2012-05-31 Thread Jeff Janes
On Thu, May 31, 2012 at 11:09 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 31 May 2012, Simon Riggs wrote: That struck me as a safe and easy optimisation. This was a problem I'd been trying to optimise for 9.2, so I've written a patch that appears simple and clean enough to be

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Jeff Janes
On Wed, May 30, 2012 at 6:10 PM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Wed, 30 May 2012, Jeff Janes wrote: But anyway, is idt_match a fairly static table?  If so, I'd partition that into 16 tables, and then have each one of your tasks join against a different one of those tables.  

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-05-31 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Now, measuring time skew is potentially a useful thing to do, if we believe that this will actually give us an accurate measurement of what the time skew is, because there are a whole series of things that people want to do which involve subtracting a

Re: [HACKERS] slow dropping of tables, DropRelFileNodeBuffers, tas

2012-05-31 Thread Simon Riggs
On 31 May 2012 19:09, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 31 May 2012, Simon Riggs wrote: That struck me as a safe and easy optimisation. This was a problem I'd been trying to optimise for 9.2, so I've written a patch that appears simple and clean enough to be applied