Re: [HACKERS] reindex creates predicate lock on index root

2011-06-08 Thread Heikki Linnakangas
On 08.06.2011 06:37, Kevin Grittner wrote: You're right; that one was a false alarm. While REINDEX was reading the heap to build the index it got an SIREAD lock on a *heap* page. We never take locks on heap pages directly, so it must've been a promotion from heap tuple locks. While that

Re: [HACKERS] [PATCH] Bug in XPATH() if expression returns a scalar value

2011-06-08 Thread Florian Pflug
On Jun6, 2011, at 14:56 , Peter Eisentraut wrote: On tis, 2011-05-31 at 16:19 +0200, Florian Pflug wrote: If people deem this to be a problem, we could instead add a separate function XPATH_VALUE() that returns VARCHAR, and make people use that for scalar-value-returning expressions. Why

[HACKERS] SSI heap_insert and page-level predicate locks

2011-06-08 Thread Heikki Linnakangas
heap_insert() calls CheckForSerializableConflictIn(), which checks if there is a predicate lock on the whole relation, or on the page we're inserting to. It does not check for tuple-level locks, because there can't be any locks on a tuple that didn't exist before. AFAICS, the check for page

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-08 Thread Dan Ports
On Tue, Jun 07, 2011 at 10:14:30PM -0400, Tom Lane wrote: Do you mean page zero, as in the metapage (for most index types), or do you mean the root page? If the former, how is that not an outright bug, since it corresponds to no data? If the latter, how is that not a serious performance

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-08 Thread Robert Creager
On Jun 7, 2011, at 12:42 PM, Alex Hunsaker bada...@gmail.com wrote: On Tue, Jun 7, 2011 at 12:22, Tom Lane t...@sss.pgh.pa.us wrote: Alex Hunsaker bada...@gmail.com writes: Im looking at the raw perl 5.10.0 source... I wonder if apple is shipping a modified version? You could find out by

Re: [HACKERS] smallserial / serial2

2011-06-08 Thread Mike Pultz
Yup- it's attached. Mike From: Brar Piening [mailto:b...@gmx.de] Sent: Tuesday, June 07, 2011 6:58 PM To: Mike Pultz Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] smallserial / serial2 On Wed, 20 Apr 2011 21:27:27 -0400, Mike Pultz mailto:m...@mikepultz.com

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-08 Thread Robert Creager
On Jun 7, 2011, at 3:01 PM, Tom Lane wrote: Robert Creager robert.crea...@oracle.com writes: The configure log mentioned upthread says it's finding /usr/bin/perl, so apparently the buildfarm is running with a different PATH than you're using here. But that log also shows

Re: [HACKERS] SSI heap_insert and page-level predicate locks

2011-06-08 Thread Dan Ports
On Wed, Jun 08, 2011 at 11:23:48AM +0300, Heikki Linnakangas wrote: AFAICS, the check for page lock is actually unnecessary. A page-level lock on a heap only occurs when tuple-level locks are promoted. It is just a coarser-grain representation of holding locks on all tuples on the page,

Re: [HACKERS] SSI heap_insert and page-level predicate locks

2011-06-08 Thread Heikki Linnakangas
On 08.06.2011 12:36, Dan Ports wrote: On Wed, Jun 08, 2011 at 11:23:48AM +0300, Heikki Linnakangas wrote: AFAICS, the check for page lock is actually unnecessary. A page-level lock on a heap only occurs when tuple-level locks are promoted. It is just a coarser-grain representation of holding

[HACKERS] .gitignore for some of cygwin files

2011-06-08 Thread Radosław Smogura
After compilation I got big list of differences about binaries and other files. You may see it in patch. Regards, Radekdiff --git a/.gitignore b/.gitignore old mode 100644 new mode 100755 index 81c4d5e..75e1d9a --- a/.gitignore +++ b/.gitignore @@ -20,6 +20,14 @@ lcov.info *.vcproj

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-08 Thread Heikki Linnakangas
On 08.06.2011 03:16, Kevin Grittner wrote: + /* +* It's OK to remove the old lock first because of the ACCESS +* EXCLUSIVE lock on the heap relation when this is called. It is +* desirable to do so

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-08 Thread Heikki Linnakangas
On 08.06.2011 14:18, Heikki Linnakangas wrote: I just looked back at your old email where you listed the different DDL operations, and notice that we missed VACUUM FULL as well (http://archives.postgresql.org/message-id/4dbd7e9102250003d...@gw.wicourts.gov). I'll look into that. Never mind

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-08 Thread Heikki Linnakangas
(sorry for repeatedly replying to self. I'll go for a coffee after this...) On 08.06.2011 14:18, Heikki Linnakangas wrote: Committed after adjusting that comment. I did a lot of other cosmetic changes too, please double-check that I didn't screw up anything. Also, it would be nice to have

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 1:59 AM, Merlin Moncure mmonc...@gmail.com wrote: There's probably an obvious explanation that I'm not seeing, ... Yep. :-) but if you're not delegating the work of writing the buffers out to someone else, why do you need to lock the per backend buffer at all?  That

[HACKERS] Autoanalyze and OldestXmin

2011-06-08 Thread Pavan Deolasee
Hi All, I was running some pgbench tests and observed this phenomenon. This might be a known issue, but I thought its nevertheless worth mentioning. Auto-analyze process grabs a MVCC snapshot. If it runs on a very large table, it may take considerable time and would stop the OldestXmin from

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 4:59 AM, Dan Ports d...@csail.mit.edu wrote: On Tue, Jun 07, 2011 at 10:14:30PM -0400, Tom Lane wrote: Do you mean page zero, as in the metapage (for most index types), or do you mean the root page?  If the former, how is that not an outright bug, since it corresponds to

Re: [HACKERS] SSI heap_insert and page-level predicate locks

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 5:36 AM, Dan Ports d...@csail.mit.edu wrote: On Wed, Jun 08, 2011 at 11:23:48AM +0300, Heikki Linnakangas wrote: AFAICS, the check for page lock is actually unnecessary. A page-level lock on a heap only occurs when tuple-level locks are promoted. It is just a

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Merlin Moncure
On Wed, Jun 8, 2011 at 7:44 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 8, 2011 at 1:59 AM, Merlin Moncure mmonc...@gmail.com wrote: There's probably an obvious explanation that I'm not seeing, ... Yep.  :-) but if you're not delegating the work of writing the buffers out to

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-08 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: (sorry for repeatedly replying to self. I'll go for a coffee after this...) That's so nice of you to try to make me feel better for the serious brain fade I suffered yesterday. ;-) On 08.06.2011 14:18, Heikki Linnakangas

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-08 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: I'm wondering if this shouldn't be linked to whether the scan is using an MVCC snapshot, rather than inserting exceptions for specific operations. Yeah, that was raised before somewhere and I spaced it. Grabbing predicate locks for non-MVCC

Re: [HACKERS] Error in PQsetvalue

2011-06-08 Thread Merlin Moncure
On Mon, Jun 6, 2011 at 1:42 PM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, Jun 6, 2011 at 7:09 AM, Pavel Golub pa...@microolap.com wrote: Sorry for delay in answer. Yeah, I'm glad to. Should I apply this patch by myself? sure, run it against your test case and make sure it works. if so,

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 10:18 AM, Merlin Moncure mmonc...@gmail.com wrote: One point i'm missing though.  Getting back to your original idea, how does writing to shmem prevent you from having to keep buffers pinned? I'm reading your comment here: Those buffers are stamped with a fake LSN that

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-08 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: *** a/src/backend/storage/lmgr/predicate.c --- b/src/backend/storage/lmgr/predicate.c *** *** 274,279 --- 274,280 #define SkipSerialization(relation) \ ((!IsolationIsSerializable()) \ ||

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-08 Thread Heikki Linnakangas
On 08.06.2011 18:09, Kevin Grittner wrote: Robert Haasrobertmh...@gmail.com wrote: I'm wondering if this shouldn't be linked to whether the scan is using an MVCC snapshot, rather than inserting exceptions for specific operations. Yeah, that was raised before somewhere and I spaced it.

Re: [HACKERS] Error in PQsetvalue

2011-06-08 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: I went ahead and tested andrew's second patch -- can we get this reviewed and committed? Add it to the upcoming commitfest. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Autoanalyze and OldestXmin

2011-06-08 Thread Greg Stark
On Jun 8, 2011 1:49 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Hi All, There is a PROC_IN_ANALYZE flag, but we don't seem to be using it anywhere. Since acquire_sample_rows() returns palloced tuples, can't we let OldestXmin advance after scanning a page by ignoring procs with the flag

Re: [HACKERS] SSI heap_insert and page-level predicate locks

2011-06-08 Thread Alvaro Herrera
Excerpts from Heikki Linnakangas's message of mié jun 08 05:45:35 -0400 2011: On 08.06.2011 12:36, Dan Ports wrote: The only thing I'd be worried about is that at some future point we might add heap page locks -- say, for sequential scans that don't read the entire relation -- and expect

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-08 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: The attached patch has not yet been tested, but I'll test it today along with the latest committed code. You can't use GetActiveSnapshot() for this. Yeah, it didn't take much testing to find that out. I had a naive assumption

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Jim Nasby
On Jun 7, 2011, at 8:24 AM, Stephen Frost wrote: * Alvaro Herrera (alvhe...@commandprompt.com) wrote: I note that if 2nd Quadrant is interested in having a game-changing platform without having to wait a full year for 9.2, they can obviously distribute a modified version of Postgres that

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-08 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: You can have one snapshot pushed to the active snapshot stack, and do a DDL operation like reindex using a different snapshot. You'll have to check the snapshot in the HeapScanDesc.

Re: [HACKERS] Error in PQsetvalue

2011-06-08 Thread Merlin Moncure
On Wed, Jun 8, 2011 at 10:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: I went ahead and tested andrew's second patch -- can we get this reviewed and committed? Add it to the upcoming commitfest. It's a client crashing bug in PQsetvalue that goes back to

Re: [HACKERS] Range Types and extensions

2011-06-08 Thread Jeff Davis
On Tue, 2011-06-07 at 10:20 -0700, Jeff Davis wrote: BTW, Jeff, have you worked out the implications of collations for textual range types? Well, it seems to work is about as far as I've gotten. As far as the implications, I'll need to do a little more research and thinking. But I don't

Re: [HACKERS] SSI heap_insert and page-level predicate locks

2011-06-08 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Heikki Linnakangas's message of mié jun 08 05:45:35 -0400 2011: On 08.06.2011 12:36, Dan Ports wrote: The only thing I'd be worried about is that at some future point we might add heap page locks -- say, for sequential scans that

Re: [HACKERS] Error in PQsetvalue

2011-06-08 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: On Wed, Jun 8, 2011 at 10:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: I went ahead and tested andrew's second patch -- can we get this reviewed and committed? Add it to the upcoming commitfest. It's a client

Re: [HACKERS] Range Types and extensions

2011-06-08 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: It looks like the type input function may be a problem, because it doesn't look like it knows what the collation is yet. In other words, PG_GET_COLLATION() is zero for the type input function. Yeah, we've assumed that I/O functions do not need to know

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-08 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: *** a/src/backend/storage/lmgr/predicate.c --- b/src/backend/storage/lmgr/predicate.c *** *** 274,279 --- 274,280 #define SkipSerialization(relation) \

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Simon Riggs
On Wed, Jun 8, 2011 at 5:19 AM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Mon, Jun 6, 2011 at 10:49 AM, Simon Riggs si...@2ndquadrant.com wrote: My point was that we have in the past implemented performance changes to increase scalability at the last minute, and also that

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-08 Thread Alex Hunsaker
On Tue, Jun 7, 2011 at 12:42, Alex Hunsaker bada...@gmail.com wrote: On Tue, Jun 7, 2011 at 12:22, Tom Lane t...@sss.pgh.pa.us wrote: Alex Hunsaker bada...@gmail.com writes: Im looking at the raw perl 5.10.0 source... I wonder if apple is shipping a modified version? You could find out by

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 11:39 AM, Jim Nasby j...@nasby.net wrote: On Jun 7, 2011, at 8:24 AM, Stephen Frost wrote: * Alvaro Herrera (alvhe...@commandprompt.com) wrote: I note that if 2nd Quadrant is interested in having a game-changing platform without having to wait a full year for 9.2, they

Re: [HACKERS] Error in PQsetvalue

2011-06-08 Thread Merlin Moncure
On Wed, Jun 8, 2011 at 11:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: On Wed, Jun 8, 2011 at 10:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: I went ahead and tested andrew's second patch -- can we get this reviewed

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Simon Riggs
On Wed, Jun 8, 2011 at 6:02 AM, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Simon is right that we slipped the vxid patch into 8.3 when a Postgres user I talked to at Linuxworld mentioned high vacuum freeze activity and simple calculations showed the many

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Simon Riggs
On Wed, Jun 8, 2011 at 5:33 AM, Bruce Momjian br...@momjian.us wrote: One more thing --- when Tom applied that patch during 8.3 beta it was with everyone's agreement, so the policy should be that if we are going to break the rules, everyone has to agree --- if anyone disagrees, the rules

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 12:25 PM, Simon Riggs si...@2ndquadrant.com wrote: As a result of this, I've been insulted, told I have no respect for process and even suggested there was a threat of patch war. Well, you've pretty much said flat out you don't like the process, and you don't agree with

Re: [HACKERS] Error in PQsetvalue

2011-06-08 Thread Andrew Chernow
On 6/8/2011 12:03 PM, Tom Lane wrote: Merlin Moncuremmonc...@gmail.com writes: On Wed, Jun 8, 2011 at 10:18 AM, Tom Lanet...@sss.pgh.pa.us wrote: Merlin Moncuremmonc...@gmail.com writes: I went ahead and tested andrew's second patch -- can we get this reviewed and committed? Add it to

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Simon Riggs
On Wed, Jun 8, 2011 at 5:32 PM, Robert Haas robertmh...@gmail.com wrote: It took me about 3 days to write the patch.  I've now spent the better part of a day on this scheduling discussion.  I would rather have spent that time improving the patch.  Or working on some other patch.  Or getting

Re: [HACKERS] Domains versus polymorphic functions, redux

2011-06-08 Thread Tom Lane
I wrote: Anyway, I think we're out of time to do anything about the issue for 9.1. I think what we'd better do is force a downcast in the context of matching to an ANYARRAY parameter, and leave the other cases to revisit later. Attached is a draft patch to do the above. It's only lightly

Re: [HACKERS] Autoanalyze and OldestXmin

2011-06-08 Thread Pavan Deolasee
On Wed, Jun 8, 2011 at 9:03 PM, Greg Stark st...@mit.edu wrote: On Jun 8, 2011 1:49 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Hi All, There is a PROC_IN_ANALYZE flag, but we don't seem to be using it anywhere. Since acquire_sample_rows() returns palloced tuples, can't we let

[HACKERS] [PATCH] Allow star syntax in GROUP BY, as a shorthand for all table columns

2011-06-08 Thread Marti Raudsepp
Hi, This patch enables the syntax GROUP BY tablename.* in cases where earlier you'd get the error field must appear in the GROUP BY clause or be used in an aggregate function I've often needed to write queries like this: SELECT a.x, a.y, a.z, sum(b.w) FROM a JOIN b USING (a_id) GROUP BY a.x,

Re: [HACKERS] Autoanalyze and OldestXmin

2011-06-08 Thread Tom Lane
Pavan Deolasee pavan.deola...@gmail.com writes: I first thought that analyze and vacuum can not run concurrently on the same table since they take a conflicting lock on the table. So even if we ignore the analyze process while calculating the OldestXmin for vacuum, we should be fine since we

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Simon Riggs
On Wed, Jun 8, 2011 at 5:44 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 8, 2011 at 12:25 PM, Simon Riggs si...@2ndquadrant.com wrote: As a result of this, I've been insulted, told I have no respect for process and even suggested there was a threat of patch war. Well, you've

Re: [HACKERS] [PATCH] Allow star syntax in GROUP BY, as a shorthand for all table columns

2011-06-08 Thread Tom Lane
Marti Raudsepp ma...@juffo.org writes: This patch enables the syntax GROUP BY tablename.* in cases where earlier you'd get the error field must appear in the GROUP BY clause or be used in an aggregate function Is this really necessary now that we know about GROUP BY primary key? The patch is

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Joshua Berkus
Simon, The point I have made is that I disagree with a feature freeze date fixed ahead of time without regard to the content of the forthcoming release. I've not said I disagree with feature freezes altogether, which would be utterly ridiculous. Fixed dates are IMHO much less important than

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-08 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mar jun 07 12:26:34 -0400 2011: It's not *that* many levels: in fact, I think md.c is the only level that would just have to pass it through without doing anything useful. I think that working from there is a saner and more efficient approach than what

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Joshua D. Drake
On 06/07/2011 11:55 AM, Tom Lane wrote: Simon Riggssi...@2ndquadrant.com writes: Before you arrived, it was quite normal to suggest tuning patches after feature freeze. *Low risk* tuning patches make sense at this stage, yes. Fooling with the lock mechanisms doesn't qualify as low risk in

[HACKERS] gcc 4.6 and hot standby

2011-06-08 Thread Alex Hunsaker
So I've been delaying moving some production boxes over to 9.0.4 from 9.0.2 because hot standby fails with: (this is on the hot standby machine that connects to the master) 2011-06-08 11:40:48 MDT [6072]: [2-1] user= LOG: entering standby mode 2011-06-08 11:40:48 MDT [6072]: [3-1] user= DEBUG:

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 1:10 PM, Simon Riggs si...@2ndquadrant.com wrote: Why do you address this to me? Many others have been committing patches against raised issues well after feature freeze. No one other than you has proposed committing anything nearly as invasive as this, and the great

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-08 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Okay, here's a patch implementing this idea. It seems to work quite well, and it solves the problem in a limited testing scenario -- I haven't yet tested on the customer machines. This seems mostly sane, except I think you have not considered

[HACKERS] Another pgindent run before beta2?

2011-06-08 Thread Tom Lane
Should we consider doing $SUBJECT? Given the plan to branch off 9.1 after we make the beta release, I think it's now or never for a second pgindent run for 9.1. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Another pgindent run before beta2?

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 2:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Should we consider doing $SUBJECT?  Given the plan to branch off 9.1 after we make the beta release, I think it's now or never for a second pgindent run for 9.1. +1. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Another pgindent run before beta2?

2011-06-08 Thread Magnus Hagander
On Wed, Jun 8, 2011 at 20:32, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 8, 2011 at 2:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Should we consider doing $SUBJECT?  Given the plan to branch off 9.1 after we make the beta release, I think it's now or never for a second pgindent run for

Re: [HACKERS] gcc 4.6 and hot standby

2011-06-08 Thread Tom Lane
Alex Hunsaker bada...@gmail.com writes: So I've been delaying moving some production boxes over to 9.0.4 from 2011-06-08 11:41:03 MDT [6078]: [1-1] user= FATAL: terminating walreceiver process due to administrator command [ repeats... ] I suppose the next step is to narrow it down to a

Re: [HACKERS] gcc 4.6 and hot standby

2011-06-08 Thread Alex Hunsaker
On Wed, Jun 8, 2011 at 12:12, Alex Hunsaker bada...@gmail.com wrote: So I've been delaying moving some production boxes over to 9.0.4 from 9.0.2 because hot standby fails with: (this is on the hot standby machine that connects to the master) [ ...] 2011-06-08 11:41:03 MDT [6072]: [18-1] user=

Re: [HACKERS] gcc 4.6 and hot standby

2011-06-08 Thread Alex Hunsaker
On Wed, Jun 8, 2011 at 12:49, Tom Lane t...@sss.pgh.pa.us wrote: Alex Hunsaker bada...@gmail.com writes: So I've been delaying moving some production boxes over to 9.0.4 from 2011-06-08 11:41:03 MDT [6078]: [1-1] user= FATAL:  terminating walreceiver process due to administrator command [

Re: [HACKERS] gcc 4.6 and hot standby

2011-06-08 Thread Tom Lane
Alex Hunsaker bada...@gmail.com writes: So I tracked it down to -fgcse, that is CFLAGS=-O2 -fno-gcse ./configure works. I then took a few guesses and compiled all of postgres with -O2, then manually recompiled xlog.c with -f-no-gcse. that combination seems to work. Huh, interesting. So the

Re: [HACKERS] contrib/citext versus collations

2011-06-08 Thread Tom Lane
I wrote: On further reflection, I'm wondering exactly how much goodness to chop off there. What I'd originally been thinking was to just lobotomize the case-folding step, and allow citext's comparison operators to still respond to input collation when comparing the folded strings. However,

Re: [HACKERS] could not truncate directory pg_serial: apparent wraparound

2011-06-08 Thread Heikki Linnakangas
On 08.06.2011 03:28, Kevin Grittner wrote: We had a report of the subject message during testing a while back and attempted to address the issue. It can result in a LOG level message and the accumulation of files in the pg_serial SLRU subdirectory. We haven't seen a recurrence, until I hit it

Re: [HACKERS] Another pgindent run before beta2?

2011-06-08 Thread Andrew Dunstan
On 06/08/2011 02:29 PM, Tom Lane wrote: Should we consider doing $SUBJECT? Given the plan to branch off 9.1 after we make the beta release, I think it's now or never for a second pgindent run for 9.1. OK, I have made sure the list on the buildfarm is up to date.

Re: [HACKERS] could not truncate directory pg_serial: apparent wraparound

2011-06-08 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 08.06.2011 03:28, Kevin Grittner wrote: We had a report of the subject message during testing a while back and attempted to address the issue. It can result in a LOG level message and the accumulation of files in the pg_serial

[HACKERS] patch for 9.2: enhanced errors

2011-06-08 Thread Pavel Stehule
Hello Attached patch implements a new erros's fields that describes table, colums related to error. This enhanced info is limited to constraints and RI. example: postgres=# create table omega(a int unique not null check (a 10)); NOTICE: 0: CREATE TABLE / UNIQUE will create implicit index

Re: [HACKERS] gcc 4.6 and hot standby

2011-06-08 Thread Tom Lane
Alex Hunsaker bada...@gmail.com writes: On Wed, Jun 8, 2011 at 12:49, Tom Lane t...@sss.pgh.pa.us wrote: It might be useful to strace the postmaster and walreceiver processes just to see if any signal is actually being sent or received. Find it attached. Well, the trace shows exactly what I

[HACKERS] tuning autovacuum

2011-06-08 Thread Euler Taveira de Oliveira
Hi, There are some releases that autovacuum was enabled by default and, up to now there is an easy way to estimate the number of autovacuum workers. I tune it observing if the number of slots are saturated for a period of time. I'm having a hard time trying to do this. I want to add a LOG

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Wed, Jun 8, 2011 at 6:02 AM, Tom Lane t...@sss.pgh.pa.us wrote: Just to set the record straight on this ... the vxid patch went in on 2007-09-05: http://archives.postgresql.org/pgsql-committers/2007-09/msg00026.php which was a day shy of a month

Re: [HACKERS] tuning autovacuum

2011-06-08 Thread Magnus Hagander
On Jun 9, 2011 12:01 AM, Euler Taveira de Oliveira eu...@timbira.com wrote: Hi, There are some releases that autovacuum was enabled by default and, up to now there is an easy way to estimate the number of autovacuum workers. I tune it observing if the number of slots are saturated for a period

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-08 Thread Tom Lane
Joshua Berkus j...@agliodbs.com writes: Simon, The point I have made is that I disagree with a feature freeze date fixed ahead of time without regard to the content of the forthcoming release. I've not said I disagree with feature freezes altogether, which would be utterly ridiculous. Fixed

Re: [HACKERS] gcc 4.6 and hot standby

2011-06-08 Thread Mark Kirkwood
On 09/06/11 06:58, Alex Hunsaker wrote: Yeah :-). However ill note it looks like its the default compiler for fedora 15, ubuntu natty and debian sid. FWIW Ubuntu natty uses gcc 4.5.2, probably just as as well in the light of your findings :-) Cheers Mark -- Sent via pgsql-hackers mailing

Re: [HACKERS] SSI heap_insert and page-level predicate locks

2011-06-08 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: heap_insert() calls CheckForSerializableConflictIn(), which checks if there is a predicate lock on the whole relation, or on the page we're inserting to. It does not check for tuple-level locks, because there can't be any locks

Re: [HACKERS] smallserial / serial2

2011-06-08 Thread Brar Piening
On Tue, 7 Jun 2011 20:35:12 -0400, Mike Pultz m...@mikepultz.com wrote: New patch attached. Review for '20110607_serial2_v2.diff': Submission review - Is the patch in context diff format? Yes. - Does it apply cleanly to the current git master? Yes. - Does it include reasonable tests,

Re: [HACKERS] Autoanalyze and OldestXmin

2011-06-08 Thread Jim Nasby
On Jun 8, 2011, at 10:33 AM, Greg Stark wrote: This is kind of like the other property it would be nice to know about transactions: that they've locked all the tables they're going to lock. That sounds like something I've wanted for a very long time: the ability for a transaction to say

[HACKERS] SSI work for 9.1

2011-06-08 Thread Kevin Grittner
From a review of recent emails I've put together a list of what I'm going to try to do this evening, in order of attack. It's ambitious and I may well not get to the end tonight, but I wanted to get the issues on record in list form. If someone spots one I'm missing or thinks I should change the

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Jim Nasby
On Jun 8, 2011, at 10:15 AM, Robert Haas wrote: That suggests to me that you have to keep them pinned anyways. I'm still a bit fuzzy on how the per-backend buffers being in shm conveys any advantage. IOW, (trying not to be obtuse) under what circumstances would backend A want to read from or

Re: [HACKERS] gcc 4.6 and hot standby

2011-06-08 Thread Alex Hunsaker
On Wed, Jun 8, 2011 at 16:20, Mark Kirkwood mark.kirkw...@catalyst.net.nz wrote: On 09/06/11 06:58, Alex Hunsaker wrote: Yeah :-). However ill note it looks like its the default compiler for fedora 15, ubuntu natty and debian sid. FWIW Ubuntu natty uses gcc 4.5.2, probably just as as well

Re: [HACKERS] smallserial / serial2

2011-06-08 Thread Jim Nasby
On Jun 8, 2011, at 5:36 PM, Brar Piening wrote: Pros Mike Pultz (patch author): since serial4 and serial8 are simply pseudo-types- effectively there for convenience, I’d argue that it should simply be there for completeness Robert Haas: We should be trying to put all types on equal footing,

Re: [HACKERS] smallserial / serial2

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 6:36 PM, Brar Piening b...@gmx.de wrote: New patch attached. Review for '20110607_serial2_v2.diff': I see you added this review to the CommitFest application - excellent. You should also change the status to either Waiting on Author or Ready for Committer based on the

Re: [HACKERS] tuning autovacuum

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 5:54 PM, Euler Taveira de Oliveira eu...@timbira.com wrote: LOG: maximum number of autovacuum workers reached HINT: Consider increasing autovacuum_max_workers (currently 5). Comments? Is the hint correct? I mean, what if there were 100 small tables that needed

Re: [HACKERS] Proposal: Another attempt at vacuum improvements

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 1:19 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: I went on to create a WIP patch based on our discussion. There are couple of issues that I stumbled upon while testing it. 1. The start-of-index-vacuum LSN that we want to track must be noted even before the heap

Re: [HACKERS] SSI work for 9.1

2011-06-08 Thread Dan Ports
On Wed, Jun 08, 2011 at 05:48:26PM -0500, Kevin Grittner wrote: (1) Pass snapshot in to some predicate.c functions. The particular functions have yet to be determined, but certainly any which acquire predicate locks, and probably all which are guarded by the SkipSerialization() macro. Skip

Re: [HACKERS] tuning autovacuum

2011-06-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I think it'd be really useful to expose some more data in this area though. One random idea is - remember the time at which a table was first observed to need vacuuming. Clear the timestamp when it gets vacuumed. Then you can do: As far as I recall

Re: [HACKERS] tuning autovacuum

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 9:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I think it'd be really useful to expose some more data in this area though.  One random idea is - remember the time at which a table was first observed to need vacuuming. Clear the

Re: [HACKERS] SSI work for 9.1

2011-06-08 Thread Kevin Grittner
Dan Ports wrote: On Wed, Jun 08, 2011 at 05:48:26PM -0500, Kevin Grittner wrote: (1) Pass snapshot in to some predicate.c functions. The particular functions have yet to be determined, but certainly any which acquire predicate locks, and probably all which are guarded by the

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 6:49 PM, Jim Nasby j...@nasby.net wrote: If backend A needs to evict a buffer with a fake LSN, it can go find the WAL that needs to be serialized, do that, flush WAL, and then evict the buffer. Isn't the only time that you'd need to evict if you ran out of buffers?

Re: [HACKERS] gcc 4.6 and hot standby

2011-06-08 Thread Fujii Masao
On Thu, Jun 9, 2011 at 5:31 AM, Tom Lane t...@sss.pgh.pa.us wrote: So it's interesting that this only happens with a particular gcc version, because it's not apparent to me why it works properly for anybody. Isn't hitting a zero record length an expected case when we run ahead of the amount of

Re: [HACKERS] tuning autovacuum

2011-06-08 Thread Euler Taveira de Oliveira
Em 08-06-2011 20:35, Robert Haas escreveu: Is the hint correct? I mean, what if there were 100 small tables that needed vacuuming all at the same time. We'd hit this limit no matter how high you set autovacuum_max_workers, but it wouldn't be right to set it to 101 just because every once in a

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Merlin Moncure
On Wed, Jun 8, 2011 at 10:21 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 8, 2011 at 6:49 PM, Jim Nasby j...@nasby.net wrote: If backend A needs to evict a buffer with a fake LSN, it can go find the WAL that needs to be serialized, do that, flush WAL, and then evict the buffer.

[HACKERS] literature on write-ahead logging

2011-06-08 Thread Robert Haas
I did a brief literature search for papers on breaking the WAL-serialization bottleneck today and hit upon this: Aether: A Scalable Approach to Logging, Ryan Johnson, Ippokratis Pandis, et al. http://infoscience.epfl.ch/record/149436/files/vldb10aether.pdf Section 5 appears to be the most

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 11:20 PM, Merlin Moncure mmonc...@gmail.com wrote: You're probably right.  I think though there is enough hypothetical upside to the private buffer case that it should be attempted just to see what breaks. The major tricky bit is dealing with the new pin/unpin mechanics.

Re: [HACKERS] SSI work for 9.1

2011-06-08 Thread Kevin Grittner
Kevin Grittner wrote: A patch is attached which just covers the predicate lock acquisition This patch rolls that up with snapshot checking in the conflict detection function called on read. The only other two functions which use that macro check for conflicts on write, and I can't see why

Re: [HACKERS] SSI work for 9.1

2011-06-08 Thread Dan Ports
On Wed, Jun 08, 2011 at 09:17:04PM -0500, Kevin Grittner wrote: A patch is attached which just covers the predicate lock acquisition, where a snapshot is available without too much pain. There are two functions which acquire predicate locks where a snapshot was not readily available:

Re: [HACKERS] tuning autovacuum

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 10:55 PM, Euler Taveira de Oliveira eu...@timbira.com wrote: Em 08-06-2011 20:35, Robert Haas escreveu: Is the hint correct?  I mean, what if there were 100 small tables that needed vacuuming all at the same time.  We'd hit this limit no matter how high you set

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Merlin Moncure
On Wed, Jun 8, 2011 at 11:27 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 8, 2011 at 11:20 PM, Merlin Moncure mmonc...@gmail.com wrote: You're probably right.  I think though there is enough hypothetical upside to the private buffer case that it should be attempted just to see what

Re: [HACKERS] Invalid byte sequence for encoding UTF8, caused due to non wide-char-aware downcase_truncate_identifier() function on WINDOWS

2011-06-08 Thread Jeevan Chalke
On Wed, Jun 8, 2011 at 6:22 AM, Robert Haas robertmh...@gmail.com wrote: 2011/6/7 Jeevan Chalke jeevan.cha...@enterprisedb.com: since we smash the identifier to lower case using downcase_truncate_identifier() function, the solution is to make this function should be wide-char aware, like

  1   2   >