[HACKERS] SET TRANSACTION .. DEFERRABLE missing docs?
I was reading through ref/set_transaction.sgml and noticed that the only documentation of DEFERRABLE is that it's a PostgreSQL language extension, not anything about what it actually does. Same for begin and start_transaction. I see it described in README-SSI and for the guc default_transaction_deferrable, but I think it needs to be documented on that ref page as well.. (in fact, the guc docs refer to the reference page, which doesn't have more info) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] German Ladies start translation project
Hello, I just wanted to inform you that Cornelia and me start the project to translate and publish the documentation German. Means the project will have a really high women concentration at the beginning :) pgsql-www suggested to take postgresql.de for publishing. I thought it is a great idea and so we will do it. Conny and me are involved in that domain anyway. Of course everybody is allowed helping us. We are on the way to set up all systems. We will keep it simple - without much graphical gadgets. We will take the original docs from source code after compiled as html. And on new release diff will get our friend. Why are we so mad? Peter once translated the whole docs and published a book. Afair it was for PostgreSQL 7.2 or 7.3. Google got much more aggressive on language and translation topic. When I take a German browser and Google for PostgreSQL + specific detail - then I first get the old Peter docs and after that I get the original postgresql.org docs - but - automatically translated from Google into German. Believe me - this translation are sometimes really worse. For me this means - it is finally time to translate / update old Peter docs. Peter and me think - just translate all again is faster then taking and checking his old texts. I am fully back - and of course I needed something reasonable todo - I know translating docs will get a life time project. I asked Cornelia if she wants to help because she already wrote a PostgreSQL book before Peter and she has lots of experiences with translations - She is involved in PHP doc translations too. She immediately said yes. As I said - we are on the way to set up all what we need on postgresql.de. Susanne -- Susanne Ebrecht Bielefeld -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] why is max standby delay only 35 minutes?
On Fri, Mar 4, 2011 at 18:19, Robert Treat r...@xzilla.net wrote: On Fri, Mar 4, 2011 at 2:03 AM, Magnus Hagander mag...@hagander.net wrote: On Fri, Mar 4, 2011 at 04:00, Robert Treat r...@xzilla.net wrote: I have a server where I wanted to do some reporting on a standby, and wanted to set the max standby delay to 1 hour. upon doing that, i get this in the logs: 2011-03-03 21:20:08 EST () [2656]: [2-1] user=,db=LOG: received SIGHUP, reloading configuration files 2011-03-03 21:20:08 EST () [2656]: [3-1] user=,db=LOG: 360 is outside the valid range for parameter max_standby_archive_delay (-1 .. 2147483) The error is clear enough, but is there some reason that the parameter is coded this way? istm people are much more likely to want to be able to set the precision in hours than in microseconds. OTOH, maybe it's a bug? The default resolution is in milliseconds, and you can't set it to anything less than that (afaict). I asked on irc and the consensus seemed to be that the internal representation is off, are we missing something? See this thread here: http://archives.postgresql.org/pgsql-hackers/2010-12/msg01517.php Summary: should be fixed, but it needs to be verified that it works across all possible codepaths. It's not an issue with just max_standby_delay. Thanks for the pointer! I guess the next question is if anyone is working on that, and/or what would need to be done to know we've done a satisfactory job of verifying nothing breaks across all codepaths were someone to take on the job? I have it sitting on my list somewhere, but I haven't actually started doing anything... A good start is to just change the code (likely quite easy) and then test all the different ways that you can set and reset and read the values of a guc (set/show/pg_settings/anythingelseyoucanthinkof), that it's passed properly across exec_backend etc - and testing tihs on multiple platforms I guess, in particular both 32 and 64-bit... At least that's my understanding of what needs to be done :-) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, 2011-03-05 at 16:13 +0900, Fujii Masao wrote:
On Sat, Mar 5, 2011 at 7:28 AM, Simon Riggs si...@2ndquadrant.com wrote:
Almost-working patch attached for the above feature. Time to stop for
the day. Patch against current repo version.
Current repo version attached here also (v20), which includes all fixes
to all known technical issues, major polishing etc..
Thanks for the patch. Now the code about the wait list looks very
simpler than before! Here are the comments:
@@ -1337,6 +1352,31 @@ pg_stat_get_wal_senders(PG_FUNCTION_ARGS)
snip
+ if (walsnd-pid != 0 walsnd-state == WALSNDSTATE_STREAMING)
+ {
+ sync_priority[i] = walsnd-sync_standby_priority;
This always reports the priority of walsender in CATCHUP state as 0.
I don't think that priority needs to be reported as 0.
Cosmetic change. We can do this, yes.
When new standby which has the same priority as current sync standby
connects, that new standby can switch to new sync one even though
current one is still running. This happens when the index of WalSnd slot
which new standby uses is ahead of that which current one uses. People
don't expect such an unexpected switchover, I think.
It is documented that the selection of standby from a set of similar
priorities is indeterminate. Users don't like it, they can change it.
+ /*
+ * Assume the queue is ordered by LSN
+ */
+ if (XLByteLT(walsndctl-lsn, proc-waitLSN))
+ return numprocs;
The code to ensure the assumption needs to be added.
Yes, just need to add the code for traversing list backwards.
--
Simon Riggs http://www.2ndQuadrant.com/books/
PostgreSQL Development, 24x7 Support, Training and Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, Mar 5, 2011 at 7:28 AM, Simon Riggs si...@2ndquadrant.com wrote: Yes, that can happen. As people will no doubt observe, this seems to be an argument for wait-forever. What we actually need is a wait that lasts longer than it takes for us to decide to failover, if the standby is actually up and this is some kind of split brain situation. That way the clients are still waiting when failover occurs. WAL is missing, but since we didn't acknowledge the client we are OK to treat that situation as if it were an abort. Oracle Data Guard in the maximum availability mode behaves that way? I'm sure that you are implementing something like the maximum availability mode rather than the maximum protection one. So I'd like to know how the data loss situation I described can be avoided in the maximum availability mode. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Saturday 05 March 2011 04:44:20 Robert Haas wrote: * Collation-related regression failure on buildfarm member pika. This is clearly a bug we need to identify, but maybe we can ship the alpha without a fix --- for one thing, getting more than one report of the problem would be helpful. pika is still red, but it claims not to have updated in 13 days, so I'm not sure what that means. I have just yesterday hit the same bug while testing some application on then HEAD, so its not just PIKA. Will try to find the bug or file some somewhat reproducible recipe. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, 2011-03-05 at 11:04 +, Simon Riggs wrote: + /* + * Assume the queue is ordered by LSN + */ + if (XLByteLT(walsndctl-lsn, proc-waitLSN)) + return numprocs; The code to ensure the assumption needs to be added. Yes, just need to add the code for traversing list backwards. I've added code to shmqueue.c to allow this. New version pushed. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, Mar 5, 2011 at 6:04 AM, Simon Riggs si...@2ndquadrant.com wrote: It is documented that the selection of standby from a set of similar priorities is indeterminate. Users don't like it, they can change it. That doesn't seem like a good argument to *change* the synchronous standby once it's already set. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, 2011-03-05 at 07:24 -0500, Robert Haas wrote: On Sat, Mar 5, 2011 at 6:04 AM, Simon Riggs si...@2ndquadrant.com wrote: It is documented that the selection of standby from a set of similar priorities is indeterminate. Users don't like it, they can change it. That doesn't seem like a good argument to *change* the synchronous standby once it's already set. If the order is arbitrary, why does it matter if it changes? The user has the power to specify a sequence, yet they have not done so. They are told the results are indeterminate, which is accurate. I can add the words and may change as new standbys connect if that helps. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, Mar 5, 2011 at 7:49 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2011-03-05 at 07:24 -0500, Robert Haas wrote: On Sat, Mar 5, 2011 at 6:04 AM, Simon Riggs si...@2ndquadrant.com wrote: It is documented that the selection of standby from a set of similar priorities is indeterminate. Users don't like it, they can change it. That doesn't seem like a good argument to *change* the synchronous standby once it's already set. If the order is arbitrary, why does it matter if it changes? The user has the power to specify a sequence, yet they have not done so. They are told the results are indeterminate, which is accurate. I can add the words and may change as new standbys connect if that helps. I just don't think that's very useful behavior. Suppose I have a master and two standbys. Both are local (or both are remote with equally good connectivity). When one of the standbys goes down, there will be a hiccup (i.e. transactions will block trying to commit) until that guy falls off and the other one takes over. Now, when he comes back up again, I don't want the synchronous standby to change again; that seems like a recipe for another hiccup. I think who the current synchronous standby is should act as a tiebreak. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Fri, Mar 4, 2011 at 11:22 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 4, 2011 at 10:44 PM, Robert Haas robertmh...@gmail.com wrote: So it seems like the only thing that is an absolute must-do is write some release notes. Here's a rough attempt at filtering the post-alpha3 commit log down to approximately the set of things worth adding to the alpha4 release notes. Working on SGML-ifying this... -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, Mar 5, 2011 at 2:05 PM, Robert Haas robertmh...@gmail.com wrote: On Sat, Mar 5, 2011 at 7:49 AM, Simon Riggs si...@2ndquadrant.com wrote: If the order is arbitrary, why does it matter if it changes? The user has the power to specify a sequence, yet they have not done so. They are told the results are indeterminate, which is accurate. I can add the words and may change as new standbys connect if that helps. I just don't think that's very useful behavior. Suppose I have a master and two standbys. Both are local (or both are remote with equally good connectivity). When one of the standbys goes down, there will be a hiccup (i.e. transactions will block trying to commit) until that guy falls off and the other one takes over. Now, when he comes back up again, I don't want the synchronous standby to change again; that seems like a recipe for another hiccup. I think who the current synchronous standby is should act as a tiebreak. +1 TLDR part: The first one might be noticed by users because it takes tens of seconds before the sync switch. The second hiccup is hardly noticable. However limiting the # switches of sync standby to the absolute minimum is also good if e.g. (if there was a hook for it) cluster middleware is notified of the sync replica change. That might either introduce a race condition or be even completely unreliable if the notify is sent asynchronous, or it might introduce a longer lag if the master waits for confirmation of the sync replica change message. At that point sync replica changes become more expensive than they are currently. regards, Yeb Havinga
Re: [HACKERS] SET TRANSACTION .. DEFERRABLE missing docs?
Magnus Hagander wrote: I was reading through ref/set_transaction.sgml and noticed that the only documentation of DEFERRABLE is that it's a PostgreSQL language extension, not anything about what it actually does. Same for begin and start_transaction. I see it described in README-SSI and for the guc default_transaction_deferrable, but I think it needs to be documented on that ref page as well.. (in fact, the guc docs refer to the reference page, which doesn't have more info) Thanks for spotting that. I'll put something together. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Sat, Mar 5, 2011 at 8:48 AM, Alexander Korotkov korot...@intaro.ru wrote: On Sat, Mar 5, 2011 at 7:22 AM, Robert Haas robertmh...@gmail.com wrote: Here's a rough attempt at filtering the post-alpha3 commit log down to approximately the set of things worth adding to the alpha4 release notes. Seems that support LIKE and ILIKE index searches via contrib/pg_trgm indexes is not mentioned here. Good catch, thanks. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Sat, Mar 5, 2011 at 8:41 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 4, 2011 at 11:22 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 4, 2011 at 10:44 PM, Robert Haas robertmh...@gmail.com wrote: So it seems like the only thing that is an absolute must-do is write some release notes. Here's a rough attempt at filtering the post-alpha3 commit log down to approximately the set of things worth adding to the alpha4 release notes. Working on SGML-ifying this... OK, first attempt committed. Proof-reading and double-checking that I haven't left things out is very welcome. I'm not entirely convinced the sections this is divided up into are the best possible choices, but neither am I sure exactly what would be better. Suggestions welcome. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On 03/04/2011 10:22 PM, Robert Haas wrote: On Fri, Mar 4, 2011 at 10:44 PM, Robert Haasrobertmh...@gmail.com wrote: So it seems like the only thing that is an absolute must-do is write some release notes. Here's a rough attempt at filtering the post-alpha3 commit log down to approximately the set of things worth adding to the alpha4 release notes. Support unlogged tables. The contents of an unlogged table are WAL-logged; um.. are _not_ WAL-logged? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions vs. shared procedural language handler functions
Tom Lane t...@sss.pgh.pa.us writes: I can imagine that someplace down the road we might want to allow multiple extensions to own the same SQL object; I know that RPMs can share ownership of files, for comparison. But today is not that day. […] Anyone have a different answer? What could be done is to have a common extension that installs the functions, then both plperl and plperlu would require the common bits. That's only practical when we have automatic dependency resolution at install and remove times (it should not be hard to do, but well). So for 9.1, I think you took the simplest path available. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Sat, Mar 5, 2011 at 9:44 AM, Andy Colson a...@squeakycode.net wrote: Support unlogged tables. The contents of an unlogged table are WAL-logged; um.. are _not_ WAL-logged? Uh, yeah. It looks like I fixed that in the version I committed, but introduced another, similar mistake which I have now also fixed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, 2011-03-05 at 14:44 +0100, Yeb Havinga wrote: On Sat, Mar 5, 2011 at 2:05 PM, Robert Haas robertmh...@gmail.com wrote: On Sat, Mar 5, 2011 at 7:49 AM, Simon Riggs si...@2ndquadrant.com wrote: If the order is arbitrary, why does it matter if it changes? The user has the power to specify a sequence, yet they have not done so. They are told the results are indeterminate, which is accurate. I can add the words and may change as new standbys connect if that helps. I just don't think that's very useful behavior. Suppose I have a master and two standbys. Both are local (or both are remote with equally good connectivity). When one of the standbys goes down, there will be a hiccup (i.e. transactions will block trying to commit) until that guy falls off and the other one takes over. Now, when he comes back up again, I don't want the synchronous standby to change again; that seems like a recipe for another hiccup. I think who the current synchronous standby is should act as a tiebreak. +1 TLDR part: The first one might be noticed by users because it takes tens of seconds before the sync switch. The second hiccup is hardly noticable. However limiting the # switches of sync standby to the absolute minimum is also good if e.g. (if there was a hook for it) cluster middleware is notified of the sync replica change. That might either introduce a race condition or be even completely unreliable if the notify is sent asynchronous, or it might introduce a longer lag if the master waits for confirmation of the sync replica change message. At that point sync replica changes become more expensive than they are currently. I'm not in favour. If the user has a preferred order, they can specify it. If there is no preferred order, how will we maintain that order? What are the rules for maintaining this arbitrary order? No, this is not something we need prior to commit, if ever. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On 03/05/2011 08:54 AM, Robert Haas wrote: On Sat, Mar 5, 2011 at 9:44 AM, Andy Colsona...@squeakycode.net wrote: Support unlogged tables. The contents of an unlogged table are WAL-logged; um.. are _not_ WAL-logged? Uh, yeah. It looks like I fixed that in the version I committed, but introduced another, similar mistake which I have now also fixed. I think you have this section twice: When an autovacuum worker... but it is doubly cool... so... :-) -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, 2011-03-05 at 20:08 +0900, Fujii Masao wrote: On Sat, Mar 5, 2011 at 7:28 AM, Simon Riggs si...@2ndquadrant.com wrote: Yes, that can happen. As people will no doubt observe, this seems to be an argument for wait-forever. What we actually need is a wait that lasts longer than it takes for us to decide to failover, if the standby is actually up and this is some kind of split brain situation. That way the clients are still waiting when failover occurs. WAL is missing, but since we didn't acknowledge the client we are OK to treat that situation as if it were an abort. Oracle Data Guard in the maximum availability mode behaves that way? I'm sure that you are implementing something like the maximum availability mode rather than the maximum protection one. So I'd like to know how the data loss situation I described can be avoided in the maximum availability mode. This is important, so I am taking time to formulate a full reply. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On 03/05/2011 08:54 AM, Robert Haas wrote: On Sat, Mar 5, 2011 at 9:44 AM, Andy Colsona...@squeakycode.net wrote: Support unlogged tables. The contents of an unlogged table are WAL-logged; um.. are _not_ WAL-logged? Uh, yeah. It looks like I fixed that in the version I committed, but introduced another, similar mistake which I have now also fixed. Improved support for parallel make, make -k, and make -q Can we add a line saying -j still doesnt work, dont use it yet or make -j2 works great now. I admit I've never tried to use -j before... is this telling me its ok to use now? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, Mar 5, 2011 at 9:21 PM, Simon Riggs si...@2ndquadrant.com wrote:
I've added code to shmqueue.c to allow this.
New version pushed.
New comments;
It looks odd to report the sync_state of walsender in BACKUP
state as ASYNC.
+SyncRepCleanupAtProcExit(int code, Datum arg)
+{
+ if (WaitingForSyncRep !SHMQueueIsDetached((MyProc-syncrep_links)))
+ {
+ LWLockAcquire(SyncRepLock, LW_EXCLUSIVE);
+ SHMQueueDelete((MyProc-syncrep_links));
+ LWLockRelease(SyncRepLock);
+ }
+
+ if (MyProc != NULL)
+ DisownLatch(MyProc-waitLatch);
Can MyProc really be NULL here? If yes, MyProc != NULL should be
checked before seeing MyProc-syncrep_links.
Even though postmaster dies, the waiting backend keeps waiting until
the timeout expires. Instead, the backends should periodically check
whether postmaster is alive, and then they should exit immediately
if it's not alive, as well as other process does? If the timeout is
disabled, such backends would get stuck infinitely.
Though I commented about the issue related to shutdown, that was
pointless. So change of ProcessInterrupts is not required unless we
find the need again. Sorry for the noise..
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sun, Mar 6, 2011 at 12:07 AM, Simon Riggs si...@2ndquadrant.com wrote: I'm not in favour. If the user has a preferred order, they can specify it. If there is no preferred order, how will we maintain that order? What are the rules for maintaining this arbitrary order? Probably what Robert, Yeb and I think is to leave the current sync standby in sync mode until either its connection is closed or higher priority standby connects. No complicated rule is required. To do that, how about tracking which standby is currently in sync mode? Each walsender checks whether its priority is higher than that of current sync one, and if yes, it takes over. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On lör, 2011-03-05 at 09:33 -0600, Andy Colson wrote: Can we add a line saying -j still doesnt work, dont use it yet or make -j2 works great now. I admit I've never tried to use -j before... is this telling me its ok to use now? Has make -j ever made any sense? Other than for locking up your system? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Mar 5, 2011, at 11:17 AM, Fujii Masao masao.fu...@gmail.com wrote: On Sun, Mar 6, 2011 at 12:07 AM, Simon Riggs si...@2ndquadrant.com wrote: I'm not in favour. If the user has a preferred order, they can specify it. If there is no preferred order, how will we maintain that order? What are the rules for maintaining this arbitrary order? Probably what Robert, Yeb and I think is to leave the current sync standby in sync mode until either its connection is closed or higher priority standby connects. No complicated rule is required. To do that, how about tracking which standby is currently in sync mode? Each walsender checks whether its priority is higher than that of current sync one, and if yes, it takes over. That is precisely what I would expect to happen, and IMHO quite useful. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
El 05/03/2011 11:18, Fujii Masao masao.fu...@gmail.com escribió: On Sun, Mar 6, 2011 at 12:07 AM, Simon Riggs si...@2ndquadrant.com wrote: I'm not in favour. If the user has a preferred order, they can specify it. If there is no preferred order, how will we maintain that order? What are the rules for maintaining this arbitrary order? Probably what Robert, Yeb and I think is to leave the current sync standby in sync mode until either its connection is closed or higher priority standby connects. No complicated rule is required. It's not better to remove the code to manage * in synchronous_standby_names? Once we do that there is no chance of having 2 standbys with the same priority. After all, most of the times the dba will need to change the * for a real list of names anyway. At least in IMHO -- Jaime Casanovawww.2ndQuadrant.com
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
Andres Freund and...@anarazel.de writes: On Saturday 05 March 2011 04:44:20 Robert Haas wrote: * Collation-related regression failure on buildfarm member pika. This is clearly a bug we need to identify, but maybe we can ship the alpha without a fix --- for one thing, getting more than one report of the problem would be helpful. I have just yesterday hit the same bug while testing some application on then HEAD, so its not just PIKA. What platform, and what locale/encoding environment? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sun, 2011-03-06 at 01:17 +0900, Fujii Masao wrote: On Sun, Mar 6, 2011 at 12:07 AM, Simon Riggs si...@2ndquadrant.com wrote: I'm not in favour. If the user has a preferred order, they can specify it. If there is no preferred order, how will we maintain that order? What are the rules for maintaining this arbitrary order? Probably what Robert, Yeb and I think is to leave the current sync standby in sync mode until either its connection is closed or higher priority standby connects. No complicated rule is required. No, it is complex. The code is intentionally stateless, so unless you have a rule you cannot work out which one is sync standby. It is much more important to have robust takeover behaviour. Changing this will require rethinking how that takeover works. And I'm not doing that for something that is documented as indeterminate. If you care about the sequence then set the supplied parameter, which I have gone to some trouble to provide. To do that, how about tracking which standby is currently in sync mode? Each walsender checks whether its priority is higher than that of current sync one, and if yes, it takes over. Regards, -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, 2011-03-05 at 11:42 -0500, Jaime Casanova wrote: El 05/03/2011 11:18, Fujii Masao masao.fu...@gmail.com escribió: On Sun, Mar 6, 2011 at 12:07 AM, Simon Riggs si...@2ndquadrant.com wrote: I'm not in favour. If the user has a preferred order, they can specify it. If there is no preferred order, how will we maintain that order? What are the rules for maintaining this arbitrary order? Probably what Robert, Yeb and I think is to leave the current sync standby in sync mode until either its connection is closed or higher priority standby connects. No complicated rule is required. It's not better to remove the code to manage * in synchronous_standby_names? Once we do that there is no chance of having 2 standbys with the same priority. Yes, there is, because we don't do duplicate name checking. I've changed the default so it is no longer * by default, to avoid complaints. After all, most of the times the dba will need to change the * for a real list of names anyway. At least in IMHO -- Jaime Casanovawww.2ndQuadrant.com -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sun, 2011-03-06 at 00:42 +0900, Fujii Masao wrote:
On Sat, Mar 5, 2011 at 9:21 PM, Simon Riggs si...@2ndquadrant.com wrote:
I've added code to shmqueue.c to allow this.
New version pushed.
New comments;
None of the requested changes are in v21, as yet.
It looks odd to report the sync_state of walsender in BACKUP
state as ASYNC.
Cool.
+SyncRepCleanupAtProcExit(int code, Datum arg)
+{
+ if (WaitingForSyncRep !SHMQueueIsDetached((MyProc-syncrep_links)))
+ {
+ LWLockAcquire(SyncRepLock, LW_EXCLUSIVE);
+ SHMQueueDelete((MyProc-syncrep_links));
+ LWLockRelease(SyncRepLock);
+ }
+
+ if (MyProc != NULL)
+ DisownLatch(MyProc-waitLatch);
Can MyProc really be NULL here? If yes, MyProc != NULL should be
checked before seeing MyProc-syncrep_links.
OK
Even though postmaster dies, the waiting backend keeps waiting until
the timeout expires. Instead, the backends should periodically check
whether postmaster is alive, and then they should exit immediately
if it's not alive, as well as other process does? If the timeout is
disabled, such backends would get stuck infinitely.
Will wake them every 60 seconds
Though I commented about the issue related to shutdown, that was
pointless. So change of ProcessInterrupts is not required unless we
find the need again. Sorry for the noise..
Yep, all gone now.
--
Simon Riggs http://www.2ndQuadrant.com/books/
PostgreSQL Development, 24x7 Support, Training and Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sun, Mar 6, 2011 at 12:42 AM, Fujii Masao masao.fu...@gmail.com wrote:
New comments;
Another one;
+ longtimeout = SyncRepGetWaitTimeout();
snip
+ else if (timeout 0
+ TimestampDifferenceExceeds(wait_start, now,
timeout))
+ {
The third argument of TimestampDifferenceExceeds is
expressed in milliseconds. But you wrongly give that the
microseconds.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Sat, Mar 05, 2011 at 11:46:13AM -0500, Tom Lane wrote:
Andres Freund and...@anarazel.de writes:
On Saturday 05 March 2011 04:44:20 Robert Haas wrote:
* Collation-related regression failure on buildfarm member pika. This
is clearly a bug we need to identify, but maybe we can ship the alpha
without a fix --- for one thing, getting more than one report of the
problem would be helpful.
I have just yesterday hit the same bug while testing some application on
then
HEAD, so its not just PIKA.
What platform, and what locale/encoding environment?
Hi,
i got the same issue using git head and managed to isolate a small testcase.
git head on debian/sid x86_64, gcc45
Reproducible with:
bin/initdb -D data/ --encoding=UTF-8 --locale=en_US
bin/postgres -B16384 -D data/ -p -h localhost
create database test with encoding='UTF8'
create table ad_window (ad_client_id varchar(32) not null , name varchar(60)
not null, ad_module_id varchar(32) not null);
create UNIQUE INDEX ad_window_name on ad_window(ad_client_id, name,
ad_module_id);
insert into ad_window values ('0','Tables and Columns','0');
insert into ad_window values ('0','Reference','0');
The second insert fails consistently with:
WARNING: text_cmp 1-1-0'Tables and Columns0-0Reference0
WARNING: called from varlena: 0'Tables and Columns0 0Reference0 for
collid: (null)
ERROR: locale operation to be invoked, but no collation was derived
The added elogs show the compared values on the second insert which first
pointed me to the unique index as the probable trigger.
Backtrace of the failing text_cmp (not sure if useful):
#1 0x0077054b in bttextcmp (fcinfo=0x7fffbc1f9980) at varlena.c:1595
#2 0x007cade4 in FunctionCall2 (flinfo=0x254e7b8,
arg1=140410956296568, arg2=39118720) at fmgr.c:1387
#3 0x00492330 in _bt_compare ()
#4 0x00491e6a in _bt_binsrch ()
#5 0x0048a381 in _bt_doinsert ()
#6 0x0049077d in btinsert ()
#7 0x007cb230 in FunctionCall6 (flinfo=0x247bc60,
arg1=14041761456, arg2=140736349578208, arg3=140736349578176,
arg4=39118460, arg5=14041755632, arg6=1) at fmgr.c:1500
#8 0x00488ef0 in index_insert (indexRelation=0x7fb402706630,
values=0x7fffbc1fa3e0, isnull=0x7fffbc1fa3c0 ,
heap_t_ctid=0x254e67c, heapRelation=0x7fb402704f70,
checkUnique=UNIQUE_CHECK_YES) at indexam.c:205
#9 0x005cce68 in ExecInsertIndexTuples ()
#10 0x005ddafc in ExecInsert ()
#11 0x005de9e1 in ExecModifyTable ()
#12 0x005c032a in ExecProcNode ()
#13 0x005be204 in ExecutePlan ()
#14 0x005bcabe in standard_ExecutorRun ()
#15 0x005bc9b2 in ExecutorRun ()
#16 0x006d2d9b in ProcessQuery (plan=0x2527350,
sourceText=0x245ebd0 insert into ad_window values ('0','Reference','0');,
params=0x0, dest=0x2527430,
completionTag=0x7fffbc1faa30 ) at pquery.c:187
Regards,
Stefan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions vs. shared procedural language handler functions
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: The only easy fix I can see at the moment is to arbitrarily create two pg_proc entries --- they can both point at the same C function, but there need to be two of 'em. So for 9.1, I think you took the simplest path available. It's never that easy :-(. I've been trying to figure out why frogmouth (Windows/cygwin buildfarm member) suddenly started failing: CREATE EXTENSION plpython2u; -- really stupid function just to get the module loaded CREATE FUNCTION stupid() RETURNS text AS 'return zarkon' LANGUAGE plpythonu; ! ERROR: could not load library c:/mingw/msys/1.0/home/pgrunner/bf/root/HEAD/inst/lib/postgresql/plpython.dll: Invalid access to memory location. + select stupid(); + ERROR: function stupid() does not exist and it just hit me what must be going on. plpython's makefile tries to symlink plpython.dll to plpython2.dll, but that trick evidently doesn't work on Windows: the system doesn't understand they're the same library and so trying to load both of them at once fails as above. The next question is how come this regression test ever worked on that platform. The reason is that up till my changes for $SUBJECT, when you issued CREATE LANGUAGE plpython2u in a database that already had plpythonu installed, CREATE LANGUAGE found C functions of the expected names already present and so it didn't create new ones. This meant that only plpython.dll ever got loaded, not plpython2.dll, despite what the pg_pltemplate entry alleges about the shlib name for the latter. IMO this is all pretty Rube Goldbergian and it's amazing it didn't fail on more platforms. What I propose to do about it is get rid of the plpython.dll symlink and just have the pg_pltemplate entry for plpythonu reference the plpython2 shlib. People who want to switch the referent for plpythonu to be Python3 will have an extra thing to do, but I haven't heard of very many people doing that anyway. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, Mar 5, 2011 at 5:53 PM, Simon Riggs si...@2ndquadrant.com wrote: On a positive note this is one less parameter and will improve performance as well. All above changes made. Ready to commit, barring concrete objections to important behaviour. I will do one final check tomorrow evening then commit. Will retest with new version this evening. Also curious to performance improvement, since v17 seems to be topscorer in that department. regardd, Yeb Havinga
Re: [HACKERS] Extensions vs. shared procedural language handler functions
On 03/05/2011 12:17 PM, Tom Lane wrote: Dimitri Fontainedimi...@2ndquadrant.fr writes: Tom Lanet...@sss.pgh.pa.us writes: The only easy fix I can see at the moment is to arbitrarily create two pg_proc entries --- they can both point at the same C function, but there need to be two of 'em. So for 9.1, I think you took the simplest path available. It's never that easy :-(. I've been trying to figure out why frogmouth (Windows/cygwin buildfarm member) suddenly started failing: It's mingw, not cygwin (brolga is the cygwin animal, and it doesn't build with python.) But good catch on the problem. FYI, I'm working on the MSVC issues. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Saturday 05 March 2011 17:46:13 Tom Lane wrote:
Andres Freund and...@anarazel.de writes:
On Saturday 05 March 2011 04:44:20 Robert Haas wrote:
* Collation-related regression failure on buildfarm member pika. This
is clearly a bug we need to identify, but maybe we can ship the alpha
without a fix --- for one thing, getting more than one report of the
problem would be helpful.
I have just yesterday hit the same bug while testing some application on
then HEAD, so its not just PIKA.
What platform, and what locale/encoding environment?
One debian and one ubuntu x64 box.
I have a WIP patch fixing one of the two issues.
Several places in selfuncs.c didn't setup collations. That lead for example to
errors during patternsel.
The eqproc don't actually currently need the collation information, but I
think it is more consistent.
(attached)
I am currently looking at the other one. Its quite strange:
I can trigger it reliably on my machines.
with gdb attached:
b indexam.c:875 (thats index_getprocinfo)
...
test=# CREATE TABLE foo(a int, b text, c int);
CREATE TABLE
Time: 65.535 ms
test=# INSERT INTO foo VALUES (1, '1', 2);
INSERT 0 1
Time: 66.777 ms
test=# INSERT INTO foo VALUES (1, '2', 2);
INSERT 0 1
Time: 40.687 ms
#play with gdb
test=# CREATE INDEX foo_abc ON foo (a, b,c);
ERROR: locale operation to be invoked, but no collation was derived
...
(gdb) print irel-rd_indcollation[0]
$8 = 0
(gdb) print irel-rd_indcollation[1]
$9 = 100
(gdb) print irel-rd_indcollation[2]
$10 = 0
(gdb)
$11 = 0
(gdb) print irel-rd_index-indcollation.values[0]
$12 = 0
(gdb) print irel-rd_index-indcollation.values[1]
$13 = 0
(gdb) print irel-rd_index-indcollation.values[2]
$14 = 100
(gdb) print irel-rd_index-indcollation.values[3]
$15 = 0
(gdb) print irel-rd_index-indcollation
$16 = {vl_len_ = 8, ndim = 144, dataoffset = 1, elemtype = 0, dim1 = 26,
lbound1 = 3, values = {0}}
The piece of code using indcollation assumes that dataoffset = 0 which seems to
be an valid assumption...
...
fmgr_info_collation(irel-rd_index-indcollation.values[attnum-1],
locinfo);
...
Thus no valid collation is attached for the text column but a useless one for
the last int column.
Andres
From 2c33f1bb9c363cbb34767ecc37c463e1162372ee Mon Sep 17 00:00:00 2001
From: Andres Freund and...@anarazel.de
Date: Sat, 5 Mar 2011 18:30:04 +0100
Subject: [PATCH] Attach collation information to several FmgrInfo structs in selfuncs.c
---
src/backend/utils/adt/selfuncs.c |7 ++-
1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index f10110b..783ad7c 100644
*** a/src/backend/utils/adt/selfuncs.c
--- b/src/backend/utils/adt/selfuncs.c
*** var_eq_const(VariableStatData *vardata,
*** 285,290
--- 285,291
FmgrInfo eqproc;
fmgr_info(get_opcode(operator), eqproc);
+ fmgr_info_collation(DEFAULT_COLLATION_OID, eqproc);
for (i = 0; i nvalues; i++)
{
*** eqjoinsel_inner(Oid operator,
*** 2116,2121
--- 2117,2123
nmatches;
fmgr_info(get_opcode(operator), eqproc);
+ fmgr_info_collation(DEFAULT_COLLATION_OID, eqproc);
hasmatch1 = (bool *) palloc0(nvalues1 * sizeof(bool));
hasmatch2 = (bool *) palloc0(nvalues2 * sizeof(bool));
*** eqjoinsel_semi(Oid operator,
*** 2338,2343
--- 2340,2346
nmatches;
fmgr_info(get_opcode(operator), eqproc);
+ fmgr_info_collation(DEFAULT_COLLATION_OID, eqproc);
hasmatch1 = (bool *) palloc0(nvalues1 * sizeof(bool));
hasmatch2 = (bool *) palloc0(nvalues2 * sizeof(bool));
*** prefix_selectivity(PlannerInfo *root, Va
*** 5108,5115
BTGreaterEqualStrategyNumber);
if (cmpopr == InvalidOid)
elog(ERROR, no = operator for opfamily %u, opfamily);
- fmgr_info(get_opcode(cmpopr), opproc);
prefixsel = ineq_histogram_selectivity(root, vardata, opproc, true,
prefixcon-constvalue,
prefixcon-consttype);
--- 5111,5119
BTGreaterEqualStrategyNumber);
if (cmpopr == InvalidOid)
elog(ERROR, no = operator for opfamily %u, opfamily);
+ fmgr_info(get_opcode(cmpopr), opproc);
+ fmgr_info_collation(DEFAULT_COLLATION_OID, opproc);
prefixsel = ineq_histogram_selectivity(root, vardata, opproc, true,
prefixcon-constvalue,
prefixcon-consttype);
*** prefix_selectivity(PlannerInfo *root, Va
*** 5130,5135
--- 5134,5140
if (cmpopr == InvalidOid)
elog(ERROR, no operator for opfamily %u, opfamily);
fmgr_info(get_opcode(cmpopr), opproc);
+ fmgr_info_collation(DEFAULT_COLLATION_OID, opproc);
greaterstrcon = make_greater_string(prefixcon, opproc);
if (greaterstrcon)
--
1.7.1
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your
Re: [HACKERS] Extensions vs. shared procedural language handler functions
Andrew Dunstan and...@dunslane.net writes: FYI, I'm working on the MSVC issues. Ah, great. I was just about to start hacking something together, but it'd be better for somebody to do it who can test it before committing... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Saturday 05 March 2011 18:37:30 Andres Freund wrote: On Saturday 05 March 2011 17:46:13 Tom Lane wrote: Andres Freund and...@anarazel.de writes: I am currently looking at the other one. Its quite strange: The backtrace during the operations described earlier: 0 index_getprocinfo (irel=0x7f5426cbc820, attnum=1, procnum=1) at indexam.c:875 #1 0x0048ab14 in _bt_mkscankey_nodata (rel=0x7f5426cbc820) at nbtutils.c:123 #2 0x007eb628 in tuplesort_begin_index_btree (indexRel=0x7f5426cbc820, enforceUnique=0 '\000', workMem=1048576, randomAccess=0 '\000') at tuplesort.c:753 #3 0x0048c94c in _bt_spoolinit (index=0x7f5426cbc820, isunique=0 '\000', isdead=0 '\000') at nbtsort.c:165 #4 0x00487074 in btbuild (fcinfo=0x7fff68ab2220) at nbtree.c:116 #5 0x007ca99a in OidFunctionCall3 (functionId=338, arg1=13404896344, arg2=13404869664, arg3=30080592) at fmgr.c:1711 #6 0x004d598d in index_build (heapRelation=0x7f5426cc3058, indexRelation=0x7f5426cbc820, indexInfo=0x1cafe50, isprimary=0 '\000') at index.c:1729 #7 0x004d4c5c in index_create (heapRelation=0x7f5426cc3058, indexRelationName=0x1d41c90 foo_abc, indexRelationId=626424, indexInfo=0x1cafe50, indexColNames=0x1cb0318, accessMethodObjectId=403, tableSpaceId=0, collationObjectId=0x1cb03b8, classObjectId=0x1cb03d8, coloptions=0x1cb03f8, reloptions=0, isprimary=0 '\000', isconstraint=0 '\000', deferrable=0 '\000', initdeferred=0 '\000', allow_system_table_mods=0 '\000', skip_build=0 '\000', concurrent=0 '\000') at index.c:1063 #8 0x0057397f in DefineIndex (heapRelation=0x1d41ca8, indexRelationName=0x1d41c90 foo_abc, indexRelationId=0, accessMethodName=0x1d41d10 btree, tableSpaceName=0x0, attributeList=0x1d41d28, predicate=0x0, options=0x0, exclusionOpNames=0x0, unique=0 '\000', primary=0 '\000', isconstraint=0 '\000', deferrable=0 '\000', initdeferred=0 '\000', is_alter_table=0 '\000', check_rights=1 '\001', skip_build=0 '\000', quiet=0 '\000', concurrent=0 '\000') at indexcmds.c:395 #9 0x006d3d74 in standard_ProcessUtility (parsetree=0x1d30ff0, queryString=0x1d303c0 CREATE INDEX foo_abc ON foo (a, b,c);, params=0x0, isTopLevel=1 '\001', dest=0x1d31390, completionTag=0x7fff68ab2dc0 ) at utility.c:954 #10 0x006d2db3 in ProcessUtility (parsetree=0x1d30ff0, queryString=0x1d303c0 CREATE INDEX foo_abc ON foo (a, b,c);, params=0x0, isTopLevel=1 '\001', dest=0x1d31390, completionTag=0x7fff68ab2dc0 ) at utility.c:334 #11 0x006d1e82 in PortalRunUtility (portal=0x1cade40, utilityStmt=0x1d30ff0, isTopLevel=1 '\001', dest=0x1d31390, completionTag=0x7fff68ab2dc0 ) at pquery.c:1184 #12 0x006d2029 in PortalRunMulti (portal=0x1cade40, isTopLevel=1 '\001', dest=0x1d31390, altdest=0x1d31390, completionTag=0x7fff68ab2dc0 ) at pquery.c:1315 #13 0x006d15ff in PortalRun (portal=0x1cade40, count=9223372036854775807, isTopLevel=1 '\001', dest=0x1d31390, altdest=0x1d31390, completionTag=0x7fff68ab2dc0 ) at pquery.c:813 #14 0x006cb935 in exec_simple_query (query_string=0x1d303c0 CREATE INDEX foo_abc ON foo (a, b,c);) at postgres.c:1018 #15 0x006cfa47 in PostgresMain (argc=2, argv=0x1c79da8, username=0x1c79c40 andres) at postgres.c:3902 #16 0x0068444e in BackendRun (port=0x1cb1e20) at postmaster.c:3590 #17 0x00683b1a in BackendStartup (port=0x1cb1e20) at postmaster.c:3275 #18 0x00680e9a in ServerLoop () at postmaster.c:1449 #19 0x00680677 in PostmasterMain (argc=3, argv=0x1c76ef0) at postmaster.c:1110 #20 0x005fb12d in main (argc=3, argv=0x1c76ef0) at main.c:199 (gdb) Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
Andres Freund and...@anarazel.de writes: I have a WIP patch fixing one of the two issues. Several places in selfuncs.c didn't setup collations. That lead for example to errors during patternsel. Hmm. I have to say that this seems like quite the wrong way to go. If everyplace in the system that could be calling a collation-sensitive function has to be modified like this, we'll be fighting bugs of omission till h*ll freezes over. Why aren't we just setting finfo.fn_collation to DEFAULT_COLLATION_OID by default, or maybe better letting places that inspect it take zero as meaning default collation? Call sites should only need to call fmgr_info_collation() if they have an explicit non-default collation to pass in. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Saturday 05 March 2011 18:43:31 Tom Lane wrote: Andres Freund and...@anarazel.de writes: I have a WIP patch fixing one of the two issues. Several places in selfuncs.c didn't setup collations. That lead for example to errors during patternsel. Hmm. I have to say that this seems like quite the wrong way to go. If everyplace in the system that could be calling a collation-sensitive function has to be modified like this, we'll be fighting bugs of omission till h*ll freezes over. Why aren't we just setting finfo.fn_collation to DEFAULT_COLLATION_OID by default, or maybe better letting places that inspect it take zero as meaning default collation? Call sites should only need to call fmgr_info_collation() if they have an explicit non-default collation to pass in. I wondered the same. On the other hand it makes errors like the one during index build way much harder to catch... Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, Mar 5, 2011 at 11:56 AM, Simon Riggs si...@2ndquadrant.com wrote: Even though postmaster dies, the waiting backend keeps waiting until the timeout expires. Instead, the backends should periodically check whether postmaster is alive, and then they should exit immediately if it's not alive, as well as other process does? If the timeout is disabled, such backends would get stuck infinitely. Will wake them every 60 seconds I don't really see why sync rep should be responsible for solving this problem, which is an issue in many other situations as well, only for itself. In fact I think I'd prefer that it didn't, and that we wait for a more general solution that will actually fix this problem for real. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Sat, Mar 5, 2011 at 10:17 AM, Andy Colson a...@squeakycode.net wrote: On 03/05/2011 08:54 AM, Robert Haas wrote: On Sat, Mar 5, 2011 at 9:44 AM, Andy Colsona...@squeakycode.net wrote: Support unlogged tables. The contents of an unlogged table are WAL-logged; um.. are _not_ WAL-logged? Uh, yeah. It looks like I fixed that in the version I committed, but introduced another, similar mistake which I have now also fixed. I think you have this section twice: When an autovacuum worker... but it is doubly cool... so... :-) Thanks, fixed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Quick Extensions Question
Robert Haas robertmh...@gmail.com writes: On Fri, Mar 4, 2011 at 11:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: Hmm. Personally I do use createdb/dropdb but never createlang/droplang; but I'm well aware that my usage may not be typical. I'm a bit hesitant to just go and drop these without any warning. I could see deprecating them for a release or two and then dropping them ... but that doesn't solve the problem of what to do with them in 9.1. Well, if we're not going remove them altogether, then my vote would be to leave them unchanged (i.e. they'll still emit CREATE LANGUAGE) and stick a big deprecation warning on them. It isn't necessary to have every SQL command available as a standalone executable. On reflection I think it makes no sense at all to leave those tools issuing CREATE/DROP LANGUAGE. We want to move people over to managing languages via extensions, and leaving those tools unchanged will not serve that goal. However, I don't mind labeling them as deprecated and not troubling to point out that they could be used for installing non-PL extensions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Quick Extensions Question
On Mar 5, 2011, at 10:03 AM, Tom Lane wrote: On reflection I think it makes no sense at all to leave those tools issuing CREATE/DROP LANGUAGE. We want to move people over to managing languages via extensions, and leaving those tools unchanged will not serve that goal. However, I don't mind labeling them as deprecated and not troubling to point out that they could be used for installing non-PL extensions. +1 I think this is the sane thing to do in the short run. It remains to be seen whether a createext type tool is even necessary. David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Replication server timeout patch
On Mon, Feb 28, 2011 at 8:08 AM, Fujii Masao masao.fu...@gmail.com wrote: On Sun, Feb 27, 2011 at 11:52 AM, Fujii Masao masao.fu...@gmail.com wrote: There are two things that I think are pretty clear. If the receiver has wal_receiver_status_interval=0, then we should ignore replication_timeout for that connection. The patch still doesn't check that wal_receiver_status_interval is set up properly. I'll implement that later. Done. I attached the updated patch. Why does internal_flush_if_writable compute bufptr differently from internal_flush? And shouldn't it be static? It seems to me that this ought to be refactored so that you don't duplicate so much code. Maybe static int internal_flush(bool nonblocking). I don't think that the while (bufptr bufend) loop needs to contain the code to set and clear the nonblocking state. You could do the whole loop with nonblocking mode turned on and then reenable it just once at the end. Besides possibly being clearer, that would be more efficient and leave less room for unexpected failures. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Sat, Mar 5, 2011 at 10:33 AM, Andy Colson a...@squeakycode.net wrote: On 03/05/2011 08:54 AM, Robert Haas wrote: On Sat, Mar 5, 2011 at 9:44 AM, Andy Colsona...@squeakycode.net wrote: Support unlogged tables. The contents of an unlogged table are WAL-logged; um.. are _not_ WAL-logged? Uh, yeah. It looks like I fixed that in the version I committed, but introduced another, similar mistake which I have now also fixed. Improved support for parallel make, make -k, and make -q Can we add a line saying -j still doesnt work, dont use it yet or make -j2 works great now. I admit I've never tried to use -j before... is this telling me its ok to use now? I've been using -j3 for a long time. These changes just make it able to parallelize a bit better. As Peter says, -j without arguments is mostly just a footgun. I can't imagine why that option hasn't been removed long since. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions vs. shared procedural language handler functions
Tom Lane t...@sss.pgh.pa.us writes: The next question is how come this regression test ever worked on that platform. The reason is that up till my changes for $SUBJECT, when you issued CREATE LANGUAGE plpython2u in a database that already had plpythonu installed, CREATE LANGUAGE found C functions of the expected names already present and so it didn't create new ones. This meant that only plpython.dll ever got loaded, not plpython2.dll, despite what the pg_pltemplate entry alleges about the shlib name for the latter. Well if we would have a plpython_wrapper extension that loads the common functions and the shared object, then a plpythonu and a plpython2u that depends on the plpython_wrapper extension, would it solve the problem? Instead of an ERROR when an extension you require isn't installed, the code would have to recurse into installing it. That means maintaining a stack of current extension being loaded, I guess. Also, for external PLs it would allow people to distribute a couple of extensions each time. The wrapper one that you have to install as a superuser, possibly in template1, and the PL one that you can install as the database owner. It could be a big enough deal in colo environments. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] German Ladies start translation project
On 05.03.2011 11:57, Susanne Ebrecht wrote: For me this means - it is finally time to translate / update old Peter docs. Peter and me think - just translate all again is faster then taking and checking his old texts. I'm curious, what percentage of the text has changed since 7.3? I did this: ~/git-sandbox/postgresql/doc/src/sgml (master)$ cat *.sgml ref/*.sgml | wc -l 249740 ~/git-sandbox/postgresql/doc/src/sgml (master)$ git diff --stat origin/REL7_3_STABLE -- *.sgml ref/*.sgml | tail -n 1 317 files changed, 206915 insertions(+), 68733 deletions(-) So the number of lines has roughly doubled since, and about a quarter of the 7.3 lines have changed. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On 2011-03-05 18:25, Yeb Havinga wrote: On Sat, Mar 5, 2011 at 5:53 PM, Simon Riggs si...@2ndquadrant.com mailto:si...@2ndquadrant.com wrote: On a positive note this is one less parameter and will improve performance as well. All above changes made. Ready to commit, barring concrete objections to important behaviour. I will do one final check tomorrow evening then commit. Will retest with new version this evening. Also curious to performance improvement, since v17 seems to be topscorer in that department. Summary of preliminary testing: 1) it is confusing to show messages/ contents of stat_replication that hints at syncrep, when synchronous_replication is on. 2) should guc settings for synchronous_replication be changed so it can only be set in the config file, and hence only change with reload_conf()? 3) speed is comparable to v17 :-) regards, Yeb Havinga So the biggest change is perhaps that you cannot start 'working' immediately after a initdb with synchronous_replication=on, without a connected standby; I needed to create a role for the repuser to make a backup, but the master halted. The initial bootstrapping has to be done with synchronous_replication = off. So I did, made backup, started standby's while still in not synchronous mode. What followed was confusing: LOG: 0: standby standby2 is now the synchronous standby with priority 2 postgres=# show synchronous_replication ; show synchronous_standby_names; select application_name,state,sync_state from pg_stat_replication; synchronous_replication - off (1 row) synchronous_standby_names standby1,standby2,standby3 (1 row) application_name | state | sync_state --+---+ standby2 | STREAMING | SYNC asyncone | STREAMING | ASYNC (2 rows) Is it really sync? pgbench test got 1464 tps.. seems a bit high. psql postgres, set synchronous_replication = on; - no errors, and show after disconnect showed this parameter was still on. My guess: we have syncrep! A restart or reload config was not necessary. pgbench test got 1460 tps. pg_reload_conf(); with syncrep = on in postgresql.conf pgbench test got 819 tps So now this is synchronous. Disabled the asynchronous standby pgbench test got 920 tps. I also got a first first 1000 tps score :-) (yeah you have to believe me there really was a sync standby server) $ pgbench -c 10 -M prepared -T 30 test starting vacuum...end. transaction type: TPC-B (sort of) scaling factor: 50 query mode: prepared number of clients: 10 number of threads: 1 duration: 30 s number of transactions actually processed: 30863 tps = 1027.493807 (including connections establishing) tps = 1028.183618 (excluding connections establishing)
Re: [HACKERS] Sync Rep v19
On 2011-03-05 21:11, Yeb Havinga wrote: Summary of preliminary testing: 1) it is confusing to show messages/ contents of stat_replication that hints at syncrep, when synchronous_replication is on. s/on/off/ Also forgot to mention these tests are againt the latest v21 syncrep patch. Y
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On lör, 2011-03-05 at 12:43 -0500, Tom Lane wrote: Why aren't we just setting finfo.fn_collation to DEFAULT_COLLATION_OID by default, or maybe better letting places that inspect it take zero as meaning default collation? Because then you'd just get silently wrong results instead of an error. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sat, Mar 5, 2011 at 3:11 PM, Yeb Havinga yebhavi...@gmail.com wrote: Summary of preliminary testing: 1) it is confusing to show messages/ contents of stat_replication that hints at syncrep, when synchronous_replication is on. [for the record, Yeb explain he means OFF not on...] the thing is that once you put a server name in synchronous_standby_names that standby is declared to be synchronous and because synchronous_replication can change at any time for any given backend we need to know priority of any declared sync standby. if you want pure async standbys remove their names from synchronous_standby_names 2) should guc settings for synchronous_replication be changed so it can only be set in the config file, and hence only change with reload_conf()? no. the thing is that we can determine for which data we want to pay the price of synchrony and for which data don't 3) speed is comparable to v17 :-) yeah... it's a lot better than before, good work Simon :) -- Jaime Casanova www.2ndQuadrant.com Professional PostgreSQL: Soporte y capacitación de PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Mar 4, 2011, at 7:44 PM, Robert Haas wrote: So it seems like the only thing that is an absolute must-do is write some release notes. What about this? Yeah, the real problem in my mind is not so much citext as whether the current representation of a type's collation property is sane in the first place. Once we've thrashed that out, I'll be happy with a simple UPDATE pg_type kluge in the citext upgrade script. Doing anything cleaner-looking than that is a future project (and might never happen, seeing that we've never felt a need for ALTER TYPE commands for other properties of a basic type). So really I guess the release-blocker issue is http://archives.postgresql.org/message-id/27152(dot)1299015062(at)sss(dot)pgh(dot)pa(dot)us Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
David E. Wheeler da...@kineticode.com writes: On Mar 4, 2011, at 7:44 PM, Robert Haas wrote: So it seems like the only thing that is an absolute must-do is write some release notes. What about this? http://archives.postgresql.org/message-id/27152(dot)1299015062(at)sss(dot)pgh(dot)pa(dot)us Well, nobody else seems to be excited about it. And it's not like there isn't a clear upgrade path if we decide to change CREATE TYPE later --- it'll just be a bit clunkier. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
Ah. Finally after trying to stare down the code for some more time the issue
is pretty simple.
index_getprocinfo did this:
/* Initialize the lookup info if first time through */
if (locinfo-fn_oid == InvalidOid)
{
...
fmgr_info_cxt(procId, locinfo, irel-rd_indexcxt);
fmgr_info_collation(irel-rd_index-indcollation.values[attnum-1],
locinfo);
}
which is not a good idea because irel-rd_index-indcollation is field after
the variable lenght indkey field.
Indkey is of int2vector type which is important because it means that there is
enough space for a second key without making direct access to indcollation
wrong (which is 4byte alligned). That explains why the issue could only be
triggered by a composite index covering at least 3 columns...
RelationInitIndexAccessInfo already fills the more convenient
Relation.rd_indcollation which makes the fix trivial.
It was a good bug though, because now I understand the relcache to some degree
which I formerly definitely did not ;-)
On Saturday 05 March 2011 18:37:30 Andres Freund wrote:
test=# CREATE TABLE foo(a int, b text, c int);
CREATE TABLE
Time: 65.535 ms
test=# INSERT INTO foo VALUES (1, '1', 2);
INSERT 0 1
Time: 66.777 ms
test=# INSERT INTO foo VALUES (1, '2', 2);
INSERT 0 1
Time: 40.687 ms
test=# CREATE INDEX foo_abc ON foo (a, b,c);
ERROR: locale operation to be invoked, but no collation was derived
Fixed after applying the patch.
Andres
From 91cf261c2af263e0965f5b302129730a7530ff38 Mon Sep 17 00:00:00 2001
From: Andres Freund and...@anarazel.de
Date: Sat, 5 Mar 2011 23:45:22 +0100
Subject: [PATCH] Fix one more 'locale operation to be invoked, but no collation was derived' error
The error was caused by directly accessing Relation.rd_index.indcollation.
The latter is positioned after a variable length field and thus cannot be
accessed directly. Instead use Relation.indcollation which is provided for
that purpose.
---
src/backend/access/index/indexam.c |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 6e6af18..f489400 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -872,7 +872,7 @@ index_getprocinfo(Relation irel,
procnum, attnum, RelationGetRelationName(irel));
fmgr_info_cxt(procId, locinfo, irel-rd_indexcxt);
- fmgr_info_collation(irel-rd_index-indcollation.values[attnum-1],
+ fmgr_info_collation(irel-rd_indcollation[attnum-1],
locinfo);
}
--
1.7.1
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Parallel make problem with git master
I am seeing the following compile problem with gmake -j2: /bin/sh ../../../config/install-sh -c -d '/usr/local/pgsql/lib' /bin/sh ../../../../config/install-sh -c -m 644 ./plpgsql.control '/usr/local/pgsql/share/extension' /bin/sh ../../../config/install-sh -c -m 644 ./plperl.control '/usr/local/pgsql/share/extension' /bin/sh ../../../../config/install-sh -c -m 644 ./plpgsql--1.0.sql '/usr/local/pgsql/share/extension' /bin/sh ../../../config/install-sh -c -m 644 ./plperl--1.0.sql '/usr/local/pgsql/share/extension' /bin/sh ../../../../config/install-sh -c -m 644 ./plpgsql--unpackaged--1.0.sql '/usr/local/pgsql/share/extension' /bin/sh ../../../config/install-sh -c -m 644 ./plperl--unpackaged--1.0.sql '/usr/local/pgsql/share/extension' /bin/sh ../../../../config/install-sh -c -d '/usr/local/pgsql/share/extension' /bin/sh ../../../config/install-sh -c -m 644 ./plperlu.control '/usr/local/pgsql/share/extension' mkdir: /usr/local/pgsql/share/extension: File exists /bin/sh ../../../config/install-sh -c -m 644 ./plperlu--1.0.sql '/usr/local/pgsql/share/extension' mkdir: /usr/local/pgsql/share/extension: File exists gmake[3]: *** [installdirs] Error 1 gmake[3]: Leaving directory `/usr/var/local/src/gen/pgsql/postgresql/src/pl/plpgsql/src' gmake[2]: *** [install] Error 2 gmake[2]: Leaving directory `/usr/var/local/src/gen/pgsql/postgresql/src/pl/plpgsql' gmake[1]: *** [install-plpgsql-recurse] Error 2 gmake[1]: *** Waiting for unfinished jobs /bin/sh ../../../config/install-sh -c -m 644 ./plperlu--unpackaged--1.0.sql '/usr/local/pgsql/share/extension' /bin/sh ../../../config/install-sh -c -d '/usr/local/pgsql/share/extension' /bin/sh ../../../config/install-sh -c -m 755 plperl.so '/usr/local/pgsql/lib/plperl.so' mkdir: /usr/local/pgsql/share/extension: File exists mkdir: /usr/local/pgsql/share/extension: File exists gmake[2]: *** [installdirs] Error 1 gmake[2]: Leaving directory `/usr/var/local/src/gen/pgsql/postgresql/src/pl/plperl' gmake[1]: *** [install-plperl-recurse] Error 2 gmake[1]: Leaving directory `/usr/var/local/src/gen/pgsql/postgresql/src/pl' gmake: *** [install-pl-recurse] Error 2 This only happens with parallel gmake and I think is caused by the assumption that mkdir extension will happen before any files are installed, which doesn't happen with parallel gmake. I have fixed the bug with the attached, applied patch which moves 'installdirs' to a dependency of the extension directory file install, rather than a more top-level target so the parallel gmake always creates the directory first. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + diff --git a/src/pl/plperl/GNUmakefile b/src/pl/plperl/GNUmakefile index 71e2cef..155b60f 100644 --- a/src/pl/plperl/GNUmakefile +++ b/src/pl/plperl/GNUmakefile @@ -74,14 +74,14 @@ Util.c: Util.xs $(PERL) $(perl_privlibexp)/ExtUtils/xsubpp -typemap $(perl_privlibexp)/ExtUtils/typemap $ $@ -install: all installdirs install-lib install-data +install: all install-lib install-data installdirs: installdirs-lib $(MKDIR_P) '$(DESTDIR)$(datadir)/extension' uninstall: uninstall-lib uninstall-data -install-data: +install-data: installdirs @for file in $(addprefix $(srcdir)/, $(DATA)); do \ echo $(INSTALL_DATA) $$file '$(DESTDIR)$(datadir)/extension'; \ $(INSTALL_DATA) $$file '$(DESTDIR)$(datadir)/extension'; \ diff --git a/src/pl/plpgsql/src/Makefile b/src/pl/plpgsql/src/Makefile index d748ef6..52fbc1c 100644 --- a/src/pl/plpgsql/src/Makefile +++ b/src/pl/plpgsql/src/Makefile @@ -27,14 +27,14 @@ all: all-lib include $(top_srcdir)/src/Makefile.shlib -install: all installdirs install-lib install-data +install: all install-lib install-data installdirs: installdirs-lib $(MKDIR_P) '$(DESTDIR)$(datadir)/extension' uninstall: uninstall-lib uninstall-data -install-data: +install-data: installdirs @for file in $(addprefix $(srcdir)/, $(DATA)); do \ echo $(INSTALL_DATA) $$file '$(DESTDIR)$(datadir)/extension'; \ $(INSTALL_DATA) $$file '$(DESTDIR)$(datadir)/extension'; \ diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile index baf22f3..86d8741 100644 --- a/src/pl/plpython/Makefile +++ b/src/pl/plpython/Makefile @@ -106,14 +106,14 @@ all: all-lib distprep: spiexceptions.h -install: all installdirs install-lib install-data +install: all install-lib install-data installdirs: installdirs-lib $(MKDIR_P) '$(DESTDIR)$(datadir)/extension' uninstall: uninstall-lib uninstall-data -install-data: +install-data: installdirs @for file in $(addprefix $(srcdir)/, $(DATA)); do \ echo $(INSTALL_DATA) $$file '$(DESTDIR)$(datadir)/extension'; \
[HACKERS] fix for pg_upgrade
I got a report from someone using pg_upgrade coming from PG 8.3 --- turns out we didn't rename toast tables to match the new relfilenode in pre-8.4, so the attached applied patch avoids the check for these cases. This check is new in pg_upgrade for 9.1. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + commit 9e5bed2df1693a46dfaed862d7462ba2379f8f79 Author: Bruce Momjian br...@momjian.us Date: Sat Mar 5 21:12:21 2011 -0500 Restructure pg_upgrade checks because pre-8.4 Postgres did not rename toast file names to match the new relfilenode. diff --git a/contrib/pg_upgrade/info.c b/contrib/pg_upgrade/info.c index a502507..ad7edc4 100644 *** a/contrib/pg_upgrade/info.c --- b/contrib/pg_upgrade/info.c *** gen_db_file_maps(DbInfo *old_db, DbInfo *** 51,59 RelInfo*new_rel = new_db-rel_arr.rels[relnum]; if (old_rel-reloid != new_rel-reloid) ! pg_log(PG_FATAL, mismatch of relation id: database \%s\, old relid %d, new relid %d\n, old_db-db_name, old_rel-reloid, new_rel-reloid); ! create_rel_filename_map(old_pgdata, new_pgdata, old_db, new_db, old_rel, new_rel, maps + num_maps); num_maps++; --- 51,68 RelInfo*new_rel = new_db-rel_arr.rels[relnum]; if (old_rel-reloid != new_rel-reloid) ! pg_log(PG_FATAL, Mismatch of relation id: database \%s\, old relid %d, new relid %d\n, old_db-db_name, old_rel-reloid, new_rel-reloid); ! ! /* toast names were not renamed to match their relfilenodes in pre-8.4 */ ! if (GET_MAJOR_VERSION(old_cluster.major_version) = 804 ! (strcmp(old_rel-nspname, new_rel-nspname) != 0 || ! strcmp(old_rel-relname, new_rel-relname) != 0)) ! pg_log(PG_FATAL, Mismatch of relation names: database \%s\, ! old rel %s.%s, new rel %s.%s\n, ! old_db-db_name, old_rel-nspname, old_rel-relname, ! new_rel-nspname, new_rel-relname); ! create_rel_filename_map(old_pgdata, new_pgdata, old_db, new_db, old_rel, new_rel, maps + num_maps); num_maps++; *** create_rel_filename_map(const char *old_ *** 104,115 /* new_relfilenode will match old and new pg_class.oid */ map-new_relfilenode = new_rel-relfilenode; - if (strcmp(old_rel-nspname, new_rel-nspname) != 0 || - strcmp(old_rel-relname, new_rel-relname) != 0) - pg_log(PG_FATAL, mismatch of relation id: database \%s\, old rel %s.%s, new rel %s.%s\n, - old_db-db_name, old_rel-nspname, old_rel-relname, - new_rel-nspname, new_rel-relname); - /* used only for logging and error reporing, old/new are identical */ snprintf(map-nspname, sizeof(map-nspname), %s, old_rel-nspname); snprintf(map-relname, sizeof(map-relname), %s, old_rel-relname); --- 113,118 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
Robert, Some minor fixes: 197listitem 198 para 199 emphasisImplement a truly serializable isolation level/emphasis 200 /para 201/listitem Should be: emphasisImplement Serializable Snapshot Isolation, in order to provide a more robust serializable transaction mode/emphasis 212 emphasisAllow multiple collations to be used within a single 213 database/emphasis Should be: Allow collations to be declared at the column level, permitting multiple collations within a single database. I agree that the sections you have are not ideal, but I don't think it's necessary to fix them for an alpha release. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On Sat, Mar 5, 2011 at 5:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: David E. Wheeler da...@kineticode.com writes: On Mar 4, 2011, at 7:44 PM, Robert Haas wrote: So it seems like the only thing that is an absolute must-do is write some release notes. What about this? http://archives.postgresql.org/message-id/27152(dot)1299015062(at)sss(dot)pgh(dot)pa(dot)us Well, nobody else seems to be excited about it. And it's not like there isn't a clear upgrade path if we decide to change CREATE TYPE later --- it'll just be a bit clunkier. I am pretty suspicious that it's not a good design, but I don't understand it well enough to propose a better one (if there is a better one). -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sun, Mar 6, 2011 at 2:59 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Mar 5, 2011 at 11:56 AM, Simon Riggs si...@2ndquadrant.com wrote: Even though postmaster dies, the waiting backend keeps waiting until the timeout expires. Instead, the backends should periodically check whether postmaster is alive, and then they should exit immediately if it's not alive, as well as other process does? If the timeout is disabled, such backends would get stuck infinitely. Will wake them every 60 seconds I don't really see why sync rep should be responsible for solving this problem, which is an issue in many other situations as well, only for itself. In fact I think I'd prefer that it didn't, and that we wait for a more general solution that will actually fix this problem for real. I agree if such a general solution will be committed together with sync rep. Otherwise, because of sync rep, the backend can easily get stuck *infinitely*. When postmaster is not alive, all the existing walsenders exit immediately and no new walsender can appear. So there is no way to release the waiting backend. I think that some solutions for this issue which is likely to happen are required. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Sync Rep v19
On Sun, Mar 6, 2011 at 1:53 AM, Simon Riggs si...@2ndquadrant.com wrote:
On Sat, 2011-03-05 at 20:08 +0900, Fujii Masao wrote:
On Sat, Mar 5, 2011 at 7:28 AM, Simon Riggs si...@2ndquadrant.com wrote:
Yes, that can happen. As people will no doubt observe, this seems to be
an argument for wait-forever. What we actually need is a wait that lasts
longer than it takes for us to decide to failover, if the standby is
actually up and this is some kind of split brain situation. That way the
clients are still waiting when failover occurs. WAL is missing, but
since we didn't acknowledge the client we are OK to treat that situation
as if it were an abort.
Oracle Data Guard in the maximum availability mode behaves that way?
I'm sure that you are implementing something like the maximum availability
mode rather than the maximum protection one. So I'd like to know how
the data loss situation I described can be avoided in the maximum
availability
mode.
It can't. (Oracle or otherwise...)
Once we begin waiting for sync rep, if the transaction or backend ends
then other backends will be able to see the changed data. The only way
to prevent that is to shutdown the database to ensure that no readers or
writers have access to that.
Oracle's protection mechanism is to shutdown the primary if there is no
sync standby available. Maximum Protection. Any other mode must
therefore be less than maximum protection, according to Oracle, and me.
Available here means one that has not timed out, via parameter.
Shutting down the main server is cool, as long as you failover to one of
the standbys. If there aren't any standbys, or you don't have a
mechanism for switching quickly, you have availability problems.
What shutting down the server doesn't do is keep the data safe for
transactions that were in their commit-wait phase when the disconnect
occurs. That data exists, yet will not have been transferred to the
standby.
From now, I also say we should wait forever. It is the safest mode and I
want no argument about whether sync rep is safe or not. We can introduce
a more relaxed mode later with high availability for the primary. That
is possible and in some cases desirable.
Now, when we lose last sync standby we have three choices:
1. reconnect the standby, or wait for a potential standby to catchup
2. immediate shutdown of master and failover to one of the standbys
3. reclassify an async standby as a sync standby
More than likely we would attempt to do (1) for a while, then do (2)
This means that when we startup the primary will freeze for a while
until the sync standbys connect. Which is OK, since they try to
reconnect every 5 seconds and we don't plan on shutting down the primary
much anyway.
I've removed the timeout parameter, plus if we begin waiting we wait
until released, forever if that's how long it takes.
The recommendation to use more than one standby remains.
Fast shutdown will wake backends from their latch and there isn't any
changed interrupt behaviour any more.
synchronous_standby_names = '*' is no longer the default
On a positive note this is one less parameter and will improve
performance as well.
All above changes made.
Ready to commit, barring concrete objections to important behaviour.
I will do one final check tomorrow evening then commit.
I agree with this change.
One comment; what about introducing built-in function to wake up all the
waiting backends? When replication connection is closed, if we STONITH
the standby, we can safely (for not physical data loss but logical one)
switch the primary to standalone mode. But there is no way to wake up
the waiting backends for now. Setting synchronous_replication to OFF
and reloading the configuration file doesn't affect the existing waiting
backends. The attached patch introduces the pg_wakeup_all_waiters
(better name?) function which wakes up all the backends on the queue.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
***
*** 13905,13910 SELECT set_config('log_statement_stats', 'off', false);
--- 13905,13917
entrytypeboolean/type/entry
entryTerminate a backend/entry
/row
+ row
+entry
+ literalfunctionpg_wakeup_all_waiters()/function/literal
+ /entry
+entrytypevoid/type/entry
+entryWake up all the backends waiting for replication/entry
+ /row
/tbody
/tgroup
/table
***
*** 13939,13944 SELECT set_config('log_statement_stats', 'off', false);
--- 13946,13956
subprocess.
/para
+para
+ functionpg_wakeup_all_waiters/ signals all the backends waiting
+ for replication, to wake up and complete the transaction.
+/para
+
indexterm
primarybackup/primary
/indexterm
*** a/src/backend/replication/syncrep.c
Re: [HACKERS] Sync Rep v19
On Sun, Mar 6, 2011 at 4:51 PM, Fujii Masao masao.fu...@gmail.com wrote: One comment; what about introducing built-in function to wake up all the waiting backends? When replication connection is closed, if we STONITH the standby, we can safely (for not physical data loss but logical one) switch the primary to standalone mode. But there is no way to wake up the waiting backends for now. Setting synchronous_replication to OFF and reloading the configuration file doesn't affect the existing waiting backends. The attached patch introduces the pg_wakeup_all_waiters (better name?) function which wakes up all the backends on the queue. If unfortunately all connection slots are used by backends waiting for replication, we cannot execute such a function. So it makes more sense to introduce something like pg_ctl standalone command? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
