Re: [HACKERS] gSoC add MERGE command new patch -- merge_v104
On 24/08/10 23:56, Andres Freund wrote: I have to ask one question: On a short review of the discussion and the patch I didn't find anything about the concurrency issues involved (at least nodeModifyTable.c didnt show any). The SQL spec doesn't require MERGE to be an atomic upsert operation. Whats the plan to go forward at that subject? I think the patch needs to lock tables exclusively (the pg level, not access exclusive) as long as there is no additional handling... Well, you can always do LOCK TABLE before calling MERGE if that's what you want, but I don't think doing that automatically would make people happy. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: extensible enums
On 08/23/2010 07:34 PM, Bruce Momjian wrote: I looked at the pg_upgrade ramifications of this change and it seems some adjustments will have to be made. Right now pg_upgrade creates an empty enum type: CREATE TYPE etest AS ENUM (); and then it calls EnumValuesCreate() to create the enum labels. EnumValuesCreate() is called from within DefineEnum() where the enum type is created, and that assumes the enums are always created initially sorted. That would not be true when pg_upgrade is calling EnumValuesCreate() directly with oid assignment as part of an upgrade. I think the cleanest solution would be to modify pg_dump.c so that it creates an empty ENUM type like before, but uses the new ALTER TYPE myenum ADD 'newlabel' syntax to add the enum labels (with oid assignment like we do for CREATE TYPE, etc.) The existing code had to hack to call EnumValuesCreate() but with this new syntax it will no longer be necessary. The call to EnumValuesCreate() for enums is the only time pg_upgrade_support calls into a function rather than just assigning an oid to a global variable, so it would be great to remove that last direct function call usage. I've just been taking another look at this suggestion. I think it will work quite cleanly. As long as we add the enums in the correct order it should just do the Right Thing (tm). To answer your other question, Oid wraparound will not be a problem. Will get coding. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gSoC add MERGE command new patch -- merge_v104
On 2010-08-25 9:26 AM +0300, Heikki Linnakangas wrote: Whats the plan to go forward at that subject? I think the patch needs to lock tables exclusively (the pg level, not access exclusive) as long as there is no additional handling... Well, you can always do LOCK TABLE before calling MERGE if that's what you want, but I don't think doing that automatically would make people happy. I don't think having a MERGE that throws UNIQUE violations would make people happy either. Regards, Marko Tiikkaja -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN doesn't show the actual function expression for FunctionScan
Argument List?
--
dim
Le 24 août 2010 à 18:06, Tom Lane t...@sss.pgh.pa.us a écrit :
I wrote:
Robert Haas robertmh...@gmail.com writes:
If you try to put all that on the same line, I think it might get
awkwardly long. Perhaps something like:
Function Scan on function_name
Expression: function_name(function_arg1, function_arg2, ...)
Yeah, that's what I had in mind, but I'm still fumbling for the right
label. Expression seems a bit too generic.
How about Function Call?
A moment's hacking later:
regression=# explain verbose SELECT lv|| op|| rv FROM unnest(ARRAY[
ROW('1.2.2'::semver, '='::text, '1.2.2'::semver),
ROW('1.2.23', '=', '1.2.23')
]) AS f(lv semver, op text, rv semver);
QUERY PLAN
Function Scan on pg_catalog.unnest f (cost=0.00..1.50 rows=100 width=96)
Output: (((lv)::text || op) || (rv)::text)
Function Call: unnest(ARRAY[ROW(('1.2.2'::text)::semver, '='::text,
('1.2.2'::text)::semver), ROW('1.2.23', '=', '1.2.23')])
(3 rows)
Look reasonable?
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Backups from the standby (Incrementally Updated Backups), open item
On Wed, Aug 25, 2010 at 5:44 AM, Josh Berkus j...@agliodbs.com wrote: Again, given that this is a method which is (a) fairly minority-need, and (b) not at all tested in the field, I do not think it belongs in the main docs. Let's put it on the wiki and blog about it, and AFTER we've collected bug reports and have something relatively bulletproof for 9.1, THEN we put it in the main docs. We really shouldn't have anything in the main docs related to backup which isn't 100% tested. The procedure has been since 8.2. We should remove it from the documents since 8.2? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On Wed, Aug 25, 2010 at 07:11, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: 1. The new conversion seems to have stolen the apostrophe from D'Arcy J.M. Cain da...@druid.net, rendering him DArcy J.M. Cain da...@druid.net. Yeah, I see that too. It's probably bad input rather than the converter's fault ;-) indeed. Wrong type of escaping. For some reason I used '' when I should've used \'. I wonder where I got that idea :D 2. Any non-ASCII characters in, for example, contributor's names show up differently in the two repos. Generally, the original repo is OK and the new repo is garbled; although I found one very old example that went the other way. What it looks like to me is that a Latin1-UTF8 conversion has been applied to the log text. Which might be a good idea if it all *was* Latin1, but a fair-sized percentage isn't. Applying this conversion to UTF8 entries results in garbage, of course. Even if this could be done reliably, I think this counts as editorializing on the historical record, and should be switched off if possible. I think the problem is that we have a mix of them :( git requires it to be utf8. cvs2git is configured to try, in order, latin1, utf8 and ascii, and use whichever first returns correct result. In this case it seems it does return saying things are right, because the result is valid utf8 - just not the utf8 we expected. I can give it a try the other way around - trying utf8 *before* latin1, to see if that makes it better - utf8 tends to be more strict. There are also a number of commits that differ in order between the two repos, and an even larger number where commits are duplicated or merged in one repository relative to the other. I suspect that this is an artifact of the converter trying to merge nearby commits into one commit, which it more or less *has* to do for sanity since CVS commits aren't atomic. I don't have a problem with the concept, but I notice cases where the converted commit has a timestamp some minutes later than what the cvs2cl output claims. I suspect this is what the converter was using as a cutoff time. Would it be possible to make sure that the converted commit is always timestamped with the latest individual file update timestamp from the included CVS commits? I can't comment o nthis part - Michael or Max? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Backups from the standby (Incrementally Updated Backups), open item
On Tue, 2010-08-24 at 11:04 -0700, Josh Berkus wrote: I've been looking at the open item which belongs with this doc: http://www.postgresql.org/docs/9.0/static/backup-incremental-updated.html I'm back from holidays today, so will begin looking at this and related open-ish items. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] trace_recovery_messages
On Thu, 2010-08-19 at 19:06 -0400, Tom Lane wrote: Fujii Masao masao.fu...@gmail.com writes: The explanation of trace_recovery_messages in the document is inconsistent with the definition of it in guc.c. I've applied a patch for this. I was tempted to propose that we just rip out trace_recovery_messages altogether, but I suppose Simon would object. Thanks for keeping it in, hopefully it will help diagnose any errors. I laughed when I saw the commit message, so thanks for that. This is definitely a stop-gap facility. If you were to propose a more general facility for increasing log level of specific modules, I'm sure the rest of us would see that implemented across the rest of the code. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gSoC add MERGE command new patch -- merge_v104
On Wed, Aug 25, 2010 at 09:26:51AM +0300, Heikki Linnakangas wrote: On 24/08/10 23:56, Andres Freund wrote: I have to ask one question: On a short review of the discussion and the patch I didn't find anything about the concurrency issues involved (at least nodeModifyTable.c didnt show any). The SQL spec doesn't require MERGE to be an atomic upsert operation. Whats the plan to go forward at that subject? I think the patch needs to lock tables exclusively (the pg level, not access exclusive) as long as there is no additional handling... Well, you can always do LOCK TABLE before calling MERGE if that's what you want, but I don't think doing that automatically would make people happy. But randomly loosing tuples will make much more people unhappy. At a much more problematic point of time (in production). There is no locking prohibiting situations like trying to update a tuple which was concurrently deleted and thus loosing a tuple. Unless I miss something. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gSoC add MERGE command new patch -- merge_v104
On 25/08/10 12:41, Andres Freund wrote: But randomly loosing tuples will make much more people unhappy. At a much more problematic point of time (in production). Hmm, how would you lose tuples? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
On Fri, 2010-08-20 at 15:59 -0400, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Hmmm. It seems to me that we'd need a sharelock on the referenced row both times. No, we don't. The first update knows that it's updating a pre-existing referencing row and not changing the FK value. If someone were to try to delete the referenced row, they would see the original version of the referencing row as good and hence fail their FK deletion check. The case where we need a sharelock is for insertion of a new referencing row. It's to prevent the race condition where somebody deletes the referenced row and thinks it's OK because he doesn't see the new referencing row yet. In principle we don't need to sharelock ... ISTM that the cause of this issue is that we don't need a *share* lock at all, we need something slightly less than that. We place the share lock because we want to ensure that the PK value is not removed by either UPDATE or DELETE. There is no need to forbid UPDATEs that do not change the PK value on the referenced table. So I propose that we have a new kind of lock: nodelete lock. This is a regular row lock type and acts almost exactly same as a sharelock. Any attempt to change PK or DELETE the value must wait for the current lock holders transactions to complete. Other UPDATEs are possible - the locked state would be passed down the lock chain to latest version. We would change the RI code to use nodelete locks rather than share locks, which would then avoid the issue. It would not be possible to mix both nodeletelocks and sharelocks since the multixact infrastructure only allows one lockref. That's not likely to be a problem since sharelocks are mostly only used by RI anyway. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On 25/08/10 09:18, Magnus Hagander wrote: On Wed, Aug 25, 2010 at 07:11, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: 2. Any non-ASCII characters in, for example, contributor's names show up differently in the two repos. Generally, the original repo is OK and the new repo is garbled; although I found one very old example that went the other way. What it looks like to me is that a Latin1-UTF8 conversion has been applied to the log text. Which might be a good idea if it all *was* Latin1, but a fair-sized percentage isn't. Applying this conversion to UTF8 entries results in garbage, of course. Even if this could be done reliably, I think this counts as editorializing on the historical record, and should be switched off if possible. I think the problem is that we have a mix of them :( git requires it to be utf8. cvs2git is configured to try, in order, latin1, utf8 and ascii, and use whichever first returns correct result. In this case it seems it does return saying things are right, because the result is valid utf8 - just not the utf8 we expected. I can give it a try the other way around - trying utf8 *before* latin1, to see if that makes it better - utf8 tends to be more strict. *Every* byte sequence is valid latin1, therefore if you try latin1, utf8, ascii in that order, latin1 will always be used. You most likely want utf8, latin1 (no point also including ascii since it's a strict subset of latin1). There are also a number of commits that differ in order between the two repos, and an even larger number where commits are duplicated or merged in one repository relative to the other. I suspect that this is an artifact of the converter trying to merge nearby commits into one commit, which it more or less *has* to do for sanity since CVS commits aren't atomic. I don't have a problem with the concept, but I notice cases where the converted commit has a timestamp some minutes later than what the cvs2cl output claims. I suspect this is what the converter was using as a cutoff time. Would it be possible to make sure that the converted commit is always timestamped with the latest individual file update timestamp from the included CVS commits? I can't comment o nthis part - Michael or Max? cvs2git will try to use the timestamps from the commits, but sometimes the ordering of how revisions and tags relate to each other will actually disagree with the timestamps. In such a case, cvs2git nudges commit timestamps forward in time, to force the defined temporal ordering into consistency with the topological ordering of events. In other words, no, you can't make cvs2git *always* use the timestamp from a cvs commit, but it should have a good reason for doing so when it deviates from that. Max. signature.asc Description: OpenPGP digital signature
Re: [HACKERS] git: uh-oh
On Wed, Aug 25, 2010 at 13:03, Max Bowsher m...@f2s.com wrote: On 25/08/10 09:18, Magnus Hagander wrote: On Wed, Aug 25, 2010 at 07:11, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: 2. Any non-ASCII characters in, for example, contributor's names show up differently in the two repos. Generally, the original repo is OK and the new repo is garbled; although I found one very old example that went the other way. What it looks like to me is that a Latin1-UTF8 conversion has been applied to the log text. Which might be a good idea if it all *was* Latin1, but a fair-sized percentage isn't. Applying this conversion to UTF8 entries results in garbage, of course. Even if this could be done reliably, I think this counts as editorializing on the historical record, and should be switched off if possible. I think the problem is that we have a mix of them :( git requires it to be utf8. cvs2git is configured to try, in order, latin1, utf8 and ascii, and use whichever first returns correct result. In this case it seems it does return saying things are right, because the result is valid utf8 - just not the utf8 we expected. I can give it a try the other way around - trying utf8 *before* latin1, to see if that makes it better - utf8 tends to be more strict. *Every* byte sequence is valid latin1, therefore if you try latin1, utf8, ascii in that order, latin1 will always be used. You most likely want utf8, latin1 (no point also including ascii since it's a strict subset of latin1). Yup. I re-ran it with utf8, latin1, ascii and that commit looks better now. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On Tue, Aug 24, 2010 at 11:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Aug 20, 2010 at 1:56 PM, Max Bowsher m...@f2s.com wrote: My guess at this point is that there may be a (very old?) version of cvs which, when adding a file to a branch, actually misrecorded the file as having existed on the branch from the moment it was first added to trunk - this would explain this anomaly. I think this is what is happening, except I'm unable to account for it by the age of the CVS version we're runnning. The machine the CVS repo is running on is running 1.11.17-FreeBSD (client/server). Um, how old do you think that is? A look at the cvs sources says 2004... Oh, really? I didn't look that carefully; I just checked the date on the download directory, which was 2008. But I guess the actual code is older. The odder cases are the ones involving deletion. There are a couple of branches/tags that, or so I'm guessing, are only present for a subset of the files in the repository: ecpg_big_bison, creation, Release-1-6-0, MANUAL_1_0, REL2_0B, and SUPPORT. I'm wondering if we shouldn't just nuke those, or at least nuke them from the copy of the repository upon which we are running the conversion. Yeah, I noticed some of those in my copy of the test repository too, but I see a slightly different set: remotes/origin/REL2_0B remotes/origin/REL6_4 remotes/origin/Release_1_0_3 remotes/origin/WIN32_DEV remotes/origin/ecpg_big_bison I doubt they're of any more than archaeological interest, but do we want to be deleting history? Well, I think what those represent are partial tags. git has no equivalent, so anything that pops out this way is going to be totally wacko. We're not really deleting history; we're just declining to convert things that git can't represent accurately. It is sort of an interesting question why REL6_4 would fall into this category, but I can't imagine we care about any of the other ones. And if we do, well, we're not deleting the CVS tree. What seemed more likely to be artifacts were these: remotes/origin/unlabeled-1.44.2 remotes/origin/unlabeled-1.51.2 remotes/origin/unlabeled-1.59.2 remotes/origin/unlabeled-1.87.2 remotes/origin/unlabeled-1.90.2 Any idea where those came from? No; I don't see anything like that. What command did you run? This series of commits also seems pretty messed up: http://archives.postgresql.org/pgsql-committers/2007-04/msg00222.php http://archives.postgresql.org/pgsql-committers/2007-04/msg00223.php You can find out about the reasons for that in this *other* discussion of conversion to git: http://archives.postgresql.org/pgsql-hackers/2007-04/msg00670.php particularly here: http://archives.postgresql.org/pgsql-hackers/2007-04/msg00685.php ... pretty crazy. I think we should try to do something to clean this up, perhaps by doctoring the file on the CVS side. On the whole I feel that you're moving the goalposts. AFAIR the agreed criteria for an acceptable SCM conversion were that it reproduce the historical states of our tree at least at all the release tags, and that it provide a close approximation of the CVS commit logs. I think that manufactured commits that correspond to CVS's artifacts might be a bit ugly, but trying to get rid of them sounds way too much like putting lipstick on a pig. And if it means removing real, if ugly, history, I'm not sure I'm in favor of it at all. Well, when did it become a goal to get this git conversion done as soon as humanly possible? We *cannot* retroactively fix these issues after the conversion is done; or at least not without rewriting the entire repository history, which is something we do not want to do lightly - it is a major inconvenience for anyone who has already cloned, and particularly for, ahem, any companies that might be merging off of the repo. I don't think we should decide that we're unwilling to fix these issues without even discussing whether that's feasible or what would be involved. I don't think we're talking about removing history; I think we're talking about cleaning up corruption in CVS that will be irretrievably baked-in by the conversion. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] trace_recovery_messages
On Wed, Aug 25, 2010 at 5:36 AM, Simon Riggs si...@2ndquadrant.com wrote: This is definitely a stop-gap facility. If you were to propose a more general facility for increasing log level of specific modules, I'm sure the rest of us would see that implemented across the rest of the code. Yeah, I was thinking about that, too. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On 25/08/10 01:15, Robert Haas wrote: On Fri, Aug 20, 2010 at 1:56 PM, Max Bowsher m...@f2s.com wrote: My guess at this point is that there may be a (very old?) version of cvs which, when adding a file to a branch, actually misrecorded the file as having existed on the branch from the moment it was first added to trunk - this would explain this anomaly. I think this is what is happening, except I'm unable to account for it by the age of the CVS version we're runnning. The machine the CVS repo is running on is running 1.11.17-FreeBSD (client/server). I don't know how long it's been that way, but there are examples of this in the relatively recent past - like July 2nd of this year. I am 100% positive that what I did was 'cvs add' one new file, 'cvs delete' one old file, modify a few other things, and commit the whole deal. But in the git conversion there are two commits, one of which adds a copy of the file as it exists in HEAD and the other of which contains the balance of the changes. Every recent manufactured commit is of this same form: it immediately precedes the commit of which (in my view) it should be considered a part. Looking back a bit further in history, there is some stranger stuff. commit ec0274633871c43da670fa90d0ac4fd7090639f2 Author: PostgreSQL Daemon webmas...@postgresql.org Date: Mon Jun 6 16:30:43 2005 + This commit was manufactured by cvs2svn to create branch 'REL8_0_STABLE'. Cherrypick from master 2005-06-06 16:30:42 UTC Bruce Momjian br...@momjian. doc/src/FAQ/FAQ_hungarian.html And then, much later, the following completely empty commit: commit 446b749c2eaeff3c0611d33bc12b3df28e2cf8fa Author: Bruce Momjian br...@momjian.us Date: Tue Oct 4 14:17:44 2005 + Add FAQ_hungarian.html to 8.0.X branch. What really happened is: http://archives.postgresql.org/pgsql-committers/2005-10/msg00044.php So that's pretty much the same thing, except the time lag between the two commits that should be married is much larger. Yup, exact same problem, the file was added to the branch, and CVS erroneously recorded that it *had existed on the branch* from the moment it was created on trunk. The odder cases are the ones involving deletion. There are a couple of branches/tags that, or so I'm guessing, are only present for a subset of the files in the repository: ecpg_big_bison, creation, Release-1-6-0, MANUAL_1_0, REL2_0B, and SUPPORT. I'm wondering if we shouldn't just nuke those, or at least nuke them from the copy of the repository upon which we are running the conversion. Well, I'd caution against being too revisionist with your history, but if you're convinced you want to drop certain tags/branches, you can configure cvs2git to ignore them (see the symbol strategy rules part of the options file). Max. signature.asc Description: OpenPGP digital signature
[HACKERS] ECPG fix for mixed case cursor names
Hi,
PostgreSQL allows in plain SQL to declare a cursor
e.g. in all lower case and fetch from is in all upper case.
We need to allow this from ECPG, too, but strictly when
the cursor name is not in a variable. Otherwise this code
below doesn't notice the cursor's double declaration
and complains using an undeclared cursor:
==
#include stdio.h
#include sqlda.h
int main(void)
{
EXEC SQL BEGIN DECLARE SECTION;
char*connstr = z...@localhost:;
EXEC SQL END DECLARE SECTION;
sqlda_t*sqlda;
EXEC SQL CONNECT TO :connstr;
EXEC SQL DECLARE mycur CURSOR FOR SELECT * FROM t1;
EXEC SQL DECLARE MYCUR CURSOR FOR SELECT * FROM t1;
EXEC SQL OPEN mYCur;
EXEC SQL FETCH ALL FROM mYcUr INTO DESCRIPTOR sqlda;
EXEC SQL CLOSE MyCuR;
EXEC SQL DISCONNECT ALL;
return 0;
}
==
Patch is attached.
Best regards,
Zoltán Böszörményi
diff -dcrpN pgsql.orig/src/interfaces/ecpg/preproc/ecpg.addons pgsql/src/interfaces/ecpg/preproc/ecpg.addons
*** pgsql.orig/src/interfaces/ecpg/preproc/ecpg.addons 2010-03-31 10:45:18.0 +0200
--- pgsql/src/interfaces/ecpg/preproc/ecpg.addons 2010-08-25 12:53:07.0 +0200
*** ECPG: DeclareCursorStmtDECLAREcursor_nam
*** 302,311
struct cursor *ptr, *this;
char *cursor_marker = $2[0] == ':' ? make_str($0) : mm_strdup($2);
char *comment, *c1, *c2;
for (ptr = cur; ptr != NULL; ptr = ptr-next)
{
! if (strcmp($2, ptr-name) == 0)
{
if ($2[0] == ':')
mmerror(PARSE_ERROR, ET_ERROR, using variable \%s\ in different declare statements is not supported, $2+1);
--- 302,312
struct cursor *ptr, *this;
char *cursor_marker = $2[0] == ':' ? make_str($0) : mm_strdup($2);
char *comment, *c1, *c2;
+ int (* strcmp_fn)(const char *, const char *) = ($2[0] == ':' ? strcmp : pg_strcasecmp);
for (ptr = cur; ptr != NULL; ptr = ptr-next)
{
! if (strcmp_fn($2, ptr-name) == 0)
{
if ($2[0] == ':')
mmerror(PARSE_ERROR, ET_ERROR, using variable \%s\ in different declare statements is not supported, $2+1);
diff -dcrpN pgsql.orig/src/interfaces/ecpg/preproc/ecpg.header pgsql/src/interfaces/ecpg/preproc/ecpg.header
*** pgsql.orig/src/interfaces/ecpg/preproc/ecpg.header 2010-05-28 08:44:14.0 +0200
--- pgsql/src/interfaces/ecpg/preproc/ecpg.header 2010-08-25 12:56:40.0 +0200
*** add_additional_variables(char *name, boo
*** 386,395
{
struct cursor *ptr;
struct arguments *p;
for (ptr = cur; ptr != NULL; ptr=ptr-next)
{
! if (strcmp(ptr-name, name) == 0)
break;
}
--- 386,396
{
struct cursor *ptr;
struct arguments *p;
+ int (* strcmp_fn)(const char *, const char *) = (name[0] == ':' ? strcmp : pg_strcasecmp);
for (ptr = cur; ptr != NULL; ptr=ptr-next)
{
! if (strcmp_fn(ptr-name, name) == 0)
break;
}
diff -dcrpN pgsql.orig/src/interfaces/ecpg/preproc/ecpg.trailer pgsql/src/interfaces/ecpg/preproc/ecpg.trailer
*** pgsql.orig/src/interfaces/ecpg/preproc/ecpg.trailer 2010-05-28 08:44:14.0 +0200
--- pgsql/src/interfaces/ecpg/preproc/ecpg.trailer 2010-08-25 12:58:20.0 +0200
*** ECPGCursorStmt: DECLARE cursor_name cur
*** 289,301
{
struct cursor *ptr, *this;
char *cursor_marker = $2[0] == ':' ? make_str($0) : mm_strdup($2);
struct variable *thisquery = (struct variable *)mm_alloc(sizeof(struct variable));
const char *con = connection ? connection : NULL;
char *comment;
for (ptr = cur; ptr != NULL; ptr = ptr-next)
{
! if (strcmp($2, ptr-name) == 0)
{
/* re-definition is a bug */
if ($2[0] == ':')
--- 289,302
{
struct cursor *ptr, *this;
char *cursor_marker = $2[0] == ':' ? make_str($0) : mm_strdup($2);
+ int (* strcmp_fn)(const char *, const char *) = ($2[0] == ':' ? strcmp : pg_strcasecmp);
struct variable *thisquery = (struct variable *)mm_alloc(sizeof(struct variable));
const char *con = connection ? connection : NULL;
char *comment;
for (ptr = cur; ptr != NULL; ptr = ptr-next)
{
! if (strcmp_fn($2, ptr-name) == 0)
{
/* re-definition is a bug */
if ($2[0] == ':')
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] No documentation for filtering dictionary feature?
On Tue, 24 Aug 2010, Tom Lane wrote: There's an entry in the 9.0 release notes saying that we've got filtering dictionaries now. Cool, but I don't see any documentation of the feature in textsearch.sgml. Shouldn't there be some? Something like http://developer.postgresql.org/pgdocs/postgres/unaccent.html ? Regards, Oleg _ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On 25/08/10 14:03, Max Bowsher wrote: On 25/08/10 09:18, Magnus Hagander wrote: On Wed, Aug 25, 2010 at 07:11, Tom Lanet...@sss.pgh.pa.us wrote: Robert Haasrobertmh...@gmail.com writes: There are also a number of commits that differ in order between the two repos, and an even larger number where commits are duplicated or merged in one repository relative to the other. I suspect that this is an artifact of the converter trying to merge nearby commits into one commit, which it more or less *has* to do for sanity since CVS commits aren't atomic. I don't have a problem with the concept, but I notice cases where the converted commit has a timestamp some minutes later than what the cvs2cl output claims. I suspect this is what the converter was using as a cutoff time. Would it be possible to make sure that the converted commit is always timestamped with the latest individual file update timestamp from the included CVS commits? I can't comment o nthis part - Michael or Max? cvs2git will try to use the timestamps from the commits, but sometimes the ordering of how revisions and tags relate to each other will actually disagree with the timestamps. In such a case, cvs2git nudges commit timestamps forward in time, to force the defined temporal ordering into consistency with the topological ordering of events. Hmm, why does it force that consistency? AFAIK git is happy with a commit with an older timestamp following a commit with a newer timestamp. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On Wed, Aug 25, 2010 at 13:19, Robert Haas robertmh...@gmail.com wrote: What seemed more likely to be artifacts were these: remotes/origin/unlabeled-1.44.2 remotes/origin/unlabeled-1.51.2 remotes/origin/unlabeled-1.59.2 remotes/origin/unlabeled-1.87.2 remotes/origin/unlabeled-1.90.2 Any idea where those came from? No; I don't see anything like that. What command did you run? They were the originally. I later removed them from the repo - I bet Tom just managed to clone before I did. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On 25/08/10 04:21, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: What seemed more likely to be artifacts were these: remotes/origin/unlabeled-1.44.2 remotes/origin/unlabeled-1.51.2 remotes/origin/unlabeled-1.59.2 remotes/origin/unlabeled-1.87.2 remotes/origin/unlabeled-1.90.2 Any idea where those came from? These occur when there are numbered revisions in one or more RCS files, which lack a branch tag to identify their name. The most likely cause is deleting a branch after having committed to it. Indeed, all of these five correspond to a commit with the message: Make the world at least somewhat safe for zero-column tables, and remove the special case in ALTER DROP COLUMN to prohibit dropping a table's last column. I have an idea you can fix this by running the following on your live CVS repository: cvs rtag -D 2002-09-23 20:43:41 UTC zero-column-tables pgsql cvs rtag -F -B -r 1.44.2 zero-column-tables \ pgsql/src/backend/commands/tablecmds.c cvs rtag -F -B -r 1.90.2 zero-column-tables \ pgsql/src/backend/parser/parse_target.c cvs rtag -F -B -r 1.90.2 zero-column-tables \ pgsql/src/backend/access/common/tupdesc.c cvs rtag -F -B -r 1.59.2 zero-column-tables \ pgsql/src/backend/executor/execTuples.c cvs rtag -F -B -r 1.87.2 zero-column-tables \ pgsql/src/backend/executor/nodeAgg.c,v cvs rtag -F -B -r 1.51.2 zero-column-tables \ pgsql/src/test/regress/expected/alter_table.out (Untested as yet, I have a test conversion running.) This series of commits also seems pretty messed up: http://archives.postgresql.org/pgsql-committers/2007-04/msg00222.php http://archives.postgresql.org/pgsql-committers/2007-04/msg00223.php You can find out about the reasons for that in this *other* discussion of conversion to git: http://archives.postgresql.org/pgsql-hackers/2007-04/msg00670.php particularly here: http://archives.postgresql.org/pgsql-hackers/2007-04/msg00685.php ... pretty crazy. I think we should try to do something to clean this up, perhaps by doctoring the file on the CVS side. On the whole I feel that you're moving the goalposts. AFAIR the agreed criteria for an acceptable SCM conversion were that it reproduce the historical states of our tree at least at all the release tags, and that it provide a close approximation of the CVS commit logs. I think that manufactured commits that correspond to CVS's artifacts might be a bit ugly, but trying to get rid of them sounds way too much like putting lipstick on a pig. And if it means removing real, if ugly, history, I'm not sure I'm in favor of it at all. I'm mostly with Tom on this one. Basically you are now discovering what a mess CVS has made. The mess has always existed, but only now do you have the tools to notice this. Your options are: 1) Accept that. 2) Retroactively modify history to say that those generated files NEVER existed in the repository. 3) Retroactively modify history to say that those generated files are actually included in all those release tags. Max. signature.asc Description: OpenPGP digital signature
Re: [HACKERS] git: uh-oh
On 25/08/10 12:36, Heikki Linnakangas wrote: On 25/08/10 14:03, Max Bowsher wrote: On 25/08/10 09:18, Magnus Hagander wrote: On Wed, Aug 25, 2010 at 07:11, Tom Lanet...@sss.pgh.pa.us wrote: Robert Haasrobertmh...@gmail.com writes: There are also a number of commits that differ in order between the two repos, and an even larger number where commits are duplicated or merged in one repository relative to the other. I suspect that this is an artifact of the converter trying to merge nearby commits into one commit, which it more or less *has* to do for sanity since CVS commits aren't atomic. I don't have a problem with the concept, but I notice cases where the converted commit has a timestamp some minutes later than what the cvs2cl output claims. I suspect this is what the converter was using as a cutoff time. Would it be possible to make sure that the converted commit is always timestamped with the latest individual file update timestamp from the included CVS commits? I can't comment o nthis part - Michael or Max? cvs2git will try to use the timestamps from the commits, but sometimes the ordering of how revisions and tags relate to each other will actually disagree with the timestamps. In such a case, cvs2git nudges commit timestamps forward in time, to force the defined temporal ordering into consistency with the topological ordering of events. Hmm, why does it force that consistency? AFAIK git is happy with a commit with an older timestamp following a commit with a newer timestamp. Um. Good point. Why do enforce that? Michael, do you think anything would break if we just removed the ensure monotonicity code? Max. signature.asc Description: OpenPGP digital signature
Re: [HACKERS] No documentation for filtering dictionary feature?
Oleg Bartunov o...@sai.msu.su writes: On Tue, 24 Aug 2010, Tom Lane wrote: There's an entry in the 9.0 release notes saying that we've got filtering dictionaries now. Cool, but I don't see any documentation of the feature in textsearch.sgml. Shouldn't there be some? Something like http://developer.postgresql.org/pgdocs/postgres/unaccent.html ? No --- that's documentation for a specific contrib module, not documentation about the feature in general. Currently the general description of dictionaries says that it's impossible for a dictionary to modify the input lexeme. Clearly that needs some adjustment, but I'm not sure what, because the feature is not explained anywhere ;-) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
Simon, On 08/25/2010 11:53 AM, Simon Riggs wrote: ..we want to ensure that the PK value.. ..or any other possibly referenced attributes? Markus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
On Wed, 2010-08-25 at 15:51 +0200, Markus Wanner wrote: Simon, On 08/25/2010 11:53 AM, Simon Riggs wrote: ..we want to ensure that the PK value.. ..or any other possibly referenced attributes? Don't think that's relevant. referenced meaning by an RI constraint, which only ever refers to PKs in other tables. As a result the proposal is safe and useful for 99% of cases. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
Max Bowsher m...@f2s.com writes: On 25/08/10 12:36, Heikki Linnakangas wrote: On 25/08/10 14:03, Max Bowsher wrote: cvs2git will try to use the timestamps from the commits, but sometimes the ordering of how revisions and tags relate to each other will actually disagree with the timestamps. In such a case, cvs2git nudges commit timestamps forward in time, to force the defined temporal ordering into consistency with the topological ordering of events. Hmm, why does it force that consistency? AFAIK git is happy with a commit with an older timestamp following a commit with a newer timestamp. Um. Good point. Why do enforce that? Michael, do you think anything would break if we just removed the ensure monotonicity code? Yes, the cases that I noticed all had to do with some curious condition, like a time-extended CVS commit overlapping with another one on a disjoint set of files. (The sets of files had to be disjoint or CVS would have failed one commit at some point.) AFAICS there is no reason the git conversion can't arbitrarily choose one order or the other, and I would like it to choose an order based on real file commit timestamps rather than made-up ones. Some other cases that I noticed involved these manufactured commits that we've been whining about --- the real commit that straightens things out tends to be displaced by a minute or so, to no purpose whatsoever since in most cases there are no nearby commits. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Performance Farm Release
[resending after noticing that reply all resulted in sending to pgsql-hackers-owner rather than pgsql-hackers] Luxenberg, Scott I. scott.luxenb...@noblis.org wrote: This is just my email to notify you all that the project I've been working on with Stephen, the PostgreSQL Performance Farm, has been released. As of now, it only supports 9.0, due to the use of workers. More details can be found in the readme. The Git repository is located here: http://github.com/slux/Postgre-Performance-Farm If this is covered in a file that I missed, or on a Wiki somewhere, please point me in the right direction and accept my apologies. I looked at the README and didn't find enough to really answer my questions. Would this project be useful for someone trying to assess the performance impact of a proposed patch (at least on the developer's machine)? What would someone do to use it in this way? Thanks for any info. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
2010/8/25 Simon Riggs si...@2ndquadrant.com: referenced meaning by an RI constraint, which only ever refers to PKs in other tables. FK constraints can also point to non-PK UNIQUE columns. Nicolas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
On Wed, Aug 25, 2010 at 10:02 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-08-25 at 15:51 +0200, Markus Wanner wrote: Simon, On 08/25/2010 11:53 AM, Simon Riggs wrote: ..we want to ensure that the PK value.. ..or any other possibly referenced attributes? Don't think that's relevant. referenced meaning by an RI constraint, which only ever refers to PKs in other tables. That doesn't appear to be correct: rhaas=# create table p (a integer primary key, b integer not null, unique (b)); NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index p_pkey for table p NOTICE: CREATE TABLE / UNIQUE will create implicit index p_b_key for table p CREATE TABLE rhaas=# create table r (b integer not null references p (b)); CREATE TABLE -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Version Numbering
Hi, On 08/21/2010 10:11 PM, Peter Geoghegan wrote: We changed 8.5 to 9.0 explicitly because doing so was good marketing, That's exactly how I see this as well. The current scheme allows for some flexibility for marketing purposes while still being self-consistent and logical in numbering. sarcasm And it allows us to have heated debates about whether or not the next release is worth a major version bump. /sarcasm Regards Markus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN doesn't show the actual function expression for FunctionScan
Dimitri Fontaine dfonta...@hi-media.com writes:
Argument List?
Well, as shown in the example I posted, it's not just the argument list
but the whole call:
Function Call: unnest(ARRAY[ROW(('1.2.2'::text)::semver, '='::text,
('1.2.2'::text)::semver), ROW('1.2.23', '=', '1.2.23')])
Now you might suggest that the function itself is redundant with the
information given in the FunctionScan node line and so we need only
show the argument list. Unfortunately there are cases where this fails;
in particular, the named function could have been inlined by the
planner, meaning that the actual expression could be just about anything
at all. So I think that trying to be cute is a bad idea and we should
just print the nodetree as-is.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: Add JSON datatype to PostgreSQL (GSoC, WIP)
On Wed, Aug 25, 2010 at 1:34 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: * Should we accept a scalar value as a valid JSON? According to RFC, the root element of JSON text must be an object or array. But to_json() and from_json() accept scalar values. This seems a bit like the XML document/content distinction, which I've never really understood. If [[1], false] is a legal JSON value, then it seems odd that [1] should be legal but false not. * JSON to a scalar value by from_json() How about to have json_to_array(JSON) instead of from_json()? JSON value is always an array or object, it's nonsense to convert it to a scalar value directly; to an array seems to match better. In addition, an array can be indexed with GIN; index-able JSON type is very attractive. Yeah, I don't like the excessive use of polymorphism either. * On-disk format of JSON values (There might be some discussions before... What is the conclusion?) The current code stores the original input text, but we can use some kinds of pre-parsed format to store JSON, like hstore. It can be different from BSON. I see no value to choosing a different on-disk format. It might speed up indexing, but I/O will be slower. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
On Wed, 2010-08-25 at 16:14 +0200, Nicolas Barbier wrote: 2010/8/25 Simon Riggs si...@2ndquadrant.com: referenced meaning by an RI constraint, which only ever refers to PKs in other tables. FK constraints can also point to non-PK UNIQUE columns. You're exactly correct and I now understand Markus' comment. Do you think that exact meaning prevents my proposal from being useful? -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
2010/8/25 Simon Riggs si...@2ndquadrant.com: On Wed, 2010-08-25 at 16:14 +0200, Nicolas Barbier wrote: 2010/8/25 Simon Riggs si...@2ndquadrant.com: referenced meaning by an RI constraint, which only ever refers to PKs in other tables. FK constraints can also point to non-PK UNIQUE columns. You're exactly correct and I now understand Markus' comment. Do you think that exact meaning prevents my proposal from being useful? Not at all, because I guess that updates to non-UNIQUE columns are way more common that updates to UNIQUE columns. Nicolas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
On Wed, Aug 25, 2010 at 3:20 PM, Simon Riggs si...@2ndquadrant.com wrote: FK constraints can also point to non-PK UNIQUE columns. You're exactly correct and I now understand Markus' comment. Do you think that exact meaning prevents my proposal from being useful? I think it just shows it needs more thought. Do we want the nodelete locks to prevent updates to any unique keys? Or to specify the specific unique key that it's concerned with? Can we allow multiple nodelete locks on different keys? I'm concerned about the proliferation of special types of locks too. Having lots of different lock types tends to create more deadlocks rather than eliminate them so this requires some careful analysis of the interaction with all the other types of locks. And most importantly :) I don't like the name nodelete. Maybe record pins? Or keep locks? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
Nicolas Barbier nicolas.barb...@gmail.com writes: 2010/8/25 Simon Riggs si...@2ndquadrant.com: You're exactly correct and I now understand Markus' comment. Do you think that exact meaning prevents my proposal from being useful? Not at all, because I guess that updates to non-UNIQUE columns are way more common that updates to UNIQUE columns. In particular, HOT updates are known useful even though they have that restriction and more. It strikes me that a possibly useful simplification of the idea is a lock type that allows HOT updates and not non-HOT ones; or more precisely not ones that change any indexed columns --- if the row ends up having to go off-page for lack of space, that need not concern us. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQLSTATE of notice PGresult
Tom Lane escreveu: You didn't actually read what I said, did you? That patch will have precisely zero effect on the OP's example. Oh, I see your point. Didn't pay attention at the OP's example. I was only worried about the successful queries that doesn't return SQLSTATE but as you point out, that part of the code deserves a refactoring to cover OP's case too. -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HS/SR on AIX
Tom Lane wrote: Steve Singer ssin...@ca.afilias.info writes: I think I've been able to reproduce the issue floating around with streaming replication on AIX. Excellent, because we weren't getting much from the original reporter. I'm withdrawing my comment, today on a clean install of the binaries I am not able to reproduce any of this. Today with beta4 I can stream replication to the standby and bring the standby up to read-write without issues. Yesterday when I had put beta4 on top of beta2 without explicitly deleting all of the beta2 binaries/libraries first. I'm thinking maybe some portions of beta2 where still laying around. It is also possible that an old version of a shared library was still sitting in memory and was being picked up by the newer postgresql (man slibclean on AIX) I will do another clean build from the beta4 source tar to confirm that I'm not still having the issue but I'm thinking the original reporter might have done something similar and had some old artifacts laying around. This worked fine with beta2 but now seems to be an issue on beta4. If I do export LIBPATH=/opt/dbs/pgsql9-beta2/lib/ before starting the standby postmaster then it seems to work. Fascinating. That seems to prove that it's an rpath problem. My first guess is that the relevant change between beta2 and beta4 is my LDFLAGS changes. See http://archives.postgresql.org/pgsql-committers/2010-07/msg00060.php and following commits. regards, tom lane -- Steve Singer Afilias Canada Data Services Developer 416-673-1142 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
On 08/25/2010 04:57 PM, Tom Lane wrote: It strikes me that a possibly useful simplification of the idea is a lock type that allows HOT updates and not non-HOT ones; or more precisely not ones that change any indexed columns --- if the row ends up having to go off-page for lack of space, that need not concern us. Nice idea. And as Simon puts it, it probably covers most use cases. So called hot locks ;-) OTOH, if it doesn't cover HOT exactly (i.e. the go off-page case), it probably shouldn't be called that. And for helping in the OPs case, it would suffice to let the lock protect against changes to any attributes that are covered by a UNIQUEness constraint. So, the question probably is, what other use cases for such a lock type do exist? Regards Markus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
Max Bowsher wrote: On 25/08/10 12:36, Heikki Linnakangas wrote: On 25/08/10 14:03, Max Bowsher wrote: cvs2git will try to use the timestamps from the commits, but sometimes the ordering of how revisions and tags relate to each other will actually disagree with the timestamps. In such a case, cvs2git nudges commit timestamps forward in time, to force the defined temporal ordering into consistency with the topological ordering of events. Hmm, why does it force that consistency? AFAIK git is happy with a commit with an older timestamp following a commit with a newer timestamp. Um. Good point. Why do enforce that? Shallow answers: * It was adopted from cvs2svn, where timestamp monotonicity is not quite required but definitely advantageous. * Non-monotonic timestamps give one a spooky feeling of time travel. Deeper answers: * Even though git is tolerant of timestamps that are out of order, that doesn't mean that they are desirable. The most common reason for out-of-order CVS timestamps is that the CVS clients and/or server had clocks that were incorrect. (Several projects have reported that their server had a dead CMOS battery, causing the clock to be reset to 1970 for a while.) So often the enforced-monotonic timestamps produced by cvs2svn are an improvement on the CVS timestamps. * CVS (when functioning correctly) cannot generate events that are out of chronological order. Therefore, non-chronological events ipso facto represent repository corruption and it would be silly to try to preserve them. (The exception is that cvs2svn might combine commits too aggressively within its 5-minute timestamp window.) So I think that it makes sense to keep at least part of the ensure monotonicity behavior. However, there is one big difference between Subversion and git: Subversion requires a total ordering of commits, whereas git only requires a topological ordering. Currently, the ensure monotonicity code is applied after the commits have been totally ordered. Therefore, any mistakes made in choosing a total order among those consistent with the topological ordering constraints can lead to monotonicity fixups that are not justified by the topology. It might make sense, in the case of DVCSs, to fix up timestamps at an earlier step in the conversion. Michael, do you think anything would break if we just removed the ensure monotonicity code? No. It might be interesting to turn it off and see where the differences appear. Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
Max Bowsher m...@f2s.com writes: On 25/08/10 04:21, Tom Lane wrote: What seemed more likely to be artifacts were these: remotes/origin/unlabeled-1.44.2 remotes/origin/unlabeled-1.51.2 remotes/origin/unlabeled-1.59.2 remotes/origin/unlabeled-1.87.2 remotes/origin/unlabeled-1.90.2 Any idea where those came from? These occur when there are numbered revisions in one or more RCS files, which lack a branch tag to identify their name. The most likely cause is deleting a branch after having committed to it. Indeed, all of these five correspond to a commit with the message: Make the world at least somewhat safe for zero-column tables, and remove the special case in ALTER DROP COLUMN to prohibit dropping a table's last column. It seems likely to me that this has something to do with the aborted early branch for 7.4 development: http://archives.postgresql.org/pgsql-hackers/2002-09/msg01733.php If you read that thread you'll find an agreement that we'd continue development on HEAD and then do a mega back-patch into REL7_3_STABLE, but there is no mega back-patch later in the CVS logs. What actually happened is explained here: http://archives.postgresql.org/pgsql-hackers/2002-11/msg00113.php The first actual commit into REL7_3_STABLE that cvs2cl finds is a mass delete pursuant to my comment there. I am not sure exactly what Marc did to move the REL7_3_STABLE tag up to today, but I'll bet that the funny state of the 2002-09-28 commit has something to do with that, as it was the first commit into HEAD after Marc originally established the REL7_3_STABLE branch. Max's proposed fix seems to involve recognizing those extra versions as a legitimate branch, which I think we don't really want. It'd be better if we deleted them. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HS/SR on AIX
Steve Singer wrote: Tom Lane wrote: Steve Singer ssin...@ca.afilias.info writes: I think I've been able to reproduce the issue floating around with streaming replication on AIX. I will do another clean build from the beta4 source tar to confirm that I'm not still having the issue but I'm thinking the original reporter might have done something similar and had some old artifacts laying around. A clean build from the beta4 source tarball where I'm careful to install into a clean (ie no old beta2 artifacts laying around waiting to be overwritten) isn't reproducing the issue. I'm happy to try other things if people suggest them (or if the original reporter is still getting this after making sure he cleans up old files first) but I'm thinking that was the issue. -- Steve Singer Afilias Canada Data Services Developer 416-673-1142 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On 25/08/10 16:43, Tom Lane wrote:
Max Bowsher m...@f2s.com writes:
On 25/08/10 04:21, Tom Lane wrote:
What seemed more likely to be artifacts were these:
remotes/origin/unlabeled-1.44.2
remotes/origin/unlabeled-1.51.2
remotes/origin/unlabeled-1.59.2
remotes/origin/unlabeled-1.87.2
remotes/origin/unlabeled-1.90.2
Any idea where those came from?
These occur when there are numbered revisions in one or more RCS files,
which lack a branch tag to identify their name. The most likely cause is
deleting a branch after having committed to it.
Indeed, all of these five correspond to a commit with the message:
Make the world at least somewhat safe for zero-column tables, and
remove the special case in ALTER DROP COLUMN to prohibit dropping a
table's last column.
It seems likely to me that this has something to do with the aborted
early branch for 7.4 development:
http://archives.postgresql.org/pgsql-hackers/2002-09/msg01733.php
If you read that thread you'll find an agreement that we'd continue
development on HEAD and then do a mega back-patch into REL7_3_STABLE,
but there is no mega back-patch later in the CVS logs. What actually
happened is explained here:
http://archives.postgresql.org/pgsql-hackers/2002-11/msg00113.php
The first actual commit into REL7_3_STABLE that cvs2cl finds is
a mass delete pursuant to my comment there. I am not sure exactly
what Marc did to move the REL7_3_STABLE tag up to today, but I'll
bet that the funny state of the 2002-09-28 commit has something to
do with that, as it was the first commit into HEAD after Marc
originally established the REL7_3_STABLE branch.
Max's proposed fix seems to involve recognizing those extra versions
as a legitimate branch, which I think we don't really want. It'd be
better if we deleted them.
In that case, either employ an ExcludeRegexpStrategyRule('unlabeled-.*')
in the cvs2git options file, or drop those refs after converting to git.
Max.
signature.asc
Description: OpenPGP digital signature
Re: [HACKERS] Performance Farm Release
Kevin, * Kevin Grittner (kevin.gritt...@wicourts.gov) wrote: Would this project be useful for someone trying to assess the performance impact of a proposed patch (at least on the developer's machine)? What would someone do to use it in this way? The goal is to have this running in a similar manner as the build farm to identify when a patch has an impact on performance (good or bad). Hackers would then be able to view performance farm reports similar to viewing build farm reports. Not sure if we'd have alerts or something, but I'd think in alot of cases a given hacker would know that they're commiting something performance-impacting (or saw someone else commit something that could be) and they'd go check out the performance farm reports a few days later to determine if there was a change. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] git: uh-oh
Robert Haas wrote: This series of commits also seems pretty messed up: http://archives.postgresql.org/pgsql-committers/2007-04/msg00222.php http://archives.postgresql.org/pgsql-committers/2007-04/msg00223.php The commit messages make it clear that CVS did something funky, although it's not exactly clear retrospectively what it was. At any rate, it's evidently still not right, because in the converted repository we get a whole slough of commits like this: commit c50da22b6050e0bdd5e2ef97541d91aa1d2e63fb Author: PostgreSQL Daemon webmas...@postgresql.org Date: Sat Dec 2 08:36:42 2006 + This commit was manufactured by cvs2svn to create branch 'REL8_2_STABLE'. Sprout from master 2006-12-02 08:36:41 UTC PostgreSQL Daemon webmas...@post Delete: src/backend/parser/gram.c src/interfaces/ecpg/preproc/pgc.c src/interfaces/ecpg/preproc/preproc.c There are similar (but separate) commits for tag REL8_2_RC1, REL8_2_BETA3, REL8_2_BETA2, REL8_2_BETA1, REL8_1_STABLE, REL8_1_0_RC1, REL8_1_0BETA4, REL8_1_0BETA3, REL8_1_0BETA2, REL8_1_0BETA1, REL8_0_0, REL8_0_0RC5, REL8_0_0RC4, REL8_0_0RC3, REL8_0_0RC2, REL8_0_0RC1, REL8_0_0BETA5, REL8_0_0BETA4, REL8_0_0BETA3, REL8_0_0BETA2, REL8_0_0BETA1, REL7_4_STABLE, REL7_4_BETA5, REL7_4_BETA4, REL7_4_BETA3, REL7_4_BETA2, REL7_4_BETA1, REL7_2_STABLE, REL7_2, REL7_2_RC2, REL7_2_RC1, REL7_2_BETA5, REL7_2_BETA4, REL7_2_BETA3, REL7_2_BETA2, REL7_2_BETA1, REL7_1_STABLE, REL7_1_BETA3, REL7_1_BETA2, REL7_0_PATCHES, REL7_0, REL6_5_PATCHES, and release-6-3. That's pretty crazy. I think we should try to do something to clean this up, perhaps by doctoring the file on the CVS side. This is probably caused by cvs2svn's failure to consider file deletions when choosing the best revision from which to branch [1]. It would be better to branch all of these symbols from the commit *after* the files were deleted, which would make them all exact copies of the original (rather than requiring a fixup branch). I don't think that this can be fixed by doctoring the CVS repository (at least, not short of removing the three files from the entire project history). It could be fixed post-conversion by using grafts, or by shifting the tags and rebasing the branches. I must say, it is refreshing to have users who actually care about their conversion, as opposed to the usual rabble who think that git-cvsimport is Just Fine :-) I guess if the postgresql project didn't care about data integrity then we would all have to worry :-) Michael [1] http://cvs2svn.tigris.org/issues/show_bug.cgi?id=55 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Performance Farm Release
Stephen Frost sfr...@snowman.net wrote: The goal is to have this running in a similar manner as the build farm to identify when a patch has an impact on performance (good or bad). Hackers would then be able to view performance farm reports similar to viewing build farm reports. Not sure if we'd have alerts or something, but I'd think in alot of cases a given hacker would know that they're commiting something performance- impacting (or saw someone else commit something that could be) and they'd go check out the performance farm reports a few days later to determine if there was a change. I actually understood that part, but was already wondering if it could be bent to slightly different purposes. It seems as though there would be value to using it to evaluate the performance impact of a proposed patch, at least on a limited basis, *before* a commit. If there's not an immediately obvious way to put it to that alternative use, that's OK; I just thought I'd ask. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On Wed, Aug 25, 2010 at 12:02 PM, Michael Haggerty mhag...@alum.mit.edu wrote: I think we should try to do something to clean this up, perhaps by doctoring the file on the CVS side. This is probably caused by cvs2svn's failure to consider file deletions when choosing the best revision from which to branch [1]. It would be better to branch all of these symbols from the commit *after* the files were deleted, which would make them all exact copies of the original (rather than requiring a fixup branch). I don't think that this can be fixed by doctoring the CVS repository (at least, not short of removing the three files from the entire project history). It could be fixed post-conversion by using grafts, or by shifting the tags and rebasing the branches. Well, the history here is pretty weird. In relevant part, here's the result of cvs log on src/backend/parser/gram.c: revision 2.92 date: 2007/04/17 01:06:27; author: tgl; state: dead; lines: +0 -0 And remove 'em again ... revision 2.91 date: 2007/04/17 01:05:07; author: tgl; state: Exp; lines: +0 -12088 Temporarily re-add derived files, in hopes of straightening out their CVS status. revision 2.90 date: 1999/05/07 01:22:54; author: vadim; state: Exp; lines: +6001 -5942 branches: 2.90.2; Fix LMGR for MVCC. Get rid of Extend lock mode. revision 2.89 date: 1999/03/28 20:32:04; author: vadim; state: Exp; lines: +3292 -3225 1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted. revision 2.88 date: 1999/03/20 18:43:49; author: tgl; state: dead; lines: +1 -1 Remove yacc/lex output files from CVS repository. The fact that the file was modified twice after being removed at rev 2.88 seems really wacko. Are you sure that's not contributing to what we're seeing here? I must say, it is refreshing to have users who actually care about their conversion, as opposed to the usual rabble who think that git-cvsimport is Just Fine :-) I guess if the postgresql project didn't care about data integrity then we would all have to worry :-) I laughed when I read this - yeah, we're kind of paranoid about that. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Performance Farm Release
* Kevin Grittner (kevin.gritt...@wicourts.gov) wrote: I actually understood that part, but was already wondering if it could be bent to slightly different purposes. It seems as though there would be value to using it to evaluate the performance impact of a proposed patch, at least on a limited basis, *before* a commit. If there's not an immediately obvious way to put it to that alternative use, that's OK; I just thought I'd ask. You can certainly run it yourself locally w/o setting it up to report back to the build or performance farm.. So, yes, you can, you'll just have to look through the outputs yourself and it won't necessairly make much sense unless you've been doing those runs for a period of time to get a feel for how volatile the speed is on your system.. I guess one issue would be figuring out how to inject your patch into the process.. If you have it committed to a git instance somewhere, you could tell it to pull from that after running on the main PG w/o the patch.. Stephen signature.asc Description: Digital signature
Re: [HACKERS] Deadlock bug
It strikes me that a possibly useful simplification of the idea is a lock type that allows HOT updates and not non-HOT ones; or more precisely not ones that change any indexed columns --- if the row ends up having to go off-page for lack of space, that need not concern us. While an improvement over the current, that's still more restrictive than we actually need for FKs. FKs just need to lock the value of the reference column(s); they don't care if *other* indexes are updated. Thus, for an RI reference, we care about one and exactly one unique/PK index on the referenced table. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On Wed, Aug 25, 2010 at 5:35 PM, Robert Haas robertmh...@gmail.com wrote: Well, the history here is pretty weird. In relevant part, here's the result of cvs log on src/backend/parser/gram.c: Interestingly this weirdness first surfaced due to a previous discussion of using git about 3 and a half years ago: http://thread.gmane.org/gmane.comp.db.postgresql.devel.general/80800/focus=80809 -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
Robert Haas robertmh...@gmail.com writes: The fact that the file was modified twice after being removed at rev 2.88 seems really wacko. Are you sure that's not contributing to what we're seeing here? Yeah, that was discussed in the earlier git-conversion thread that I pointed to. We never did figure out how that happened, though I speculated it might have been due to weirdness in Vadim's local checkout. Is it possible to just delete those two revisions from the CVS repository, and if so would it help? We certainly don't need 'em. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
Robert Haas wrote: Well, the history here is pretty weird. In relevant part, here's the result of cvs log on src/backend/parser/gram.c: revision 2.92 date: 2007/04/17 01:06:27; author: tgl; state: dead; lines: +0 -0 And remove 'em again ... revision 2.91 date: 2007/04/17 01:05:07; author: tgl; state: Exp; lines: +0 -12088 Temporarily re-add derived files, in hopes of straightening out their CVS status. revision 2.90 date: 1999/05/07 01:22:54; author: vadim; state: Exp; lines: +6001 -5942 branches: 2.90.2; Fix LMGR for MVCC. Get rid of Extend lock mode. revision 2.89 date: 1999/03/28 20:32:04; author: vadim; state: Exp; lines: +3292 -3225 1. Vacuum is updated for MVCC. 2. Much faster btree tuples deletion in the case when first on page index tuple is deleted (no movement to the left page(s)). 3. Remember blkno of new root page in BTPageOpaque of left/right siblings when root page is splitted. revision 2.88 date: 1999/03/20 18:43:49; author: tgl; state: dead; lines: +1 -1 Remove yacc/lex output files from CVS repository. The fact that the file was modified twice after being removed at rev 2.88 seems really wacko. Are you sure that's not contributing to what we're seeing here? I think this is the normal behavior when a file is deleted then re-added. In version 2.89 the file was re-added, and its delta is against the pre-deleted version (presumably 2.87). (Actually, even deleted versions can have deltas, so technically the delta in 2.89 is against the hidden content of version 2.88. Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
Josh Berkus j...@agliodbs.com writes: It strikes me that a possibly useful simplification of the idea is a lock type that allows HOT updates and not non-HOT ones; or more precisely not ones that change any indexed columns --- if the row ends up having to go off-page for lack of space, that need not concern us. While an improvement over the current, that's still more restrictive than we actually need for FKs. FKs just need to lock the value of the reference column(s); they don't care if *other* indexes are updated. That is true, but tracking exactly which indexes are relevant for that, at the extremely low level that this would have to take effect, doesn't seem like a bright plan to me. It's already ugly beyond words that heapam.c knows enough about indexes to enforce the HOT restriction; I do *not* want it having to know about FKs. There would also be new locking restrictions added by the mere fact of trying to do that, because DDL operations that previously didn't have to lock out SELECT FOR SHARE now would. With Simon's patch that reduces ALTER TABLE ADD FOREIGN KEY to not take AccessExclusiveLock, that's not a vacuous concern anymore. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
Michael Haggerty mhag...@alum.mit.edu writes: Robert Haas wrote: The fact that the file was modified twice after being removed at rev 2.88 seems really wacko. Are you sure that's not contributing to what we're seeing here? I think this is the normal behavior when a file is deleted then re-added. In version 2.89 the file was re-added, and its delta is against the pre-deleted version (presumably 2.87). The thing that was confusing was that Vadim apparently saw the file as being still live, while none of the rest of us did. I don't think he did an explicit cvs add to make it live again, because if he had, that should have propagated to the repository and the rest of us would have seen it. It took some considerable fooling around (though unfortunately I don't recall the exact details) to persuade my checkout that gram.c wasn't deleted so that I could delete it again. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
On Wed, Aug 25, 2010 at 6:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: That is true, but tracking exactly which indexes are relevant for that, at the extremely low level that this would have to take effect, doesn't seem like a bright plan to me. It's already ugly beyond words that heapam.c knows enough about indexes to enforce the HOT restriction; I do *not* want it having to know about FKs. Well the alternative is teaching FKs how to handle locks. Ie, if you could lock just certain columns of a row then heapam.c only needs to check if those columns are being updated. It doesn't have to understand why those columns are the ones that matter. It's still not a very practical idea at least at first glance. It would mean storing a variable sized list of columns somewhere that can be consulted when the update happens. I don't know how the share lock infrastructure works but I don't think it's obvious that there is such a place. There would also be new locking restrictions added by the mere fact of trying to do that, because DDL operations that previously didn't have to lock out SELECT FOR SHARE now would. I think the above would solve that too. Since the FK trigger would explicitly list the columns being referenced, dropping or adding an index wouldn't change which columns were already locked in the rows that were already looked up using the index. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] initdb fails to allocate shared memory
A.M. age...@themactionfaction.com writes: On Aug 25, 2010, at 12:28 PM, Tom Lane wrote: However, it's odd that you got this variant of the HINT and not the one that suggests increasing SHMMAX. Looking at the code, that means that shmget returned ENOMEM, not EINVAL, which surprises me. I was already running an instance of pgsql 8.4, so that sucked up all the shared mem space and there was none left for 9.0b4. Ah-hah. I now realize that we've been interpreting ENOMEM too narrowly. The Single Unix Spec defines ENOMEM from shmget thusly: [ENOMEM] A shared memory identifier and associated shared memory segment are to be created but the amount of available physical memory is not sufficient to fill the request. from which you'd assume that the only way to fix it is (a) buy more RAM or (b) decrease the request size. Thus we only suggest (b). However, it appears from your report that OS X is also using ENOMEM when SHMALL is exceeded, which is not all that surprising because actually none of the spec-defined error codes cover SHMALL exhaustion. The Linux manpage for shmget says that ENOSPC is used when SHMALL is exhausted, which is equally reasonable but it's *not* what we read in SUS: [ENOSPC] A shared memory identifier is to be created but the system-imposed limit on the maximum number of allowed shared memory identifiers system-wide would be exceeded. So apparently different implementations are not very consistent about which code they return for SHMALL exhaustion, and we should change the wording for the ENOMEM hint to mention SHMALL as a possible cause. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
Greg Stark gsst...@mit.edu writes: It's still not a very practical idea at least at first glance. It would mean storing a variable sized list of columns somewhere that can be consulted when the update happens. I don't know how the share lock infrastructure works but I don't think it's obvious that there is such a place. Yeah, there are all sorts of practical issues to be solved before this idea is more than a pipe dream; one being that the method for marking a row as locked involves setting its xmax, which is none too compatible with having somebody else actually update it. Maybe you could make it work by copying the xmax forward to the new version, but it seems ticklish. However, minimizing the amount of state needed to determine whether an update is allowed would clearly help to surmount at least some of the practical problems, which is why I suggested piggybacking on the HOT logic. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Performance Farm Release
(resending as I also accidentally CC'd pgsql-hackers-owner, not the list) On 25/08/10 02:25, Luxenberg, Scott I. wrote: This is just my email to notify you all that the project I've been working on with Stephen, the PostgreSQL Performance Farm, has been released. As of now, it only supports 9.0, due to the use of workers. More details can be found in the readme. The Git repository is located here: http://github.com/slux/Postgre-Performance-Farm Umm, how about renaming it to PostgreSQL-Performance-Farm before the name gets too engrained everywhere and impossible to change anymore? The project is not called Postgre.. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git: uh-oh
On Wed, Aug 25, 2010 at 1:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The fact that the file was modified twice after being removed at rev 2.88 seems really wacko. Are you sure that's not contributing to what we're seeing here? Yeah, that was discussed in the earlier git-conversion thread that I pointed to. We never did figure out how that happened, though I speculated it might have been due to weirdness in Vadim's local checkout. Is it possible to just delete those two revisions from the CVS repository, and if so would it help? We certainly don't need 'em. cvs admin -o ? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] new notify payload as string
With 9.0b4, I am testing the new NOTIFY payload feature. One thing I noticed is that it seems impossible to differentiate at the receiving end from: NOTIFY test; and NOTIFY test,''; So, it is impossible to differentiate between a notification with an empty string payload and a notification without a payload due to the backend protocol defining the payload as a string. Perhaps a boolean could be added to the backend protocol to account for this and then extra could be set to NULL in libpq. This could be handy when converting codebases that use LISTEN/NOTIFY when one wishes to be warned of old NOTIFYs being used without a payload. Cheers, M -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] initdb fails to allocate shared memory
I wrote: it appears from your report that OS X is also using ENOMEM when SHMALL is exceeded, which is not all that surprising because actually none of the spec-defined error codes cover SHMALL exhaustion. A look into http://www.opensource.apple.com/source/xnu/xnu-1504.7.4/bsd/kern/sysv_shm.c confirms this: if ((user_ssize_t)(shm_committed + btoc(total_size)) shminfo.shmall) return ENOMEM; So this behavior is probably common among BSDen. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 2.7 deprecated the PyCObject API?
On tis, 2010-08-17 at 21:48 +0300, Peter Eisentraut wrote: On tis, 2010-08-17 at 20:55 +0300, Peter Eisentraut wrote: On fre, 2010-08-13 at 20:20 -0400, Tom Lane wrote: According to a discussion over in Fedora-land, $subject is true: http://lists.fedoraproject.org/pipermail/devel/2010-August/140995.html I see several calls in plpython.c that seem to refer to PyCObject stuff. Anybody have any idea if we need to do something about this? Some exception handling might be good, but I think we don't need to abandon the API yet. It's only pending deprecation. Here is a patch. The crash is reproducible, if warnings are turned into errors. I have applied this patch back to 8.0. 7.4's plpython crashes with Python = 2.5; it's probably not worth rescuing. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] new notify payload as string
A.M. age...@themactionfaction.com writes: So, it is impossible to differentiate between a notification with an empty string payload and a notification without a payload due to the backend protocol defining the payload as a string. That's correct. This was baked into the FE/BE protocol in 2003; we're not going to change it now, especially not without a pretty convincing argument why it's critical functionality to have. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] No documentation for filtering dictionary feature?
On Wed, 25 Aug 2010, Tom Lane wrote: Oleg Bartunov o...@sai.msu.su writes: On Tue, 24 Aug 2010, Tom Lane wrote: There's an entry in the 9.0 release notes saying that we've got filtering dictionaries now. Cool, but I don't see any documentation of the feature in textsearch.sgml. Shouldn't there be some? Something like http://developer.postgresql.org/pgdocs/postgres/unaccent.html ? No --- that's documentation for a specific contrib module, not documentation about the feature in general. Currently the general description of dictionaries says that it's impossible for a dictionary to modify the input lexeme. Clearly that needs some adjustment, but I'm not sure what, because the feature is not explained anywhere ;-) Well, http://www.pgcon.org/2009/schedule/attachments/92_2009.pdf contains description of Dictionary API and new TSLexeme-flags: TSL_FILTER If dictionary returns only one lexeme with TSL_FILTER flag, then that lexeme will be used as an input for the subsequent dictionaries in the chain. I'm not sure, how to incorporate this into current documentation, since there is no built-in filtering dictionary. Probably, it'd be nice to add filtering option to simple dictionary to illustrate this feature. Unfortunately, I'm busy-busy :( Regards, Oleg _ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] No documentation for filtering dictionary feature?
Oleg Bartunov o...@sai.msu.su writes: On Wed, 25 Aug 2010, Tom Lane wrote: No --- that's documentation for a specific contrib module, not documentation about the feature in general. Currently the general description of dictionaries says that it's impossible for a dictionary to modify the input lexeme. Clearly that needs some adjustment, but I'm not sure what, because the feature is not explained anywhere ;-) Well, http://www.pgcon.org/2009/schedule/attachments/92_2009.pdf contains description of Dictionary API and new TSLexeme-flags: TSL_FILTER If dictionary returns only one lexeme with TSL_FILTER flag, then that lexeme will be used as an input for the subsequent dictionaries in the chain. OK. I'll see what can be done with the docs. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
On Wed, 2010-08-25 at 14:10 -0400, Tom Lane wrote: Greg Stark gsst...@mit.edu writes: It's still not a very practical idea at least at first glance. It would mean storing a variable sized list of columns somewhere that can be consulted when the update happens. I don't know how the share lock infrastructure works but I don't think it's obvious that there is such a place. Yeah, there are all sorts of practical issues to be solved before this idea is more than a pipe dream; one being that the method for marking a row as locked involves setting its xmax, which is none too compatible with having somebody else actually update it. Maybe you could make it work by copying the xmax forward to the new version, but it seems ticklish. That's the plan. Can't see a problem, but will let you know. However, minimizing the amount of state needed to determine whether an update is allowed would clearly help to surmount at least some of the practical problems, which is why I suggested piggybacking on the HOT logic. If the row is key share locked (as opposed to tuple share locks we already have), then an UPDATE would only work if it was a non-HOT UPDATE. Yes, that would save us some effort in working out whether to allow the UPDATE or not. It *is* more restrictive than strictly necessary, but much better than the current situation. So at least we know that part of it has an easy solution. I propose to make RI checks use FOR KEY SHARE LOCK rather than FOR SHARE LOCK. So we keep the semantics for share locking for explicit users, just add a new flavour. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Committers info for the git migration - URGENT!
Magnus Hagander mag...@hagander.net writes: The current mapping used is the same one as on git.postgresql.org (see attached file). BTW, I noticed that this list omits several old committers: 162 bryanh 20 byronn 6 julian 1 mcguirk (the numbers are the number of commits I find in cvs2cl for each name). I am pretty sure of the first two: Bryan Henderson bry...@giraffe.netgate.net Byron Nikolaidis byr...@insightdist.com and I think the others are Julian Assange pr...@suburbia.net Dan McGuirk mcgu...@indirect.com though they're before my time. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deadlock bug
On 8/25/10 1:35 PM, Simon Riggs wrote: If the row is key share locked (as opposed to tuple share locks we already have), then an UPDATE would only work if it was a non-HOT UPDATE. Yes, that would save us some effort in working out whether to allow the UPDATE or not. It *is* more restrictive than strictly necessary, but much better than the current situation. So at least we know that part of it has an easy solution. I agree that this would be an improvement. It still has the issue of being baffling to users (why did I get a deadlock this time, and not THAT time?) but current behavior has that problem. Heck, current behavior is often baffling to *me*. The other thing which came out of this incident (and many user reports) is the rather extreme opacity of our locking information, despite the improvements of the last 2 versions. However, I don't have a proposal on how that should be fixed yet. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Backups from the standby (Incrementally Updated Backups), open item
Simon Riggs wrote: On Tue, 2010-08-24 at 11:04 -0700, Josh Berkus wrote: I've been looking at the open item which belongs with this doc: http://www.postgresql.org/docs/9.0/static/backup-incremental-updated.html I'm back from holidays today, so will begin looking at this and related open-ish items. I talked to several people on IM about this today and there was general agreement that the procedure as outlined, and even as proposed with pg_start_backup(), is not known to be reliable enough to put in our docs. I have therefore removed it from the 9.0 docs, and created a wiki for it: http://wiki.postgresql.org/wiki/Incrementally_Updated_Backups I looked at the 8.4 doc mention: http://www.postgresql.org/docs/8.4/static/warm-standby.html#BACKUP-INCREMENTAL-UPDATED and it was sufficiently vague that no one has obviously implemented it or we would have gotten complaints that it does not work, so I am not sure about removing all previous mentions. I think the fact it might work with full page writes allows it to work, but it is not something we do not want to continue recommending. If we get enough research that the wiki-listed procedure is good, we can move it back into the docs. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bg worker: patch 1 of 6 - permanent process
On Tue, Jul 13, 2010 at 11:31 PM, Markus Wanner mar...@bluegap.ch wrote: This patch turns the existing autovacuum launcher into an always running process, partly called the coordinator. If autovacuum is disabled, the coordinator process still gets started and keeps around, but it doesn't dispatch vacuum jobs. I think this part is a reasonable proposal, but... The coordinator process now uses imessages to communicate with background (autovacuum) workers and to trigger a vacuum job. It also adds two new controlling GUCs: min_spare_background_workers and max_spare_background_workers. Other changes in the patch doesn't seem be always needed for the purpose. In other words, the patch is not minimal. The original purpose could be done without IMessage. Also, min/max_spare_background_workers are not used in the patch at all. (BTW, min/max_spare_background_*helpers* in postgresql.conf.sample is maybe typo.) The most questionable point for me is why you didn't add any hook functions in the coordinator process. With the patch, you can extend the coordinator protocols with IMessage, but it requires patches to core at handle_imessage(). If you want fully-extensible workers, we should provide a method to develop worker codes in an external plugin. Is it possible to develop your own codes in the plugin? If possible, you can use IMessage as a private protocol freely in the plugin. Am I missing something? -- Itagaki Takahiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bg worker: patch 1 of 6 - permanent process
On Wed, Aug 25, 2010 at 9:39 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Tue, Jul 13, 2010 at 11:31 PM, Markus Wanner mar...@bluegap.ch wrote: This patch turns the existing autovacuum launcher into an always running process, partly called the coordinator. If autovacuum is disabled, the coordinator process still gets started and keeps around, but it doesn't dispatch vacuum jobs. I think this part is a reasonable proposal, but... It's not clear to me whether it's better to have a single coordinator process that handles both autovacuum and other things, or whether it's better to have two separate processes. The coordinator process now uses imessages to communicate with background (autovacuum) workers and to trigger a vacuum job. It also adds two new controlling GUCs: min_spare_background_workers and max_spare_background_workers. Other changes in the patch doesn't seem be always needed for the purpose. In other words, the patch is not minimal. The original purpose could be done without IMessage. Also, min/max_spare_background_workers are not used in the patch at all. (BTW, min/max_spare_background_*helpers* in postgresql.conf.sample is maybe typo.) The most questionable point for me is why you didn't add any hook functions in the coordinator process. With the patch, you can extend the coordinator protocols with IMessage, but it requires patches to core at handle_imessage(). If you want fully-extensible workers, we should provide a method to develop worker codes in an external plugin. I agree with this criticism, but the other thing that strikes me as a nonstarter is having the postmaster participate in the imessages framework. Our general rule is that the postmaster must avoid touching shared memory; else a backend that scribbles on shared memory might take out the postmaster, leading to a failure of the crash-and-restart logic. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Committers info for the git migration - URGENT!
On Aug 25, 2010, at 6:49 PM, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: The current mapping used is the same one as on git.postgresql.org (see attached file). BTW, I noticed that this list omits several old committers: Julian Assange pr...@suburbia.net That is _the_ Julian Assange who is in the news now. Very cool! http://en.wikipedia.org/wiki/Julian_Assange#Career_as_computer_programmer -M
Re: [HACKERS] bg worker: patch 1 of 6 - permanent process
On Thu, Aug 26, 2010 at 11:39 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jul 13, 2010 at 11:31 PM, Markus Wanner mar...@bluegap.ch wrote: This patch turns the existing autovacuum launcher into an always running process, partly called the coordinator. It's not clear to me whether it's better to have a single coordinator process that handles both autovacuum and other things, or whether it's better to have two separate processes. Ah, we can separate the proposal to two topics: A. Support to run non-vacuum jobs from autovacuum launcher B. Support user defined background processes A was proposed in the original 1 of 6 patch, but B might be more general. If we have a separated coordinator, B will be required. Markus, do you need B? Or A + standard backend processes are enough? If you need B eventually, starting with B might be better. -- Itagaki Takahiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Packaging of PG 9.0RC1
FYI, we are planning to package Postgres 9.0RC1 tomorrow/Thursday, with release on Monday/Tuesday of next week. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Committers info for the git migration - URGENT!
A.M. age...@themactionfaction.com writes: On Aug 25, 2010, at 6:49 PM, Tom Lane wrote: BTW, I noticed that this list omits several old committers: Julian Assange pr...@suburbia.net That is _the_ Julian Assange who is in the news now. Very cool! Yowza ... *that* Julian Assange? Cool, but maybe we shouldn't advertise the connection ;-) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HS/SR on AIX
On Thu, Aug 26, 2010 at 12:45 AM, Steve Singer ssin...@ca.afilias.info wrote: A clean build from the beta4 source tarball where I'm careful to install into a clean (ie no old beta2 artifacts laying around waiting to be overwritten) isn't reproducing the issue. I'm happy to try other things if people suggest them (or if the original reporter is still getting this after making sure he cleans up old files first) but I'm thinking that was the issue. Thanks for the report! Alanoly, could you do a clean install and try the test again? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
