Re: [HACKERS] Aggregate ORDER BY patch
Andrew Gierth wrote: Herewith a patch to implement agg(foo ORDER BY bar) with or without DISTINCT, etc. What does that mean? Aggregate functions are supposed to be commutative, right? No artificial restrictions are imposed on what syntactical combinations are allowed. However, ORDER BY is not allowed with aggregates used as window functions (as per the existing restriction on DISTINCT). How is this different from window functions? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] IS DISTINCT FROM vs. ANY
Folks, Shouldn't this work and produce a true? SELECT NULL IS NOT DISTINCT FROM ANY(ARRAY['a',NULL]); ERROR: syntax error at or near ANY LINE 1: SELECT NULL IS NOT DISTINCT FROM ANY(ARRAY['a',NULL]); Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, Nov 13, 2009 at 2:15 AM, Bruce Momjian br...@momjian.us wrote: Personally, I would not propose to impose this rule of first-time contributors, or even second-time contributors. But by about patch #3 I think everyone should be pitching in. I hate to ask, but how would we enforce this? Do we no longer apply patches for 3rd-time submitters who have not reviewed? That seems to be hurting us more than them. Are we prepared to discard valid patches for this reason? What about people who contribute hours and hours of their time in other ways? Are they required to contribute even more of their time to review as well, just to help their own occasional code contributions get through the process? (yes, I am thinking largely of me, working tens of hours per week on Postgres, but perhaps submiting one relatively small patch per release) -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] IS DISTINCT FROM vs. ANY
David Fetter wrote: Folks, Shouldn't this work and produce a true? SELECT NULL IS NOT DISTINCT FROM ANY(ARRAY['a',NULL]); ERROR: syntax error at or near ANY LINE 1: SELECT NULL IS NOT DISTINCT FROM ANY(ARRAY['a',NULL]); It should, but probably depends on whether IS NOT DISTINCT should be considered an operator. http://www.postgresql.org/docs/8.4/static/functions-comparisons.html#AEN16561 Got caught by the same thing a couple of days ago. -- Richard Huxton Archonet Ltd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, 2009-11-13 at 08:33 +, Dave Page wrote: On Fri, Nov 13, 2009 at 2:15 AM, Bruce Momjian br...@momjian.us wrote: Personally, I would not propose to impose this rule of first-time contributors, or even second-time contributors. But by about patch #3 I think everyone should be pitching in. I hate to ask, but how would we enforce this? Do we no longer apply patches for 3rd-time submitters who have not reviewed? That seems to be hurting us more than them. Are we prepared to discard valid patches for this reason? What about people who contribute hours and hours of their time in other ways? Are they required to contribute even more of their time to review as well, just to help their own occasional code contributions get through the process? Yes, but the rule needs to be hard for a few reasons. If your employer encourages you to write a patch then they need to be made aware that you must also do all the other things too: write docs, supply tests, explain it *and* include review time. Since reviewing your own patch isn't possible, we must require patch review of other's patches. The rule is deliberately hard on the contributor to ensure the sponsor understands the rule and allows more time. I don't expect it to need to be enforced. If everybody understands it will be enforced and that there will be some polite reminders, then it will likely never need to be enforced. For me, reviewing patches is a great way to learn how to code properly, learn PostgreSQL and get new ideas. All the CF manager needs to do is ensure that every patch submitted chalks up one review. If you think about it, we wouldn't actually need any rr reviewers at all then, because if we have 20 patches we would have 20 reviews due. So the whole scheme is self-balancing. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On tor, 2009-11-12 at 16:06 -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Here's the patch to support Python =3.1 with PL/Python. The compatibility code is mostly in line with the usual 2-3 C porting practice and is documented inline. There was considerable debate earlier about whether we wanted to treat Python 3 as a separate PL so it could be available in parallel with plpython 2, because of the user-level coding incompatibilities. It looks like this patch simply ignores that problem. Exactly how to package that is something to be determined by the packagers, and we can give them the support they need. But first you need code that works with Python 3, which is what this patch does. What is going to happen to plpython functions that depend on 2.x behavior? The porting path from 2 to 3 is pretty well established. You first port to 2.6, then remove all the old features, then move to 3.x. This is not something we have to reinvent. The only significant differences that you can't use in 2.6 without future imports are unicode literals and the print function, both of which are not in common use in PL/Python. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest 2009-09, two weeks on
On Nov 13, 2009, at 8:06 AM, Michael Meskes wrote: On Thu, Nov 12, 2009 at 03:07:27PM -0500, Robert Haas wrote: If you want to submit patches in a series like this one, they need to be considered standalone, I think. The Linux kernel devs work differently than us here. Zoltan broke them up because Michael asked him to do so. Actually these patchsets add different features. I see no reason why they should be done as one patch. However, I haven't had the time to look into the latest ones, but at least that was the situation when I asked Zoltan to split the patch. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: mes...@jabber.org VfL Borussia! Forca Barca! Go SF 49ers! Use: Debian GNU/Linux, PostgreSQL good morning, are there some pending technical issues with those patches or can we basically review and commit? many thanks, hans -- Cybertec Schönig Schönig GmbH Reyergasse 9 / 2 A-2700 Wiener Neustadt Web: www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] write ahead logging in standby (streaming replication)
On Fri, Nov 13, 2009 at 3:17 PM, Greg Smith g...@2ndquadrant.com wrote: Yeah, that's the other parts of the industry I was referring to. MySQL uses semi-synchronous to distinguish between its completely asynchronous default replication mode and one where it provides a somewhat safer implementation. The description reads more as asynchronous with some synchronous elements, not one style of synchronous implementation. None of their documentation wanders into the problem area here by calling it a true synchronous solution when it's really not--MySQL Cluster is their synchronous vehicle. It's fine to adopt the term semi-synchronous, as it's become quite popular and people are going to label the PG implementation with it regardless of what is settled on here. But we should all try to be careful to use it as correctly as possible. OK. Let's think over what recv ACK and fsync ACK synchronization modes should be called later. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Check constraint on domain over an array not executed for array literals
Florian G. Pflug wrote:
While trying to create a domain over an array type to enforce a certain
shape or certain contents of an array (like the array being only
one-dimensional or not containing NULLs), I've stumbled over what I
believe to be a bug in postgresql 8.4
It seems that check constraints on domains are *not* executed for
literals of the domain-over-array-type - in other words, for expressions
like:
array[...]::my-domain-over-array-type.
They are, however, executed if I first force the array to be of the base
type, and then cast it to the array type.
...
I still have the feeling that this a bug, though. First, because it
leaves you with no way at guarantee that values of a given domain always
fulfill certain constraints. And second because array[...]::arraytype
at least *looks* like a cast, and hence should behave like one too.
Agreed, it's a bug. A simpler example is just:
postgres=# create domain myintarray as int[] check (value[1] 10);
CREATE DOMAIN
postgres=# SELECT array['20']::myintarray; -- should fail
array
───
{20}
(1 row)
There's a special case in transformExpr function to handle the
ARRAY[...]::arraytype construct, which skips the usual type-casting
and just constructs an ArrayExpr with the right target type. However,
it's not taking into account that the target type can be a domain.
Attached patch fixes that. Anyone see a problem with it?
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index 04127bd..8ca4a2f 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -169,6 +169,20 @@ transformExpr(ParseState *pstate, Node *expr)
targetType,
elementType,
targetTypmod);
+
+ /*
+ * If the target array type is a domain, we still need
+ * to check the domain constraint. (coerce_to_domain
+ * is a no-op otherwise)
+ */
+ result = coerce_to_domain(result,
+ InvalidOid,
+ -1,
+ targetType,
+ COERCE_IMPLICIT_CAST,
+ tc-location,
+ false,
+ true);
break;
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
On Fri, Nov 13, 2009 at 7:54 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Andrew Gierth wrote: Herewith a patch to implement agg(foo ORDER BY bar) with or without DISTINCT, etc. What does that mean? Aggregate functions are supposed to be commutative, right? We certainly have non-commutative agggregates currently, notably array_agg() I suspect it would be good to have a flag marking the commutative ones -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
Unfortunately with all that payload-length discussion, the other part of my email regarding ACID compliant behavior got completely lost. I would appreciate some input on that part also... Thanks, Joachim On Thu, Nov 12, 2009 at 12:17 PM, Joachim Wieland j...@mcknight.de wrote: On Thu, Nov 12, 2009 at 4:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: One possible solution would be to write to the queue before committing and adding the TransactionID. Then other backends can check if our TransactionID has successfully committed or not. Not sure if this is worth the overhead however... That sounds reasonable, and it's certainly no more overhead than the tuple visibility checks that happen in the current implementation. I am not too concerned about the runtime of the visibility checks, instead I suppose that most of the overhead will come from waiting for another transaction to either commit or abort... If transaction t1 scans the queue and at some point finds notifications from t2, then it will ask for the status of t2. If it finds out that t2 is still running, then it has no other option than to stop working on the queue and wait (it will be signalled again later once t2 has finished). This also means that we cannot at the same time write notifications to the queue and read from it and if a transaction places a few million notifications into the queue, readers need to wait until it has finished and only after that they can continue and read the notifications... And it means that if the queue is full, we might run into a deadlock... A transaction adding notifications will wait for the readers to proceed and the readers wait for the transaction to commit or abort... One option could be to write new notifications to a subdirectory, and create a bunch of new segment files there. Once this is done, the segment files could be moved over and renamed, so that they continue the slru queue... If we run out of disk space while filling that temporary subdirectory, then we can just delete the subdirectory and nobody has been blocked. We could still run into errors moving and renaming the segment files (e.g. permission problems) so that we still might need to abort the transaction... 2. The payload parameter is optional. A notifying client can either call NOTIFY foo; or NOTIFY foo 'payload';. The length of the payload is currently limited to 128 characters... Not sure if we should allow longer payload strings... Might be a good idea to make the max the same as the max length for prepared transaction GUIDs? Not sure anyone would be shipping those around, but it's a pre-existing limit of about the same size. Yes, sounds reasonable to have the same limit for user-defined identifiers... Joachim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git.postgresql.org vs. REL8_1_STABLE
Joshua Tolley eggyk...@gmail.com writes: Am I the only one having problems building 8.1 from git? (Am I the only one building 8.1 from git?) In a clean repository, I've checked out REL8_1_STABLE, configured with only one argument, to set --prefix, and make gives me this: Still does not compile. I trying to compile 8.1 from git://git.postgresql.org/git/postgresql.git and can not do this. git clone git://git.postgresql.org/git/postgresql.git git checkout -b REL8_1_STABLE origin/REL8_1_STABLE ./configure --prefix=$HOME/inst/pg-dev --enable-nls --enable-debug --enable-depend --enable-cassert --enable-thread-safety --with-pgport=5433 --with-python --with-perl (configured ok) make ../../../../src/include/nodes/execnodes.h:23:29: error: nodes/tidbitmap.h: No such file or directory ../../../../src/include/nodes/execnodes.h:934: error: expected specifier-qualifier-list before 'TIDBitmap' ../../../../src/include/nodes/execnodes.h:959: error: expected specifier-qualifier-list before 'TIDBitmap' make[4]: *** [indexvalid.o] Error 1 postgresql-8.1.18.tar.bz2 from www.postgresql.org/ compiled successful. -- Sergey Burladyan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, Nov 13, 2009 at 8:50 AM, Simon Riggs si...@2ndquadrant.com wrote: What about people who contribute hours and hours of their time in other ways? Are they required to contribute even more of their time to review as well, just to help their own occasional code contributions get through the process? Yes, but the rule needs to be hard for a few reasons. If your employer encourages you to write a patch then they need to be made aware that you must also do all the other things too: write docs, supply tests, explain it *and* include review time. Since reviewing your own patch isn't possible, we must require patch review of other's patches. The rule is deliberately hard on the contributor to ensure the sponsor understands the rule and allows more time. Which is fine for sponsored patches, but my employer doesn't sponsor me to hack on PostgreSQL directly - we have other staff that are far more capable of doing that :-). My annual submissions are for my own personal development/enjoyment/whatever, and are never chosen for my employer, but because they seem interesting, are common requests from users, or would help pgAdmin provide improved functionality for users. The bottom line is, if I'm required to review patches, it will impact on other community work, which I am almost certainly more qualified to do and would be more productive at. That said - in general I don't disagree with the idea of everyone chipping in, and I do try to do so myself on appropriate patches (for example, the recent Windows DACL bug fix which I spent a few hours reviewing and testing - and am still waiting for Magnus to commit :-p). I just think that *requiring* people to review will ultimately be counter productive. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
On Fri, Nov 13, 2009 at 1:57 AM, Robert Haas robertmh...@gmail.com wrote: I agree. We frequently reject features on the basis that someone might do something stupid with them. It's lame and counterproductive, and we should stop. The world contains infinite amounts of lameness, but that's the world's problem, not ours. There is zero evidence that this feature is only useful for stupid purposes, and some evidence (namely, the opinions of esteemed community members) that it is useful for at least some non-stupid purposes. This is BS. The problem is not that someone might do something stupid with this feature. The problem is that we're making these other use cases into requirements which will influence the design. This is a classic feature creep situation and the result is normally products which solve none of the use cases especially well. Remember this queue has to live in shared memory which is a fixed size resource. If you're designing a queue mechanism then you would naturally use something like a queue or priority queue. You expect to spill to disk and need an efficient storage mechanism. The natural implementation of this in Postgres would be a table, not the slru. If you're designing a condition-variable mechanism then you would naturally use a hash table which can probably live in a single page with a simple flag for each variable. The comment in another thread that this mechanism should implement ACID properties just reinforces my reaction. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Application name patch - v3
Hi Andres,
On Thu, Nov 12, 2009 at 11:32 PM, Andres Freund and...@anarazel.de wrote:
I had some free time so I started to take a look at that patch:
+ PostgresPollingStatusType
+ pqAppnamePoll(PGconn *conn)
...
+ case APPNAME_STATE_OPTION_WAIT:
...
+ else
+ {
+ /* Query finished, so we're done */
+ conn-setenv_state =
APPNAME_STATE_IDLE;
+ return PGRES_POLLING_OK;
+ }
+ break;
+ }
Shouldnt that set appname_state?
Yup, well spotted.
The attached patch fixes this and also a couple occurances of trailing
whitespace.
Thanks.
What about pg_dump/psql setting fallback_application_name?
Per Tom, I'm waiting on the possible new array-based libpq connect API
which will make a conversion of those utilities from PQsetdbLogin a
lot cleaner than moving to PQconnectdb (and all the ugly connection
string building that would require).
--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On tor, 2009-11-12 at 18:42 -0700, James Pye wrote: For me, plpython has never been what I would call a pleasure to use, and many of the gripes that I have with it are, IMO, entrenched far enough into the implementation that any efforts to change it would(should? =) cause unacceptable breakage in user applications(?). Has this list of gripes ever been brought up and discussed in this forum? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
On fre, 2009-11-13 at 03:16 +, Andrew Gierth wrote: Caveat: as discussed earlier, this patch changes the behaviour of array_agg(DISTINCT x) when applied to NULL inputs. Formerly, the NULLs were unconditionally skipped; now, they are treated just like DISTINCT or GROUP BY normally do. The right answer to that should be in the SQL standard. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest 2009-09, two weeks on
2009/11/13 Hans-Jürgen Schönig h...@cybertec.at: On Nov 13, 2009, at 8:06 AM, Michael Meskes wrote: On Thu, Nov 12, 2009 at 03:07:27PM -0500, Robert Haas wrote: If you want to submit patches in a series like this one, they need to be considered standalone, I think. The Linux kernel devs work differently than us here. Zoltan broke them up because Michael asked him to do so. Actually these patchsets add different features. I see no reason why they should be done as one patch. However, I haven't had the time to look into the latest ones, but at least that was the situation when I asked Zoltan to split the patch. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: mes...@jabber.org VfL Borussia! Forca Barca! Go SF 49ers! Use: Debian GNU/Linux, PostgreSQL good morning, are there some pending technical issues with those patches or can we basically review and commit? *scratches head* How is anyone supposed to answer that question? It is in the process of reviewing them that one decides whether there are any technical issues... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Experimental patch: generating BKI revisited
John Naylor escribió: Hello everyone, I was quite intrigued by a discussion that happened this past summer regarding generation of bootstrap files such as postgres.bki, and the associated pain points of maintaining the DATA() statements in catalog headers. It occurred to me that the current system is backwards: Instead of generating the bootstrap files from hard-coded strings contained in various header files, it seems it would be a cleaner design to generate both from a human-readable high-level description of the system catalogs. I had a look at this some time ago and I must admit that I find it pretty interesting. The technology choices make it obviously impossible to merge -- not only the particular Perl modules used, but the mere fact that Perl is used (and that such a recent version is required). But you're already aware of all this so I'm not going to say more. As far as the data file is concerned, I think having it all in a single file is a loser. I'd go for a file per catalog. Also, I don't like the fact that the column descriptions are lost because of being in a YAML comment. I think it'd be better if the generated pg_foo.h files had them. One thing I loved about this is that it's trivial to add a column to pg_proc and that this not mean that I have to edit almost every single line of the damn monster file. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
On Fri, Nov 13, 2009 at 5:35 AM, Greg Stark gsst...@mit.edu wrote: On Fri, Nov 13, 2009 at 1:57 AM, Robert Haas robertmh...@gmail.com wrote: I agree. We frequently reject features on the basis that someone might do something stupid with them. It's lame and counterproductive, and we should stop. The world contains infinite amounts of lameness, but that's the world's problem, not ours. There is zero evidence that this feature is only useful for stupid purposes, and some evidence (namely, the opinions of esteemed community members) that it is useful for at least some non-stupid purposes. This is BS. The problem is not that someone might do something stupid with this feature. The problem is that we're making these other use cases into requirements which will influence the design. This is a classic feature creep situation and the result is normally products which solve none of the use cases especially well. Removing a length restriction is feature creep? Having an flexible payload mechanism improves on notify in the same way that epoll improves on poll. Yes, epoll is overdesigned, highly overused, etc. but it does vastly improve server handling/responsiveness in some situations. Delivering a notification with data saves a round trip back to the server and a transaction which is both helpful in terms of server load and improving latency. On top of that, I don't think saying: hello; here's some data is groundbreaking in terms of network communication paradigms. My interest in this feature is not academic, the project I'm working on could use it with great benefit immediately. Arguments that I am using notify for the set list of use cases improvised by the original authors are not going to hold much water with me :-). IMNSHO, I don't think that keeping payloads limited to a tiny size 'improves' this feature is a winnable argument. That said, I do appreciate simple designs and very much understand trying to keep things simple. So let me ask you this: *) Are you sure that putting a variable length payload into the slru is going to complicate things that badly in terms of implementing this feature? If so, how? *) Wouldn't you agree that variable length would actually benefit 'proper' (small) payloads by allowing more of them to fit in the slru page? *) 8k should be enough for anybody :-) ...so if a variable length structure can be made why not max the payload length at blcksz-hdrsz and call it a day (yes, I am aware that extending the structure will reduce payload maximum length)? I think this should fit quite nicely into the OP's approach and benefits both people who use small payloads and large ones...(I DO think spanning pages is complex and probably unnecessary) merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, 2009-11-13 at 10:26 +, Dave Page wrote: On Fri, Nov 13, 2009 at 8:50 AM, Simon Riggs si...@2ndquadrant.com wrote: What about people who contribute hours and hours of their time in other ways? Are they required to contribute even more of their time to review as well, just to help their own occasional code contributions get through the process? Yes, but the rule needs to be hard for a few reasons. If your employer encourages you to write a patch then they need to be made aware that you must also do all the other things too: write docs, supply tests, explain it *and* include review time. Since reviewing your own patch isn't possible, we must require patch review of other's patches. The rule is deliberately hard on the contributor to ensure the sponsor understands the rule and allows more time. Which is fine for sponsored patches, but my employer doesn't sponsor me to hack on PostgreSQL directly - we have other staff that are far more capable of doing that :-). My annual submissions are for my own personal development/enjoyment/whatever, and are never chosen for my employer, but because they seem interesting, are common requests from users, or would help pgAdmin provide improved functionality for users. The bottom line is, if I'm required to review patches, it will impact on other community work, which I am almost certainly more qualified to do and would be more productive at. That said - in general I don't disagree with the idea of everyone chipping in, and I do try to do so myself on appropriate patches (for example, the recent Windows DACL bug fix which I spent a few hours reviewing and testing - and am still waiting for Magnus to commit :-p). I just think that *requiring* people to review will ultimately be counter productive. Requiring people to write docs or any other patch submission rules has never been counterproductive. People could easily say, English is not my first language, therefore I skip all comments and docs. But they don't, because we require that, as a hard rule. Nobody has ever said enforcing *those* rules is counter productive. I don't see why adding new requirements would be a problem - especially since they aim to address problems with the flow of patches. Change will always seem strange, but just like commitfests themselves the new way of working has been quickly adopted without much complaint. Bottom line is if we all spend all of our time developing and no time reviewing, then we shouldn't be surprised if there is a review bottleneck. Everybody wants their patches to go through, and the *fastest* way is actually for people to assist with review. We just need an easily understood way of implementing that. The 1:1 suggestion is one way, there may be others. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CTE containing ambiguous columns
Robert Haas wrote: create table animals (id serial primary key, name varchar not null); ... with beings as (select * from animals a1, animals a2) select * from beings where id = 1; Doctor, it hurts when I do this. So stop doing that. Can't you disambiguate it using a column list on beings? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
Joachim Wieland wrote: 1. Instead of placing the queue into shared memory only I propose to create a new subdirectory pg_notify/ and make the queue slru-based, such that we do not risk blocking. Several people here have pointed out that blocking is a true no-go for a new listen/notify implementation. With an slru-based queue we have so much space that blocking is really unlikely even in periods with extreme notify bursts. Regarding performance, the slru-queue is not fsync-ed to disk so most activity would be in the OS file cache memory anyway and most backends will probably work on the same pages most of the time. However more locking overhead is required in comparison to a shared-memory-only implementation. Note, however, that pg_subtrans is also supposed to use the same pages from memory most of the time and it still is a performance bottleneck in some cases, and enlarging NUM_SUBTRANS_BUFFERS is a huge boon. I think holding AsyncCtlLock in exclusive mode for every notify add (which may involve I/O) is going to be a hard hit on scalability. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, Nov 13, 2009 at 1:31 PM, Simon Riggs si...@2ndquadrant.com wrote: Requiring people to write docs or any other patch submission rules has never been counterproductive. People could easily say, English is not my first language, therefore I skip all comments and docs. But they don't, because we require that, as a hard rule. Nobody has ever said enforcing *those* rules is counter productive. Requiring that someone document their own work is very different from requiring that they spend time reviewing someone elses entirely unrelated work, possibly in areas of which they have little or no understanding (which may well be an issue at times). -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest 2009-09, two weeks on
Alvaro Herrera írta: Boszormenyi Zoltan escribió: Alvaro Herrera írta: I have applied this patch after some tinkering. I mainly added support for fetch_args: FORWARD opt_from_in name and BACKWARD opt_from_in name in ecpg.addons which apparently you forgot. Thanks. Your fix is correct if this patch is considered standalone. This means I have to re-post the later patches to fix the reject this fix causes in them. Yeah. BTW I don't think the rest of the pieces in this series make sense to apply separately, because they don't do anything useful by themselves (one of them introduces an unused function, what good is in that?). I think they should be submitted as a single patch. If you want to submit patches in a series like this one, they need to be considered standalone, I think. The Linux kernel devs work differently than us here. Dan started reviewing the dynamic cursorname patch. He looked at it in the original form and he said that he's not familiar with the ECPG code. I have drafted the docs for the generated ECPG grammar (it was applied mainstream by Michael shortly after being posted) and have split this patch in question to help Dan in the review. The patch pieces explain the various problems about the implementation. Is it really *that* apparent that I read too much LKML? :-D -- Bible has answers for everything. Proof: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. (Matthew 5:37) - basics of digital technology. May your kingdom come - superficial description of plate tectonics -- Zoltán Böszörményi Cybertec Schönig Schönig GmbH http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
Robert Haas wrote: I am personally quite tired of reviewing patches for people who don't in turn review mine (or someone's). It makes me feel like not working on this project. If we can solve that problem without implementing a policy of this type, that is good. I would much prefer to run by the honor system rather than having to threaten to drop patches, but only if the honor system actually works. Organizing contributors on a project like this is like herding cats. Threats and penalties are unlikely to be effective. This is essentially a charity where people give in ways that work for them, and you take whatever they have to give. I'm extremely uncomfortable with the idea of a prescriptive system. I've proposed them myself in the past, but I have since come to the realization that it will simply drive people away. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
spill to disk and need an efficient storage mechanism. The natural implementation of this in Postgres would be a table, not the slru. If This is what I think the people's real problem is, the implementation becomes a more complex when including payloads (larger ones even more so). I think its a side-track to discuss queue vs condition variables. Whether a notify is 20 bytes through the network or 8192 bytes doesn't change its design or purpose, only its size. Calling this a creeping feature is quite a leap. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
Simon Riggs wrote: Requiring people to write docs or any other patch submission rules has never been counterproductive. People could easily say, English is not my first language, therefore I skip all comments and docs. But they don't, because we require that, as a hard rule. Nobody has ever said enforcing *those* rules is counter productive. I don't see why adding new requirements would be a problem - especially since they aim to address problems with the flow of patches. Change will always seem strange, but just like commitfests themselves the new way of working has been quickly adopted without much complaint. Bottom line is if we all spend all of our time developing and no time reviewing, then we shouldn't be surprised if there is a review bottleneck. Everybody wants their patches to go through, and the *fastest* way is actually for people to assist with review. We just need an easily understood way of implementing that. The 1:1 suggestion is one way, there may be others. The docs case is a good example. We do ask people to write docs, but I don't think we will reject patches if people don't supply docs. I am not against any of the ideas suggested in this thread --- I am just pointing out we are heading in a very new direction with the _requirements_ mentioned. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, 2009-11-13 at 13:34 +, Dave Page wrote: On Fri, Nov 13, 2009 at 1:31 PM, Simon Riggs si...@2ndquadrant.com wrote: Requiring people to write docs or any other patch submission rules has never been counterproductive. People could easily say, English is not my first language, therefore I skip all comments and docs. But they don't, because we require that, as a hard rule. Nobody has ever said enforcing *those* rules is counter productive. Requiring that someone document their own work is very different from requiring that they spend time reviewing someone elses entirely unrelated work, possibly in areas of which they have little or no understanding (which may well be an issue at times). Of course: one requirement is for docs, the other for review. OTOH they are both additional requirements around submitting a patch. Once people accept that, it will all work. All patches require review. If we have no mechanism for providing review time, then *all* patches will stall. I think it is unfair and unwise to assume that reviewers just turn up as needed. The reason we are having this discussion is they plainly don't. We were worried about Tom getting burnt out by it, now Robert is. I've no problem with arguing against my specific idea for producing more review time, but if there is no alternative proposal then all you are saying is lets not fix the current problem. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
On Fri, Nov 13, 2009 at 1:47 PM, Andrew Chernow a...@esilo.com wrote: This is what I think the people's real problem is, the implementation becomes a more complex when including payloads (larger ones even more so). I think its a side-track to discuss queue vs condition variables. Whether a notify is 20 bytes through the network or 8192 bytes doesn't change its design or purpose, only its size. Calling this a creeping feature is quite a leap. It's true that the real creep is having the payload at all rather than not having it. But having fixed-size data also makes the implementation much simpler as well. One person described stuffing the payload with the primary key of the record being invalidated. This means the requirements have just gone from holding at most some small fixed number of records bounded by the number of tables or other shared data structures to holding a large number of records bounded only by the number of records in their tables which is usually much much larger. Now you're talking about making the payloads variable size, which means you need to do free space management within shared pages to keep track of how much space is free and available for reuse. So we've gone from a simple hash table of fixed size entries containing an oid or name datum where we expect the hash table to fit in memory and a simple lru can handle old pages that aren't part of the working set to something that's going to look a lot like a database table -- it has to handle reusing space in collections of variable size data and scale up to millions of entries. And I note someone else in the thread was suggesting it needed ACID properties which makes space reuse even more complex and will need something like vacuum to implement it. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, 2009-11-13 at 08:46 -0500, Andrew Dunstan wrote: Organizing contributors on a project like this is like herding cats. Threats and penalties are unlikely to be effective. People have spoken against this because of the enforcement issue. If we talk about this like we were suggesting hands be removed, then sure, everybody will be against it. If we enforce it like we do other rules, such as if your patch is late, we bump to next commit fest. That is enforced in a draconian manner but nobody thinks that is bad. If you imagine the suggestion with a softer focus, where we each act in a way that respects the need for an action that needs to happen, then it will work. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs head doesn't pass make check on one of the machines here
Grzegorz Jaśkiewicz wrote: Dnia czwartek 12 listopad 2009 o 22:14:14 Andrew Dunstan napisał(a): Grzegorz Jaskiewicz wrote: consistently fails when compiled on ubuntu 9.10 here (on mini 10v). + ERROR: incompatible library /home/kgrittn/postgresql-8.4.0/src/test/regress/refint.so: version mismatch + DETAIL: Server is version 8.4, library is version 8.3. You might take this as a clue as to what you're doing wrong ... and where abouts is that in the file that I included ? can't see it ? As per Tom's - yes, this laptop has LANG set to UTF8 Polish. Setting it back to EN actually makes this error go away. Now question (probably being asked already), is why isn't regression test set the locales ? Oh. I must have looked at thwe wrong file. I apologise. You diffs were gzipped so I had to extract them. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
Simon Riggs wrote: On Fri, 2009-11-13 at 08:47 -0500, Bruce Momjian wrote: We do ask people to write docs, but I don't think we will reject patches if people don't supply docs. Yes, that is a good example. It's a rule, plain and simple. Nobody gets their spleen removed for breaking it, yet it is still somehow enforced. I find it strange that suggesting a new rule is opposed on the general basis that *any* rule cannot be enforced; surely therefore we cannot have new rules at all, ever? We clearly do have new rules from time to time. So what's wrong with this new rule? Should we update the FAQ to say, enclosing docs with a patch is a rule, but actually its not really and you only suffer mild rebuke if you break it and can therefore be ignored? Well, right now we ask for docs, but if they are not supplied, I think we just write them ourselves. Is a different enforcement method being suggested here? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, 2009-11-13 at 09:31 -0500, Bruce Momjian wrote: Well, right now we ask for docs, but if they are not supplied, I think we just write them ourselves. Is a different enforcement method being suggested here? And we never bump late patches, nor reject them if sent in missing format etc. We enforce all manner of rules. You tell me this is your main function on the project even. Is there now no enforcement of any rule, just because I propose a new one? Address the main question: how will we get more review time? There are many other possible proposals, so please lets hear them. (And how would they be enforced?) -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, Nov 13, 2009 at 8:47 AM, Bruce Momjian br...@momjian.us wrote: The docs case is a good example. We do ask people to write docs, but I don't think we will reject patches if people don't supply docs. I am not against any of the ideas suggested in this thread --- I am just pointing out we are heading in a very new direction with the _requirements_ mentioned. We reject patches for lack of docs all the time. We certainly don't have a policy that the reviewer or committer will write the docs for you if you fail to write them yourself. Sometimes the reviewer or committer will help copy edit, or will revise, but in most cases they won't write them from scratch. Of course, we don't reject such patches PERMANENTLY - people just add the docs and resubmit. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, 2009-11-13 at 08:47 -0500, Bruce Momjian wrote: We do ask people to write docs, but I don't think we will reject patches if people don't supply docs. Yes, that is a good example. It's a rule, plain and simple. Nobody gets their spleen removed for breaking it, yet it is still somehow enforced. I find it strange that suggesting a new rule is opposed on the general basis that *any* rule cannot be enforced; surely therefore we cannot have new rules at all, ever? We clearly do have new rules from time to time. So what's wrong with this new rule? Should we update the FAQ to say, enclosing docs with a patch is a rule, but actually its not really and you only suffer mild rebuke if you break it and can therefore be ignored? -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs head doesn't pass make check on one of the machines here
Dnia czwartek 12 listopad 2009 o 22:14:14 Andrew Dunstan napisał(a): Grzegorz Jaskiewicz wrote: consistently fails when compiled on ubuntu 9.10 here (on mini 10v). + ERROR: incompatible library /home/kgrittn/postgresql-8.4.0/src/test/regress/refint.so: version mismatch + DETAIL: Server is version 8.4, library is version 8.3. You might take this as a clue as to what you're doing wrong ... and where abouts is that in the file that I included ? can't see it ? As per Tom's - yes, this laptop has LANG set to UTF8 Polish. Setting it back to EN actually makes this error go away. Now question (probably being asked already), is why isn't regression test set the locales ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
Robert Haas wrote: On Fri, Nov 13, 2009 at 8:47 AM, Bruce Momjian br...@momjian.us wrote: The docs case is a good example. We do ask people to write docs, but I don't think we will reject patches if people don't supply docs. I am not against any of the ideas suggested in this thread --- I am just pointing out we are heading in a very new direction with the _requirements_ mentioned. We reject patches for lack of docs all the time. We certainly don't have a policy that the reviewer or committer will write the docs for you if you fail to write them yourself. Sometimes the reviewer or committer will help copy edit, or will revise, but in most cases they won't write them from scratch. Of course, we don't reject such patches PERMANENTLY - people just add the docs and resubmit. In that case people are working on their own patches. That's quite different from asking/requiring them to work on somebody else's. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
Calling this a creeping feature is quite a leap. It's true that the real creep is having the payload at all rather than not having it. Not having the payload at all is like santa showing up without his bag of toys. Instead, you have to drive/fly to the north pole where he just came from to get them. One person described stuffing the payload with the primary key of the record being invalidated. This means the requirements have just gone from holding at most some small fixed number of records bounded by the number of tables or other shared data structures to holding a large number of records bounded only by the number of records in their tables which is usually much much larger. Now you're talking about making the payloads variable size, which means you need to do free space management within shared pages to keep track of how much space is free and available for reuse. So we've gone from a simple hash table of fixed size entries containing an oid or name datum where we expect the hash table to fit in memory and a simple lru can handle old pages that aren't part of the working set to something that's going to look a lot like a database table -- it has to handle reusing space in collections of variable size data and scale up to millions of entries. And I note someone else in the thread was suggesting it needed ACID properties which makes space reuse even more complex and will need something like vacuum to implement it. I think the original OP was close. The structure can still be fixed length but maybe we can bump it to 8k (BLCKSZ)? -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
Peter Eisentraut pete...@gmx.net writes: On fre, 2009-11-13 at 03:16 +, Andrew Gierth wrote: Caveat: as discussed earlier, this patch changes the behaviour of array_agg(DISTINCT x) when applied to NULL inputs. Formerly, the NULLs were unconditionally skipped; now, they are treated just like DISTINCT or GROUP BY normally do. The right answer to that should be in the SQL standard. It's not. The standard defines the behavior of certain specific aggregates; it doesn't provide general rules that would apply to user-defined aggregates. In particular, all the standard aggregates are strict and so they just ignore nulls anyhow. The proposed change would only affect the behavior of aggregates with non-strict transition functions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] plperl and inline functions -- first draft
Joshua Tolley wrote: Some items of note include that this makes the regression tests add not only plperl to the test database but also plperlu, which is a new thing. I can't see why this might cause problems, but thought I'd mention it. The tests specifically try to verify that plperl doesn't allow 'use Data::Dumper', and plperlu does. Since Data::Dumper is part of perl core, that seemed safe, but it is another dependency, and perhaps we don't want to do that. If not, is there some other useful way of testing plperlu vs. plperl, and does it really matter? Loading both plperl and plperlu could have problems, as there are some platforms where we can't use them both in the same session, due to some perl oddities. We would need to test this on one such - I don't recall which they are. Config might be a better choice than Data::Dumper. The Perl team or some packagers could drop Data::Dumper some day, but they aren't likely to drop Config. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, Nov 13, 2009 at 8:46 AM, Andrew Dunstan and...@dunslane.net wrote: Robert Haas wrote: I am personally quite tired of reviewing patches for people who don't in turn review mine (or someone's). It makes me feel like not working on this project. If we can solve that problem without implementing a policy of this type, that is good. I would much prefer to run by the honor system rather than having to threaten to drop patches, but only if the honor system actually works. Organizing contributors on a project like this is like herding cats. Threats and penalties are unlikely to be effective. This is essentially a charity where people give in ways that work for them, and you take whatever they have to give. I'm extremely uncomfortable with the idea of a prescriptive system. I've proposed them myself in the past, but I have since come to the realization that it will simply drive people away. I think you're entirely missing the point. If you want to spend large amounts of your time reviewing patches from people who won't in turn review yours, that is *absolutely wonderful*. It would be a huge help to the project, especially because you have the ability not only to review them, but also commit them, an area in which we are hugely strapped for qualified people who have the ability to commit substantial time to the project. I would be more than happy to cut back on managing CommitFests and reviewing patches and focus on writing my own patches and letting you fix them up and commit them. I have several good, interesting ideas for really cool patches that I don't have time to write, and they are posted on the developer Wiki if you'd like to go read them. Personally, I believe that I take more of an interest in other people's patches than the average contributor. I am interested in reviewing them, and I would be willing to put even more work in exchange for a commit bit, but that offer has not yet been forthcoming; maybe some day. I have done a large percentage of the management work for both of the last two CommitFests, during which time I have also reviewed 8 patches. That work was long, hard, difficult, and time-consuming. I wish to get something back for it, and specifically what I want to get back for it is the willingness of other contributors to review my patches when and if I can get enough of a break from running CommitFests and reviewing patches to write them. I don't find that setting that expectation is either unreasonable or unfair, and I'm sorry that you and Bruce apparently feel otherwise. What I would like to know is - assuming you don't want to do it yourself, and you don't want to require other *regular contributors* to do it - then who is going to review MY patches? Keep in mind that this is a problem that *does not apply to you*. You are a committer. If no one reviews your patch, you will eventually go ahead and commit it anyway. If no one reviews my patch, it doesn't go in. In fact, even if someone DOES review it, it doesn't necessarily go in, but at least the odds are better. Please don't sabotage my effort to ensure an adequate supply of reviewers unless you have a competing proposal. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
On Fri, Nov 13, 2009 at 10:00 AM, Andrew Chernow a...@esilo.com wrote: I think the original OP was close. The structure can still be fixed length but maybe we can bump it to 8k (BLCKSZ)? The problem with this (which I basically agree with) is that this will greatly increase the size of the queue for all participants of this feature if they use the payload or not. I think it boils down to this: is there a reasonably effective way of making the payload variable length (now or in the future)? If not, let's compromise and maybe go with a larger size, maybe 256 or 512 bytes. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
* Andrew Dunstan and...@dunslane.net [091113 09:52]: In that case people are working on their own patches. That's quite different from asking/requiring them to work on somebody else's. But is it? Just s/patches/itches/ i.e. The patched code implenting feature $X is their main itch... They scratch that, and massage some other places (like docs and tests) to make their itch more palatable/pleasant for the rest of the community, because the community expects docs, tests, and reviews. a. -- Aidan Van Dyk Create like a god, ai...@highrise.ca command like a king, http://www.highrise.ca/ work like a slave. signature.asc Description: Digital signature
[HACKERS] [Patch] Fix enum type mismatch
Attached patch fixed following warning: ../../../src/include/nodes/parsenodes.h, line 487: warning: enumerator value overflows INT_MAX (2147483647) The reason is clear, enum is int not unsigned. It is short fix, but I'm thinking about enum conversion to #define. We use e.g. in the same file. 60 typedef uint32 AclMode; /* a bitmask of privilege bits */ 61 62 #define ACL_INSERT (10) /* for relations */ 63 #define ACL_SELECT (11) 64 #define ACL_UPDATE (12) 65 #define ACL_DELETE (13) 66 #define ACL_TRUNCATE(14) 67 #define ACL_REFERENCES (15) 68 #define ACL_TRIGGER (16) Zdenek diff -r 68b8827f4738 src/include/nodes/parsenodes.h --- a/src/include/nodes/parsenodes.h Fri Nov 13 11:17:04 2009 + +++ b/src/include/nodes/parsenodes.h Fri Nov 13 16:21:38 2009 +0100 @@ -484,7 +484,7 @@ CREATE_TABLE_LIKE_INDEXES = 1 2, CREATE_TABLE_LIKE_STORAGE = 1 3, CREATE_TABLE_LIKE_COMMENTS = 1 4, - CREATE_TABLE_LIKE_ALL = 0x + CREATE_TABLE_LIKE_ALL = 0x7FFF /* enum is int */ } CreateStmtLikeOption; /* -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
Greg Stark gsst...@mit.edu writes: On Fri, Nov 13, 2009 at 7:54 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Andrew Gierth wrote: Herewith a patch to implement agg(foo ORDER BY bar) with or without DISTINCT, etc. What does that mean? Aggregate functions are supposed to be commutative, right? We certainly have non-commutative agggregates currently, notably array_agg() Right. The fact that none of the standard aggregates are order-sensitive doesn't mean that it's not useful to have user-defined ones that are. Currently we suggest fetching from an ordered sub-select if you want to use an aggregate that is input order sensitive. This patch just provides an alternative (and equally nonstandard) notation for that. I'm not entirely convinced that adding ORDER BY here is a good idea, partly because it goes so far beyond the spec and partly because it's not going to be easily optimizable. But I can see that there is a use-case. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
On fre, 2009-11-13 at 10:01 -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On fre, 2009-11-13 at 03:16 +, Andrew Gierth wrote: Caveat: as discussed earlier, this patch changes the behaviour of array_agg(DISTINCT x) when applied to NULL inputs. Formerly, the NULLs were unconditionally skipped; now, they are treated just like DISTINCT or GROUP BY normally do. The right answer to that should be in the SQL standard. It's not. The standard defines the behavior of certain specific aggregates; it doesn't provide general rules that would apply to user-defined aggregates. But array_agg is in the standard. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
On fre, 2009-11-13 at 10:35 -0500, Tom Lane wrote: I'm not entirely convinced that adding ORDER BY here is a good idea, partly because it goes so far beyond the spec This is exactly the syntax that is in the spec AFAICT. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Check constraint on domain over an array not executed for array literals
Heikki Linnakangas wrote: Agreed, it's a bug. A simpler example is just: [snipped] Will this fix for this be included in 8.4.2 (or .3), or will it have to wait for 8.4 because it changes behavior? There's a special case in transformExpr function to handle the ARRAY[...]::arraytype construct, which skips the usual type-casting and just constructs an ArrayExpr with the right target type. However, it's not taking into account that the target type can be a domain. Attached patch fixes that. Anyone see a problem with it? I'm not familiar with the parser so I can't really judge this. However, I've applied the patch to my development db and it seems to work fine, and fixes the bug. Thanks for the quick response! best regards, Florian Pflug smime.p7s Description: S/MIME Cryptographic Signature
Re: [HACKERS] PL/Python array support
CREATE OR REPLACE FUNCTION incr(stuff int[]) RETURNS int[] AS $$
for x in stuff:
yield x+1
$$
LANGUAGE 'plpythonu';
# select incr(ARRAY[1,2,3]);
ERROR: invalid memory alloc request size 18446744073709551608
CONTEXT: while creating return value
PL/Python function incr
Suppose, it could be fixed by additional check in PLy_function_handler near line
947 :
if (proc-is_setof) {
...
}
else if (PyIter_Check(plrv))
{
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg(returned object should be iterated),
errdetail(PL/Python returns iterable object in non-setof returning
context)));
}
Peter Eisentraut wrote:
On ons, 2009-11-04 at 16:02 +0200, Peter Eisentraut wrote:
Here is a patch to support arrays in PL/Python as parameters and
return values.
Slightly updated version with fixed reference counting.
--
Teodor Sigaev E-mail: teo...@sigaev.ru
WWW: http://www.sigaev.ru/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, Nov 13, 2009 at 8:34 AM, Dave Page dp...@pgadmin.org wrote: On Fri, Nov 13, 2009 at 1:31 PM, Simon Riggs si...@2ndquadrant.com wrote: Requiring people to write docs or any other patch submission rules has never been counterproductive. People could easily say, English is not my first language, therefore I skip all comments and docs. But they don't, because we require that, as a hard rule. Nobody has ever said enforcing *those* rules is counter productive. Requiring that someone document their own work is very different from requiring that they spend time reviewing someone elses entirely unrelated work, possibly in areas of which they have little or no understanding (which may well be an issue at times). I think this is overstating the level of contribution that is being asked, at least by me. Every CommitFest is full of a bunch of little patches that make small changes and need to be reviewed. Anyone who is reasonably familiar with C (and most of our regular contributors are) can take a little time to understand what one of those patches is doing and check it for style and functionality. I can usually review one of those patches in about 2 hours knowing nothing about that area of the system. What I object to about the present system is that the last two CommitFests have gone like this, for me: 1. First, I assign reviewers to as many patches as we have reviewers for. 2. Then, I ask the reviewers who haven't done so to actually complete the reviews. 3. Then, I ask the patch submitters who haven't done so to update their patches, and/or bounce the patches for lack of an update. 4. Then, I ask the reviewers who don't do it without prompting to check over the revised patches. ...at this point we are about 2 weeks into the CommitFest... 5. Then, I try to find reviewers for the patches that haven't been reviewed yet and start steps 1-4 over with the remaining patches. 6. Concurrently, I review several patches myself. Step #5 is the one that is really irritating to me. If there are 45 patches in the CommitFest and 15 to 20 people who didn't submit anything sign up to review and the committers take 7 or 8 patches directly, why is the number of reviewers still 10 or 15 less than the number of patches? My diagnosis is that the people who submit 2 or 3 or 4 or 5 patches to the CommitFest think to themselves well, if I sign up to review, I might not have time to update all of these patches and get them committed during this CommitFest, so I'll let someone else do it. As far as I can see, this is the exact opposite of how the process is supposed to work: during the CommitFest, people are supposed to stop working on their own patches and review patches belonging to other people. As it is, the people who are actually willing to review are getting asked to review multiple patches so that other people can review none at all. Some people are willing to do that, and that is fine, but many are not, and even for the ones who are, after a certain point, it seems unfair to ask it. To put this another way, if everyone who submitted a patch reviewed a patch, we could finish up each CommitFest in 2-3 weeks instead of a whole month, except that the committing would drag out for the rest of the month unless someone other than Tom is willing to help to a greater degree than in the most recent CommitFest. That would substantially increase the time available for everyone to work on their own patches. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CTE containing ambiguous columns
On Fri, Nov 13, 2009 at 8:32 AM, Andrew Dunstan and...@dunslane.net wrote: Robert Haas wrote: create table animals (id serial primary key, name varchar not null); ... with beings as (select * from animals a1, animals a2) select * from beings where id = 1; Doctor, it hurts when I do this. So stop doing that. Can't you disambiguate it using a column list on beings? Sure, after I figured out what the real problem was. Maybe I'm a dope, but when I get an error cursor pointed at an ambiguous column reference, my thought is oh, I need to qualify that reference - not oh, some completely unrelated part of the query has an *-expansion that contains duplicate columns. Something like: HINT: alias contains multiple columns named colname ...would help a lot. I don't feel strongly about it, I just thought it was confusing. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Experimental patch: generating BKI revisited
On Fri, Nov 13, 2009 at 8:16 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: I had a look at this some time ago and I must admit that I find it pretty interesting. The technology choices make it obviously impossible to merge -- not only the particular Perl modules used, but the mere fact that Perl is used (and that such a recent version is required). But you're already aware of all this so I'm not going to say more. Really? Perl is a show-stopper? That seems harsh. Perl is pretty ubiquitous - in fact, given that we now support Windows, arguably more so than awk. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
Simon Riggs si...@2ndquadrant.com writes: All the CF manager needs to do is ensure that every patch submitted chalks up one review. If you think about it, we wouldn't actually need any rr reviewers at all then, because if we have 20 patches we would have 20 reviews due. So the whole scheme is self-balancing. Well, no, that's *far* too optimistic/simplistic, because it imagines that every review is worth the same. What we lack is not just review time but qualified review time, ie, comments from someone who's already familiar with the portion of the code base that's being patched. Now when the current reviewing process was proposed, there were two separate goals in mind. One was to take whatever incremental load we could off the eventual committer's work, by catching obvious mistakes, making sure the docs were up to snuff, etc. The other was the idea that reviewing would in itself improve the skills of our development community, by making people read code that they wouldn't have read otherwise, and that eventually we'd have more committer-grade people just because of all the reviewing they'd done. (The jury is still out on whether that will work, but in any case it's a long-term project.) The problem at the moment seems to not be lack of first-level review time but lack of qualified review. I don't know what we do about that. Requiring people who have submitted one or two patches to do reviews isn't going to produce more of the latter, it's going to produce more of the former. Especially if the patches available to be reviewed are working in areas they haven't looked at before. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Experimental patch: generating BKI revisited
On Fri, Nov 13, 2009 at 3:54 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Nov 13, 2009 at 8:16 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: I had a look at this some time ago and I must admit that I find it pretty interesting. The technology choices make it obviously impossible to merge -- not only the particular Perl modules used, but the mere fact that Perl is used (and that such a recent version is required). But you're already aware of all this so I'm not going to say more. Really? Perl is a show-stopper? That seems harsh. Perl is pretty ubiquitous - in fact, given that we now support Windows, arguably more so than awk. Building in VC++ on Windows already requires Perl. And if you're building in mingw, you've probably already got it, or can get it pretty easily. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
Merlin Moncure mmonc...@gmail.com writes: The problem with this (which I basically agree with) is that this will greatly increase the size of the queue for all participants of this feature if they use the payload or not. I think it boils down to this: is there a reasonably effective way of making the payload variable length (now or in the future)? If not, let's compromise and maybe go with a larger size, maybe 256 or 512 bytes. Yeah, if the payload is not variable length then we are not going to be able to make it more than a couple hundred bytes without taking a significant performance hit. (By the way, has anyone yet tried to compare the speed of this implementation to the old code?) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, Nov 13, 2009 at 10:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: All the CF manager needs to do is ensure that every patch submitted chalks up one review. If you think about it, we wouldn't actually need any rr reviewers at all then, because if we have 20 patches we would have 20 reviews due. So the whole scheme is self-balancing. Well, no, that's *far* too optimistic/simplistic, because it imagines that every review is worth the same. What we lack is not just review time but qualified review time, ie, comments from someone who's already familiar with the portion of the code base that's being patched. Right, but I think we're more likely to find such people among the pool of existing contributors than we are among people who don't write patches themselves but happen to volunteer to review. I think Simon's idea of requiring 1 review per patch probably IS a bit overly simplistic - for one thing, someone who submits 10 patches, as I did in the July CommitFest, can scarcely be expected to also review 10 patches. (Even if they were willing, it would make the CommitFest longer, not shorter.) But I don't think they should get by reviewing none, either, especially if they're submitting patches to every CommitFest. It's not my idea that we should punish someone like Dave Page who does a lot of PostgreSQL work and occasionally writes a patch. What I'm complaining about is people who submit patches regularly and rarely or never review. We have enough volunteers to cover new and occasional patch submitters; sometimes those reviews are not quite as thorough, but new and occasional contributors tend to submit relatively simple patches anyway, so it's not a catastrophe. It's the regular patch submitters who, IMHO, most need to be involved. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Check constraint on domain over an array not executed for array literals
Florian G. Pflug wrote: Heikki Linnakangas wrote: Agreed, it's a bug. A simpler example is just: [snipped] Will this fix for this be included in 8.4.2 (or .3), or will it have to wait for 8.4 because it changes behavior? It's a regression; 8.3 and earlier used to check the domain constraint correctly. I just committed it to CVS HEAD and REL8_4_STABLE. Thanks for the report! -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Experimental patch: generating BKI revisited
On Fri, Nov 13, 2009 at 11:00 AM, Dave Page dp...@pgadmin.org wrote: On Fri, Nov 13, 2009 at 3:54 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Nov 13, 2009 at 8:16 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: I had a look at this some time ago and I must admit that I find it pretty interesting. The technology choices make it obviously impossible to merge -- not only the particular Perl modules used, but the mere fact that Perl is used (and that such a recent version is required). But you're already aware of all this so I'm not going to say more. Really? Perl is a show-stopper? That seems harsh. Perl is pretty ubiquitous - in fact, given that we now support Windows, arguably more so than awk. Building in VC++ on Windows already requires Perl. And if you're building in mingw, you've probably already got it, or can get it pretty easily. Yep, it's only on UNIX-ish systems where Perl isn't necessarily required, and realistically I think it is probably present on nearly all of those, too. We can even include the autogenerated files in make distprep so that it's only required for development, not to compile a release tarball. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
On Fri, Nov 13, 2009 at 10:35 AM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark gsst...@mit.edu writes: On Fri, Nov 13, 2009 at 7:54 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Andrew Gierth wrote: Herewith a patch to implement agg(foo ORDER BY bar) with or without DISTINCT, etc. What does that mean? Aggregate functions are supposed to be commutative, right? We certainly have non-commutative agggregates currently, notably array_agg() Right. The fact that none of the standard aggregates are order-sensitive doesn't mean that it's not useful to have user-defined ones that are. Currently we suggest fetching from an ordered sub-select if you want to use an aggregate that is input order sensitive. This patch just provides an alternative (and equally nonstandard) notation for that. I'm not entirely convinced that adding ORDER BY here is a good idea, partly because it goes so far beyond the spec and partly because it's not going to be easily optimizable. But I can see that there is a use-case. Yeah, for sure. I currently handle this, when necessary, by using subselects, but it would sure be nice to have a more compact notation, if there's a good way to do that. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Experimental patch: generating BKI revisited
On Fri, Nov 13, 2009 at 4:12 PM, Robert Haas robertmh...@gmail.com wrote: Yep, it's only on UNIX-ish systems where Perl isn't necessarily required, and realistically I think it is probably present on nearly all of those, too. Exactly. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
Merlin Moncure wrote: On Fri, Nov 13, 2009 at 10:00 AM, Andrew Chernow a...@esilo.com wrote: I think the original OP was close. The structure can still be fixed length but maybe we can bump it to 8k (BLCKSZ)? The problem with this (which I basically agree with) is that this will greatly increase the size of the queue for all participants of this feature if they use the payload or not. I think it boils down to this: is there a reasonably effective way of making the payload variable length (now or in the future)? If not, let's compromise and maybe go with a larger size, maybe 256 or 512 bytes. My original intention was to have the queue as a circular buffer where the size of the entries was variable, but possibly bounded. Certainly using fixed length records of large size seems somewhat wasteful. But maybe that doesn't fit with what Joachim has done. Incidentally, I'd like to thank Joachim personally for having done this work, that I have been trying to get to for a couple of years, and that circumstances kept conspiring to prevent me from doing. It's been a big monkey on my back. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Check constraint on domain over an array not executed for array literals
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Florian G. Pflug wrote: It seems that check constraints on domains are *not* executed for literals of the domain-over-array-type - in other words, for expressions like: array[...]::my-domain-over-array-type. There's a special case in transformExpr function to handle the ARRAY[...]::arraytype construct, which skips the usual type-casting and just constructs an ArrayExpr with the right target type. However, it's not taking into account that the target type can be a domain. Attached patch fixes that. Anyone see a problem with it? Hm. I concur that this special-case code is failing to consider the possibility that the target type is domain-over-array-type rather than just array-type. I think though that this patch is a bit of a kluge, because it delivers a mislabeled expression tree. The result of the transformArrayExpr() is not really of type myintarray. What it is is a plain int[] value; we shouldn't label it as being already myintarray, because it hasn't passed the domain checks. At the moment there is probably not any visible effect of that, but in the future it could lead to misoptimization, so I think it's important to get it right. My inclination is to apply getBaseTypeAndTypmod to the targetType and pass that as the array type to transformArrayExpr, then instead of considering the job done, fall through to transformTypeCast, which will either do nothing or attach a domain coercion node. I'm not sure about the typmod handling (need more caffeine to work that out). Do you want to have another go at it, or shall I? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
Heikki == Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Herewith a patch to implement agg(foo ORDER BY bar) with or without DISTINCT, etc. Heikki What does that mean? Aggregate functions are supposed to be Heikki commutative, right? The SQL spec defines two non-commutative aggregates that we implement: array_agg(x ORDER BY ...) xmlagg(x ORDER BY ...) In addition, of course, we allow user-defined aggregates, which are perfectly free to be non-commutative. -- Andrew (irc:RhodiumToad) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
Peter == Peter Eisentraut pete...@gmx.net writes: I'm not entirely convinced that adding ORDER BY here is a good idea, partly because it goes so far beyond the spec Peter This is exactly the syntax that is in the spec AFAICT. Right. The spec defines this syntax for array_agg and xmlagg (only). The patch goes beyond the spec in that it allows ORDER BY for any aggregate at all, and also allows combination of ORDER BY and DISTINCT (the spec only allows DISTINCT with general set operation which doesn't include array_agg or xmlagg, so there is nowhere that both are allowed). But it would be entirely unreasonable, the way postgres works, to implement ORDER BY for only specific aggregates. (Note also that combining ORDER BY and DISTINCT can change the behaviour of DISTINCT. e.g. foo(distinct x order by x using ) will use whatever definition of equality is implied by the hypothetical operator when comparing values of x for distinctness.) As for the null handling, the spec is no help there for this reason: it allows DISTINCT only for general set operation, and for all general set operations whether DISTINCT or not, NULLs are always discarded from the input before processing (i.e. the behaviour we implement for strict aggregates). So even the fact that we allow array_agg(distinct x) at all is going beyond the spec and therefore doesn't have a defined result. -- Andrew (irc:RhodiumToad) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Experimental patch: generating BKI revisited
Dave Page dp...@pgadmin.org writes: On Fri, Nov 13, 2009 at 4:12 PM, Robert Haas robertmh...@gmail.com wrote: Yep, it's only on UNIX-ish systems where Perl isn't necessarily required, and realistically I think it is probably present on nearly all of those, too. Exactly. Yeah. Although the project policy is that we don't require Perl to build on Unix, there was a bug in the makefiles that made it effectively required, and nobody noticed for several years. I don't think it would be a hard sell to change that policy if we got a significant benefit out of it. (Depending on non-core Perl modules is a totally different thing though.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
Andrew Gierth and...@tao11.riddles.org.uk writes: Peter == Peter Eisentraut pete...@gmx.net writes: Peter This is exactly the syntax that is in the spec AFAICT. Right. The spec defines this syntax for array_agg and xmlagg (only). Cool, I had forgotten that they added that in the latest revisions. I withdraw the complaint that this patch goes too far beyond the spec. But it would be entirely unreasonable, the way postgres works, to implement ORDER BY for only specific aggregates. Quite. This is another instance of the thing I complained of before, that the SQL committee likes to define the behavior of specific aggregates instead of inducing a generic aggregate-behavior definition. So we're on our own to extract one, and this proposal seems pretty reasonable to me: it's useful and it's consistent with the query-level behavior of DISTINCT and ORDER BY. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
Robert Haas wrote: On Fri, Nov 13, 2009 at 10:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: All the CF manager needs to do is ensure that every patch submitted chalks up one review. If you think about it, we wouldn't actually need any rr reviewers at all then, because if we have 20 patches we would have 20 reviews due. So the whole scheme is self-balancing. Well, no, that's *far* too optimistic/simplistic, because it imagines that every review is worth the same. What we lack is not just review time but qualified review time, ie, comments from someone who's already familiar with the portion of the code base that's being patched. Right, but I think we're more likely to find such people among the pool of existing contributors than we are among people who don't write patches themselves but happen to volunteer to review. I think Simon's idea of requiring 1 review per patch probably IS a bit overly simplistic - for one thing, someone who submits 10 patches, as I did in the July CommitFest, can scarcely be expected to also review 10 patches. (Even if they were willing, it would make the CommitFest longer, not shorter.) But I don't think they should get by reviewing none, either, especially if they're submitting patches to every CommitFest. It's not my idea that we should punish someone like Dave Page who does a lot of PostgreSQL work and occasionally writes a patch. What I'm complaining about is people who submit patches regularly and rarely or never review. We have enough volunteers to cover new and occasional patch submitters; sometimes those reviews are not quite as thorough, but new and occasional contributors tend to submit relatively simple patches anyway, so it's not a catastrophe. It's the regular patch submitters who, IMHO, most need to be involved. I think we (the commitfest manager?) should simply send polite message to any regulars who submits patches but hasn't volunteered for review. Along the lines of: Hi XXX. You've submitted a patch to the commitfest. As you know, this is a community process and the it depends on volunteers like you. It would be extremely helpful if you could pick a patch that interests you from the list at commitfest.postgresql.org, mark yourself as a reviewer, and review and/or test it as thoroughly as you can. Remember that the faster other patches are reviewed, the faster others get to review your patch and the faster the commitfest can be closed and the version can be released. The commitfest manager can apply common sense: if there's plenty of reviewers already and not many patches, there's no need to reach out to more people. OTOH, if there's a shortage, he can e.g. go through old commitfests too and beg people who have contributed in the past. Think of a fundraiser who calls around people, begging for donations. This is the same, except that we're begging for people's time instead of money. Or think of donating blood. If there's a shortage of blood of a certain type, they will send emails to past donors, asking to come donate. I agree with Tom though that we don't really need a huge pool of people who chip in with one hour per month. We need people who know the codebase pretty well, and who can spend a fair amount of time to do thorough review of complex patches. There is a few dozen or so people out there who have enough knowledge on various parts of the system, or are good at writing documentation, or good at testing. The question is how to get them to donate more time. I'm proposing that we ask them to. It can't hurt. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Check constraint on domain over an array not executed for array literals
Tom Lane wrote: Hm. I concur that this special-case code is failing to consider the possibility that the target type is domain-over-array-type rather than just array-type. I think though that this patch is a bit of a kluge, because it delivers a mislabeled expression tree. The result of the transformArrayExpr() is not really of type myintarray. What it is is a plain int[] value; we shouldn't label it as being already myintarray, because it hasn't passed the domain checks. At the moment there is probably not any visible effect of that, but in the future it could lead to misoptimization, so I think it's important to get it right. My inclination is to apply getBaseTypeAndTypmod to the targetType and pass that as the array type to transformArrayExpr, then instead of considering the job done, fall through to transformTypeCast, which will either do nothing or attach a domain coercion node. Hmm, yeah that's more accurate. I'm not sure about the typmod handling (need more caffeine to work that out). Seems straightforward to me. The ArrayExpr should have the typmod of the base type, as returned by getBaseTypeAndTypmod, and domains don't have typmods. Do you want to have another go at it, or shall I? I'll give it a shot. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
On Friday 13 November 2009 16:35:08 Tom Lane wrote: Greg Stark gsst...@mit.edu writes: On Fri, Nov 13, 2009 at 7:54 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Andrew Gierth wrote: Herewith a patch to implement agg(foo ORDER BY bar) with or without DISTINCT, etc. What does that mean? Aggregate functions are supposed to be commutative, right? We certainly have non-commutative agggregates currently, notably array_agg() Right. The fact that none of the standard aggregates are order-sensitive doesn't mean that it's not useful to have user-defined ones that are. Currently we suggest fetching from an ordered sub-select if you want to use an aggregate that is input order sensitive. This patch just provides an alternative (and equally nonstandard) notation for that. I'm not entirely convinced that adding ORDER BY here is a good idea, partly because it goes so far beyond the spec and partly because it's not going to be easily optimizable. But I can see that there is a use-case. The spec supports the ORDER BY syntax for the xmlagg aggregate... Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
Simon Riggs wrote: On Fri, 2009-11-13 at 09:31 -0500, Bruce Momjian wrote: Well, right now we ask for docs, but if they are not supplied, I think we just write them ourselves. Is a different enforcement method being suggested here? And we never bump late patches, nor reject them if sent in missing format etc. We enforce all manner of rules. You tell me this is your main function on the project even. Is there now no enforcement of any rule, just because I propose a new one? Enforcing civil behavior in the group, in a private way, is different from having a public policy that requires patch review. Again, I am not against this change --- I am just pointing out it is new territory for us. Address the main question: how will we get more review time? There are many other possible proposals, so please lets hear them. (And how would they be enforced?) Not sure, but I would like to point out the bottleneck is not currently the reviewers but the ability to give final approval and commit it. And again, the commit process is very fast --- it is that final approval that is hard. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Inspection of row types in pl/pgsql and pl/sql
Hi I'm currently working on a project where we need to build a global cache table containing all values of certain types found in any of the other tables. Currently, a seperate insert, update and delete (plpgsql) trigger function exists for each table in the database which is auto-generated by a (plpgsql) function which queries the system catalogs to find all fields with a certain type, and then generates the appropriate plpgsql function using EXECUTE '...'. I'd like to replace this function-generating function by a generic trigger function that works for all tables. Due to the lack of any way to inspect the *structure* of a record type, however, I'd have to use a C language function for that, which induces quite some maintenance headaches (especially if deployed on windows). I'm therefore thinking about implementing the following generate-purpose inspection functions for row types record_length(record) returns smallint Returns the number of fields in the given record. record_names(record) returns name[] Returns the names of the record's fields. Array will contain NULLs if one or more fields are unnamed. record_types(record) returns regtype[]; Returns the OIDs of the record's types. Array won't contain NULLs record_value(record, name, anyelement) returns anyelement Returns the value of a certain (named) field. The type of the third argument defines the return type (its value is ignored). The field's value is cast to that type if possible, otherwise an error is raised. record_value(record, smallint, anyelement) returns anyelement Returns the value of the field at the given position. record_values(record, regtype, anyelement) returns anyarray Returns an array of all values of all fields with the given type or whose type is a domain over the given type. No casting is done. Any comment/critique is appreciated. Would anyone else find those functions useful? best regards, Florian Pflug smime.p7s Description: S/MIME Cryptographic Signature
Re: [HACKERS] next CommitFest
2009/11/14 Heikki Linnakangas heikki.linnakan...@enterprisedb.com: I think we (the commitfest manager?) should simply send polite message to any regulars who submits patches but hasn't volunteered for review. Along the lines of: I certainly endorse Heikki's suggestion, but I wonder if we can do more to make reviewing patches an attractive option. As Tom notes, reviewing somebody else's patch just isn't as much fun as working on your own. Robert notes that reviewing patches is a great way to learn about the codebase, and I concur. But in terms of hacker satisfaction it just doesn't compare to creating something, and I think the ratios of submitters to reviewers in recent CFs vindicate me here. I'm thinking of something like a Reviewer Hall of Fame, or Honour Roll. During and after a commitfest, it shows how many reviews have been completed by each person [1]. This could be included in the Weekly News at the CF's conclusion. One of the things that people get out of contributing to an OSS project is the recognition of their peers. Well then, let's leverage that by acclaiming the people who put in a lot of effort reviewing, loudly and publicly. The louder and more public, the more powerful the incentive. Cheers, BJ [1] perhaps with some subjective weighting with respect to patch complexity / depth of review. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 This is BS. The problem is not that someone might do something stupid with this feature. The problem is that we're making these other use cases into requirements which will influence the design. This is a classic feature creep situation and the result is normally products which solve none of the use cases especially well. Feature creep? The payload has been on the roadmap for a long time. I don't recall anyone objecting when Andrew was working on the next version of Listen/Notify around what is probably a couple of years ago now. Remember this queue has to live in shared memory which is a fixed size resource. If you're designing a queue mechanism then you would naturally use something like a queue or priority queue. Right, but we're not discussing a queue, we're discussing the listen/notify system. If people want to mis-use it as a queue when they should be using something else, so be it. Talk of efficiency also seems silly here - using shared memory is already way more efficient than our current listen/notify system. - -- Greg Sabino Mullane g...@turnstep.com PGP Key: 0x14964AC8 200911131234 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAkr9mL0ACgkQvJuQZxSWSshkvACg6OQ/SRjkvmozzUogTX3weuio 4ZoAnRVfvcrdMmo+iKtkyXmhAlZqViqF =6fzv -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
Robert Haas wrote: Please don't sabotage my effort to ensure an adequate supply of reviewers unless you have a competing proposal. I don't think you can reasonably demand this. If I don't think your suggestion is going to improve matters I have a right to say so. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, Nov 13, 2009 at 12:18 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I agree with Tom though that we don't really need a huge pool of people who chip in with one hour per month. We need people who know the codebase pretty well, and who can spend a fair amount of time to do thorough review of complex patches. There is a few dozen or so people out there who have enough knowledge on various parts of the system, or are good at writing documentation, or good at testing. The question is how to get them to donate more time. I'm proposing that we ask them to. It can't hurt. I agree. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, Nov 13, 2009 at 12:32 PM, Brendan Jurd dire...@gmail.com wrote: I'm thinking of something like a Reviewer Hall of Fame, or Honour Roll. During and after a commitfest, it shows how many reviews have been completed by each person [1]. This could be included in the Weekly News at the CF's conclusion. One of the things that people get out of contributing to an OSS project is the recognition of their peers. Well then, let's leverage that by acclaiming the people who put in a lot of effort reviewing, loudly and publicly. The louder and more public, the more powerful the incentive. I think this would be a nice thing to do. +1 for considering complexity of patches in making this determination. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inspection of row types in pl/pgsql and pl/sql
Florian G. Pflug f...@phlo.org writes: I'd like to replace this function-generating function by a generic trigger function that works for all tables. Due to the lack of any way to inspect the *structure* of a record type, however, I'd have to use a C language function for that, which induces quite some maintenance headaches (especially if deployed on windows). Trying to do this in plpgsql is doomed to failure and heartache, because it's fundamentally a strongly typed language. The proposed functions won't fix that and hence will be unusable in practice. I'd suggest either using C, or using one of the less-strongly-typed PLs. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
Greg Sabino Mullane g...@turnstep.com writes: Talk of efficiency also seems silly here - using shared memory is already way more efficient than our current listen/notify system. Except that the proposed implementation spills to disk. Particularly if it has to have support for large payloads, it could very well end up being a lot SLOWER than what we have now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
Simon Riggs wrote: All the CF manager needs to do is ensure that every patch submitted chalks up one review. If you think about it, we wouldn't actually need any rr reviewers at all then, because if we have 20 patches we would have 20 reviews due. So the whole scheme is self-balancing In fact, just suggesting the guideline that everyone who submits a patch should review one here was sufficient to pull in a number of submitters who volunteered to do a single review as well, moving some distance toward what you're describing. It seems we had a perception here that joining rrreviewers subscribed you to doing multiple patch reviews; I've let multiple submitters who were trying to help out know it's OK to just grab one patch and review without even getting involved on that list. Take a look at https://commitfest.postgresql.org/action/commitfest_view?id=4 right now. I've been suggesting to people that they assign themselves to the patches they like, and it's nearing completely populated two days before the CommitFest has even started. I have 6 reviewers that haven't been assigned anything yet and there are only 8 unassigned patches out there. In several cases, assigning the reviewer turned out to be quite easy because so many submitters joined in--just assign someone who submitted a patch in the same area. So it far it looks sufficient to introduce the expectation that submitters should also do a review, without even needing to make that a firm rule. That helps increase the reviewer pool significantly, addressing the general problem Robert has been fighting, while not forcing people like Dave who have other pulls on their time into a review role. We'll see whether the follow-through here is good or not, maybe this will decay yet. For now, simply telling submitters that the review of their own patches might be influenced by whether they do a good job reviewing someone else's has improved things considerably over past CommitFests, and it's hard to imagine how someone could complain about a guideline that fair. The most difficult part here remains finding reviewers for the really big patches. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs head doesn't pass make check on one of the machines here
On fre, 2009-11-13 at 15:05 +0100, Grzegorz Jaśkiewicz wrote: As per Tom's - yes, this laptop has LANG set to UTF8 Polish. Setting it back to EN actually makes this error go away. The Polish locale isn't supported by the regression tests. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
Heikki == Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: No artificial restrictions are imposed on what syntactical combinations are allowed. However, ORDER BY is not allowed with aggregates used as window functions (as per the existing restriction on DISTINCT). Heikki How is this different from window functions? Window functions return a row for each row of input, aggregates don't. The reason I didn't tackle the case of aggregate functions used as window functions is that the spec allows constructs like this: array_agg(a order by b) over (order by c) which can't be represented using the aggregate-as-window-function mechanism as it currently stands, since you'd have to re-sort the window each time. -- Andrew (irc:RhodiumToad) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, Nov 13, 2009 at 12:37 PM, Andrew Dunstan and...@dunslane.net wrote: Robert Haas wrote: Please don't sabotage my effort to ensure an adequate supply of reviewers unless you have a competing proposal. I don't think you can reasonably demand this. If I don't think your suggestion is going to improve matters I have a right to say so. I've never disputed the right of you or anyone else to say whatever they like. Just to be clear, I don't think that mandating reviews is the best idea anyone has ever had, and I don't rule out the possibility that in solving one problem it might create some others. I think those problems are likely solvable, but I might be wrong, and in any event, it's clearly better for it to be a voluntary system. As far as I can tell, the major objection to having it be mandatory is that it might drive some people away. My major argument for why that isn't the case is that the mere fact that we are even *discussing* whether it should be mandatory has led to a bumper crop of reviewers, including several of the people who fall into the category I've been discussing. So maybe we don't need to make it mandatory: maybe we just need to discuss making it mandatory every 6 months or so. :-) Anyhow, as Bruce pointed out on another message, in some sense we are getting sidetracked. Good reviewers opting out of the system *is* a problem, but lack of a sufficient number of sufficiently involved committers is a bigger one. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
Tom Lane wrote: Greg Sabino Mullane g...@turnstep.com writes: Talk of efficiency also seems silly here - using shared memory is already way more efficient than our current listen/notify system. Except that the proposed implementation spills to disk. Particularly if it has to have support for large payloads, it could very well end up being a lot SLOWER than what we have now. True, but do you really consider it to be a common case that the notify system gets soo bogged down that it starts to crawl? The problem would be the collective size of notify structures + payloads and whether that would fit in memory or not. This leads me to believe that the only safety in smaller payloads is *possibly* a smaller chance of bogging it down, but that all depends on the usage pattern of smaller vs. larger payloads which is system specific. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
2009/11/14 Andrew Gierth and...@tao11.riddles.org.uk: Heikki == Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: No artificial restrictions are imposed on what syntactical combinations are allowed. However, ORDER BY is not allowed with aggregates used as window functions (as per the existing restriction on DISTINCT). Heikki How is this different from window functions? Window functions return a row for each row of input, aggregates don't. The reason I didn't tackle the case of aggregate functions used as window functions is that the spec allows constructs like this: array_agg(a order by b) over (order by c) which can't be represented using the aggregate-as-window-function mechanism as it currently stands, since you'd have to re-sort the window each time. Now I'm about to send my patch to introduce more frame types, aggregate cache mechanism in window functions may be broken sometimes, and it is *possible* to put order-by clause in argument list if we prepare tuplesort as in nodeAgg. But I don't see useful cases and it seems so hard task that I'm not sold. Regards, -- Hitoshi Harada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On Nov 13, 2009, at 4:47 AM, Peter Eisentraut wrote: Has this list of gripes ever been brought up and discussed in this forum? Some are TODOs, so in part by other people. Some were briefly touched on in the recent past discussions(around the time that I announced the WIP). Native typing vs conversion, function fragments vs function modules. I don't think native typing has seen any actual discussion, but I do recall mentioning it as something that I wanted to do(implicitly griped?). ... There is a difference in the situation from the discussion before. Prior, it was, I would like to implement a new PL for Python 3 with these features, and now, it is, I have implemented a new PL for Python 3 with these features. Simply, -hackers can choose among moving forward with Python 3 support in plpython or going with plpython3 or even both, I suppose(?). Naturally, I'm biased toward something that involves plpython3, so I don't think I can(should?) be of much help to -hackers as a Python PG user in any such decision. Of course, excepting the provision of justifications for my implementation/design choices... I would really love to see some input from Python users. I certainly don't want to waste time trying to get something into pgsql that Python users don't want. [here's a gripe that I haven't brought up as I think it is a matter of taste] I find (plpython) trigger functions to be a bit odd. I think it's the way in which manipulation/suppression decisions are made in BEFORE ROW triggers(return OK, SKIP, etc).. [label this as opinion at this point as I have yet to be able to nail down what, specifically, is wrong or un-pythonic about them.] Also, having distinct entry points to handle trigger events helps reduce potential errors by forcing the user to explicitly state the events that the trigger function can handle. Currently, in plpython, users *should* do sanity checks. Function modules opened the door for implementing this in a natural way, multiple functions(entry points) in the function module. http://python.projects.postgresql.org/pldocs/plpython3-programming.html#PLPYTHON3-FUNCTIONS-TRIGGERS -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] next CommitFest
On Fri, November 13, 2009 1:04 pm, Robert Haas wrote: the mere fact that we are even *discussing* whether it should be mandatory has led to a bumper crop of reviewers, Non sequitur. I think it is more likely that the bumper crop of reviewers is due to the lengthy discussion about the lack of reviewers. -- nathan wagner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
2009/11/14 Tom Lane t...@sss.pgh.pa.us: Andrew Gierth and...@tao11.riddles.org.uk writes: Peter == Peter Eisentraut pete...@gmx.net writes: Peter This is exactly the syntax that is in the spec AFAICT. Right. The spec defines this syntax for array_agg and xmlagg (only). Cool, I had forgotten that they added that in the latest revisions. I withdraw the complaint that this patch goes too far beyond the spec. But it would be entirely unreasonable, the way postgres works, to implement ORDER BY for only specific aggregates. Quite. This is another instance of the thing I complained of before, that the SQL committee likes to define the behavior of specific aggregates instead of inducing a generic aggregate-behavior definition. So we're on our own to extract one, and this proposal seems pretty reasonable to me: it's useful and it's consistent with the query-level behavior of DISTINCT and ORDER BY. It's not only in aggregates but also window function as well as plain functions like substring(x from t). In window functions, IGNORE NULLS is defined in spec for those first_vlaue(), last_value(), lead(), lag(), etc. but not for generic use. I'm +1 for an approach to apply them for generic cases. Regards, -- Hitoshi Harada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Experimental patch: generating BKI revisited
Tom Lane escribió: Dave Page dp...@pgadmin.org writes: On Fri, Nov 13, 2009 at 4:12 PM, Robert Haas robertmh...@gmail.com wrote: Yep, it's only on UNIX-ish systems where Perl isn't necessarily required, and realistically I think it is probably present on nearly all of those, too. Exactly. Yeah. Although the project policy is that we don't require Perl to build on Unix, there was a bug in the makefiles that made it effectively required, and nobody noticed for several years. I don't think it would be a hard sell to change that policy if we got a significant benefit out of it. (Depending on non-core Perl modules is a totally different thing though.) Well, this is a pretty fortunate turn of events. I had two paragraphs in my original email that I edited out (... so I'm not going to say more) on how to workaround the lack of Perl. If we're all OK now on requiring some basic Perl installation then all the better. I certainly have no trouble with it. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Missing feature in plpgsql EXECUTE ... USING support
I just noticed that plpgsql's OPEN cursor FOR EXECUTE command lacks a USING clause, so the only way to put parameters into the string is textual insertion. Seems like an oversight, since every other variant of EXECUTE in plpgsql can do USING. Did we omit that intentionally? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Aggregate ORDER BY patch
On Fri, Nov 13, 2009 at 5:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Quite. This is another instance of the thing I complained of before, that the SQL committee likes to define the behavior of specific aggregates instead of inducing a generic aggregate-behavior definition. I think this makes sense from the point of view of the spec authors. They're trying to standardize the behaviour of the functions that their existing implementations provide without creating extra demands on themselves to implement new features. Even if some of them do implement more general solutions the path of least resistance to getting their syntax standardized will be the one which imposes the least cost on the other members of the committee. So we're on our own to extract one, and this proposal seems pretty reasonable to me: it's useful and it's consistent with the query-level behavior of DISTINCT and ORDER BY. ++ -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Experimental patch: generating BKI revisited
Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane escribió: Yeah. Although the project policy is that we don't require Perl to build on Unix, there was a bug in the makefiles that made it effectively required, and nobody noticed for several years. I don't think it would be a hard sell to change that policy if we got a significant benefit out of it. (Depending on non-core Perl modules is a totally different thing though.) Well, this is a pretty fortunate turn of events. I had two paragraphs in my original email that I edited out (... so I'm not going to say more) on how to workaround the lack of Perl. If we're all OK now on requiring some basic Perl installation then all the better. I certainly have no trouble with it. Although actually, we could still keep that policy if Perl is needed to build .bki files --- we just have to build those files in distprep and ship them as part of tarballs. It's already the case that you need Perl to build from a CVS pull, it's only tarball users who don't need it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Experimental patch: generating BKI revisited
On Fri, Nov 13, 2009 at 2:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane escribió: Yeah. Although the project policy is that we don't require Perl to build on Unix, there was a bug in the makefiles that made it effectively required, and nobody noticed for several years. I don't think it would be a hard sell to change that policy if we got a significant benefit out of it. (Depending on non-core Perl modules is a totally different thing though.) Well, this is a pretty fortunate turn of events. I had two paragraphs in my original email that I edited out (... so I'm not going to say more) on how to workaround the lack of Perl. If we're all OK now on requiring some basic Perl installation then all the better. I certainly have no trouble with it. Although actually, we could still keep that policy if Perl is needed to build .bki files --- we just have to build those files in distprep and ship them as part of tarballs. It's already the case that you need Perl to build from a CVS pull, it's only tarball users who don't need it. I just said the same thing a few hours ago... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs head doesn't pass make check on one of the machines here
Peter Eisentraut pete...@gmx.net writes: On fre, 2009-11-13 at 15:05 +0100, Grzegorz JaÅkiewicz wrote: As per Tom's - yes, this laptop has LANG set to UTF8 Polish. Setting it back to EN actually makes this error go away. The Polish locale isn't supported by the regression tests. With only one result-ordering difference, it seems like we could easily support that if there were enough demand. I'd want somebody running a buildfarm machine in Polish locale, though, to catch future breakages. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inspection of row types in pl/pgsql and pl/sql
Tom Lane wrote: Florian G. Pflug f...@phlo.org writes: I'd like to replace this function-generating function by a generic trigger function that works for all tables. Due to the lack of any way to inspect the *structure* of a record type, however, I'd have to use a C language function for that, which induces quite some maintenance headaches (especially if deployed on windows). Trying to do this in plpgsql is doomed to failure and heartache, because it's fundamentally a strongly typed language. The proposed functions won't fix that and hence will be unusable in practice. I'd suggest either using C, or using one of the less-strongly-typed PLs. Well, the proposed functions at least allow for some more flexibility in working with row types, given that you know in advance which types you will be dealing with (but not necessarily the precise ordering and number of the record's fields). They might feel a bit kludgy because of the anyelement dummy argument that bridges the gap between the statically typed nature of SQL and the rather dynamic RECORDs, but the kludgy-ness factor is still within reasonable limits I think. Since all the other PLs (except C) are not nearly as integrated with the postgres type system, using them for this task does not really buy anything IMHO. AFAIK, all these PLs will convert any SQL type which isn't specifically mapped to one of the PLs types to a string. *That* I can do with pl/pgsql too, by simply using record_out() and then parsing the result... C of course lets me work around all these problems - but at the cost of a longer development time and (more importantly) more maintenance headaches (especially on windows, where a C compiler is not just one apt-get/yum/whatever call away). Regarding usability - the proposed function would for example allow you to implement a wide-range of row-to-text conversion functions in pure pl/pgsql by calling record_value(record, name, anyelement) with NULL::varchar as the last argument for each field, and then concatinating the resulting text together any way you like. best regards, Florian Pflug smime.p7s Description: S/MIME Cryptographic Signature
Re: [HACKERS] Check constraint on domain over an array not executed for array literals
Heikki Linnakangas wrote: Tom Lane wrote: Hm. I concur that this special-case code is failing to consider the possibility that the target type is domain-over-array-type rather than just array-type. I think though that this patch is a bit of a kluge, because it delivers a mislabeled expression tree. The result of the transformArrayExpr() is not really of type myintarray. What it is is a plain int[] value; we shouldn't label it as being already myintarray, because it hasn't passed the domain checks. At the moment there is probably not any visible effect of that, but in the future it could lead to misoptimization, so I think it's important to get it right. My inclination is to apply getBaseTypeAndTypmod to the targetType and pass that as the array type to transformArrayExpr, then instead of considering the job done, fall through to transformTypeCast, which will either do nothing or attach a domain coercion node. Hmm, yeah that's more accurate. Ok, committed another patch doing exactly that. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Hot standby, overflowed snapshots, testing
In GetSnapshotData(), we set subcount to -1 if the snapshot was overflowed:
subcount = GetKnownAssignedTransactions(snapshot-subxip,
xmin, xmax, overflow);
/*
* See if we have removed any subxids from KnownAssignedXids
that
* we might need to see. If so, mark snapshot overflowed.
*/
if (overflow)
subcount = -1; /* overflowed */
In XidInMVCCSnapshot we do this:
/*
* In recovery we store all xids in the subxact array because it
* is by far the bigger array, and we mostly don't know which
xids
* are top-level and which are subxacts. The xip array is empty.
*
* We start by searching subtrans, if we overflowed.
*/
if (snapshot-subxcnt 0)
{
/* overflowed, so convert xid to top-level */
xid = SubTransGetTopmostTransaction(xid);
/*
* If xid was indeed a subxact, we might now have an
xid xmin, so
* recheck to avoid an array scan. No point in
rechecking xmax.
*/
if (TransactionIdPrecedes(xid, snapshot-xmin))
return false;
}
/*
* We now have either a top-level xid higher than xmin or an
* indeterminate xid. We don't know whether it's top level or
subxact
* but it doesn't matter. If it's present, the xid is visible.
*/
for (j = 0; j snapshot-subxcnt; j++)
{
if (TransactionIdEquals(xid, snapshot-subxip[j]))
return true;
}
Note that if subxcnt is -1 to mark that the snapshot is overflowed, the
for-loop will do nothing. IOW, if the snapshot is overflowed,
XidInMVCCSnapshot always returns false.
This seems pretty straightforward to fix, we'll just need separate flag
to mark whether the subxid array has overflowed, but I'm bringing this
up as a separate email because this is the 2nd bug in XidInMVCCSnapshot
already, the first one being the silly one that the if-condition was
backwards. It seems that no-one still has done any testing of the
subxact stuff and visibility, and that is pretty scary.
I got the impression earlier that you had some test environment set up
to test hot standby. Can you share any details of what test cases you've
run?
I think we're going to have a good number of volunteers to test Hot
Standby, but it would be useful to define some specific test cases to
exercise all the hairy recovery transaction tracking and snapshot
related things. Or at least provide a list of the difficult parts so
that people know to what to test. If people know roughly what the tricky
areas are, we'll get better coverage than if people just kick the tires.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Missing feature in plpgsql EXECUTE ... USING support
2009/11/13 Tom Lane t...@sss.pgh.pa.us: I just noticed that plpgsql's OPEN cursor FOR EXECUTE command lacks a USING clause, so the only way to put parameters into the string is textual insertion. Seems like an oversight, since every other variant of EXECUTE in plpgsql can do USING. Did we omit that intentionally? It's my mistake. I am sure, so OPEN FOR EXECUTE needs USING clause. It's a paradox so Oracle OPEN function missing any support for dynamic query. We are better than Oracle :). Pavel regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs head doesn't pass make check on one of the machines here
On 13 Nov 2009, at 19:39, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On fre, 2009-11-13 at 15:05 +0100, Grzegorz Jaśkiewicz wrote: As per Tom's - yes, this laptop has LANG set to UTF8 Polish. Setting it back to EN actually makes this error go away. The Polish locale isn't supported by the regression tests. With only one result-ordering difference, it seems like we could easily support that if there were enough demand. I'd want somebody running a buildfarm machine in Polish locale, though, to catch future breakages. Setting the LANG variable doesn't change behaviour, unfortunately - and it passes on other computers. But changing it on the machine with Polish locale does make it pass, weird. (I barely use Polish locales, even tho I was born long time ago in that country, so it is hard to test it for me - on daily basis. But I will try to re-test it every so often). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify rewrite
My original intention was to have the queue as a circular buffer where the size of the entries was variable, but possibly bounded. Certainly using fixed length records of large size seems somewhat wasteful. Maybe we should do away with 'spill to disk' all together and either hard-code an overflow behavior or make it a knob. Possible overflow behaviors could be block until space is available, throw an error or silently drop it. Can the size of the shared memory segment for notifications be configurable? That would allow those with large payloads or a huge number of notifications to bump memory to avoid overflow cases. By removing the disk and making shmem usage configurable, I think the notify system would be flexible and could scale nicely. Another added benefit is the payload limit can be much higher than previously considered, because memory/performance concerns are now in the hands of the DBA. Incidentally, I'd like to thank Joachim personally for having done this work, +1 -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
