Re: [HACKERS] Hot Standby on git
On Sat, Sep 26, 2009 at 5:49 AM, Simon Riggs si...@2ndquadrant.com wrote: Just a note to say that Hot Standby patch is now on git repository git://git.postgresql.org/git/users/simon/postgres Branch name: hot_standby Awesome! Thanks for taking the time to get this set up. The complete contents of that repository are BSD licenced contributions to the PostgreSQL project. Excellent... Any further changes to that will be by agreement here on hackers. From now, I will be submitting each individual change as patch-on-patch to allow people to see and discuss them and to confirm them as open source contributions. Sounds good. I request anybody else interested to do the same to allow us to work together. All contributions welcome. My record of agreed changes is here http://wiki.postgresql.org/wiki/Hot_Standby#Remaining_Work_Items You'll notice that I've already completed 8 changes (10 commits); those are all fairly minor changes, so submitted here as a combined patch. There are 9 pending changes, so far, none of which appear to be major obstacles to resolve. Many thanks to Heikki for a thorough review which has identified nearly all of those change requests. I estimate that making the remaining changes noted on the Wiki and fully testing them will take at least 2 weeks. Gabriele Bartolini is assisting in this area, though neither of us are able to work full time on this. We still have ample time to complete the project in this release. Sounds like you are making rapid progress. If you think there's something useful I could do, let me know and I'll take a look. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Unicode escapes in E'...' strings Author: Marko Kreen
On 9/26/09, Peter Eisentraut pete...@gmx.net wrote: On Sat, 2009-09-26 at 00:18 +0300, Marko Kreen wrote: On 9/26/09, Tom Lane t...@sss.pgh.pa.us wrote: Maybe it doesn't need to know, but I think it would be disastrous from a maintenance standpoint to not keep the two sets of flex rules in strict correspondence. It would soon become unclear whether or how to apply changes in the backend lexer to psql. Patch attached. That patch results in the following message from flex: psqlscan.l:1039: warning, -s option given but default rule can be matched Agh. Well, that just means the xeu state must be commented out: -%x xeu +/* %x xeu */ -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Unicode escapes in E'...' strings Author: Marko Kreen
On Sat, 2009-09-26 at 00:18 +0300, Marko Kreen wrote: On 9/26/09, Tom Lane t...@sss.pgh.pa.us wrote: Maybe it doesn't need to know, but I think it would be disastrous from a maintenance standpoint to not keep the two sets of flex rules in strict correspondence. It would soon become unclear whether or how to apply changes in the backend lexer to psql. Patch attached. That patch results in the following message from flex: psqlscan.l:1039: warning, -s option given but default rule can be matched -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] syslog_line_prefix
On Fri, Sep 25, 2009 at 6:12 PM, Andrew Dunstan and...@dunslane.net wrote: Tom Lane wrote: Joshua Tolley eggyk...@gmail.com writes: Having just sent two messages to the discussion about the wrong patch, I'll apologize, and shut up now :) No need to apologize --- this really is, and should be, all one conversation. I think the main problem I've got with applying either patch is that I don't believe we have consensus on the direction the logging code should go. Without that, it's a bad idea to accept incremental patches, even if they're arguably harmless by themselves. Agreed. The discussion does have en element of /déją vu,/ too. The the whole idea behind log_line_prefix was to allow people to make easier and better log splitting decisions after the fact. Like you I'm wary of adding too much extra processing into the elog code. I think we have consensus that this patch isn't clearly moving us in the right direction, and might be moving us in the wrong direction, so I am going to mark it as Rejected. I also agree with Tom's comments that we don't have consensus on where this should go. I think it would help a lot if someone put together a design document (perhaps on the wiki) and tried to enumerate at a high level the logging requirements that aren't being satisfied by the current system. Then we could have a conversation about the right way to address them. By writing the code first, I think we're putting the cart before the horse. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Join optimization for inheritance tables
On Tue, 2009-09-22 at 18:16 -0400, Emmanuel Cecchet wrote: If the partitioning implementation does not make progress (and does not make it for 8.5) Manu, not sure if you are referring to Kedar's patch I reviewed earlier in July, but that patch didn't implement anything like the right internals for this to work. As Tom says, what we need is a single internal data structure that describes all the partitions. That can then be used to automatically route inserts, perform the push down merge joins and speed-up constraint exclusion. We should build this once and store it in the relcache for the parent relation and so would need some changes in relcache invalidation as well to make sure child ddl changes invalidate that. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO item: Allow more complex user/database default GUC settings
Alvaro Herrera alvhe...@commandprompt.com writes: Right, that works. Updated patch attached; should solve the issues raised in the thread. I renamed the catalog pg_db_role_setting as suggested by Tom. ... I have also added a view, whose only purpose is to convert the role and database OIDs into names. It's been named pg_db_role_settings, but if anyone has a better suggestion I'm all ears. I dislike the idea of having a catalog and a view whose names are the same except for a plural. It's confusing as heck, because no one will remember which is which. Since pg_settings is the existing user view, I think pg_db_role_settings is a reasonable choice for the new view, but then we need a different name for the catalog. The only thing that comes to mind right now is pg_db_role_default, but I don't like it much. Anybody have other suggestions? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Hot Standby on git
Just a note to say that Hot Standby patch is now on git repository git://git.postgresql.org/git/users/simon/postgres Branch name: hot_standby The complete contents of that repository are BSD licenced contributions to the PostgreSQL project. Any further changes to that will be by agreement here on hackers. From now, I will be submitting each individual change as patch-on-patch to allow people to see and discuss them and to confirm them as open source contributions. I request anybody else interested to do the same to allow us to work together. All contributions welcome. My record of agreed changes is here http://wiki.postgresql.org/wiki/Hot_Standby#Remaining_Work_Items You'll notice that I've already completed 8 changes (10 commits); those are all fairly minor changes, so submitted here as a combined patch. There are 9 pending changes, so far, none of which appear to be major obstacles to resolve. Many thanks to Heikki for a thorough review which has identified nearly all of those change requests. I estimate that making the remaining changes noted on the Wiki and fully testing them will take at least 2 weeks. Gabriele Bartolini is assisting in this area, though neither of us are able to work full time on this. We still have ample time to complete the project in this release. Many thanks to Magnus and Aidan for helping me resolve my git-wrestling contest and apologies for the delay while that bout happened. -- Simon Riggs www.2ndQuadrant.com *** a/doc/src/sgml/backup.sgml --- b/doc/src/sgml/backup.sgml *** *** 1934,1941 if (!triggered) /para para ! Read-only here means no writes to the permanent database tables. So ! there are no problems with queries that make use of temporary sort and work files will be used. Temporary tables cannot be created and therefore cannot be used at all in recovery mode. /para --- 1934,1941 /para para ! Read-only here means no writes to the permanent database tables. ! There are no problems with queries that make use of temporary sort and work files will be used. Temporary tables cannot be created and therefore cannot be used at all in recovery mode. /para *** *** 1983,1989 if (!triggered) /listitem listitem para !LOCK, with restrictions, see later /para /listitem listitem --- 1983,1989 /listitem listitem para !LOCK TABLE, though only when explicitly IN ACCESS SHARE MODE /para /listitem listitem *** *** 2000,2014 if (!triggered) /para para ! These actions will produce error messages itemizedlist listitem para !DML - Insert, Update, Delete, COPY FROM, Truncate which all write data. ! Any RULE which generates DML will throw error messages as a result. ! Note that there is no action possible that can result in a trigger ! being executed. /para /listitem listitem --- 2000,2013 /para para ! These actions produce error messages itemizedlist listitem para !DML - Insert, Update, Delete, COPY FROM, Truncate. ! Note that there are no actions that result in a trigger ! being executed during recovery. /para /listitem listitem *** *** 2024,2029 if (!triggered) --- 2023,2041 /listitem listitem para +RULEs on SELECT statements that generate DML commands. RULEs on DML + commands that produce only SELECT statements are already disallowed + during read-only transactions. + /para + /listitem + listitem + para +LOCK TABLE, in short default form, since it requests ACCESS EXCLUSIVE MODE. +LOCK TABLE that explicitly requests a lock other than ACCESS SHARE MODE. + /para + /listitem + listitem + para Transaction management commands that explicitly set non-read only state itemizedlist listitem *** *** 2069,2077 if (!triggered) para Note that current behaviour of read only transactions when not in ! recovery is to allow the last two actions, so there is a small and ! subtle difference in behaviour between standby read-only transactions ! and read only transactions during normal running. It is possible that the restrictions on LISTEN, UNLISTEN, NOTIFY and temporary tables may be lifted in a future release, if their internal implementation is altered to make this possible. --- 2081,2089 para Note that current behaviour of read only transactions when not in ! recovery is to allow the last two actions, so there are small and ! subtle differences in behaviour between read-only transactions ! run on standby and during normal running. It is possible that the restrictions on LISTEN, UNLISTEN, NOTIFY and temporary tables may be lifted in a future release, if their
Re: [HACKERS] WIP - syslogger infrastructure changes
On Fri, Sep 25, 2009 at 5:22 PM, Robert Haas robertmh...@gmail.com wrote: AIUI the problem is that when logging_collector is on, we throw away the original stderr. That's OK as long as you never try to switch back to it. BTW, this seems like it could be fixed with some appropriate file descriptor management in postmaster. No? Anyway, marking this Rejected for now, as that seems to be the consensus. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby on git
On Sat, 2009-09-26 at 09:29 -0400, Robert Haas wrote: I estimate that making the remaining changes noted on the Wiki and fully testing them will take at least 2 weeks. Gabriele Bartolini is assisting in this area, though neither of us are able to work full time on this. We still have ample time to complete the project in this release. Sounds like you are making rapid progress. Only because Heikki has put so much effort into review and because most of the remaining issues are coding related, rather than theoretical. We moving along and are in no way threatened by time. If you think there's something useful I could do, let me know and I'll take a look. I feel like I need a better way of unit testing new code. Some of the code in the patch is to handle corner cases, so recreating them is fairly hard. It is a nagging feeling that I am missing some knowledge here and would welcome some insight, or research, into better ways of doing general case unit testing. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby on git
On 09/26/2009 10:04 AM, Simon Riggs wrote: If you think there's something useful I could do, let me know and I'll take a look. I feel like I need a better way of unit testing new code. Some of the code in the patch is to handle corner cases, so recreating them is fairly hard. It is a nagging feeling that I am missing some knowledge here and would welcome some insight, or research, into better ways of doing general case unit testing. You might try and steal ideas from EasyMock / PowerMock - but not sure how well the ideas map to C. Generally it means allowing the functions to be called from a mock environment, where subroutine calls that might be called are stubbed out to return sample data that would simulate your scenario. Object oriented languages that require every object to provide an interface where most object methods can be overridden are more ideal for performing this sort of test. I rarely ever see this sort of stuff in FOSS projects, and never that I can remember in FOSS C projects. It's not easy, though. I assume you are doing it through code changing right now. Commenting out lines, replacing them with others, etc? Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby on git
On Sat, Sep 26, 2009 at 10:45:17AM -0400, Mark Mielke wrote: On 09/26/2009 10:04 AM, Simon Riggs wrote: If you think there's something useful I could do, let me know and I'll take a look. I feel like I need a better way of unit testing new code. Some of the code in the patch is to handle corner cases, so recreating them is fairly hard. It is a nagging feeling that I am missing some knowledge here and would welcome some insight, or research, into better ways of doing general case unit testing. You might try and steal ideas from EasyMock / PowerMock - but not sure how well the ideas map to C. Generally it means allowing the functions to be called from a mock environment, where subroutine calls that might be called are stubbed out to return sample data that would simulate your scenario. Object oriented languages that require every object to provide an interface where most object methods can be overridden are more ideal for performing this sort of test. I rarely ever see this sort of stuff in FOSS projects, and never that I can remember in FOSS C projects. It's not easy, though. I assume you are doing it through code changing right now. Commenting out lines, replacing them with others, etc? Cheers, mark -- Mark Mielkem...@mielke.cc There are a variety of projects dedicated to creating C unit test frameworks. I don't have a lot of experience with them, but I have heard good things about check and cunit. Here's a link I found with a longer list of frameworks. http://www.opensourcetesting.org/unit_c.php -- --Dan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP - syslogger infrastructure changes
Robert Haas robertmh...@gmail.com writes: On Fri, Sep 25, 2009 at 5:22 PM, Robert Haas robertmh...@gmail.com wrote: AIUI the problem is that when logging_collector is on, we throw away the original stderr. That's OK as long as you never try to switch back to it. BTW, this seems like it could be fixed with some appropriate file descriptor management in postmaster. No? Yeah, probably. In the current system design it didn't seem to be necessary because collecting the collector process's own stderr output is usually not all that critical. If we had the postmaster dup its original stderr and the collector dup it back, that would be a more complete but more complex solution. (dup2 works on Windows, no?) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP - syslogger infrastructure changes
On Sat, Sep 26, 2009 at 17:43, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Sep 25, 2009 at 5:22 PM, Robert Haas robertmh...@gmail.com wrote: AIUI the problem is that when logging_collector is on, we throw away the original stderr. That's OK as long as you never try to switch back to it. BTW, this seems like it could be fixed with some appropriate file descriptor management in postmaster. No? Yeah, probably. In the current system design it didn't seem to be necessary because collecting the collector process's own stderr output is usually not all that critical. If we had the postmaster dup its original stderr and the collector dup it back, that would be a more complete but more complex solution. (dup2 works on Windows, no?) Yeah, they should. Not sure how that works through process boundaries though, we do some special fiddling for any handles that need to be inherited. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Unicode escapes in E'...' strings Author: Marko Kreen
Marko Kreen mark...@gmail.com writes: On 9/26/09, Peter Eisentraut pete...@gmx.net wrote: That patch results in the following message from flex: psqlscan.l:1039: warning, -s option given but default rule can be matched Agh. Well, that just means the xeu state must be commented out: -%x xeu +/* %x xeu */ Ick --- that breaks the whole concept of keeping the two sets of flex rules in sync. And it's quite unclear why it fixes the problem, too. At the very least, if you do it that way, it needs a comment explaining exactly why it's different from the backend. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] syslog_line_prefix
Robert Haas robertmh...@gmail.com writes: I also agree with Tom's comments that we don't have consensus on where this should go. I think it would help a lot if someone put together a design document (perhaps on the wiki) and tried to enumerate at a high level the logging requirements that aren't being satisfied by the current system. Then we could have a conversation about the right way to address them. By writing the code first, I think we're putting the cart before the horse. +1 ... that seems like a much more sensible way to proceed than submitting patches without prior discussion. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Unicode escapes in E'...' strings Author: Marko Kreen
Resend...
On 9/26/09, Tom Lane t...@sss.pgh.pa.us wrote:
Marko Kreen mark...@gmail.com writes:
On 9/26/09, Peter Eisentraut pete...@gmx.net wrote:
That patch results in the following message from flex:
psqlscan.l:1039: warning, -s option given but default rule can be
matched
Agh. Well, that just means the xeu state must be commented out:
-%x xeu
+/* %x xeu */
Ick --- that breaks the whole concept of keeping the two sets of
flex rules in sync. And it's quite unclear why it fixes the problem,
too. At the very least, if you do it that way, it needs a comment
explaining exactly why it's different from the backend.
The commenting-out fixes the problem, because I copy pasted the state
declaration without any rules in it.
Anyway, now I attached a patch, where I filled the section but without
referring it from anywhere. The rules itself are now equal. Is that OK?
--
marko
diff --git a/src/bin/psql/psqlscan.l b/src/bin/psql/psqlscan.l
index 7f08da2..8309577 100644
--- a/src/bin/psql/psqlscan.l
+++ b/src/bin/psql/psqlscan.l
@@ -158,6 +158,7 @@ static void emit(const char *txt, int len);
* xdolq $foo$ quoted strings
* xui quoted identifier with Unicode escapes
* xus quoted string with Unicode escapes
+ * xeu Unicode surrogate pair in extended quoted string.
*/
%x xb
@@ -169,6 +170,7 @@ static void emit(const char *txt, int len);
%x xdolq
%x xui
%x xus
+%x xeu
/* Additional exclusive states for psql only: lex backslash commands */
%x xslashcmd
%x xslasharg
@@ -253,6 +255,8 @@ xeinside [^\\']+
xeescape [\\][^0-7]
xeoctesc [\\][0-7]{1,3}
xehexesc [\\]x[0-9A-Fa-f]{1,2}
+xeunicode [\\](u[0-9A-Fa-f]{4}|U[0-9A-Fa-f]{8})
+xeunicodefail [\\](u[0-9A-Fa-f]{0,3}|U[0-9A-Fa-f]{0,7})
/* Extended quote
* xqdouble implements embedded quote,
@@ -511,6 +515,20 @@ other .
xe{xeinside} {
ECHO;
}
+xe{xeunicode} {
+ ECHO;
+}
+xeu{xeunicode} {
+ ECHO;
+}
+xeu. |
+xeu\n |
+xeuEOF {
+ ECHO;
+}
+xe,xeu{xeunicodefail} {
+ ECHO;
+}
xe{xeescape} {
ECHO;
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO item: Allow more complex user/database default GUC settings
On Sep 26, 2009, at 11:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Right, that works. Updated patch attached; should solve the issues raised in the thread. I renamed the catalog pg_db_role_setting as suggested by Tom. ... I have also added a view, whose only purpose is to convert the role and database OIDs into names. It's been named pg_db_role_settings, but if anyone has a better suggestion I'm all ears. I dislike the idea of having a catalog and a view whose names are the same except for a plural. It's confusing as heck, because no one will remember which is which. Since pg_settings is the existing user view, I think pg_db_role_settings is a reasonable choice for the new view, but then we need a different name for the catalog. The only thing that comes to mind right now is pg_db_role_default, but I don't like it much. Anybody have other suggestions? The problem of having both a table and a closely related view is, IME, one that comes up a lot. I think you just need to pick a convention and stick with it. Mine is to append _view to the table name. Renaming the underlying table doesn't seem like it helps at all. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby on git
I feel like I need a better way of unit testing new code. Some of the code in the patch is to handle corner cases, so recreating them is fairly hard. It is a nagging feeling that I am missing some knowledge here and would welcome some insight, or research, into better ways of doing general case unit testing. There's always pgtap. Whenever we find a new corner case, we add it to the development test suite. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Join optimization for inheritance tables
Herodotos Herodotou h...@cs.duke.edu writes: On Tue, Sep 22, 2009 at 8:06 PM, Jeff Davis pg...@j-davis.com wrote: I think you mean that the planning time is in milliseconds, not seconds. The planning time is actually in seconds. This is exactly why I think this is a dead-end approach. Trying to do theorem proving from an unstructured collection of constraints simply cannot scale to hundreds of partitions, at least not if you want reasonable planning performance. There are other aspects of our current partitioning approach that don't scale either, eg the complexity of the insert redirection triggers. We could handle standard cases where there's a simple partitioning rule with far less overhead than this, if we had an explicit model of the partitioning rule inside the system. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 TODO: Add comments to output indicating version of pg_dump and of the database server
On Fri, 2009-09-25 at 16:59 -0400, Tom Lane wrote: shakahsha...@gmail.com shakahsha...@gmail.com writes: From pg_dump/pg_restore section (9.2 of the Todo page on the PostgreSQL Wiki), is the following item Add comments to output indicating version of pg_dump and of the database server simply asking for a change to the pg_dump header from: I think so, but what's not clear is whether this is a good idea to do in the default output. It might only be appropriate in verbose mode, so as not to introduce unnecessary diffs between logically identical dumps. Well, a diff of the same database made by different (major) versions of pg_dump will already be different in most situations, so adding the pg_dump version number in it is essentially free from this perspective. What is the use case for adding the server version? I can imagine something like wanting to know exactly where the dump came from, but then host name and such would be better. (And then you can infer the server version from that.) Another issue is that it's not all that clear what to do or how to do it for archive dumps --- do you then want both pg_dump and pg_restore to tell you about themselves? I don't see a good reason for pg_restore to get involved. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby on git
On 09/26/2009 02:28 PM, Dan Colish wrote: There are a variety of projects dedicated to creating C unit test frameworks. I don't have a lot of experience with them, but I have heard good things about check and cunit. Here's a link I found with a longer list of frameworks. http://www.opensourcetesting.org/unit_c.php Looking at check and cunit - I don't see what sort of mock function facility they would provide? One part of unit testing is arranging for functions to be called, tested, and results reported on. This can take you a certain amount of the way. Pure functions, for example, that always generate the same output for the same input parameters, are perfect for this situation. Perhaps test how a qsort() or bsearch() method works under various scenarios? Most real life code gets a little more complicated. For example, what if we want to simulate a network failure or out of disk space condition? What if we want to test out what happens when the Y2038 date is reached? This requires either complex test case setup that is difficult to run reproducibly, or another approach - mock. It means doing things like overriding the write() method, and making it return successful N times, and then failing on the (N+1)th time with ENOSPC. It means overriding the gettimeofday() method to return a time in the future. A major benefit of this sort of testing is that it should not require source changes in order to perform the test. This sort of stuff is a LOT easier to do in OO languages. I see it done in Java a lot. I can't remember ever having seen it done in C. I think it's just too hard compared to the value obtained from the effort. In your list above, it does show a few attempts - CMock sticks out as a for example. It looks more complicated, though. It takes a .h file and generates stubs for you to fill in? That could be difficult to manage for a large project with thousands or many times more unit tests. OO is easier because you can override *only* particular methods, and you can safely call the super method that it overrides to provide the underlying behaviour in the success cases. Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP - syslogger infrastructure changes
On Sat, Sep 26, 2009 at 11:43:46AM -0400, Tom Lane wrote: complete but more complex solution. (dup2 works on Windows, no?) Unless I'm misreading syslogger.c, dup2() gets called on every platform. I've started the wiki page in question: http://wiki.postgresql.org/wiki/Logging_Brainstorm It doesn't have anything linking to it right now, which might be a bad thing. -- Joshua Tolley / eggyknap End Point Corporation http://www.endpoint.com signature.asc Description: Digital signature
Re: [HACKERS] Join optimization for inheritance tables
Hi Simon, Thanks for the insight. I might take that as a long term project. I have to discuss that with my colleagues at Aster. It would certainly help to put together a wiki page with the key insights on the design of such implementation to be able to better scope the project and agree on what is the right approach for this. manu On Tue, 2009-09-22 at 18:16 -0400, Emmanuel Cecchet wrote: If the partitioning implementation does not make progress (and does not make it for 8.5) Manu, not sure if you are referring to Kedar's patch I reviewed earlier in July, but that patch didn't implement anything like the right internals for this to work. As Tom says, what we need is a single internal data structure that describes all the partitions. That can then be used to automatically route inserts, perform the push down merge joins and speed-up constraint exclusion. We should build this once and store it in the relcache for the parent relation and so would need some changes in relcache invalidation as well to make sure child ddl changes invalidate that. -- Emmanuel Cecchet Aster Data Systems Web: http://www.asterdata.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Issues for named/mixed function notation patch
On Tue, 2009-09-15 at 10:51 +0200, Pavel Stehule wrote: My renonc, please, try new patch. I forgot mark regproc.c file. I think the documentation around calling functions is disorganized: Variadic functions, functions with defaults, SRFs, out parameters, and polymorphism are all explained in 34.4, which is about SQL functions specifically. Overloading is in chapter 34 also, but not specifically in the SQL function section like the rest. Function calls themselves are only given 5 lines of explanation in 4.2.6, with no mention of things like the VARIADIC keyword. These complaints aren't about the patch, but we might want to consider some reorganization of those sections (probably a separate doc patch). The interaction with variadic functions appears to be misdocumented. From the code and tests, the VARIADIC keyword appears to be optional when using named notation, but required when using positional notation. But the documentation says: However, a named variadic argument can only be called the way shown in the example above. The VARIADIC keyword must not be specified and a variadic notation of all arguments is not supported. To use variadic argument lists you must use positional notation instead. What is the intended behavior? I think we should always require VARIADIC to be specified regardless of using named notation. I'm still reviewing the code. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 TODO: Add comments to output indicating version of pg_dump and of the database server
On Sat, Sep 26, 2009 at 11:02:55PM +0300, Peter Eisentraut wrote: On Fri, 2009-09-25 at 16:59 -0400, Tom Lane wrote: shakahsha...@gmail.com shakahsha...@gmail.com writes: From pg_dump/pg_restore section (9.2 of the Todo page on the PostgreSQL Wiki), is the following item Add comments to output indicating version of pg_dump and of the database server simply asking for a change to the pg_dump header from: I think so, but what's not clear is whether this is a good idea to do in the default output. It might only be appropriate in verbose mode, so as not to introduce unnecessary diffs between logically identical dumps. Well, a diff of the same database made by different (major) versions of pg_dump will already be different in most situations, so adding the pg_dump version number in it is essentially free from this perspective. What is the use case for adding the server version? There have been cases where pg_restore doesn't fix infelicities. For example, there was a time when it was a good idea to run adddepend after the reload. Knowing what server version the dump came from could be handy for this kind of case. I can imagine something like wanting to know exactly where the dump came from, but then host name and such would be better. (And then you can infer the server version from that.) You can infer the server version until the next upgrade, at which point the information is lost. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby on git
On Sep 26, 2009, at 12:33 PM, Josh Berkus wrote: There's always pgtap. Whenever we find a new corner case, we add it to the development test suite. Also, for C TAP, there's [libtap](http://jc.ngo.org.uk/trac-bin/trac.cgi/wiki/LibTap ). You can then use `prove` which you likely already have on your system, to harness the test output. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP - syslogger infrastructure changes
Joshua Tolley wrote: On Sat, Sep 26, 2009 at 11:43:46AM -0400, Tom Lane wrote: complete but more complex solution. (dup2 works on Windows, no?) Unless I'm misreading syslogger.c, dup2() gets called on every platform. Yes. Windows supports dup2(). See for example http://msdn.microsoft.com/en-us/library/8syseb29(VS.80).aspx cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby on git
Mark Mielke escribió: Most real life code gets a little more complicated. For example, what if we want to simulate a network failure or out of disk space condition? What if we want to test out what happens when the Y2038 date is reached? This requires either complex test case setup that is difficult to run reproducibly, or another approach - mock. It means doing things like overriding the write() method, and making it return successful N times, and then failing on the (N+1)th time with ENOSPC. I remember a kernel simulator based on User-Mode Linux called UMLsim, with which you could stuff like this. It's dead at this point though, and I don't know if there's any replacement. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Unicode escapes in E'...' strings Author: Marko Kreen
Marko Kreen mark...@gmail.com writes: Anyway, now I attached a patch, where I filled the section but without referring it from anywhere. The rules itself are now equal. Is that OK? Well, you also have to track the state changes (BEGIN). In comparing the scanners I realized I'd forgotten to sync psql myself when I was fooling around with the plpgsql scanner :-(. So mea culpa as well ... Fixed and applied. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq port number handling
Sam Mason s...@samason.me.uk writes: Hum, why is PG doing an (unchecked) atoi on the user specified port rather than leaving it up to getaddrinfo to resolve the port? It would seem to require changing UNIXSOCK_PATH to accept a string as the port number, which is probably a bit much of a change. The included doesn't feel very nice, but is probably more acceptable. Applied, thanks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO item: Allow more complex user/database default GUC settings
Robert Haas escribió: The problem of having both a table and a closely related view is, IME, one that comes up a lot. I think you just need to pick a convention and stick with it. Mine is to append _view to the table name. That would make the difference clear, but since what the user normally wants to see is the view, it seems a poor solution to make the view the more difficult one to type (and the one that isn't tab-completed first in psql). I'd go with naming the view pg_db_role_setting and append _internal to the catalog or something similar, except that we don't have any catalog with such a bad name yet and I don't want to start. Maybe name the table pg_configuration? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Issues for named/mixed function notation patch
However, a named variadic argument can only be called the way shown in the example above. The VARIADIC keyword must not be specified and a variadic notation of all arguments is not supported. To use variadic argument lists you must use positional notation instead. What is the intended behavior? I think we should always require VARIADIC to be specified regardless of using named notation. maybe we could to support variadic named parameters in future - then using VARIADIC keyword should be necessary - like foo(10 AS p1, 20 AS p1, 30 AS p3) is equalent of foo(VARIADIC ARRAY[10,20] AS p1, 30 AS p3) if we plan this feature, the VARIADIC keyword have to be mandatory. Regards Pavel Stehule I'm still reviewing the code. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Using results from INSERT ... RETURNING
Robert Haas robertmh...@gmail.com writes: Well, part of the problem is that I've not had a lot of luck trying to understand how the executor really works (what's a tuple table slot and why do we need to know in advance how many of them there are?). You know, that's actually a really good question. There is not, as far as I can see, much advantage to keeping the Slots in an array. We could perfectly well keep them in a List and eliminate the notational overhead of having to count them in advance. There'd be a bit more palloc overhead (for the list cells) but avoiding the counting work would more or less make up for that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
