Re: [HACKERS] Hot Standby: Relation-specific deferred conflict resolution
On Fri, Jan 29, 2010 at 9:03 AM, Simon Riggs si...@2ndquadrant.com wrote: That was not the feedback I have received. Nobody has commented on that to me, though many have commented on the need for the current patch. As mentioned, I went to the trouble of running a meeting to gain additional feedback and the result was very clear. I don't have a technical opinion about this problem yet as I haven't tested HS+SR yet but I'm not sure it's a good idea to base technical decisions and priorities on user polls (I'm pretty sure most of them don't use HS+SR as much as Heikki these days). If you ask people what they want in their future cars, they won't answer they want wheels or an engine: it's something obvious for them. AFAICS (but I might be wrong), you asked this question to people who are interested in HS+SR but don't have any idea of what it's like to use HS+SR daily with or without this limitation. There are perhaps better arguments for not doing it but this one seems a bit weird to me. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] creating index names automatically?
On Wed, Dec 23, 2009 at 7:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm still not really eager to introduce yet another category of keywords, but perhaps it is worth doing. Comments? If we consider that they can now be a problem in pl/pgsql, it might be a good idea to consider it for this release. (I'm thinking of http://archives.postgresql.org/message-id/17728.1261002...@sss.pgh.pa.us for instance) -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing config files in a directory
On Sun, Oct 25, 2009 at 10:08 AM, Peter Eisentraut pete...@gmx.net wrote: As a file point, I would prefer something like include 'whatever/*.conf' +1 for that. That's what Apache does and it works well for the users and the packagers. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Deprecation
On Fri, Oct 16, 2009 at 9:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: So, while I do think it's something we should leave alone until it gets in the way, this is a sufficiently large value of in the way that I'm willing to talk about removing add_missing_from. I'm just concerned about the impact of that, considering that an app that still depends on it came up as recently as yesterday. 8.4 is going to be supported for several years so they can document it's the last version supported by the current version of OpenACS. If they want the benefits from 8.5 and higher (for themselves or their users), they'll do the work needed to remove the various bad practices from their code. Note that the removal of the implicit casts to text was far more problematic for a lot of apps than add_missing_from=off - which is the de facto standard for several years now. And as you can see in http://openacs.org/forums/message-view?message_id=2471518 , they did the ground work to support the tsearch2 changes from contrib to core. A friendly email to them explaining why it needs to be fixed in their code should be sufficient (and it might be already fixed, btw). -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP - syslogger infrastructure changes
Hi Magnus, On Mon, Sep 14, 2009 at 9:41 PM, Magnus Hagander mag...@hagander.net wrote: First, the patch removes the logging_collector parameter and basically assumes that logging_collector is always on. Alvaro commited this a few days ago: http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/config.sgml?r1=1.225r2=1.226 Any consequence? Second, data transfer from the backends is now in the form of structured data, and the actual logging message is put together in the syslogger (today,it's put together in the backend and just sent as a string to the syslogger). Yes, this means that we will likely send more data than what's eventually going to be logged, since all fields don't go out (except with CVS logging, I guess). But you usually don't send *that* much data in the log. I don't know if it will make a real difference but some of us log quite a lot of queries. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release timetable, again
On Thu, Aug 27, 2009 at 12:03 AM, Dimitri Fontainedfonta...@hi-media.com wrote: Is the offering good enough? We might need to run some kind of tutorials for users to be able to run large tests easily, and maybe think about some newer tools allowing to compare logs of two application runs in two database versions (capture all queries and result in a database, then have a way to diff). Then beta testing would mean having a spare machine where to run the magic regression test suite against some internal application. Well, people may recall I spent a lot of time testing 8.3 before and during beta. The important words are a lot of time, probably one month full time spread on 3 months to find *only* 3 problems Tom, Alvarro and Andrew fixed: yes one month for only 3 problems identified, reported, discussed and fixed. The problem isn't to connect your application to the database - it's the easy part: if you have a large one, you probably won't see the anomalies. The application I used for my tests is displaying every SQL query at the bottom of the page with the time spent executing the query, I was switching from the 8.1 site to the 8.3 site to check everything manually. I also got all the urls of this application (more than one million), and use a load test tool to load each page and pgFouine to grab any error from the PostgreSQL logs. Even with these information and this work, I'm pretty sure I would have missed a join problem which would have returned 2 or 3 more rows even with the time I spent working on it. My plan at the time was to develop an application which would parse the query logs from the server, replay the queries on 2 PG servers with both versions and report me the anomalies (difference in the number of rows, difference in the content, queries slower than with the old version). I haven't had the opportunity to work on the 8.4 beta test due to my daily (and often nightly) job work load but the idea is still there and IMHO it's really necessary if we want to be able to detect the problems we only discovered after the release. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Table and Index compression
Pierre, On Thu, Aug 6, 2009 at 11:39 AM, PFCli...@peufeu.com wrote: The best for this is lzo : very fast decompression, a good compression ratio on a sample of postgres table and indexes, and a license that could work. The license of lzo doesn't allow us to include it in PostgreSQL without relicensing PostgreSQL as GPL. I'm not sure of what you imply by a license that could work. Note that it doesn't change the interest of your approach. It's just that I'm not sure we can find a performance-acceptable BSD licensed compression library (it was discussed a lot of times here). -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL state in log_line_prefix
On Fri, Jul 3, 2009 at 9:14 PM, Peter Eisentrautpete...@gmx.net wrote: On Sunday 10 May 2009 15:32:26 Guillaume Smet wrote: I attached a patch which allows to add the SQL state code into the log_line_prefix. I used %e (as in error) as %s is already used. Committed. Thanks Peter. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 development schedule
On Wed, Jul 1, 2009 at 11:42 PM, Andrew Dunstanand...@dunslane.net wrote: You have correctly identified the requirement in the sentence quoted above. I for one am quite prepared to support core or some person designated by core having such authority. I agree with you that without something like that not much will improve. That's the role of a release manager. Perhaps it's time to think about designating one for each release to endorse this responsability. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GRANT ON ALL IN schema
2009/6/17 Petr Jelinek pjmo...@pjmodos.net: I agree that Default ACLs are more important and I already offered Stephen help on that. But I've seen countless requests for granting on all tables to a user and I already got some positive feedback outside of the list, so I believe there is demand for this. Also to paraphrase you Tom, by that logic you can tell people to write half of administration functionality as plpgsql functions. Indeed. How to do default ACLs and wildcards for GRANT is by far the most common question asked by our customers. And they don't understand why it's not by default in PostgreSQL. Installing a script/function for that on every database is just painful. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Suggested TODO: allow ALTERing of typemods without heap/index rebuild
On Mon, Jun 1, 2009 at 10:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: We could certainly put in a quick hack that just covered a few of the cases for built-in types, but it's not very pleasing ... Jonah proposed a patch for that a long time ago. See http://archives.postgresql.org/pgsql-patches/2006-10/msg00154.php . -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Clean shutdown and warm standby
On Thu, May 28, 2009 at 5:02 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: So you check. This solves Guillaume's immediate concern. If you have a suggestion for further improvements, I'm all ears. Thanks for applying the patch. Yes, the problem is that before this change, even with a working replication and a clean shutdown, you still had to replicate the last WAL file by hand. Personnally, I have an eye on each postgresql log file when I switch from one server to another so I can see if anything is going wrong (that said, it could be a problem with more than 2 servers...). This patch just fixes this problem not the other concerns and corner cases we might have. If we want to go further, we need to agree on what we want exactly and which corner cases we want to cover but it's probably 8.5 material at this point. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Clean shutdown and warm standby
On Thu, May 28, 2009 at 5:36 PM, Simon Riggs si...@2ndquadrant.com wrote: If you feel we have moved forwards, that's good, but since no part of the *safe* maintenance procedure has changed, I don't see that myself. Only the unsafe way of doing it got faster. I disagree with you. The situation was: - you stop the master; - everything seems to be OK in the log files (archiving and so on); - it's broken anyway as you don't have the last log file; - you have to copy the last log file manually. - you can start the slave. It is now: - you stop the master; - if everything is OK in the log files, the last log file has been archived (and yes I check it manually too) and it's done. If not (and it's the exception, not the rule) I have to copy manually the missing WAL files; - you can start the slave. I think it's a step forward, maybe not sufficient for you but I prefer the situation now than before. It's safer because of the principle of least surprise: I'm pretty sure a lot of people didn't even think that the last WAL file was systematically missing. As Heikki stated it, if you have concrete proposals of how we can fix the other corner cases, we're all ears. Considering my current level of knowledge, that's all I can do by myself. IMHO, that's something that needs to be treated in the massive replication work planned for 8.5. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Clean shutdown and warm standby
On Thu, May 28, 2009 at 6:06 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2009-05-28 at 17:50 +0200, Guillaume Smet wrote: I think it's a step forward, maybe not sufficient for you but I prefer the situation now than before. It's safer because of the principle of least surprise: I'm pretty sure a lot of people didn't even think that the last WAL file was systematically missing. If I hadn't spoken out, I think you would have assumed you were safe and so would everybody else. Time is saved only if you perform the step manually - if time saving was your objective you should have been using a script in the first place. If you're using a script, carry on using it: nothing has changed, you still need to check. You might think that but I won't have. I will still monitor my log files carefully and check the last WAL file is received and treated on the slave as I currently do. I prefer checking it visually than using a script. At least, now, I have a chance to have it working without a manual intervention. It's good you submitted a patch, I have no problem there, BTW, but applying a patch during beta, should either fix the problem or not be applied at all. Well, I don't think we'll agree on that. Anyway, have a nice day :). -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Clean shutdown and warm standby
On Tue, Apr 28, 2009 at 5:35 PM, Guillaume Smet guillaume.s...@gmail.com wrote: On Tue, Apr 28, 2009 at 5:22 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: At a normal startup, the checkpoint record would be there as usual. And an archive recovery starts at the location indicated by the backup label. AFAICS calling RequestXLogSwitch() before CreateCheckPoint would be equivalent to calling pg_switch_xlog() just before shutting down. That's what I had in mind when writing the patch but I didn't know the implications of this particular checkpoint. So moving the call before CreateCheckPoint is what I really intended now that I have in mind these implications and I don't know why it would be a problem to miss this checkpoint in the logs archived. What do we decide about this problem? Should we just call RequestXLogSwitch() before the creation of the shutdown checkpoint or do we need a more complex patch? If so can anybody explain the potential problem of this approach so we can figure how to fix it? Thanks. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] SQL state in log_line_prefix
Hi -hackers,
A customer of us recently wanted to separate user errors from system
errors as different teams are working on these 2 types of errors and
while they might want to ignore user errors (bad-written SQL queries
and so on), they want to be sure system errors are analyzed carefully.
A good way to differentiate the 2 types of errors is to use the SQL state.
Currently, the only way to have the SQL state in the standard log file
is to do verbose logging which is, well..., verbose.
I attached a patch which allows to add the SQL state code into the
log_line_prefix. I used %e (as in error) as %s is already used.
Any comment?
--
Guillaume
Index: doc/src/sgml/config.sgml
===
RCS file: /projects/cvsroot/pgsql/doc/src/sgml/config.sgml,v
retrieving revision 1.216
diff -c -r1.216 config.sgml
*** doc/src/sgml/config.sgml 27 Apr 2009 16:27:35 - 1.216
--- doc/src/sgml/config.sgml 10 May 2009 12:20:20 -
***
*** 3049,3054
--- 3049,3059
entryno/entry
/row
row
+ entryliteral%e/literal/entry
+ entrySQL state/entry
+ entryno/entry
+ /row
+ row
entryliteral%q/literal/entry
entryProduces no output, but tells non-session
processes to stop at this point in the string; ignored by
Index: src/backend/utils/error/elog.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/utils/error/elog.c,v
retrieving revision 1.213
diff -c -r1.213 elog.c
*** src/backend/utils/error/elog.c 2 Mar 2009 21:18:43 - 1.213
--- src/backend/utils/error/elog.c 10 May 2009 12:20:23 -
***
*** 139,145
} while (0)
! static void log_line_prefix(StringInfo buf);
static void send_message_to_server_log(ErrorData *edata);
static void send_message_to_frontend(ErrorData *edata);
static char *expand_fmt_string(const char *fmt, ErrorData *edata);
--- 139,145
} while (0)
! static void log_line_prefix(StringInfo buf, ErrorData *edata);
static void send_message_to_server_log(ErrorData *edata);
static void send_message_to_frontend(ErrorData *edata);
static char *expand_fmt_string(const char *fmt, ErrorData *edata);
***
*** 1585,1591
* Format tag info for log lines; append to the provided buffer.
*/
static void
! log_line_prefix(StringInfo buf)
{
/* static counter for line numbers */
static long log_line_number = 0;
--- 1585,1591
* Format tag info for log lines; append to the provided buffer.
*/
static void
! log_line_prefix(StringInfo buf, ErrorData *edata)
{
/* static counter for line numbers */
static long log_line_number = 0;
***
*** 1722,1727
--- 1722,1730
case 'x':
appendStringInfo(buf, %u, GetTopTransactionIdIfAny());
break;
+ case 'e':
+ appendStringInfo(buf, %s, unpack_sql_state(edata-sqlerrcode));
+ break;
case '%':
appendStringInfoChar(buf, '%');
break;
***
*** 1978,1984
formatted_log_time[0] = '\0';
! log_line_prefix(buf);
appendStringInfo(buf, %s: , error_severity(edata-elevel));
if (Log_error_verbosity = PGERROR_VERBOSE)
--- 1981,1987
formatted_log_time[0] = '\0';
! log_line_prefix(buf, edata);
appendStringInfo(buf, %s: , error_severity(edata-elevel));
if (Log_error_verbosity = PGERROR_VERBOSE)
***
*** 2002,2036
{
if (edata-detail_log)
{
! log_line_prefix(buf);
appendStringInfoString(buf, _(DETAIL: ));
append_with_tabs(buf, edata-detail_log);
appendStringInfoChar(buf, '\n');
}
else if (edata-detail)
{
! log_line_prefix(buf);
appendStringInfoString(buf, _(DETAIL: ));
append_with_tabs(buf, edata-detail);
appendStringInfoChar(buf, '\n');
}
if (edata-hint)
{
! log_line_prefix(buf);
appendStringInfoString(buf, _(HINT: ));
append_with_tabs(buf, edata-hint);
appendStringInfoChar(buf, '\n');
}
if (edata-internalquery)
{
! log_line_prefix(buf);
appendStringInfoString(buf, _(QUERY: ));
append_with_tabs(buf, edata-internalquery);
appendStringInfoChar(buf, '\n');
}
if (edata-context)
{
! log_line_prefix(buf);
appendStringInfoString(buf, _(CONTEXT: ));
append_with_tabs(buf, edata-context);
appendStringInfoChar(buf, '\n');
--- 2005,2039
{
if (edata-detail_log)
{
! log_line_prefix(buf, edata);
appendStringInfoString(buf, _(DETAIL: ));
append_with_tabs(buf, edata-detail_log);
appendStringInfoChar(buf, '\n');
}
else if (edata-detail)
{
! log_line_prefix(buf, edata);
appendStringInfoString(buf, _(DETAIL: ));
append_with_tabs(buf, edata-detail);
appendStringInfoChar(buf, '\n');
}
if
Re: [HACKERS] SQL state in log_line_prefix
On Sun, May 10, 2009 at 2:32 PM, Guillaume Smet guillaume.s...@gmail.com wrote: I attached a patch which allows to add the SQL state code into the log_line_prefix. I used %e (as in error) as %s is already used. Patch added to the next commit fest page. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL state in log_line_prefix
On Sun, May 10, 2009 at 3:29 PM, Andrew Dunstan and...@dunslane.net wrote: If we're going to do this I think it needs to be added to CSV log output too. AFAICS, it's already done for the CSV output hence my the only way to have the SQL state in the *standard* log file. Did I miss something? Thanks for your feedback. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL state in log_line_prefix
On Sun, May 10, 2009 at 4:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: This might or might not be worth doing, but it seems pretty nearly entirely useless for the claimed purpose. What is your division between user errors and system errors, and how will the SQLSTATE help you make that? We use http://www.postgresql.org/docs/8.3/interactive/errcodes-appendix.html to sort the errors. For example: Class 22 — Data Exception - user Class 53 — Insufficient Resources - system Class XX — Internal Error - system It's not perfect but it's the best way we found to classify them at the moment. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Clean shutdown and warm standby
On Tue, Apr 28, 2009 at 5:22 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: At a normal startup, the checkpoint record would be there as usual. And an archive recovery starts at the location indicated by the backup label. AFAICS calling RequestXLogSwitch() before CreateCheckPoint would be equivalent to calling pg_switch_xlog() just before shutting down. That's what I had in mind when writing the patch but I didn't know the implications of this particular checkpoint. So moving the call before CreateCheckPoint is what I really intended now that I have in mind these implications and I don't why it would be a problem to miss this checkpoint in the logs archived. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Clean shutdown and warm standby
Hi, On Wed, Apr 8, 2009 at 9:11 PM, I wrote: Following the discussion here http://archives.postgresql.org/message-id/49d9e986.8010...@pse-consulting.de , I wrote a small patch which rotates the last XLog file on shutdown [snip] Any comment or advice on how I can fix it with a different method if this one is considered wrong? Original message and patch here: http://archives.postgresql.org/message-id/1d4e0c10904081211p2c0f1cdepe620c11d1271c...@mail.gmail.com Thanks. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New trigger option of pg_standby
On Mon, Apr 13, 2009 at 7:52 AM, Fujii Masao masao.fu...@gmail.com wrote: 1. the trigger file containing smart is created. 2. pg_standby is executed. 2-1. nextWALfile is restored. 2-2. the trigger file is deleted because nextWALfile+1 doesn't exist. 3. the restored nextWALfile is applied. 4. pg_standby is executed again to restore nextWALfile+1. I don't think it should happen. IMHO, it's an acceptable compromise to replay all the WAL files present when I created the trigger file. So if I have the smart shutdown trigger file and I don't have any nextWALfile+1, I can remove the trigger file and stop the recovery: pg_standby won't be executed again after that, even if a nextWALfile+1 appeared while replaying the previous WAL file. That said, stupid question: do we have a way to know the nextWALfile+1 name to test if it exists? nextWALfile is transmitted through the restore_command API and I'm wondering if we can have nextWALfile+1 name without changing the restore_command API. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New trigger option of pg_standby
On Fri, Apr 10, 2009 at 5:47 AM, Fujii Masao masao.fu...@gmail.com wrote: One idea to solve this problem is to tell pg_standby as a command-line argument about whether the trigger file can be removed. That parameter value can be set to 'true' when the last applied record is re-fetched. Though pg_standby is called to restore timeline history files also after that point, the trigger file is already unnecessary (pg_standby doesn't wait for history file). Specifically, if restore_command contains new % option (%e?), it's replaced by the boolean value which indicates whether the trigger file can be deleted. This value is set to 'true' when the startup process re-fetches the last valid record, 'false' otherwise. In smart mode, pg_standby determines whether to delete the trigger file according to that value. Comments? Hmmm, it seems overly complicated but I don't know the code of pg_standby. Or, do you have any better idea? Wouldn't it be possible to have a global switch (let's name it startCluster, default to false) which is set to true when the trigger file is found for the first time? You would then be able to remove the trigger file and let the cluster start by checking this variable. One more time, I don't know the code of pg_standby so it may be a stupid idea. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Clean shutdown and warm standby
On Thu, Apr 9, 2009 at 5:00 AM, Fujii Masao masao.fu...@gmail.com wrote: RequestXLogSwitch() doesn't wait until the switched WAL file has actually been archived. So, some WAL files still may not exist in the standby server also after clean shutdown of the primary. Thanks for your comment. RequestXLogSwitch() doesn't wait for archiving but the shutdown process takes care of it AFAICS. As far as I understand the shutdown code, we have the following sequence (I just explain here the steps involved in the XLog and archiver shutdown): - postmaster.c line 2693: PM_WAIT_BACKENDS state: we start the bgwriter and shut it down. It calls ShutdownXLog which creates the shutdown checkpoint and, with my patch, switch to a new XLog file. Then we are in PM_SHUTDOWN state. - postmaster.c line 2244: the reaper is called for the bgwriter child just shutdown and wakens the archiver one last time: the archive command is executed for our last XLog file. Did I miss something? -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Clean shutdown and warm standby
Hi,
Following the discussion here
http://archives.postgresql.org/message-id/49d9e986.8010...@pse-consulting.de
, I wrote a small patch which rotates the last XLog file on shutdown
so that the archive command is also executed for this file and we are
sure we have all the useful XLog files when we perform a clean
shutdown of master + switch to the failover server. This rotation is
done only if the archive mode is active and an archive command is set.
It's currently really difficult to switch easily (ie without copying
the file manually) to the failover server without any data loss.
Is there any problem I've missed? Could we consider the inclusion of
such a patch or something similar?
Comments?
Regards,
--
Guillaume
Index: src/backend/access/transam/xlog.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/access/transam/xlog.c,v
retrieving revision 1.335
diff -c -r1.335 xlog.c
*** src/backend/access/transam/xlog.c 7 Apr 2009 00:31:26 - 1.335
--- src/backend/access/transam/xlog.c 8 Apr 2009 13:18:40 -
***
*** 5950,5956
--- 5950,5965
if (RecoveryInProgress())
CreateRestartPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
else
+ {
CreateCheckPoint(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_IMMEDIATE);
+ /*
+ * if archiving is active and an archive command is set, we need to
+ * rotate the last XLog file containing useful information so that
+ * the archive command is also executed for it
+ */
+ if(XLogArchivingActive() XLogArchiveCommandSet())
+ RequestXLogSwitch();
+ }
ShutdownCLOG();
ShutdownSUBTRANS();
ShutdownMultiXact();
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New trigger option of pg_standby
On Fri, Apr 3, 2009 at 5:42 AM, Fujii Masao masao.fu...@gmail.com wrote: Here is the patch; - Smart failover is chosen if the trigger file labeled smart or an empty one exists. - Fast failover is chosen if the trigger file labeled fast exists, the signal (SIGUSR1 or SIGINT) is received or the wait timeout happens. After some further thoughts, +1 for this approach too. I think you imply 'containing smart' not 'labeled smart'. Labeled is confusing IMHO. +1 to change the default behaviour too. All the people discovering the current behaviour are totally surprised. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [Snowball-discuss] Snowball release cycle ?
On Wed, Apr 1, 2009 at 11:44 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Where does GPL come into it? (I hadn't seen that mentioned before for either product.) Richard is one of the developers of Snowball so he might want to keep its license compatible with GPL2. Moreover, Snowball is used by a lot of projects, and probably some of them are GPL2. Note that ASL2 is however compatible with GPL3. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 release notes proof reading 1/2
On Sat, Mar 28, 2009 at 11:05 PM, Bruce Momjian br...@momjian.us wrote: Based on mentions of his name in previous release notes, you are correct; change committed. Thanks Bruce. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] display previous query string of idle-in-transaction
2009/3/27 Tatsuhito Kasahara kasahara.tatsuh...@oss.ntt.co.jp: But if I can also check last query string, I guess which apllication do that and point out the problem point. Oh, I just understand why you want this patch. I usually have one database per server so I didn't see your point. Considering this usage, +1 for me. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 open items list
On Fri, Mar 27, 2009 at 2:58 AM, Robert Haas robertmh...@gmail.com wrote: I think we should also boot everything in the pre-existing bugs category, and the first two items from the questions category, which don't seem important enough to worry about at this stage of the game. That would leave us with 14 items, all of which look reasonably relevant and 8.4-related. Comments? FWIW, I posted my comments in the wiki. I tried to post useful information about the status of each item I have an opinion about. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] display previous query string of idle-in-transaction
On Fri, Mar 27, 2009 at 9:07 AM, Simon Riggs si...@2ndquadrant.com wrote: Or take it further back still and think about why idle in transaction occurs at all and fix *that*. Maybe not in Postgres at all, possibly in the driver or even higher up client stack. From my experience, the main reasons for this problem are: - a poorly coded connection pool (a lot of people think they really should write their connection pooling solution themselves...); - a poorly coded application; - bugs in the code. So you really should try to fix them in your application first. That said, I have one case in mind where I wasn't able to fix completely the connection pool and the application and we still encounter idle in transaction connections from time to time. For this sort of case, a timeout would be a nice solution. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New trigger option of pg_standby
On Fri, Mar 27, 2009 at 12:56 PM, Peter Eisentraut pete...@gmx.net wrote: Simon Riggs wrote: If we go with this, I would suggest we make *neither* the default by removing -t, and adopting two new options: something like -f == fast failover, -p == patient failover. -m smart|fast|immediate :-) The advantage of having 2 options (or the ability to put a string value in the trigger file) is that you can choose the behaviour when you need to trigger it (you just have to use the 2 options with 2 different filenames). I don't think it's the case with your proposal. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New trigger option of pg_standby
On Fri, Mar 27, 2009 at 3:38 AM, Fujii Masao masao.fu...@gmail.com wrote: OK, I'll change the patch as Simon suggested; removing -t and adding two new options: -f = fast failover (existing behavior), -p patient failover. Also I'll default the patient failover, so it's performed when the signal (SIGINT or SIGUSR1) is received. I'm wondering if we should consider backpatching this one. Even if the feature works as advertised in the documentation. It's a very surprising behaviour and I'm pretty sure someone will shoot himself in the foot with it, if not already done. Considering backpatching might change the way we want to fix it. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] display previous query string of idle-in-transaction
On Fri, Mar 27, 2009 at 8:27 AM, Guillaume Smet guillaume.s...@gmail.com wrote: 2009/3/27 Tatsuhito Kasahara kasahara.tatsuh...@oss.ntt.co.jp: But if I can also check last query string, I guess which apllication do that and point out the problem point. Oh, I just understand why you want this patch. I usually have one database per server so I didn't see your point. Thinking a bit more about it: the datname column in the pg_stat_activity view gives you the database concerned and usename the user used. So I still don't see your point: you can use different user to distinguish the applications. Moreover, if you're using connection pooling (which is more and more common) and the same user for connecting to the database, you won't be able to know if it's really the last query which causes the problem (from my experience, it's usually not). Being able to detect which application is running which query on the very same database with the very same user seems like something not so obvious and the use case seems to be pretty narrow. And IMHO, even if we suppose you can make the difference between the applications with only one query, you won't be able to limit your investigation to this application. So, in fact, I'm still not convinced. Could you detail a bit more how you plan to use it? -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 open items list
On Fri, Mar 27, 2009 at 4:24 AM, Robert Haas robertmh...@gmail.com wrote: Perhaps so, but again, it's not a new regression, so why should it be considered a blocker for 8.4beta? I agree they shouldn't. You were talking about bumping them to 8.5 which is a totally different thing. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 open items list
On Fri, Mar 27, 2009 at 9:38 AM, Magnus Hagander mag...@hagander.net wrote: That can be argued to just be completing the pg_hba rewrite stuff that happened long before november with the final logical step. I guess if you stretch that definition as well, this could also be an extension to that :) Yes, that was my point. I think it's better to be consistent for all the configuration files. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New trigger option of pg_standby
On Thu, Mar 26, 2009 at 2:51 AM, Fujii Masao masao.fu...@gmail.com wrote: What does the default mean? You mean that new trigger should use the existing trigger option character (-t)? Yes, that's my point. I understand it seems weird to switch the options but I'm pretty sure a lot of persons currently using -t would be surprised by the current behaviour. Moreover playing all the remaining WALs before starting up should be the most natural option when people are looking in the help. That said, it would be nice to hear from people really using pg_standby to know if they understand how it works now and if it's what they intended when they set it up. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] 8.4 release notes proof reading 1/2
Hi, Just to warn people that I'm making a comprehensive proof reading of the release notes. Here are the first comments: - This was available previously via a configure --enable-integer-datetimes (Neil Conway) - I don't think we need Neil's name in the details - New semi- and ansi-joins (Tom) - s/ansi/anti/ - While semi-joins merely replace existing IN joins, anti-joins are a new capability for NOT IN and NOT EXIST clauses (Tom) This improves optimization possibilities. - remove the (Tom), s/EXIST/EXISTS/ - IN and EXISTs now provide similar performance for equivalent clauses. - s/EXISTs/EXISTS/ - A new constraint_exclusion setting, partition, which matches this behavior. - this sentence is weird - Move that server statistics file into the subdirectory pg_stat_tmp - s/that/the/ ? I think this item can be grouped with the following one Allow the location of the new pg_stat_tmp to be configured via the stats_temp_directory setting I'll post the other part in a few hours. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 release notes proof reading 1/2
Bruce, Here is the second set of comments: - pg_hba.conf: it seems to me the format has changed which may break existing pg_hba.conf (it broke the default one of the RPM packaging). We should make it very visible as the format hasn't changed for a while. I suppose we'll put it at the top but I just don't want us to forget it so I mention it here. - Allow recovery.conf boolean variables to take the same range of string alues as postgresql.conf (Bruce) - s/alues/values/ - Add support for Unicode sting and identifier specifications using code points, e.g. U'd\0061t\+61' (Peter) - s/sting/string/ - Allow CREATE OR REPLACE VIEW to add columns to the _end_ of the view - formatting problem with the underscores? - Previously EXPLAIN VERBOSE output an internal representation of the query plan - s/output/outputs/ ? - Autovacuum options uses to be stored in a system table. - s/uses/used/ ? - Allows dumps to fail if unable to acquire shared table within the specified amount of time. - shared lock instead of shared table? - Add configure option --disable-float4-byval to use the old behavior. tgl Pass float8 and int8 by value in the server where possible (Zoltan Boszormenyi) - tgl + there's a carriage return missing (and a bullet) - Add -M (query mode) to /contrib/pgbench (ITAGAKI Takahiro) -Itagaki san's name inconsistent with other mentions of his name I can provide a patch for all this if needed. I thought you'd prefer validate the changes. Really nice work and really interesting reading. Thanks. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New trigger option of pg_standby
Hi Simon. On Thu, Mar 26, 2009 at 11:50 AM, Simon Riggs si...@2ndquadrant.com wrote: Earlier, we discussed having a single trigger file that contains an option rather than two distinct trigger files. That design is better because it allows the user to choose at failover time, rather than making a binding decision at config time. That solution would be the ideal one, IMHO, because it gives user more choice - and would allow us to keep the -t option meaningfully. In that case the default should be patience. Or you can define both files in your command line to have the choice. I like the idea of removing -t and adding 2 new options so that people are warned about the intended behavior. Anyway, I don't have a strong opinion about how we should fix it as I don't use pg_standby personnally, just that we should. The two options you mention have their own merits. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] maintenance_work_mem and autovacuum
On Thu, Mar 26, 2009 at 7:34 PM, Simon Riggs si...@2ndquadrant.com wrote: Why do we have separate parameters for autovacuum and vacuum, except for maintenance_work_mem? Should we also have autovacuum_work_mem? We already discussed it here: http://archives.postgresql.org/message-id/49353a69.20...@hagander.net It resulted in a doc patch - not sure it's sufficient but it's interesting to read this thread before discussing further. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 open items list
On Fri, Mar 27, 2009 at 2:58 AM, Robert Haas robertmh...@gmail.com wrote: That includes a whole slough of patches that weren't submitted until after November 1st and which I think should probably be bumped en masse to 8.5: postgresql.conf: patch to have ParseConfigFile report all parsing errors, then bail Not sure about this one. A similar patch for the pg_hba.conf file submitted after the commit fest (IIRC) has already been commited. pg_standby trigger behavior is dangerous This one has to be fixed IMHO. I'm not sure how but we have to take a decision. The current behaviour is really dangerous for most of our users. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 release notes proof reading 1/2
On Fri, Mar 27, 2009 at 2:44 AM, Bruce Momjian br...@momjian.us wrote: Guillaume Smet wrote: - Add -M (query mode) to /contrib/pgbench (ITAGAKI Takahiro) -Itagaki san's name inconsistent with other mentions of his name Above all fixed, thanks. I think you fixed this one the wrong way. It should be s/ITAGAKI Takahiro/Takahiro Itagaki/g, shouldn't it? -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New trigger option of pg_standby
On Wed, Mar 25, 2009 at 7:29 AM, Fujii Masao masao.fu...@gmail.com wrote: Attached patch fixes the above problem by adding a new trigger option to pg_standby; the presence of this new trigger file causes recovery to end after replaying all the available WAL files. Shouldn't it be the default? It seems like the most expected behaviour to me. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New trigger option of pg_standby
On Wed, Mar 25, 2009 at 9:44 AM, Fujii Masao masao.fu...@gmail.com wrote: Yeah, I agree... but there may be scripts for warm-standby based on the existing default behavior. So, I didn't make a new trigger the default. I don't use pg_standby personnaly but I admit I'm quite surprised by the current behaviour. I'm pretty sure a lot of the current users would be surprised too. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New trigger option of pg_standby
On Wed, Mar 25, 2009 at 2:59 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: I find it hard to imagine a use case for the existing default behavior. I thought a bit about it and I think it can be useful when your priority is the availability of the service and you don't consider a data loss that important: even if you have a lot of WALs segments to replay, you may want to have your service up immediately in case of a major problem. Keeping it is a good idea IMHO but I don't think it should be the default. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] display previous query string of idle-in-transaction
On Wed, Mar 25, 2009 at 5:48 PM, hubert depesz lubaczewski dep...@depesz.com wrote: I would love to get it, but when I suggested it some time in the past Tom shot it down as bad idea. http://archives.postgresql.org/message-id/20071016132131.ga4...@depesz.com I agree with Tom here. I tracked a lot of idle in transaction problems and you need the whole sequence of queries of all the backends to really understand what happens. The last query executed is mostly useless to solve this sort of problem. At least, it was for me in every case I had to deal with. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] contrib function naming, and upgrade issues
On Sun, Mar 22, 2009 at 10:46 PM, Tom Lane t...@sss.pgh.pa.us wrote: There might (or might not) be some use-case for being able to declare that module A depends on module B, Typically, earthdistance requires cube so the module dependency is already something that might be useful. But as you said, it doesn't require a second level of grouping, just a way to define dependencies. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Should SET ROLE inherit config params?
On Fri, Mar 13, 2009 at 2:39 AM, Josh Berkus j...@agliodbs.com wrote: SET ROLE special WITH SETTINGS ... or similar; I'd need to find an existing keyword which works. Perhaps something like SET ROLE special NEW SESSION;. It solves a problem mentioned by Tom as it's very clear that it's a new session so when you reset the settings to what they were at session start, you take the default settings of special. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] small parallel restore optimization
On Fri, Mar 6, 2009 at 6:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: Can you put together even a weakly reproducible test case? Something that only fails every tenth or hundredth time would still help. It seems that Olivier can reproduce the problem at will on Unixware. I don't know if it's easy to find useful information to debug the problem on this platform though. See http://archives.postgresql.org/pgsql-hackers/2009-03/msg00201.php -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [BUGS] BUG #4689: Expanding the length of a VARCHAR column should not induce a table rewrite
On Wed, Mar 4, 2009 at 11:06 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: While this isn't a bug, it's a reasonable feature request. I've added this to the TODO page: http://wiki.postgresql.org/wiki/Todo#ALTER Patches are welcome. I remember someone proposed a patch for that a long time ago: http://archives.postgresql.org/pgsql-patches/2006-10/msg00154.php AFAICS there were 2 possible follow-ups to this patch: - only manage the case of char/varchar (which is the most common case anyway); - develop a complete infrastructure to handle all the cases as suggested by Gregory. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [BUGS] BUG #4689: Expanding the length of a VARCHAR column should not induce a table rewrite
On Wed, Mar 4, 2009 at 11:50 AM, Peter Eisentraut pete...@gmx.net wrote: The question is how you want to implement this in a data type independent fashion. You can't assume that increasing the typmod is a noop for all data types. Sure. See my previous answer on -hackers (I don't think this discussion belong to -bugs) and especially the discussion in the archives about Jonas' patch. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Okay to change TypeCreate() signature in back branches?
On Mon, Feb 23, 2009 at 8:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: There are a number of options at this point, including fixing the problem only in HEAD, fixing back to 8.1 but no further, or making wrapper functions in the back branches to preserve the existing argument lists of heap_create_with_catalog and/or TypeCreate. I'd go for fixing it properly back to 8.1. 8.1 is the oldest version people still put into production with new applications IMHO (due mainly to its inclusion in current versions of RHEL and SLES). I'm not sure it's worth it to backport it to older versions. It seems like a minor problem and older versions I still see in production usually aren't running the latest minor version anyway. I can't measure the time needed to write the wrapper functions though. If it's just a matter of a couple of minutes and it's not risky, perhaps it's better to fix it right now. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules Bernd Helmle
On Sat, Jan 24, 2009 at 1:17 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think that we probably want the rules to show up automatically during an upgrade from an older version I'm really not convinced by that. Is it required by the standard? It's really far from being compliant with the principle of least surprise. Personnally, I don't expect my views to become updatable. There should be an easy way to make a view become updatable but making all of them updatable automagically on upgrade seems weird. Another question related to the choice of explicit rules for the implementation: if we change the way these rules are generated in 8.5, will we upgrade all the existing rules? What if the user modified one of them on purpose? -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Automatic view update rules Bernd Helmle
On Sun, Jan 25, 2009 at 1:07 PM, Bernd Helmle maili...@oopsware.de wrote: I didn't find such a notion in the standard. A view is automatically updatable if it meets the criteria of updatability). If you don't want your view writable, you have to GRANT the necessary ACLs. Perhaps I'm a bit old school on this one but I don't see the point of creating a bunch of rules on every view I create, even if I don't especially want them to be updatable (and I think it's a very common use case - especially because we're used to it). Yes, I can remove them but I don't see the point of going through every view to remove the rules. Especially, creating these rules on upgrade seems really weird as there is no chance the application is using updatable views: they didn't exist in prior versions and if the application is using it, the view already has its own set of rules for that. I originally had the idea of a GUC which controls wether automatic rules will be generated or not. But I abonded this idea, since this has some kind of parametrized SQL standard functionality. I'm more for some syntactical sugar which allows to create view with the updatable property and remove this property from the view. I don't know if it's possible though and it's just MVHO on this subject. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery Test Framework
On Mon, Jan 12, 2009 at 3:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: However, we are getting off onto a tangent. I wasn't trying to start a discussion about general project policies, but about the specific status of this particular group of patches. I concur with Gregory on this one. IM(Very)HO, it's really too late in the cycle to commit these features (ie sync rep and hot standby). They are supposed to guarantee high availability and data security and they must be rock solid. Having them commited just before the release seems to me like a very dangerous way to publish them. A lot of users are waiting for these features so they really should be usable and rock solid before they get released to the public. One more year without them is perhaps better than causing problems on critical databases. Apart from the features themselves, what people expect the most (at least the ones I met) is a replication feature which is simple to set up and integrated. A polished user interface is probably what is the most important from the user point of view (correctness and stability are a minimum). That's what is going to make a difference with what already existed (for the users I know). I'm just handwaving but I think there's probably need for at least one more month to get these patches reviewed and ready to commit (considering there are very few people able to review them and to fix problem in this set of patches). Note that I don't question the quality of the patches, just that there will be very little time to test the final code commited before the release. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery Test Framework
On Mon, Jan 12, 2009 at 4:56 PM, Merlin Moncure mmonc...@gmail.com wrote: I disagree at least with hot standby. I've been using/testing (as have others) it under a variety of workloads for several months now with no issues outside of corrected issues in the very early patches. Also, a relatively few amount of people update/build from cvs frequently so being committed late in the release cycle isn't as important as you are claiming...the real 'wider net' testing happens when the beta period begins. Update/build from CVS != Update/build from CVS + apply the replication patches + test them explicitely. That said, I didn't have the time to test them myself so I feel also responsible for that. My point is that what Simon currently has (and so what you tested) is different from what is going to be commited (note the final in what I wrote) and I suspect there will be a certain number of non negligible adjustments (see the last discussions between Simon and Heikki and I don't think Tom has taken a look at these patches yet). I'm not sure that the beta/rc testing cycle is sufficient for such a critical feature and that we probably need some time to polish it. But once again, it's just MVHO. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery Test Framework
On Mon, Jan 12, 2009 at 5:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: Basically I think we are up against the same type of project management decision we've had several times before: are we willing to slip the 8.4 release schedule for however long it will take for hot standby and the other replication-related features to be ready? At this point I think there can be no question that it will not be a small slip; in fact I'm not even prepared to guess at how long it will take. What I wouldn't like to see is the replication patches becoming another Bitmap index on disk patch. If we release 8.4 and postpone replication to 8.5, we really need a plan to concentrate the efforts of the few people capable of making it happen in the very few months of the 8.5 cycle. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery Test Framework
On Mon, Jan 12, 2009 at 5:48 PM, Dave Page dp...@pgadmin.org wrote: I would. PostgreSQL is not a commercial application which has to be released on schedule to satisfy shareholders - it's an Open Source project that aims to provide it's users with useful features. It has nothing to do with commercial/non commercial. It's basically a decision of time based releases vs features based releases. For many years now, new versions of PostgreSQL have been released on a time based schedule (one version/year in december) even if it was not a strong decision. We have two extremely useful features here (hot standby and sync replication) which together will make this a killer release for many people - we can delay a month or two as required to polish and get them ready for release, or decide we're willing to wait another 12 - 14 months for them to be available for end users. IMHO, if it takes 4 months to have these patches in the tree, it's not worth it: do we accept other patches during this period or not? If so, on which basis, if not how are we going to deal with 3-4 months of patches waiting for review. Note that delaying 8.4 is also delaying the other features of 8.4 which are ready (new FSM, CTE, Windowing functions). Personnaly, integrated replication is by far the most expected feature but I'm not sure it's the case for everyone, especially if they have to wait 3-4 more months. It's really a matter of how far we are of having these patches in their final form (and I mean after reviewing). And as Tom stated, it's currently hard to know. IMHO, Simon's proposal to identify which parts especially needs attention is a very good idea. I really think these patches need a thorough review sooner than later: we won't make it happen by letting Simon write code alone without feedback. After this first review, we should be able to know if it's a matter of 1 month or 3. Without knowing this, it's hard to take a good decision. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] foreign_data test fails with non-C locale
On Fri, Jan 9, 2009 at 5:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: However, the de facto policy is that we try to keep them passing in locales that are used by any of the regular developers. I think it would be useful to have buildfarm members testing in a few common locales. If you define common locales, I can set up as many new animals as needed to cover the locales needed for any branch we'd like to test. Perhaps we should add a parameter to the buildfarm config file so that the buildfarm script can check the locale is accepted and set it directly. Considering that we won't have the locale information in the animal description, it's a good way to have it in the report. Just let me know. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] visibility map - what do i miss?
On Sat, Dec 6, 2008 at 12:53 PM, hubert depesz lubaczewski [EMAIL PROTECTED] wrote: First run - without visibility maps, timing of vacuums: Time: 267844.822 ms Time: 138854.592 ms Time: 305467.950 ms Time: 487133.179 ms Second run - on head: Time: 252218.609 ms Time: 234388.763 ms Time: 334016.413 ms Time: 575698.750 ms Now - as I understand the change - visilibity maps should make second run much faster? If I understand correctly what Heikki explained, not if you run VACUUM only once (and you confirmed me on IRC you run it only once). It's the VACUUM which sets the PD_ALL_VISIBLE flag on the pages so the first VACUUM should be slower with HEAD than with 8.3 as it has far more work to do. The second VACUUM should then be faster. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] default statistics target testing (was: Simple postgresql.conf wizard)
Robert, On Fri, Dec 5, 2008 at 7:24 PM, Greg Smith [EMAIL PROTECTED] wrote: comparing to 8.2.9 results: - q1 was 32% slower with dst = 10, 33% slower with dst = 1000 - other queries were 19% slower with dst = 10, 25% slower with dst = 1000 You mean that HEAD is slower than 8.2.9 or I don't understand correctly what you wrote? -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] default statistics target testing (was: Simple postgresql.conf wizard)
On Fri, Dec 5, 2008 at 8:11 PM, Robert Haas [EMAIL PROTECTED] wrote: Correct. As compared with 8.2.9, ANALYZE was substantially faster, but query planning was significantly slower. Thanks also to Greg for reposting my emails. I didn't even realize I hadn't sent them to the list. Any chance you could do the same test with a 8.3? It could be interesting to see if it's a HEAD thing or if the slow down was introduced in 8.3. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] maintenance memory vs autovac
On Wed, Dec 3, 2008 at 10:49 AM, Magnus Hagander [EMAIL PROTECTED] wrote: The autovacuum workers change that and make it a default behaviour (as we can have 3*maintenance_work_mem by default). It's still one per process, it's just that autovac uses more than one process. I agree. What I implied is that by default you have 3 autovacuum workers so the behaviour has changed, even if it didn't change in a technical way. It's probably worthwhile to add a note about the effects of autovacuum around the documentation of maintenance_work_mem, though. +1 A lot of people set maintenance_work_mem quite high because of the old behaviour. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] maintenance memory vs autovac
On Wed, Dec 3, 2008 at 2:00 AM, Tom Lane [EMAIL PROTECTED] wrote: It seems like mostly a confusion-generator to me. Is there any actual evidence that autovac should use a different maintenance_work_mem than other processes? IMHO, the point is that we were used to consider the maintenance_work_mem as a one process at a time thing. Even if it's not really true, we usually didn't do maintenance task on a concurrent basis. The autovacuum workers change that and make it a default behaviour (as we can have 3*maintenance_work_mem by default). From my point of view, the best solution would be to share the maintenance_work_mem amongst all the workers but I suppose it's not technically possible. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Simple postgresql.conf wizard
Greg, On Mon, Dec 1, 2008 at 3:17 AM, Greg Smith [EMAIL PROTECTED] wrote: ./pgtune -i ~/data/postgresql.conf First, thanks for your work: it will really help a lot of people to have a decent default configuration. A couple of comments from reading the code (I didn't run it yet): - it would be really nice to make it work with Python 2.4 as RHEL 5 is a Python 2.4 thing and it is a very widespread platform out there, - considering the audience of this tool, I think you should explain in the usage text which type of workload implies each database type (DW, OLTP, Web, Mixed, Desktop). - not sure of the wording of The file this needs to operate correctly can be generated with: - it would be nice to be able to define the architecture (32-64 bits) from the command line (especially considering I won't be able to run it on our target boxes which are all RHEL 5 :)) I'll see if I have more feedback while testing it for real. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: Hot standby
On Fri, Nov 28, 2008 at 6:45 PM, Tom Lane [EMAIL PROTECTED] wrote: Hm. People have complained of that fact from time to time in normal usage as well. Should we simply change SIGINT handling to allow it to cancel an idle transaction? If this means that we would be able to able to easily rollback the current transaction of a backend stuck in idle in transaction status, a big +1. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [bugfix] DISCARD ALL does not release advisory locks
On Wed, Nov 26, 2008 at 8:13 PM, Gregory Stark [EMAIL PROTECTED] wrote: DISCARD ALL was specifically added in 8.3 for the purpose of connection poolers to be a big hammer that exactly emulates a new session. I'm somewhat skeptical that there are any applications using it directly at all, and doubly so that they would be using it and expecting advisory locks to persist. I think the second and third points are pretty convincing. I kinda agree with you. The only problem IMHO is that we described in the doc exactly what it does and not simply as the big hammer it was supposed to be. See http://www.postgresql.org/docs/8.3/interactive/sql-discard.html . You can't guarantee that nobody checked the doc to see if it discards advisory locks even if it's highly unlikely. That said, I'm more for the backpatching too. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [bugfix] DISCARD ALL does not release advisory locks
On Wed, Nov 26, 2008 at 9:45 PM, Tom Lane [EMAIL PROTECTED] wrote: Well, the *first* sentence there says it resets the session to its initial state, so it seems to me the intent is clear. But maybe we should alter the second sentence to read, say, This _currently_ has the same effect as ..., thereby making it clear that this is implementation detail and not the controlling definition. +1. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] no XLOG during COPY?
On Thu, Sep 11, 2008 at 9:01 PM, Andrew Dunstan [EMAIL PROTECTED] wrote: Could someone please point me at where this optimization was committed? I'm having trouble locating it. I think it's this one: http://archives.postgresql.org/pgsql-committers/2007-01/msg00296.php -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] migrate data 6.5.3 - 8.3.1
On Fri, Aug 15, 2008 at 9:54 AM, alexander lunyov [EMAIL PROTECTED] wrote: Thanks for the tip. I want to try new pg_dump to connect to old server, but i can't - old postgres doesn't listening to network socket. Why postgres 6.5.3 not binding to network socket? It started with this line: (Forgot the list, sorry for the double email, Alexander) Do you have something like tcpip_socket in your postgresql.conf (I don't know if it was in 6.5)? If so, you have to set it to true to make PostgreSQL listen on the network. As for the upgrade path, I'll go with: - dump with 7.3 pg_dump, insert your dump into a 7.3 db and run adddepends contrib shipped with 7.3 on your db (it should fix the foreign key problem reported by Zdenek); - then dump your 7.3 db with 8.3 pg_dump and insert your dump into your 8.3 server. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] migrate data 6.5.3 - 8.3.1
On Fri, Aug 15, 2008 at 11:42 AM, alexander lunyov [EMAIL PROTECTED] wrote: i didn't find anything like postgresql.conf on old server. Right now i'm tried to start 6.5.3 on windows (downloaded binary from ftp archive on postgresql.org, installed last cygwin) with the data dir from old server, but there's errors: Add the -i option to your startup command line. You should then be able to connect using the network. That said, you should also be able to connect using the unix domain socket using -h /path/to/domain option. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CVS Head psql bug?
On Thu, Jun 26, 2008 at 3:04 AM, Tom Lane [EMAIL PROTECTED] wrote: Okay, it seems fairly close to RHEL-4. Doesn't look like there are any RHEL4 or CentOS4 machines in the buildfarm, which is why we didn't find out about it earlier. FYI, I set up a CentOS 4 vserver on one of our buildfarm boxes. It's called guanaco: http://www.pgbuildfarm.org/cgi-bin/show_history.pl?nm=guanacobr=HEAD -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: reducing overhead for repeat de-TOASTing
Mark, On Tue, Jul 1, 2008 at 1:15 PM, Mark Cave-Ayland [EMAIL PROTECTED] wrote: Thanks very much for supplying the WIP patch. In the interest of testing and feedback, I've just downloaded PostgreSQL CVS head and applied your patch, compiled up a slightly modified version of PostGIS (without RECHECKs on the GiST opclass) and loaded in the same dataset. From the discussion we had a few months ago, I don't think the no RECHECK trick works with CVS tip anymore. See my post on the Remove lossy-operator RECHECK flag? thread: http://archives.postgresql.org/pgsql-hackers/2008-04/msg00847.php and follow-ups. That said, perhaps it's not the only problem you have but I thought it was worth mentioning. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upcoming back-branch update releases
On Wed, May 28, 2008 at 4:10 PM, Tom Lane [EMAIL PROTECTED] wrote: If you've got any bug fixes you've been working on, now is a good time to get them finished up and sent in... Has the s/\x09//g patch for psql from Bruce and you been backported to 8.3? I didn't see it on pgsql-commiters. Regards, -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upcoming back-branch update releases
On Thu, May 29, 2008 at 4:14 PM, Tom Lane [EMAIL PROTECTED] wrote: No, nothing's been done about that AFAIK. What's the consensus, do we want to change that behavior in 8.3.2? IIRC, noone voted against backpatching it after Alvaro and you agreed with doing so. Archives link: http://archives.postgresql.org/pgsql-hackers/2008-05/msg00243.php It would be nice to have it fixed in 8.3.2, even if it's not really blocking for the release. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upcoming back-branch update releases
On Thu, May 29, 2008 at 4:58 PM, Tom Lane [EMAIL PROTECTED] wrote: IIRC I made a few cosmetic cleanups along with the actual bug fix. I'll take a look this afternoon and put it in. Thanks. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] [NOVICE] encoding problems
On Fri, May 9, 2008 at 4:38 AM, Bruce Momjian [EMAIL PROTECTED] wrote: No, we don't change behaviors in back branches unless we get lots of complaints, and we haven't in this case. I suspect it's annoying for a lot of people, just not annoying enough to make them complain about it. I understand your point of view but I really think it's more a regression fix than a behavior change. That said, if nobody is following me on this one, I'll live with it - it's just annoying, not blocking. Thanks for fixing it anyway. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [NOVICE] encoding problems
On Wed, Mar 12, 2008 at 4:35 PM, Tom Lane [EMAIL PROTECTED] wrote: Martijn van Oosterhout [EMAIL PROTECTED] writes: The other alternative is to convert tabs to spaces on output. Can't remember why we didn't do that. Yeah. The idea I had was to invent a parameter specifying the number of spaces a tab should expand to --- setting this to zero would give you the current \x09 behavior. I'm not sure how painful it would be to expand tabs properly in mbprint, but it seems possible. Is there any plan to fix this behaviour in 8.3 branch? For instance, the pg_stat_activity output is far less readable than before when people use tabs to indent the SQL queries in their applications. From my experience, it's pretty common and it's really a regression from the usability point of view. Regards, -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Column storage positions
On Mon, Apr 2, 2007 at 2:44 AM, Phil Currier [EMAIL PROTECTED] wrote: On 4/1/07, Guillaume Smet [EMAIL PROTECTED] wrote: Phil, did you make any progress with your patch? Your results seemed very encouraging and your implementation interesting. IIRC, the problem was that you weren't interested in working on the visual/mysqlish column ordering. As the plan was to decouple column ordering in three different orderings, I don't think it's really a problem if your implementation doesn't support one of them (at least if it doesn't prevent us from having the visual one someday). I haven't done much with it since February, largely because my available free time evaporated. But I do intend to get back to it when I have a chance. But you're right, the storage position stuff I've worked on is completely independent from display positions, and certainly wouldn't prevent that being added separately. Is there any chance you keep us posted with your progress and post a preliminary patch exposing your design choices? This could allow other people to see if there are interesting results with their particular database and workload. Yeah, I'll try to clean things up and post a patch eventually. And if anyone feels like working on the display position piece, let me know; perhaps we could pool our efforts for 8.4. Hi Phil, Did you make any progress on this cleanup? It seems like a good timing to revive this project if we want it for 8.4. Thanks for your feedback. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] -DCLOBBER_CACHE_ALWAYS build takes far longer than before
Hi all, While working on our network problems to send the full build result to the buildfarm server, I just saw that a full buildfarm run with -DCLOBBER_CACHE_ALWAYS option takes nearly 2 hours more than before (from ~ 5h40 to 7h30). This phenomenon has started with this run: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=pigeondt=2008-04-12%2008:00:02 which has taken into account these commits: pgsql/README 1.35 pgsql/contrib/ltree/ltree_io.c 1.15 pgsql/src/backend/parser/parse_type.c 1.95 pgsql/src/backend/utils/adt/tsquery.c 1.17 pgsql/src/backend/utils/adt/arrayfuncs.c 1.143 pgsql/src/backend/utils/adt/geo_ops.c 1.100 pgsql/src/backend/utils/mmgr/aset.c 1.77 pgsql/src/port/dirmod.c 1.53 pgsql/src/include/pg_config_manual.h 1.31 pgsql/src/include/port.h 1.120 pgsql/doc/TODO 1.2415 pgsql/doc/src/FAQ/TODO.html 1.921 The additional time is spread like this: make check: 40 minutes make installcheck: 34 minutes make contrib installcheck: 10 minutes I don't know if it's normal or not but I thought it's worth mentioning. The standard build run on the same box isn't affected. Olivier, do you have the same behaviour with jaguar? -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] -DCLOBBER_CACHE_ALWAYS build takes far longer than before
On Sun, Apr 20, 2008 at 6:02 PM, Tom Lane [EMAIL PROTECTED] wrote: It's probably due to the RANDOMIZE_ALLOCATED_MEMORY overhead added here: pgsql/src/backend/utils/mmgr/aset.c 1.77 Right now that's on by default in assert-enabled builds. I'm not sure if we want to leave it like that for long ... OK. It's not a problem by itself. I just wanted to point it out. Thanks. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove lossy-operator RECHECK flag?
On Fri, Apr 11, 2008 at 7:41 PM, Tom Lane [EMAIL PROTECTED] wrote: Offhand I suspect only the PostGIS project would be affected. Just wanted to point out that I personnally use the capability to remove the RECHECK of PostGIS opclass (I define a similar opclass without the recheck) when I enforce the SRID in the application: the overhead of rechecking the rows returned by the index scan is really noticeable (for nothing if your SRID is the same in all your application and data - that's what I read some time ago, hope it's still true). I don't know how you think PostGIS should fix their opclass after RECHECK removal. But I'd like to have a way to keep the ability to remove the RECHECK on my own if it's possible so I hope it won't be hardcoded in C code somewhere. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove lossy-operator RECHECK flag?
On Fri, Apr 11, 2008 at 9:02 PM, Tom Lane [EMAIL PROTECTED] wrote: To be blunt, that seems like a really bad idea, and I have not the slightest hesitation about breaking your ability to do it. How do you know that the recheck-need corresponds to what you are testing on the application side? From what I read when I did that a few months ago, the recheck was added to provide SRID checking. If you don't have it, you don't have the SRID mismatch error when SRIDs don't match. If there's actually some safe, consistent use for such behavior then I think you need to lobby the PostGIS project to provide access to it. In the general case, there isn't. If you enforce the SRID (using a wrapper/constraints/whatever), there is. After some googling, I finally found the post of Mark Cave-Ayland on postgis-users which made me take this decision: http://www.mail-archive.com/[EMAIL PROTECTED]/msg01206.html Don't know if there is a better way to do it in PostGIS itself but the ability to take this decision for a specific database (or even column) is really convenient. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] Integer datetime by default
On Sun, Mar 30, 2008 at 6:29 PM, Tom Lane [EMAIL PROTECTED] wrote: Neil Conway [EMAIL PROTECTED] writes: Applied to HEAD. At this point it would probably be a good idea if a couple of buildfarm machines were to start testing builds with --disable-integer-datetimes ... any volunteers out there? I set up a new animal called marten with --disable-integer-datetimes. http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=martendt=2008-04-09%2020:45:22 -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] jaguar is failing
On Mon, Mar 31, 2008 at 4:02 PM, Tom Lane [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] writes: Did you commit a patch already, do you want me to rerun the build? Yes; no --- looks like it'll run by itself in an hour anyay. I set up a new box with -DCLOBBER_CACHE_ALWAYS called pigeon so that we now have 2 boxes running this option. The build is scheduled once a day. For the record, it takes nearly 6 hours to run all the tests. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New boxes available for QA
FYI, we (Stefan and I) started a wiki page to organize this effort: http://wiki.postgresql.org/wiki/Performances_QA_testing . Ideas and participation are very welcome. I also described the platform we have here and the usage of each server: http://wiki.postgresql.org/wiki/QA_Platform_hosted_at_Open_Wide_%28France%29 . I started working on it this week-end. I'll update this page as servers are booked/used and when we add more boxes. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New boxes available for QA
On Wed, Apr 2, 2008 at 1:53 AM, Greg Smith [EMAIL PROTECTED] wrote: As far as the other more powerful machines you mentioned go, would need to know a bit more about the disks and disk controller in there to comment about whether those are worth the trouble to integrate. The big missing piece of community hardware that remains elusive would be a system with Here we go: - a couple of 1750 servers: dual Xeon 2.8 boxes with PERC 4/DI, 2 internal disks, from 2 to 3 GB of RAM, we can probably get one of them up to 4 GB if needed - a PV 220 S disk array with: 4 x 36 GB + 5 x 73 GB. I think I can get 8 identical disks in the box by switching the 73 GB disks with the 36 GB ones from the other boxes but I'm not sure we can make only one RAID array from the 2 parts of the PV 220 S. - one of the above boxes also has a PERC 4/DC and is connected to the disk array. - a 6650 box: quad Xeon MP 2.2 with 4 GB: it has 2 internal disks and an external attachment to the disk array. All the disks are 10k rpm. What I was thinking about is that it can be useful to have several boxes connected to validate features too, not only performances (who says read access to a warm standby?). Note that if we don't find any good usage for them, it won't be a problem to affect them to our internal test platform. If everything goes well, we plan to buy a big box for internal PostgreSQL benchmarking and testing. It's obvious we won't use it night and day so I may be able to provide windows of time when the community can use it. This one is hypothetical though, the other ones are real and dedicated to community usage (yeah, it wasn't an April's fool). -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] New boxes available for QA
-hackers, As I announced it a couple of months ago, apart from the boxes donated to PostgreSQLFr (affected to the web team IIRC), Continuent also donated 7 servers and a Gb/s switch to us for QA testing. It took some time to set them up but they're now up and running and available. These servers are available 24/7 to PostgreSQL QA and won't be used for other purposes. The servers are mostly P4 2.8-3 GHz with 512 to 3 GB of RAM and SATA disk(s) and they are running CentOS 5. The purposes I had in mind when I asked these servers to Robert Hodges were to use them for: - running buildfarm animals with unusual options: perhaps another box with -DCLOBBER_CACHE_ALWAYS or the recent --disable-integer-datetimes option and virtually any future options we'd like to test (Tom, any thoughts?) - feel free to ask and I can give people access if they want to be able to set up the animals by themselves (Andrew?); - running benchfarm clients the day we'll have a benchfarm; - give (well-known) people of the community who don't have access to several servers the ability to perform tests on this platform (depending on how many servers we dedicate to the 2 above points). I'm open to any suggestions as they are really here to serve the community and I'd really like to use them for any sort of QA possible. Concerning the second point, I wonder if it's not worth it to have a very simple thing already reporting results as the development cycle for 8.4 has already started (perhaps several pgbench unit tests testing various type of queries with a daily tree). Thoughts? The good news is that we will add a couple of new boxes to this platform soon. These new servers are dual Xeon boxes with more than 2GB RAM (from 2 to 4) and SCSI/SAS disks. We also have a quad Xeon MP 2.2 GHz box and a quad Xeon MP 700 Mhz which may be affected to the project if we really need them (I know sometimes people are looking for slow multi processors boxes so the quad Xeon 700 box may be a good choice) - they are huge 6U boxes so if we don't need them for specific purposes, I prefer affecting 1U boxes to the community. If we need them, it's the good moment to ask for them. The new boxes are donated by Cityvox. All these boxes are hosted in Villeurbanne, France by Open Wide, the company I work for. I'm looking forward to your comments and ideas. Regards, -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [JDBC] Re: [HACKERS] How embarrassing: optimization of a one-shot query doesn't work
On Tue, Apr 1, 2008 at 8:06 AM, Tom Lane [EMAIL PROTECTED] wrote: Yeah, the lack of any formal testing of the extended-Query protocol is a real problem. I'm not sure of a good fix, but it bears some thinking about. Not only do we not have an automated way to notice if we broke functionality, but we don't really notice for either extended or basic protocol if we hurt performance. I just posted something to -hackers about the availability of boxes for QA purposes. It doesn't solve the problem by itself though. A good answer is probably to plan optional JDBC benchmarks in the benchfarm design - not all people want to run Java on their boxes but we have servers of our own to do so. Andrew? -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [JDBC] Re: [HACKERS] How embarrassing: optimization of a one-shot query doesn't work
On Wed, Apr 2, 2008 at 2:05 AM, Greg Smith [EMAIL PROTECTED] wrote: I'm not sure if all of those changes are net positive for PostgreSQL though, they weren't last time I played with this. I fixed most of the bugs of JDBCBench I found when I benchmarked Sequoia a long time ago. Totally forgot about it. I'll see if I can find the sources somewhere. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New boxes available for QA
On Wed, Apr 2, 2008 at 1:53 AM, Greg Smith [EMAIL PROTECTED] wrote: The pgbench-tools utilities I was working on at one point anticipated this sort of test starting one day. You can't really get useful results out of pgbench without running it enough times that you get average or median values. I dump everything into a results database which can be separated from the databases used for running the test, and then it's easy to compare day to day aggregate results across different query types. I already used your pgbench tools but I just used the ability to draw graphs with gnuplot, I didn't test the database thing. I haven't had a reason to work on that recently, but if you've got a semi-public box ready for benchmarks now I do. Won't be able to run any serious benchmarks on the systems you described, but should be great for detecting basic regressions and testing less popular compile-time options as you describe. Yeah, that's exactly what they are for. As far as the other more powerful machines you mentioned go, would need to know a bit more about the disks and disk controller in there to comment about whether those are worth the trouble to integrate. The big missing piece of community hardware that remains elusive would be a system with =4 cores, =8GB RAM, and =8 disks with a usable write-caching controller in it. All the other boxes are Dell boxes (1750/1850/2950/6850) with PERC 4 or 5 depending on the servers. Two of them have external attachments to a disk array but it's an old one with 2 separated arrays (4 disks + 5 disks IIRC). They aren't big beasts but I think they can be useful to hackers who don't have any hardware fully available and also run more serious continuous tests than the other ones. I'll post the specs of the servers that may be fully available for community purposes tomorrow. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New boxes available for QA
On Tue, Apr 1, 2008 at 3:29 PM, Stephen Frost [EMAIL PROTECTED] wrote: I'm almost done scripting up everything to load the TIGER/Line Shapefiles from the US Census into PostgreSQL/PostGIS. Once it's done and working I would be happy to provide it to whomever asks, and it might be an interesting data set to load/query and look at benchmarks with. There's alot of GIST index creation, as well as other indexes like soundex(), and I'm planning to use partitioning of some sort for the geocoder. We could, for example, come up with some set of arbitrary addresses to geocode and see what the performance of that is. It's just a thought, and it's a large/real data set to play with. I must admit that the first step I want to be achieved is to have the most simple regression tests running on a daily basis. A real database with advanced features can be very interesting for the future. I'm not sure loading the full database will provide useful results on this hardware but we can always work on a subset of it. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [JDBC] Re: [HACKERS] How embarrassing: optimization of a one-shot query doesn't work
On Wed, Apr 2, 2008 at 2:53 AM, Dave Cramer [EMAIL PROTECTED] wrote: I for one would be very interested in the JDBCBench code. OK, I didn't make anything fancy, I just fixed the problem I encountered when profiling Sequoia (I mostly used it as an injector). I'll post the code tomorrow if I can find it somewhere (I lost a couple of disks and I don't remember the box I used to run the tests). -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [JDBC] Re: [HACKERS] How embarrassing: optimization of a one-shot query doesn't work
On Tue, Apr 1, 2008 at 7:35 AM, Tom Lane [EMAIL PROTECTED] wrote: Well, it's not like we have never before changed internal APIs in a minor update. (There have been security-related cases where we gave *zero* notice of such changes.) Nor am I willing to surrender the option to do so again. If there's somebody out there with a real problem with this change, they need to speak up. +1 to fix it ASAP. A lot of people usually wait for 8.x.1 before considering a migration and people are usually quite cautious with the 8.3 migration because of all the cast errors in the existing app. Another question is how we can be sure it doesn't happen again. The easiest way to test this is probably to have a JDBC test testing this exact feature in the future benchfarm. Any comment? -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Ideas input sought for this year's SOC page
On Wed, Mar 12, 2008 at 7:52 PM, Merlin Moncure [EMAIL PROTECTED] wrote: the read only standby would be what i'd most like to see. +1 for this one. Working on setting up a benchfarm could be a good idea too. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Maximum statistics target
On Mon, Mar 10, 2008 at 11:36 AM, Peter Eisentraut [EMAIL PROTECTED] wrote: The time to analyze is also quite constant, just before you run out of memory. :) The MaxAllocSize is the limiting factor in all this. In my example, statistics targets larger than about 80 created pg_statistic rows that would have been larger than 1GB, so they couldn't be stored. From my experience on real life examples, the time to analyze is far from being constant when you raise the statistics target but it may be related to the schema of our tables. cityvox=# \timing Timing is on. cityvox=# show default_statistics_target ; default_statistics_target --- 10 (1 row) Time: 0.101 ms cityvox=# ANALYZE evenement; ANALYZE Time: 406.069 ms cityvox=# ANALYZE evenement; ANALYZE Time: 412.355 ms cityvox=# set default_statistics_target = 30; SET Time: 0.165 ms cityvox=# ANALYZE evenement; ANALYZE Time: 1419.161 ms cityvox=# ANALYZE evenement; ANALYZE Time: 1381.754 ms cityvox=# set default_statistics_target = 100; SET Time: 1.853 ms cityvox=# ANALYZE evenement; ANALYZE Time: 5211.785 ms cityvox=# ANALYZE evenement; ANALYZE Time: 5178.764 ms That said I totally agree that it's not a good idea to have a strict maximum value if we haven't technical reasons for that. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] insert ... delete ... returning ... ?
Hi Mark, On Sun, Feb 24, 2008 at 10:21 PM, Mark Mielke [EMAIL PROTECTED] wrote: I'm at PostgreSQL 8.3 for my production database and everything is working great. I had no problems converting free text search from 8.2 to 8.3, and I really like the improvements. I tried using insert ... delete ... return ... and get a syntax error: pccyber=# insert into product_manufacturer_archived (itemno, manufacturer_id) I would expect a semicolon here. pccyber-# delete from product_manufacturer pccyber-# where not exists (select * from icitem pccyber(# where icitem.itemno = product_manufacturer.itemno and pccyber(# not inactive) pccyber-# returning itemno, manufacturer_id; ERROR: syntax error at or near delete LINE 2: delete from product_manufacturer ^ -- Guillaume ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] insert ... delete ... returning ... ?
On Sun, Feb 24, 2008 at 10:39 PM, Guillaume Smet [EMAIL PROTECTED] wrote: On Sun, Feb 24, 2008 at 10:21 PM, Mark Mielke [EMAIL PROTECTED] wrote: I'm at PostgreSQL 8.3 for my production database and everything is working great. I had no problems converting free text search from 8.2 to 8.3, and I really like the improvements. I tried using insert ... delete ... return ... and get a syntax error: pccyber=# insert into product_manufacturer_archived (itemno, manufacturer_id) I would expect a semicolon here. Mmmmh, my bad, I missed your point. Sorry for the noise :). -- Guillaume ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] PostgreSQL 8.4 development plan
On Feb 6, 2008 9:56 AM, Dave Page [EMAIL PROTECTED] wrote: Whenever a commit fest is in progress, the focus will shift from development to review, feedback and commit of patches. Each fest will continue until all patches in the queue have either been committed to the CVS repository, returned to the author for additional work, or rejected outright, and until that has happened, no new patches will be considered. If we don't have a bench farm anytime soon, I think we should consider planning a set of benchmarks after each commit fest to prevent performance regressions on different workloads. I don't expect them to be comprehensive but it could allow us to prevent the most obvious regressions. -- Guillaume ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
