Re: [HACKERS] List traffic

2010-05-15 Thread Marc G. Fournier
On Sat, 15 May 2010, Jaime Casanova wrote: On Fri, May 14, 2010 at 8:39 AM, Marc G. Fournier scra...@hub.org wrote: And IMHO, that is as much a fault of the 'old timers' on the lists as the newbies ... if nobody redirects / loosely enforces the mandates of the various lists, newbies aren't

Re: [HACKERS] List traffic

2010-05-15 Thread Rob Wultsch
Linux has *as many if not more* ... MySQL, if memory servers, has a half dozen or more ... etc ... MySQL has a bunch of lists, none of which get much traffic. Honestly, they should probably be combined. -- Rob Wultsch -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] pg_upgrade code questions

2010-05-15 Thread Heikki Linnakangas
Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Thu, May 13, 2010 at 5:06 PM, Bruce Momjian br...@momjian.us wrote: I have added SGML comments to comment out the text that mentions EDB Advanced Server. Is that enough? Should I remove the text from the SGML? Should I move it

Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle

2010-05-15 Thread Rob Wultsch
On Fri, May 14, 2010 at 7:32 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Oracle, and all other MVCC databases I've read about outside of PostgreSQL, use an update in place with a rollback log technique. Have you looked at PBXT (which is explicitly NOT SERIALIZABLE)? -- Rob Wultsch

Re: [HACKERS] List traffic

2010-05-15 Thread Marc G. Fournier
[redirected to -chat] On Fri, 14 May 2010, Rob Wultsch wrote: Linux has *as many if not more* ... MySQL, if memory servers, has a half dozen or more ... etc ... MySQL has a bunch of lists, none of which get much traffic. Honestly, they should probably be combined. Except, when you do

Re: [HACKERS] Synchronous replication patch built on SR

2010-05-15 Thread Heikki Linnakangas
BTW, What I'd like to see as a very first patch first is to change the current poll loops in walreceiver and walsender to, well, not poll. That's a requirement for synchronous replication, is very useful on its own, and requires a some design and implementation effort to get right. It would be

Re: [HACKERS] predefined macros for various BSD-based systems?

2010-05-15 Thread Giles Lean
Tom Lane t...@sss.pgh.pa.us wrote: I suppose that at least some of the *BSD herd really do predefine some of the symbols being attributed to them here, but I would like to see something authoritative about which and what. Documentation follows, but first the summary: FreeBSD: __FreeBSD__

Re: [HACKERS] predefined macros for various BSD-based systems?

2010-05-15 Thread Bruce Momjian
Peter Eisentraut wrote: On l?r, 2010-05-15 at 00:23 -0400, Robert Haas wrote: It's a commercial distribution of BSD. I remember it being pretty nice when I used it 10+ years ago, but it sounds like it's dead now. BSDI is the company that produced BSD/OS, which was Bruce's main

Re: [HACKERS] predefined macros for various BSD-based systems?

2010-05-15 Thread Andrew Dunstan
Bruce Momjian wrote: Peter Eisentraut wrote: On l?r, 2010-05-15 at 00:23 -0400, Robert Haas wrote: It's a commercial distribution of BSD. I remember it being pretty nice when I used it 10+ years ago, but it sounds like it's dead now. BSDI is the company that produced BSD/OS,

Re: [HACKERS] predefined macros for various BSD-based systems?

2010-05-15 Thread Bruce Momjian
Tom Lane wrote: I'm not even too sure what bsdi is, but I'm suspicious of that branch too. A search of our code finds contrib/pg_upgrade/file.c: 248: #elif defined(freebsd) || defined(bsdi) || defined(__darwin__) || defined(openbsd) src/backend/utils/misc/ps_status.c: 67: #elif

Re: [HACKERS] predefined macros for various BSD-based systems?

2010-05-15 Thread Bruce Momjian
Giles Lean wrote: Tom Lane t...@sss.pgh.pa.us wrote: I suppose that at least some of the *BSD herd really do predefine some of the symbols being attributed to them here, but I would like to see something authoritative about which and what. Documentation follows, but first the

Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle

2010-05-15 Thread Kevin Grittner
Rob Wultsch wrote: Have you looked at PBXT (which is explicitly NOT SERIALIZABLE)? I hadn't heard of it; so I took a quick look based on your post. It seems to a new engine to use with MySQL which has MVCC without a rollback log, so it presumably uses techniques at least vaguely similar to

Re: [HACKERS] How to know killed by pg_terminate_backend

2010-05-15 Thread Tatsuo Ishii
Seems reasonable. Does the victim backend currently know why it has been killed? I don't think so. One idea is postmaster sets a flag in the shared memory area indicating it rceived SIGTERM before forwarding the signal to backends. Backend check the flag and if it's not set, it

Re: [HACKERS] HS/SR Assert server crash

2010-05-15 Thread Simon Riggs
On Thu, 2010-05-13 at 18:01 -0400, Bruce Momjian wrote: I was able to easily crash the standby server today just by starting it and connecting to it via psql. The master was idle. The failure was: LOG: streaming replication successfully connected to primary TRAP:

[HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Simon Riggs
Patch adds a keepalive message to ensure max_standby_delay is useful. No WAL format changes, no libpq changes. Just an additional message type for the streaming replication protocol, sent once per main loop in WALsender. Plus docs. Comments? -- Simon Riggs www.2ndQuadrant.com diff

Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle

2010-05-15 Thread Rob Wultsch
On Sat, May 15, 2010 at 4:09 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote:  Anything in particular you wanted me to notice about it besides that? Nope. It was just a counter point to your previous comment. -- Rob Wultsch wult...@gmail.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] List traffic

2010-05-15 Thread Rob Wultsch
On Fri, May 14, 2010 at 11:50 PM, Rob Wultsch wult...@gmail.com wrote: Linux has *as many if not more* ... MySQL, if memory servers, has a half dozen or more ... etc ... MySQL has a bunch of lists, none of which get much traffic. Honestly, they should probably be combined. -- Rob Wultsch

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Patch adds a keepalive message to ensure max_standby_delay is useful. The proposed placement of the keepalive-send is about the worst it could possibly be. It needs to be done right before pq_flush to ensure minimum transfer delay. Otherwise any

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Simon Riggs
On Sat, 2010-05-15 at 11:45 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: Patch adds a keepalive message to ensure max_standby_delay is useful. The proposed placement of the keepalive-send is about the worst it could possibly be. It needs to be done right before pq_flush

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Heikki Linnakangas
Simon Riggs wrote: On Sat, 2010-05-15 at 11:45 -0400, Tom Lane wrote: I'm also extremely dubious that it's a good idea to set recoveryLastXTime from this. Using both that and the timestamps from the wal log flies in the face of everything I remember about control theory. We should be doing

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Simon Riggs
On Sat, 2010-05-15 at 19:30 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: On Sat, 2010-05-15 at 11:45 -0400, Tom Lane wrote: I'm also extremely dubious that it's a good idea to set recoveryLastXTime from this. Using both that and the timestamps from the wal log flies in the face of

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Heikki Linnakangas
Simon Riggs wrote: On Sat, 2010-05-15 at 19:30 +0300, Heikki Linnakangas wrote: Doesn't feel right to me either. If you want to expose the keepalive-time to queries, it should be a separate field, something like lastMasterKeepaliveTime and a pg_last_master_keepalive() function to read it.

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Simon Riggs
On Sat, 2010-05-15 at 20:05 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: On Sat, 2010-05-15 at 19:30 +0300, Heikki Linnakangas wrote: Doesn't feel right to me either. If you want to expose the keepalive-time to queries, it should be a separate field, something like

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Simon Riggs
On Sat, 2010-05-15 at 18:24 +0100, Simon Riggs wrote: I will recode using that concept. There's some behaviours that aren't helpful here: Startup gets new pointer when it runs out of data to replay. That might or might not include an updated keepalive timestamp, since there's no exact

Re: [HACKERS] recovery consistent != hot standby

2010-05-15 Thread Robert Haas
On Fri, May 14, 2010 at 5:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: PM_RECOVERY_CONSISTENT - PM_HOT_STANDBY PMSIGNAL_RECOVERY_CONSISTENT - PMSIGNAL_BEGIN_HOT_STANDBY +1.  From the point of view of the postmaster, whether the state transition happens

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Heikki Linnakangas
Simon Riggs wrote: WALSender sleeps even when it might have more WAL to send, it doesn't check it just unconditionally sleeps. At least WALReceiver loops until it has no more to receive. I just can't imagine why that's useful behaviour. Good catch. That should be fixed. I also note that

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Simon Riggs wrote: WALSender sleeps even when it might have more WAL to send, it doesn't check it just unconditionally sleeps. At least WALReceiver loops until it has no more to receive. I just can't imagine why that's useful behaviour. Good catch. That should be

Re: [HACKERS] [PATCH] Add SIGCHLD catch to psql

2010-05-15 Thread David Fetter
On Fri, May 14, 2010 at 04:24:43PM -0400, Bruce Momjian wrote: Stephen Frost wrote: -- Start of PGP signed section. Greetings, Toying around with FETCH_COUNT today, I discovered that it didn't do the #1 thing I really wanted to use it for- query large tables without having to

Re: [HACKERS] [PATCH] Add SIGCHLD catch to psql

2010-05-15 Thread Robert Haas
On Sat, May 15, 2010 at 7:46 PM, David Fetter da...@fetter.org wrote:   Anyway, this makes FETCH_COUNT alot more useful, and, in my view, the   current behaviour of completely ignoring $PAGER exiting is a bug. Plesae add this to the next commit-fest:      

[HACKERS] Unexpected page allocation behavior on insert-only tables

2010-05-15 Thread Michael Renner
While preparing a replication test setup with 9.0beta1 I noticed strange page allocation patterns which Andrew Gierth found interesting enough to report here. I've written a simple tool to generate traffic on a database [1], which did about 30 TX/inserts per second to a table. Upon inspecting

Re: [HACKERS] [PATCH] Add SIGCHLD catch to psql

2010-05-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sat, May 15, 2010 at 7:46 PM, David Fetter da...@fetter.org wrote: Wouldn't this count as a bug fix? Possibly, but changes to signal handlers are pretty global and can sometimes have surprising side effects. I'm all in favor of someone reviewing

Re: [HACKERS] Unexpected page allocation behavior on insert-only tables

2010-05-15 Thread Tom Lane
Michael Renner michael.ren...@amd.co.at writes: I've written a simple tool to generate traffic on a database [1], which did about 30 TX/inserts per second to a table. Upon inspecting the data in the table, I noticed the expected grouping of tuples which came from a single backend to

Re: [HACKERS] Unexpected page allocation behavior on insert-only tables

2010-05-15 Thread Michael Renner
On 16.05.2010 02:16, Tom Lane wrote: Michael Rennermichael.ren...@amd.co.at writes: I've written a simple tool to generate traffic on a database [1], which did about 30 TX/inserts per second to a table. Upon inspecting the data in the table, I noticed the expected grouping of tuples which came

Re: [HACKERS] [PATCH] Add SIGCHLD catch to psql

2010-05-15 Thread Bruce Momjian
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Sat, May 15, 2010 at 7:46 PM, David Fetter da...@fetter.org wrote: Wouldn't this count as a bug fix? Possibly, but changes to signal handlers are pretty global and can sometimes have surprising side effects. I'm all in favor

[HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Tom Lane
I noticed by accident that there are some cases where the planner fails to inline the SQL functions that underlie the || operator for text vs non-text cases. The reason is that these functions are marked immutable, but their expansion involves a coercion to text that might not be immutable. The

[HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Bruce Momjian
FYI, I test pg_upgrade by loading the old cluster's regression database from a pg_dump output file, then after the upgrade, I dump the regression database of the new cluster and diff the changes. The problem I just encountered is that pg_dump uses extra_float_digits=-3 for 9.0, while previous

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Andrew Dunstan
Bruce Momjian wrote: FYI, I test pg_upgrade by loading the old cluster's regression database from a pg_dump output file, then after the upgrade, I dump the regression database of the new cluster and diff the changes. The problem I just encountered is that pg_dump uses extra_float_digits=-3

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Bruce Momjian
Andrew Dunstan wrote: Bruce Momjian wrote: FYI, I test pg_upgrade by loading the old cluster's regression database from a pg_dump output file, then after the upgrade, I dump the regression database of the new cluster and diff the changes. The problem I just encountered is that

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Bruce Momjian wrote: FYI, I test pg_upgrade by loading the old cluster's regression database from a pg_dump output file, then after the upgrade, I dump the regression database of the new cluster and diff the changes. The problem I just encountered

Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Jaime Casanova
On Sat, May 15, 2010 at 8:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: Is it reasonable to fix this now, and if so should I bump catversion or leave it alone?  My own preference is to fix it in pg_proc.h but not touch catversion; but you could argue that different ways. are you planning to

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Bruce Momjian wrote: FYI, I test pg_upgrade by loading the old cluster's regression database from a pg_dump output file, then after the upgrade, I dump the regression database of the new cluster and diff the changes. The

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Bruce Momjian
Andrew Dunstan wrote: The problem I just encountered is that pg_dump uses extra_float_digits=-3 for 9.0, while previous releases used '2'. I had to do hack each server version to get a dump output that would match without rounding errors --- it did eventually work and validated.

Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Tom Lane
Jaime Casanova ja...@2ndquadrant.com writes: On Sat, May 15, 2010 at 8:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: Is it reasonable to fix this now, and if so should I bump catversion or leave it alone?  My own preference is to fix it in pg_proc.h but not touch catversion; but you could argue

Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Jaime Casanova
On Sat, May 15, 2010 at 10:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jaime Casanova ja...@2ndquadrant.com writes: On Sat, May 15, 2010 at 8:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: Is it reasonable to fix this now, and if so should I bump catversion or leave it alone?  My own preference is to

Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Tom Lane
Jaime Casanova ja...@2ndquadrant.com writes: On Sat, May 15, 2010 at 10:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jaime Casanova ja...@2ndquadrant.com writes: On Sat, May 15, 2010 at 8:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: Is it reasonable to fix this now, and if so should I bump

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Andrew Dunstan
Bruce Momjian wrote: Maybe I have misunderstood. How exactly is the server version being hacked here? I know it's only for testing, but it still seems to me that lying to a program as heavily version dependant as pg_dump is in general a bad idea. The code in pg_dump 9.0 is: /*

Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Jaime Casanova
On Sat, May 15, 2010 at 10:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jaime Casanova ja...@2ndquadrant.com writes: On Sat, May 15, 2010 at 10:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jaime Casanova ja...@2ndquadrant.com writes: On Sat, May 15, 2010 at 8:51 PM, Tom Lane t...@sss.pgh.pa.us