Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Heikki Linnakangas
Tom Lane wrote: > So I think that labeling textanycat/anytextcat as immutable was a > thinko, and we instead ought to label them volatile so that the planner > can inline them no matter what the behavior of the underlying text cast > is. That feels backwards, having to label functions as more vola

Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Jaime Casanova
On Sat, May 15, 2010 at 10:21 PM, Tom Lane wrote: > Jaime Casanova writes: >> On Sat, May 15, 2010 at 10:16 PM, Tom Lane wrote: >>> Jaime Casanova writes: On Sat, May 15, 2010 at 8:51 PM, Tom Lane wrote: Is it reasonable to fix this now, and if so should I bump catversion or lea

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Andrew Dunstan
Bruce Momjian wrote: Maybe I have misunderstood. How exactly is the server version being hacked here? I know it's only for testing, but it still seems to me that lying to a program as heavily version dependant as pg_dump is in general a bad idea. The code in pg_dump 9.0 is: /*

Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Tom Lane
Jaime Casanova writes: > On Sat, May 15, 2010 at 10:16 PM, Tom Lane wrote: >> Jaime Casanova writes: >>> On Sat, May 15, 2010 at 8:51 PM, Tom Lane wrote: >>> Is it reasonable to fix this now, and if so should I bump catversion >>> or leave it alone?  My own preference is to fix it in pg_proc.h

Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Jaime Casanova
On Sat, May 15, 2010 at 10:16 PM, Tom Lane wrote: > Jaime Casanova writes: >> On Sat, May 15, 2010 at 8:51 PM, Tom Lane wrote: >>> Is it reasonable to fix this now, and if so should I bump catversion >>> or leave it alone?  My own preference is to fix it in pg_proc.h but >>> not touch catversion

Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Tom Lane
Jaime Casanova writes: > On Sat, May 15, 2010 at 8:51 PM, Tom Lane wrote: >> Is it reasonable to fix this now, and if so should I bump catversion >> or leave it alone?  My own preference is to fix it in pg_proc.h but >> not touch catversion; but you could argue that different ways. > are you pl

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Bruce Momjian
Andrew Dunstan wrote: > >>> The problem I just encountered is that pg_dump uses > >>> extra_float_digits=-3 for 9.0, while previous releases used '2'. I had > >>> to do hack each server version to get a dump output that would match > >>> without rounding errors --- it did eventually work and valid

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan writes: Bruce Momjian wrote: FYI, I test pg_upgrade by loading the old cluster's regression database from a pg_dump output file, then after the upgrade, I dump the regression database of the new cluster and diff the changes. The problem I just encounte

Re: [HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Jaime Casanova
On Sat, May 15, 2010 at 8:51 PM, Tom Lane wrote: > > Is it reasonable to fix this now, and if so should I bump catversion > or leave it alone?  My own preference is to fix it in pg_proc.h but > not touch catversion; but you could argue that different ways. > are you planning to backpatch this? if

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Tom Lane
Andrew Dunstan writes: > Bruce Momjian wrote: >> FYI, I test pg_upgrade by loading the old cluster's regression database >> from a pg_dump output file, then after the upgrade, I dump the >> regression database of the new cluster and diff the changes. >> >> The problem I just encountered is that p

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Bruce Momjian
Andrew Dunstan wrote: > > > Bruce Momjian wrote: > > FYI, I test pg_upgrade by loading the old cluster's regression database > > from a pg_dump output file, then after the upgrade, I dump the > > regression database of the new cluster and diff the changes. > > > > The problem I just encountered i

Re: [HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Andrew Dunstan
Bruce Momjian wrote: FYI, I test pg_upgrade by loading the old cluster's regression database from a pg_dump output file, then after the upgrade, I dump the regression database of the new cluster and diff the changes. The problem I just encountered is that pg_dump uses extra_float_digits=-3 for

[HACKERS] pg_upgrade and extra_float_digits

2010-05-15 Thread Bruce Momjian
FYI, I test pg_upgrade by loading the old cluster's regression database from a pg_dump output file, then after the upgrade, I dump the regression database of the new cluster and diff the changes. The problem I just encountered is that pg_dump uses extra_float_digits=-3 for 9.0, while previous rele

[HACKERS] Performance problem in textanycat/anytextcat

2010-05-15 Thread Tom Lane
I noticed by accident that there are some cases where the planner fails to inline the SQL functions that underlie the || operator for text vs non-text cases. The reason is that these functions are marked immutable, but their expansion involves a coercion to text that might not be immutable. The p

Re: [HACKERS] [PATCH] Add SIGCHLD catch to psql

2010-05-15 Thread Bruce Momjian
Tom Lane wrote: > Robert Haas writes: > > On Sat, May 15, 2010 at 7:46 PM, David Fetter wrote: > >> Wouldn't this count as a bug fix? > > > Possibly, but changes to signal handlers are pretty global and can > > sometimes have surprising side effects. I'm all in favor of someone > > reviewing th

Re: [HACKERS] Unexpected page allocation behavior on insert-only tables

2010-05-15 Thread Michael Renner
On 16.05.2010 02:16, Tom Lane wrote: Michael Renner writes: I've written a simple tool to generate traffic on a database [1], which did about 30 TX/inserts per second to a table. Upon inspecting the data in the table, I noticed the expected grouping of tuples which came from a single backend to

Re: [HACKERS] Unexpected page allocation behavior on insert-only tables

2010-05-15 Thread Tom Lane
Michael Renner writes: > I've written a simple tool to generate traffic on a database [1], which > did about 30 TX/inserts per second to a table. Upon inspecting the data > in the table, I noticed the expected grouping of tuples which came from > a single backend to matching pages [2]. The stra

Re: [HACKERS] [PATCH] Add SIGCHLD catch to psql

2010-05-15 Thread Tom Lane
Robert Haas writes: > On Sat, May 15, 2010 at 7:46 PM, David Fetter wrote: >> Wouldn't this count as a bug fix? > Possibly, but changes to signal handlers are pretty global and can > sometimes have surprising side effects. I'm all in favor of someone > reviewing the patch - any volunteers? One

[HACKERS] Unexpected page allocation behavior on insert-only tables

2010-05-15 Thread Michael Renner
While preparing a replication test setup with 9.0beta1 I noticed strange page allocation patterns which Andrew Gierth found interesting enough to report here. I've written a simple tool to generate traffic on a database [1], which did about 30 TX/inserts per second to a table. Upon inspecting

Re: [HACKERS] [PATCH] Add SIGCHLD catch to psql

2010-05-15 Thread Robert Haas
On Sat, May 15, 2010 at 7:46 PM, David Fetter wrote: >> >   Anyway, this makes FETCH_COUNT alot more useful, and, in my view, the >> >   current behaviour of completely ignoring $PAGER exiting is a bug. >> >> Plesae add this to the next commit-fest: >> >>       https://commitfest.postgresql.org/ac

Re: [HACKERS] [PATCH] Add SIGCHLD catch to psql

2010-05-15 Thread David Fetter
On Fri, May 14, 2010 at 04:24:43PM -0400, Bruce Momjian wrote: > Stephen Frost wrote: > -- Start of PGP signed section. > > Greetings, > > > > Toying around with FETCH_COUNT today, I discovered that it didn't do > > the #1 thing I really wanted to use it for- query large tables without > > h

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Heikki Linnakangas
Heikki Linnakangas wrote: > Simon Riggs wrote: >> WALSender sleeps even when it might have more WAL to send, it doesn't >> check it just unconditionally sleeps. At least WALReceiver loops until >> it has no more to receive. I just can't imagine why that's useful >> behaviour. > > Good catch. That

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Heikki Linnakangas
Simon Riggs wrote: > WALSender sleeps even when it might have more WAL to send, it doesn't > check it just unconditionally sleeps. At least WALReceiver loops until > it has no more to receive. I just can't imagine why that's useful > behaviour. Good catch. That should be fixed. I also note that w

Re: [HACKERS] recovery consistent != hot standby

2010-05-15 Thread Robert Haas
On Fri, May 14, 2010 at 5:23 PM, Tom Lane wrote: > Robert Haas writes: >> PM_RECOVERY_CONSISTENT -> PM_HOT_STANDBY >> PMSIGNAL_RECOVERY_CONSISTENT -> PMSIGNAL_BEGIN_HOT_STANDBY > > +1.  From the point of view of the postmaster, whether the state > transition happens immediately upon reaching cons

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Simon Riggs
On Sat, 2010-05-15 at 18:24 +0100, Simon Riggs wrote: > I will recode using that concept. There's some behaviours that aren't helpful here: Startup gets new pointer when it runs out of data to replay. That might or might not include an updated keepalive timestamp, since there's no exact relation

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Simon Riggs
On Sat, 2010-05-15 at 20:05 +0300, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Sat, 2010-05-15 at 19:30 +0300, Heikki Linnakangas wrote: > >> Doesn't feel right to me either. If you want to expose the > >> keepalive-time to queries, it should be a separate field, something like > >> lastM

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Heikki Linnakangas
Simon Riggs wrote: > On Sat, 2010-05-15 at 19:30 +0300, Heikki Linnakangas wrote: >> Doesn't feel right to me either. If you want to expose the >> keepalive-time to queries, it should be a separate field, something like >> lastMasterKeepaliveTime and a pg_last_master_keepalive() function to >> read

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Simon Riggs
On Sat, 2010-05-15 at 19:30 +0300, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Sat, 2010-05-15 at 11:45 -0400, Tom Lane wrote: > >> I'm also extremely dubious that it's a good idea to set > >> recoveryLastXTime from this. Using both that and the timestamps from > >> the wal log flies in

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Heikki Linnakangas
Simon Riggs wrote: > On Sat, 2010-05-15 at 11:45 -0400, Tom Lane wrote: >> I'm also extremely dubious that it's a good idea to set >> recoveryLastXTime from this. Using both that and the timestamps from >> the wal log flies in the face of everything I remember about control >> theory. We should b

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Simon Riggs
On Sat, 2010-05-15 at 11:45 -0400, Tom Lane wrote: > Simon Riggs writes: > > Patch adds a keepalive message to ensure max_standby_delay is useful. > > The proposed placement of the keepalive-send is about the worst it could > possibly be. It needs to be done right before pq_flush to ensure > min

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Tom Lane
Simon Riggs writes: > Patch adds a keepalive message to ensure max_standby_delay is useful. The proposed placement of the keepalive-send is about the worst it could possibly be. It needs to be done right before pq_flush to ensure minimum transfer delay. Otherwise any attempt to measure clock sk

Re: [HACKERS] List traffic

2010-05-15 Thread Rob Wultsch
On Fri, May 14, 2010 at 11:50 PM, Rob Wultsch wrote: >> Linux has *as many if not more* ... MySQL, if memory servers, has a half >> dozen or more ... etc ... > > MySQL has a bunch of lists, none of which get much traffic. Honestly, > they should probably be combined. > > -- > Rob Wultsch "They" w

Re: [HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

2010-05-15 Thread Rob Wultsch
On Sat, May 15, 2010 at 4:09 AM, Kevin Grittner wrote: >  Anything in particular you wanted me to notice about it besides that? Nope. It was just a counter point to your previous comment. -- Rob Wultsch wult...@gmail.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) T

[HACKERS] Keepalive for max_standby_delay

2010-05-15 Thread Simon Riggs
Patch adds a keepalive message to ensure max_standby_delay is useful. No WAL format changes, no libpq changes. Just an additional message type for the streaming replication protocol, sent once per main loop in WALsender. Plus docs. Comments? -- Simon Riggs www.2ndQuadrant.com diff -

Re: [HACKERS] HS/SR Assert server crash

2010-05-15 Thread Simon Riggs
On Thu, 2010-05-13 at 18:01 -0400, Bruce Momjian wrote: > I was able to easily crash the standby server today just by starting it > and connecting to it via psql. The master was idle. The failure was: > > LOG: streaming replication successfully connected to primary > TRAP: FailedAs

Re: [HACKERS] How to know killed by pg_terminate_backend

2010-05-15 Thread Tatsuo Ishii
> >> Seems reasonable. Does the victim backend currently know why it has been > >> killed? > > > > I don't think so. > > > > One idea is postmaster sets a flag in the shared memory area > > indicating it rceived SIGTERM before forwarding the signal to > > backends. > > > > Backend check the flag an

Re: [HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

2010-05-15 Thread Kevin Grittner
Rob Wultsch wrote: > Have you looked at PBXT (which is explicitly NOT SERIALIZABLE)? I hadn't heard of it; so I took a quick look based on your post. It seems to a new engine to use with MySQL which has MVCC without a rollback log, so it presumably uses techniques at least vaguely similar to

Re: [HACKERS] predefined macros for various BSD-based systems?

2010-05-15 Thread Bruce Momjian
Giles Lean wrote: > > Tom Lane wrote: > > > I suppose that at least some of the *BSD herd really do predefine some > > of the symbols being attributed to them here, but I would like to see > > something authoritative about which and what. > > Documentation follows, but first the summary: > > F

Re: [HACKERS] predefined macros for various BSD-based systems?

2010-05-15 Thread Bruce Momjian
Tom Lane wrote: > I'm not even too sure what "bsdi" is, but I'm suspicious of that branch > too. A search of our code finds > > contrib/pg_upgrade/file.c: 248: #elif defined(freebsd) || defined(bsdi) || > defined(__darwin__) || defined(openbsd) > src/backend/utils/misc/ps_status.c: 67: #elif (de

Re: [HACKERS] predefined macros for various BSD-based systems?

2010-05-15 Thread Andrew Dunstan
Bruce Momjian wrote: Peter Eisentraut wrote: On l?r, 2010-05-15 at 00:23 -0400, Robert Haas wrote: It's a commercial distribution of BSD. I remember it being pretty nice when I used it 10+ years ago, but it sounds like it's dead now. BSDI is the company that produced BSD/OS,

Re: [HACKERS] predefined macros for various BSD-based systems?

2010-05-15 Thread Bruce Momjian
Peter Eisentraut wrote: > On l?r, 2010-05-15 at 00:23 -0400, Robert Haas wrote: > > It's a commercial distribution of BSD. I remember it being pretty > > nice when I used it 10+ years ago, but it sounds like it's dead now. > > BSDI is the company that produced BSD/OS, which was Bruce's main > dev

Re: [HACKERS] predefined macros for various BSD-based systems?

2010-05-15 Thread Giles Lean
Tom Lane wrote: > I suppose that at least some of the *BSD herd really do predefine some > of the symbols being attributed to them here, but I would like to see > something authoritative about which and what. Documentation follows, but first the summary: FreeBSD: __FreeBSD__ NetBSD: __NetBSD_

Re: [HACKERS] Synchronous replication patch built on SR

2010-05-15 Thread Heikki Linnakangas
BTW, What I'd like to see as a very first patch first is to change the current poll loops in walreceiver and walsender to, well, not poll. That's a requirement for synchronous replication, is very useful on its own, and requires a some design and implementation effort to get right. It would be nice

Re: [HACKERS] List traffic

2010-05-15 Thread Marc G. Fournier
[redirected to -chat] On Fri, 14 May 2010, Rob Wultsch wrote: Linux has *as many if not more* ... MySQL, if memory servers, has a half dozen or more ... etc ... MySQL has a bunch of lists, none of which get much traffic. Honestly, they should probably be combined. Except, when you do post,

Re: [HACKERS] Row-level Locks & SERIALIZABLE transactions, postgres vs. Oracle

2010-05-15 Thread Rob Wultsch
On Fri, May 14, 2010 at 7:32 AM, Kevin Grittner wrote: > Oracle, and all other MVCC databases I've read about outside of PostgreSQL, > use > an "update in place with a rollback log" technique. Have you looked at PBXT (which is explicitly NOT SERIALIZABLE)? -- Rob Wultsch wult...@gmail.com --

Re: [HACKERS] pg_upgrade code questions

2010-05-15 Thread Heikki Linnakangas
Tom Lane wrote: > Magnus Hagander writes: >> On Thu, May 13, 2010 at 5:06 PM, Bruce Momjian wrote: >>> I have added SGML comments to comment out the text that mentions EDB >>> Advanced Server. Is that enough? Should I remove the text from the >>> SGML? Should I move it to the bottom of the SGM