Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Tue, 2011-03-01 at 15:25 +0900, Fujii Masao wrote: > On Tue, Mar 1, 2011 at 9:19 AM, Simon Riggs wrote: > > On Mon, 2011-02-28 at 18:40 +, Simon Riggs wrote: > >> > SyncRepReleaseWaiters should be called when walsender exits. Otherwise, > >> > if the standby crashes while a transaction is w

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Fujii Masao
On Tue, Mar 1, 2011 at 3:39 AM, Simon Riggs wrote: >> PREPARE TRANSACTION and ROLLBACK PREPARED should wait for >> replication as well as COMMIT PREPARED? > > PREPARE - Yes > ROLLBACK - No > > Further discussion welcome If we don't make ROLLBACK PREPARED wait for replication, we might need to iss

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Fujii Masao
Thanks for update of the patch! On Tue, Mar 1, 2011 at 3:40 AM, Simon Riggs wrote: >> SyncRepRemoveFromQueue seems not to be as short-term as we can >> use the spinlock. Instead, LW lock should be used there. You seem to have forgotten to fix the above-mentioned issue. A spinlock can be used onl

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Fujii Masao
On Tue, Mar 1, 2011 at 9:19 AM, Simon Riggs wrote: > On Mon, 2011-02-28 at 18:40 +, Simon Riggs wrote: >> > SyncRepReleaseWaiters should be called when walsender exits. Otherwise, >> > if the standby crashes while a transaction is waiting for replication, >> > it waits infinitely. >> >> Will t

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Tom Lane
Marko Tiikkaja writes: > On 2011-02-28 9:36 PM, Tom Lane wrote: >> OK, so the intent is that in all cases, we just advance CID and don't >> take a new snapshot between queries that were generated (by rule >> expansion) from a single original parsetree? But we still take a new >> snap between orig

Re: [HACKERS] Snapshot synchronization, again...

2011-02-28 Thread Joachim Wieland
On Mon, Feb 28, 2011 at 6:38 PM, Robert Haas wrote: >> Remember that it's not only about saving shared memory, it's also >> about making sure that the snapshot reflects a state of the database >> that has actually existed at some point in the past. Furthermore, we >> can easily invalidate a snapsh

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-02-28 Thread Greg Stark
On Tue, Mar 1, 2011 at 1:43 AM, David Christensen wrote: > Was this cluster upgraded to 8.4.4 from 8.4.0?  It sounds to me like a known > bug in 8.4.0 which was fixed by this commit: > The reproduction script described was running vacuum repeatedly. A single vacuum run out to be sufficient to cl

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-02-28 Thread Fujii Masao
On Tue, Mar 1, 2011 at 10:43 AM, David Christensen wrote: > Was this cluster upgraded to 8.4.4 from 8.4.0?  It sounds to me like a known > bug in 8.4.0 which was fixed by this commit: > > commit 7fc7a7c4d082bfbd579f49e92b046dd51f1faf5f > Author: Tom Lane > Date:   Mon Aug 24 02:18:32 2009 +

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-02-28 Thread David Christensen
On Feb 28, 2011, at 3:28 PM, daveg wrote: > On Wed, Jan 12, 2011 at 10:46:14AM +0200, Heikki Linnakangas wrote: >> On 12.01.2011 06:21, Fujii Masao wrote: >>> On Sat, Dec 25, 2010 at 2:09 PM, Maxim Boguk wrote: While I trying create reproducible test case for BUG #5798 I encountered ve

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Mon, 2011-02-28 at 18:40 +, Simon Riggs wrote: > > SyncRepReleaseWaiters should be called when walsender exits. Otherwise, > > if the standby crashes while a transaction is waiting for replication, > > it waits infinitely. > > Will think on this. The behaviour seems correct to me: If allo

Re: [HACKERS] WIP: cross column correlation ...

2011-02-28 Thread Rod Taylor
> > But it's not the same as tracking *sections of a table*. > > I dunno. I imagine if you have a "section" of a table in different > storage than other sections, you created a tablespace and moved the > partition holding that section there. Otherwise, how do you prevent the > tuples from moving

Re: [HACKERS] SSI bug?

2011-02-28 Thread Dan Ports
An updated patch to address this issue is attached. It fixes a couple issues related to use of the backend-local lock table hint: - CheckSingleTargetForConflictsIn now correctly handles the case where a lock that's being held is not reflected in the local lock table. This fixes the asser

Re: [HACKERS] Native XML

2011-02-28 Thread Andrew Dunstan
On 02/28/2011 05:28 PM, Kevin Grittner wrote: Anton wrote: it was actually the focal point of my considerations: whether to store plain text or 'something else'. There seems to be an almost universal assumption that storing XML in its native form (i.e. a text stream) is going to produc

[HACKERS] PL developer summit, May 21 at PgCon

2011-02-28 Thread Selena Deckelmann
Hi! I've been contacting PL developers directly over the last couple of weeks about having a PL summit at PgCon this year. The overwhelming response was: yes, let's do it! So, we will have the summit starting at 9:30am, Saturday May 21 at PgCon in Ottawa. We'll meet in one of the session rooms on

Re: [HACKERS] Re: Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Chris Browne
j...@agliodbs.com (Josh Berkus) writes: >> I don't understand what you're talking about at all here. I think >> there are a lot of unsolved problems in monitoring but the one thing >> I think everyone is pretty clear on is that the right way to export >> metrics like these is to export a counter an

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Mon, 2011-02-28 at 16:55 -0500, Robert Haas wrote: > On Mon, Feb 28, 2011 at 4:36 PM, Simon Riggs wrote: > > On Mon, 2011-02-28 at 16:22 -0500, Robert Haas wrote: > > > >> > Docs changed: git://github.com/simon2ndQuadrant/postgres.git > >> > >> I'm hoping you're going to post an updated patch o

Re: [HACKERS] Re: Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Robert Treat
On Mon, Feb 28, 2011 at 4:13 PM, Greg Stark wrote: > On Mon, Feb 28, 2011 at 6:31 PM, Josh Berkus wrote: > What's needed here is for someone to write a good mrtg/rrd/whatever > replacement using postgres as its data store. If you're monitoring > something sensitive then you would store the data i

Re: [HACKERS] Native XML

2011-02-28 Thread Kevin Grittner
Anton wrote: > it was actually the focal point of my considerations: whether to > store plain text or 'something else'. Given that there were similar issues for other hierarchical data types, perhaps we need something similar to tsvector, but for hierarchical data. The extra layer of abstract

Re: [HACKERS] Native XML

2011-02-28 Thread Anton
On 02/28/2011 05:23 PM, Robert Haas wrote: > On Mon, Feb 28, 2011 at 10:30 AM, Tom Lane wrote: > >> Well, in principle we could allow them to work on both, just the same >> way that (for instance) "+" is a standardized operator but works on more >> than one datatype. But I agree that the prosp

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-02-28 Thread daveg
On Wed, Jan 12, 2011 at 10:46:14AM +0200, Heikki Linnakangas wrote: > On 12.01.2011 06:21, Fujii Masao wrote: > >On Sat, Dec 25, 2010 at 2:09 PM, Maxim Boguk wrote: > >>While I trying create reproducible test case for BUG #5798 I > >>encountered very strange effect on two of my servers (both serve

Re: [HACKERS] Re: Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Josh Berkus
> I don't understand what you're talking about at all here. I think > there are a lot of unsolved problems in monitoring but the one thing I > think everyone is pretty clear on is that the right way to export > metrics like these is to export a counter and then have some external > component perio

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-02-28 Thread Heikki Linnakangas
On 28.02.2011 23:28, daveg wrote: On Wed, Jan 12, 2011 at 10:46:14AM +0200, Heikki Linnakangas wrote: We'll likely need to go back and forth a few times with various debugging patches until we get to the heart of this.. Anything new on this? I'm seeing at on one of my clients production boxes.

Re: [HACKERS] SR standby hangs

2011-02-28 Thread Robert Haas
On Tue, Feb 22, 2011 at 11:34 AM, Tom Lane wrote: > Greg Stark writes: >> On Tue, Feb 22, 2011 at 12:55 PM, Robert Haas wrote: >>> A little OT, but ISTM that the buffer pin mechanism by its nature is >>> prone to lock upgrade hazards. > >> Except that pins don't block exclusive locks so there's

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 4:36 PM, Simon Riggs wrote: > On Mon, 2011-02-28 at 16:22 -0500, Robert Haas wrote: > >> > Docs changed: git://github.com/simon2ndQuadrant/postgres.git >> >> I'm hoping you're going to post an updated patch once the current rash >> of updates is all done. > > Immediately pr

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Mon, 2011-02-28 at 16:22 -0500, Robert Haas wrote: > > Docs changed: git://github.com/simon2ndQuadrant/postgres.git > > I'm hoping you're going to post an updated patch once the current rash > of updates is all done. Immediately prior to commit, yes. Everybody else has been nudging me towar

Re: [HACKERS] Spatio-Temporal Functions

2011-02-28 Thread Robert Haas
On Sat, Feb 26, 2011 at 1:43 PM, Nick Raj wrote: > Hi, > I am writing some spatio-temporal function in postgis. > Like, ST_Enters( obj_geom, boundary_geom, time t1,time t2) > > For example- Does vehicle enters in ABC between time t1 to t2? > Let us suppose, i take only one object geometry at a tim

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 4:13 PM, Simon Riggs wrote: > On Sat, 2011-02-19 at 23:26 -0500, Robert Haas wrote: > >> I believe the problem is that the definition of IsOnSyncRepQueue is >> bogus, so that the loop in SyncRepWaitOnQueue always takes the first >> branch. > > Sorry, don't see that. Jaime/Y

[HACKERS] Re: Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Greg Stark
On Mon, Feb 28, 2011 at 6:31 PM, Josh Berkus wrote: > Like replacing each statistic with a series of time-based buckets, which > would then increase the size of the table by 5X to 10X.  That was the > first solution I thought of, and rejected. > I don't understand what you're talking about at all

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Sat, 2011-02-19 at 23:26 -0500, Robert Haas wrote: > I believe the problem is that the definition of IsOnSyncRepQueue is > bogus, so that the loop in SyncRepWaitOnQueue always takes the first > branch. Sorry, don't see that. Jaime/Yeb fix applied. > It was a little confusing to me setting thi

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Sat, 2011-02-19 at 22:52 -0500, Robert Haas wrote: > On Sat, Feb 19, 2011 at 3:28 AM, Simon Riggs wrote: > > First, we should be clear to explain that you are referring to the fact > > that the request > > synchronous_commit = off > > synchronous_replication = on > > makes no sense in the way

[HACKERS] OSSP gone missing? Fate of UUID?

2011-02-28 Thread Michael Glaesemann
A couple of weeks ago when installing uuid-ossp on a new server, I noticed that the ossp site is gone. I haven't found anything on the web to indicate what happened. Anyone know? Michael Glaesemann grzm seespotcode net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) T

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 2:31 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane wrote: >>> Ultimately we need to think of a reporting mechanism that's a bit >>> smarter than "rewrite the whole file for any update" ... > >> Well, we have these things called "ta

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
Euler Taveira de Oliveira writes: > Em 28-02-2011 15:50, Tom Lane escreveu: >> Ultimately we need to think of a reporting mechanism that's a bit >> smarter than "rewrite the whole file for any update" ... > What about splitting statistic file per database? That would improve matters for some usa

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
"Joshua D. Drake" writes: > On Mon, 2011-02-28 at 11:39 -0800, Josh Berkus wrote: > Spitballing here, but could sqlite be an intermediate, compromise solution? >> >> For a core PostgreSQL component ?!?!? > Sure, why not? Because it's fifty times more mechanism than we need here? We don't want

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Fri, 2011-02-25 at 16:41 +0100, Yeb Havinga wrote: > I needed to add this to make the documentation compile > > --- a/doc/src/sgml/config.sgml > +++ b/doc/src/sgml/config.sgml > @@ -2010,6 +2010,9 @@ SET ENABLE_SEQSCAN TO OFF; > You should also consider setting > hot_standby_feedback

Re: [HACKERS] EXPLAIN doesn't show sufficient info for wCTE cases

2011-02-28 Thread Tom Lane
David Fetter writes: > On Mon, Feb 28, 2011 at 11:44:06AM -0500, Robert Haas wrote: >> I think it's good to include the table name, for sure. I *think* I >> agree that it isn't necessary to include the child names. > Would this affect the use case of breaking up a too-long table into > partition

Re: [HACKERS] pl/python do not delete function arguments

2011-02-28 Thread Peter Eisentraut
On lör, 2011-02-26 at 09:43 +0100, Jan Urbański wrote: > I'm officially at a loss on how to fix that bug without some serious > gutting of how PL/Python arguments work. If someone comes up with a > brilliant way to solve this problem, we can commit it after beta, or > even during the 9.2 cycle (sho

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Euler Taveira de Oliveira
Em 28-02-2011 15:50, Tom Lane escreveu: Ultimately we need to think of a reporting mechanism that's a bit smarter than "rewrite the whole file for any update" ... What about splitting statistic file per database? -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hack

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Joshua D. Drake
On Mon, 2011-02-28 at 11:39 -0800, Josh Berkus wrote: > > Spitballing here, but could sqlite be an intermediate, compromise solution? > > For a core PostgreSQL component ?!?!? Sure, why not? It is ACID compliant, has the right kind of license, has a standard API that we are all used to. It seems

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Josh Berkus
> Spitballing here, but could sqlite be an intermediate, compromise solution? For a core PostgreSQL component ?!?!? -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com --

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Marko Tiikkaja
On 2011-02-28 9:36 PM, Tom Lane wrote: Marko Tiikkaja writes: On 2011-02-28 9:03 PM, Tom Lane wrote: OK, and which behavior is getting changed, to what? I am not interested in trying to reverse-engineer a specification from the patch. My recollection is (and the archives seem to agree) tha

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Michael Glaesemann
On Feb 28, 2011, at 14:31, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane wrote: >>> Ultimately we need to think of a reporting mechanism that's a bit >>> smarter than "rewrite the whole file for any update" ... > >> Well, we have these things called "tables

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Tom Lane
Marko Tiikkaja writes: > On 2011-02-28 9:03 PM, Tom Lane wrote: >> OK, and which behavior is getting changed, to what? I am not interested >> in trying to reverse-engineer a specification from the patch. > My recollection is (and the archives seem to agree) that normal > execution and SQL funct

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Mon, 2011-02-21 at 21:35 +0900, Tatsuo Ishii wrote: > > Well, good news all round. > > > > v17 implements what I believe to be the final set of features for sync > > rep. This one I'm actually fairly happy with. It can be enjoyed best at > > DEBUG3. > > > > The patch is very lite touch on a fe

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane wrote: >> Ultimately we need to think of a reporting mechanism that's a bit >> smarter than "rewrite the whole file for any update" ... > Well, we have these things called "tables". Any chance of using those? Having the stats coll

Re: [HACKERS] knngist - 0.8

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 2:25 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 28, 2011 at 1:53 PM, Josh Berkus wrote: Since no one has stepped up to fix these issues, I have marked this patch Returned with Feedback. > >>> This is just contrib/btree_GIST, yes? > >> Yes, core KNN

Re: [HACKERS] mysql2pgsql.perl update

2011-02-28 Thread Josh Berkus
> does foundry support git or should I just link to a repo on github? If you prefer using git, the latter. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent vi

Re: [HACKERS] knngist - 0.8

2011-02-28 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 28, 2011 at 1:53 PM, Josh Berkus wrote: >>> Since no one has stepped up to fix these issues, I have marked this >>> patch Returned with Feedback. >> This is just contrib/btree_GIST, yes? > Yes, core KNN was committed by Tom during the November CommitFest. Righ

Re: [HACKERS] EXPLAIN doesn't show sufficient info for wCTE cases

2011-02-28 Thread David Fetter
On Mon, Feb 28, 2011 at 11:44:06AM -0500, Robert Haas wrote: > On Mon, Feb 28, 2011 at 11:39 AM, Tom Lane wrote: > > EXPLAIN currently shows ModifyTable nodes as just "Insert", "Update", > > or "Delete", without any indication of the target table.  This was > > more or less good enough when there

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Marko Tiikkaja
On 2011-02-28 9:03 PM, Tom Lane wrote: Marko Tiikkaja writes: On 2011-02-28 8:22 PM, Tom Lane wrote: So: exactly what is the intended behavioral change as of now, and what is the argument supporting that change? The only intended change is what I was wondering in the original post: snapshot

Re: [HACKERS] mysql2pgsql.perl update

2011-02-28 Thread Andrew Hammond
On Mon, Feb 28, 2011 at 10:49 AM, Josh Berkus wrote: > Andrew, > >> I spent some time updating mysql2pgsql.perl. Changes were driven by an >> attempt to migrate a redmine database. Original code was failing for a >> number of reasons (regex recursion explosion, . I was wondering it >> there's a mo

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Thu, 2011-02-24 at 18:13 -0800, Daniel Farina wrote: > I have also reproduced this. Notably, things seem fine as long as > pgbench is confined to one backend, but as soon as two are used (-c 2) > by the feature I can get segfaults. Sorry that you all experienced this. I wasn't able to get conc

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 2:01 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 28, 2011 at 1:45 PM, Tom Lane wrote: >>> I'm afraid that the goals of this patch might be similarly obsolete. > >> No, I don't think so.  IIUC, the problem is that EXPLAIN ANALYZE runs >> the rewrite products w

Re: [HACKERS] knngist - 0.8

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 1:53 PM, Josh Berkus wrote: > >> Since no one has stepped up to fix these issues, I have marked this >> patch Returned with Feedback. > > This is just contrib/btree_GIST, yes? Yes, core KNN was committed by Tom during the November CommitFest. -- Robert Haas EnterpriseDB:

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Tom Lane
Marko Tiikkaja writes: > On 2011-02-28 8:22 PM, Tom Lane wrote: >> So: exactly what is the intended behavioral change as of now, and what >> is the argument supporting that change? > The only intended change is what I was wondering in the original post: > snapshot handling between normal executi

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 28, 2011 at 1:45 PM, Tom Lane wrote: >> I'm afraid that the goals of this patch might be similarly obsolete. > No, I don't think so. IIUC, the problem is that EXPLAIN ANALYZE runs > the rewrite products with different snapshot handling than the regular > execut

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Marko Tiikkaja
On 2011-02-28 8:22 PM, Tom Lane wrote: Marko Tiikkaja writes: [ latest version of snapshot-taking patch ] I started to look at this, and find myself fairly confused as to what the purpose is anymore. Reviewing the thread, there has been a lot of discussion of refactoring the API of pg_parse_

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 1:45 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 28, 2011 at 1:22 PM, Tom Lane wrote: >>> So: exactly what is the intended behavioral change as of now, and what >>> is the argument supporting that change? > >> IIUC, this is the result of countless rounds of c

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane wrote: > Josh Berkus writes: >> On 2/28/11 10:24 AM, Robert Haas wrote: >>> On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus wrote: On the other hand, anything which increases the size of pg_statistic would be a nightmare. > >>> Hmm? > >> Like repl

Re: [HACKERS] knngist - 0.8

2011-02-28 Thread Josh Berkus
> Since no one has stepped up to fix these issues, I have marked this > patch Returned with Feedback. This is just contrib/btree_GIST, yes? -- -- Josh Berkus PostgreSQL Experts Inc. http:

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
Josh Berkus writes: > On 2/28/11 10:24 AM, Robert Haas wrote: >> On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus wrote: >>> On the other hand, anything which increases the size of pg_statistic >>> would be a nightmare. >> Hmm? > Like replacing each statistic with a series of time-based buckets, wh

Re: [HACKERS] mysql2pgsql.perl update

2011-02-28 Thread Josh Berkus
Andrew, > I spent some time updating mysql2pgsql.perl. Changes were driven by an > attempt to migrate a redmine database. Original code was failing for a > number of reasons (regex recursion explosion, . I was wondering it > there's a more formal / appropriate place to put this. I'd also > appreci

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 28, 2011 at 1:22 PM, Tom Lane wrote: >> So: exactly what is the intended behavioral change as of now, and what >> is the argument supporting that change? > IIUC, this is the result of countless rounds of communal bikeshedding around: Quite :-(. But I'm not sur

Re: [HACKERS] WIP: cross column correlation ...

2011-02-28 Thread Alvaro Herrera
Excerpts from Robert Haas's message of sáb feb 26 02:24:26 -0300 2011: > On Fri, Feb 25, 2011 at 2:26 PM, Alvaro Herrera > wrote: > > Excerpts from Rod Taylor's message of vie feb 25 14:03:58 -0300 2011: > > > >> How practical would it be for analyze to keep a record of response times > >> for >

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Fri, 2011-02-25 at 16:41 +0100, Yeb Havinga wrote: > --- a/src/backend/replication/syncrep.c > +++ b/src/backend/replication/syncrep.c > @@ -274,6 +274,8 @@ SyncRepRemoveFromQueue(void) > } > else > { > + bool found = false; > + > while

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Tue, 2011-02-22 at 14:38 +0900, Fujii Masao wrote: > On Mon, Feb 21, 2011 at 6:06 PM, Fujii Masao wrote: > > I've read about a tenth of the patch, so I'll submit another comments > > about the rest later. > > Here are another comments: Thanks for your comments Code available at git://github.c

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Thu, 2011-02-24 at 22:08 +0900, Fujii Masao wrote: > On Tue, Feb 22, 2011 at 2:38 PM, Fujii Masao wrote: > > I've read about two-tenths of the patch, so I'll submit another comments > > about the rest later. Sorry for the slow reviewing... > > Here are another comments: Thanks for your commen

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Mon, 2011-02-28 at 10:31 +0100, Yeb Havinga wrote: > 1) no automatic switch to other synchronous standby > - start master server, add synchronous standby 1 > - change allow_standalone_primary to off > - add second synchronous standby > - wait until pg_stat_replication shows both standby's are i

Re: [HACKERS] Sync Rep v17

2011-02-28 Thread Simon Riggs
On Mon, 2011-02-21 at 18:06 +0900, Fujii Masao wrote: > Thanks for the patch! Thanks for the review. Code available at git://github.com/simon2ndQuadrant/postgres.git > PREPARE TRANSACTION and ROLLBACK PREPARED should wait for > replication as well as COMMIT PREPARED? PREPARE - Yes ROLLBACK - N

Re: [HACKERS] pl/python custom exceptions for SPI

2011-02-28 Thread Tom Lane
Peter Eisentraut writes: > On mån, 2011-02-28 at 12:08 -0500, Tom Lane wrote: >> I'm seeing a core dump as well as multiple inconsistencies in error >> message spelling in the plpython regression tests on a Fedora 13 box >> (python 2.6.4). Several buildfarm critters don't look too happy either.

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Josh Berkus
On 2/28/11 10:24 AM, Robert Haas wrote: > On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus wrote: >> On the other hand, anything which increases the size of pg_statistic >> would be a nightmare. > > Hmm? Like replacing each statistic with a series of time-based buckets, which would then increase the

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 1:22 PM, Tom Lane wrote: > Marko Tiikkaja writes: >> [ latest version of snapshot-taking patch ] > > I started to look at this, and find myself fairly confused as to what > the purpose is anymore.  Reviewing the thread, there has been a lot of > discussion of refactoring t

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus wrote: > On the other hand, anything which increases the size of pg_statistic > would be a nightmare. Hmm? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-02-28 Thread Tom Lane
Marko Tiikkaja writes: > [ latest version of snapshot-taking patch ] I started to look at this, and find myself fairly confused as to what the purpose is anymore. Reviewing the thread, there has been a lot of discussion of refactoring the API of pg_parse_and_rewrite and related functions exporte

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Martijn van Oosterhout
On Mon, Feb 28, 2011 at 10:04:54AM -0800, Josh Berkus wrote: > Take, for example, a problem I was recently grappling with for Nagios. > I'd like to do a check as to whether or not tables are getting > autoanalyzed often enough. After all, autovac can fall behind, and we'd > want to be alerted of t

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Josh Berkus
> Well, what we have now is a bunch of counters in pg_stat_all_tables > and pg_statio_all_tables. Right. What I'm saying is those aren't good enough, and have never been good enough. Counters without a time basis are pretty much useless for performance monitoring/management (Baron Schwartz ha

Re: [HACKERS] Snapshot synchronization, again...

2011-02-28 Thread Tom Lane
Robert Haas writes: > On Sun, Feb 27, 2011 at 8:33 PM, Joachim Wieland wrote: >> Remember that it's not only about saving shared memory, it's also >> about making sure that the snapshot reflects a state of the database >> that has actually existed at some point in the past. > But you can do all

[HACKERS] mysql2pgsql.perl update

2011-02-28 Thread Andrew Hammond
Hi all. I spent some time updating mysql2pgsql.perl. Changes were driven by an attempt to migrate a redmine database. Original code was failing for a number of reasons (regex recursion explosion, . I was wondering it there's a more formal / appropriate place to put this. I'd also appreciate a sepa

Re: [HACKERS] pl/python custom exceptions for SPI

2011-02-28 Thread Peter Eisentraut
On mån, 2011-02-28 at 12:08 -0500, Tom Lane wrote: > Peter Eisentraut writes: > > On lör, 2011-02-12 at 11:58 +0100, Jan Urbański wrote: > >> Here's an updated patch with documentation. It's an incremental patch on > >> top of the latest explicit-subxacts version. > > > Committed. > > I'm seeing

Re: [HACKERS] Snapshot synchronization, again...

2011-02-28 Thread Robert Haas
On Sun, Feb 27, 2011 at 8:33 PM, Joachim Wieland wrote: > On Sun, Feb 27, 2011 at 3:04 PM, Heikki Linnakangas > wrote: >>> Why exactly, Heikki do you think the hash is more troublesome? >> It just feels wrong to rely on cryptography just to save some shared memory. > > Remember that it's not only

Re: [HACKERS] PL/pgSQL return value in after triggers

2011-02-28 Thread Tom Lane
Peter Eisentraut writes: > PL/pgSQL trigger functions currently require a value to be returned, > even though that value is not used for anything in case of a trigger > fired AFTER. I was wondering if we could relax that. I got bit by that just a couple days ago --- I supposed that a trigger tha

Re: [HACKERS] PL/pgSQL return value in after triggers

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 12:07 PM, Peter Eisentraut wrote: > PL/pgSQL trigger functions currently require a value to be returned, > even though that value is not used for anything in case of a trigger > fired AFTER.  I was wondering if we could relax that.  It would make > things a bit more robust

Re: [HACKERS] pl/python custom exceptions for SPI

2011-02-28 Thread Tom Lane
Peter Eisentraut writes: > On lör, 2011-02-12 at 11:58 +0100, Jan Urbański wrote: >> Here's an updated patch with documentation. It's an incremental patch on >> top of the latest explicit-subxacts version. > Committed. I'm seeing a core dump as well as multiple inconsistencies in error message

[HACKERS] PL/pgSQL return value in after triggers

2011-02-28 Thread Peter Eisentraut
PL/pgSQL trigger functions currently require a value to be returned, even though that value is not used for anything in case of a trigger fired AFTER. I was wondering if we could relax that. It would make things a bit more robust and produce clearer PL/pgSQL code. The specific case I'm concerned

Re: [HACKERS] knngist - 0.8

2011-02-28 Thread Robert Haas
On Fri, Feb 18, 2011 at 1:07 AM, Tom Lane wrote: > There might be more issues, I haven't read the patch in detail. > But anyway, I'm going to set it to Waiting on Author.  I think it > needs at least a day or so's work, and I can't put in that kind of > time on it now. Since no one has stepped up

Re: [HACKERS] pl/python custom exceptions for SPI

2011-02-28 Thread Peter Eisentraut
On lör, 2011-02-12 at 11:58 +0100, Jan Urbański wrote: > On 11/02/11 10:53, Jan Urbański wrote: > > On 10/02/11 22:26, Steve Singer wrote: > > Here's an updated patch with documentation. It's an incremental patch on > top of the latest explicit-subxacts version. Committed. -- Sent via pgsql-ha

Re: [HACKERS] EXPLAIN doesn't show sufficient info for wCTE cases

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 11:39 AM, Tom Lane wrote: > EXPLAIN currently shows ModifyTable nodes as just "Insert", "Update", > or "Delete", without any indication of the target table.  This was > more or less good enough when there could only be one such node per > query, but it's looking pretty inad

[HACKERS] EXPLAIN doesn't show sufficient info for wCTE cases

2011-02-28 Thread Tom Lane
EXPLAIN currently shows ModifyTable nodes as just "Insert", "Update", or "Delete", without any indication of the target table. This was more or less good enough when there could only be one such node per query, but it's looking pretty inadequate to me as I play around with data-modifying statement

Re: [HACKERS] Native XML

2011-02-28 Thread Andrew Dunstan
On 02/28/2011 10:51 AM, Tom Lane wrote: Andrew Dunstan writes: xpath_table is severely broken by design IMNSHO. We need a new design, but I'm reluctant to work on that until someone does LATERAL, because a replacement would be much nicer to design with it than without it. Well, maybe I'm mis

Re: [HACKERS] Native XML

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 10:30 AM, Tom Lane wrote: > Well, in principle we could allow them to work on both, just the same > way that (for instance) "+" is a standardized operator but works on more > than one datatype.  But I agree that the prospect of two parallel types > with essentially duplicat

Re: [HACKERS] Problem with pg_upgrade (8.4 -> 9.0) due to ALTER DATABASE SET ROLE

2011-02-28 Thread Robert Treat
On Mon, Feb 28, 2011 at 3:42 AM, Magnus Hagander wrote: > On Mon, Feb 28, 2011 at 06:21, Tom Lane wrote: >> Robert Treat writes: >>> Did anything ever come of this discussion? >> >> I think it's a TODO --- nothing done about it as yet, AFAIR. >> >>> On one of the databases I >>> was upgrading, I

Re: [HACKERS] PG signal handler and non-reentrant malloc/free calls

2011-02-28 Thread Tom Lane
Heikki Linnakangas writes: > Unless ImmediateInterruptOK is set, but it's only set around a few > blocking system calls where it is safe to do so. (Checks...) Actually, > md5_crypt_verify() looks suspicious, it does "ImmediateInterruptOK = > true", and then calls palloc() and pfree(). Hm, yeah

Re: [HACKERS] Native XML

2011-02-28 Thread Tom Lane
Andrew Dunstan writes: > xpath_table is severely broken by design IMNSHO. We need a new design, > but I'm reluctant to work on that until someone does LATERAL, because a > replacement would be much nicer to design with it than without it. Well, maybe I'm missing something, but I don't really un

Re: [HACKERS] Native XML

2011-02-28 Thread Andrew Dunstan
On 02/28/2011 10:30 AM, Tom Lane wrote: The single most pressing problem we've got with XML right now is the poor state of the XPath extensions in contrib/xml2. If we don't see a meaningful step forward in that area, a new implementation of the xml datatype isn't likely to win acceptance.

Re: [HACKERS] Native XML

2011-02-28 Thread Robert Haas
On Sun, Feb 27, 2011 at 10:20 PM, Andrew Dunstan wrote: > No, I think the xpath implementation is from libxml2. But in any case, I > think the problem is in the whole design of the xpath_table function, and > not in the library used for running the xpath queries. i.e it's our fault, > and not the

Re: [HACKERS] Native XML

2011-02-28 Thread Tom Lane
Andrew Dunstan writes: > On 02/28/2011 04:25 AM, Anton wrote: >> A question is of course, if potential new implementation must >> necessarily replace the existing one, immediately or at all. What I >> published is implemented as a new data type and thus pg_type.h and >> pg_proc.h are the only file

Re: [HACKERS] WIP: cross column correlation ...

2011-02-28 Thread Robert Haas
On Sun, Feb 27, 2011 at 5:17 PM, Josh Berkus wrote: > >> I think there would be value in giving the DBA an easier way to see >> which tables are hot, but I am really leery about the idea of trying >> to feed that directly into the query planner.  I think this is one of >> those cases where we let

Re: [HACKERS] Native XML

2011-02-28 Thread Andrew Dunstan
On 02/28/2011 04:25 AM, Anton wrote: On 02/27/2011 11:57 PM, Peter Eisentraut wrote: On sön, 2011-02-27 at 10:45 -0500, Tom Lane wrote: Hmm, so this doesn't rely on libxml2 at all? Given the amount of pain that library has caused us, getting out from under it seems like a mighty attractive

[HACKERS] Bug of the hot standby feedback

2011-02-28 Thread Fujii Masao
Hi, When I implemented the replication timeout patch, I found the bug on the HS feedback feature. When wal_receiver_status_interval is zero and hot_standby_feedback is enabled, walreceiver sends the feedback too aggressively. I think that the attached patch should be applied to fix the bug. Regar

Re: [HACKERS] Replication server timeout patch

2011-02-28 Thread Fujii Masao
On Sun, Feb 27, 2011 at 11:52 AM, Fujii Masao wrote: >> There are two things that I think are pretty clear.  If the receiver >> has wal_receiver_status_interval=0, then we should ignore >> replication_timeout for that connection. > > The patch still doesn't check that wal_receiver_status_interval

  1   2   >