[HACKERS] proposal: plpgsql, solution for derivated types of parameters

2010-10-08 Thread Pavel Stehule
Hello, the current plpgsql syntax doesn't offer a functionality to define some variable with type as element of some other array variable or reverse order. The primary goal of this proposal is enahancing plpgsql for better working with polymorphic types. I propose a following syntax: --

Re: [HACKERS] Git cvsserver serious issue

2010-10-08 Thread Magnus Hagander
On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan and...@dunslane.net wrote: On 10/07/2010 03:37 PM, Magnus Hagander wrote: On Thu, Oct 7, 2010 at 21:31, Andrew Dunstanand...@dunslane.net  wrote: On 10/07/2010 10:11 AM, Magnus Hagander wrote: OTOH, this patch seems pretty small and simple to

Re: [HACKERS] On Scalability

2010-10-08 Thread Vincenzo Romano
2010/10/7 Simon Riggs si...@2ndquadrant.com: On Thu, 2010-10-07 at 14:10 +0200, Vincenzo Romano wrote: Making these things sub-linear (whether not O(log n) or even O(1) ), provided that there's  way to, would make this RDBMS more appealing to enterprises. I mean also partial indexes (as an

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-08 Thread Fujii Masao
On Fri, Oct 8, 2010 at 10:38 AM, Robert Haas robertmh...@gmail.com wrote: Yes, let's please just implement something simple and get it committed.  k = 1.  Two GUCs (synchronous_standbys = name, name, name and synchronous_waitfor = none|recv|fsync|apply) For my cases, I'm OK with this as the

Re: [HACKERS] proposal: plpgsql, solution for derivated types of parameters

2010-10-08 Thread Itagaki Takahiro
On Fri, Oct 8, 2010 at 3:08 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I propose a following syntax: -- variable as element of some array DECLARE elementvar IS ELEMENT OF arrayvar; DECLARE elementvar IS ELEMENT OF arrayvar%type -- variable as array of some variable DECLARE arrayvar

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Dimitri Fontaine
Greg Smith g...@2ndquadrant.com writes: […] I don't see this as needing any implementation any more complicated than the usual way such timeouts are handled. Note how long you've been trying to reach the standby. Default to -1 for forever. And if you hit the timeout, mark the standby as

Re: [HACKERS] leaky views, yet again

2010-10-08 Thread Dimitri Fontaine
Hi, Stephen Frost sfr...@snowman.net writes: Wow, I just kind of assumed you could; I'm not really sure why. Perhaps it'll be possible in the future though, so might be something to think about if/when it happens. Can't see a way to abuse the view from inside a DO or in a function in the

Re: [HACKERS] proposal: plpgsql, solution for derivated types of parameters

2010-10-08 Thread Pavel Stehule
2010/10/8 Itagaki Takahiro itagaki.takah...@gmail.com: On Fri, Oct 8, 2010 at 3:08 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I propose a following syntax: -- variable as element of some array DECLARE elementvar IS ELEMENT OF arrayvar; DECLARE elementvar IS ELEMENT OF arrayvar%type

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 12:30 AM, Simon Riggs wrote: I do, but its not a parameter. The k = 1 behaviour is hardcoded and considerably simplifies the design. Moving to k 1 is additional work, slows things down and seems likely to be fragile. Perfect! So I'm all in favor of committing that, but leaving

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
Simon, On 10/08/2010 12:25 AM, Simon Riggs wrote: Asking for k 1 does *not* mean those servers are time synchronised. Yes, it's technically impossible to create a fully synchronized cluster (on the basis of shared-nothing nodes we are aiming for, that is). There always is some kind of lag on

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Heikki Linnakangas
On 07.10.2010 21:38, Markus Wanner wrote: On 10/07/2010 03:19 PM, Dimitri Fontaine wrote: I think you're all into durability, and that's good. The extra cost is service downtime It's just *reduced* availability. That doesn't necessarily mean downtime, if you combine cleverly with async

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 04:01 AM, Fujii Masao wrote: Really? I don't think that ko-count=0 means wait-forever. Telling from the documentation, I'd also say it doesn't wait forever by default. However, please note that there are different parameters for the initial wait for connection during boot up

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Heikki Linnakangas
On 08.10.2010 06:41, Fujii Masao wrote: On Thu, Oct 7, 2010 at 3:01 AM, Markus Wannermar...@bluegap.ch wrote: Of course, it doesn't make sense to wait-forever on *every* standby that ever gets added. Quorum commit is required, yes (and that's what this thread is about, IIRC). But with quorum

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Simon Riggs
On Fri, 2010-10-08 at 09:52 +0200, Markus Wanner wrote: One addendum: a timeout increases availability at the cost of increased danger of data loss and higher complexity. Don't use it, just increase (N - k) instead. Completely agree. -- Simon Riggs www.2ndQuadrant.com

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 05:41 AM, Fujii Masao wrote: But, even with quorum commit, if you choose wait-forever option, failover would decrease availability. Right after the failover, no standby has connected to new master, so if quorum = 1, all the transactions must wait for a while. That's a point,

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Heikki Linnakangas
On 08.10.2010 01:25, Simon Riggs wrote: On Thu, 2010-10-07 at 13:44 -0400, Aidan Van Dyk wrote: To get non-stale responses, you can only query those k=3 servers. But you've shot your self in the foot because you don't know which 3/10 those will be. The other 7 *are* stale (by definition).

Re: I: [HACKERS] About Our CLUSTER implementation is pessimal patch

2010-10-08 Thread Leonardo Francalanci
Applied with some significant editorialization. The biggest problem I found was that the code for expression indexes didn't really work, and would leak memory like there's no tomorrow even when it did work. Sorry I couldn't write the way it was supposed to... I'll look at the differences

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Simon Riggs
On Fri, 2010-10-08 at 10:56 +0300, Heikki Linnakangas wrote: Or what kind of customers do you think really need a no-lag solution for read-only queries? In the LAN case, the lag of async rep is negligible and in the WAN case the latencies of sync rep are prohibitive. There is a very

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-08 Thread Yeb Havinga
Robert Haas wrote: Yes, let's please just implement something simple and get it committed. k = 1. Two GUCs (synchronous_standbys = name, name, name and synchronous_waitfor = none|recv|fsync|apply), SUSET so you can change it per txn. Done. We can revise it *the day after it's committed* if

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Heikki Linnakangas
On 08.10.2010 11:25, Simon Riggs wrote: On Fri, 2010-10-08 at 10:56 +0300, Heikki Linnakangas wrote: Or what kind of customers do you think really need a no-lag solution for read-only queries? In the LAN case, the lag of async rep is negligible and in the WAN case the latencies of sync rep are

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 10:27 AM, Heikki Linnakangas wrote: Synchronous replication in the 'replay' mode is supposed to guarantee exactly that, no? The master may lag behind, so it's not strictly speaking the same data. Regards Markus Wanner -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 09:56 AM, Heikki Linnakangas wrote: Imagine a web application that's mostly read-only, but a user can modify his own personal details like name and address, for example. Imagine that the user changes his street address and clicks 'save', causing an UPDATE, and the next query

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Simon Riggs
On Fri, 2010-10-08 at 11:27 +0300, Heikki Linnakangas wrote: On 08.10.2010 11:25, Simon Riggs wrote: On Fri, 2010-10-08 at 10:56 +0300, Heikki Linnakangas wrote: Or what kind of customers do you think really need a no-lag solution for read-only queries? In the LAN case, the lag of async

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 01:44 AM, Greg Smith wrote: They'll use Sync Rep to maximize the odds a system failure doesn't cause any transaction loss. They'll use good quality hardware on the master so it's unlikely to fail. ..unlikely to fail? Ehm.. is that you speaking, Greg? ;-) But when the

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 11:00 AM, Simon Riggs wrote: From the perspective of an observer, randomly selecting a standby for load balancing purposes: No, they are not guaranteed to see the latest answer, nor even can they find out whether what they are seeing is the latest answer. I completely agree. The

[HACKERS] compiling C library under mingw

2010-10-08 Thread Marios Vodas
I have a library that compiles fine under linux. But when I try to compile it under mingw on windows 7 I get the following errors. gcc.exe -c -O2 -I/C/PostgreSQL/9.0/include/server -I/C/PostgreSQL/9.0/include/server/utils -I/C/PostgreSQL/9.0/include/server/access

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Dimitri Fontaine
Markus Wanner mar...@bluegap.ch writes: ..and how do you make sure you are not marking your second standby as degraded just because it's currently lagging? Well, in sync rep, a standby that's not able to stay under the timeout is degraded. Full stop. The presence of the timeout (or its value

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 11:41 AM, Dimitri Fontaine wrote: Same old story. Either you're able to try and fix the master so that you don't lose any data and don't even have to check for that, or you take a risk and start from a non synced standby. It's all availability against durability again. ..and a

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Dimitri Fontaine
Markus Wanner mar...@bluegap.ch writes: ..and a whole lot of manual work, that's prone to error for something that could easily be automated So, the master just crashed, first standby is dead and second ain't in sync. What's the easy and automated way out? Sorry, I need a hand here. --

Re: [HACKERS] On Scalability

2010-10-08 Thread Vincenzo Romano
2010/10/7 Simon Riggs si...@2ndquadrant.com: On Thu, 2010-10-07 at 14:10 +0200, Vincenzo Romano wrote: Making these things sub-linear (whether not O(log n) or even O(1) ), provided that there's  way to, would make this RDBMS more appealing to enterprises. I mean also partial indexes (as an

Re: [HACKERS] Git cvsserver serious issue

2010-10-08 Thread Andrew Dunstan
On 10/08/2010 02:09 AM, Magnus Hagander wrote: On Fri, Oct 8, 2010 at 03:52, Andrew Dunstanand...@dunslane.net wrote: There's a simpler solution which I have just tested. Instead of patching, use the Pg driver instead of SQLite. Set the dbname to %m. If the database doesn't exist the cvs

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 4:29 AM, Yeb Havinga yebhavi...@gmail.com wrote: Robert Haas wrote: Yes, let's please just implement something simple and get it committed.  k = 1.  Two GUCs (synchronous_standbys = name, name, name and synchronous_waitfor = none|recv|fsync|apply), SUSET so you can

Re: [HACKERS] Git cvsserver serious issue

2010-10-08 Thread Marko Kreen
On Fri, Oct 8, 2010 at 3:13 PM, Andrew Dunstan and...@dunslane.net wrote: On 10/08/2010 02:09 AM, Magnus Hagander wrote: On Fri, Oct 8, 2010 at 03:52, Andrew Dunstanand...@dunslane.net  wrote: There's a simpler solution which I have just tested. Instead of patching, use the Pg driver instead

Re: [HACKERS] Git cvsserver serious issue

2010-10-08 Thread Andrew Dunstan
On 10/08/2010 09:15 AM, Marko Kreen wrote: On Fri, Oct 8, 2010 at 3:13 PM, Andrew Dunstanand...@dunslane.net wrote: On 10/08/2010 02:09 AM, Magnus Hagander wrote: On Fri, Oct 8, 2010 at 03:52, Andrew Dunstanand...@dunslane.netwrote: There's a simpler solution which I have just tested.

Re: [HACKERS] Timeout and Synch Rep

2010-10-08 Thread Fujii Masao
On Fri, Oct 8, 2010 at 4:50 AM, Josh Berkus j...@agliodbs.com wrote: In my effort to make the discussion around the design decisions of synch rep less opaque, I'm starting a separate thread about what has developed to be one of the more contentious issues. I'm going to champion timeouts

Re: [HACKERS] Timeout and Synch Rep

2010-10-08 Thread Thom Brown
On 7 October 2010 20:50, Josh Berkus j...@agliodbs.com wrote: All, So, if something happens to the standby, and it cannot return an ack in 30 seconds, they would like it to degrade to asynch mode.  At that point, they would also like to trigger a nagios alert which will wake up the sysadmin

Re: [HACKERS] a few small bugs in plpgsql

2010-10-08 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: Now I understand to interpretation. But there is little bit difficult to understand to error message. Can be message enhanced to show a complete expression? It does already: regression=# select foo(); ERROR: column label does not exist LINE 2:

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Fujii Masao
On Fri, Oct 8, 2010 at 5:07 PM, Markus Wanner mar...@bluegap.ch wrote: On 10/08/2010 04:01 AM, Fujii Masao wrote: Really? I don't think that ko-count=0 means wait-forever. Telling from the documentation, I'd also say it doesn't wait forever by default. However, please note that there are

Re: [HACKERS] proposal: plpgsql, solution for derivated types of parameters

2010-10-08 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: the current plpgsql syntax doesn't offer a functionality to define some variable with type as element of some other array variable or reverse order. The primary goal of this proposal is enahancing plpgsql for better working with polymorphic types.

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: I don't see this as needing any implementation any more complicated than the usual way such timeouts are handled. Note how long you've been trying to reach the standby. Default to -1 for forever. And if you hit the timeout, mark the standby as

Re: [HACKERS] proposal: plpgsql, solution for derivated types of parameters

2010-10-08 Thread Pavel Stehule
2010/10/8 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: the current plpgsql syntax doesn't offer a functionality to define some variable with type as element of some other array variable or reverse order. The primary goal of this proposal is enahancing plpgsql for

Re: [HACKERS] a few small bugs in plpgsql

2010-10-08 Thread Pavel Stehule
2010/10/8 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: Now I understand to interpretation. But there is little bit difficult to understand to error message. Can be message enhanced to show a complete expression? It does already: regression=# select foo();

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Fujii Masao
On Fri, Oct 8, 2010 at 5:10 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Do we really need that? Yes. But if there is no unsent WAL when the master goes down, we can start new standby without new backup by copying the timeline history file from new master to new standby and

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 04:11 PM, Tom Lane wrote: Actually, #2 seems rather difficult even if you want it. Presumably you'd like to keep that state in reliable storage, so it survives master crashes. But how you gonna commit a change to that state, if you just lost every standby (suppose master's

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Well, actually, that's *considerably* more complicated than just a timeout. How are you going to mark the standby as degraded? The standby can't keep that information, because it's not even connected when the master makes the decision. ISTM that this

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 12:05 PM, Dimitri Fontaine wrote: Markus Wanner mar...@bluegap.ch writes: ..and a whole lot of manual work, that's prone to error for something that could easily be automated So, the master just crashed, first standby is dead and second ain't in sync. What's the easy and

[HACKERS] Total memory allocated to postgres

2010-10-08 Thread vamsi krishna
Hi I want to know how much main memory is actually allocated to be the maximum memory for postgresql, because my hardware may have 4GB main memory but at the end of the day, only 1GB may be allocated for postgres. I want to set (increase) it manually. Can someone tell me how to do it? In

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Tom Lane
Markus Wanner mar...@bluegap.ch writes: On 10/08/2010 04:11 PM, Tom Lane wrote: Actually, #2 seems rather difficult even if you want it. Presumably you'd like to keep that state in reliable storage, so it survives master crashes. But how you gonna commit a change to that state, if you just

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 04:38 PM, Tom Lane wrote: Markus Wanner mar...@bluegap.ch writes: IIUC you seem to assume that the master node keeps its master role. But users who value availability a lot certainly want automatic fail-over, Huh? Surely loss of the slaves shouldn't force a failover. Maybe

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Simon Riggs
On Fri, 2010-10-08 at 10:11 -0400, Tom Lane wrote: 1. a unique identifier for each standby (not just role names that multiple standbys might share); That is difficult because each standby is identical. If a standby goes down, people can regenerate a new standby by taking a copy from another

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Fujii Masao
On Fri, Oct 8, 2010 at 5:16 PM, Markus Wanner mar...@bluegap.ch wrote: On 10/08/2010 05:41 AM, Fujii Masao wrote: But, even with quorum commit, if you choose wait-forever option, failover would decrease availability. Right after the failover, no standby has connected to new master, so if

Re: [HACKERS] Git cvsserver serious issue

2010-10-08 Thread Aidan Van Dyk
On Fri, Oct 8, 2010 at 8:13 AM, Andrew Dunstan and...@dunslane.net wrote: That's what the default SQLite setup does anyway. The only difference here is that we are leveraging the fact that with the Pg driver the database must first exist. Of course, with Pg the database can live on a different

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Fujii Masao
On Fri, Oct 8, 2010 at 6:00 PM, Simon Riggs si...@2ndquadrant.com wrote: From the perspective of an observer, randomly selecting a standby for load balancing purposes: No, they are not guaranteed to see the latest answer, nor even can they find out whether what they are seeing is the latest

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Simon Riggs
On Fri, 2010-10-08 at 23:55 +0900, Fujii Masao wrote: On Fri, Oct 8, 2010 at 6:00 PM, Simon Riggs si...@2ndquadrant.com wrote: From the perspective of an observer, randomly selecting a standby for load balancing purposes: No, they are not guaranteed to see the latest answer, nor even can

Re: [HACKERS] Git cvsserver serious issue

2010-10-08 Thread Andrew Dunstan
On 10/08/2010 10:53 AM, Aidan Van Dyk wrote: On Fri, Oct 8, 2010 at 8:13 AM, Andrew Dunstanand...@dunslane.net wrote: That's what the default SQLite setup does anyway. The only difference here is that we are leveraging the fact that with the Pg driver the database must first exist. Of

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 04:47 PM, Simon Riggs wrote: Yes, I really want to avoid such issues and likely complexities we get into trying to solve them. In reality they should not be common because it only happens if the sysadmin has not configured sufficient number of redundant standbys. Well, full

Re: [HACKERS] Sync Replication with transaction-controlled durability

2010-10-08 Thread Simon Riggs
On Tue, 2010-09-14 at 18:48 +0100, Simon Riggs wrote: I'm working on a patch to implement synchronous replication for PostgreSQL, with user-controlled durability specified on the master. The design also provides high throughput by allowing concurrent processes to handle the WAL stream. The

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Markus Wanner
On 10/08/2010 04:48 PM, Fujii Masao wrote: I believe many systems require write-availability. Sure. Make sure you have enough standbies to fail over to. (I think there are even more situations where read-availability is much more important, though). Start with 0 (i.e. replication off), then

[HACKERS] Bug in information_schema: column names don't match spec

2010-10-08 Thread Tom Lane
While reading the proposed triggers-on-views patch I noticed that information_schema.sql defines these columns in the TRIGGERS view: CAST( CASE WHEN t.tgtype 2 = 2 THEN 'BEFORE' WHEN t.tgtype 64 = 64 THEN 'INSTEAD OF' ELSE 'AFTER' END AS character_data) AS

Re: [HACKERS] WIP: Triggers on VIEWs

2010-10-08 Thread Tom Lane
Bernd Helmle maili...@oopsware.de writes: I would like to do some more tests/review, but going to mark this patch as Ready for Committer, so that someone more qualified on the executor part can have a look on it during this commitfest, if that's okay for us? I've started looking at this

[HACKERS] Italian PGDay 2010, Call for papers

2010-10-08 Thread Gabriele Bartolini
The fourth edition of the Italian PostgreSQL Day (PGDay.IT 2010) will be held on December 10 in Rome. It will cover topics for users, developers and contributors to PostgreSQL, as well as decision and policy makers. For more information about the conference, please visit the website at

Re: [HACKERS] On Scalability

2010-10-08 Thread Greg Stark
On Fri, Oct 8, 2010 at 3:20 AM, Vincenzo Romano vincenzo.rom...@notorand.it wrote: Do the same conclusions apply to partial indexes? I mean, if I have a large number (n=100 or n=1000) of partial indexes on a single very large table (m=10**12), how good is the planner to choose the right

Re: [HACKERS] Sync Replication with transaction-controlled durability

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 11:10 AM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, 2010-09-14 at 18:48 +0100, Simon Riggs wrote: I'm working on a patch to implement synchronous replication for PostgreSQL, with user-controlled durability specified on the master. The design also provides high

Re: [HACKERS] GIN vs. Partial Indexes

2010-10-08 Thread Josh Berkus
IMO, what's needed is to fix GIN so it doesn't go insane for empty values or non-restrictive queries, by ensuring there's at least one index entry for every row. This has been discussed before; see the TODO section for GIN. Well, what is that fix waiting on, then? Oleg, Teodor? We may even

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-08 Thread Josh Berkus
On 10/07/2010 06:38 PM, Robert Haas wrote: Yes, let's please just implement something simple and get it committed. k = 1. Two GUCs (synchronous_standbys = name, name, name and synchronous_waitfor = none|recv|fsync|apply), SUSET so you can change it per txn. Done. We can revise it *the day

Re: [HACKERS] Timeout and Synch Rep

2010-10-08 Thread Josh Berkus
So, if something happens to the standby, and it cannot return an ack in 30 seconds, they would like it to degrade to asynch mode. At that point, they would also like to trigger a nagios alert which will wake up the sysadmin with flashing red lights. How? TBD, and before 9.1. It's clear to

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Josh Berkus
And, I'd like to know whether the master waits forever because of the standby failure in other solutions such as Oracle DataGuard, MySQL semi-synchronous replication. MySQL used to be fond of simiply failing sliently. Not sure what 5.4 does, or Oracle. In any case MySQL's replication has

Re: [HACKERS] WIP: Triggers on VIEWs

2010-10-08 Thread Dean Rasheed
On 8 October 2010 16:50, Tom Lane t...@sss.pgh.pa.us wrote: Bernd Helmle maili...@oopsware.de writes: I would like to do some more tests/review, but going to mark this patch as Ready for Committer, so that someone more qualified on the executor part can have a look on it during this

Re: [HACKERS] WIP: Triggers on VIEWs

2010-10-08 Thread Tom Lane
Dean Rasheed dean.a.rash...@gmail.com writes: On 8 October 2010 16:50, Tom Lane t...@sss.pgh.pa.us wrote: I've started looking at this patch now.  I think it would have been best submitted as two patches: one to add the SQL-spec INSTEAD OF trigger functionality, and a follow-on to extend

Re: [HACKERS] GIN vs. Partial Indexes

2010-10-08 Thread Robert Haas
On Thu, Oct 7, 2010 at 10:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: I thought we fixed this in 8.4.4, but apparently not.  In the event that you have a GIN index containing a WHERE clause which is sufficiently restrictive, PostgreSQL will attempt to use

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 12:29 PM, Josh Berkus j...@agliodbs.com wrote: On 10/07/2010 06:38 PM, Robert Haas wrote: Yes, let's please just implement something simple and get it committed.  k = 1.  Two GUCs (synchronous_standbys = name, name, name and synchronous_waitfor = none|recv|fsync|apply),

Re: [HACKERS] WIP: Triggers on VIEWs

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 11:50 AM, Tom Lane t...@sss.pgh.pa.us wrote:  I think the right thing here is to replace before with a three-valued enum, perhaps called timing, so as to force people to take another look at any code that touches the field directly. +1. That seems much nicer. Although

Re: [HACKERS] GIN vs. Partial Indexes

2010-10-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Oct 7, 2010 at 10:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: IMO, what's needed is to fix GIN so it doesn't go insane for empty values or non-restrictive queries, by ensuring there's at least one index entry for every row.  This has been

Re: [HACKERS] WIP: Triggers on VIEWs

2010-10-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Oct 8, 2010 at 11:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Although we already have macros TRIGGER_FIRED_AFTER/TRIGGER_FIRED_BEFORE that seem to mask the details here, the changes you had to make in contrib illustrate that the macros' callers

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Rob Wultsch
* On 10/8/10, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Oct 8, 2010 at 5:10 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Do we really need that? Yes. But if there is no unsent WAL when the master goes down, we can start new standby without new backup by copying

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Heikki Linnakangas
On 08.10.2010 17:26, Fujii Masao wrote: On Fri, Oct 8, 2010 at 5:10 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Do we really need that? Yes. But if there is no unsent WAL when the master goes down, we can start new standby without new backup by copying the timeline

Re: [HACKERS] WIP: Triggers on VIEWs

2010-10-08 Thread Tom Lane
BTW, while I'm looking at this: it seems like the index arrays in struct TrigDesc are really a lot more complication than they are worth. It'd be far easier to dispense with them and instead iterate through the main trigger array, skipping any triggers whose tgtype doesn't match what we need. If

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Greg Smith
Markus Wanner wrote: ..and how do you make sure you are not marking your second standby as degraded just because it's currently lagging? Effectively degrading the utterly needed one, because your first standby has just bitten the dust? People are going to monitor the standby lag. If it

[HACKERS] getting set up on git (finally)

2010-10-08 Thread Joe Conway
I'm finally trying to get current with the switch to git, following this wiki page: http://wiki.postgresql.org/wiki/Committing_with_Git Specifically, I am trying to do: Dependent Clone per Branch, Pushing and Pulling From a Local Repository -plus- Clone Locally, Repoint Origin

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Greg Smith
Tom Lane wrote: How are you going to mark the standby as degraded? The standby can't keep that information, because it's not even connected when the master makes the decision. From a high level, I'm assuming only that the master has a list in memory of the standby system(s) it believes are

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Simon Riggs
On Fri, 2010-10-08 at 17:06 +0200, Markus Wanner wrote: Well, full cluster outages are infrequent, but sadly cannot be avoided entirely. (Murphy's laughing). IMO we should be prepared to deal with those. I've described how I propose to deal with those. I'm not waving away these issues, just

Re: [HACKERS] Issues with Quorum Commit

2010-10-08 Thread Simon Riggs
On Fri, 2010-10-08 at 16:34 -0400, Greg Smith wrote: Tom Lane wrote: How are you going to mark the standby as degraded? The standby can't keep that information, because it's not even connected when the master makes the decision. From a high level, I'm assuming only that the master has

Re: [HACKERS] GIN vs. Partial Indexes

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 1:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Oct 7, 2010 at 10:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: IMO, what's needed is to fix GIN so it doesn't go insane for empty values or non-restrictive queries, by ensuring

Re: [HACKERS] Sync Replication with transaction-controlled durability

2010-10-08 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: It seems like it would be more helpful if you were working on implementing a design that had more than one vote. As far as I can tell, we have rough consensus that for the first commit we should only worry about the case where k = 1; that is, only one

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2010-10-08 Thread Marko Tiikkaja
On 2010-10-04 5:31 PM +0300, Tom Lane wrote: Marko Tiikkajamarko.tiikk...@cs.helsinki.fi writes: Nope. I think I grepped contrib/ at some point and none of those were using pg_parse_and_rewrite() so this is all just speculation. And yes, it's not really part of any stable API but breaking

Re: [HACKERS] Sync Replication with transaction-controlled durability

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 5:59 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: It seems like it would be more helpful if you were working on implementing a design that had more than one vote.  As far as I can tell, we have rough consensus that for the

[HACKERS] Docs for archive_cleanup_command are poor

2010-10-08 Thread Brendan Jurd
Hi folks, I have just set up HS+SR for the first time, and for the most part, the docs were excellent. The one exception for me was the discussion of archive_cleanup_command. This is a pretty important part of constructing a healthy standby server, and IMO the docs don't give it the treatment

[HACKERS] Keywords in pg_hba.conf should be field-specific

2010-10-08 Thread Brendan Jurd
Hi folks, One of the speedbumps I hit when setting up HS+SR was naming the user the slave would connect as for streaming replication. At first I picked 'replication', which seemed quite natural to me (and I don't doubt will seem natural to others as well). When I started up the slave, I got

Re: [HACKERS] GIN vs. Partial Indexes

2010-10-08 Thread David E. Wheeler
On Oct 8, 2010, at 1:47 PM, Tom Lane wrote: How so? In a typical application, there would not likely be very many such rows --- we're talking about cases like documents containing zero indexable words. In any case, the problem right now is that GIN has significant functional limitations

Re: [HACKERS] Keywords in pg_hba.conf should be field-specific

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 10:13 PM, Brendan Jurd dire...@gmail.com wrote: One of the speedbumps I hit when setting up HS+SR was naming the user the slave would connect as for streaming replication.  At first I picked 'replication', which seemed quite natural to me (and I don't doubt will seem

Re: [HACKERS] getting set up on git (finally)

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 4:31 PM, Joe Conway m...@joeconway.com wrote: I'm finally trying to get current with the switch to git, following this wiki page:  http://wiki.postgresql.org/wiki/Committing_with_Git Specifically, I am trying to do: Dependent Clone per Branch, Pushing

Re: [HACKERS] Total memory allocated to postgres

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 10:37 AM, vamsi krishna vamsikrishna1...@gmail.com wrote: I want to know how much main memory  is actually allocated to be the maximum memory for postgresql, because my hardware may have 4GB main memory but at the end of the day, only 1GB may be allocated for postgres.