Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Heikki Linnakangas
On 02.01.2011 00:40, Josh Berkus wrote: On 1/1/11 5:59 AM, Stefan Kaltenbrunner wrote: well you keep saying that but to be honest I cannot really even see a usecase for me - what is only a random one of a set of servers is sync at any time and I don't really know which one. My usecases would al

Re: [HACKERS] SSI SLRU low-level functions first cut

2011-01-02 Thread Heikki Linnakangas
On 01.01.2011 23:21, Kevin Grittner wrote: I've got low-level routines coded for interfacing predicate.c to SLRU to handle old committed transactions, so that SSI can deal with situations where a large number of transactions are run during the lifetime of a single serializable transaction. I'm

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Stefan Kaltenbrunner
On 01/02/2011 09:35 AM, Heikki Linnakangas wrote: On 02.01.2011 00:40, Josh Berkus wrote: On 1/1/11 5:59 AM, Stefan Kaltenbrunner wrote: well you keep saying that but to be honest I cannot really even see a usecase for me - what is only a random one of a set of servers is sync at any time and

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 10:35 +0200, Heikki Linnakangas wrote: BTW, there's a bunch of replication related stuff that we should work to close, that are IMHO more important than synchronous replication. Like making the standby follow timeline changes, to make failovers smoother, and the facility

Re: [HACKERS] [COMMITTERS] pgsql: Basic foreign table support.

2011-01-02 Thread Magnus Hagander
Typo, I think: - (errmsg(skipping \%s\ --- cannot vacuum indexes, views, or special system tables, + (errmsg(skipping \%s\ --- cannot only non-tables or special system tables, //Magnus On Sun, Jan 2, 2011 at 05:48, Robert Haas rh...@postgresql.org wrote: Basic

Re: [HACKERS] management of large patches

2011-01-02 Thread Magnus Hagander
On Sun, Jan 2, 2011 at 06:32, Robert Haas robertmh...@gmail.com wrote: We're coming the end of the 9.1 development cycle, and I think that there is a serious danger of insufficient bandwidth to handle the large patches we have outstanding.  For my part, I am hoping to find the bandwidth to

Re: [HACKERS] Libpq PGRES_COPY_BOTH - version compatibility

2011-01-02 Thread Magnus Hagander
On Wed, Dec 29, 2010 at 19:49, Robert Haas robertmh...@gmail.com wrote: On Dec 29, 2010, at 10:14 AM, Magnus Hagander mag...@hagander.net wrote: We can be held responsible for the packaging decisions if they use *our* make install commands, imho. Yep. So, as I see it there are two ways of

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sat, 2011-01-01 at 22:11 -0500, Aidan Van Dyk wrote: On Sat, Jan 1, 2011 at 6:08 PM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2011-01-01 at 14:40 -0800, Josh Berkus wrote: Standby in general deals with the A,D,R triangle (Availability, Durability, Response time). Any one

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 10:35 +0200, Heikki Linnakangas wrote: Frankly, if Simon hadn't already submitted code, I'd be pushing for single-standby-only for 9.1, instead of any one. Yes, we are awfully late, but let's not panic. Yes, we're about a year late. Getting a simple feature like

[HACKERS] Support for negative index values in array fetching

2011-01-02 Thread Valtonen, Hannu
Hi, I ran into the problem of getting the last n elements out of an array and while some workarounds do exist: (http://stackoverflow.com/questions/2949881/getting-the-last-element-of-a-postgres-array-declaratively) I was still annoyed that I couldn't just ask for the last n values in an array

Re: [HACKERS] Extension upgrade, patch v0: debug help needed

2011-01-02 Thread Dimitri Fontaine
Dimitri Fontaine dimi...@2ndquadrant.fr writes: The problem occurs on ALTER OPERATOR FAMILY ... SET EXTENSION, that's what dichotomy on the citext.upgrade.sql tells me. The code in question was copy/pasted from the SET SCHEMA code path in gram.y then other related files. So I just tested a

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Hannu Krosing
On 2.1.2011 5:36, Robert Haas wrote: On Sat, Jan 1, 2011 at 6:54 AM, Simon Riggssi...@2ndquadrant.com wrote: Yes, working out the math is a good idea. Things are much clearer if we do that. Let's assume we have 98% availability on any single server. 1. Having one primary and 2 standbys,

Re: [HACKERS] Extension upgrade, patch v0: debug help needed

2011-01-02 Thread Dimitri Fontaine
Dimitri Fontaine dimi...@2ndquadrant.fr writes: make -C contrib/citext install psql -f .../head/share/contrib/citext.sql psql dim=# do $$ begin execute 'alter operator class public.citext_ops using btree set schema utils'; end; $$; server closed the connection unexpectedly

Re: [HACKERS] Support for negative index values in array fetching

2011-01-02 Thread Florian Pflug
On Jan2, 2011, at 11:45 , Valtonen, Hannu wrote: I ran into the problem of getting the last n elements out of an array and while some workarounds do exist: (http://stackoverflow.com/questions/2949881/getting-the-last-element-of-a-postgres-array-declaratively) I was still annoyed that I

Re: [HACKERS] [COMMITTERS] pgsql: Basic foreign table support.

2011-01-02 Thread Robert Haas
On Sun, Jan 2, 2011 at 4:24 AM, Magnus Hagander mag...@hagander.net wrote: Typo, I think: -               (errmsg(skipping \%s\ --- cannot vacuum indexes, views, or special system tables, +               (errmsg(skipping \%s\ --- cannot only non-tables or special system tables, Oops, fixed.

Re: [HACKERS] Libpq PGRES_COPY_BOTH - version compatibility

2011-01-02 Thread Robert Haas
On Sun, Jan 2, 2011 at 4:36 AM, Magnus Hagander mag...@hagander.net wrote: On Wed, Dec 29, 2010 at 19:49, Robert Haas robertmh...@gmail.com wrote: On Dec 29, 2010, at 10:14 AM, Magnus Hagander mag...@hagander.net wrote: We can be held responsible for the packaging decisions if they use *our*

Re: [HACKERS] management of large patches

2011-01-02 Thread Robert Haas
On Sun, Jan 2, 2011 at 4:29 AM, Magnus Hagander mag...@hagander.net wrote: As for priority between those that *were* submitted earlier, and have been reworked (which is how the system is supposed to work), it's a lot harder. And TBH, I think we're going to have a problem getting all those

Base Backup Streaming (was: [HACKERS] Sync Rep Design)

2011-01-02 Thread Dimitri Fontaine
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: BTW, there's a bunch of replication related stuff that we should work to close, that are IMHO more important than synchronous replication. Like making the standby follow timeline changes, to make failovers smoother, and the

Re: [HACKERS] management of large patches

2011-01-02 Thread KaiGai Kohei
(2011/01/02 14:32), Robert Haas wrote: We're coming the end of the 9.1 development cycle, and I think that there is a serious danger of insufficient bandwidth to handle the large patches we have outstanding. For my part, I am hoping to find the bandwidth to two, MAYBE three major commits

Re: [HACKERS] [BUGS] BUG #5662: Incomplete view

2011-01-02 Thread Peter Eisentraut
On mån, 2010-12-06 at 14:47 +0200, Peter Eisentraut wrote: On sön, 2010-09-19 at 14:28 -0400, Tom Lane wrote: Or maybe we could implement that function, call it like this CAST((pg_sequence_parameters(c.oid)).max_value AS cardinal_number) AS maximum_value, and plan on

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sat, 2011-01-01 at 23:36 -0500, Robert Haas wrote: On Sat, Jan 1, 2011 at 6:54 AM, Simon Riggs si...@2ndquadrant.com wrote: Yes, working out the math is a good idea. Things are much clearer if we do that. Let's assume we have 98% availability on any single server. 1. Having one

Re: [HACKERS] SSI SLRU low-level functions first cut

2011-01-02 Thread Kevin Grittner
Heikki Linnakangas wrote: Nothing checking for the hi-bit flag AFAICS. I guess the code that uses that would do check it. Right. After getting this layer done, I went off to watch the Badgers in the Rose Bowl, leaving that coding for today. ;-) But wouldn't it be simpler to mark the

Re: [HACKERS] How to know killed by pg_terminate_backend

2011-01-02 Thread Tatsuo Ishii
Seems reasonable. Does the victim backend currently know why it has been killed? I don't think so. One idea is postmaster sets a flag in the shared memory area indicating it rceived SIGTERM before forwarding the signal to backends. Backend check the flag and if it's not set, it

Re: [HACKERS] How to know killed by pg_terminate_backend

2011-01-02 Thread Tatsuo Ishii
Seems reasonable. Does the victim backend currently know why it has been killed? I don't think so. One idea is postmaster sets a flag in the shared memory area indicating it rceived SIGTERM before forwarding the signal to backends. Backend check the flag and if it's not set, it

Re: [HACKERS] Libpq PGRES_COPY_BOTH - version compatibility

2011-01-02 Thread Peter Eisentraut
On tis, 2010-12-28 at 13:13 +0100, Magnus Hagander wrote: My pg_streamrecv no longer works with 9.1, because it returns PGRES_COPY_BOTH instead of PGRES_COPY_OUT when initating a copy. That's fine. So I'd like to make it work on both. Specifically, I would like it to check for

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Kevin Grittner
Simon Riggs wrote: On Sat, 2011-01-01 at 23:36 -0500, Robert Haas wrote: On Sat, Jan 1, 2011 at 6:54 AM, Simon Riggs wrote: Yes, working out the math is a good idea. Things are much clearer if we do that. Let's assume we have 98% availability on any single server. 1. Having one primary

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 08:08 -0600, Kevin Grittner wrote: I think you're talking about different metrics, and you're both right. With two servers configured in sync rep your chance of having an available (running) server is 99.9992%. The chance that you know that you have one that is totally

Re: [HACKERS] Base Backup Streaming

2011-01-02 Thread Heikki Linnakangas
On 02.01.2011 14:47, Dimitri Fontaine wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: BTW, there's a bunch of replication related stuff that we should work to close, that are IMHO more important than synchronous replication. Like making the standby follow timeline changes,

Re: [HACKERS] How to know killed by pg_terminate_backend

2011-01-02 Thread Tom Lane
Tatsuo Ishii is...@postgresql.org writes: Comments are welcome. This is a bad idea. It makes an already-poorly-tested code path significantly more fragile, in return for nothing of value. regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Heikki Linnakangas
On 02.01.2011 15:41, Simon Riggs wrote: On Sat, 2011-01-01 at 23:36 -0500, Robert Haas wrote: On Sat, Jan 1, 2011 at 6:54 AM, Simon Riggssi...@2ndquadrant.com wrote: Yes, working out the math is a good idea. Things are much clearer if we do that. Let's assume we have 98% availability on any

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Kevin Grittner
Simon Riggs wrote: Do you agree that requiring response from 2 sync standbys, or locking up, gives us 94% server availability, but 99.9992% data durability? I'm not sure how to answer that. The calculations so far have been based around up-time and the probabilities that you have a

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 11:11 -0600, Kevin Grittner wrote: Simon Riggs wrote: Do you agree that requiring response from 2 sync standbys, or locking up, gives us 94% server availability, but 99.9992% data durability? I'm not sure how to answer that. The calculations so far have been

Re: [HACKERS] Base Backup Streaming

2011-01-02 Thread Dimitri Fontaine
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: http://archives.postgresql.org/message-id/4c80d9b8.2020...@enterprisedb.com That just needs to be polished into shape, and documentation. Wow, cool! I don't know how but I've missed it. +1. Or maybe it would be better make it a

Re: [HACKERS] management of large patches

2011-01-02 Thread Kevin Grittner
Robert Haas wrote: - true serializability - not entirely sure of the status of this I try to keep the status section of the Wiki page up-to-date. I have just reviewed it and tweaked it for the latest events: http://wiki.postgresql.org/wiki/Serializable#Current_Status There are a number

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread MARK CALLAGHAN
On Thu, Dec 30, 2010 at 9:02 PM, Robert Haas robertmh...@gmail.com wrote: reads MySQL documentation I see now that you've tried to design this feature in a way that is similar to MySQL's offering, which does have some value.  But it appears to me that the documentation you've written here is

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Jeff Janes
On Sat, Jan 1, 2011 at 8:35 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2011-01-01 at 05:13 -0800, Jeff Janes wrote: On 12/31/10, Simon Riggs si...@2ndquadrant.com wrote: 2. sync does not guarantee that the updates to the standbys are in any way coordinated. You can run a query on

Re: [HACKERS] managment of large patches

2011-01-02 Thread pasman pasmański
Hello. Maybe are any often bugs? they may be found by more asserts to track internal state of structures. Or tools like lastly developed script for c++ keywords. -- Sent from my mobile device pasman -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 12:13 -0800, MARK CALLAGHAN wrote: On Thu, Dec 30, 2010 at 9:02 PM, Robert Haas robertmh...@gmail.com wrote: reads MySQL documentation I see now that you've tried to design this feature in a way that is similar to MySQL's offering, which does have some value. But it

Re: [HACKERS] Base Backup Streaming

2011-01-02 Thread Magnus Hagander
On Sun, Jan 2, 2011 at 18:53, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: http://archives.postgresql.org/message-id/4c80d9b8.2020...@enterprisedb.com That just needs to be polished into shape, and documentation. I have an

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 18:54 +0200, Heikki Linnakangas wrote: I believe we all agree that there's different use cases that require different setups. Both first-past-the-post and wait-for-all-to-ack have their uses. Robert's analysis is that first-past-the-post doesn't actually improve the

Re: [HACKERS] Base Backup Streaming

2011-01-02 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes: Yes, especially since we discussed it in Stuttgart. I guess it may have been during the party... I remember we talked about it, I didn't remember a patch had reached the list… Yes, if it should go in any of the current binaries, initdb would be the

Re: [HACKERS] and it's not a bunny rabbit, either

2011-01-02 Thread Peter Eisentraut
On lör, 2011-01-01 at 17:21 -0500, Robert Haas wrote: I don't see anything wrong with having 20 or 30 messages of variants of foo cannot be used on bar without placeholders. Well, that's OK with me. It seems a little grotty, but manageably so. Questions: 1. Should we try to

Re: [HACKERS] contrib/snapshot

2011-01-02 Thread Jim Nasby
On Dec 31, 2010, at 1:35 PM, Joel Jacobson wrote: 2010/12/31 Simon Riggs si...@2ndquadrant.com Please call it something other than snapshot. There's already about 3 tools called something similar and a couple of different meanings of the term in the world of Postgres. Thanks, good point.

Re: [HACKERS] C++ keywords in headers (was Re: [GENERAL] #include funcapi.h)

2011-01-02 Thread Peter Geoghegan
I believe that Dave Page wants to move to building pg for windows using visual C++ 2010 some time this year. That alone may be enough of a reason to check for C++0x keywords in headers: http://blogs.msdn.com/b/vcblog/archive/2010/04/06/c-0x-core-language-features-in-vc10-the-table.aspx I think

Re: [HACKERS] How to know killed by pg_terminate_backend

2011-01-02 Thread Tatsuo Ishii
Tatsuo Ishii is...@postgresql.org writes: Comments are welcome. This is a bad idea. It makes an already-poorly-tested code path significantly more fragile, in return for nothing of value. Are you saying that procsignal.c is the already-poorly-tested one? If so, why? As for value, I have

Re: [HACKERS] Recovery conflict monitoring

2011-01-02 Thread Simon Riggs
On Mon, 2010-12-27 at 14:39 +0100, Magnus Hagander wrote: On Thu, Dec 23, 2010 at 13:09, Magnus Hagander mag...@hagander.net wrote: This patch adds counters and views to monitor hot standby generated recovery conflicts. It extends the pg_stat_database view with one column with the total

Re: [HACKERS] page compression

2011-01-02 Thread Simon Riggs
On Tue, 2010-12-28 at 09:10 -0600, Andy Colson wrote: I know its been discussed before, and one big problem is license and patent problems. Would like to see a design for that. There's a few different ways we might want to do that, and I'm interested to see if its possible to get compressed

Re: [HACKERS] contrib/snapshot

2011-01-02 Thread Joel Jacobson
2011/1/2 Jim Nasby j...@nasby.net Renamed to fsnapshot. Is it actually limited to functions? ISTM this concept would be valuable for anything that's not in pg_class (in other words, anything that doesn't have user data in it). My ambition is to primarily support functions. Support for

Re: [HACKERS] contrib/snapshot

2011-01-02 Thread Joel Jacobson
2011/1/3 Joel Jacobson j...@gluefinance.com 2011/1/2 Jim Nasby j...@nasby.net Is it actually limited to functions? ISTM this concept would be valuable for anything that's not in pg_class (in other words, anything that doesn't have user data in it). Instead of limiting the support to

Re: [HACKERS] contrib/snapshot

2011-01-02 Thread Andrew Dunstan
On 01/02/2011 07:44 PM, Joel Jacobson wrote: Also, I'm not sure why this needs to be in contrib vs pgFoundry. Good point. It's actually in neither of them right now, it's only at github.com http://github.com :) I merely used the prefix contrib/ in the subject line to indicate it's not

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Hannu Krosing
On 2.1.2011 5:36, Robert Haas wrote: On Sat, Jan 1, 2011 at 6:54 AM, Simon Riggssi...@2ndquadrant.com wrote: Yes, working out the math is a good idea. Things are much clearer if we do that. Let's assume we have 98% availability on any single server. 1. Having one primary and 2 standbys,

Re: [HACKERS] Re: new patch of MERGE (merge_204) a question about duplicated ctid

2011-01-02 Thread Greg Smith
Marko Tiikkaja wrote: I'm confused. Are you saying that the patch is supposed to lock the table against concurrent INSERT/UPDATE/DELETE/MERGE? Because I don't see it in the patch, and the symptoms you're having are a clear indication of the fact that it's not happening. I also seem to

[HACKERS] Visual Studio 2010/Windows SDK 7.1 support

2011-01-02 Thread Brar Piening
Hi, i'v created a patch enables support for building PostgreSQL with Visual Studio 2010 or Microsoft Windows SDK for Windows 7 and .NET Framework 4 (Windows SDK 7.1). You can grab it from http://www.piening.info/VS2010.patch It only touches the .pl, .pm and .bat files in src/tools/msvc so