Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 11:26, Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: Simon Riggs si...@2ndquadrant.com wrote: The proposal is to use WAL to generate the logical change stream. That has been shown in testing to be around x4 faster than having a separate

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Heikki Linnakangas
On 20.06.2012 01:27, Kevin Grittner wrote: Andres Freundand...@2ndquadrant.com wrote: Yes, thats definitely a valid use-case. But that doesn't preclude the other - also not uncommon - use-case where you want to have different master which all contain up2date data. I agree. I was just

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 08:35, Robert Haas robertmh...@gmail.com wrote: I expect it would be fine to have a tool that pulls LCRs out of WAL to prepare that to be sent to remote locations.  Is that what you have in mind? Yes.  I think it should be possible to generate LCRs from WAL, but I think that

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 14:40, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: The reason we need an origin id in this scenario is that otherwise this will happen: 1. A row is updated on node A 2. Node B receives the WAL record from A, and updates the corresponding row in B. This

Re: [HACKERS] pgbench--new transaction type

2012-06-20 Thread Heikki Linnakangas
On 01.06.2012 03:02, Jeff Janes wrote: I've attached a new patch which addresses several of your concerns, and adds the documentation. The description is much longer than the descriptions of other nearby options, which mostly just give a simple statement of what they do rather than a

Re: [HACKERS] pgbench--new transaction type

2012-06-20 Thread Simon Riggs
On 20 June 2012 15:32, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 01.06.2012 03:02, Jeff Janes wrote: I've attached a new patch which addresses several of your concerns, and adds the documentation.  The description is much longer than the descriptions of other nearby

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Heikki Linnakangas
On 20.06.2012 10:32, Simon Riggs wrote: On 20 June 2012 14:40, Heikki Linnakangas And I'm worried it might not even be enough in more complicated scenarios. It is not the only required conflict mechanism, and has never been claimed to be so. It is simply one piece of information needed, at

[HACKERS] Too frequent message of pgbench -i?

2012-06-20 Thread Tatsuo Ishii
Currently pgbench -i prints following message every 10k tuples created. fprintf(stderr, %d tuples done.\n, j); I think it's long time ago when the frequency of message seemed to be appropriate because computer is getting so fast these days and every 10k message seems to

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 15:45, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 20.06.2012 10:32, Simon Riggs wrote: On 20 June 2012 14:40, Heikki Linnakangas And I'm worried it might not even be enough in more complicated scenarios. It is not the only required conflict

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Heikki Linnakangas
On 20.06.2012 11:17, Simon Riggs wrote: On 20 June 2012 15:45, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 20.06.2012 10:32, Simon Riggs wrote: On 20 June 2012 14:40, Heikki Linnakangas And I'm worried it might not even be enough in more complicated scenarios. It is

[HACKERS] Release versioning inconsistency

2012-06-20 Thread Marti Raudsepp
Hi list, The recent 9.2 beta releases have used a slightly different numbering scheme than all previous releases. It used to be that tarballs for version $VER were always available at: http://ftp.postgresql.org/pub/source/v$VER/postgresql-$VER.tar.bz2 However, the new releases now use

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 16:23, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: It's only needed for multi-master replication, where the same table can be updated from multiple nodes. Just leave that out for now. There's plenty of functionality and issues left even without that. Huh?

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Heikki Linnakangas
On 20.06.2012 11:34, Simon Riggs wrote: On 20 June 2012 16:23, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: It's only needed for multi-master replication, where the same table can be updated from multiple nodes. Just leave that out for now. There's plenty of functionality and

Re: [HACKERS] Too frequent message of pgbench -i?

2012-06-20 Thread Heikki Linnakangas
On 20.06.2012 11:04, Tatsuo Ishii wrote: Currently pgbench -i prints following message every 10k tuples created. fprintf(stderr, %d tuples done.\n, j); I think it's long time ago when the frequency of message seemed to be appropriate because computer is getting so fast

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Andres Freund
On Wednesday, June 20, 2012 02:35:59 AM Robert Haas wrote: On Tue, Jun 19, 2012 at 5:59 PM, Christopher Browne cbbro...@gmail.com wrote: On Tue, Jun 19, 2012 at 5:46 PM, Robert Haas robertmh...@gmail.com wrote: Btw, what do you mean with conflating the stream? I don't really see that

Re: [HACKERS] Release versioning inconsistency

2012-06-20 Thread Magnus Hagander
On Wed, Jun 20, 2012 at 10:28 AM, Marti Raudsepp ma...@juffo.org wrote: Hi list, The recent 9.2 beta releases have used a slightly different numbering scheme than all previous releases. It used to be that tarballs for version $VER were always available at:  

Re: [HACKERS] Release versioning inconsistency

2012-06-20 Thread Marti Raudsepp
On Wed, Jun 20, 2012 at 12:18 PM, Magnus Hagander mag...@hagander.net wrote: (I do believe that using the v9.2.0beta marker is *better*, because then it sorts properly. But likely not enough much better to be inconsistent with previous versions) Good point. Maybe that's a reason to change the

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 16:44, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 20.06.2012 11:34, Simon Riggs wrote: On 20 June 2012 16:23, Heikki Linnakangas heikki.linnakan...@enterprisedb.com  wrote: It's only needed for multi-master replication, where the same table can be

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Peter Eisentraut
On sön, 2012-06-17 at 23:58 +0100, Peter Geoghegan wrote: So if you take the word Aßlar here - that is equivalent to Asslar, and so strcoll(Aßlar, Asslar) will return 0 if you have the right LC_COLLATE This is not actually correct. glibc will sort Asslar before Aßlar, and that is correct in

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

2012-06-20 Thread Andres Freund
Hi Robert, Hi All! On Wednesday, June 20, 2012 03:08:48 AM Robert Haas wrote: On Tue, Jun 19, 2012 at 2:23 PM, Andres Freund and...@2ndquadrant.com wrote: Well, the words are fuzzy, but I would define logical replication to be something which is independent of the binary format in which

Re: [HACKERS] performance regression in 9.2 when loading lots of small tables

2012-06-20 Thread Jeff Janes
On Tue, Jun 19, 2012 at 8:06 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 19, 2012 at 10:56 PM, Jeff Janes jeff.ja...@gmail.com wrote: But in the 9.2 branch, the slow phenotype was re-introduced in 1575fbcb795fc331f4, although perhaps the details of who is locking what differs.  I

Re: [HACKERS] WIP Patch: Selective binary conversion of CSV file foreign tables

2012-06-20 Thread Etsuro Fujita
Hi KaiGai-san, Thank you for the review. -Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kohei KaiGai Sent: Wednesday, June 20, 2012 1:26 AM To: Etsuro Fujita Cc: Robert Haas; pgsql-hackers@postgresql.org

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Peter Geoghegan
On 20 June 2012 11:00, Peter Eisentraut pete...@gmx.net wrote: On sön, 2012-06-17 at 23:58 +0100, Peter Geoghegan wrote: So if you take the word Aßlar here - that is equivalent to Asslar, and so strcoll(Aßlar, Asslar) will return 0 if you have the right LC_COLLATE This is not actually

Re: [HACKERS] libpq compression

2012-06-20 Thread Florian Pflug
On Jun19, 2012, at 17:36 , Robert Haas wrote: On Mon, Jun 18, 2012 at 1:42 PM, Martijn van Oosterhout klep...@svana.org wrote: On Sun, Jun 17, 2012 at 12:29:53PM -0400, Tom Lane wrote: The fly in the ointment with any of these ideas is that the configure list is not a list of exact cipher

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-20 Thread Andres Freund
On Wednesday, June 20, 2012 05:01:16 AM Robert Haas wrote: On Tue, Jun 19, 2012 at 4:22 PM, Andres Freund and...@2ndquadrant.com wrote: 1. dllist.h has double the element overhead by having an inline value pointer (which is not needed when embedding) and a pointer to the list (which I

Re: [HACKERS] Ability to listen on two unix sockets

2012-06-20 Thread Honza Horak
On 06/15/2012 05:40 PM, Honza Horak wrote: I realized the patch has some difficulties -- namely the socket path in the data dir lock file, which currently uses one port for socket and the same for interface. So to allow users to use arbitrary port for all unix sockets, we'd need to add

Re: [HACKERS] [ADMIN] pg_basebackup blocking all queries with horrible performance

2012-06-20 Thread Magnus Hagander
On Mon, Jun 11, 2012 at 6:06 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Jun 12, 2012 at 12:47 AM, Magnus Hagander mag...@hagander.net wrote: On Mon, Jun 11, 2012 at 5:37 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Jun 11, 2012 at 3:24 AM, Magnus Hagander mag...@hagander.net

Re: [HACKERS] WAL format changes

2012-06-20 Thread Magnus Hagander
On Tue, Jun 19, 2012 at 5:57 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 19, 2012 at 4:14 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Well, that was easier than I thought. Attached is a patch to make XLogRecPtr a uint64, on top of my other WAL format patches.

Re: [HACKERS] Release versioning inconsistency

2012-06-20 Thread Magnus Hagander
On Wed, Jun 20, 2012 at 11:23 AM, Marti Raudsepp ma...@juffo.org wrote: On Wed, Jun 20, 2012 at 12:18 PM, Magnus Hagander mag...@hagander.net wrote: (I do believe that using the v9.2.0beta marker is *better*, because then it sorts properly. But likely not enough much better to be inconsistent

Re: [HACKERS] Release versioning inconsistency

2012-06-20 Thread Dickson S. Guedes
2012/6/20 Magnus Hagander mag...@hagander.net: On Wed, Jun 20, 2012 at 11:23 AM, Marti Raudsepp ma...@juffo.org wrote: On Wed, Jun 20, 2012 at 12:18 PM, Magnus Hagander mag...@hagander.net wrote: (I do believe that using the v9.2.0beta marker is *better*, because then it sorts properly. But

Re: [HACKERS] [ADMIN] pg_basebackup blocking all queries with horrible performance

2012-06-20 Thread Simon Riggs
On 11 June 2012 23:47, Magnus Hagander mag...@hagander.net wrote You agreed to add something like NOSYNC option into START_REPLICATION command? I'm on the fence. I was hoping somebody else would chime in with an opinion as well. Why would you add it to synchronous_standby_names and then

Re: [HACKERS] Release versioning inconsistency

2012-06-20 Thread Peter Eisentraut
On ons, 2012-06-20 at 13:26 +0200, Magnus Hagander wrote: On Wed, Jun 20, 2012 at 11:23 AM, Marti Raudsepp ma...@juffo.org wrote: On Wed, Jun 20, 2012 at 12:18 PM, Magnus Hagander mag...@hagander.net wrote: (I do believe that using the v9.2.0beta marker is *better*, because then it sorts

[HACKERS] reviving AC_PROG_INSTALL

2012-06-20 Thread Peter Eisentraut
As I had written in [0], using /usr/bin/install instead of install-sh can significantly speed up the time to run make install, and hence make check. Using /usr/bin/install is standard in all autotools-using projects. PostgreSQL has removed the use of /usr/bin/install because of the incident

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Peter Geoghegan
On 19 June 2012 19:44, Peter Geoghegan pe...@2ndquadrant.com wrote: PostgreSQL supported Unicode before 2005, when the tie-breaker was introduced. I know at least one Swede who used Postgres95. I just took a look at the REL6_4 branch, and it looks much the same in 1999 as it did in 2005, in

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Kevin Grittner
Heikki Linnakangas wrote: I don't like the idea of adding the origin id to the record header. It's only required in some occasions, and on some record types. Right. And I'm worried it might not even be enough in more complicated scenarios. Perhaps we need a more generic WAL record

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 6:59 AM, Andres Freund and...@2ndquadrant.com wrote: My guess is that it wouldn't be too hard to remove some of the extra pointers.  Anyone who is using Dllist as a non-inline list could be converted to List * instead. There are only three users of the whole dllist.h.

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 5:15 AM, Andres Freund and...@2ndquadrant.com wrote: As I said before, I definitely agree that we want to have a separate transport format once we have decoding nailed down. We still need to ship wal around if the decoding happens in a different instance, but *after*

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-20 Thread Andres Freund
On Wednesday, June 20, 2012 02:51:30 PM Robert Haas wrote: On Wed, Jun 20, 2012 at 6:59 AM, Andres Freund and...@2ndquadrant.com wrote: Also, the performance-critical things could be reimplemented as macros. I question, though, whether we really need both single and doubly linked

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 5:47 AM, Simon Riggs si...@2ndquadrant.com wrote: The idea that logical rep is some kind of useful end goal in itself is slightly misleading. If the thought is to block multi-master completely on that basis, that would be a shame. Logical rep is the mechanism for

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Andres Freund
On Wednesday, June 20, 2012 03:19:55 PM Robert Haas wrote: On Wed, Jun 20, 2012 at 5:47 AM, Simon Riggs si...@2ndquadrant.com wrote: The idea that logical rep is some kind of useful end goal in itself is slightly misleading. If the thought is to block multi-master completely on that basis,

Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata

2012-06-20 Thread Amit Kapila
I'm almost inclined to suggest that we not get next-LSN from WAL, but by scanning all the pages in the main data store and computing the max observed LSN. This is clearly not very attractive from a performance standpoint, but it would avoid the obvious failure mode where you lost some

[HACKERS] [PATCH] Add some more documentation for array indexes/operators

2012-06-20 Thread Ryan Kelly
I had trouble finding what operators arrays supported or which ones had index support or even determining that arrays could be indexed from the documentation from the array data type. So, patch. -Ryan Kelly diff --git a/doc/src/sgml/array.sgml b/doc/src/sgml/array.sgml index 3508ba3..51d996d

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 9:12 AM, Andres Freund and...@2ndquadrant.com wrote: Uh. I don't want to just go around and replace anything randomly. Actually I don't want to change anything for now except whats necessary to get the patch in. The point I tried to make was just that the relatively

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 21:19, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 20, 2012 at 5:47 AM, Simon Riggs si...@2ndquadrant.com wrote: The idea that logical rep is some kind of useful end goal in itself is slightly misleading. If the thought is to block multi-master completely on that

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-20 Thread Andres Freund
On Wednesday, June 20, 2012 03:24:58 PM Robert Haas wrote: On Wed, Jun 20, 2012 at 9:12 AM, Andres Freund and...@2ndquadrant.com wrote: Uh. I don't want to just go around and replace anything randomly. Actually I don't want to change anything for now except whats necessary to get the patch

Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata

2012-06-20 Thread Aidan Van Dyk
On Wed, Jun 20, 2012 at 9:21 AM, Amit Kapila amit.kap...@huawei.com wrote: Example Scenario - Now assume we have Data files and WAL files intact and only control file is lost. Just so I understand correctly, the aim of this is to fix the situation where out of the thousands of files and

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 9:25 AM, Simon Riggs si...@2ndquadrant.com wrote: On 20 June 2012 21:19, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 20, 2012 at 5:47 AM, Simon Riggs si...@2ndquadrant.com wrote: The idea that logical rep is some kind of useful end goal in itself is slightly

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Andres Freund
On Wednesday, June 20, 2012 03:02:28 PM Robert Haas wrote: On Wed, Jun 20, 2012 at 5:15 AM, Andres Freund and...@2ndquadrant.com wrote: One bit is fine if you have only very simple replication topologies. Once you think about globally distributed databases its a bit different. You describe

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 20:37, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Heikki Linnakangas  wrote: I don't like the idea of adding the origin id to the record header. It's only required in some occasions, and on some record types. Right. Wrong, as explained. And I'm worried it might not

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Hannu Valtonen
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/19/2012 01:47 AM, Christopher Browne wrote: That numbering scheme gets pretty anti-intuitive fairly quickly, from whence we took the approach of having a couple digits indicating data centre followed by a digit indicating which node in that

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 9:43 AM, Andres Freund and...@2ndquadrant.com wrote: If you do that, then, yes, everything that you need to disentangle various network topologies must be present in WAL.  But what I'm saying is: don't do it like that.  Generate the LCRs just ONCE, at the origin node,

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Andres Freund
On Wednesday, June 20, 2012 03:42:39 PM Robert Haas wrote: On Wed, Jun 20, 2012 at 9:25 AM, Simon Riggs si...@2ndquadrant.com wrote: On 20 June 2012 21:19, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 20, 2012 at 5:47 AM, Simon Riggs si...@2ndquadrant.com wrote: The idea that

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Andres Freund
On Wednesday, June 20, 2012 03:54:43 PM Robert Haas wrote: On Wed, Jun 20, 2012 at 9:43 AM, Andres Freund and...@2ndquadrant.com wrote: If you do that, then, yes, everything that you need to disentangle various network topologies must be present in WAL. But what I'm saying is: don't do it

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 21:42, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 20, 2012 at 9:25 AM, Simon Riggs si...@2ndquadrant.com wrote: On 20 June 2012 21:19, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 20, 2012 at 5:47 AM, Simon Riggs si...@2ndquadrant.com wrote: The idea that

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Greg Stark
On Sun, Jun 17, 2012 at 9:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: The trick for hashing such datatypes is to be able to guarantee that equal values hash to the same hash code, which is typically possible as long as you know the equality rules well enough.  We could possibly do that for text

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Peter Geoghegan
On 20 June 2012 15:10, Greg Stark st...@mit.edu wrote: On Sun, Jun 17, 2012 at 9:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: The trick for hashing such datatypes is to be able to guarantee that equal values hash to the same hash code, which is typically possible as long as you know the equality

Re: [HACKERS] foreign key locks

2012-06-20 Thread Jaime Casanova
On Thu, Jun 14, 2012 at 11:41 AM, Alvaro Herrera alvhe...@alvh.no-ip.org wrote: Hi, This is v12 of the foreign key locks patch. Hi Álvaro, Just noticed that this patch needs a rebase because of the refactoring Tom did in ri_triggers.c -- Jaime Casanova         www.2ndQuadrant.com

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 16:23, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 20.06.2012 11:17, Simon Riggs wrote: On 20 June 2012 15:45, Heikki Linnakangas heikki.linnakan...@enterprisedb.com  wrote: On 20.06.2012 10:32, Simon Riggs wrote: On 20 June 2012 14:40, Heikki

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Greg Stark
On Wed, Jun 20, 2012 at 3:19 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: It occurs to me that strxfrm would answer this question. If we made the hash function hash the result of strxfrm then we could make equality use strcoll and not fall back to strcmp. What about per-column collations?

Re: [HACKERS] [ADMIN] PANIC while doing failover (streaming replication)

2012-06-20 Thread Matheus Ricardo Espanhol
Hi, Any news on this issue? My slave server is not trusted, all the warnings below are related to indexes of the main tables: 2012-06-14 11:46:23 BRT [18654]: [34603-1] user=,db= LOG: recovery restart point at 435/4E899CE8 2012-06-14 11:46:23 BRT [18654]: [34604-1] user=,db= DETAIL: last

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: I think that this change may have made the difference between the Hungarians getting away with it and not getting away with it. Might it have been that for text, they were using some operator that wasn't '=' (perhaps one which has no fastpath, and

[HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread David Fetter
Folks, A co-worker filed a bug against file_fdw where the columns in a FOREIGN TABLE were scrambled on SELECT. It turned out that this comes from the (yes, it's documented, but since it's documented in a place not obviously linked to the bug, it's pretty useless) feature of COPY CSV HEADER

Re: [HACKERS] libpq compression

2012-06-20 Thread Alvaro Herrera
Excerpts from Florian Pflug's message of mié jun 20 06:35:29 -0400 2012: On Jun19, 2012, at 17:36 , Robert Haas wrote: On Mon, Jun 18, 2012 at 1:42 PM, Martijn van Oosterhout klep...@svana.org wrote: On Sun, Jun 17, 2012 at 12:29:53PM -0400, Tom Lane wrote: The fly in the ointment with

Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata

2012-06-20 Thread Tom Lane
Amit Kapila amit.kap...@huawei.com writes: I'm almost inclined to suggest that we not get next-LSN from WAL, but by scanning all the pages in the main data store and computing the max observed LSN. This is clearly not very attractive from a performance standpoint, but it would avoid the

Re: [HACKERS] Release versioning inconsistency

2012-06-20 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On ons, 2012-06-20 at 13:26 +0200, Magnus Hagander wrote: That might actually be a good idea. We can't really change the way we named the betas, but it's not too late to consider naming the actual release as 9.2.0... The final release was always going

Re: [HACKERS] Pg default's verbosity?

2012-06-20 Thread Peter Eisentraut
On tis, 2012-06-19 at 02:15 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: There might be something to the idea of demoting a few of the things we've traditionally had as NOTICEs, though. IME, the following two messages account for a huge percentage of the chatter:

Re: [HACKERS] libpq compression

2012-06-20 Thread Tom Lane
Florian Pflug f...@phlo.org writes: I wonder though if shouldn't restrict the allowed ciphers list to being a simple list of supported ciphers. If our goal is to support multiple SSL libraries transparently then surely having openssl-specific syntax in the config file isn't exactly great

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: Kevin Grittner kevin.gritt...@wicourts.gov wrote: Heikki Linnakangas wrote: I don't like the idea of adding the origin id to the record header. It's only required in some occasions, and on some record types. Right. Wrong, as explained. The

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 10:02 AM, Andres Freund and...@2ndquadrant.com wrote: Were not the only ones here that are performing scope creep though... I think about all people who have posted in the whole thread except maybe Tom and Marko are guilty of doing so. I still think its rather sensible

Re: [HACKERS] libpq compression

2012-06-20 Thread Florian Pflug
On Jun20, 2012, at 17:34 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: I wonder though if shouldn't restrict the allowed ciphers list to being a simple list of supported ciphers. If our goal is to support multiple SSL libraries transparently then surely having openssl-specific syntax

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread Tom Lane
David Fetter da...@fetter.org writes: Rather than being totally ignored in the COPY OUT (CSV HEADER) case, the header line in should be parsed to establish which columns are where and rearranging the output if needed. This is not fix a POLA violation. This is a non-backwards-compatible new

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread Andrew Dunstan
On 06/20/2012 11:02 AM, David Fetter wrote: Folks, A co-worker filed a bug against file_fdw where the columns in a FOREIGN TABLE were scrambled on SELECT. It turned out that this comes from the (yes, it's documented, but since it's documented in a place not obviously linked to the bug, it's

Re: [HACKERS] libpq compression

2012-06-20 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: I looked at the code (apps/ciphers.c) and it looks pretty easy to obtain the list of ciphers starting from the stringified configuration parameter and iterate on them. Do you mean that it will produce an expansion of the set of ciphers meeting

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Andres Freund
On Wednesday, June 20, 2012 05:34:42 PM Kevin Grittner wrote: Simon Riggs si...@2ndquadrant.com wrote: This is not transaction metadata, it is WAL record metadata required for multi-master replication, see later point. We need to add information to every WAL record that is used as the

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 10:08 AM, Simon Riggs si...@2ndquadrant.com wrote: But I think getting even single-master logical replication working well in a single release cycle is going to be a job and a half. OK, so your estimate is 1.5 people to do that. And if we have more people, should they

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Peter Geoghegan
On 20 June 2012 15:55, Tom Lane t...@sss.pgh.pa.us wrote: Peter Geoghegan pe...@2ndquadrant.com writes: I think that this change may have made the difference between the Hungarians getting away with it and not getting away with it. Might it have been that for text, they were using some

Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata

2012-06-20 Thread Fujii Masao
On Wed, Jun 20, 2012 at 12:40 PM, Amit Kapila amit.kap...@huawei.com wrote: I believe if WAL files are proper as mentioned in Alvaro's mail, the purposed logic should generate correct values. Do you see any problem in logic purposed in my original mail. Can I resume my work on this feature?

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread David Fetter
On Wed, Jun 20, 2012 at 11:47:14AM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: Rather than being totally ignored in the COPY OUT (CSV HEADER) case, the header line in should be parsed to establish which columns are where and rearranging the output if needed. This is not

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread Tom Lane
David Fetter da...@fetter.org writes: OK, new proposal: COPY FROM (Thanks, Andrew! Must not post while asleep...) should have an option which requires that HEADER be enabled and mandates that the column names in the header match the columns coming in. Has someone got a better name for this

[HACKERS] proposal and patch : support INSERT INTO...RETURNING with partitioned table using rule

2012-06-20 Thread John Lumby
--- Problem I'm trying to solve:     For partitioned tables,  make it possible to use RETURNING clause on INSERT INTO    together with DO INSTEAD rule [  Note  -  wherever I say INSERT I also mean UPDATE and DELETE ] ---

Re: [HACKERS] libpq compression

2012-06-20 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mié jun 20 11:49:51 -0400 2012: Alvaro Herrera alvhe...@commandprompt.com writes: I looked at the code (apps/ciphers.c) and it looks pretty easy to obtain the list of ciphers starting from the stringified configuration parameter and iterate on them.

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: No, I'm suggesting it would probably be at least a bit of a win here to cache the constant, and only have to do a strxfrm() + strcmp() per comparison. Um, have you got any hard evidence to support that notion? The traditional advice is that

Re: [HACKERS] libpq compression

2012-06-20 Thread Magnus Hagander
On Wed, Jun 20, 2012 at 12:35 PM, Florian Pflug f...@phlo.org wrote: On Jun19, 2012, at 17:36 , Robert Haas wrote: On Mon, Jun 18, 2012 at 1:42 PM, Martijn van Oosterhout klep...@svana.org wrote: On Sun, Jun 17, 2012 at 12:29:53PM -0400, Tom Lane wrote: The fly in the ointment with any of

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread David Fetter
On Wed, Jun 20, 2012 at 12:18:45PM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: OK, new proposal: COPY FROM (Thanks, Andrew! Must not post while asleep...) should have an option which requires that HEADER be enabled and mandates that the column names in the header match

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread Alvaro Herrera
Excerpts from David Fetter's message of mié jun 20 12:43:31 -0400 2012: On Wed, Jun 20, 2012 at 12:18:45PM -0400, Tom Lane wrote: In your original proposal, I was rather wondering what should happen if the incoming file didn't have the same set of columns called out in the COPY command's

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Andres Freund
Hi, On Wednesday, June 20, 2012 05:44:09 PM Robert Haas wrote: On Wed, Jun 20, 2012 at 10:02 AM, Andres Freund and...@2ndquadrant.com wrote: Were not the only ones here that are performing scope creep though... I think about all people who have posted in the whole thread except maybe Tom

Re: [HACKERS] foreign key locks

2012-06-20 Thread Tom Lane
Jaime Casanova ja...@2ndquadrant.com writes: On Thu, Jun 14, 2012 at 11:41 AM, Alvaro Herrera alvhe...@alvh.no-ip.org wrote: This is v12 of the foreign key locks patch. Just noticed that this patch needs a rebase because of the refactoring Tom did in ri_triggers.c Hold on a bit before you

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread Andrew Dunstan
On 06/20/2012 12:18 PM, Tom Lane wrote: David Fetterda...@fetter.org writes: OK, new proposal: COPY FROM (Thanks, Andrew! Must not post while asleep...) should have an option which requires that HEADER be enabled and mandates that the column names in the header match the columns coming in.

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread Andrew Dunstan
On 06/20/2012 12:50 PM, Alvaro Herrera wrote: Another related case: you have a file with headers and columns (n, t, x, y, z) but your table only has n and t. How would you tell COPY to discard the junk columns? Currently it just complains that they are there. That's one of the main use

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Simon Riggs
On 20 June 2012 23:34, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Simon Riggs si...@2ndquadrant.com wrote: Kevin Grittner kevin.gritt...@wicourts.gov wrote: Heikki Linnakangas  wrote: I don't like the idea of adding the origin id to the record header. It's only required in some

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread Josh Berkus
In the past people have asked me to have copy use the CSV header line in place of supplying a column list in the COPY command. I can certainly see some utility in that, and I think it might achieve what David wants. Using that scenario it would be an error to supply an explicit column list

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Christopher Browne
On Wed, Jun 20, 2012 at 11:50 AM, Andres Freund and...@2ndquadrant.com wrote: On Wednesday, June 20, 2012 05:34:42 PM Kevin Grittner wrote: Simon Riggs si...@2ndquadrant.com wrote: This is not transaction metadata, it is WAL record metadata required for multi-master replication, see later

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 12:53 PM, Andres Freund and...@2ndquadrant.com wrote: I would prefer the event trigger way because that seems to be the most extensible/reusable. It would allow a fully replicated databases and catalog only instances. I think we need to design event triggers in a way

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 12:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: The fact is that this is likely to be a fairly significant performance win, because strxfrm() is quite simply the way you're supposed to do collation-aware sorting, and is documented as such. For that reason, C standard

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread Marc Mamin
-Ursprüngliche Nachricht- Von: pgsql-hackers-ow...@postgresql.org im Auftrag von Josh Berkus Gesendet: Mi 6/20/2012 7:06 An: pgsql-hackers@postgresql.org Betreff: Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER ... (1) is valuable for backwards

Re: [HACKERS] sortsupport for text

2012-06-20 Thread Peter Geoghegan
On 20 June 2012 17:41, Tom Lane t...@sss.pgh.pa.us wrote: Peter Geoghegan pe...@2ndquadrant.com writes: No, I'm suggesting it would probably be at least a bit of a win here to cache the constant, and only have to do a strxfrm() + strcmp() per comparison. Um, have you got any hard evidence to

Re: [HACKERS] Nasty, propagating POLA violation in COPY CSV HEADER

2012-06-20 Thread Josh Berkus
4) MAP_HEADER ('column 1'- 'col_1', 'Date' - 'input_date' ...) to cover the case when column names do not match. Personally, I think that's going way beyond what we want to do with COPY. At that point, check out the CSV-array FDW. Of course, if someone writes a WIP patch which implements

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-20 Thread Andres Freund
On Wednesday, June 20, 2012 07:17:57 PM Robert Haas wrote: On Wed, Jun 20, 2012 at 12:53 PM, Andres Freund and...@2ndquadrant.com wrote: I would prefer the event trigger way because that seems to be the most extensible/reusable. It would allow a fully replicated databases and catalog only

Re: [HACKERS] pgbench--new transaction type

2012-06-20 Thread Robert Haas
On Wed, Jun 20, 2012 at 3:48 AM, Simon Riggs si...@2ndquadrant.com wrote: I'm sure Jeff submitted this because of the need for a standard test, rather than the wish to actually modify pgbench itself. Can I suggest that we include a list of standard scripts with pgbench for this purpose? These

Re: [HACKERS] WAL format changes

2012-06-20 Thread Fujii Masao
On Wed, Jun 20, 2012 at 8:19 PM, Magnus Hagander mag...@hagander.net wrote: On Tue, Jun 19, 2012 at 5:57 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 19, 2012 at 4:14 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Well, that was easier than I thought. Attached is

  1   2   >