Re: [HACKERS] xml_is_document and selective pg_re_throw

2012-06-13 Thread Nikhil Sontakke
No, I don't see any particular risk there. The places that might throw ERRCODE_INVALID_XML_DOCUMENT are sufficiently few (as in, exactly one, in this usage) that we can have reasonable confidence we know what the system state is when we catch that error. Hmmm, I was writing some code in

Re: [HACKERS] hint bit i/o reduction

2012-06-13 Thread Amit Kapila
1) Keep track # times the last transaction id was repeatedly seen in tqual.c (resetting as soon as a new xid was touched. We can do this just for xmin, or separately for both xmin and xmax. Will this be done when we see a new xid duricg scan of tuple, if yes then Won't this logic impact

Re: [HACKERS] Minimising windows installer password confusion

2012-06-13 Thread Dave Page
On Wed, Jun 13, 2012 at 2:12 AM, Craig Ringer cr...@postnewspapers.com.au wrote: Users don't remember passwords, though. It's one of those constants, and is why practically every web site etc out there offers password recovery. The installer IMO needs to store the postgres account password in

Re: [HACKERS] Minimising windows installer password confusion

2012-06-13 Thread Dave Page
On Wed, Jun 13, 2012 at 2:18 AM, Craig Ringer cr...@postnewspapers.com.au wrote: On 06/12/2012 08:08 PM, Dave Page wrote: Some background: By default the installer will use 'postgres' for both the service (OS) account, and the database superuser account. It will use the same password for both

[HACKERS] errdetail_busy_db plural forms

2012-06-13 Thread Peter Eisentraut
I was looking for missing use of gettext plural forms, which led me to errdetail_busy_db(). While we can't do much about this: errdetail(There are %d other session(s) and %d prepared transaction(s) using the database., notherbackends, npreparedxacts); I think it's still worth

Re: [HACKERS] Minimising windows installer password confusion

2012-06-13 Thread Dave Page
On Wed, Jun 13, 2012 at 3:07 AM, Craig Ringer cr...@postnewspapers.com.au wrote: On 06/13/2012 01:19 AM, Sachin Srivastava wrote: On Tue, Jun 12, 2012 at 7:43 PM, Dave Page dp...@pgadmin.org mailto:dp...@pgadmin.org wrote:    On Tue, Jun 12, 2012 at 2:57 PM, Robert Haas

Re: [HACKERS] Minimising windows installer password confusion

2012-06-13 Thread Florian Pflug
On Jun13, 2012, at 11:10 , Dave Page wrote: On Wed, Jun 13, 2012 at 2:12 AM, Craig Ringer cr...@postnewspapers.com.au wrote: Users don't remember passwords, though. It's one of those constants, and is why practically every web site etc out there offers password recovery. The installer IMO

Re: [HACKERS] initdb and fsync

2012-06-13 Thread Peter Eisentraut
On tis, 2012-06-12 at 21:09 -0700, Jeff Davis wrote: On Sun, 2012-03-25 at 19:59 -0700, Jeff Davis wrote: On Sat, 2012-03-17 at 17:48 +0100, Cédric Villemain wrote: I agree with Andres. I believe we should use sync_file_range (_before?) with linux. And we can use

[HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

2012-06-13 Thread Andres Freund
Hi everyone, This mail contains the highlevel design description of how our prototype of in-core logical replication works. The individual patches will be posted as replies to this email. I obviously welcome all sorts of productive comments to both the individual patches and the architecture.

[HACKERS] [PATCH 03/16] Add a new syscache to fetch a pg_class entry via its relfilenode

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de This patch is problematic because formally indexes used by syscaches needs to be unique, this one is not though because of 0/InvalidOids entries for nailed/shared catalog entries. Those values aren't allowed to be queried though. It might be nicer to add

[HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de Adds a single and a double linked list which can easily embedded into other datastructures and can be used without any additional allocations. Problematic: It requires USE_INLINE to be used. It could be remade to fallback to to externally defined functions

[HACKERS] [PATCH 02/16] Add zeroRecPtr as a shortcut for initializing a local variable to {0, 0}

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de This is locally defined in lots of places and would get introduced frequently in the next commits. It is expected that this can be defined in a header-only manner as soon as the XLogInsert scalability groundwork from Heikki gets in. ---

[HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de The individual changes need to be identified by an xid. The xid can be a subtransaction or a toplevel one, at commit those can be reintegrated by doing a k-way mergesort between the individual transaction. Callbacks for apply_begin, apply_change and

[HACKERS] [PATCH 01/16] Overhaul walsender wakeup handling

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de The previous coding could miss xlog writeouts at several places. E.g. when wal was written out by the background writer or even after a commit if synchronous_commit=off. This could lead to delays in sending data to the standby of up to 7 seconds. To fix

[HACKERS] [PATCH 07/16] Log enough data into the wal to reconstruct logical changes from it if wal_level=logical

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de This adds a new wal_level value 'logical' Missing cases: - heap_multi_insert - primary key changes for updates - no primary key - LOG_NEWPAGE --- src/backend/access/heap/heapam.c| 135 ---

[HACKERS] [PATCH 12/16] Add state to keep track of logical replication

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de In order to have restartable replication with minimal additional writes its very useful to know up to which point we have replayed/received changes from a foreign node. One representation of that is the lsn of changes at the originating cluster. We need to

[HACKERS] [PATCH 09/16] Decode wal (with wal_level=logical) into changes in an ApplyCache instance

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de This requires an up2date catalog and can thus only be run on a replica. Missing: - HEAP_NEWPAGE support - HEAP2_MULTI_INSERT support - DDL integration. *No* ddl, including TRUNCATE is possible atm --- src/backend/replication/logical/Makefile |2 +-

[HACKERS] [PATCH 11/16] Add infrastructure for manipulating multiple streams of wal on a segment handling level

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de For that add a 'node_id' parameter to most commands dealing with wal segments. A node_id thats 'InvalidMultimasterNodeId' references local wal, every other node_id referes to wal in a new pg_lcr directory. Using duplicated code would reduce the impact of

[HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de One solution to avoid loops when doing wal based logical replication in topologies which are more complex than one unidirectional transport is introducing the concept of a 'origin_id' into the wal stream. Luckily there is some padding in the XLogRecord

[HACKERS] [PATCH 06/16] Add support for a generic wal reading facility dubbed XLogReader

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to do so is not tightly integrated into xlog.c is rather hard and would require changes to rather

[HACKERS] [PATCH 16/16] current version of the design document

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de --- src/backend/replication/logical/DESIGN | 209 1 file changed, 209 insertions(+) create mode 100644 src/backend/replication/logical/DESIGN diff --git a/src/backend/replication/logical/DESIGN

[HACKERS] [PATCH 15/16] Activate/Implement the apply process which applies received updates from another node

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de One apply process currently can only apply changes from one database in another cluster (with a specific node_id). Currently synchronous_commit=off is statically set in the apply process because after a crash we can safely recover all changes which we

[HACKERS] [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de We decided to use low level functions to do the apply instead of producing sql statements containing the data (or using prepared statements) because both, the text conversion and the full executor overhead aren't introduce a significant overhead which is

[HACKERS] [PATCH 13/16] Introduction of pair of logical walreceiver/sender

2012-06-13 Thread Andres Freund
From: Andres Freund and...@anarazel.de A logical WALReceiver is started directly by Postmaster when we enter PM_RUN state and the new parameter multimaster_conninfo is set. For now only one of those is started, but the code doesn't rely on that. In future multiple ones should be allowed. To

[HACKERS] [PATCH 05/16] Preliminary: Introduce Bgworker process

2012-06-13 Thread Andres Freund
From: Simon Riggs si...@2ndquadrant.com Early prototype that allows for just 1 bgworker which calls a function called do_applyprocess(). Expect major changes in this, but not in ways that would effect the apply process. --- src/backend/postmaster/Makefile |4 +-

Re: [HACKERS] Ability to listen on two unix sockets

2012-06-13 Thread Honza Horak
On 06/10/2012 12:37 AM, Peter Eisentraut wrote: On sön, 2012-06-10 at 00:25 +0200, Andres Freund wrote: We already have the ability to configure the Unix socket directory in postgresql.conf. All you need to do is turn that into a list. That doesn't help libpq using clients. There is no

Re: [HACKERS] hint bit i/o reduction

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 3:57 AM, Amit Kapila amit.kap...@huawei.com wrote: 1) Keep track # times the last transaction id was repeatedly seen in tqual.c (resetting as soon as a new xid was touched.  We can do this just for xmin, or separately for both xmin and xmax. Will this be done when we

Re: [HACKERS] Ability to listen on two unix sockets

2012-06-13 Thread Honza Horak
On 06/10/2012 03:41 PM, Robert Haas wrote: On Sun, Jun 10, 2012 at 8:36 AM, Tom Lanet...@sss.pgh.pa.us wrote: Peter Eisentrautpete...@gmx.net writes: On lör, 2012-06-09 at 18:26 -0400, Tom Lane wrote: That's not actually quite the same thing as what I suggest above. Currently,

Re: [HACKERS] hint bit i/o reduction

2012-06-13 Thread Amit Kapila
Yes, but only in the unhinted case -- in oltp workloads tuples get hinted fairly quickly so I doubt this would be a huge impact. Hinted scans will work exactly as they do now. In the unhinted case for OLTP a few tests are added but that's noise compared to the other stuff going on. I

Re: [HACKERS] WIP patch for Todo Item : Provide fallback_application_name in contrib/pgbench, oid2name, and dblink

2012-06-13 Thread Amit Kapila
I have created the patch by including fallback_application_name for dblink as well. In this I have used the name of fallback_application_name as dblink. Please let me know your suggestions regarding the same. -Original Message- From: Robert Haas [mailto:robertmh...@gmail.com] Sent:

Re: [HACKERS] [PATCH 16/16] current version of the design document

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 6:28 AM, Andres Freund and...@2ndquadrant.com wrote: +synchronized catalog at the decoding site. That adds some complexity to use +cases like replicating into a different database or cross-version +replication. For those it is relatively straight-forward to develop a

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jun 12, 2012 at 9:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: The code for this is as attached.  Note that I'd rip out the normal-path tracking of line boundaries; it seems better to have a second scan of the data in the error case and save the

Re: [HACKERS] [PATCH 16/16] current version of the design document

2012-06-13 Thread Andres Freund
Hi Merlin, On Wednesday, June 13, 2012 04:21:12 PM Merlin Moncure wrote: On Wed, Jun 13, 2012 at 6:28 AM, Andres Freund and...@2ndquadrant.com wrote: +synchronized catalog at the decoding site. That adds some complexity to use +cases like replicating into a different database or

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Robert Haas
On Wed, Jun 13, 2012 at 10:35 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jun 12, 2012 at 9:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: The code for this is as attached.  Note that I'd rip out the normal-path tracking of line boundaries; it seems

Re: [HACKERS] hint bit i/o reduction

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 9:02 AM, Amit Kapila amit.kap...@huawei.com wrote: Yes, but only in the unhinted case -- in oltp workloads tuples get hinted fairly quickly so I doubt this would be a huge impact.  Hinted scans will work exactly as they do now.  In the unhinted case for OLTP a  few

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Andres Freund
On Wednesday, June 13, 2012 05:03:38 PM Robert Haas wrote: On Wed, Jun 13, 2012 at 10:35 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jun 12, 2012 at 9:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: The code for this is as attached. Note that I'd rip

Re: [HACKERS] Ability to listen on two unix sockets

2012-06-13 Thread Florian Pflug
On Jun13, 2012, at 15:14 , Honza Horak wrote: Since systemd with it's PrivateTmp feature is going to be used in more and more distros, there will probably be a bigger need to solve in-accessible default unix socket directory /tmp in the future. Thus, I'd like to ask if anybody is aware of

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Robert Haas
On Wed, Jun 13, 2012 at 11:06 AM, Andres Freund and...@2ndquadrant.com wrote: On Wednesday, June 13, 2012 05:03:38 PM Robert Haas wrote: On Wed, Jun 13, 2012 at 10:35 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jun 12, 2012 at 9:52 PM, Tom Lane

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Andres Freund
On Wednesday, June 13, 2012 05:18:22 PM Robert Haas wrote: On Wed, Jun 13, 2012 at 11:06 AM, Andres Freund and...@2ndquadrant.com wrote: On Wednesday, June 13, 2012 05:03:38 PM Robert Haas wrote: On Wed, Jun 13, 2012 at 10:35 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas

[HACKERS] 9.3devel branch

2012-06-13 Thread Robert Haas
Seeing as we're about to start a CommitFest, I think it's about time to create the 9.3devel branch. I'm happy to do it, but I believe it's usually Tom's gig. Tom? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [PATCH 07/16] Log enough data into the wal to reconstruct logical changes from it if wal_level=logical

2012-06-13 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: This adds a new wal_level value 'logical' Missing cases: - heap_multi_insert - primary key changes for updates - no primary key - LOG_NEWPAGE First, Wow! I look forward to the point where we can replace our trigger-based replication with

Re: [HACKERS] [PATCH 16/16] current version of the design document

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 9:40 AM, Andres Freund and...@2ndquadrant.com wrote: Hi Merlin, On Wednesday, June 13, 2012 04:21:12 PM Merlin Moncure wrote: On Wed, Jun 13, 2012 at 6:28 AM, Andres Freund and...@2ndquadrant.com wrote: +synchronized catalog at the decoding site. That adds some

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

2012-06-13 Thread Robert Haas
On Wed, Jun 13, 2012 at 7:27 AM, Andres Freund and...@2ndquadrant.com wrote: Unless somebody objects I will add most of the individual marked as RFC to the current commitfest. I hope that with comments stemming from that round we can get several of the patches into the first or second

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On Wednesday, June 13, 2012 05:18:22 PM Robert Haas wrote: According to my testing, the main cost of an exception block catching a division-by-zero error is that of generating the error message, primarily sprintf()-type stuff. The cost of scanning

Re: [HACKERS] 9.3devel branch

2012-06-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Seeing as we're about to start a CommitFest, I think it's about time to create the 9.3devel branch. I'm happy to do it, but I believe it's usually Tom's gig. There are a couple open issues (like reversion of that checkpoint-skipping patch) that

Re: [HACKERS] [PATCH 07/16] Log enough data into the wal to reconstruct logical changes from it if wal_level=logical

2012-06-13 Thread Andres Freund
On Wednesday, June 13, 2012 05:27:06 PM Kevin Grittner wrote: Andres Freund and...@2ndquadrant.com wrote: This adds a new wal_level value 'logical' Missing cases: - heap_multi_insert - primary key changes for updates - no primary key - LOG_NEWPAGE First, Wow! Thanks ;) I hope

Re: [HACKERS] 9.3devel branch

2012-06-13 Thread Robert Haas
On Wed, Jun 13, 2012 at 11:57 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Seeing as we're about to start a CommitFest, I think it's about time to create the 9.3devel branch. I'm happy to do it, but I believe it's usually Tom's gig. There are a couple

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Robert Haas
On Wed, Jun 13, 2012 at 11:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On Wednesday, June 13, 2012 05:18:22 PM Robert Haas wrote: According to my testing, the main cost of an exception block catching a division-by-zero error is that of generating the

Re: [HACKERS] [PATCH 16/16] current version of the design document

2012-06-13 Thread Andres Freund
Hi, On Wednesday, June 13, 2012 05:39:36 PM Merlin Moncure wrote: On Wed, Jun 13, 2012 at 9:40 AM, Andres Freund and...@2ndquadrant.com wrote: Let's take the case where I have N small-ish schema identical database shards that I want to aggregate into a single warehouse -- something that

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

2012-06-13 Thread Andres Freund
On Wednesday, June 13, 2012 05:53:31 PM Robert Haas wrote: On Wed, Jun 13, 2012 at 7:27 AM, Andres Freund and...@2ndquadrant.com wrote: Unless somebody objects I will add most of the individual marked as RFC to the current commitfest. I hope that with comments stemming from that round we

Re: [HACKERS] Re: [GENERAL] pg_upgrade from 9.0.7 to 9.1.3: duplicate key pg_authid_oid_index

2012-06-13 Thread Bruce Momjian
On Mon, Jun 04, 2012 at 10:16:45AM -0400, Bruce Momjian wrote: I think the checks that are actually needed here are (1) bootstrap superusers are named the same, and (2) there are no roles other than the bootstrap superuser in the new cluster. You are right that it is more complex than I

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jun 13, 2012 at 11:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: In any case, the proposed scheme for providing context requires that you know where the error is before you can identify the context. I considered schemes that would keep track of

[HACKERS] uncataloged tables are a vestigial husk

2012-06-13 Thread Robert Haas
While working on some code today, I noticed that RELKIND_UNCATALOGED appears to serve no useful purpose. In the few places where we check for it at all, we treat it in exactly the same way as RELKIND_RELATION. It seems that it's only purpose is to serve as a placeholder inside each newly-created

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Robert Haas
On Wed, Jun 13, 2012 at 12:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jun 13, 2012 at 11:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: In any case, the proposed scheme for providing context requires that you know where the error is before you can

Re: [HACKERS] initdb and fsync

2012-06-13 Thread Jeff Davis
included in your patch, which would be a suitable place to discuss this briefly. Thank you, I added that. Regards, Jeff Davis initdb-fsync-20120613.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I like some of these changes - in particular, the use of errcontext(), but some of them still seem off. ! DETAIL: Token ' is invalid. ! CONTEXT: JSON data, line 1: '... This doesn't make sense to me. Well, the input is two single quotes and the

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

2012-06-13 Thread Steve Singer
On 12-06-13 07:27 AM, Andres Freund wrote: Its also available in the 'cabal-rebasing' branch on git.postgresql.org/users/andresfreund/postgres.git . That branch will modify history though. That branch has a merge error in f685a11ce43b9694cbe61ffa42e396c9fbc32b05 gcc -O2 -Wall

Re: [HACKERS] uncataloged tables are a vestigial husk

2012-06-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: While working on some code today, I noticed that RELKIND_UNCATALOGED appears to serve no useful purpose. In the few places where we check for it at all, we treat it in exactly the same way as RELKIND_RELATION. It seems that it's only purpose is to

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

2012-06-13 Thread Andres Freund
On Wednesday, June 13, 2012 07:11:40 PM Steve Singer wrote: On 12-06-13 07:27 AM, Andres Freund wrote: Its also available in the 'cabal-rebasing' branch on git.postgresql.org/users/andresfreund/postgres.git . That branch will modify history though. That branch has a merge error in

Re: [HACKERS] uncataloged tables are a vestigial husk

2012-06-13 Thread Robert Haas
On Wed, Jun 13, 2012 at 1:13 PM, Tom Lane t...@sss.pgh.pa.us wrote: The attached patch cleans it up by removing RELKIND_UNCATALOGED and teaching RelationBuildLocalRelation() to set the relkind itself. I think there are probably some places to fix in the docs too. catalogs.sgml doesn't include

[HACKERS] Is cachedFetchXidStatus provably valid?

2012-06-13 Thread Merlin Moncure
It's probably an academic concern, but what happens if a backend saves off cachedFetchXidStatus and then sleeps for a very long time. During that time an xid wraparound happens and the backend wakes up and happens to read another unhinted tuple with the same xid and a different commit status.

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

2012-06-13 Thread Andres Freund
On Wednesday, June 13, 2012 07:11:40 PM Steve Singer wrote: On 12-06-13 07:27 AM, Andres Freund wrote: Its also available in the 'cabal-rebasing' branch on git.postgresql.org/users/andresfreund/postgres.git . That branch will modify history though. That branch has a merge error in

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-13 Thread Andres Freund
The previous mail contained a patch with a mismerge caused by reording commits. Corrected version attached. Thanks to Steve Singer for noticing this quickly. Andres -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services From

Re: [HACKERS] uncataloged tables are a vestigial husk

2012-06-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jun 13, 2012 at 1:13 PM, Tom Lane t...@sss.pgh.pa.us wrote: The attached patch cleans it up by removing RELKIND_UNCATALOGED and teaching RelationBuildLocalRelation() to set the relkind itself. I think there are probably some places to fix in

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

2012-06-13 Thread Andres Freund
Hi, The patch as of yet doesn't contain how you actually can use the prototype... Obviously at this point its not very convenient: I have two config files: Node 1: port = 5501 wal_level = logical max_wal_senders = 10 wal_keep_segments = 200 multimaster_conninfo = 'port=5502 host=/tmp'

Re: [HACKERS] [RFC][PATCH] BDR Prototype startup instructions (not prematurely sent this time)

2012-06-13 Thread Andres Freund
Hi, The patch as of yet doesn't contain how you actually can use the prototype... Obviously at this point its not very convenient: I have two config files: Node 1: port = 5501 wal_level = logical max_wal_senders = 10 wal_keep_segments = 200 multimaster_conninfo = 'port=5502 host=/tmp'

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Robert Haas
On Wed, Jun 13, 2012 at 1:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I like some of these changes - in particular, the use of errcontext(), but some of them still seem off. ! DETAIL:  Token ' is invalid. ! CONTEXT:  JSON data, line 1: '... This

[HACKERS] Re: [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

2012-06-13 Thread Noah Misch
On Mon, Jun 11, 2012 at 09:18:39PM -0400, Robert Haas wrote: I guess the remaining question is whether to do it only for LOCAL TEMP tables or also for GLOBAL TEMP ones. A survey of what other products do might be of some value. Thanks for investigating. Sybase ASE, which I include only

Re: [HACKERS] [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

2012-06-13 Thread Christopher Browne
On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund and...@2ndquadrant.com wrote: From: Andres Freund and...@anarazel.de We decided to use low level functions to do the apply instead of producing sql statements containing the data (or using prepared statements) because both, the text conversion

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jun 13, 2012 at 1:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: ! DETAIL: Character with value 0x0a must be escaped. ! CONTEXT: JSON data, line 1: abc ! ... This seems an odd way to present this,

Re: [HACKERS] [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

2012-06-13 Thread Tom Lane
Noah Misch n...@leadboat.com writes: Given that, how about warning on GLOBAL only but having the documentation equally discourage use of both? Yeah, that's about what I was thinking, too. Any thoughts on the wording of the WARNING message? regards, tom lane -- Sent

Re: [HACKERS] [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

2012-06-13 Thread Andres Freund
On Wednesday, June 13, 2012 08:50:42 PM Christopher Browne wrote: On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund and...@2ndquadrant.com wrote: From: Andres Freund and...@anarazel.de We decided to use low level functions to do the apply instead of producing sql statements containing the

Re: [HACKERS] Backup docs

2012-06-13 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes: I would like to see that page changed to list pg_basebackup as the default way of doing base backups, and then list the manual way as an option if you need more flexibility. +1 I'd also like to add pg_basebackup -x under standalone hot backups,

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-06-13 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: I thought I already pointed that out, but: we have *extensions*. What we don't have is a convenient method of dealing with functions that need to be migrated across extensions, or from an extension to core, between one major release and the next. It would

[HACKERS] Re: [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

2012-06-13 Thread Noah Misch
On Wed, Jun 13, 2012 at 02:56:58PM -0400, Tom Lane wrote: Noah Misch n...@leadboat.com writes: Given that, how about warning on GLOBAL only but having the documentation equally discourage use of both? Yeah, that's about what I was thinking, too. Any thoughts on the wording of the

Re: [HACKERS] temporal support patch

2012-06-13 Thread Miroslav Šimulčík
There would be no problem to make my solution compatible with SQL 2011, but the standard is not freely available. Can anybody provide me with this standard? 2012/5/20 Pavel Stehule pavel.steh...@gmail.com Hello 2012/5/18 Miroslav Šimulčík simulcik.m...@gmail.com: Hello. SQL 2011

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

2012-06-13 Thread Gabriele Bartolini
Hi Noah, Il 10/06/12 22:53, Noah Misch ha scritto: This has bitrotted; please refresh. Also, please evaluate Peter's feedback: http://archives.postgresql.org/message-id/1333693277.32606.9.ca...@vanquo.pezone.net Our goal is to work on this patch from the next commit fest. What we are about

Re: [HACKERS] temporal support patch

2012-06-13 Thread Miroslav Šimulčík
2012/5/30 Jim Nasby j...@nasby.net On 5/18/12 2:06 AM, Miroslav Šimulčík wrote: - no data redundancy - in my extension current versions of entries are stored only once in original table (in table_log - entries are inserted to both original and log table) That's not necessarily a

Re: [HACKERS] Is cachedFetchXidStatus provably valid?

2012-06-13 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: It's probably an academic concern, but what happens if a backend saves off cachedFetchXidStatus and then sleeps for a very long time. During that time an xid wraparound happens and the backend wakes up and happens to read another unhinted tuple with

Re: [HACKERS] temporal support patch

2012-06-13 Thread Miroslav Šimulčík
* I'd very much like to see you make use of Range Types from 9.2; in particular, TSTZRANGE would be much better than holding two timestamps. If a standard requires you to display two timestamps in certain situations, perhaps you could use ranges internally and display the boundaries as

Re: [HACKERS] temporal support patch

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 4:10 PM, Miroslav Šimulčík simulcik.m...@gmail.com wrote: I have working patch for postgresql version 9.0.4, but it needs refactoring before i can submit it, because some parts don't meet formatting requirements yet. And yes, changes are large, so it will be better to

Re: [HACKERS] Is cachedFetchXidStatus provably valid?

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 3:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: It's probably an academic concern, but what happens if a backend saves off cachedFetchXidStatus and then sleeps for a very long time.  During that time an xid wraparound happens and

Re: [HACKERS] [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

2012-06-13 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Wed, Jun 13, 2012 at 02:56:58PM -0400, Tom Lane wrote: Any thoughts on the wording of the WARNING message? My patch used GLOBAL is deprecated in temporary table creation, which still seems fine to me. Here's an update based on this discussion. Applied

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

2012-06-13 Thread Noah Misch
On Wed, Jun 13, 2012 at 10:12:18PM +0200, Gabriele Bartolini wrote: Our goal is to work on this patch from the next commit fest. What we are about to do for this commit fest is to split the previous patch and send a small one just for the array_remove() and array_replace() functions.

Re: [HACKERS] Ability to listen on two unix sockets

2012-06-13 Thread Peter Eisentraut
On ons, 2012-06-13 at 15:14 +0200, Honza Horak wrote: Thus, I'd like to ask if anybody is aware of any issue connected with simply patching pg_config_manual.h, same as Debian does it already? For example, is there any piece of software, that simply rely on /tmp location of the socket and

Re: [HACKERS] Ability to listen on two unix sockets

2012-06-13 Thread Peter Eisentraut
On mån, 2012-06-11 at 18:07 -0400, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On sön, 2012-06-10 at 17:24 -0400, Robert Haas wrote: Why would that matter? If you configure M ports and N Unix socket locations, you get M*N actual sockets created. ...I *seriously* doubt that

[HACKERS] Re: [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

2012-06-13 Thread Noah Misch
On Wed, Jun 13, 2012 at 05:50:36PM -0400, Tom Lane wrote: Applied with some further wordsmithing on docs and comments. We can still tweak this if anyone objects, of course, but I thought it'd probably save work to get it in before the branch. Thanks. The SQL standard also distinguishes

Re: [HACKERS] Ability to listen on two unix sockets

2012-06-13 Thread Peter Eisentraut
On mån, 2012-06-11 at 18:45 -0500, Michael Nolan wrote: What about entries in pg_hba.conf? Will they need to be able to specify both the directory and the port number? I think the philosophy behind pg_hba.conf is that you distinguish client *systems*. So one client might be Kerberos-capable,

Re: [HACKERS] Ability to listen on two unix sockets

2012-06-13 Thread Peter Eisentraut
On tis, 2012-06-12 at 14:47 +0200, Honza Horak wrote: This could be true in case all listening ports are equal, which I guess isn't a good idea, because IIUIC the port number as a part of the socket name is used for distinguish sockets of various postmasters in the same directory. In that

Re: [HACKERS] Ability to listen on two unix sockets

2012-06-13 Thread Peter Eisentraut
On ons, 2012-06-13 at 15:25 +0200, Honza Horak wrote: It seems unix_socket_directory could be turned into list and probably renamed to unix_socket_directories, since it would be confusing if a list value is in singular. Well, it would also be annoying to rename the parameter name for a

Re: [HACKERS] [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

2012-06-13 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Wed, Jun 13, 2012 at 05:50:36PM -0400, Tom Lane wrote: The SQL standard also distinguishes between global and local temporary tables, where a local temporary table is only visible within a specific SQL module, though its definition is still shared

[HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-06-13 Thread Alexander Korotkov
Hackers, attached patch implements quad-tree on ranges. Some performance results in comparison with current GiST indexing. Index creation is slightly slower. Probably, it need some investigation. Search queries on SP-GiST use much more pages. However this comparison can be not really correct,

Re: [HACKERS] Minimising windows installer password confusion

2012-06-13 Thread Craig Ringer
On 06/13/2012 05:10 PM, Dave Page wrote: The idea of storing the password in clear text in the registry gives me nervous twitches. Me too. It's horrible, and I really dislike the idea. I can't imagine that Microsoft don't have a better solution to this. I talked to some Microsoft people at

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

2012-06-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: ! DETAIL: Character with value 0x0a must be escaped. ! CONTEXT: JSON data, line 1: abc ! ... This seems an odd way to present this, especially if the goal is to NOT include the character needing escaping in the log unescaped, which I thought was

Re: [HACKERS] Minimising windows installer password confusion

2012-06-13 Thread Craig Ringer
On 06/13/2012 05:14 PM, Dave Page wrote: On Wed, Jun 13, 2012 at 2:18 AM, Craig Ringer cr...@postnewspapers.com.au wrote: On 06/12/2012 08:08 PM, Dave Page wrote: Some background: By default the installer will use 'postgres' for both the service (OS) account, and the database superuser

Re: [HACKERS] Ability to listen on two unix sockets

2012-06-13 Thread Bruce Momjian
On Thu, Jun 14, 2012 at 01:31:31AM +0300, Peter Eisentraut wrote: On ons, 2012-06-13 at 15:25 +0200, Honza Horak wrote: It seems unix_socket_directory could be turned into list and probably renamed to unix_socket_directories, since it would be confusing if a list value is in singular.

Re: [HACKERS] Minimising windows installer password confusion

2012-06-13 Thread Craig Ringer
On 06/13/2012 05:18 PM, Dave Page wrote: On Wed, Jun 13, 2012 at 3:07 AM, Craig Ringer Why using the windows control panel ? Because when I wrote the email I was looking for a simple solution that wouldn't require writing code that has potential to fail depending on how the users

Re: [HACKERS] Minimising windows installer password confusion

2012-06-13 Thread Craig Ringer
On 06/13/2012 06:32 PM, Florian Pflug wrote: Some further googling indicates that, yes, the service account passwords are stored in the registry, but are only accessible to the LocalSystem account [2]. Querying them from the postgres installer thus isn't really an option. But what you could do,

[HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-06-13 Thread Greg Stark
On Mon, Jun 4, 2012 at 8:49 PM, Christopher Browne cbbro...@gmail.com wrote: What if the two servers are in different time zones? NTP shouldn't have any problem; it uses UTC underneath.  As does PostgreSQL, underneath. As an aside, this is not strictly speaking true. NTP doesn't use UTC --

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-06-13 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: I thought I already pointed that out, but: we have *extensions*. What we don't have is a convenient method of dealing with functions that need to be migrated across extensions, or from an extension to core,

  1   2   >