Re: [HACKERS] Changeset Extraction v7.6.1

2014-06-01 Thread Euler Taveira
On 01-06-2014 02:57, Andres Freund wrote: On 2014-06-01 00:50:58 -0500, Jim Nasby wrote: On 5/31/14, 9:11 AM, Andres Freund wrote: On 2014-02-21 15:14:15 -0600, Jim Nasby wrote: On 2/17/14, 7:31 PM, Robert Haas wrote: But do you really want to keep that snapshot around long enough to copy

Re: [HACKERS] Changeset Extraction v7.6.1

2014-06-01 Thread Jim Nasby
On 6/1/14, 10:49 AM, Euler Taveira wrote: On 01-06-2014 02:57, Andres Freund wrote: On 2014-06-01 00:50:58 -0500, Jim Nasby wrote: On 5/31/14, 9:11 AM, Andres Freund wrote: On 2014-02-21 15:14:15 -0600, Jim Nasby wrote: On 2/17/14, 7:31 PM, Robert Haas wrote: But do you really want to keep

Re: [HACKERS] Changeset Extraction v7.6.1

2014-05-31 Thread Andres Freund
On 2014-02-21 15:14:15 -0600, Jim Nasby wrote: On 2/17/14, 7:31 PM, Robert Haas wrote: But do you really want to keep that snapshot around long enough to copy the entire database? I bet you don't: if the database is big, holding back xmin for long enough to copy the whole thing isn't likely

Re: [HACKERS] Changeset Extraction v7.6.1

2014-05-31 Thread Jim Nasby
On 5/31/14, 9:11 AM, Andres Freund wrote: On 2014-02-21 15:14:15 -0600, Jim Nasby wrote: On 2/17/14, 7:31 PM, Robert Haas wrote: But do you really want to keep that snapshot around long enough to copy the entire database? I bet you don't: if the database is big, holding back xmin for long

Re: [HACKERS] Changeset Extraction v7.6.1

2014-05-31 Thread Andres Freund
On 2014-06-01 00:50:58 -0500, Jim Nasby wrote: On 5/31/14, 9:11 AM, Andres Freund wrote: On 2014-02-21 15:14:15 -0600, Jim Nasby wrote: On 2/17/14, 7:31 PM, Robert Haas wrote: But do you really want to keep that snapshot around long enough to copy the entire database? I bet you don't: if the

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-27 Thread Andres Freund
On 2014-02-24 12:50:03 -0500, Robert Haas wrote: On Mon, Feb 24, 2014 at 9:48 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-15 17:29:04 -0500, Robert Haas wrote: On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund and...@2ndquadrant.com wrote: + /* +* XXX: It's

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-27 Thread Robert Haas
On Thu, Feb 27, 2014 at 11:06 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-24 12:50:03 -0500, Robert Haas wrote: On Mon, Feb 24, 2014 at 9:48 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-15 17:29:04 -0500, Robert Haas wrote: On Fri, Feb 14, 2014 at 4:55 AM,

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-24 Thread Andres Freund
On 2014-02-15 17:29:04 -0500, Robert Haas wrote: On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund and...@2ndquadrant.com wrote: + /* +* XXX: It's impolite to ignore our argument and keep decoding until the +* current position. +*/ Eh, what? So, the

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-24 Thread Robert Haas
On Mon, Feb 24, 2014 at 9:48 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-15 17:29:04 -0500, Robert Haas wrote: On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund and...@2ndquadrant.com wrote: + /* +* XXX: It's impolite to ignore our argument and keep decoding until

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Andres Freund
Hi, On 2014-02-19 13:01:02 -0500, Robert Haas wrote: I think it should be fairly easy to relax the restriction to creating a slot, but not getting data from it. Do you think that would that be sufficient? That would be a big improvement, for sure, and might be entirely sufficient.

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Andres Freund
On 2014-02-19 13:31:06 -0500, Robert Haas wrote: TBH, as compared to what you've got now, I think this mostly boils down to a question of quoting and escaping. I'm not really concerned with whether we ship something that's perfectly efficient, or that has filtering capabilities, or that has a

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Andres Freund
On 2014-02-19 13:07:11 -0500, Robert Haas wrote: On Tue, Feb 18, 2014 at 4:07 AM, Andres Freund and...@2ndquadrant.com wrote: 2. I think the snapshot-export code is fundamentally misdesigned. As I said before, the idea that we're going to export one single snapshot at one particular point

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Robert Haas
On Fri, Feb 21, 2014 at 6:07 AM, Andres Freund and...@2ndquadrant.com wrote: I can sympathize with the too much during init argument, but I don't see how moving stuff to the first call would get rid of the problems. If we fail later it's going to be just as confusing. No, it isn't. If you

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Andres Freund
On 2014-02-21 08:16:59 -0500, Robert Haas wrote: On Fri, Feb 21, 2014 at 6:07 AM, Andres Freund and...@2ndquadrant.com wrote: I can sympathize with the too much during init argument, but I don't see how moving stuff to the first call would get rid of the problems. If we fail later it's

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Robert Haas
On Fri, Feb 21, 2014 at 8:27 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-21 08:16:59 -0500, Robert Haas wrote: On Fri, Feb 21, 2014 at 6:07 AM, Andres Freund and...@2ndquadrant.com wrote: I can sympathize with the too much during init argument, but I don't see how moving

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Andres Freund
On 2014-02-21 08:51:03 -0500, Robert Haas wrote: On Fri, Feb 21, 2014 at 8:27 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-21 08:16:59 -0500, Robert Haas wrote: On Fri, Feb 21, 2014 at 6:07 AM, Andres Freund and...@2ndquadrant.com wrote: I can sympathize with the too much

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Jim Nasby
On 2/17/14, 7:31 PM, Robert Haas wrote: But do you really want to keep that snapshot around long enough to copy the entire database? I bet you don't: if the database is big, holding back xmin for long enough to copy the whole thing isn't likely to be fun. I can confirm that this would be epic

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-19 Thread Robert Haas
On Sat, Feb 15, 2014 at 6:59 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-15 17:29:04 -0500, Robert Haas wrote: On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund and...@2ndquadrant.com wrote: [ new patches ] 0001 already needs minor Hm? If there are conflicts, I'll push/send

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-19 Thread Robert Haas
On Sun, Feb 16, 2014 at 12:12 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-15 17:29:04 -0500, Robert Haas wrote: On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund and...@2ndquadrant.com wrote: [ new patches ] 0001 already needs minor + * copied stuff from tuptoaster.c. Perhaps

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-19 Thread Robert Haas
On Tue, Feb 18, 2014 at 4:07 AM, Andres Freund and...@2ndquadrant.com wrote: 2. I think the snapshot-export code is fundamentally misdesigned. As I said before, the idea that we're going to export one single snapshot at one particular point in time strikes me as extremely short-sighted. I

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-19 Thread Robert Haas
On Tue, Feb 18, 2014 at 4:33 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-17 21:35:23 -0500, Robert Haas wrote: What I don't understand is why we're not taking the test_decoding module, polishing it up a little to produce some nice, easily machine-parseable output, calling it

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-19 Thread Euler Taveira
On 18-02-2014 06:33, Andres Freund wrote: I really hope there will be nicer ones by the time 9.4 is released. Euler did send in a json plugin http://archives.postgresql.org/message-id/52A5BFAE.1040209%2540timbira.com.br , but there hasn't too much feedback yet. It's hard to start discussing

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-18 Thread Andres Freund
Hi Robert, On 2014-02-17 20:31:34 -0500, Robert Haas wrote: 1. How safe is it to try to do decoding inside of a regular backend? What we're doing here is entering a special mode where we forbid the use of regular snapshots in favor of requiring the use of decoding snapshots, and forbid access

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-18 Thread Andres Freund
On 2014-02-17 21:10:26 -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: 1. How safe is it to try to do decoding inside of a regular backend? What we're doing here is entering a special mode where we forbid the use of regular snapshots in favor of requiring the use of

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-18 Thread Andres Freund
On 2014-02-17 18:49:59 -0800, Peter Geoghegan wrote: On Mon, Feb 17, 2014 at 6:35 PM, Robert Haas robertmh...@gmail.com wrote: What I actually suspect is going to happen if we ship this as-is is that people are going to start building logical replication solutions on top of the

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-18 Thread Andres Freund
On 2014-02-17 21:35:23 -0500, Robert Haas wrote: What I don't understand is why we're not taking the test_decoding module, polishing it up a little to produce some nice, easily machine-parseable output, calling it basic_decoding, and shipping that. Then people who want something else can

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-17 Thread Robert Haas
On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund and...@2ndquadrant.com wrote: [ patches ] Having now had a little bit of opportunity to reflect on the State Of This Patch, I'd like to step back from the minutia upon which I've been commenting in my previous emails and articulate three high-level

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Having now had a little bit of opportunity to reflect on the State Of This Patch, I'd like to step back from the minutia upon which I've been commenting in my previous emails and articulate three high-level concerns about this patch. In so doing, I

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-17 Thread Robert Haas
On Mon, Feb 17, 2014 at 9:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: 3. As this feature is proposed, the only plugin we'll ship with 9.4 is a test_decoding plugin which, as its own documentation says, doesn't do anything especially useful. What exactly do we gain by forcing users who want to

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-17 Thread Peter Geoghegan
On Mon, Feb 17, 2014 at 6:35 PM, Robert Haas robertmh...@gmail.com wrote: What I actually suspect is going to happen if we ship this as-is is that people are going to start building logical replication solutions on top of the test_decoding module even though it explicitly says that it's just

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-16 Thread Andres Freund
On 2014-02-15 17:29:04 -0500, Robert Haas wrote: On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund and...@2ndquadrant.com wrote: [ new patches ] 0001 already needs minor + * copied stuff from tuptoaster.c. Perhaps there should be toast_internal.h? Yes, please. If you can submit a

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-15 Thread Robert Haas
On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund and...@2ndquadrant.com wrote: [ new patches ] 0001 already needs minor + * copied stuff from tuptoaster.c. Perhaps there should be toast_internal.h? Yes, please. If you can submit a separate patch creating this file and relocating this stuff

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-15 Thread Andres Freund
On 2014-02-15 17:29:04 -0500, Robert Haas wrote: On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund and...@2ndquadrant.com wrote: [ new patches ] 0001 already needs minor Hm? If there are conflicts, I'll push/send a rebased tomorrow or monday. +* the transaction's invalidations. This