Re: [HACKERS] A Modest Upgrade Proposal

On Thu, Jul 28, 2016 at 09:22:17AM +0800, Craig Ringer wrote: > It might, actually. One approach for online upgrade is to: > > * pg_basebackup the master > * start the replica and let it catch up > * create a logical replication slot on the master > * replace the replication.conf on the

Re: [HACKERS] A Modest Upgrade Proposal

On Sun, Jul 17, 2016 at 02:55:20PM -0400, Jan Wieck wrote: > On Sun, Jul 17, 2016 at 2:08 PM, Jim Nasby wrote: > > On 7/13/16 2:06 PM, Joshua D. Drake wrote: > >> On 07/07/2016 01:01 PM, Robert Haas wrote: > >> There was an unconference session on this topic at PGCon and

Re: [HACKERS] A Modest Upgrade Proposal

On Thu, Jul 28, 2016 at 10:22 AM, Craig Ringer wrote: > It might, actually. One approach for online upgrade is to: > > * pg_basebackup the master > * start the replica and let it catch up > * create a logical replication slot on the master > * replace the replication.conf

Re: [HACKERS] A Modest Upgrade Proposal

On 28 July 2016 at 04:35, Bruce Momjian wrote: > On Fri, Jul 8, 2016 at 12:18:28AM +0100, Simon Riggs wrote: > > On 7 July 2016 at 21:10, Robert Haas wrote: > > > > pg_upgrade does that, kinda. I'd like to have something better, but > > in the

Re: [HACKERS] A Modest Upgrade Proposal

On Fri, Jul 8, 2016 at 12:18:28AM +0100, Simon Riggs wrote: > On 7 July 2016 at 21:10, Robert Haas wrote: > > pg_upgrade does that, kinda.  I'd like to have something better, but > in the absence of that, I think it's quite wrong to think about > deprecating

Re: [HACKERS] A Modest Upgrade Proposal

On 07/18/2016 03:17 PM, Jim Nasby wrote: On 7/17/16 2:22 PM, Petr Jelinek wrote: I do agree that DDL "feels better" (which I think is what JD was alluding too). Yes and no. It reads better and is more clear to those who are not developers or have a developer background which is, many in

Re: [HACKERS] A Modest Upgrade Proposal

On 7/17/16 2:22 PM, Petr Jelinek wrote: I generally agree, but I think the more important question is "Why?". Is it becouse DDL looks more like a sentence? Is it because arrays are a PITA? Is it too hard to call functions? For me it's many small reasons. I want to store it in catalogs and some

Re: [HACKERS] A Modest Upgrade Proposal

On 07/17/2016 11:55 AM, Jan Wieck wrote: Yeah, I haven't meet anyone yet that would like to have: select replicate_these_relations('['public']); vs: ALTER SCHEMA public ENABLE REPLICATION; (or something like that). I generally agree, but I think

Re: [HACKERS] A Modest Upgrade Proposal

On 17/07/16 20:50, Robert Haas wrote: It's the same with cluster-wide management, dump and restore of replication state to re-create a replication setup elsewhere, etc. We have to build the groundwork first. Trying to pour the top storey concrete when the bottom storey isn't even there yet

Re: [HACKERS] A Modest Upgrade Proposal

On 17/07/16 20:08, Jim Nasby wrote: On 7/13/16 2:06 PM, Joshua D. Drake wrote: On 07/07/2016 01:01 PM, Robert Haas wrote: There was an unconference session on this topic at PGCon and quite a number of people there stated that they found DDL to be an ease-of-use feature and wanted to have it.

Re: [HACKERS] A Modest Upgrade Proposal

On Sun, Jul 17, 2016 at 2:08 PM, Jim Nasby wrote: > On 7/13/16 2:06 PM, Joshua D. Drake wrote: > >> On 07/07/2016 01:01 PM, Robert Haas wrote: >> >> There was an unconference session on this topic at PGCon and quite a >>> number of people there stated that they found

Re: [HACKERS] A Modest Upgrade Proposal

On Thu, Jul 14, 2016 at 2:29 AM, Craig Ringer wrote: > Yes, I'd like that too. I'd also like to have fully parallized writeable > queries right now. But we can't build everything all at once. I agree. > Before doing parallelized writes, things like dsm, dsm queues, group

Re: [HACKERS] A Modest Upgrade Proposal

On 7/7/16 8:17 PM, Simon Riggs wrote: Simplicity is key, I agree. But that's just a user interface feature, not a comment on what's underneath the covers. pg_upgrade is not simple and is never likely to be so, under the covers. Right, and what I'd prefer effort put into is making managing

Re: [HACKERS] A Modest Upgrade Proposal

On 7/13/16 2:06 PM, Joshua D. Drake wrote: On 07/07/2016 01:01 PM, Robert Haas wrote: There was an unconference session on this topic at PGCon and quite a number of people there stated that they found DDL to be an ease-of-use feature and wanted to have it. Yeah, I haven't meet anyone yet

Re: [HACKERS] A Modest Upgrade Proposal

On 14 July 2016 at 03:26, Robert Haas wrote: > On Fri, Jul 8, 2016 at 5:47 AM, Craig Ringer > wrote: > >> DDL is our standard way of getting things into the system catalogs. > >> We have no system catalog metadata that is intended to be populated by

Re: [HACKERS] A Modest Upgrade Proposal

On 14 July 2016 at 03:06, Robert Haas wrote: > Physical replication has > the same issue. Users don't want to configure archive_command and > wal_keep_segments and max_wal_senders and wal_level and set up an > archive and create recovery.conf on the standby. They want

Re: [HACKERS] A Modest Upgrade Proposal

On 13/07/16 21:06, Robert Haas wrote: We have much to discuss in terms of security, the way it should work and what options to support and a sidetrack into syntax isn't warranted at this early stage. Please lets discuss those important things first, then return to whether DDL makes sense or

Re: [HACKERS] A Modest Upgrade Proposal

On 13 July 2016 at 15:06, Robert Haas wrote: > On Thu, Jul 7, 2016 at 9:25 PM, Simon Riggs wrote: > > I note also that replication slots aren't backed up by pg_dump; I see > > analogy here and think that at least some parts of logical replication >

Re: [HACKERS] A Modest Upgrade Proposal

On Fri, Jul 8, 2016 at 5:47 AM, Craig Ringer wrote: >> DDL is our standard way of getting things into the system catalogs. >> We have no system catalog metadata that is intended to be populated by >> any means other than DDL. > > Replication slots? (Arguably not catalogs, I

Re: [HACKERS] A Modest Upgrade Proposal

On Thu, Jul 7, 2016 at 9:25 PM, Simon Riggs wrote: > I note also that replication slots aren't backed up by pg_dump; I see > analogy here and think that at least some parts of logical replication will > be similar and not require DDL at all, just as slots do not. I agree

Re: [HACKERS] A Modest Upgrade Proposal

On 07/07/2016 01:01 PM, Robert Haas wrote: There was an unconference session on this topic at PGCon and quite a number of people there stated that they found DDL to be an ease-of-use feature and wanted to have it. Yeah, I haven't meet anyone yet that would like to have: select

Re: [HACKERS] A Modest Upgrade Proposal

On Thu, Jul 7, 2016 at 8:53 PM, Simon Riggs wrote: >> I thought I sat through, at least, most of it, but you barely gave >> anyone else a chance to talk, which kind of misses the point of an >> unconference. The portion which I attended was not about how to move >> the

Re: [HACKERS] A Modest Upgrade Proposal

On 08/07/16 12:47, Craig Ringer wrote: On 8 July 2016 at 09:41, Robert Haas > wrote: If you want to add a column to a table, you say ALTER TABLE .. ADD COLUMN. If you want to add a column to an extension, you say ALTER EXTENSION

Re: [HACKERS] A Modest Upgrade Proposal

On 8 July 2016 at 11:18, Alvaro Herrera wrote: > Tom Lane wrote: > > Simon Riggs writes: > > > > pg_am has existed for decades without supporting DDL > > > > That argument has been obsoleted by events ;-) ... and in any case, the > > reason we

Re: [HACKERS] A Modest Upgrade Proposal

On 8 July 2016 at 09:41, Robert Haas wrote: > > > Personally, I'm in the group of people that don't see the need for DDL. > > There are already many successful features that don't utilize DDL, such > as > > backup, advisory locks and some features that use DDL that don't

Re: [HACKERS] A Modest Upgrade Proposal

On 8 July 2016 at 11:09, Petr Jelinek wrote: > > One interesting thing will be making sure we can replicate from physical > standby in the future as you mentioned elsewhere in the thread but I think > that should be possible as long as you define the catalogs on master (not

Re: [HACKERS] A Modest Upgrade Proposal

On 08/07/16 10:59, Simon Riggs wrote: On 8 July 2016 at 03:55, Tom Lane > wrote: > One of my examples was full text search and it does have > DDL, but that was an anti-example; all the feedback I have is that it was > much easier to use

Re: [HACKERS] A Modest Upgrade Proposal

On 8 July 2016 at 03:55, Tom Lane wrote: > > One of my examples was full text search and it does have > > DDL, but that was an anti-example; all the feedback I have is that it was > > much easier to use before it had DDL and that forcing it to use DDL > pretty > > much

Re: [HACKERS] A Modest Upgrade Proposal

Tom Lane wrote: > Simon Riggs writes: > > pg_am has existed for decades without supporting DDL > > That argument has been obsoleted by events ;-) ... and in any case, the > reason we went without CREATE ACCESS METHOD for so long was not that we > encouraged "INSERT INTO

Re: [HACKERS] A Modest Upgrade Proposal

Simon Riggs writes: > On 8 July 2016 at 02:41, Robert Haas wrote: >> DDL is our standard way of getting things into the system catalogs. >> We have no system catalog metadata that is intended to be populated by >> any means other than DDL. > pg_am

Re: [HACKERS] A Modest Upgrade Proposal

On 8 July 2016 at 02:41, Robert Haas wrote: > > Personally, I'm in the group of people that don't see the need for DDL. > > There are already many successful features that don't utilize DDL, such > as > > backup, advisory locks and some features that use DDL that don't

Re: [HACKERS] A Modest Upgrade Proposal

On 8 July 2016 at 02:41, Robert Haas wrote: > On Thu, Jul 7, 2016 at 7:15 PM, Simon Riggs wrote: > > Yes, I ran the unconference session. It was a shame you weren't able to > stay > > for the whole discussion. > > I thought I sat through, at least,

Re: [HACKERS] A Modest Upgrade Proposal

On Thu, Jul 7, 2016 at 7:15 PM, Simon Riggs wrote: > Yes, I ran the unconference session. It was a shame you weren't able to stay > for the whole discussion. I thought I sat through, at least, most of it, but you barely gave anyone else a chance to talk, which kind of

Re: [HACKERS] A Modest Upgrade Proposal

On 8 July 2016 at 01:47, Joshua D. Drake wrote: > * Long running transaction > And of course you can't run any transactions at all during pg_upgrade, not just long running ones. -- Simon Riggshttp://www.2ndQuadrant.com/

Re: [HACKERS] A Modest Upgrade Proposal

On 8 July 2016 at 01:47, Joshua D. Drake wrote: > It is true that something like pg_logical doesn't suffer from those three > things but it does suffer from others: > > * No DDL - Agreed, not "required" but certainly a very nice > feature. > > * Lack of

Re: [HACKERS] A Modest Upgrade Proposal

On 07/07/2016 05:14 PM, Simon Riggs wrote: I would much rather see more brain power put into pg_upgrade or in place upgrades than logical replication (as a upgrade solution). Why is that? First, let me state that I don't have a problem with logical replication as an upgrade

Re: [HACKERS] A Modest Upgrade Proposal

On 8 July 2016 at 00:48, Joshua D. Drake wrote: > On 07/07/2016 01:10 PM, Robert Haas wrote: > >> On Mon, May 16, 2016 at 8:52 PM, David Fetter wrote: >> >>> In light of the above, it is perfectly reasonable to require, at least >>> temporarily, setting

Re: [HACKERS] A Modest Upgrade Proposal

On 07/07/2016 01:10 PM, Robert Haas wrote: On Mon, May 16, 2016 at 8:52 PM, David Fetter wrote: In light of the above, it is perfectly reasonable to require, at least temporarily, setting up duplicate storage, or another node. pg_upgrade does that, kinda. I'd like to have

Re: [HACKERS] A Modest Upgrade Proposal

On 7 July 2016 at 21:10, Robert Haas wrote: > pg_upgrade does that, kinda. I'd like to have something better, but > in the absence of that, I think it's quite wrong to think about > deprecating it, even if we had logical replication fully integrated > into core today.

Re: [HACKERS] A Modest Upgrade Proposal

On 7 July 2016 at 21:01, Robert Haas wrote: > On Mon, May 16, 2016 at 9:22 PM, Alvaro Herrera > wrote: > > David Fetter wrote: > >> As a relatively (to our users) minor course correction, I would like > >> to propose the following: > > > >> -

Re: [HACKERS] A Modest Upgrade Proposal

On Mon, May 16, 2016 at 8:52 PM, David Fetter wrote: > In light of the above, it is perfectly reasonable to require, at least > temporarily, setting up duplicate storage, or another node. > > I am aware that some cases exist where this is not possible, but I > don't think we

Re: [HACKERS] A Modest Upgrade Proposal

On Mon, May 16, 2016 at 9:22 PM, Alvaro Herrera wrote: > David Fetter wrote: >> As a relatively (to our users) minor course correction, I would like >> to propose the following: > >> - Develop a logical upgrade path as a part of the (Yay! Sexy!) logical >> replication

Re: [HACKERS] A Modest Upgrade Proposal

Joshua D. Drake wrote: > On 05/16/2016 06:32 PM, Alvaro Herrera wrote: > >Joshua D. Drake wrote: > > > >>Alvaro, > >> > >>Thank you for bringing this to light. Is there a contributor FAQ for > >>PgLogical so that people can help? > > > >Hmm, I don't think there's any contributor FAQ. It's

Re: [HACKERS] A Modest Upgrade Proposal

On Mon, May 16, 2016 at 06:20:34PM -0700, Joshua D. Drake wrote: > On 05/16/2016 05:52 PM, David Fetter wrote: > > Folks, > > > This path would, of course, run either locally or across a > > network, and be testable in both cases. There would be a > > downgrade path, namely switching origin

Re: [HACKERS] A Modest Upgrade Proposal

On 05/16/2016 06:32 PM, Alvaro Herrera wrote: Joshua D. Drake wrote: Alvaro, Thank you for bringing this to light. Is there a contributor FAQ for PgLogical so that people can help? Hmm, I don't think there's any contributor FAQ. It's supposed to be a regular patch submission, after all --

Re: [HACKERS] A Modest Upgrade Proposal

Joshua D. Drake wrote: > Alvaro, > > Thank you for bringing this to light. Is there a contributor FAQ for > PgLogical so that people can help? Hmm, I don't think there's any contributor FAQ. It's supposed to be a regular patch submission, after all -- it needs user interface review, a review

Re: [HACKERS] A Modest Upgrade Proposal

On 05/16/2016 06:22 PM, Alvaro Herrera wrote: David Fetter wrote: As a relatively (to our users) minor course correction, I would like to propose the following: - Develop a logical upgrade path as a part of the (Yay! Sexy!) logical replication that's already in large part built. This

Re: [HACKERS] A Modest Upgrade Proposal

David Fetter wrote: > As a relatively (to our users) minor course correction, I would like > to propose the following: > - Develop a logical upgrade path as a part of the (Yay! Sexy!) logical > replication that's already in large part built. > > This path would, of course, run either

Re: [HACKERS] A Modest Upgrade Proposal

On 05/16/2016 05:52 PM, David Fetter wrote: Folks, This path would, of course, run either locally or across a network, and be testable in both cases. There would be a downgrade path, namely switching origin nodes. What say? What happens when the database is 5TB in size and you only have