Re: pg_basebackup ignores the existing data directory permissions

2019-11-27 Thread Michael Paquier
On Wed, Sep 04, 2019 at 04:11:17PM -0400, Alvaro Herrera wrote: > This patch has been dormant for months. There's been at lot of > discussion but it doesn't seem conclusive; it doesn't look like we know > what we actually want to do. Can I try to restart the discussion and > see if we can get to

Re: pg_basebackup ignores the existing data directory permissions

2019-09-04 Thread Alvaro Herrera
On 2019-Apr-03, Robert Haas wrote: > I am not sure what solution is best here, but it is hard to imagine > that the status quo is the right thing. This patch has been dormant for months. There's been at lot of discussion but it doesn't seem conclusive; it doesn't look like we know what we

Re: pg_basebackup ignores the existing data directory permissions

2019-04-03 Thread Robert Haas
On Fri, Mar 29, 2019 at 6:05 AM Peter Eisentraut wrote: > On 2019-03-26 03:26, Michael Paquier wrote: > > Do we really want to extend the replication protocol to control that? > > Perhaps we are losing sight of the original problem, which is that if > you create the target directory with the

Re: pg_basebackup ignores the existing data directory permissions

2019-04-02 Thread Haribabu Kommi
On Fri, Mar 29, 2019 at 9:05 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-03-26 03:26, Michael Paquier wrote: > > Do we really want to extend the replication protocol to control that? > > Perhaps we are losing sight of the original problem, which is that if > you

Re: Re: pg_basebackup ignores the existing data directory permissions

2019-03-29 Thread David Steele
On 3/26/19 3:59 AM, Haribabu Kommi wrote: I am really questioning if we should keep this stuff isolated within pg_basebackup or not.  At the same time, it may be confusing to have BASE_BACKUP only use the permissions inherited from the data directory, so some input from folks

Re: pg_basebackup ignores the existing data directory permissions

2019-03-29 Thread Peter Eisentraut
On 2019-03-26 03:26, Michael Paquier wrote: > Do we really want to extend the replication protocol to control that? Perhaps we are losing sight of the original problem, which is that if you create the target directory with the wrong permissions then ... it has the wrong permissions. And you are

Re: pg_basebackup ignores the existing data directory permissions

2019-03-25 Thread Haribabu Kommi
On Tue, Mar 26, 2019 at 1:27 PM Michael Paquier wrote: > On Sun, Mar 24, 2019 at 10:30:47PM +1100, Haribabu Kommi wrote: > > With the above additional options, the pg_basebackup is able to control > > the access permissions of the backup files, but when it comes to tar mode > > all the files are

Re: pg_basebackup ignores the existing data directory permissions

2019-03-25 Thread Michael Paquier
On Sun, Mar 24, 2019 at 10:30:47PM +1100, Haribabu Kommi wrote: > With the above additional options, the pg_basebackup is able to control > the access permissions of the backup files, but when it comes to tar mode > all the files are sent from the server and stored as it is in backup, to > support

Re: pg_basebackup ignores the existing data directory permissions

2019-03-25 Thread Michael Paquier
On Mon, Mar 25, 2019 at 09:08:23AM -0400, Robert Haas wrote: > If we're going to go with -g {inherit|none|group} then -g inherit > ought to do what was originally proposed on this thread -- i.e. set > the directory permissions to match the contents. I don't think that's > a change that can or

Re: pg_basebackup ignores the existing data directory permissions

2019-03-25 Thread Robert Haas
On Thu, Mar 21, 2019 at 8:42 PM Michael Paquier wrote: > > Why not? > > Because we have released v11 so as we respect the permissions set on > the source instead from which the backup is taken for all the folder's > content. If we begin to enforce it we may break some cases. If a new > option

Re: pg_basebackup ignores the existing data directory permissions

2019-03-24 Thread Haribabu Kommi
On Sat, Mar 23, 2019 at 2:23 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-03-22 05:00, Michael Paquier wrote: > > On Fri, Mar 22, 2019 at 02:45:24PM +1100, Haribabu Kommi wrote: > >> How about letting the pg_basebackup to decide group permissions of the > >> standby

Re: pg_basebackup ignores the existing data directory permissions

2019-03-22 Thread Peter Eisentraut
On 2019-03-22 05:00, Michael Paquier wrote: > On Fri, Mar 22, 2019 at 02:45:24PM +1100, Haribabu Kommi wrote: >> How about letting the pg_basebackup to decide group permissions of the >> standby directory irrespective of the primary directory permissions. >> >> Default - permissions are same as

Re: pg_basebackup ignores the existing data directory permissions

2019-03-21 Thread Michael Paquier
On Fri, Mar 22, 2019 at 02:45:24PM +1100, Haribabu Kommi wrote: > How about letting the pg_basebackup to decide group permissions of the > standby directory irrespective of the primary directory permissions. > > Default - permissions are same as primary > --allow-group-access - standby directory

Re: pg_basebackup ignores the existing data directory permissions

2019-03-21 Thread Haribabu Kommi
On Fri, Mar 22, 2019 at 11:42 AM Michael Paquier wrote: > On Thu, Mar 21, 2019 at 02:56:24PM -0400, Robert Haas wrote: > > On Tue, Mar 19, 2019 at 2:29 AM Michael Paquier > wrote: > >> Hm. We have been assuming that the contents of a base backup inherit > >> the permission of the source when

Re: pg_basebackup ignores the existing data directory permissions

2019-03-21 Thread Michael Paquier
On Thu, Mar 21, 2019 at 02:56:24PM -0400, Robert Haas wrote: > On Tue, Mar 19, 2019 at 2:29 AM Michael Paquier wrote: >> Hm. We have been assuming that the contents of a base backup inherit >> the permission of the source when using pg_basebackup because this >> allows users to keep a nodes in a

Re: pg_basebackup ignores the existing data directory permissions

2019-03-21 Thread Robert Haas
On Tue, Mar 19, 2019 at 2:29 AM Michael Paquier wrote: > Hm. We have been assuming that the contents of a base backup inherit > the permission of the source when using pg_basebackup because this > allows users to keep a nodes in a consistent state without deciding > which option to use. Do you

Re: pg_basebackup ignores the existing data directory permissions

2019-03-20 Thread Peter Eisentraut
On 2019-03-19 08:34, Haribabu Kommi wrote: > How about the following change? > > pg_basebackup  --> copies the contents of the src directory (with group > access)  > and even the root directory permissions. > > pg_basebackup --no-group-access   --> copies the contents of the src > directory  >

Re: pg_basebackup ignores the existing data directory permissions

2019-03-20 Thread Peter Eisentraut
On 2019-03-18 16:45, Robert Haas wrote: >> I'm strongly in favor of keeping initdb and pg_basebackup options >> similar and consistent. They are both ways to initialize data directories. >> >> You'll note that initdb does not behave the way you describe. It's not >> unreasonable behavior, but

Re: pg_basebackup ignores the existing data directory permissions

2019-03-19 Thread Haribabu Kommi
On Tue, Mar 19, 2019 at 5:29 PM Michael Paquier wrote: > On Mon, Mar 18, 2019 at 11:45:05AM -0400, Robert Haas wrote: > > So you want to default to no group access regardless of the directory > > permissions, with an option to enable group access that must be > > explicitly specified? That

Re: pg_basebackup ignores the existing data directory permissions

2019-03-19 Thread Michael Paquier
On Mon, Mar 18, 2019 at 11:45:05AM -0400, Robert Haas wrote: > So you want to default to no group access regardless of the directory > permissions, with an option to enable group access that must be > explicitly specified? That seems like a reasonable option to me; note > that initdb does seem to

Re: pg_basebackup ignores the existing data directory permissions

2019-03-18 Thread Robert Haas
On Mon, Mar 18, 2019 at 11:36 AM Peter Eisentraut wrote: > On 2019-03-18 14:47, Robert Haas wrote: > >> Based on the feedback gathered, having a separate option to enforce > >> the default and not touching the behavior implemented until now, > >> sounds fine to me. > > That's not what I'm

Re: pg_basebackup ignores the existing data directory permissions

2019-03-18 Thread Peter Eisentraut
On 2019-03-18 14:47, Robert Haas wrote: >> Based on the feedback gathered, having a separate option to enforce >> the default and not touching the behavior implemented until now, >> sounds fine to me. > That's not what I'm proposing. I think the behavior implemented until > now is not best,

Re: pg_basebackup ignores the existing data directory permissions

2019-03-18 Thread Robert Haas
On Mon, Mar 18, 2019 at 4:16 AM Michael Paquier wrote: > On Mon, Mar 18, 2019 at 08:32:44AM +0100, Magnus Hagander wrote: > > On Mon, Mar 18, 2019 at 7:08 AM Stephen Frost wrote: > >> I definitely think that we should add an option to allow the user to > >> tell us explicitly what they want

Re: pg_basebackup ignores the existing data directory permissions

2019-03-18 Thread Peter Eisentraut
On 2019-03-16 15:29, Robert Haas wrote: > Another option would be to provide a pg_basebackup option to allow the > user to specify what they intended i.e. --[no-]group-read. (Tying it > to -R doesn't sound like a good decision to me.) I was actually surprised to learn how it works right now. I

Re: pg_basebackup ignores the existing data directory permissions

2019-03-18 Thread Kyotaro HORIGUCHI
At Mon, 18 Mar 2019 17:16:01 +0900, Michael Paquier wrote in <20190318081601.gi1...@paquier.xyz> > On Mon, Mar 18, 2019 at 08:32:44AM +0100, Magnus Hagander wrote: > > On Mon, Mar 18, 2019 at 7:08 AM Stephen Frost wrote: > >> I definitely think that we should add an option to allow the user to

Re: pg_basebackup ignores the existing data directory permissions

2019-03-18 Thread Michael Paquier
On Mon, Mar 18, 2019 at 08:32:44AM +0100, Magnus Hagander wrote: > On Mon, Mar 18, 2019 at 7:08 AM Stephen Frost wrote: >> I definitely think that we should add an option to allow the user to >> tell us explicitly what they want here, even if we also go based on what >> the created directory has

Re: pg_basebackup ignores the existing data directory permissions

2019-03-18 Thread Magnus Hagander
On Mon, Mar 18, 2019 at 7:08 AM Stephen Frost wrote: > Greetings, > > * Robert Haas (robertmh...@gmail.com) wrote: > > On Thu, Mar 14, 2019 at 7:34 PM Peter Eisentraut > > wrote: > > > I think the potential problems of getting this wrong are bigger than > the > > > issue we are trying to fix. >

Re: pg_basebackup ignores the existing data directory permissions

2019-03-18 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Mar 14, 2019 at 7:34 PM Peter Eisentraut > wrote: > > I think the potential problems of getting this wrong are bigger than the > > issue we are trying to fix. > > I think the question is: how do we know what the user intended?

Re: pg_basebackup ignores the existing data directory permissions

2019-03-16 Thread Robert Haas
On Thu, Mar 14, 2019 at 7:34 PM Peter Eisentraut wrote: > I think the potential problems of getting this wrong are bigger than the > issue we are trying to fix. I think the question is: how do we know what the user intended? If the user wants the directory to be accessible only to the owner,

Re: pg_basebackup ignores the existing data directory permissions

2019-03-15 Thread Haribabu Kommi
On Fri, Mar 15, 2019 at 10:33 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-03-09 02:19, Haribabu Kommi wrote: > > Yes, I agree that it may be a problem if the existing data directory > > permissions > > are 0700 to changing it to 0750. But it may not be a problem for

Re: pg_basebackup ignores the existing data directory permissions

2019-03-14 Thread Peter Eisentraut
On 2019-03-09 02:19, Haribabu Kommi wrote: > Yes, I agree that it may be a problem if the existing data directory > permissions > are 0700 to changing it to 0750. But it may not be a problem for the > scenarios, > where the existing data permissions  >=0750, to the upstream permissions. > Because

Re: pg_basebackup ignores the existing data directory permissions

2019-03-08 Thread Haribabu Kommi
On Fri, Mar 8, 2019 at 11:59 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > pg_basebackup copies the data directory permission mode from the > upstream server. But it doesn't copy the ownership. So if say the > upstream server allows group access and things are owned by >

Re: pg_basebackup ignores the existing data directory permissions

2019-03-08 Thread Peter Eisentraut
pg_basebackup copies the data directory permission mode from the upstream server. But it doesn't copy the ownership. So if say the upstream server allows group access and things are owned by postgres:postgres, and I want to make a copy for local development and make a backup into a directory

Re: pg_basebackup ignores the existing data directory permissions

2019-02-22 Thread Haribabu Kommi
On Wed, Feb 20, 2019 at 7:40 PM Magnus Hagander wrote: > > On Wed, Feb 20, 2019 at 5:17 AM Haribabu Kommi > wrote: > >> >> On Fri, Feb 15, 2019 at 10:15 AM Michael Paquier >> wrote: >> >>> On Thu, Feb 14, 2019 at 11:21:19PM +1100, Haribabu Kommi wrote: >>> > On Thu, Feb 14, 2019 at 8:57 PM

Re: pg_basebackup ignores the existing data directory permissions

2019-02-20 Thread Magnus Hagander
On Wed, Feb 20, 2019 at 5:17 AM Haribabu Kommi wrote: > > On Fri, Feb 15, 2019 at 10:15 AM Michael Paquier > wrote: > >> On Thu, Feb 14, 2019 at 11:21:19PM +1100, Haribabu Kommi wrote: >> > On Thu, Feb 14, 2019 at 8:57 PM Magnus Hagander >> wrote: >> >> I think it could be argued that neither

Re: pg_basebackup ignores the existing data directory permissions

2019-02-19 Thread Michael Paquier
On Wed, Feb 20, 2019 at 03:16:48PM +1100, Haribabu Kommi wrote: > Lack of complaints from the users, how about making this change in the HEAD? Fine by me. I would tend to patch pg_basebackup so as the target folder gets the correct umask instead of nuking the chmod call in initdb so I think that

Re: pg_basebackup ignores the existing data directory permissions

2019-02-19 Thread Haribabu Kommi
On Fri, Feb 15, 2019 at 10:15 AM Michael Paquier wrote: > On Thu, Feb 14, 2019 at 11:21:19PM +1100, Haribabu Kommi wrote: > > On Thu, Feb 14, 2019 at 8:57 PM Magnus Hagander > wrote: > >> I think it could be argued that neither initdb *or* pg_basebackup should > >> change the permissions on an

Re: pg_basebackup ignores the existing data directory permissions

2019-02-17 Thread Michael Paquier
On Fri, Feb 15, 2019 at 09:24:15AM +0900, Kyotaro HORIGUCHI wrote: > I disagree that pg_basebackup rejects directories other than > specific permissions, since it is just a binary backup tool, > which is not exclusive to making replication-standby. It ought to > be runnable and actually runnable

Re: pg_basebackup ignores the existing data directory permissions

2019-02-14 Thread Kyotaro HORIGUCHI
At Fri, 15 Feb 2019 08:15:24 +0900, Michael Paquier wrote in <20190214231524.gc2...@paquier.xyz> > On Thu, Feb 14, 2019 at 11:21:19PM +1100, Haribabu Kommi wrote: > > On Thu, Feb 14, 2019 at 8:57 PM Magnus Hagander wrote: > >> I think it could be argued that neither initdb *or* pg_basebackup

Re: pg_basebackup ignores the existing data directory permissions

2019-02-14 Thread Haribabu Kommi
On Thu, Feb 14, 2019 at 8:57 PM Magnus Hagander wrote: > On Thu, Feb 14, 2019 at 9:10 AM Michael Paquier > wrote: > >> On Thu, Feb 14, 2019 at 06:34:07PM +1100, Haribabu Kommi wrote: >> > we have an application that is used to create the data directory with >> >> Well, initdb would do that

Re: pg_basebackup ignores the existing data directory permissions

2019-02-14 Thread Magnus Hagander
On Thu, Feb 14, 2019 at 9:10 AM Michael Paquier wrote: > On Thu, Feb 14, 2019 at 06:34:07PM +1100, Haribabu Kommi wrote: > > we have an application that is used to create the data directory with > > Well, initdb would do that happily, so there is no actual any need to > do that to begin with.

Re: pg_basebackup ignores the existing data directory permissions

2019-02-14 Thread Michael Paquier
On Thu, Feb 14, 2019 at 06:34:07PM +1100, Haribabu Kommi wrote: > we have an application that is used to create the data directory with Well, initdb would do that happily, so there is no actual any need to do that to begin with. Anyway.. > owner access (0700), but with initdb group permissions

Re: pg_basebackup ignores the existing data directory permissions

2019-02-13 Thread Haribabu Kommi
On Thu, Feb 14, 2019 at 3:04 PM Michael Paquier wrote: > On Wed, Feb 13, 2019 at 06:42:36PM +1100, Haribabu Kommi wrote: > > This should back-patch till 11 where the group access is introduced. > > Because of lack of complaints, I agree with you that there is no need of > > further back-patch. >

Re: pg_basebackup ignores the existing data directory permissions

2019-02-13 Thread Michael Paquier
On Wed, Feb 13, 2019 at 06:42:36PM +1100, Haribabu Kommi wrote: > This should back-patch till 11 where the group access is introduced. > Because of lack of complaints, I agree with you that there is no need of > further back-patch. I am confused by the link with group access. The patch you are

Re: pg_basebackup ignores the existing data directory permissions

2019-02-12 Thread Haribabu Kommi
On Wed, Feb 13, 2019 at 12:42 PM Michael Paquier wrote: > On Tue, Feb 12, 2019 at 06:03:37PM +1100, Haribabu Kommi wrote: > > During the testing allow group access permissions on the standby database > > directory, one of my colleague found the issue, that pg_basebackup > > doesn't verify

pg_basebackup ignores the existing data directory permissions

2019-02-11 Thread Haribabu Kommi
Hi Hackers, During the testing allow group access permissions on the standby database directory, one of my colleague found the issue, that pg_basebackup doesn't verify whether the existing data directory has the required permissions or not? This issue is not related allow group access