Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2017-10-07 Thread Mark Kirkwood
On 26/09/17 20:44, Mark Kirkwood wrote: $ pg_basebackup -D . WARNING:  could not read symbolic link "pg_tblspc/space1": Invalid argument pg_basebackup: directory "/data0/pgdata/11/pg_tblspc/space1" exists but is not empty pg_basebackup: removing contents of data directory "." Err -

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2017-09-26 Thread Mark Kirkwood
On 29/04/15 09:35, Bruce Momjian wrote: On Fri, Apr 24, 2015 at 01:05:03PM -0400, Bruce Momjian wrote: This way, both pg_dump and pg_upgrade will issue warnings, though, of course, those warnings can be ignored. I am hopeful these two warnings will be sufficient and we will not need make

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-28 Thread Bruce Momjian
On Fri, Apr 24, 2015 at 01:05:03PM -0400, Bruce Momjian wrote: This way, both pg_dump and pg_upgrade will issue warnings, though, of course, those warnings can be ignored. I am hopeful these two warnings will be sufficient and we will not need make these errors, with the possible

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-24 Thread Robert Haas
On Thu, Apr 23, 2015 at 4:30 PM, Andres Freund and...@anarazel.de wrote: On 2015-04-23 16:26:09 -0400, Robert Haas wrote: But pg_upgrade automates all that, so you can't use pg_upgrade in that case. If we add a GUC as I suggested, you can still use pg_upgrade. But we also have to live with

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-24 Thread Bruce Momjian
On Wed, Apr 22, 2015 at 10:41:02PM -0400, Bruce Momjian wrote: josh=# create tablespace tbl2 location '/home/josh/pg94/data/pg_xlog/'; CREATE TABLESPACE It really seems like we ought to block *THAT*. Of course, if we block tablespace creation in PGDATA generally, then that's covered.

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Bruce Momjian
On Thu, Apr 23, 2015 at 09:13:52AM -0400, Robert Haas wrote: On Wed, Apr 22, 2015 at 10:41 PM, Bruce Momjian br...@momjian.us wrote: What is a real problem is that we don't block creating tablespaces anywhere at all, including in obviously problematic places like the transaction log

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Bruce Momjian
On Thu, Apr 23, 2015 at 05:05:14PM +0200, Andres Freund wrote: On 2015-04-23 11:00:43 -0400, Bruce Momjian wrote: On Thu, Apr 23, 2015 at 09:13:52AM -0400, Robert Haas wrote: I think this is a good thing to do, but I sure wish we could go further and block it completely. That may require

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Jim Nasby
On 4/22/15 9:41 PM, Bruce Momjian wrote: The case this doesn't catch is referencing a symbolic link that points to the same directory. We can't make it an error so people can use pg_upgrade these setups. Couldn't we make it an ERROR unless IsBinaryUpgrade? -- Jim Nasby, Data Architect, Blue

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Andres Freund
On 2015-04-23 11:00:43 -0400, Bruce Momjian wrote: On Thu, Apr 23, 2015 at 09:13:52AM -0400, Robert Haas wrote: I think this is a good thing to do, but I sure wish we could go further and block it completely. That may require more thought than we have time to put in at this stage of the

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Andres Freund
On 2015-04-23 15:17:55 -0500, Jim Nasby wrote: Yes, but only after creating a brand new cluster from scratch, which would then disallow them from putting tablespaces in $PGDATA. pg_dumpall output includes tablespaces. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread David Steele
On 4/23/15 4:30 PM, Andres Freund wrote: On 2015-04-23 16:26:09 -0400, Robert Haas wrote: But pg_upgrade automates all that, so you can't use pg_upgrade in that case. If we add a GUC as I suggested, you can still use pg_upgrade. But we also have to live with data directories being in a shit

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Robert Haas
On Thu, Apr 23, 2015 at 11:00 AM, Bruce Momjian br...@momjian.us wrote: I have developed the attached patch to warn about creating tablespaces inside the data directory. The case this doesn't catch is referencing a symbolic link that points to the same directory. We can't make it an

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Andres Freund
On 2015-04-23 15:46:20 -0400, Robert Haas wrote: Well, we've made backward-incompatible changes before. Not to this specific thing, but in general. I don't think there's anything preventing us from doing so here, except that we don't want to annoy too many users. I think the number of users

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Jim Nasby
On 4/23/15 11:01 AM, Andres Freund wrote: On April 23, 2015 6:12:05 PM GMT+03:00, Jim Nasby jim.na...@bluetreble.com wrote: On 4/22/15 9:41 PM, Bruce Momjian wrote: The case this doesn't catch is referencing a symbolic link that points to the same directory. We can't make it an error so

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Andres Freund
On 2015-04-23 16:26:09 -0400, Robert Haas wrote: But pg_upgrade automates all that, so you can't use pg_upgrade in that case. If we add a GUC as I suggested, you can still use pg_upgrade. But we also have to live with data directories being in a shit state forever onward. We won't really be

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Robert Haas
On Thu, Apr 23, 2015 at 3:57 PM, Andres Freund and...@anarazel.de wrote: On 2015-04-23 15:46:20 -0400, Robert Haas wrote: Well, we've made backward-incompatible changes before. Not to this specific thing, but in general. I don't think there's anything preventing us from doing so here, except

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Robert Haas
On Wed, Apr 22, 2015 at 10:41 PM, Bruce Momjian br...@momjian.us wrote: What is a real problem is that we don't block creating tablespaces anywhere at all, including in obviously problematic places like the transaction log directory: josh=# create tablespace tbl2 location

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-22 Thread Bruce Momjian
On Fri, Jan 30, 2015 at 01:26:22PM -0800, Josh Berkus wrote: Robert, Stephen, etc.: Apparently you can create a tablespace in the tablespace directory: josh=# create tablespace tbl location '/home/josh/pg94/data/pg_tblspc/'; CREATE TABLESPACE josh=# create table test_tbl ( test text )

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-31 Thread Marc Mamin
it is just as likely they simply are not aware of the downsides and the only reason they put it is $PGDATA is that it seemed like a logical place to put a directory that is intended to hold database data. Yes, this is the reason why we got in this issue. The name PGDATA is misleading. The

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Bruce Momjian
On Fri, Jan 30, 2015 at 11:12:43AM -0500, Robert Haas wrote: I think everyone who has read this mailing list for a while is probably already aware of this problem. When you create a tablespace somewhere inside the data directory, weird things happen. If you pg_upgrade and then incautiously

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Joshua D. Drake
On 01/30/2015 08:19 AM, Bruce Momjian wrote: On Fri, Jan 30, 2015 at 11:12:43AM -0500, Robert Haas wrote: I think everyone who has read this mailing list for a while is probably already aware of this problem. When you create a tablespace somewhere inside the data directory, weird things

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Given all this, it seems like a good idea to at least give a warning if somebody tries to create a tablespace instead the data directory. A warning seems like a good idea. I actually thought we *did* prevent it.. Arguably, we should prohibit it

[HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Robert Haas
I think everyone who has read this mailing list for a while is probably already aware of this problem. When you create a tablespace somewhere inside the data directory, weird things happen. If you pg_upgrade and then incautiously run the delete_old_cluster.sh script thus created, you will blow

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Robert Haas
On Fri, Jan 30, 2015 at 11:43 AM, Joshua D. Drake j...@commandprompt.com wrote: I mean yes a warning is good but it is after the fact, the tablespace is already created. We know that tablespaces in $PGDATA are a bad idea, why not protect the user? Please go back and read the discussion of

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread David Steele
On 1/30/15 11:43 AM, Joshua D. Drake wrote: On 01/30/2015 08:19 AM, Bruce Momjian wrote: On Fri, Jan 30, 2015 at 11:12:43AM -0500, Robert Haas wrote: I think everyone who has read this mailing list for a while is probably already aware of this problem. When you create a tablespace somewhere

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Josh Berkus
Robert, Stephen, etc.: Apparently you can create a tablespace in the tablespace directory: josh=# create tablespace tbl location '/home/josh/pg94/data/pg_tblspc/'; CREATE TABLESPACE josh=# create table test_tbl ( test text ) tablespace tbl; CREATE TABLE josh=# \q

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread David G Johnston
Robert Haas wrote Arguably, we should prohibit it altogether, but there are obviously people that want to do it, and there could even be somewhat valid reasons for that, Lots of hand-waving here and it is just as likely they simply are not aware of the downsides and the only reason they put

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Josh Berkus
On 01/30/2015 09:19 AM, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: Given all this, it seems like a good idea to at least give a warning if somebody tries to create a tablespace instead the data directory. A warning seems like a good idea. I actually thought we *did*