Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-04-21 Thread Tom Lane
Robert Haas writes: > On Thu, Apr 21, 2022 at 1:53 PM Tom Lane wrote: >> 0001 attached is a revised patch that does it that way. This seems >> like a clearly better answer. >> 0002 contains the perhaps-slightly-more-controversial changes of >> changing the macro names and explicitly pinning no

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-04-21 Thread Robert Haas
On Thu, Apr 21, 2022 at 1:53 PM Tom Lane wrote: > Robert Haas writes: > > On Wed, Apr 20, 2022 at 4:56 PM Tom Lane wrote: > >> Having just had to bury my nose in renumber_oids.pl, I thought of a > >> different approach we could take to expose these OIDs to Catalog.pm. > >> That's to invent a

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-04-21 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 20, 2022 at 4:56 PM Tom Lane wrote: >> Having just had to bury my nose in renumber_oids.pl, I thought of a >> different approach we could take to expose these OIDs to Catalog.pm. >> That's to invent a new macro that Catalog.pm recognizes, and write >> something

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-04-20 Thread Robert Haas
On Wed, Apr 20, 2022 at 4:56 PM Tom Lane wrote: > Having just had to bury my nose in renumber_oids.pl, I thought of a > different approach we could take to expose these OIDs to Catalog.pm. > That's to invent a new macro that Catalog.pm recognizes, and write > something about like this in

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-04-20 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 20, 2022 at 2:34 PM Tom Lane wrote: >> The attached draft patch attempts to improve this situation. >> It reserves these OIDs, and creates the associated macros, through >> the normal BKI infrastructure by adding entries in pg_database.dat. >> We have to delete

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-04-20 Thread Robert Haas
On Wed, Apr 20, 2022 at 2:34 PM Tom Lane wrote: > The attached draft patch attempts to improve this situation. > It reserves these OIDs, and creates the associated macros, through > the normal BKI infrastructure by adding entries in pg_database.dat. > We have to delete those rows again during

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-04-20 Thread Tom Lane
Robert Haas writes: > On Sat, Jan 22, 2022 at 2:20 AM Shruthi Gowda wrote: >> Agree. In the latest patch, the template0 and postgres OIDs are fixed >> to unused manually assigned OIDs 4 and 5 respectively. These OIDs are >> no more listed as unused OIDs. > Thanks. Committed with a few more

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-03-22 Thread Robert Haas
On Mon, Mar 21, 2022 at 8:52 PM Andres Freund wrote: > I noticed this still has an open CF entry: > https://commitfest.postgresql.org/37/3296/ > I assume it can be marked as committed? Yeah, done now. But don't forget that we still need to do something on the "wrong fds used for refilenodes

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-03-21 Thread Andres Freund
On 2022-01-24 14:44:10 -0500, Robert Haas wrote: > On Sat, Jan 22, 2022 at 2:20 AM Shruthi Gowda wrote: > > Agree. In the latest patch, the template0 and postgres OIDs are fixed > > to unused manually assigned OIDs 4 and 5 respectively. These OIDs are > > no more listed as unused OIDs. > >

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-03-12 Thread Justin Pryzby
On Tue, Jan 25, 2022 at 10:19:53AM +0530, Shruthi Gowda wrote: > On Tue, Jan 25, 2022 at 1:14 AM Robert Haas wrote: > > On Sat, Jan 22, 2022 at 2:20 AM Shruthi Gowda wrote: > > > Agree. In the latest patch, the template0 and postgres OIDs are fixed > > > to unused manually assigned OIDs 4 and 5

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-24 Thread Shruthi Gowda
On Tue, Jan 25, 2022 at 1:14 AM Robert Haas wrote: > > On Sat, Jan 22, 2022 at 2:20 AM Shruthi Gowda wrote: > > Agree. In the latest patch, the template0 and postgres OIDs are fixed > > to unused manually assigned OIDs 4 and 5 respectively. These OIDs are > > no more listed as unused OIDs. > >

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-24 Thread Robert Haas
On Sat, Jan 22, 2022 at 2:20 AM Shruthi Gowda wrote: > Agree. In the latest patch, the template0 and postgres OIDs are fixed > to unused manually assigned OIDs 4 and 5 respectively. These OIDs are > no more listed as unused OIDs. Thanks. Committed with a few more cosmetic changes. -- Robert

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-24 Thread Bruce Momjian
On Sat, Jan 22, 2022 at 12:47:35PM +0530, Shruthi Gowda wrote: > On Sat, Jan 22, 2022 at 12:27 AM Tom Lane wrote: > > > > Robert Haas writes: > > > It seems to me that what this comment is saying is that OIDs in the > > > second and third categories are doled out by counters. Therefore, we > > >

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-21 Thread Shruthi Gowda
On Sat, Jan 22, 2022 at 12:17 AM Robert Haas wrote: > > On Fri, Jan 21, 2022 at 8:40 AM Shruthi Gowda wrote: > > From what I see in the code, template0 and postgres are the last > > things that get created in initdb phase. The system OIDs that get > > assigned to these DBs vary from release to

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-21 Thread Shruthi Gowda
On Sat, Jan 22, 2022 at 12:27 AM Tom Lane wrote: > > Robert Haas writes: > > It seems to me that what this comment is saying is that OIDs in the > > second and third categories are doled out by counters. Therefore, we > > can't know which of those OIDs will get used, or how many of them will > >

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-21 Thread Tom Lane
Robert Haas writes: > It seems to me that what this comment is saying is that OIDs in the > second and third categories are doled out by counters. Therefore, we > can't know which of those OIDs will get used, or how many of them will > get used, or which objects will get which OIDs. Therefore, I

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-21 Thread Robert Haas
On Fri, Jan 21, 2022 at 8:40 AM Shruthi Gowda wrote: > From what I see in the code, template0 and postgres are the last > things that get created in initdb phase. The system OIDs that get > assigned to these DBs vary from release to release. At present, the > system assigned OIDs of template0 and

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-21 Thread Shruthi Gowda
On Fri, Jan 21, 2022 at 1:08 AM Robert Haas wrote: > > On Thu, Jan 20, 2022 at 11:03 AM Shruthi Gowda wrote: > > It is not required for PostgresObjectId. The unused_oids script > > provides a list of unused oids in the manually-assignable OIDs range > > (1-). > > Well, so ... why are we not

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-20 Thread Robert Haas
On Thu, Jan 20, 2022 at 11:03 AM Shruthi Gowda wrote: > It is not required for PostgresObjectId. The unused_oids script > provides a list of unused oids in the manually-assignable OIDs range > (1-). Well, so ... why are we not treating the OIDs for these two databases the same? If there's a

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-20 Thread Shruthi Gowda
On Thu, Jan 20, 2022 at 7:57 PM Robert Haas wrote: > > On Thu, Jan 20, 2022 at 7:09 AM Shruthi Gowda wrote: > > > Here's an updated version in which I've reverted the changes to gram.y > > > and tried to improve the comments and documentation. Could you have a > > > look at implementing (2)

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-20 Thread Robert Haas
On Thu, Jan 20, 2022 at 7:09 AM Shruthi Gowda wrote: > > Here's an updated version in which I've reverted the changes to gram.y > > and tried to improve the comments and documentation. Could you have a > > look at implementing (2) above? > > Attached is the patch that implements comment (2).

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-20 Thread Shruthi Gowda
On Tue, Jan 18, 2022 at 2:34 AM Robert Haas wrote: > > On Mon, Jan 17, 2022 at 9:57 AM Shruthi Gowda wrote: > > I have rebased and generated the patches on top of PostgreSQL commit > > ID cf925936ecc031355cd56fbd392ec3180517a110. > > Kindly apply

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-17 Thread Shruthi Gowda
On Tue, Jan 18, 2022 at 12:35 AM Robert Haas wrote: > > On Tue, Dec 14, 2021 at 1:21 PM Shruthi Gowda wrote: > > Thanks, Robert for the updated version. I reviewed the changes and it > > looks fine. > > I also tested the patch. The patch works as expected. > > Thanks. > > > > - I adjusted the

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-17 Thread Robert Haas
On Mon, Jan 17, 2022 at 9:57 AM Shruthi Gowda wrote: > I have rebased and generated the patches on top of PostgreSQL commit > ID cf925936ecc031355cd56fbd392ec3180517a110. > Kindly apply v8-0001-pg_upgrade-Preserve-relfilenodes-and-tablespace-O.patch > first and then

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-17 Thread Robert Haas
On Tue, Dec 14, 2021 at 1:21 PM Shruthi Gowda wrote: > Thanks, Robert for the updated version. I reviewed the changes and it > looks fine. > I also tested the patch. The patch works as expected. Thanks. > > - I adjusted the function header comment for heap_create. Your > > proposed comment

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-17 Thread Shruthi Gowda
On Sat, Jan 15, 2022 at 11:17 AM Julien Rouhaud wrote: > > Hi, > > On Fri, Dec 17, 2021 at 01:03:06PM +0530, Shruthi Gowda wrote: > > > > I have updated the DBOID preserve patch to handle this case and > > generated the latest patch on top of your v7-001-preserve-relfilenode > > patch. > > The

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-14 Thread Julien Rouhaud
Hi, On Fri, Dec 17, 2021 at 01:03:06PM +0530, Shruthi Gowda wrote: > > I have updated the DBOID preserve patch to handle this case and > generated the latest patch on top of your v7-001-preserve-relfilenode > patch. The cfbot reports that the patch doesn't apply anymore:

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-12-16 Thread Shruthi Gowda
On Mon, Dec 13, 2021 at 8:43 PM Shruthi Gowda wrote: > > On Mon, Dec 6, 2021 at 11:25 PM Robert Haas wrote: > > > > On Sun, Dec 5, 2021 at 11:44 PM Sadhuprasad Patro wrote: > > > 3. > > > @@ -504,11 +525,15 @@ createdb(ParseState *pstate, const CreatedbStmt > > > *stmt) > > > */ > > >

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-12-15 Thread tushar
On 12/15/21 12:09 AM, tushar wrote: I spent much of today reviewing 0001. Here's an updated version, so far only lightly tested. Please check whether I've broken anything. Thanks Robert, I tested from v96/12/13/v14 -> v15( with patch) things are working fine i.e table /index relfilenode is

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-12-14 Thread tushar
On 12/14/21 2:35 AM, Robert Haas wrote: I spent much of today reviewing 0001. Here's an updated version, so far only lightly tested. Please check whether I've broken anything. Thanks Robert, I tested from v96/12/13/v14 -> v15( with patch) things are working fine i.e table /index relfilenode is

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-12-14 Thread Shruthi Gowda
On Tue, Dec 14, 2021 at 2:35 AM Robert Haas wrote: > > On Mon, Dec 13, 2021 at 9:40 AM Shruthi Gowda wrote: > > > I am reviewing another patch > > > "v5-0001-Preserve-relfilenode-and-tablespace-OID-in-pg_upg" as well > > > and will provide the comments soon if any... > > I spent much of today

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-12-13 Thread Robert Haas
On Mon, Dec 13, 2021 at 9:40 AM Shruthi Gowda wrote: > > I am reviewing another patch > > "v5-0001-Preserve-relfilenode-and-tablespace-OID-in-pg_upg" as well > > and will provide the comments soon if any... I spent much of today reviewing 0001. Here's an updated version, so far only lightly

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-12-13 Thread Shruthi Gowda
On Mon, Dec 6, 2021 at 11:25 PM Robert Haas wrote: > > On Sun, Dec 5, 2021 at 11:44 PM Sadhuprasad Patro wrote: > > 3. > > @@ -504,11 +525,15 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt) > > */ > > pg_database_rel = table_open(DatabaseRelationId, RowExclusiveLock); > > > > - do

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-12-13 Thread Shruthi Gowda
On Mon, Dec 6, 2021 at 10:14 AM Sadhuprasad Patro wrote: > > On Tue, Oct 26, 2021 at 6:55 PM Shruthi Gowda wrote: > > > > > > I have revised the patch w.r.t the way 'create_storage' is interpreted > > in heap_create() along with some minor changes to preserve the DBOID > > patch. > > > > Hi

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-12-06 Thread Robert Haas
On Sun, Dec 5, 2021 at 11:44 PM Sadhuprasad Patro wrote: > 1. > --- a/doc/src/sgml/ref/create_database.sgml > +++ b/doc/src/sgml/ref/create_database.sgml > @@ -31,7 +31,8 @@ CREATE DATABASE class="parameter">name > - [ IS_TEMPLATE [=] class="parameter">istemplate ] ] > + [

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-12-05 Thread Sadhuprasad Patro
On Tue, Oct 26, 2021 at 6:55 PM Shruthi Gowda wrote: > > > I have revised the patch w.r.t the way 'create_storage' is interpreted > in heap_create() along with some minor changes to preserve the DBOID > patch. > Hi Shruthi, I am reviewing the attached patches and providing a few comments here

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-10-26 Thread Shruthi Gowda
On Thu, Oct 7, 2021 at 7:33 PM Robert Haas wrote: > > On Thu, Oct 7, 2021 at 3:24 AM Shruthi Gowda wrote: > > Every other > > caller/flow passes false for 'create_storage' and we still need to > > create storage in heap_create() if relkind has storage. > > That seems surprising. I have revised

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-10-07 Thread Robert Haas
On Thu, Oct 7, 2021 at 3:24 AM Shruthi Gowda wrote: > Every other > caller/flow passes false for 'create_storage' and we still need to > create storage in heap_create() if relkind has storage. That seems surprising. -- Robert Haas EDB: http://www.enterprisedb.com

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-10-07 Thread Shruthi Gowda
On Thu, Oct 7, 2021 at 2:05 AM Robert Haas wrote: > > On Mon, Oct 4, 2021 at 12:44 PM Shruthi Gowda wrote: > > Thanks for the inputs, Robert. In the v4 patch, an unused OID (i.e, 4) > > is fixed for the template0 and the same is removed from unused oid > > list. > > > > In addition to the review

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-10-06 Thread Robert Haas
On Mon, Oct 4, 2021 at 12:44 PM Shruthi Gowda wrote: > Thanks for the inputs, Robert. In the v4 patch, an unused OID (i.e, 4) > is fixed for the template0 and the same is removed from unused oid > list. > > In addition to the review comment fixes, I have removed some code that > is no longer

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-10-04 Thread Shruthi Gowda
On Fri, Sep 24, 2021 at 12:44 AM Robert Haas wrote: > > On Wed, Sep 22, 2021 at 3:07 PM Shruthi Gowda wrote: > > > - The comment in binary_upgrade_set_pg_class_oids() is still not > > > accurate. You removed the sentence which says "Indexes cannot have > > > toast tables, so we need not make

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-09-23 Thread Robert Haas
On Wed, Sep 22, 2021 at 3:07 PM Shruthi Gowda wrote: > > - The comment in binary_upgrade_set_pg_class_oids() is still not > > accurate. You removed the sentence which says "Indexes cannot have > > toast tables, so we need not make this probe in the index code path" > > but the immediately

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-09-22 Thread Shruthi Gowda
On Tue, Aug 24, 2021 at 2:27 AM Robert Haas wrote: > It's pretty clear from the discussion, I think, that the database OID > one is going to need rework to be considered. > > Regarding the other one: > > - The comment in binary_upgrade_set_pg_class_oids() is still not > accurate. You removed the

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Bruce Momjian
On Thu, Aug 26, 2021 at 01:24:46PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Thu, Aug 26, 2021 at 01:03:54PM -0400, Stephen Frost wrote: > > > Yes, we're talking about either incremental (or perhaps differential) > > > backup where only the

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Bruce Momjian
On Thu, Aug 26, 2021 at 01:20:38PM -0400, Robert Haas wrote: > So I think this proposed change is in the safe direction. If > relfilenodes were currently preserved and we wanted to make them not > be preserved, then I think you would be quite right to say "whoa, > whoa, that could be a problem."

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Aug 26, 2021 at 01:03:54PM -0400, Stephen Frost wrote: > > Yes, we're talking about either incremental (or perhaps differential) > > backup where only the files which are actually different would be backed > > up. Just like with PG,

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Robert Haas
On Thu, Aug 26, 2021 at 12:51 PM Bruce Momjian wrote: > I just don't want to add requirements/complexity to pg_upgrade without > clearly stated reasons because future database changes will need to > honor this new preservation behavior. Well, I agree that it's good to have reasons clearly stated

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Bruce Momjian
On Thu, Aug 26, 2021 at 01:03:54PM -0400, Stephen Frost wrote: > Yes, we're talking about either incremental (or perhaps differential) > backup where only the files which are actually different would be backed > up. Just like with PG, I can't provide any complete guarantees that > we'd be able to

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Aug 26, 2021 at 12:34:56PM -0400, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Thu, Aug 26, 2021 at 11:36:51AM -0400, Stephen Frost wrote: > > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > > On

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Bruce Momjian
On Thu, Aug 26, 2021 at 12:37:19PM -0400, Robert Haas wrote: > On Thu, Aug 26, 2021 at 11:48 AM Bruce Momjian wrote: > > I am find to add it if it is minor, but I want to see the calculus of > > its value vs complexity, which I have not seen spelled out. > > I don't think it's going to be all

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Bruce Momjian
On Thu, Aug 26, 2021 at 12:34:56PM -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Thu, Aug 26, 2021 at 11:36:51AM -0400, Stephen Frost wrote: > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > On Thu, Aug 26, 2021 at 11:00:47AM -0400, Robert Haas wrote: > > > >

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Aug 26, 2021 at 11:39 AM Stephen Frost wrote: > > This looks like a pretty good analysis to me. As it relates to the > > question about allowing users to specify an OID, I'd be inclined to > > allow it but only for OIDs >64k.

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Robert Haas
On Thu, Aug 26, 2021 at 11:48 AM Bruce Momjian wrote: > I am find to add it if it is minor, but I want to see the calculus of > its value vs complexity, which I have not seen spelled out. I don't think it's going to be all that complicated, but we're going to have to wait until we have something

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Aug 26, 2021 at 11:36:51AM -0400, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Thu, Aug 26, 2021 at 11:00:47AM -0400, Robert Haas wrote: > > > > Anyone see a flaw in that analysis? > > > > > > I am

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Robert Haas
On Thu, Aug 26, 2021 at 11:39 AM Stephen Frost wrote: > This looks like a pretty good analysis to me. As it relates to the > question about allowing users to specify an OID, I'd be inclined to > allow it but only for OIDs >64k. We've certainly reserved things in the > past and I don't see any

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Bruce Momjian
On Thu, Aug 26, 2021 at 11:36:51AM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Thu, Aug 26, 2021 at 11:00:47AM -0400, Robert Haas wrote: > > > Anyone see a flaw in that analysis? > > > > I am still waiting to hear the purpose of this

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Bruce Momjian
On Thu, Aug 26, 2021 at 11:35:01AM -0400, Robert Haas wrote: > On Thu, Aug 26, 2021 at 11:24 AM Bruce Momjian wrote: > > On Thu, Aug 26, 2021 at 11:00:47AM -0400, Robert Haas wrote: > > > Anyone see a flaw in that analysis? > > > > I am still waiting to hear the purpose of this preservation. As

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Aug 17, 2021 at 2:50 PM Robert Haas wrote: > > > Less sure that this is a good idea, though. In particular, I do not > > > think that you can make it work in the face of > > > alter database template1 rename to oops; > >

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Aug 26, 2021 at 11:00:47AM -0400, Robert Haas wrote: > > Anyone see a flaw in that analysis? > > I am still waiting to hear the purpose of this preservation. As long as > you don't apply the patch, I guess I will just stop asking.

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Robert Haas
On Thu, Aug 26, 2021 at 11:24 AM Bruce Momjian wrote: > On Thu, Aug 26, 2021 at 11:00:47AM -0400, Robert Haas wrote: > > Anyone see a flaw in that analysis? > > I am still waiting to hear the purpose of this preservation. As long as > you don't apply the patch, I guess I will just stop asking.

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Bruce Momjian
On Thu, Aug 26, 2021 at 11:00:47AM -0400, Robert Haas wrote: > Anyone see a flaw in that analysis? I am still waiting to hear the purpose of this preservation. As long as you don't apply the patch, I guess I will just stop asking. -- Bruce Momjian https://momjian.us EDB

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-26 Thread Robert Haas
On Tue, Aug 17, 2021 at 2:50 PM Robert Haas wrote: > > Less sure that this is a good idea, though. In particular, I do not > > think that you can make it work in the face of > > alter database template1 rename to oops; > > create database template1; > > That is a really good

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Bruce Momjian
On Tue, Aug 24, 2021 at 02:34:26PM -0400, Robert Haas wrote: > On Tue, Aug 24, 2021 at 2:16 PM Bruce Momjian wrote: > > One other issue --- the more that pg_upgrade preserves, the more likely > > pg_upgrade will break when some internal changes happen in Postgres. > > Therefore, if you want

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Robert Haas
On Tue, Aug 24, 2021 at 2:16 PM Bruce Momjian wrote: > One other issue --- the more that pg_upgrade preserves, the more likely > pg_upgrade will break when some internal changes happen in Postgres. > Therefore, if you want pg_upgrade to preserve something, you have to > have a good reason ---

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Bruce Momjian
On Tue, Aug 24, 2021 at 12:43:20PM -0400, Bruce Momjian wrote: > Yes, it is a trade-off. If we had pg_upgrade create the new cluster, > the pg_upgrade instructions would be simpler, but pg_upgrade would be > more complex since it has to adjust _everything_ properly so pg_upgrade > works --- I

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Aug 24, 2021 at 12:43 PM Bruce Momjian wrote: > > Yes, it is a trade-off. If we had pg_upgrade create the new cluster, > > the pg_upgrade instructions would be simpler, but pg_upgrade would be > > more complex since it has to

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Robert Haas
On Tue, Aug 24, 2021 at 12:43 PM Bruce Momjian wrote: > Yes, it is a trade-off. If we had pg_upgrade create the new cluster, > the pg_upgrade instructions would be simpler, but pg_upgrade would be > more complex since it has to adjust _everything_ properly so pg_upgrade > works --- I never got

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Aug 23, 2021 at 5:12 PM Stephen Frost wrote: > > Regarding that ... I have to wonder just what promises we feel we've > > made when it comes to what a user is expected to be able to do with the > > new cluster *before* pg_upgrade

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Robert Haas
On Tue, Aug 24, 2021 at 12:04 PM Tom Lane wrote: > Per upthread discussion, it seems impractical to fully guarantee > that database OIDs match, which seems to mean that the whole premise > collapses. Like Bruce, I want to see a plausible use case justifying > any partial-guarantee scenario

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Bruce Momjian
On Tue, Aug 24, 2021 at 11:24:21AM -0400, Robert Haas wrote: > On Mon, Aug 23, 2021 at 5:12 PM Stephen Frost wrote: > > Regarding that ... I have to wonder just what promises we feel we've > > made when it comes to what a user is expected to be able to do with the > > new cluster *before*

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Bruce Momjian
On Tue, Aug 24, 2021 at 11:28:37AM -0400, Robert Haas wrote: > On Mon, Aug 23, 2021 at 8:29 PM Bruce Momjian wrote: > > I assume this patch is not going to be applied until there is an actual > > use case for preserving these values. > > My interpretation of the preceding discussion was that

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Bruce Momjian
On Tue, Aug 24, 2021 at 12:04:00PM -0400, Tom Lane wrote: > Robert Haas writes: > > On Mon, Aug 23, 2021 at 8:29 PM Bruce Momjian wrote: > >> I assume this patch is not going to be applied until there is an actual > >> use case for preserving these values. > > > ... > > > That being said, if

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Tom Lane
Robert Haas writes: > On Mon, Aug 23, 2021 at 8:29 PM Bruce Momjian wrote: >> I assume this patch is not going to be applied until there is an actual >> use case for preserving these values. > ... > That being said, if you or somebody else thinks that this is a bad > idea or that the reasons

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Robert Haas
On Mon, Aug 23, 2021 at 8:29 PM Bruce Momjian wrote: > I assume this patch is not going to be applied until there is an actual > use case for preserving these values. My interpretation of the preceding discussion was that several people thought this change was a good idea regardless of whether

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Robert Haas
On Mon, Aug 23, 2021 at 5:12 PM Stephen Frost wrote: > Regarding that ... I have to wonder just what promises we feel we've > made when it comes to what a user is expected to be able to do with the > new cluster *before* pg_upgrade is run on it. For my part, I sure feel > like it's "nothing", in

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-24 Thread Shruthi Gowda
On Tue, Aug 24, 2021 at 5:59 AM Bruce Momjian wrote: > > On Mon, Aug 23, 2021 at 04:57:31PM -0400, Robert Haas wrote: > > On Fri, Aug 20, 2021 at 1:36 PM Shruthi Gowda wrote: > > > Thanks Robert for your comments. > > > I have split the patch into two portions. One that handles DB OID and > > >

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-23 Thread Bruce Momjian
On Mon, Aug 23, 2021 at 04:57:31PM -0400, Robert Haas wrote: > On Fri, Aug 20, 2021 at 1:36 PM Shruthi Gowda wrote: > > Thanks Robert for your comments. > > I have split the patch into two portions. One that handles DB OID and > > the other that > > handles tablespace OID and relfilenode OID. >

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-23 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Aug 20, 2021 at 1:36 PM Shruthi Gowda wrote: > > Thanks Robert for your comments. > > I have split the patch into two portions. One that handles DB OID and > > the other that > > handles tablespace OID and relfilenode OID. > >

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-23 Thread Robert Haas
On Fri, Aug 20, 2021 at 1:36 PM Shruthi Gowda wrote: > Thanks Robert for your comments. > I have split the patch into two portions. One that handles DB OID and > the other that > handles tablespace OID and relfilenode OID. It's pretty clear from the discussion, I think, that the database OID one

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-20 Thread Shruthi Gowda
> The rest of this email will be detailed review comments on the patch > as presented, and thus probably only interesting to someone actually > working on the patch. Feel free to skip if that's not you. > > - I suggest splitting the patch into one portion that deals with > database OID and another

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-17 Thread Robert Haas
On Tue, Aug 17, 2021 at 1:54 PM Tom Lane wrote: > Right. If pg_upgrade explicitly ignores template0 then its OID > need not be stable ... at least, not unless there's a chance it > could conflict with some other database OID, which would become > a live possibility if we let users get at "WITH

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-17 Thread Bruce Momjian
On Tue, Aug 17, 2021 at 11:56:30AM -0400, Robert Haas wrote: > On Wed, Aug 11, 2021 at 3:41 AM Shruthi Gowda wrote: > > I have fixed all the issues and now the patch is working as expected. > > Hi, > > I'm changing the subject line since the patch does something which was > discussed on that

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-17 Thread Shruthi Gowda
On Tue, Aug 17, 2021 at 11:07 PM Robert Haas wrote: > > On Tue, Aug 17, 2021 at 12:42 PM Tom Lane wrote: > > Actually though ... I've not read the patch, but what does it do about > > the fact that the postgres and template0 DBs do not have stable OIDs? > > I cannot imagine any way to force

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-17 Thread Tom Lane
I wrote: > Robert Haas writes: >> The only hard requirement for this feature is if we >> use the database OID for some kind of encryption or integrity checking >> or checksum type feature. > It's fairly unclear to me why that is so important as to justify the > amount of klugery that this line

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-17 Thread Tom Lane
Robert Haas writes: > I wasn't able to properly understand that comment, and to be honest > I'm not sure I precisely understand your concern either. I don't quite > see why the template0 database matters. I think that database isn't > going to be dumped, or restored, so as far as pg_upgrade is

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-17 Thread Robert Haas
On Tue, Aug 17, 2021 at 12:42 PM Tom Lane wrote: > Actually though ... I've not read the patch, but what does it do about > the fact that the postgres and template0 DBs do not have stable OIDs? > I cannot imagine any way to force those to match across PG versions > that would not be an

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-17 Thread Tom Lane
Stephen Frost writes: > Also agreed on this, though I wonder- do we actually need to explicitly > make CREATE DATABASE q WITH OID = 1234; only work during binary upgrade > mode in the backend? That strikes me as perhaps doing more work than we > really need to while also preventing something

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-17 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > To me, adding a separate TOC entry for a thing that is not really a > > separate object seems like a scary hack that might come back to bite > > us. Unfortunately, I don't know enough about pg_dump to say exactly > > how

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-17 Thread Tom Lane
Robert Haas writes: > To me, adding a separate TOC entry for a thing that is not really a > separate object seems like a scary hack that might come back to bite > us. Unfortunately, I don't know enough about pg_dump to say exactly > how it might come back to bite us, which leaves wide open the >

preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-17 Thread Robert Haas
On Wed, Aug 11, 2021 at 3:41 AM Shruthi Gowda wrote: > I have fixed all the issues and now the patch is working as expected. Hi, I'm changing the subject line since the patch does something which was discussed on that thread but isn't really related to the old email subject. In general, I think