Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-07 Thread Simon Riggs
On Fri, 7 Jan 2022 at 16:09, Justin Pryzby wrote: > > On Fri, Jan 07, 2022 at 03:53:51PM +, Finnerty, Jim wrote: > > I'd still like a plan to retire the "double xmax" representation > > eventually. Previously I suggested that this could be done as a > > post-process, before upgrade is

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-07 Thread Bruce Momjian
On Fri, Jan 7, 2022 at 03:53:51PM +, Finnerty, Jim wrote: > Re: The "prepare" approach was the first tried. > https://github.com/postgrespro/pg_pageprep But it appears to be > very difficult and unreliable. After investing many months into > pg_pageprep, "double xmax" approach appears to be

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-07 Thread Peter Eisentraut
On 07.01.22 06:18, Fujii Masao wrote: On 2022/01/06 19:24, Simon Riggs wrote: On Thu, 30 Dec 2021 at 13:19, Maxim Orlov wrote: Your opinions are very much welcome! This is a review of the Int64 options patch, "v6-0001-Add-64-bit-GUCs-for-xids.patch" Do we really need to support both

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-07 Thread Justin Pryzby
On Fri, Jan 07, 2022 at 03:53:51PM +, Finnerty, Jim wrote: > I'd still like a plan to retire the "double xmax" representation eventually. > Previously I suggested that this could be done as a post-process, before > upgrade is complete, but that could potentially make upgrade very slow. >

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-07 Thread Finnerty, Jim
Re:The "prepare" approach was the first tried. https://github.com/postgrespro/pg_pageprep But it appears to be very difficult and unreliable. After investing many months into pg_pageprep, "double xmax" approach appears to be very fast to implement and reliable. I'd still like

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-07 Thread Finnerty, Jim
Re: clog page numbers, as returned by TransactionIdToPage - int pageno = TransactionIdToPage(xid); /* get page of parent */ + int64 pageno = TransactionIdToPage(xid); /* get page of parent */ ... - int pageno =

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Fujii Masao
On 2022/01/06 19:24, Simon Riggs wrote: On Thu, 30 Dec 2021 at 13:19, Maxim Orlov wrote: Your opinions are very much welcome! This is a review of the Int64 options patch, "v6-0001-Add-64-bit-GUCs-for-xids.patch" Do we really need to support both int32 and int64 options? Isn't it enough

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Alexander Korotkov
,On Thu, Jan 6, 2022 at 4:15 PM Finnerty, Jim wrote: > (Maxim) Re: -- If after upgrade page has no free space for special data, > tuples are >converted to "double xmax" format: xmin became virtual >FrozenTransactionId, xmax occupies the whole 64bit. >Page converted to new

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Peter Geoghegan
On Tue, Jan 4, 2022 at 9:40 PM Fujii Masao wrote: > Could you tell me what happens if new tuple with XID larger than xid_base + > 0x is inserted into the page? Such new tuple is not allowed to be > inserted into that page? I fear that this patch will have many bugs along these lines.

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Finnerty, Jim
Re:Most software has a one-stage upgrade model. What you propose would have us install 2 things, with a step in-between, which makes it harder to manage. The intended benefit would be that the code doesn't need to handle the possibility of 2 different XID representations for the

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Simon Riggs
On Thu, 6 Jan 2022 at 13:15, Finnerty, Jim wrote: > > (Maxim) Re: -- If after upgrade page has no free space for special data, > tuples are >converted to "double xmax" format: xmin became virtual >FrozenTransactionId, xmax occupies the whole 64bit. >Page converted to new

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Finnerty, Jim
(Maxim) Re: -- If after upgrade page has no free space for special data, tuples are converted to "double xmax" format: xmin became virtual FrozenTransactionId, xmax occupies the whole 64bit. Page converted to new format when vacuum frees enough space. A better way would be

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Simon Riggs
On Thu, 30 Dec 2021 at 13:19, Maxim Orlov wrote: > Your opinions are very much welcome! This is a review of the Int64 options patch, "v6-0001-Add-64-bit-GUCs-for-xids.patch" Applies cleanly, with some fuzz, compiles cleanly and passes make check. Patch eyeballs OK, no obvious defects. Tested

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-05 Thread Alexander Korotkov
Hi! On Thu, Jan 6, 2022 at 3:02 AM Bruce Momjian wrote: > > On Thu, Dec 30, 2021 at 03:15:16PM +0300, Maxim Orlov wrote: > > PFA updated working patch v6 for PG15 development cycle. > > It is based on a patch by Alexander Korotkov version 5 [5] with a few fixes, > > refactoring and was rebased

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-05 Thread Bruce Momjian
On Wed, Jan 5, 2022 at 06:12:26PM -0600, Justin Pryzby wrote: > On Wed, Jan 05, 2022 at 06:51:37PM -0500, Bruce Momjian wrote: > > On Tue, Jan 4, 2022 at 10:22:50PM +, Finnerty, Jim wrote: > > > I'm concerned about the maintainability impact of having 2 new > > > on-disk page formats. It's

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-05 Thread Justin Pryzby
On Wed, Jan 05, 2022 at 06:51:37PM -0500, Bruce Momjian wrote: > On Tue, Jan 4, 2022 at 10:22:50PM +, Finnerty, Jim wrote: > > I'm concerned about the maintainability impact of having 2 new > > on-disk page formats. It's already complex enough with XIDs and > > multixact-XIDs. > > > > If the

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-05 Thread Bruce Momjian
On Tue, Jan 4, 2022 at 05:49:07PM +, Finnerty, Jim wrote: > Hi Maxim, > I’m glad to see that you’re trying to carry the 64-bit XID work forward. > I > had not noticed that my earlier patch (also derived from Alexander Kortkov’s > patch) was responded to back in September. Perhaps we

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-05 Thread Bruce Momjian
On Thu, Dec 30, 2021 at 03:15:16PM +0300, Maxim Orlov wrote: > PFA updated working patch v6 for PG15 development cycle. > It is based on a patch by Alexander Korotkov version 5 [5] with a few fixes, > refactoring and was rebased to PG15. > > Main changes: > - Change TransactionId to 64bit > -

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-05 Thread Bruce Momjian
On Tue, Jan 4, 2022 at 10:22:50PM +, Finnerty, Jim wrote: > I'm concerned about the maintainability impact of having 2 new > on-disk page formats. It's already complex enough with XIDs and > multixact-XIDs. > > If the lack of space for the two epochs in the special data area is > a problem

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-05 Thread Simon Riggs
On Thu, 30 Dec 2021 at 13:19, Maxim Orlov wrote: > > Hi, hackers! > > Long time wraparound was a really big pain for highly loaded systems. One > source of performance degradation is the need to vacuum before every > wraparound. > And there were several proposals to make XIDs 64-bit like [1],

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-04 Thread Fujii Masao
On 2021/12/30 21:15, Maxim Orlov wrote: Hi, hackers! Long time wraparound was a really big pain for highly loaded systems. One source of performance degradation is the need to vacuum before every wraparound. And there were several proposals to make XIDs 64-bit like [1], [2], [3] and [4] to

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-04 Thread Finnerty, Jim
On 1/4/22, 2:35 PM, "Stephen Frost" wrote: >> >>Not saying that I've got any idea how to fix that case offhand, and we >>don't really support such a thing today as the server would just stop >>instead, ... >> Perhaps that's a >> worthwhile tradeoff for being able to generally

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-04 Thread Stephen Frost
Greetings, * Maxim Orlov (orlo...@gmail.com) wrote: > Long time wraparound was a really big pain for highly loaded systems. One > source of performance degradation is the need to vacuum before every > wraparound. > And there were several proposals to make XIDs 64-bit like [1], [2], [3] and > [4]

<    1   2   3