Re: Add 64-bit XIDs into PostgreSQL 15

2025-06-11 Thread Yura Sokolov
11.06.2025 09:00, Evgeny Voropaev wrote: > 2) About repairing fragmentation. > > The original approach implemented in PG18 assumes that fragmentation > occurs during every `prune_freeze` operation. It happens because the > logic of the "redo"-function `heap_xlog_prune_freeze` assumes that > fra

Re: Add 64-bit XIDs into PostgreSQL 15

2025-06-10 Thread Evgeny Voropaev
> What exactly would break if we did perform `heap_page_prune_execute` > with NO fragmentation repair during `heap_page_prepare_for_xid`? Correction into the last question: What exactly would break if we did invoke `heap_page_prune_execute` with `repairFragmentation=true` during performing of `

Re: Add 64-bit XIDs into PostgreSQL 15

2025-06-10 Thread Evgeny Voropaev
Glad to see the activity around the full version of xid64! 1) About extra freezing in `freeze_single_heap_page`. At xid64v62 `freeze_single_heap_page` performs freezing via a call to `heap_page_prune_and_freeze`. Why, then, does `freeze_single_heap_page` perform freezing a second time? ``` f

Re: Add 64-bit XIDs into PostgreSQL 15

2024-12-17 Thread wenhui qiu
Hi Evgeny > we are causing the target solution to become staled I think the community is not positive enough about xid64,There is also a lot of resistance, such as the wal log becomes larger >I also hope you and the community will support attempts to retain the > whole xid64 solution in the con

Re: Add 64-bit XIDs into PostgreSQL 15

2024-12-17 Thread Evgeny Voropaev
Hi Evgeny      xid64 path split several threads ,The current one should be this:(https://www.postgresql.org/message-id/flat/CACG=ezawg7_nt-8ey4akv2w9lculthhknwcawmbgeetnjrj...@mail.gmail.com) ,We can do some tests on path so that  can merge earlier Thanks Hello Wenhui! Thank you for pointing

Re: Add 64-bit XIDs into PostgreSQL 15

2024-12-16 Thread wenhui qiu
Hi Evgeny xid64 path split several threads ,The current one should be this:( https://www.postgresql.org/message-id/flat/CACG=ezawg7_nt-8ey4akv2w9lculthhknwcawmbgeetnjrj...@mail.gmail.com) ,We can do some tests on path so that can merge earlier Thanks On Tue, Dec 10, 2024 at 7:47 PM Evgeny

Re: Add 64-bit XIDs into PostgreSQL 15

2024-09-12 Thread Aleksander Alekseev
Hi, > Agree. > > To me it seems like we didn't reach a consensus regarding switching to > 64-bit XIDs. Given that and the fact that the patchset is rather > difficult to rebase (and review) I suggest focusing on something we > reached a consensus for. I'm going to close a CF entry for this > parti

Re: Add 64-bit XIDs into PostgreSQL 15

2024-09-12 Thread Aleksander Alekseev
Michael, Maxim, > Apparently, the original thread will inevitably disintegrate into many > separate ones. > For me, looks like some kind of road map. One for 64-bit GUCs, another one to > remove > short file names from SLRUs and, to make things more complicated, [1] for cf > entry [0], > to get

Re: Add 64-bit XIDs into PostgreSQL 15

2024-09-12 Thread Maxim Orlov
Apparently, the original thread will inevitably disintegrate into many separate ones. For me, looks like some kind of road map. One for 64-bit GUCs, another one to remove short file names from SLRUs and, to make things more complicated, [1] for cf entry [0], to get rid of MultiXactOffset wraparound

Re: Add 64-bit XIDs into PostgreSQL 15

2024-09-11 Thread Michael Paquier
On Thu, Jul 25, 2024 at 02:09:10PM +0300, Heikki Linnakangas wrote: > On 25/07/2024 13:19, Peter Eisentraut wrote: >> Conversely, if there is still some threshold (not disaster, but >> efficiency or something else), would it still be useful to keep these >> settings well below 2^31?  In which case,

Re: Add 64-bit XIDs into PostgreSQL 15

2024-07-25 Thread wenhui qiu
Maybe a setting similar to max_wal_size could be better for that? +1 Thanks Peter Eisentraut 于2024年7月25日周四 21:31写道: > On 25.07.24 13:09, Heikki Linnakangas wrote: > >> However, if there is no more disaster threshold at 2^31, what is the > >> guidance for setting these? Or more radically, why e

Re: Add 64-bit XIDs into PostgreSQL 15

2024-07-25 Thread Peter Eisentraut
On 25.07.24 13:09, Heikki Linnakangas wrote: However, if there is no more disaster threshold at 2^31, what is the guidance for setting these?  Or more radically, why even run transaction-count-based vacuum at all? To allow the CLOG to be truncated. There's no disaster anymore, but without fre

Re: Add 64-bit XIDs into PostgreSQL 15

2024-07-25 Thread Chris Travers
On Thu, Jul 25, 2024 at 5:19 PM Peter Eisentraut wrote: > On 23.07.24 11:13, Aleksander Alekseev wrote: > >> Here is the fix. It can be tested like this: > >> [...] > > > > PFA the rebased patchset. > > I'm wondering about the 64-bit GUCs. > > At first, it makes sense that if there are settings t

Re: Add 64-bit XIDs into PostgreSQL 15

2024-07-25 Thread Heikki Linnakangas
On 25/07/2024 13:19, Peter Eisentraut wrote: I'm wondering about the 64-bit GUCs. At first, it makes sense that if there are settings that are counted in terms of transactions, and transaction numbers are 64-bit integers, then those settings should accept 64-bit integers. But what is the pur

Re: Add 64-bit XIDs into PostgreSQL 15

2024-07-25 Thread Peter Eisentraut
On 23.07.24 11:13, Aleksander Alekseev wrote: Here is the fix. It can be tested like this: [...] PFA the rebased patchset. I'm wondering about the 64-bit GUCs. At first, it makes sense that if there are settings that are counted in terms of transactions, and transaction numbers are 64-bit i

Re: Add 64-bit XIDs into PostgreSQL 15

2024-01-18 Thread Shubham Khanna
On Wed, Dec 13, 2023 at 5:56 PM Maxim Orlov wrote: > > Hi! > > Just to keep this thread up to date, here's a new version after recent > changes in SLRU. > I'm also change order of the patches in the set, to make adding initdb MOX > options after the > "core 64 xid" patch, since MOX patch is unli

Re: Add 64-bit XIDs into PostgreSQL 15

2023-12-15 Thread wenhui qiu
Hi Pavel Borisov Many thanks Best whish Pavel Borisov 于2023年12月15日周五 17:13写道: > Hi, Wenhui! > > On Fri, 15 Dec 2023 at 05:52, wenhui qiu wrote: > >> Hi Maxim Orlov >> Good news,xid64 has achieved a successful first phase,I tried to >> change the path status (https://commitfest.postgres

Re: Add 64-bit XIDs into PostgreSQL 15

2023-12-15 Thread Pavel Borisov
Hi, Wenhui! On Fri, 15 Dec 2023 at 05:52, wenhui qiu wrote: > Hi Maxim Orlov > Good news,xid64 has achieved a successful first phase,I tried to > change the path status (https://commitfest.postgresql.org/43/3594/) ,But > it seems incorrect > > Maxim Orlov 于2023年12月13日周三 20:26写道: > >> Hi! >>

Re: Add 64-bit XIDs into PostgreSQL 15

2023-12-14 Thread wenhui qiu
Hi Maxim Orlov Good news,xid64 has achieved a successful first phase,I tried to change the path status (https://commitfest.postgresql.org/43/3594/) ,But it seems incorrect Maxim Orlov 于2023年12月13日周三 20:26写道: > Hi! > > Just to keep this thread up to date, here's a new version after recent > c

Re: Add 64-bit XIDs into PostgreSQL 15

2023-07-04 Thread Aleksander Alekseev
Hi, > This patch hasn't applied in quite some time, and the thread has moved to > discussing higher lever items rather than the suggested patch, so I'm closing > this as Returned with Feedback. Please feel free to resubmit when there is > renewed interest and a concensus on how/what to proceed wi

Re: Add 64-bit XIDs into PostgreSQL 15

2023-07-04 Thread Daniel Gustafsson
This patch hasn't applied in quite some time, and the thread has moved to discussing higher lever items rather than the suggested patch, so I'm closing this as Returned with Feedback. Please feel free to resubmit when there is renewed interest and a concensus on how/what to proceed with. -- Danie

Re: Add 64-bit XIDs into PostgreSQL 15

2023-01-10 Thread Aleksander Alekseev
Hi Maxim, > Anyway. Let's discuss on-disk page format, shall we? Here are my two cents. > AFAICS, we have a following options: > [...] > 2. Put special in every page where base for XIDs are stored. This is what we > have done in the current patch set. The approach of using special space IMO is

Re: Add 64-bit XIDs into PostgreSQL 15

2022-12-30 Thread adherent postgres
Konstantin Knizhnik ; Nikita Glukhov ; Yura Sokolov ; Simon Riggs 主题: Re: Add 64-bit XIDs into PostgreSQL 15 Hi! I want to make a quick summary here. 1. An overall consensus has been reached: we shall focus on committing SLRU changes first. 2. I've created an appropriate patch set he

Re: Add 64-bit XIDs into PostgreSQL 15

2022-12-28 Thread Maxim Orlov
Hi! I want to make a quick summary here. 1. An overall consensus has been reached: we shall focus on committing SLRU changes first. 2. I've created an appropriate patch set here [0]. 3. How [0] is waiting for a review. As always, all opinions will be welcome. 4. While discussing error/warning mes

Re: Add 64-bit XIDs into PostgreSQL 15

2022-12-09 Thread Maxim Orlov
On Fri, 9 Dec 2022 at 16:54, adherent postgres < adherent_postg...@hotmail.com> wrote: > Hi Aleksander Alekseev > I think the xids 32bit transformation project has been dragged on for too > long. Huawei's openGauss referenced this patch to implement xids 64bit, and > Postgrespro also implemented

Re: Add 64-bit XIDs into PostgreSQL 15

2022-12-09 Thread Pavel Borisov
Hi, Adherent! On Fri, 9 Dec 2022 at 17:54, adherent postgres wrote: > > Hi Aleksander Alekseev > I think the xids 32bit transformation project has been dragged on for too > long. Huawei's openGauss referenced this patch to implement xids 64bit, and > Postgrespro also implemented xids 64bit, wh

Re: Add 64-bit XIDs into PostgreSQL 15

2022-12-09 Thread Maxim Orlov
> So I'd vote for an evolutionary approach and give my +1 for > undertaking efforts to first committing [1] to 16. > > [1]: > https://www.postgresql.org/message-id/CAFiTN-uudj2PY8GsUzFtLYFpBoq_rKegW3On_8ZHdxB1mVv3-A%40mail.gmail.com > > Kind regards, > Pavel Borisov, > Supabase. > +1 Totally suppo

Re: Add 64-bit XIDs into PostgreSQL 15

2022-12-09 Thread adherent postgres
l has no reason not to implement xid 64 bit. What about your opinion? Best whish 发件人: Aleksander Alekseev 发送时间: 2022年12月9日 20:49 收件人: pgsql-hackers@lists.postgresql.org 抄送: adherent postgres ; Chris Travers ; Chris Travers ; Bruce Momjian 主题: Re: Add 64-bit

Re: Add 64-bit XIDs into PostgreSQL 15

2022-12-09 Thread Pavel Borisov
Hi, Robert! > Lest we miss the forest for the trees, there is an aspect of this > patch that I find to be an extremely good idea and think we should try > to get committed even if the rest of the patch set ends up in the > rubbish bin. Specifically, there are a couple of patches in here that > have

Re: Add 64-bit XIDs into PostgreSQL 15

2022-12-09 Thread Aleksander Alekseev
Hi adherent, > Robertmhaas said that the project Zheap is > dead(https://twitter.com/andy_pavlo/status/1590703943176589312), which means > that we cannot use Zheap to deal with the issue of xid wraparound and dead > tuples in tables. The dead tuple issue is not a big deal because I can sti

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-30 Thread Peter Geoghegan
On Wed, Nov 30, 2022 at 8:13 AM Robert Haas wrote: > I haven't checked the patches to see whether they look correct, and > I'm concerned in particular about upgrade scenarios. But if there's a > way we can get that part committed, I think it would be a clear win. +1 -- Peter Geoghegan

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-30 Thread Robert Haas
On Tue, Nov 29, 2022 at 9:35 PM Chris Travers wrote: > My proposal would be to make the threshold configurable and start warning on > every transaction after that. There are a couple reasons to do that. > > The first is that noisy warnings are extremely easy to see. You get them in > cron emai

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Chris Travers
On Tue, Nov 29, 2022 at 5:57 PM Bruce Momjian wrote: > On Tue, Nov 29, 2022 at 11:41:07AM -0500, Robert Haas wrote: > > My argument is that removing xidStopLimit is totally fine, because it > > only serves to stop the database. What to do about xidWarnLimit is a > > slightly more complex question

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Greg Stark
On Mon, 28 Nov 2022 at 16:53, Peter Geoghegan wrote: > > Imagine if we actually had 64-bit XIDs -- let's assume for a moment > that it's a done deal. This raises a somewhat awkward question: do you > just let the system get further and further behind on freezing, > forever? We can all agree that

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Bruce Momjian
On Tue, Nov 29, 2022 at 11:41:07AM -0500, Robert Haas wrote: > My argument is that removing xidStopLimit is totally fine, because it > only serves to stop the database. What to do about xidWarnLimit is a > slightly more complex question. Certainly it can't be left untouched, > because warning that

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Robert Haas
On Tue, Nov 29, 2022 at 8:03 AM Chris Travers wrote: > To be clear, I never suggested shutting down the database. What I have > suggested is that repurposing the current approaching-xid-wraparound warnings > to start complaining loudly when a threshold is exceeded would be helpful. > I think

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Robert Haas
On Mon, Nov 28, 2022 at 4:53 PM Peter Geoghegan wrote: > Imagine if we actually had 64-bit XIDs -- let's assume for a moment > that it's a done deal. This raises a somewhat awkward question: do you > just let the system get further and further behind on freezing, > forever? We can all agree that 2

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Jan Wieck
On 11/29/22 09:46, Bruce Momjian wrote: As far as I know, all our freeze values are focused on avoiding XID wraparound. If XID wraparound is no longer an issue, we might find that our freeze limits can be much higher than they are now. I'd be careful in that direction as the values together w

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Bruce Momjian
On Tue, Nov 29, 2022 at 02:35:20PM +0100, Chris Travers wrote: > So I think the problem is that PostgreSQL is becoming more and more scalabile, > hardware is becoming more capable, and certain use cases are continuing to > scale up.  Over time, we tend to find ourselves approaching the end of the >

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Chris Travers
On Mon, Nov 28, 2022 at 11:06 PM Peter Geoghegan wrote: > On Mon, Nov 28, 2022 at 1:52 PM Bruce Momjian wrote: > > I think the problem is that we still have bloat with 64-bit XIDs, > > specifically pg_xact and pg_multixact files. Yes, that bloat is less > > serious, but it is still an issue wor

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Chris Travers
Hi; I suppose I must not have been clear in what I am suggesting we do and why. I will try to answer specific points below and then restate what I think the problem is, and what I think should be done about it. On Mon, Nov 28, 2022 at 5:53 PM Robert Haas wrote: > On Sat, Nov 26, 2022 at 4:08

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Peter Geoghegan
On Mon, Nov 28, 2022 at 1:52 PM Peter Geoghegan wrote: > I'm not claiming to know how to build this "better xidStopLimit > mechanism", by the way. I'm not seriously proposing it. Mostly I'm > just saying that the question "where do you draw the line if not at 2 > billion XIDs?" is a very pertinent

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Peter Geoghegan
On Mon, Nov 28, 2022 at 1:52 PM Bruce Momjian wrote: > I think the problem is that we still have bloat with 64-bit XIDs, > specifically pg_xact and pg_multixact files. Yes, that bloat is less > serious, but it is still an issue worth reporting in the server logs, > though not serious enough to st

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Peter Geoghegan
On Mon, Nov 28, 2022 at 1:30 PM Robert Haas wrote: > What is the purpose of using 64-bit XIDs, if not to avoid having to > stop the world when we run short of XIDs? I agree that the xidStopLimit mechanism was designed with the specific goal of preventing "true" physical XID wraparound that result

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Bruce Momjian
On Mon, Nov 28, 2022 at 04:30:22PM -0500, Robert Haas wrote: > What is the purpose of using 64-bit XIDs, if not to avoid having to > stop the world when we run short of XIDs? > > I'd say that if this patch, or any patch with broadly similar goals, > fails to remove xidStopLimit, it might as well n

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 4:09 PM Peter Geoghegan wrote: > Granted, the specifics of the current XidStopLimit mechanism are > unlikely to directly carry over to 64-bit XIDs. XidStopLimit is > structured in a way that doesn't actually consider freeze debt in > units like unfrozen pages. Like Chris, I

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Peter Geoghegan
On Mon, Nov 28, 2022 at 8:53 AM Robert Haas wrote: > It is true that if the table is progressively bloating, it is likely > to be more bloated by the time you are 8 billion XIDs behind than it > was when you were 800 million XIDs behind. I don't see that as a very > good reason not to adopt this p

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Robert Haas
On Sat, Nov 26, 2022 at 4:08 AM Chris Travers wrote: > I didn't see any changes to pg_upgrade to make this change possible on > upgrade. Is that also outside of the scope of your patch set? If so how is > that continuity supposed to be ensured? The scheme is documented in their 0006 patch, in

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-26 Thread Chris Travers
Hi; Trying to discuss where we are talking past eachother. On Fri, Nov 25, 2022 at 9:38 AM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi hackers, > > > I'm wondering whether the safest way to handle this is by creating a > > new TAM called "heap64", so that all storage changes

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-25 Thread Aleksander Alekseev
Hi hackers, > I'm wondering whether the safest way to handle this is by creating a > new TAM called "heap64", so that all storage changes happens there. > Many current users see stability as one of the greatest strengths of > Postgres, so while I very much support this move, I wonder if this > gi

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-24 Thread Peter Geoghegan
On Sun, Nov 20, 2022 at 11:58 PM Chris Travers wrote: > I can start by saying I think it would be helpful (if the other issues are > approached reasonably) to have 64-bit xids, but there is an important piece > of context in reventing xid wraparounds that seems missing from this patch > unless

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-24 Thread Simon Riggs
On Fri, 21 Oct 2022 at 17:09, Maxim Orlov wrote: > Reviews and opinions are very welcome! I'm wondering whether the safest way to handle this is by creating a new TAM called "heap64", so that all storage changes happens there. (Obviously there are still many other changes in core, but they are m

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-24 Thread Aleksander Alekseev
Hi Chris, > XID wraparound doesn't happen to healthy databases > If you disagree, I would like to hear why. Consider the case when you run a slow OLAP query that takes 12h to complete and 100K TPS of fast OLTP-type queries on the same system. The fast queries will consume all 32-bit XIDs in less

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-23 Thread Chris Travers
On Tue, Nov 22, 2022 at 10:01 AM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi Chris, > > > Right now the way things work is: > > 1. Database starts throwing warnings that xid wraparound is approaching > > 2. Database-owning team initiates an emergency response, may take > downtime

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-22 Thread Dilip Kumar
On Tue, Nov 22, 2022 at 7:44 PM Aleksander Alekseev wrote: > > Hi hackers, > > [ Excluding my personal e-mail from cc:, not sure how it got there. > Please don't cc: to afis...@gmail.com, I'm not using it for reading > pgsql-hackers@. ] > > > I agree with Alexander, that notifications for DBA are

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-22 Thread Aleksander Alekseev
Hi hackers, [ Excluding my personal e-mail from cc:, not sure how it got there. Please don't cc: to afis...@gmail.com, I'm not using it for reading pgsql-hackers@. ] > I agree with Alexander, that notifications for DBA are a little bit > outside the scope of the activity in this thread unless we'

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-22 Thread Pavel Borisov
Hi, Alexander! On Tue, 22 Nov 2022 at 13:01, Aleksander Alekseev wrote: > > Hi Chris, > > > Right now the way things work is: > > 1. Database starts throwing warnings that xid wraparound is approaching > > 2. Database-owning team initiates an emergency response, may take downtime > > or degrad

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-22 Thread Aleksander Alekseev
Hi Chris, > Right now the way things work is: > 1. Database starts throwing warnings that xid wraparound is approaching > 2. Database-owning team initiates an emergency response, may take downtime > or degradation of services as a result > 3. People get frustrated with PostgreSQL because this

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Chris Travers
On Mon, Nov 21, 2022 at 10:40 AM Pavel Borisov wrote: > > I have a very serious concern about the current patch set. as someone > who has faced transaction id wraparound in the past. > > > > I can start by saying I think it would be helpful (if the other issues > are approached reasonably) to hav

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Chris Travers
On Mon, Nov 21, 2022 at 12:25 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi hackers, > > > > I have a very serious concern about the current patch set. as someone > who has faced transaction id wraparound in the past. > > > > [...] > > > > I had a similar stance when I started wor

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Andres Freund
Hi, On 2022-11-21 15:16:46 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2022-11-21 14:21:35 -0500, Tom Lane wrote: > >> pg_resetwal does seem like a better, more useful home for this; it'd > >> allow you to adjust these numbers after initial creation which might be > >> useful. I'm not

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Tom Lane
Andres Freund writes: > On 2022-11-21 14:21:35 -0500, Tom Lane wrote: >> pg_resetwal does seem like a better, more useful home for this; it'd >> allow you to adjust these numbers after initial creation which might be >> useful. I'm not sure how flexible it is right now in terms of where >> you ca

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Andres Freund
Hi, On 2022-11-21 14:21:35 -0500, Tom Lane wrote: > Peter Eisentraut writes: > >> To date testing database cluster wraparund was not easy as initdb has > >> always > >> inited it with default xid/mxid/mxoff. The option to specify any valid > >> xid/mxid/mxoff at cluster startup will make these t

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Tom Lane
Peter Eisentraut writes: >> To date testing database cluster wraparund was not easy as initdb has always >> inited it with default xid/mxid/mxoff. The option to specify any valid >> xid/mxid/mxoff at cluster startup will make these things easier. > Doesn't pg_resetwal already provide that functio

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Peter Eisentraut
Question about """ Subject: [PATCH v50 5/8] Add initdb option to initialize cluster with non-standard xid/mxid/mxoff. To date testing database cluster wraparund was not easy as initdb has always inited it with default xid/mxid/mxoff. The option to specify any valid xid/mxid/mxoff at cluster sta

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Aleksander Alekseev
Hi hackers, > > I have a very serious concern about the current patch set. as someone who > > has faced transaction id wraparound in the past. > > [...] > > I had a similar stance when I started working on this patch. Of > course, it seemed horrible just to postpone the consequences of > inadequa

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Pavel Borisov
> I have a very serious concern about the current patch set. as someone who has > faced transaction id wraparound in the past. > > I can start by saying I think it would be helpful (if the other issues are > approached reasonably) to have 64-bit xids, but there is an important piece > of context

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-20 Thread Chris Travers
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested I have a very serious concern about the current patch set. as someone who has

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-16 Thread Aleksander Alekseev
Hi hackers, > Dilip Kumar asked a good question in the thread about the 0001..0003 > subset [1]. I would like to duplicate it here to make sure it was not > missed by mistake: > > """ > Have we measured the WAL overhead because of this patch set? maybe > these particular patches will not impact bu

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-14 Thread Aleksander Alekseev
Hi hackers, > Thanks, done! Dilip Kumar asked a good question in the thread about the 0001..0003 subset [1]. I would like to duplicate it here to make sure it was not missed by mistake: """ Have we measured the WAL overhead because of this patch set? maybe these particular patches will not impac

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-14 Thread Thom Brown
On Thu, 3 Nov 2022 at 08:12, Maxim Orlov wrote: > > Hi! > >> I attach an additional V48-0009 patch as they are just comments, apply it if >> you want to. > > Big thank you for your review. I've applied your addition in the recent patch > set below. > > Besides, mentioned above, next changes are

Re: Add 64-bit XIDs into PostgreSQL 15

2022-10-21 Thread Zhang Mingli
Hi, On Oct 22, 2022, 00:09 +0800, Maxim Orlov , wrote: > > > > Done! Thanks! Here is the rebased version. > > > > This version has bug fix for multixact replication. Previous versions of > > the patch set does not write pd_multi_base in WAL. Thus, this field was set > > to 0 upon WAL reply on r

Re: Add 64-bit XIDs into PostgreSQL 15

2022-10-12 Thread Michael Paquier
On Fri, Oct 07, 2022 at 02:04:09PM +0300, Maxim Orlov wrote: > As always, reviews are very welcome! This patch set needs a rebase, as far as I can see. -- Michael signature.asc Description: PGP signature

Re: Add 64-bit XIDs into PostgreSQL 15

2022-09-20 Thread Zhang Mingli
Hi, On Sep 20, 2022, 17:26 +0800, Justin Pryzby , wrote: > On Tue, Sep 20, 2022 at 03:37:47PM +0800, Zhang Mingli wrote: > > I want to have a look at these patches, but apply on master failed: > > Yeah, it's likely to break every week or more often. > > You have a few options: > > 0) resolve the co

Re: Add 64-bit XIDs into PostgreSQL 15

2022-09-20 Thread Justin Pryzby
On Tue, Sep 20, 2022 at 03:37:47PM +0800, Zhang Mingli wrote: > I want to have a look at these patches, but apply on master failed: Yeah, it's likely to break every week or more often. You have a few options: 0) resolve the conflict yourself; 1) apply the patch to the commit that the authors se

Re: Add 64-bit XIDs into PostgreSQL 15

2022-09-20 Thread Zhang Mingli
Hi, With these patches, it seems that we don’t need to handle wraparound in GetNextLocalTransactionId() too, as LocalTransactionId is unit64 now. ``` LocalTransactionId GetNextLocalTransactionId(void) {     LocalTransactionId result;     /* loop to avoid returning InvalidLocalTransactionId at wr

Re: Add 64-bit XIDs into PostgreSQL 15

2022-09-20 Thread Zhang Mingli
Hi, Is this patch target to PG16 now? I want to have a look at these patches, but apply on master failed: Applying: Use 64-bit numbering of SLRU pages. Applying: Use 64-bit format to output XIDs Applying: Use 64-bit FullTransactionId instead of Epoch:xid Applying: Use 64-bit pages representation

Re: Add 64-bit XIDs into PostgreSQL 15

2022-09-16 Thread Maxim Orlov
I have to say, to my embarrassment, after sending the previous email, I've notice minor imperfections in a patch set caused by the last rebase. These imperfections led to cf bot fail. I'll address this issue in the next iteration in order not to generate excessive flow. -- Best regards, Maxim Or

Re: Add 64-bit XIDs into PostgreSQL 15

2022-09-04 Thread Dilip Kumar
On Sun, Sep 4, 2022 at 9:53 AM Dilip Kumar wrote: > > On Mon, Jul 18, 2022 at 2:54 PM Pavel Borisov wrote: > >> > >> > I can agree with you that sending rebased patches too often can be a > >> > little annoying. On the other hand, otherwise, it's just red in Cfbot. I > >> > suppose it's much ea

Re: Add 64-bit XIDs into PostgreSQL 15

2022-09-03 Thread Dilip Kumar
On Mon, Jul 18, 2022 at 2:54 PM Pavel Borisov wrote: >> >> > I can agree with you that sending rebased patches too often can be a >> > little annoying. On the other hand, otherwise, it's just red in Cfbot. I >> > suppose it's much easier and more comfortable to review the patches that >> > at l

Re: Add 64-bit XIDs into PostgreSQL 15

2022-08-26 Thread Justin Pryzby
On Wed, Jan 05, 2022 at 06:12:26PM -0600, Justin Pryzby wrote: > On Wed, Jan 05, 2022 at 06:51:37PM -0500, Bruce Momjian wrote: > > On Tue, Jan 4, 2022 at 10:22:50PM +, Finnerty, Jim wrote: > > > I'm concerned about the maintainability impact of having 2 new > > > on-disk page formats. It's al

Re: Add 64-bit XIDs into PostgreSQL 15

2022-07-15 Thread Aleksander Alekseev
Hi hackers, > I can agree with you that sending rebased patches too often can be a little > annoying. On the other hand, otherwise, it's just red in Cfbot. I suppose > it's much easier and more comfortable to review the patches that at least > apply cleanly and pass all tests. So if Cfbot is re

Re: Add 64-bit XIDs into PostgreSQL 15

2022-07-15 Thread Pavel Borisov
On Fri, 15 Jul 2022 at 16:17, Justin Pryzby wrote: > On Fri, Jul 15, 2022 at 03:23:29PM +0400, Pavel Borisov wrote: > > On Fri, 15 Jul 2022 at 15:20, Pavel Borisov > wrote: > > > Hi, hackers! > > > v42 stopped applying, so we rebased it to v43. Attached is a GitHub > link, > > > but I see Cfbot

Re: Add 64-bit XIDs into PostgreSQL 15

2022-07-15 Thread Justin Pryzby
On Fri, Jul 15, 2022 at 03:23:29PM +0400, Pavel Borisov wrote: > On Fri, 15 Jul 2022 at 15:20, Pavel Borisov wrote: > > Hi, hackers! > > v42 stopped applying, so we rebased it to v43. Attached is a GitHub link, > > but I see Cfbot hasn't become green. Apparently, it hasn't seen changes in > > GitH

Re: Add 64-bit XIDs into PostgreSQL 15

2022-07-15 Thread Pavel Borisov
On Fri, 15 Jul 2022 at 15:20, Pavel Borisov wrote: > Hi, hackers! > v42 stopped applying, so we rebased it to v43. Attached is a GitHub link, > but I see Cfbot hasn't become green. Apparently, it hasn't seen changes in > GitHub link relative to v42 attached as a patch. > Github link is as follow

Re: Add 64-bit XIDs into PostgreSQL 15

2022-07-15 Thread Pavel Borisov
Hi, hackers! v42 stopped applying, so we rebased it to v43. Attached is a GitHub link, but I see Cfbot hasn't become green. Apparently, it hasn't seen changes in GitHub link relative to v42 attached as a patch. -- Best regards, Pavel Borisov

Re: Add 64-bit XIDs into PostgreSQL 15

2022-06-30 Thread Pavel Borisov
> > This seems to be causing cfbot/cirrusci to time out. > > Here's the build history > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/38/3594 > > https://cirrus-ci.com/task/4809278652416000 4 weeks ago on macos > https://cirrus-ci.com/task/5559884417597440 2 weeks ago on macos

Re: Add 64-bit XIDs into PostgreSQL 15

2022-06-30 Thread Justin Pryzby
This seems to be causing cfbot/cirrusci to time out. Here's the build history https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/38/3594 https://cirrus-ci.com/task/4809278652416000 4 weeks ago on macos https://cirrus-ci.com/task/5559884417597440 2 weeks ago on macos https://cirru

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-19 Thread Pavel Borisov
> > > Do you know that you can test a branch on cirrus without using CF bot or > mailing the patch to the list ? See src/tools/ci/README > Yes, sure! The main reason to post updates of this patchset is for hackers that are interested in the progress have relevant version with updates. This patch

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-19 Thread Justin Pryzby
On Fri, Mar 11, 2022 at 08:26:26PM +0300, Aleksander Alekseev wrote: > Hi hackers, > > > Here is a new version of the patchset. SLRU refactoring was moved to a > > separate patch. Both v14-0003 (XID_FMT macro) and v14-0004 (SLRU > > refactoring) can be delivered in PG15. > > Here is a new version

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-18 Thread Andres Freund
Hi, On 2022-03-18 18:22:00 +0300, Maxim Orlov wrote: > Here is v22 with following changes: > - use explicit unsigned long long cast for printf/elog XIDs instead of > macro XID_TYPE > - add *.po localization > - fix forgotten XIDs format changes in pg_resetwal.c > - 0006 patch refactoring FWIW, 00

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-18 Thread Pavel Borisov
Hi, hackers! While working on the patchset I've noticed that FullTransactionId lost its semantics in its scope. TransactionId is supposed to be 64bit (default) and epoch approach becomes outdated. What do you think of fully removing FullTransactionId and its support functions and macro? -- Best

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-17 Thread Pavel Borisov
> I forked this thread as requested by several people in the discussion [1]. > > The new thread contains two patches that are targeting PG15. I replaced > the thread in the current CF to [1]. This thread was added to the next CF. > I suggest we continue discussing changes targeting PG >= 16 here. >

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-17 Thread Aleksander Alekseev
Hi hackers, I forked this thread as requested by several people in the discussion [1]. The new thread contains two patches that are targeting PG15. I replaced the thread in the current CF to [1]. This thread was added to the next CF. I suggest we continue discussing changes targeting PG >= 16 her

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-15 Thread Kyotaro Horiguchi
At Tue, 15 Mar 2022 18:48:34 +0300, Maxim Orlov wrote in > Hi Kyotaro! > > 0001: > > > > The XID_FMT has quite bad impact on the translatability of error > > messages. 3286065651 has removed INT64_FORMAT from translatable > > texts for the reason. This re-introduces that in several places.

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-15 Thread Michael Banck
Hi, On Mon, Mar 14, 2022 at 01:32:04PM +0300, Aleksander Alekseev wrote: > IMO v16-0001 and v16-0002 are in pretty good shape and are as much as > we are going to deliver in PG15. I'm going to change the status of the > CF entry to "Ready for Committer" somewhere this week unless someone > believe

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-14 Thread Kyotaro Horiguchi
At Mon, 14 Mar 2022 19:43:40 +0400, Pavel Borisov wrote in > > I'd like to add a few quick notes on what's been done in v17. I have some commens by a quick look-through. Apologize in advance for wrong comments from the lack of the knowledge of the whole patch-set. > > Patches 0001 and 0002 th

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-14 Thread Pavel Borisov
> > Hi, Hackers! > >> Hi! Here is updated version of the patch, based on Alexander's ver16. >> > I'd like to add a few quick notes on what's been done in v17. > > Patches 0001 and 0002 that are planned to be committed to PG15 are almost > unchanged with the exception of one unnecessary cast in 0002

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-14 Thread Pavel Borisov
Hi, Hackers! > Hi! Here is updated version of the patch, based on Alexander's ver16. > I'd like to add a few quick notes on what's been done in v17. Patches 0001 and 0002 that are planned to be committed to PG15 are almost unchanged with the exception of one unnecessary cast in 0002 removed. We'

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-14 Thread Aleksander Alekseev
Hi hackers, > > Here is a new version of the patchset. SLRU refactoring was moved to a > > separate patch. Both v14-0003 (XID_FMT macro) and v14-0004 (SLRU > > refactoring) can be delivered in PG15. > > Here is a new version of the patchset. The changes compared to v14 are > minimal. Most importan

  1   2   >