Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-04-10 Thread Kartyshov Ivan
I did some experiments over synchronous replications and got that cascade replication can`t be synchronous. And  pg_wal_replay_wait() allows us to read your writes consistency on cascade replication. Beyond that, I added more tests on multi-standby replication and cascade replications. -- Ivan

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-04-02 Thread Kartyshov Ivan
On 2024-04-02 13:15, Bharath Rupireddy wrote: On Tue, Apr 2, 2024 at 3:41 PM Kartyshov Ivan wrote: 8 years, we tried to add this feature, and now we suggest the best way for this feature is to commit the minimal version first. Just curious, do you or anyone else have an immediate use

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-04-02 Thread Kartyshov Ivan
On 2024-04-02 11:14, Andy Fan wrote: If so, when users call pg_wal_replay_wait, they can get informed when the wal is replaied to the target_lsn, but they can't know how long time it waits unless they check it in application side, I think such information will be useful for monitor purpose

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-30 Thread Kartyshov Ivan
Thank you Alexander for working on patch, may be we should change some names: 1) test 043_wait_lsn.pl -> to 043_waitlsn.pl like waitlsn.c and waitlsn.h In waitlsn.c and waitlsn.h variables: 2) targret_lsn -> trgLSN like curLSN 3) lsn -> trgLSN like curLSN -- Ivan Kartyshov Postgres

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-26 Thread Kartyshov Ivan
Thank you for your interest to the patch. I understand you questions, but I fully support Alexander Korotkov idea to commit the minimal required functionality. And then keep working on other improvements. On 2024-03-24 05:39, Bharath Rupireddy wrote: On Fri, Mar 22, 2024 at 4:28 AM Peter

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-22 Thread Kartyshov Ivan
Thank you for your feedback. On 2024-03-20 12:11, Alexander Korotkov wrote: On Wed, Mar 20, 2024 at 12:34 AM Kartyshov Ivan wrote: > 4.2 With an unreasonably high future LSN, BEGIN command waits > unboundedly, shouldn't we check if the specified LSN is more than > pg_last_wal_re

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-22 Thread Kartyshov Ivan
Thank you for your feedback. On 2024-03-20 12:11, Alexander Korotkov wrote: On Wed, Mar 20, 2024 at 12:34 AM Kartyshov Ivan wrote: > 4.2 With an unreasonably high future LSN, BEGIN command waits > unboundedly, shouldn't we check if the specified LSN is more than > pg_last_wal_re

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-22 Thread Kartyshov Ivan
On 2024-03-20 12:11, Alexander Korotkov wrote: On Wed, Mar 20, 2024 at 12:34 AM Kartyshov Ivan wrote: > 4.2 With an unreasonably high future LSN, BEGIN command waits > unboundedly, shouldn't we check if the specified LSN is more than > pg_last_wal_receive_lsn() error out? I think

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-19 Thread Kartyshov Ivan
Bharath Rupireddy, thank you for you review. But here is some points. On 2024-03-16 10:02, Bharath Rupireddy wrote: 4.1 With invalid LSN succeeds, shouldn't it error out? Or at least, add a fast path/quick exit to WaitForLSN()? BEGIN AFTER '0/0'; In postgresql '0/0' is Valid pg_lsn, but it is

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-19 Thread Kartyshov Ivan
Intro == The main purpose of the feature is to achieve read-your-writes-consistency, while using async replica for reads and primary for writes. In that case lsn of last modification is stored inside application. We cannot store this lsn inside database, since reads are distributed across

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-15 Thread Kartyshov Ivan
On 2024-03-15 22:59, Kartyshov Ivan wrote: On 2024-03-11 13:44, Alexander Korotkov wrote: I picked the second option and left only the AFTER clause for the BEGIN statement. I think this should be enough for the beginning. Thank you for your rework on your patch, here I made some fixes: 0

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-15 Thread Kartyshov Ivan
On 2024-03-11 13:44, Alexander Korotkov wrote: I picked the second option and left only the AFTER clause for the BEGIN statement. I think this should be enough for the beginning. Thank you for your rework on your patch, here I made some fixes: 0) autocomplete 1) less jumps 2) more description

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-07 Thread Kartyshov Ivan
Intro == The main purpose of the feature is to achieve read-your-writes-consistency, while using async replica for reads and primary for writes. In that case lsn of last modification is stored inside application. We cannot store this lsn inside database, since reads are distributed across

Fwd: Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-02-01 Thread Kartyshov Ivan
Updated, rebased, fixed Ci and added documentation. We left two different solutions. Help me please to choose the best. 1) Classic (wait_classic_v6.patch) https://www.postgresql.org/message-id/3cc883048264c2e9af022033925ff8db%40postgrespro.ru == advantages: multiple wait events, separate

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-01-17 Thread Kartyshov Ivan
Add some fixes and rebase. -- Ivan Kartyshov Postgres Professional: www.postgrespro.comdiff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml index 4a42999b18..657a217e27 100644 --- a/doc/src/sgml/ref/allfiles.sgml +++ b/doc/src/sgml/ref/allfiles.sgml @@ -188,6 +188,7 @@

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-01-11 Thread Kartyshov Ivan
Rebased and ready for review. I left only versions (due to irreparable problems) 1) Classic (wait_classic_v4.patch) https://www.postgresql.org/message-id/3cc883048264c2e9af022033925ff8db%40postgrespro.ru == advantages: multiple wait events, separate WAIT FOR statement disadvantages: new

Re: [HACKERS] make async slave to wait for lsn to be replayed

2023-12-08 Thread Kartyshov Ivan
Should rise disscusion on separate utility statement or find case where procedure version is failed. 1) Classic (wait_classic_v3.patch) https://www.postgresql.org/message-id/3cc883048264c2e9af022033925ff8db%40postgrespro.ru == advantages: multiple wait events, separate WAIT FOR statement

Re: [HACKERS] make async slave to wait for lsn to be replayed

2023-12-08 Thread Kartyshov Ivan
On 2023-11-27 03:08, Alexander Korotkov wrote: I've retried my case with v6 and it doesn't fail anymore. But I wonder how safe it is to reset xmin within the user-visible function? We have no guarantee that the function is not called inside the complex query. Then how will the rest of the

Re: [HACKERS] make async slave to wait for lsn to be replayed

2023-03-06 Thread Kartyshov Ivan
Fix build.meson troubles -- Ivan Kartyshov Postgres Professional: http://www.postgrespro.com The Russian Postgres Companydiff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..422bb1ed82 100644 ---

Re: [HACKERS] make async slave to wait for lsn to be replayed

2023-03-06 Thread Kartyshov Ivan
Update patch to fix conflict with master -- Ivan Kartyshov Postgres Professional: http://www.postgrespro.com The Russian Postgres Companydiff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..422bb1ed82 100644 ---

Re: [HACKERS] make async slave to wait for lsn to be replayed

2023-03-04 Thread Kartyshov Ivan
Here I made new patch of feature, discussed above. WAIT FOR procedure - waits for certain lsn on pause == Synopsis == SELECT pg_wait_lsn(‘LSN’, timeout) returns boolean Where timeout = 0, will wait infinite without timeout And if timeout = 1, then just check if lsn

Re: [HACKERS] make async slave to wait for lsn to be replayed

2023-02-28 Thread Kartyshov Ivan
Intro== The main purpose of the feature is to achieve read-your-writes-consistency, while using async replica for reads and primary for writes. In that case lsn of last modification is stored inside application. We cannot store this lsn inside database, since reads are distributed

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-07 Thread Kartyshov Ivan
On 2020-04-08 00:27, Tom Lane wrote: Alexander Korotkov writes: » WAIT FOR LSN lsn [ TIMEOUT timeout ] This seems like a really carelessly chosen syntax —- *three* new keywords, when you probably didn't need any. Are you not aware that there is distributed overhead in the grammar for

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-06 Thread Kartyshov Ivan
On 2020-04-04 03:14, Alexander Korotkov wrote: I think that now we would be fine with single LSN and single TIMEOUT. In future we may add multiple LSNs/TIMEOUTs or/and support for expressions as LSNs/TIMEOUTs if we figure out it's necessary. I also think it's good to couple waiting for lsn with

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-04-04 Thread Kartyshov Ivan
On 2020-04-03 21:51, Anna Akenteva wrote: I did some code cleanup and added tests - both for the standalone WAIT FOR statement and for WAIT FOR as a part of BEGIN. The new patch is attached. I did more cleanup and code optimization on waiting events on latch. And rebase patch. -- Ivan

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-03-26 Thread Kartyshov Ivan
Anna, thank you for your review. On 2020-03-25 21:10, Anna Akenteva wrote: On 2020-03-21 14:16, Kartyshov Ivan wrote: and event is: LSN value [options] TIMESTAMP value I would maybe remove WAIT FOR TIMESTAMP. As Robert Haas has pointed out, it seems a lot like

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-03-21 Thread Kartyshov Ivan
As it was discussed earlier, I added wait for statement into begin/start statement. Synopsis == BEGIN [ WORK | TRANSACTION ] [ transaction_mode[, ...] ] wait_for_event where transaction_mode is one of: ISOLATION LEVEL { SERIALIZABLE | REPEATABLE READ | READ COMMITTED

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-03-17 Thread Kartyshov Ivan
I made some improvements over old implementation WAIT FOR. Synopsis == WAIT FOR [ANY | SOME | ALL] event [, event ...] and event is: LSN value options TIMESTAMP value and options is: TIMEOUT delay UNTIL TIMESTAMP timestamp ALL - option used

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-03-06 Thread Kartyshov Ivan
Sorry, I have some troubles on email sending. On 2020-03-06 08:54, Kyotaro Horiguchi wrote: The syntax seems getting confused. What happens if we typed in the command "WAIT FOR TIMESTAMP '...' UNTIL TIMESTAMP ''"? It seems to me the options is useles. Couldn't the TIMEOUT option be a part

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-03-06 Thread Kartyshov Ivan
On 2020-03-06 08:54, Kyotaro Horiguchi wrote: The syntax seems getting confused. What happens if we typed in the command "WAIT FOR TIMESTAMP '...' UNTIL TIMESTAMP ''"? It seems to me the options is useles. Couldn't the TIMEOUT option be a part of event? I know gram.y doesn't accept that

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-03-05 Thread Kartyshov Ivan
On 2018-03-06 14:50, Simon Riggs wrote: On 6 March 2018 at 11:24, Dmitry Ivanov wrote: In PG11, I propose the following command, sticking mostly to Ants' syntax, and allowing to wait for multiple events before it returns. It doesn't hold snapshot and will not get cancelled by Hot Standby.