Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-23 Thread Bertrand Drouvot
Hi, On Tue, Jan 23, 2024 at 02:50:06PM +0900, Michael Paquier wrote: > On Mon, Jan 22, 2024 at 09:07:45AM +, Bertrand Drouvot wrote: > > I've rewritten some of that, and applied the patch down to v16. Thanks! > > Yeah, I can clearly see how this patch helps from a theoritical perspective

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-22 Thread Michael Paquier
On Mon, Jan 22, 2024 at 09:07:45AM +, Bertrand Drouvot wrote: > On Mon, Jan 22, 2024 at 03:54:44PM +0900, Michael Paquier wrote: >> Also, wouldn't it be better to document in the test why >> txid_current_snapshot() is chosen? Contrary to txid_current(), it >> does not generate a

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-22 Thread Bertrand Drouvot
Hi, On Mon, Jan 22, 2024 at 03:54:44PM +0900, Michael Paquier wrote: > On Fri, Jan 19, 2024 at 09:03:01AM +, Bertrand Drouvot wrote: > > On Fri, Jan 19, 2024 at 09:00:01AM +0300, Alexander Lakhin wrote: > +# Launch $sql and wait for a new snapshot that has a newer horizon before > +# doing

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-21 Thread Michael Paquier
On Fri, Jan 19, 2024 at 09:03:01AM +, Bertrand Drouvot wrote: > On Fri, Jan 19, 2024 at 09:00:01AM +0300, Alexander Lakhin wrote: >> 15.01.2024 12:00, Alexander Lakhin wrote: >>> If this approach looks promising to you, maybe we could add a submodule to >>> perl/PostgreSQL/Test/ and use this

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-19 Thread Bertrand Drouvot
Hi, On Fri, Jan 19, 2024 at 09:00:01AM +0300, Alexander Lakhin wrote: > Hello, > > 15.01.2024 12:00, Alexander Lakhin wrote: > > If this approach looks promising to you, maybe we could add a submodule to > > perl/PostgreSQL/Test/ and use this functionality in other tests (e.g., in > >

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-18 Thread Alexander Lakhin
Hello, 15.01.2024 12:00, Alexander Lakhin wrote: If this approach looks promising to you, maybe we could add a submodule to perl/PostgreSQL/Test/ and use this functionality in other tests (e.g., in 019_replslot_limit) as well. Personally I think that having such a functionality for using in

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-15 Thread Alexander Lakhin
Hello Michael and Bertrand, 15.01.2024 06:59, Michael Paquier wrote: The WAL records related to standby snapshots are playing a lot with the randomness of the failures we are seeing. Alexander has mentioned offlist something else: using SIGSTOP on the bgwriter to avoid these records and make

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-15 Thread Bertrand Drouvot
Hi, On Mon, Jan 15, 2024 at 01:11:26PM +0900, Michael Paquier wrote: > On Sun, Jan 14, 2024 at 11:08:39PM -0500, Tom Lane wrote: > > Michael Paquier writes: > >> While thinking about that, a second idea came into my mind: a > >> superuser-settable developer GUC to disable such WAL records to be

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-14 Thread Michael Paquier
On Sun, Jan 14, 2024 at 11:08:39PM -0500, Tom Lane wrote: > Michael Paquier writes: >> While thinking about that, a second idea came into my mind: a >> superuser-settable developer GUC to disable such WAL records to be >> generated within certain areas of the test. This requires a small >>

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-14 Thread Tom Lane
Michael Paquier writes: > While thinking about that, a second idea came into my mind: a > superuser-settable developer GUC to disable such WAL records to be > generated within certain areas of the test. This requires a small > implementation, but nothing really huge, while being portable >

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-14 Thread Michael Paquier
On Fri, Jan 12, 2024 at 01:46:08PM +, Bertrand Drouvot wrote: > 1) Michael's proposal up-thread (means tweak the test with a retry logic, > retrying > things if such a standby snapshot is found). > > 2) Don't report a test error for active slots in case its catalog_xmin > advanced. > > I'd

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-12 Thread Bertrand Drouvot
Hi, On Fri, Jan 12, 2024 at 02:00:01PM +0300, Alexander Lakhin wrote: > Hi, > > 12.01.2024 10:15, Bertrand Drouvot wrote: > > > > For this one, the "good" news is that it looks like that we don’t see the > > "terminating" message not followed by an "obsolete" message (so the engine > > behaves

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-11 Thread Bertrand Drouvot
Hi, On Fri, Jan 12, 2024 at 07:01:55AM +0900, Michael Paquier wrote: > On Thu, Jan 11, 2024 at 11:00:01PM +0300, Alexander Lakhin wrote: > > Bertrand, I've relaunched tests in the same slowed down VM with both > > patches applied (but with no other modifications) and got a failure > > with

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-11 Thread Michael Paquier
On Thu, Jan 11, 2024 at 11:00:01PM +0300, Alexander Lakhin wrote: > Bertrand, I've relaunched tests in the same slowed down VM with both > patches applied (but with no other modifications) and got a failure > with pg_class, similar to what we had seen before: > 9   #   Failed test 'activeslot

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-11 Thread Bertrand Drouvot
Hi, On Thu, Jan 11, 2024 at 01:00:00PM +0300, Alexander Lakhin wrote: > Hi Bertrand, > > 10.01.2024 19:32, Bertrand Drouvot wrote: > > > > > is an absent message 'obsolete replication slot "row_removal_activeslot"' > > Looking at > >

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-10 Thread Bertrand Drouvot
Hi, On Wed, Jan 10, 2024 at 05:00:01PM +0300, Alexander Lakhin wrote: > 10.01.2024 12:46, Bertrand Drouvot wrote: > > > Would it be possible to also send the standby logs? > > Yes, please look at the attached logs. This time I've build postgres with > -DWAL_DEBUG and ran tests with TEMP_CONFIG

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-10 Thread Bertrand Drouvot
Hi, On Tue, Jan 09, 2024 at 08:00:00PM +0300, Alexander Lakhin wrote: > Michael, it definitely increases stability of the test (tens of iterations > with 20 tests in parallel performed successfully), Thanks for testing! > although I've managed to > see another interesting failure (twice): > 13  

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-09 Thread Michael Paquier
On Tue, Jan 09, 2024 at 08:00:00PM +0300, Alexander Lakhin wrote: > Thus just adding FREEZE is not enough, seemingly. It makes me wonder if > 0174c2d21 should be superseded by a patch like discussed (or just have > autovacuum = off added)... Adding an extra FREEZE offers an extra insurance, so I

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-09 Thread Alexander Lakhin
Hello Michael and Bertrand, I'd also like to note that even with FREEZE added [1], I happened to see the test failure: 5   #   Failed test 'inactiveslot slot invalidation is logged with vacuum on pg_class' 5   #   at t/035_standby_logical_decoding.pl line 222. 5 5   #   Failed test

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2023-05-31 Thread Drouvot, Bertrand
Hi, On 5/30/23 12:34 PM, Drouvot, Bertrand wrote: Hi hackers, Please find attached a patch proposal to $SUBJECT. Indeed, we have seen occurrences in [1] that some slots were not invalidated (while we expected vacuum to remove dead rows leading to slots invalidation on the standby). Though we

Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2023-05-30 Thread Drouvot, Bertrand
Hi hackers, Please find attached a patch proposal to $SUBJECT. Indeed, we have seen occurrences in [1] that some slots were not invalidated (while we expected vacuum to remove dead rows leading to slots invalidation on the standby). Though we don't have strong evidences that this was due to