Re: ***Conflict with recovery error***

2023-01-20 Thread Laurenz Albe 
On Fri, 2023-01-20 at 09:59 +, Abhishek Prakash wrote:
> We had set max_standby_streaming_delay = -1, but faced storage issues
> nearly 3.5 TB of storage was consumed.

Then either don't run queries that take that long or run fewer data
modifications on the primary.

Or invest in a few more TB disk storage.

Yours,
Laurenz Albe

RE: ***Conflict with recovery error***

2023-01-20 Thread Abhishek Prakash 
Hi Laurenz,

Thanks for your reply.
We had set max_standby_streaming_delay = -1, but faced storage issues nearly 
3.5 TB of storage was consumed. 

Regards,
Abhishek P

-Original Message-
From: Laurenz Albe  
Sent: Friday, January 20, 2023 3:26 PM
To: Abhishek Prakash ; 
pgsql-gene...@lists.postgresql.org; pgsql-hackers@lists.postgresql.org; 
usergro...@postgresql.org
Subject: Re: ***Conflict with recovery error***

[**EXTERNAL EMAIL**]

On Fri, 2023-01-20 at 08:56 +, Abhishek Prakash wrote:
> We are facing below issue with read replica we did work arounds by 
> setting hot_standby_feedback, max_standby_streaming_delay and 
> max_standby_archive_delay, which indeed caused adverse effects on 
> primary DB and storage. As our DB is nearly 6 TB which runs as AWS Postgres 
> RDS.
>
> Even the below error occurs on tables where vacuum is disabled and no 
> DML operations are permitted. Will there be any chances to see row 
> versions being changed even if vacuum is disabled.
> Please advise.
>
> 2023-01-13 07:20:12 
> UTC:10.64.103.75(61096):ubpreplica@ubprdb01:[17707]:ERROR:  canceling 
> statement due to conflict with recovery
> 2023-01-13 07:20:12 
> UTC:10.64.103.75(61096):ubpreplica@ubprdb01:[17707]:DETAIL:  User query might 
> have needed to see row versions that must be removed.

It could be HOT chain pruning or an anti-wraparound autovacuum (which runs even 
if autovacuum is disabled).
Disabling autovacuum is not a smart idea to begin with.

Your best bet is to set "max_standby_streaming_delay = -1".

More reading:
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.cybertec-postgresql.com%2Fen%2Fstreaming-replication-conflicts-in-postgresql%2F=05%7C01%7Cabhishek.prakash08%40infosys.com%7Ce50f15f9ec4a497669a208dafacc8a3c%7C63ce7d592f3e42cda8ccbe764cff5eb6%7C0%7C0%7C638098053794261389%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=%2FlYwVKhkjP23vza5yhuJfw6mcOYynDVbNIhnKRBwUu4%3D=0

Yours,
Laurenz Albe

Re: ***Conflict with recovery error***

2023-01-20 Thread Laurenz Albe 
On Fri, 2023-01-20 at 08:56 +, Abhishek Prakash wrote:
> We are facing below issue with read replica we did work arounds by setting
> hot_standby_feedback, max_standby_streaming_delay and 
> max_standby_archive_delay,
> which indeed caused adverse effects on primary DB and storage. As our DB is
> nearly 6 TB which runs as AWS Postgres RDS. 
>  
> Even the below error occurs on tables where vacuum is disabled and no DML
> operations are permitted. Will there be any chances to see row versions
> being changed even if vacuum is disabled.
> Please advise.
>  
> 2023-01-13 07:20:12 
> UTC:10.64.103.75(61096):ubpreplica@ubprdb01:[17707]:ERROR:  canceling 
> statement due to conflict with recovery
> 2023-01-13 07:20:12 
> UTC:10.64.103.75(61096):ubpreplica@ubprdb01:[17707]:DETAIL:  User query might 
> have needed to see row versions that must be removed.

It could be HOT chain pruning or an anti-wraparound autovacuum (which runs
even if autovacuum is disabled).
Disabling autovacuum is not a smart idea to begin with.

Your best bet is to set "max_standby_streaming_delay = -1".

More reading:
https://www.cybertec-postgresql.com/en/streaming-replication-conflicts-in-postgresql/

Yours,
Laurenz Albe