Hi,

On Mon, Nov 03, 2025 at 06:51:41AM +0000, Daniele Paolo wrote:
> Sorry to bother you and for my poor English. I hope you understand.
> 
> I work with Debian 12 and the following products:
> 
>   *   patroni 4.0.7
>   *   etcd Version: 3.4.23
>   *   pgBackRest 2.56.0
>   *   postgresql 17
> 
> In the latest DEBIAN update, the products have been updated to:

Well, it was an update of the PostgreSQL community packages from
apt.postgresql.org (Debian 12 has patroni 3.0.2-1), but opening a bug
here is fine.

> Before migration :
> + Cluster: PG1 (7537608679052463361) ----+-----------+----+-----------+
> | Member     | Host            | Role    | State     | TL | Lag in MB |
> +------------+-----------------+---------+-----------+----+-----------+
> | vdn-pg-t11 | 148.196.186.106 | Leader  | running   | 47 |           |
> | vdn-pg-t12 | 148.196.186.107 | Replica | streaming | 47 |         0 |
> +------------+-----------------+---------+-----------+----+-----------+

You run etcd on the local nodes apparantly, right?

> root@vdn-pg-t11:~# apt upgrade -y
> Reading package lists... Done
> Building dependency tree... Done
> Reading state information... Done
> Calculating upgrade... Done
> The following packages will be upgraded:
>   bind9-dnsutils bind9-host bind9-libs intel-microcode patroni pgbackrest
[...]
> Setting up patroni (4.1.0-1.pgdg12+1) ...
> Installing new version of config file /etc/patroni/config.yml.in ...
> Job for patroni.service failed because a timeout was exceeded.
> See "systemctl status patroni.service" and "journalctl -xeu patroni.service" 
> for details.
> invoke-rc.d: initscript patroni, action "restart" failed.
> ● patroni.service - Runners to orchestrate a high-availability PostgreSQL
>      Loaded: loaded (/lib/systemd/system/patroni.service; enabled; preset: 
> enabled)
>      Active: activating (auto-restart) (Result: timeout) since Thu 2025-10-30 
> 09:21:19 CET; 6ms ago
>     Process: 8164 ExecStart=/usr/bin/patroni /etc/patroni/config.yml 
> (code=exited, status=0/SUCCESS)
>    Main PID: 8164 (code=exited, status=0/SUCCESS)
>         CPU: 1.478s
> dpkg: error processing package patroni (--configure):
> installed patroni package post-installation script subprocess returned error 
> exit status 1
[...]
> Errors were encountered while processing:
> patroni
> E: Sub-process /usr/bin/dpkg returned an error code (1)
> 
> After migration :
> 
> root@vdn-pg-t11:~# patronictl list
> + Cluster: PG1 (7537608679052463361) 
> ----+-----------+----+-------------+-----+-------------+-----+
> | Member     | Host            | Role    | State     | TL | Receive LSN | Lag 
> |  Replay LSN | Lag |
> +------------+-----------------+---------+-----------+----+-------------+-----+-------------+-----+
> | vdn-pg-t11 | 148.196.186.106 | Replica | streaming | 48 | 6C/B60001E0 |   0 
> | 6C/B60001E0 |   0 |
> | vdn-pg-t12 | 148.196.186.107 | Leader  | running   | 48 |             |     
> |             |     |
> +------------+-----------------+---------+-----------+----+-------------+-----+-------------+-----+
> …
> root@vdn-pg-t11:~# patronictl list
> + Cluster: PG1 (7537608679052463361) 
> ----+-----------+----+-------------+-----+-------------+-----+
> | Member     | Host            | Role    | State     | TL | Receive LSN | Lag 
> |  Replay LSN | Lag |
> +------------+-----------------+---------+-----------+----+-------------+-----+-------------+-----+
> | vdn-pg-t11 | 148.196.186.106 | Replica | streaming | 69 | 6C/CA0001E0 |   0 
> | 6C/CA0001E0 |   0 |
> | vdn-pg-t12 | 148.196.186.107 | Leader  | running   | 69 |             |     
> |             |     |
> +------------+-----------------+---------+-----------+----+-------------+-----+-------------+-----+
> …
> root@vdn-pg-t11:~# patronictl list
> + Cluster: PG1 (7537608679052463361) 
> ----+-----------+-----+-------------+-----+-------------+-----+
> | Member     | Host            | Role    | State     |  TL | Receive LSN | 
> Lag |  Replay LSN | Lag |
> +------------+-----------------+---------+-----------+-----+-------------+-----+-------------+-----+
> | vdn-pg-t11 | 148.196.186.106 | Replica | streaming | 277 | 6D/980001A8 |   
> 0 | 6D/980001A8 |   0 |
> | vdn-pg-t12 | 148.196.186.107 | Leader  | running   | 277 |             |    
>  |             |     |
> +------------+-----------------+---------+-----------+-----+-------------+-----+-------------+-----+
> 
> The various logs are attached.

You only attached logs for one of the machines (vdn-pg-t11), right? Can
you also attach the ones for the other?

It is hard to figure out what went wrong from them, but the beginning of
the patroni logs has this:

|2025-10-30 09:27:20,725 ERROR: Request to server http://148.196.186.106:2379 
failed:ReadTimeoutError("HTTPConnectionPool(host='148.196.186.106', port=2379): 
Read timed out. (read timeout=3.3328031433333343)")

Which sounds like an etcd problem. Did you try to install etcd-client
and run the following:

etcdctl member list
etcdctl endpoint status
etcdctl endpoint health

It also looks like pgbackrest is very unhappy:

|2025-10-30 10:51:59.912 P00  ERROR: [045]: WAL file '000000C40000006D00000048' 
already exists in the repo1 archive with a different checksum
|2025-10-30 10:52:00.013 CET - 3 - 28698 -  - @ - 0LOG:  archive command failed 
with exit code 45
|2025-10-30 10:52:00.013 CET - 4 - 28698 -  - @ - 0DETAIL:  The failed archive 
command was: pgbackrest --stanza=stanza_PG archive-push 
/db/PG1/pgdata/pg_wal/000000C40000006D00000048

In general, I am not (yet) convinced this is a problem with Patroni 4.1.0,
there's not enough logs to say something for now.


Cheers,

Michael

Reply via email to