Re: In-order pg_dump (or in-order COPY TO)

2025-08-26 Thread Adrian Klaver
On 8/26/25 12:43, Dimitrios Apostolou wrote: Hello list, I am storing dumps of a database (pg_dump custom format) in a de- duplicating backup server. Each dump is many terabytes in size, so deduplication is very important. And de-duplication itself is based on rolling checksums which is prett

Re: In-order pg_dump (or in-order COPY TO)

2025-08-26 Thread Tom Lane
Ron Johnson writes: > On Tue, Aug 26, 2025 at 6:08 PM Tom Lane wrote: >> I'd expect pg_dump/pg_restore to preserve the physical row ordering, >> simply because it doesn't do anything that would change that. > But the rolling checksums are against a pg_dump file, not a pg_basebackup > file. Oh,

Re: In-order pg_dump (or in-order COPY TO)

2025-08-26 Thread Tom Lane
Ron Johnson writes: > On Tue, Aug 26, 2025 at 4:31 PM David G. Johnston < > david.g.johns...@gmail.com> wrote: >> The logical dump has no ordering - it will come out however it comes out. >> "COPY TO ..." doesn't have an order by clause - there is no way to >> make or communicate to it that order

Re: In-order pg_dump (or in-order COPY TO)

2025-08-26 Thread Ron Johnson
On Tue, Aug 26, 2025 at 6:08 PM Tom Lane wrote: > Dimitrios Apostolou writes: > > Unfortunately after I did pg_restore to a new server, I notice that the > > dumps from the new server are not being de-duplicated, all blocks are > > considered new. > > > This means that the data has been signific

Re: In-order pg_dump (or in-order COPY TO)

2025-08-26 Thread Ron Johnson
On Tue, Aug 26, 2025 at 4:31 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Tue, Aug 26, 2025 at 12:43 PM Dimitrios Apostolou > wrote: > >> Could the >> row-order have changed when doing COPY FROM with pg_restore? > > > There is no reliable, meaningful, row ordering when it comes

Re: In-order pg_dump (or in-order COPY TO)

2025-08-26 Thread Tom Lane
Dimitrios Apostolou writes: > Unfortunately after I did pg_restore to a new server, I notice that the > dumps from the new server are not being de-duplicated, all blocks are > considered new. > This means that the data has been significantly altered. The new dumps > contain the same rows but pro

Re: In-order pg_dump (or in-order COPY TO)

2025-08-26 Thread Ron Johnson
On Tue, Aug 26, 2025 at 3:44 PM Dimitrios Apostolou wrote: > Hello list, > > I am storing dumps of a database (pg_dump custom format) in a > de-duplicating backup server. Each dump is many terabytes in size, so > deduplication is very important. And de-duplication itself is based on > rolling che

Re: In-order pg_dump (or in-order COPY TO)

2025-08-26 Thread David G. Johnston
On Tue, Aug 26, 2025 at 12:43 PM Dimitrios Apostolou wrote: > Could the > row-order have changed when doing COPY FROM with pg_restore? There is no reliable, meaningful, row ordering when it comes to the physical files. Sure, cluster does make an attempt, but it is quite limited in practice.

Re: How to configure client-side TLS ciphers for streaming replication?

2025-08-26 Thread Laurenz Albe
On Tue, 2025-08-26 at 20:34 +0800, xx Z wrote: > Thanks for your suggestion. > But I still want to know why we can't set "ssl_ciphers" on the client side. I'd say because nobody implemented it, perhaps because nobody felt the need. > This is still considered a security issue in some cases, and Po

In-order pg_dump (or in-order COPY TO)

2025-08-26 Thread Dimitrios Apostolou
Hello list, I am storing dumps of a database (pg_dump custom format) in a de-duplicating backup server. Each dump is many terabytes in size, so deduplication is very important. And de-duplication itself is based on rolling checksums which is pretty flexible, it can compensate for blocks movin

Re: Feature request: A method to configure client-side TLS ciphers for streaming replication

2025-08-26 Thread Ron Johnson
On Tue, Aug 26, 2025 at 9:09 AM xx Z wrote: > Hello, > Thank you for the reply and for the advice about our PostgreSQL version. > We will plan to update it. > To clarify what I meant by "standby (client)": In a streaming replication > setup, the standby server connects to the primary server to re

Re: How to configure client-side TLS ciphers for streaming replication?

2025-08-26 Thread DINESH NAIR
Hi , Found an article which might be of help, configuring through HAProxy as a TLS proxy to control cipher suites. https://stackoverflow.com/questions/53198588/how-to-disable-specific-cipher-suites-from-haproxy-can-i-do-this-ssl-default [https://cdn.sstatic.net/Sites/stackoverflow/Img/apple-tou

Re: DISABLE TRIGGER doc wrong?

2025-08-26 Thread Tom Lane
Ron Johnson writes: > Like I said, it's not completely linear. The real question, though, is > whether PG looks for gaps in oid allocation once it wants to try and > allocate an oid of uint32 max. OID counter wraparound is not particularly a problem. regards, tom lane

Re: How to configure client-side TLS ciphers for streaming replication?

2025-08-26 Thread Rob Sargent
> On Aug 26, 2025, at 5:35 AM, xx Z wrote: > >  > Thanks for your suggestion. > But I still want to know why we can't set "ssl_ciphers" on the client side. > This is still considered a security issue in some cases, and PostgreSQL has > mature capabilities on the master side to implement this

Re: Feature request: A method to configure client-side TLS ciphers for streaming replication

2025-08-26 Thread xx Z
Hello, Thank you for the reply and for the advice about our PostgreSQL version. We will plan to update it. To clarify what I meant by "standby (client)": In a streaming replication setup, the standby server connects to the primary server to receive data. In this specific network connection, the sta

Re: DISABLE TRIGGER doc wrong?

2025-08-26 Thread Ron Johnson
On Tue, Aug 26, 2025 at 9:01 AM Dominique Devienne wrote: > On Tue, Aug 26, 2025 at 2:54 PM Ron Johnson > wrote: > > On Tue, Aug 26, 2025 at 3:01 AM Dominique Devienne > wrote: > >> ERROR: permission denied: "RI_ConstraintTrigger_c_1226298044" is a > >> (yes, that's a large OID... For a 1 year

Re: DISABLE TRIGGER doc wrong?

2025-08-26 Thread Dominique Devienne
On Tue, Aug 26, 2025 at 2:54 PM Ron Johnson wrote: > On Tue, Aug 26, 2025 at 3:01 AM Dominique Devienne > wrote: >> ERROR: permission denied: "RI_ConstraintTrigger_c_1226298044" is a >> (yes, that's a large OID... For a 1 year old DB) > > PG's OID allocation of "user-land" OIDs doesn't start at

DMS error where postgres is the destination

2025-08-26 Thread Siraj G
Hello Experts! We are replicating data from Oracle to Postgres through GCP DMS. We are getting these errors in the DMS.. I do not think this is a postgres related issue, but trying to see if anyone can shed more light on this. Error: [DATABASE] recover row error: can't recover rollback row AABQb

Re: Feature request: A method to configure client-side TLS ciphers for streaming replication

2025-08-26 Thread Ron Johnson
On Tue, Aug 26, 2025 at 3:28 AM xx Z wrote: > Hello PostgreSQL community, > > I have a question regarding the configuration of streaming replication. > > When setting up streaming replication over TLS, I've noticed that while > the primary server can restrict its supported encryption algorithms u

Re: DISABLE TRIGGER doc wrong?

2025-08-26 Thread Ron Johnson
On Tue, Aug 26, 2025 at 3:01 AM Dominique Devienne wrote: [snip] > ERROR: permission denied: "RI_ConstraintTrigger_c_1226298044" is a > system trigger > > (yes, that's a large OID... For a 1 year old DB) > PG's OID allocation of "user-land" OIDs doesn't start at 16384 anymore. And it can seem q

Re: How to configure client-side TLS ciphers for streaming replication?

2025-08-26 Thread xx Z
Thanks for your suggestion. But I still want to know why we can't set "ssl_ciphers" on the client side. This is still considered a security issue in some cases, and PostgreSQL has mature capabilities on the master side to implement this functionality. Greetings, Yunfei Zhou Laurenz Albe 于2025年8月2

Re: How to configure client-side TLS ciphers for streaming replication?

2025-08-26 Thread Laurenz Albe
On Tue, 2025-08-26 at 19:48 +0800, xx Z wrote: > Is there a way for a streaming replication standby (client) to restrict its > list > of supported TLS ciphers, similar to how the ssl_ciphers parameter works on > the > primary server? > We need this for security compliance but can't find an equiva

How to configure client-side TLS ciphers for streaming replication?

2025-08-26 Thread xx Z
Hello, Is there a way for a streaming replication standby (client) to restrict its list of supported TLS ciphers, similar to how the ssl_ciphers parameter works on the primary server? We need this for security compliance but can't find an equivalent setting for the client-side connection in primary

Re: Strange deadlock with object/target of lock : transaction

2025-08-26 Thread Achilleas Mantzios
On 8/25/25 17:11, Achilleas Mantzios wrote: On 25/8/25 17:58, Adrian Klaver wrote: On 8/25/25 07:40, Achilleas Mantzios wrote: On 8/20/25 14:59, Achilleas Mantzios wrote: On 8/14/25 16:01, Achilleas Mantzios wrote: Hi Adrian On 8/14/25 15:39, Adrian Klaver wrote: On 8/14/25 00:07, Achi

Feature request: A method to configure client-side TLS ciphers for streaming replication

2025-08-26 Thread xx Z
Hello PostgreSQL community, I have a question regarding the configuration of streaming replication. When setting up streaming replication over TLS, I've noticed that while the primary server can restrict its supported encryption algorithms using the ssl_ciphers parameter, there doesn't seem to be

Re: Strange deadlock with object/target of lock : transaction

2025-08-26 Thread Achilleas Mantzios
On 8/26/25 07:22, Laurenz Albe wrote: On Mon, 2025-08-25 at 15:40 +0100, Achilleas Mantzios wrote: We've been hit by a weird deadlock which it took me some days to isolate and replicate. It does not have to do with order of updates or any explicit TABLE-level locking, the objects/targets of t

Re: DISABLE TRIGGER doc wrong?

2025-08-26 Thread Dominique Devienne
On Mon, Aug 25, 2025 at 7:33 PM Adrian Klaver wrote: > On 8/25/25 10:24, Dominique Devienne wrote: > > On Mon, Aug 25, 2025 at 7:13 PM Pavel Luzanov > > wrote: > >> On 25.08.2025 19:19, Dominique Devienne wrote: > >> Simple experiment shows that it is still up to date: Thanks Pavel. Indeed. See