pg_basebackup "Permission denied" error when initiating streaming replication

2024-05-07 Thread Frans Simmelvuo (Papula-Nevinpat)
, the problem I'm encountering is the following: I call pg_basebackup at the target as follows: >> pg_basebackup -h -U -p -W -D -T >> "C:\Server >> Files\Data\ProgramsData\PostgreSQL\10\data\base\demo_tablespace"=E:\PG_tablespace_data\10\demo_tablespace >

Re: pg_basebackup Restore problem

2024-01-17 Thread Johnathan Tiamoh
R/*.tar.gz | xargs -I {} tar -h -zxvf {} >>> >>> On Wed, Jan 17, 2024 at 4:07 PM Johnathan Tiamoh < >>> johnathantia...@gmail.com> wrote: >>> >>>> Yes. >>>> >>>> I am trying to restore the backups on a standby >>&

Re: pg_basebackup Restore problem

2024-01-17 Thread Ron Johnson
t;> >>> I am trying to restore the backups on a standby >>> >>> On Wed, Jan 17, 2024 at 4:04 PM Ron Johnson >>> wrote: >>> >>>> Wait a minute... *tar* is throwing the errors, not pg_basebackup, no? >>>> >>>> On We

Re: pg_basebackup Restore problem

2024-01-17 Thread Johnathan Tiamoh
ohnathan Tiamoh < > johnathantia...@gmail.com> wrote: > >> Yes. >> >> I am trying to restore the backups on a standby >> >> On Wed, Jan 17, 2024 at 4:04 PM Ron Johnson >> wrote: >> >>> Wait a minute... *tar* is throwing the errors, n

Re: pg_basebackup Restore problem

2024-01-17 Thread Ron Johnson
at 4:04 PM Ron Johnson > wrote: > >> Wait a minute... *tar* is throwing the errors, not pg_basebackup, no? >> >> On Wed, Jan 17, 2024 at 3:27 PM Johnathan Tiamoh < >> johnathantia...@gmail.com> wrote: >> >>> 1. What's in $PATH_FOLDER? >>>

Re: pg_basebackup Restore problem

2024-01-17 Thread Johnathan Tiamoh
Yes. I am trying to restore the backups on a standby On Wed, Jan 17, 2024 at 4:04 PM Ron Johnson wrote: > Wait a minute... *tar* is throwing the errors, not pg_basebackup, no? > > On Wed, Jan 17, 2024 at 3:27 PM Johnathan Tiamoh < > johnathantia...@gmail.com> wro

Re: pg_basebackup Restore problem

2024-01-17 Thread Ron Johnson
Wait a minute... *tar* is throwing the errors, not pg_basebackup, no? On Wed, Jan 17, 2024 at 3:27 PM Johnathan Tiamoh wrote: > 1. What's in $PATH_FOLDER? > > /tnt/backup/current > > 2. What pg_basebackup command did you use? > > pg_basebackup -D "$

Re: pg_basebackup Restore problem

2024-01-17 Thread Johnathan Tiamoh
1. What's in $PATH_FOLDER? /tnt/backup/current 2. What pg_basebackup command did you use? pg_basebackup -D "$baseback_dir" --format=tar \ "${comp_opts[@]}" --wal-method=stream --no-password --verbose "${PG_DUMP_OPTS[@]}" /bin/mv "$b

Re: pg_basebackup Restore problem

2024-01-17 Thread Ron Johnson
1. What's in $PATH_FOLDER? 2. What pg_basebackup command did you use? 3. Why aren't you letting pg_basebackup maintain the WAL files it needs? On Wed, Jan 17, 2024 at 11:27 AM Johnathan Tiamoh wrote: > You need to tell us the PG version number *and* show us the full command > y

Re: pg_basebackup Restore problem

2024-01-17 Thread Johnathan Tiamoh
You need to tell us the PG version number *and* show us the full command you ran. ? Postgresql Version 14.10 tar -h -zxvf $PATH_FOLDER/* .tar.gz ls $PATH_FOLDER/*.tar.gz | xargs -I {} tar -h -zxvf {} What user are you running pg_basebackup as? I ran it as postgres and now I'm restoring

Re: pg_basebackup Restore problem

2024-01-17 Thread Ron Johnson
On Wed, Jan 17, 2024 at 11:11 AM Johnathan Tiamoh wrote: > Hello, > > I am trying to restore a pg_basebackup and have the following errors. > > You need to tell us the PG version number *and* show us the full command you ran. > nohup: ignoring input > tar: /tnt/backu

pg_basebackup Restore problem

2024-01-17 Thread Johnathan Tiamoh
Hello, I am trying to restore a pg_basebackup and have the following errors. nohup: ignoring input tar: /tnt/backup/current/7400.tar.gz: Not found in archive tar: /tnt/backup/current/7401.tar.gz: Not found in archive tar: /tnt/backup/current/7402.tar.gz: Not found in archive tar: /tnt/backup

Re: pg_basebackup

2023-11-20 Thread Christoph Moench-Tegeder
## Matthias Apitz (g...@unixarea.de): > 2023-11-16 20:34:13.538 CET [6250] LOG: terminating walsender process due to > replication timeout Besides "what Lauenz said" (especially about the horribly ooutdated PostgreSQL version): check IO speed and saturation during backup and make sure you're

Re: pg_basebackup

2023-11-20 Thread Laurenz Albe
On Mon, 2023-11-20 at 07:30 +0100, Matthias Apitz wrote: > We're facing in a customer installation (PostgreSQL 13.1 on Linux) the > following problem for the first time and not reproducible: 13.1? Your immediate reaction should be "update to the latest minor release". > ${BI

pg_basebackup

2023-11-19 Thread Matthias Apitz
the current log; use a DB which will exist; # psql -U ${DBSUSER} -dpostgres -c "select pg_switch_wal();" > /dev/null # backup the cluster # printf "%s: pg_basebackup the cluster to %s ... " "`date "+%d.%m.%Y-%H:%M:%S"`" ${BACKUPDIR}-${DATE}-${NUM} ${BI

Re: pg_basebackup / recovery

2023-04-12 Thread Achilleas Mantzios
Στις 13/4/23 01:31, ο/η Michael Paquier έγραψε: On Wed, Apr 12, 2023 at 01:45:56PM +0300, Achilleas Mantzios - cloud wrote: On 4/12/23 12:32, Fabrice Chapuis wrote: During recovery process of a self contained backup, how postgres know to stop reading wal when consistency is reached? Because

Re: pg_basebackup / recovery

2023-04-12 Thread Michael Paquier
On Wed, Apr 12, 2023 at 01:45:56PM +0300, Achilleas Mantzios - cloud wrote: > On 4/12/23 12:32, Fabrice Chapuis wrote: >> During recovery process of a self contained backup, how postgres know to >> stop reading wal when consistency is reached? > > Because it knows the full packup info. It will

Re: pg_basebackup / recovery

2023-04-12 Thread Achilleas Mantzios - cloud
On 4/12/23 12:32, Fabrice Chapuis wrote: During recovery process of a self contained backup, how postgres know to stop reading wal when consistency is reached? Because it knows the full packup info. It will observe the STOP WAL LOCATION: 3BC7/4B000130 (file 00023BC7004B) inside

pg_basebackup / recovery

2023-04-12 Thread Fabrice Chapuis
During recovery process of a self contained backup, how postgres know to stop reading wal when consistency is reached?

Re: postgres replication without pg_basebackup? postgres 13.3

2022-11-07 Thread Laurenz Albe
22:57:55 CET-]LOG:  database system is shut down > > We have read we can run pg_resetwal but the Wal folder is the folder where > the Wals are archived in the primary. Would that be correct to reset them? Don't run "pg_resetwal". At the very least, it will break your standby. At this point, your standby seems to be broken. I don't know what exactly you did, but it leeks like you should run a "pg_basebackup" after all. Yours, Laurenz Albe

Re: postgres replication without pg_basebackup? postgres 13.3

2022-11-07 Thread Pilar de Teodoro
some disk failures. Now the issues are solved, we would like to make > standby the old primary > > from a copy of the new primary which is already in place. Is it > possible without using pg_basebackup? > > That's exactly what "pg_rewind" is for. It is a fast version of &

Re: postgres replication without pg_basebackup? postgres 13.3

2022-11-07 Thread Laurenz Albe
w primary which is already in place.  Is it possible > without using pg_basebackup? That's exactly what "pg_rewind" is for. It is a fast version of "pg_basebackup" for exactly that case. Yours, Laurenz Albe -- Cybertec | https://www.cybertec-postgresql.com

postgres replication without pg_basebackup? postgres 13.3

2022-11-07 Thread Pilar de Teodoro
Dear all, We have a very large database of 37TB and we had to promote our standby to primary due to some disk failures. Now the issues are solved, we would like to make standby the old primary from a copy of the new primary which is already in place. Is it possible without using pg_basebackup

Re: About pg_basebackup

2022-05-19 Thread Laurenz Albe
On Thu, 2022-05-19 at 13:18 +0900, 菊池祐 wrote: > I executed the pg_basebackup command to set up a replication configuration in > postgresql, > but the following message appears and replication didn’t complete. > > 2741/2742 tablespaces (/var/lib/pgsql/9.2/bac12609150596/1260915

About pg_basebackup

2022-05-18 Thread 菊池祐
Hi,I executed the pg_basebackup command to set up a replication configuration in postgresql, but the following message appears and replication didn’t complete.2741/2742 tablespaces (/var/lib/pgsql/9.2/bac12609150596/12609150596 kB (100%), 2741/2742 tablespaces (/var/lib/pgsql/9.2/bac12609150596

Re: pg_basebackup with hostssl ?

2022-02-02 Thread Laura Smith
Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Wednesday, February 2nd, 2022 at 17:20, Adrian Klaver wrote: > > Before you do that I would establish that you are connecting to the > > correct Postgres instance. > > Good news, all up and running ! The new

Re: pg_basebackup with hostssl ?

2022-02-02 Thread Adrian Klaver
On 2/2/22 09:18, Laura Smith wrote: Seems like I was looking for the wrong words on the right page ! Thanks, will try that. Before you do that I would establish that you are connecting to the correct Postgres instance. -- Adrian Klaver adrian.kla...@aklaver.com

Re: pg_basebackup with hostssl ?

2022-02-02 Thread Laura Smith
; Specifies parameters used to connect to the server, as a > > ; these will override any conflicting command line > > options. > > The option is called --dbname for consistency with other client > > applications, but because pg_basebackup doesn't connect to any > &

Re: pg_basebackup with hostssl ?

2022-02-02 Thread David G. Johnston
On Wed, Feb 2, 2022 at 9:37 AM Laura Smith < n5d9xq3ti233xiyif...@protonmail.ch> wrote: > The point I am making is that pg_basebackup is connecting without > encryption and I want to know how to tell it to use encryption. The docs > are silent on the subject. > It is a

Re: pg_basebackup with hostssl ?

2022-02-02 Thread Adrian Klaver
On 2/2/22 08:37, Laura Smith wrote: Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Wednesday, February 2nd, 2022 at 16:30, Adrian Klaver wrote: I am not following. pg_basebackup is a stand alone client that is not involved in replication other then establishing

Re: pg_basebackup with hostssl ?

2022-02-02 Thread Laura Smith
Forgot to add that I also have : "hostsslallall10.0.0.0/8md5"

Re: pg_basebackup with hostssl ?

2022-02-02 Thread Laura Smith
Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Wednesday, February 2nd, 2022 at 16:30, Adrian Klaver wrote: > I am not following. pg_basebackup is a stand alone client that is not > > involved in replication other then establishing a starting point from

Re: pg_basebackup with hostssl ?

2022-02-02 Thread Adrian Klaver
On 2/2/22 07:48, Laura Smith wrote: I've had a quick glance through the man page for pg_basebackup but can't see any flags to set ssl. Long story short, I've got the following in pg_hba.conf of the master: hostsslreplicationall10.0.0.0/8md5 But the slave is complaining: FATAL

pg_basebackup with hostssl ?

2022-02-02 Thread Laura Smith
I've had a quick glance through the man page for pg_basebackup but can't see any flags to set ssl. Long story short, I've got the following in pg_hba.conf of the master: hostsslreplicationall10.0.0.0/8md5 But the slave is complaining: FATAL: no pg_hba.conf entry for replication

Re: Issue with pg_basebackup v.11

2021-10-25 Thread Ninad Shah
Thanks Tom. Regards, Ninad Shah On Sat, 23 Oct 2021 at 20:12, Tom Lane wrote: > Ninad Shah writes: > > Would keepalive setting address and mitigate the issue? > > [ shrug... ] Maybe; nobody else has more information about this > situation than you do. I suggested something to experiment

Re: Issue with pg_basebackup v.11

2021-10-23 Thread Tom Lane
Ninad Shah writes: > Would keepalive setting address and mitigate the issue? [ shrug... ] Maybe; nobody else has more information about this situation than you do. I suggested something to experiment with. regards, tom lane

Re: Issue with pg_basebackup v.11

2021-10-22 Thread Ninad Shah
Hey Tom, Thank you for your response. Actually, when we copy data using scp/rsync, it works without any issue. But, it fails while attempting to transfer using pg_basebackup. Would keepalive setting address and mitigate the issue? Regards, Ninad Shah On Fri, 22 Oct 2021 at 21:39, Tom Lane

Re: Issue with pg_basebackup v.11

2021-10-22 Thread Tom Lane
ed out We have heard reports of network connections dropping while pg_basebackup is busy doing something disk-intensive such as fsync'ing. The apparent 2-hour delay here does not mean that pg_basebackup was out to lunch for 2 hours; more likely that reflects the TCP timeout delay before the

Issue with pg_basebackup v.11

2021-10-22 Thread Ninad Shah
Hello experts, I am facing an issue with a customer's production server while trying to take backup using pg_basebackup. Below is the log from pg_basebackup execution. * 115338208/1304172127 kB (8%), 0/1 tablespace (...atastaging/base/115868/154220.1) 115355616/1304172127 kB (8%), 0/1

Re: Problem in pg_basebackup of 1.7 TB database while replicating in PostgreSQL 9.4.( Master and slave both are in postgreSQL 9.4)

2021-10-04 Thread Ron
On 10/4/21 2:28 AM, Amal Chakravarty wrote: Hi all. I am working with a database of 1.7 TB size which is in PostgreSQL 9.4. Kindly suggest how to take the pg_basebackup of such a huge data while replicating from master to slave. Obligatory "9.4 is EOL" comment. -- Angular mome

Re: Problem in pg_basebackup of 1.7 TB database while replicating in PostgreSQL 9.4.( Master and slave both are in postgreSQL 9.4)

2021-10-04 Thread Tomas Vondra
On 10/4/21 9:28 AM, Amal Chakravarty wrote: Hi all. I am working with a database of 1.7 TB size which is in PostgreSQL 9.4. Kindly suggest how to take the pg_basebackup of such a huge data while replicating from master to slave. And what exactly is the issue? Does it fail in some way

Problem in pg_basebackup of 1.7 TB database while replicating in PostgreSQL 9.4.( Master and slave both are in postgreSQL 9.4)

2021-10-04 Thread Amal Chakravarty
Hi all. I am working with a database of 1.7 TB size which is in PostgreSQL 9.4. Kindly suggest how to take the pg_basebackup of such a huge data while replicating from master to slave. Regards, Amal Chakravarty.

Re: pg_basebackup fails with "COPY stream ended"

2021-06-15 Thread Julien Rouhaud
On Tue, Jun 15, 2021 at 09:53:45PM -0700, Dipanjan Das wrote: > > I am running "pg_basebackup -h -U postgres -D -X stream". It > fails with either of the following two error messages: > [...] > WARNING: terminating connection because of crash of another server process

pg_basebackup fails with "COPY stream ended"

2021-06-15 Thread Dipanjan Das
Hi, I am running "pg_basebackup -h -U postgres -D -X stream". It fails with either of the following two error messages: ERROR: Backup failed copying files. DETAILS: data transfer failure on directory '/mnt/data/barman/base/20210615T212304/data' pg_basebackup error: pg_basebackup:

Re: "missing chunk number 0 for toast value xxx in pg_toast_xxx" when pg_basebackup

2021-04-13 Thread Laurenz Albe
On Tue, 2021-04-13 at 06:36 -0700, MaXinjian wrote: > > Why do you write the WAL to /tmp/pg_wal, only to later mount that at the > > default location? > > pg_wal dir has size limitation, if wal files are too large, they will be > overwrited, right? No, they won't. You could run out of space on

Re: "missing chunk number 0 for toast value xxx in pg_toast_xxx" when pg_basebackup

2021-04-13 Thread Tom Lane
Ma Xinjian writes: > When I use pg_basebackup to backup and restore db(Let's call it A) to a > standalone instance(Let's call it B), "missing chunk number 0 for toast > value xxx in pg_toast_xxx" errors output. > PG version: 10.3 10.3 is quite a few bug fixes ago. M

Re: "missing chunk number 0 for toast value xxx in pg_toast_xxx" when pg_basebackup

2021-04-13 Thread MaXinjian
> Why do you write the WAL to /tmp/pg_wal, only to later mount that at the > default location? pg_wal dir has size limitation, if wal files are too large, they will be overwrited, right? > I see nothing wrong with what you are doing, but I may have got lost in > your complicated procedure. > You

Re: "missing chunk number 0 for toast value xxx in pg_toast_xxx" when pg_basebackup

2021-04-13 Thread Laurenz Albe
On Tue, 2021-04-13 at 02:38 -0700, Ma Xinjian wrote: > When I use pg_basebackup to backup and restore db(Let's call it A) to a > standalone instance(Let's call it B), "missing chunk number 0 for toast > value xxx in pg_toast_xxx" errors output. > > PG version: 10.

"missing chunk number 0 for toast value xxx in pg_toast_xxx" when pg_basebackup

2021-04-13 Thread Ma Xinjian
Hi, When I use pg_basebackup to backup and restore db(Let's call it A) to a standalone instance(Let's call it B), "missing chunk number 0 for toast value xxx in pg_toast_xxx" errors output. PG version: 10.3 pg_basebackup command: /usr/pgsql-10/bin/pg_basebackup -h p-rdb-c01 -

Re: "invalid record length" after restoring pg_basebackup

2020-11-16 Thread Tom Lane
cannot be continued due to the broken WAL record? There is no "broken WAL record". There is only junk following the primary's current WAL write point. > What I would expect is that pg_basebackup only transfer healthy WAL > entries so that a restored system can pick up right after t

Re: "invalid record length" after restoring pg_basebackup

2020-11-16 Thread Dennis Jacobfeuerborn
. But the system the backup was pulled from kept running fine. Also wouldn't that make re-attaching the system to the primary impossible since replication cannot be continued due to the broken WAL record? What I would expect is that pg_basebackup only transfer healthy WAL entries so that a restored s

Re: "invalid record length" after restoring pg_basebackup

2020-11-13 Thread Tom Lane
Dennis Jacobfeuerborn writes: > All of this works fine and the logs report that the db reaches a > consistent recovery state but as last entry it reports an "invalid > record length": This looks quite normal to me. If you'd pulled the power plug on the primary system at the time you made this

"invalid record length" after restoring pg_basebackup

2020-11-13 Thread Dennis Jacobfeuerborn
Hi, I've run into a strange issue after restoring a backup that I created using pg_basebackup on a standby instance. The command I use to create the backup is this: pg_basebackup -v --write-recovery-conf -h$BACKUP_HOST -p5432 -U$BACKUP_USER --format tar --wal-method stream --compress=2 -D

Re: pg_basebackup + delta base backups

2020-08-15 Thread Stephen Frost
delta capability. > > If I understood correctly the method you described, you were basically doing > a "backup" between A (primary) and B (repo) and in such a way the repo is > then compatible with the pg_data structure, but without delta support (ie. > transfering the whole

Re: canceling statement due to conflict with recovery after pg_basebackup

2020-06-03 Thread Kyotaro Horiguchi
At Wed, 3 Jun 2020 10:07:14 +0300, "Andrus" wrote in > Hi! Hi. > Async binary replication hot standby was started after pg_basebackup. > Running query in slave throws error > > ERROR: canceling statement due to conflict with recovery > > Why ? As written i

canceling statement due to conflict with recovery after pg_basebackup

2020-06-03 Thread Andrus
Hi! Async binary replication hot standby was started after pg_basebackup. Running query in slave throws error ERROR: canceling statement due to conflict with recovery Why ? Query should return table and other sizes in decreasing order. How to improve it so that this error does not occur

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-06-01 Thread Paul Förster
ter is empty, from initdb. Backup is not needed then why the mv stuff? > pg_basebackup uses environment varuables if not specified in command line. So > my script does the same thing. have you looked at the other options? > I tried > sudo --user=postgres pg_basebackup > but g

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-06-01 Thread Andrus
Hi! How to create replication server ? I always do it this way and it work for me: $ pg_basebackup -h ${PGHOST} -p ${PGPORT} -U replicator -W -R -D ${PGDATA} -P -v -Fp -Xs After that, I edit ${PGDATA}/postgresql.conf and (w/ PostgreSQL 11 and older ${PGDATA}/recovery.conf) to make it do what

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-06-01 Thread Magnus Hagander
On Mon, Jun 1, 2020 at 10:17 AM Andrus wrote: > Hi! > > > I have tried to re-initiate replica serveral times in low-use time but > this error occurs again. > >remove the whole replica's PGDATA/* and do a pg_basebackup again. But > before that, make sure wal_kee

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-06-01 Thread Paul Förster
Hi Andrus, > On 01. Jun, 2020, at 10:17, Andrus wrote: > Shell script starts server after pg_basebackup completes automatically: > > PGHOST=example.com > PGPASSWORD=mypass > PGUSER=replikaator > export PGHOST PGPASSWORD PGUSER > /etc/init.d/postgresql stop > mv /var/

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Andrus
Hi! I want to create hot standby async server using /etc/init.d/postgresql stop mv /var/lib/postgresql/12/main /var/lib/postgresql/12/mainold pg_basebackup --write-recovery-conf -D /var/lib/postgresql/12/main I don't see where the base backup is being taken from just where it is going

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Adrian Klaver
On 5/31/20 2:43 PM, Andrus wrote: Hi! In addition to my most recent questions: What are you trying to achieve? I want to create  hot standby async server using /etc/init.d/postgresql stop mv /var/lib/postgresql/12/main /var/lib/postgresql/12/mainold pg_basebackup --write-recovery-conf -D

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Andrus
. To really answer this we will need to see the exact commands you are using and the sequence they are done in. Replication server is created using /etc/init.d/postgresql stop mv /var/lib/postgresql/12/main /var/lib/postgresql/12/mainold pg_basebackup --write-recovery-conf -D /var/lib/postgresql

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Andrus
Hi! In addition to my most recent questions: What are you trying to achieve? I want to create hot standby async server using /etc/init.d/postgresql stop mv /var/lib/postgresql/12/main /var/lib/postgresql/12/mainold pg_basebackup --write-recovery-conf -D /var/lib/postgresql/12/main chmod

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Paul Förster
ve the whole replica's PGDATA/* and do a pg_basebackup again. But before that, make sure wal_keep_segments in big enough on the master and, just as much important, do a vacuumdb -a (takes much space during the process) and use archiving! > If named replication slot is used commands like > vac

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Adrian Klaver
most recent questions: What are you trying to achieve? In other words why do a pg_basebackup if you have a standby receiving WALs? Andrus. -- Adrian Klaver adrian.kla...@aklaver.com

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Adrian Klaver
On 5/31/20 2:03 PM, Andrus wrote: Hi! I'm guessing are looking for: https://www.postgresql.org/docs/12/runtime-config-replication.html 26.2.6. Replication Slots Replication slots provide an automated way to ensure that the master does not remove WAL segments until they have been received by

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Andrus
Hi! I'm guessing are looking for: https://www.postgresql.org/docs/12/runtime-config-replication.html 26.2.6. Replication Slots Replication slots provide an automated way to ensure that the master does not remove WAL segments until they have been received by all standbys, and that the master

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Paul Förster
Hi Andrus, > On 31. May, 2020, at 21:47, Andrus wrote: > replikaator@[unknown] ERROR: requested WAL segment 000102CF00E9 > has already been removed the message says it all. You need to copy the WAL file 000102CF00E9 and newer to the replica's pg_wal directory because

Re: How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Adrian Klaver
On 5/31/20 12:47 PM, Andrus wrote: Hi! pg_basebackup takes 8 hours. After it is finished,  replication slave does not start: LOG:  consistent recovery state reached at 2DE/985A5BE0 LOG:  database system is ready to accept read only connections LOG:  started streaming WAL from primary at 2DE

How to start slave after pg_basebackup. Why min_wal_size and wal_keep_segments are duplicated

2020-05-31 Thread Andrus
Hi! pg_basebackup takes 8 hours. After it is finished, replication slave does not start: LOG: consistent recovery state reached at 2DE/985A5BE0 LOG: database system is ready to accept read only connections LOG: started streaming WAL from primary at 2DE/9900 on timeline 1 replikaator

Re: pg_basebackup + delta base backups

2020-05-27 Thread Christopher Pereira
ble with the pg_data structure, but without delta support (ie. transfering the whole database)? Delta support is critical for VLDBs, so I see two alternatives to replace pg_basebackup with pgbackrest to rebuild a replica: 1) Create a temporary repo on the primary 2) Create a temporary repo on the rep

Re: pg_basebackup + incremental base backups

2020-05-26 Thread Stephen Frost
the replica is promoted and you want to remaster the former primary. > We did some tests measuring IO and traffic and are very happy with the > results. We will definitely be adding pgBackRest to our toolchain. Glad to hear that. > Regarding my initial question, I still believe that the

Re: pg_basebackup + incremental base backups

2020-05-25 Thread Christopher Pereira
will definitely be adding pgBackRest to our toolchain. Regarding my initial question, I still believe that the world deserves a simple direct pg_basebackup replacement even when putting an additional "repo host" in the middle is a better idea in the long term. As you said, all

Re: Query returns no rows in pg_basebackup cluster

2020-05-25 Thread Andrus
Hi! How to set logical replication for all user databases in cluster so that when new database is added or new tables are added to database they will start replicate automatically ? I think that it would be good if you spend some time reading the documentation on this stuff, particularly the

Re: Query returns no rows in pg_basebackup cluster

2020-05-25 Thread David G. Johnston
On Sunday, May 24, 2020, Andrus wrote: > Hi! > > Backup in created in Windows from Linux server using pg_receivewal and >>> pg_basebackup . >>> Can this backup used for PITR in Linux ? >>> >> No. Physical copies need to be based on the same platfo

Re: Query returns no rows in pg_basebackup cluster

2020-05-25 Thread Michael Paquier
On Mon, May 25, 2020 at 09:02:49AM +0300, Andrus wrote: > Will logical replication also allow two modes: > 1. PITR recovery can used if needed > 2. Hot standby: User databases in both clusters contain same data. > > How to set logical replication for all user databases in cluster so that > when

Re: Query returns no rows in pg_basebackup cluster

2020-05-25 Thread Andrus
Hi! No. Physical copies need to be based on the same platform. Does the O/S that the client software runs on really affect this? To the extent that the O/S determines text sort order, yes; see thread. The short answer here is that we aren't going to support such cases. If you try to

Re: Query returns no rows in pg_basebackup cluster

2020-05-25 Thread Andrus
Hi! Backup in created in Windows from Linux server using pg_receivewal and pg_basebackup . Can this backup used for PITR in Linux ? No. Physical copies need to be based on the same platform. If you wish to replicate a cluster without any platform, architecture or even not-too-many major

Re: Query returns no rows in pg_basebackup cluster

2020-05-24 Thread Tom Lane
"David G. Johnston" writes: > On Sun, May 24, 2020 at 4:10 PM Michael Paquier wrote: >> No. Physical copies need to be based on the same platform. > Does the O/S that the client software runs on really affect this? To the extent that the O/S determines text sort order, yes; see thread. The

Re: Query returns no rows in pg_basebackup cluster

2020-05-24 Thread David G. Johnston
On Sun, May 24, 2020 at 4:10 PM Michael Paquier wrote: > On Fri, May 22, 2020 at 09:53:37AM +0300, Andrus wrote: > > Backup in created in Windows from Linux server using pg_receivewal and > pg_basebackup . > > Can this backup used for PITR in Linux ? > > No. Physical

Re: Query returns no rows in pg_basebackup cluster

2020-05-24 Thread Michael Paquier
On Fri, May 22, 2020 at 09:53:37AM +0300, Andrus wrote: > Backup in created in Windows from Linux server using pg_receivewal and > pg_basebackup . > Can this backup used for PITR in Linux ? No. Physical copies need to be based on the same platform. If you wish to replicate a cluste

Re: pg_basebackup + incremental base backups

2020-05-24 Thread Stephen Frost
Greetings, * Christopher Pereira (krip...@imatronix.cl) wrote: > Ok, we want to use pgbackrest to *rebuild a standby that has fallen behind* > (where pg_rewind won't work). After reading the docs, we believe we should > use this setup: > > a) Primary host: primary cluster > > b) Repository

Re: pg_basebackup + incremental base backups

2020-05-24 Thread Christopher Pereira
We've contemplated adding support for something like this to pgbackrest, since all the pieces are there, but there hasn't been a lot of demand for it and it kind of goes against the idea of having a proper backup solution, really.. It'd also create quite a bit of load on the primary to

Re: Query returns no rows in pg_basebackup cluster

2020-05-22 Thread Andrus
Hi! ERROR: item order invariant violated for index "desktop_baas_liigid_idx" DETAIL: Lower index tid=(3,15) (points to index tid=(16,4098)) higher index tid=(3,16) (points to index tid=(17,4098)) page lsn=292/630C0CE8. SQL state: XX002 Uh huh ... and I'll bet the same test on the source

Re: Query returns no rows in pg_basebackup cluster

2020-05-22 Thread Tom Lane
"Andrus" writes: >> Possibly you could try running contrib/amcheck on the index in question >> and see if it reports any issues. > I tried and it reports error > ERROR: item order invariant violated for index "desktop_baas_liigid_idx" > DETAIL: Lower index tid=(3,15) (points to index

Re: Query returns no rows in pg_basebackup cluster

2020-05-22 Thread Andrus
Hi! The sorting rules for this locale must be the same in both platforms. Only locale names are different. I think they are less alike than you hoped, because if they were alike, you wouldn't be seeing this problem. Possibly you could try running contrib/amcheck on the index in question and

Re: Query returns no rows in pg_basebackup cluster

2020-05-22 Thread Tom Lane
"Andrus" writes: >> No, what it sounds like is the OP tried to physically replicate a >> database on another platform with completely different sorting rules. > The sorting rules for this locale must be the same in both platforms. > Only locale names are different. I think they are less alike

Re: pg_basebackup + incremental base backups

2020-05-22 Thread Christopher Pereira
, but this rebuilding process would only be done in case of emergency or during low load hours. pg_basebackup works fine but does not support differential/incremental backups which is a blocker. Do you know any alternative software that is able to rebuild the standby PG data dir using rsync

Re: Query returns no rows in pg_basebackup cluster

2020-05-22 Thread Andrus
and pg_basebackup . Can this backup used for PITR in Linux ? Andrus.

Re: Query returns no rows in pg_basebackup cluster

2020-05-22 Thread Andrus
Hi! Main server is in Linux and backup server is in windows. This is not a supported setup if you want to run a physical backup. Your backup and your primary need to be the same - software and hardware. Consider anything that is working to be a false negative – assume >something will break

Re: Query returns no rows in pg_basebackup cluster

2020-05-21 Thread David G. Johnston
On Thu, May 21, 2020 at 10:41 PM Andrus wrote: > Main server is in Linux and backup server is in windows. > This is not a supported setup if you want to run a physical backup. Your backup and your primary need to be the same - software and hardware. Consider anything that is working to be a

Re: Query returns no rows in pg_basebackup cluster

2020-05-21 Thread Andrus
Hi! No, what it sounds like is the OP tried to physically replicate a database on another platform with completely different sorting rules. The sorting rules for this locale must be the same in both platforms. Only locale names are different. It looks like windows server does not recognize

Re: Query returns no rows in pg_basebackup cluster

2020-05-21 Thread Andrus
Hi! The LIKE query probably doesn't use an index and thus finds the relevant data via sequential scan and equality checks on each record. Yeah, exactly. An equality condition will use a btree index if available. LIKE, however, sees the "_" as a wildcard so it cannot use an index and

Re: Query returns no rows in pg_basebackup cluster

2020-05-21 Thread Andrus
Hi! Are you referring to two different instances of Postgres on Windows? No. Main server is in Linux and backup server is in windows. Andrus.

Re: Query returns no rows in pg_basebackup cluster

2020-05-21 Thread Tom Lane
"David G. Johnston" writes: > On Thu, May 21, 2020 at 3:57 PM Adrian Klaver > wrote: >> Well what I was trying to figure out was: >> "Windows server this query returns 0 rows. >> In Windows server same query using like >> select * from firma1.desktop where baas like '_LOGIFAI' >> returns

Re: Query returns no rows in pg_basebackup cluster

2020-05-21 Thread Adrian Klaver
20 1:20 PM, Andrus wrote: >>> In windows pg_basebackup was used to create base backup from Linux server. > >> Are you referring to two different instances of Postgres on Windows? > > No, what it sounds like is the OP tried to physically replicate a > dat

Re: Query returns no rows in pg_basebackup cluster

2020-05-21 Thread David G. Johnston
On Thu, May 21, 2020 at 3:57 PM Adrian Klaver wrote: > On 5/21/20 3:47 PM, Tom Lane wrote: > > Adrian Klaver writes: > >> On 5/21/20 1:20 PM, Andrus wrote: > >>> In windows pg_basebackup was used to create base backup from Linux > server. > > > >&

Re: Query returns no rows in pg_basebackup cluster

2020-05-21 Thread Adrian Klaver
On 5/21/20 3:47 PM, Tom Lane wrote: Adrian Klaver writes: On 5/21/20 1:20 PM, Andrus wrote: In windows pg_basebackup was used to create base backup from Linux server. Are you referring to two different instances of Postgres on Windows? No, what it sounds like is the OP tried

Re: Query returns no rows in pg_basebackup cluster

2020-05-21 Thread Tom Lane
Adrian Klaver writes: > On 5/21/20 1:20 PM, Andrus wrote: >> In windows pg_basebackup was used to create base backup from Linux server. > Are you referring to two different instances of Postgres on Windows? No, what it sounds like is the OP tried to physically replicate a databas

  1   2   3   >