Re: Trying to handle db corruption 9.6

2019-05-21 Thread Tomas Vondra
On Tue, May 21, 2019 at 04:03:52PM +, Greg Clough wrote: My restore command copy the wals from archive dir in the primary to an archive dir in the secondary(different from the pg_xlog in the secondary) I think that you're restore command puts them back into the archive, and then uncompre

Re: Trying to handle db corruption 9.6

2019-05-21 Thread Mariel Cherkassky
Tomas - Well, when I run the restore_command manually it works (archive dir exists on the secondary..). Thank for the explanation on the system catalogs.. Greg - My restore command copy the wals from archive dir in the primary to an archive dir in the secondary(different from the pg_xlog in the s

Re: Trying to handle db corruption 9.6

2019-05-21 Thread Tomas Vondra
On Tue, May 21, 2019 at 12:01:31PM +0300, Mariel Cherkassky wrote: Tomas : Well, when you say it does not work, why do you think so? Does it print some error, or what? Does it even get executed? It does not seem to be the case, judging by the log (there's no archive_command message). How was th

Re: Trying to handle db corruption 9.6

2019-05-21 Thread Mariel Cherkassky
Tomas : Well, when you say it does not work, why do you think so? Does it print some error, or what? Does it even get executed? It does not seem to be the case, judging by the log (there's no archive_command message). How was the "secondary machine" created? You said you have all the WAL since th

Re: Trying to handle db corruption 9.6

2019-05-20 Thread Tomas Vondra
On Mon, May 20, 2019 at 08:20:33PM +0300, Mariel Cherkassky wrote: Hey Greg, Basically my backup was made after the first pg_resetxlog so I was wrong. Bummer. However, the customer had a secondary machine that wasn't synced for a month. I have all the walls since the moment the second

Re: Trying to handle db corruption 9.6

2019-05-20 Thread Tomas Vondra
On Mon, May 20, 2019 at 04:20:45PM +, Bimal wrote: I had ran into same issue about year back, luckily I had standby to quickly promote. But, I wish there was better a documentation on how to handle WAL log fill up and resetting them. pg_resetxlog is not a tool to deal with "WAL fil

Re: Trying to handle db corruption 9.6

2019-05-20 Thread Mariel Cherkassky
Hey Greg, Basically my backup was made after the first pg_resetxlog so I was wrong. However, the customer had a secondary machine that wasn't synced for a month. I have all the walls since the moment the secondary went out of sync. Once I started it I hoped that it will start recover the wals and f

Re: Trying to handle db corruption 9.6

2019-05-20 Thread Bimal
I had ran into same issue about year back, luckily I had standby to quickly promote.  But, I wish there was better a documentation on how to handle WAL log fill up and resetting them.   On Monday, May 20, 2019, 9:08:19 AM PDT, Mariel Cherkassky wrote: A backup was made after the corru

Re: Trying to handle db corruption 9.6

2019-05-20 Thread Mariel Cherkassky
A backup was made after the corruption appeared but before I tried using the pg_resetxlog command. Basically I just want to start the database with the data that is available in the files(I'm ok with loosing data that was in the cache and wasnt written to disk). My question is how can I continue fr

Re: Trying to handle db corruption 9.6

2019-05-20 Thread Mariel Cherkassky
Yes I understand that.. I'm trying to handle it after the backup that I have taken.. On Mon, May 20, 2019, 5:49 PM Flo Rance wrote: > Hi, > > First of all, as stated in the wiki, you'll need to do a filesystem level > copy of the database files and put them on another drive before attempting > t

Re: Trying to handle db corruption 9.6

2019-05-20 Thread Flo Rance
Hi, First of all, as stated in the wiki, you'll need to do a filesystem level copy of the database files and put them on another drive before attempting to do anything else ! https://wiki.postgresql.org/wiki/Corruption regards, Flo On Mon, May 20, 2019 at 4:40 PM Mariel Cherkassky < mariel.cher

Trying to handle db corruption 9.6

2019-05-20 Thread Mariel Cherkassky
Hey, I'm trying to handle a corruption that one of our customers is facing. His disk space was full and as a result of that he decided to run pg_resetxlog a few times(bad idea..) . When I connected to the machine I saw that the db was down. When I started the db (service postgresql start) I saw the