Yes, contradictory. We recently disabled email forwarding, so on 1/10 when the disk filled up, we never received any alerts. New position, so I was completely unaware of this database, except on a conceptual level, until Tuesday.
I think the best I can do is get this database back to 1/10. In my first restore attempt I noticed that the most recent jiraissue.resolutiondate was 1/10, so jira was running in memory the entire time and nothing was flushed to the db - probably because it was corrupted, and possibly other reasons related to neglect. My hope is that I can get the db back to 1/10 and maybe we can, with Atlassian's help, somehow sync the lucene files back to the db. I don't think I will have any postgres data to work with beyond 1/10. Does this still sound do-able with that kind of data gap? On Fri, Mar 2, 2018 at 3:44 PM, Vick Khera <vi...@khera.org> wrote: > On Fri, Mar 2, 2018 at 4:32 PM, Paul Costello <paulc1...@gmail.com> wrote: > >> I have a database that wouldn't start due to the disk filling up back on >> 1/10, unbeknownst to us until 2/27. This is jira, so it's critical data. >> It appears jira was running in memory that entire time. >> > > > Those first two sentences seem contradictory... > > >> >> I needed to run pg_resetxlog -f in order to start the database. It >> started, but upon logging in I found the system catalog and some data to be >> corrupt. >> > > > Once you did this, fixing the data is really on you. Postgres has no way > to know what any of the data mean, nor how to decide what to keep and what > to toss on those conflicting rows with duplicate keys. > > What I'd personally do is take your 1/5 backup, then merge in rows for > tickets and affiliated data from whatever you can recover in the current > database copy you have. Once that's done, run jira's built-in integrity > checker then do a full export to XML backup format. Finally re-import that > into a fresh jira so you know what's in there is consistent. You'll > probably also have to cross-reference the attachments directory for missing > tickets and clean up those files (or synthesize tickets for them). > > If your jira is configured to send email somewhere on ticket updates, > gathering those (even if it is in multiple people's mailboxes) and > recreating ticket info from them would also move you along. > > You will lose some of your data because not all of it was written to disk. > >
