On Fri, Sep 5, 2014 at 6:25 PM, Issa Ashwash <[email protected]> wrote: > Hello, > Came across an issue today where for large tables not all filenode segments > are restored (I'm assuming this means they aren't backed up, but I'm still > verifying that). > The symptom of this is that after a restore, some tables are empty. When you > compare the the directories, the restored DB takes up less space. > > So for a few very large tables, on the source box we see: >> >> 1.1G /path/postgresql/main/base/16385/50111 >> >> 1.1G /path/postgresql/main/base/16385/50111.1 >> >> 1.1G /path/postgresql/main/base/16385/50111.2 >> >> 1.1G /path/postgresql/main/base/16385/50111.3 >> >> 527M /path/postgresql/main/base/16385/50111.4 >> >> 0 /path/postgresql/main/base/16385/50111_init >> >> 1.1G /path/postgresql/main/base/16385/50112 >> >> 1.1G /path/postgresql/main/base/16385/50112.1 >> >> 1.1G /path/postgresql/main/base/16385/50112.2 >> >> 1.1G /path/postgresql/main/base/16385/50112.3 >> >> 1.1G /path/postgresql/main/base/16385/50112.4 >> >> 142M /path/postgresql/main/base/16385/50112.5 >> >> 8.0K /path/postgresql/main/base/16385/50112_init > > > But on the destination box after a restore we see: >> >> 0 /path/postgresql/main/base/16385/50111 >> >> 0 /path/postgresql/main/base/16385/50111_init >> >> 8.0K /path/postgresql/main/base/16385/50112 >> >> 8.0K /path/postgresql/main/base/16385/50112_init > > > > So the file nodes are empty, and the filenode segments are not copied over > at all. > > System: Postgres 9.1, ubuntu 12.04, wal-e version 6.10 > > If anyone has any pointers on what to look for, I'd be much obliged. > Thanks, > Issa
Do you have logs of the download? It seems like that version of wal-e should have all known scarybugs fixed. I'd suggest upgrade to v0.7 (v0.6 is out of support, but I don't recall any serious bugs that have since been found). One of the features (available only on S3, so it isn't merged) that Heroku uses is to manifest all those files as to provide better forensics and re-checking. You may want to try using that branch if you, too, use S3. It's used in Heroku Postgres production so I surmise it's pretty safe. https://github.com/fdr/wal-e/tree/snapshot-2014062701 -- You received this message because you are subscribed to the Google Groups "wal-e" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
