Re: [GENERAL] pg_basebackup failing
The error you are seeing is triggered because this relation file exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which is as well the norm for tar. I thought PostgreSQL would break the file if it grows beyond 1GB (1GB is segment size which one can modify while compiling). Am I missing something or have I got it totally wrong? Is there a solution/workaround? Not use a pg_basebackup but an external backup solution or a custom script. Break this relation file into several pieces by doing some partitions on it. Can I break up the referenced file somehow? Partitions on the parent table. Which might have other implications on the performance if the queries do not use Partition key in where clause. Best Regards, *Sameer Kumar | Database Consultant* *ASHNIK PTE. LTD. *101 Cecil Street, #11-11 Tong Eng Building, Singapore 069533 M : *+65 8110 0350* T: +65 6438 3504 | www.ashnik.com www.facebook.com/ashnikbiz | www.twitter.com/ashnikbiz [image: email patch] This email may contain confidential, privileged or copyright material and is solely for the use of the intended recipient(s). image002.jpg
Re: [GENERAL] pg_basebackup failing
On Jan 15, 2014 12:07 PM, Sameer Kumar sameer.ku...@ashnik.com wrote: The error you are seeing is triggered because this relation file exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which is as well the norm for tar. I thought PostgreSQL would break the file if it grows beyond 1GB (1GB is segment size which one can modify while compiling). Am I missing something or have I got it totally wrong? Yeah, how did you actually end up with a file that size? Do you have any non standard compiler options set when you built your server? /Magnus
Re: [GENERAL] pg_basebackup failing
On Wed, Jan 15, 2014 at 8:05 PM, Sameer Kumar sameer.ku...@ashnik.comwrote: The error you are seeing is triggered because this relation file exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which is as well the norm for tar. I thought PostgreSQL would break the file if it grows beyond 1GB (1GB is segment size which one can modify while compiling). Am I missing something or have I got it totally wrong? No, you are missing nothing. Thanks for reminding :) (This is settable with ./configure --with-segsize) So the build has been done with some uncommon option. Regards, -- Michael
Re: [GENERAL] pg_basebackup failing
Thanks all alan On Jan 15, 2014, at 6:30 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Jan 15, 2014 at 8:05 PM, Sameer Kumar sameer.ku...@ashnik.com wrote: The error you are seeing is triggered because this relation file exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which is as well the norm for tar. I thought PostgreSQL would break the file if it grows beyond 1GB (1GB is segment size which one can modify while compiling). Am I missing something or have I got it totally wrong? No, you are missing nothing. Thanks for reminding :) (This is settable with ./configure --with-segsize) So the build has been done with some uncommon option. Regards, -- Michael
Re: [GENERAL] pg_basebackup failing
On Wed, Jan 15, 2014 at 6:53 AM, Alan Nilsson anils...@apple.com wrote: Could someone give me some insight to the following error message: [mqsql06:/Volumes/SQL_Set] _postgres% pg_basebackup --user=replicate --host=mqsql03 -xP -Fp --pgdata=pgsql 19439890/65873894 kB (29%), 1/1 tablespace pg_basebackup: could not get transaction log end position from server: ERROR: archive member base/17040/25491 too large for tar format I find the error message confusing since I have the format explicitly set to plain. pg_basebackup fetches a full backup from server using tar all the time for performance purposes to reduce the amount of data sent through the stream. pg_basebackup contains some code to decode this tarball and generate a plain format from it. The error you are seeing is triggered because this relation file exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which is as well the norm for tar. Is there a solution/workaround? Not use a pg_basebackup but an external backup solution or a custom script. Break this relation file into several pieces by doing some partitions on it. Can I break up the referenced file somehow? Partitions on the parent table. Regards, -- Michael -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
