Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-27 Thread Brian Weaver
On Thu, Sep 27, 2012 at 6:43 PM, Tom Lane wrote: > Brian Weaver writes: >> OK, here is my attempt at patching and correcting the issue in this >> thread. I have done my best to test to ensure that hot standby, >> pg_basebackup, and pg_restore of older files work without is

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-27 Thread Brian Weaver
OK, here is my attempt at patching and correcting the issue in this thread. I have done my best to test to ensure that hot standby, pg_basebackup, and pg_restore of older files work without issues. I think this might be a larger patch that expected, I took some liberties of trying to clean up a bit

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-27 Thread Brian Weaver
w the pg_basebackup worked. The archive will include all the wal files if I make wal_keep_segments high enough. -- Brian On Thu, Sep 27, 2012 at 6:01 PM, Magnus Hagander wrote: > On Tue, Sep 25, 2012 at 5:08 PM, Brian Weaver wrote: >> Unless I misread the code, the tar format and streaming xlog a

[HACKERS] EVENT Keyword and CREATE TABLE

2012-09-26 Thread Brian Weaver
I think I just got bitten hard by a commit in mid July... git sha1 3855968. In some of our old tables going back several years we a column named 'event' as in: CREATE TABLE tblaudittrail ( id bigint NOT NULL, siteid integer NOT NULL, entrytype character varying(25), form character

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-25 Thread Brian Weaver
enough to focus on a single task. I'm far too interrupt driven at work. -- Brian On Tue, Sep 25, 2012 at 10:30 AM, Tom Lane wrote: > Brian Weaver writes: >> If you're willing to wait a bit on me to code and test my extensions >> to pg_basebackup I will try to address som

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-25 Thread Brian Weaver
> On 9/25/12 3:38 PM, Brian Weaver wrote: >> >> I want >> to modify pg_basebackup to include the WAL files in the tar output. > > > Doesn't pg_basebackup -x do exactly that? > > > Regards, > Marko Tiikkaja > -- /* insert witty comment here

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-25 Thread Brian Weaver
sions to pg_basebackup I will try to address some of the deficiencies as well add new features. I agree the checksum algorithm could definitely use some refactoring. I was already working on that before I retired last night. -- Brian On Mon, Sep 24, 2012 at 10:36 PM, Tom Lane wrote: > Bria

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-24 Thread Brian Weaver
f the original committer put the 'ustar00\0' string in by design? Regardless I'll look at it more tomorrow 'cause I'm calling it a night. I need to send a note to the libarchive folks too because I *think* I found a potential buffer overrun in one of their octal conversion r

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-24 Thread Brian Weaver
Um I apologize for the third e-mail on the topic. It seems that my C coding is a bit rusty from years of neglect. No sooner had I hit the send button then I realized that trying to embed a null character in a string might not work, especially when it's followed by two consecutive zeros. Here i

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-24 Thread Brian Weaver
Actually I found one other issue while continuing my investigation. The insertion of the 'ustar' and version '00' has the '00' version at the wrong offset. The patch is attached. -- Brian On Mon, Sep 24, 2012 at 7:51 PM, Brian Weaver wrote: > While research

[HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-24 Thread Brian Weaver
While researching the way streaming replication works I was examining the construction of the tar file header. By comparing documentation on the tar header format from various sources I certain the following patch should be applied to so the group identifier is put into thee header properly. While

Re: [HACKERS] Problem with multi-job pg_restore

2012-05-01 Thread Brian Weaver
Lane wrote: > Brian Weaver writes: >> Doh! I missed a script that was run by cron that does a nightly >> backup. That's the likely offender for the 'copy-to-stdout' > >> I've removed it from the nightly run. I'll see if have any better luck >> w

Re: [HACKERS] Problem with multi-job pg_restore

2012-05-01 Thread Brian Weaver
gh. Any pointers would be appreciated. On Tue, May 1, 2012 at 12:59 PM, Tom Lane wrote: > Brian Weaver writes: > > I'm confused.  A copy-to-stdout ought to be something that pg_dump > would do, not pg_restore.  Are you sure this is related at all? > >                        r

Re: [HACKERS] Problem with multi-job pg_restore

2012-05-01 Thread Brian Weaver
the runtime issues. -- Brian On Tue, May 1, 2012 at 12:59 PM, Tom Lane wrote: > Brian Weaver writes: >> I think I've discovered an issue with multi-job pg_restore on a 700 GB >> data file created with pg_dump. > > Just to clarify, you mean parallel restore, right?  Ar

[HACKERS] Problem with multi-job pg_restore

2012-05-01 Thread Brian Weaver
I think I've discovered an issue with multi-job pg_restore on a 700 GB data file created with pg_dump. Before anyone points out that the preferred procedure is to use the newest pg_dump to backup a database before doing pg_restore let just say, "Yes I'm aware of that advice and unfortunately it jus