Re: [HACKERS] pg_basebackup stream xlog to tar

On Wed, Oct 26, 2016 at 2:00 AM, Magnus Hagander wrote: > Thanks, applied and pushed. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_basebackup stream xlog to tar

On Tue, Oct 25, 2016 at 2:52 PM, Michael Paquier wrote: > On Tue, Oct 25, 2016 at 7:12 PM, Magnus Hagander > wrote: > > On Mon, Oct 24, 2016 at 7:46 AM, Michael Paquier < > michael.paqu...@gmail.com> > > wrote: > >> > >> On Sun, Oct 23, 2016 at

Re: [HACKERS] pg_basebackup stream xlog to tar

On Tue, Oct 25, 2016 at 7:12 PM, Magnus Hagander wrote: > On Mon, Oct 24, 2016 at 7:46 AM, Michael Paquier > wrote: >> >> On Sun, Oct 23, 2016 at 10:28 PM, Magnus Hagander >> wrote: >> + if (format == 'p') >> +

Re: [HACKERS] pg_basebackup stream xlog to tar

On Mon, Oct 24, 2016 at 7:46 AM, Michael Paquier wrote: > On Sun, Oct 23, 2016 at 10:28 PM, Magnus Hagander > wrote: > > It also broke the tests and invalidated some documentation. But it was > easy > > enough to fix. > > > > I've now applied

Re: [HACKERS] pg_basebackup stream xlog to tar

On Sun, Oct 23, 2016 at 10:28 PM, Magnus Hagander wrote: > It also broke the tests and invalidated some documentation. But it was easy > enough to fix. > > I've now applied this, so next time you get to do the merging :P Joking > aside, please review and let me know if you

Re: [HACKERS] pg_basebackup stream xlog to tar

On Mon, Oct 24, 2016 at 1:38 PM, Andres Freund wrote: > On 2016-10-17 14:37:05 +0900, Michael Paquier wrote: >> 2) Add an option to pg_xlogdump to be able to output its output to a >> file. That would be awkward to rely on grabbing the output data from a >> pipe... On Windows

Re: [HACKERS] pg_basebackup stream xlog to tar

On 2016-10-17 14:37:05 +0900, Michael Paquier wrote: > 2) Add an option to pg_xlogdump to be able to output its output to a > file. That would be awkward to rely on grabbing the output data from a > pipe... On Windows particularly. Thinking about it, would that > actually be useful to others?

Re: [HACKERS] pg_basebackup stream xlog to tar

On Sun, Oct 23, 2016 at 10:52 PM, Michael Paquier wrote: > On Sun, Oct 23, 2016 at 10:30 PM, Magnus Hagander wrote: >> I think both of those would be worthwhile. Just for the testability in >> itself, but such a flag to pg_xlogdump would probably

Re: [HACKERS] pg_basebackup stream xlog to tar

On Sun, Oct 23, 2016 at 10:30 PM, Magnus Hagander wrote: > On Mon, Oct 17, 2016 at 7:37 AM, Michael Paquier > wrote: >> > But independent of this patch, actually putting that test in for non-tar >> > mode would probably not be a bad idea -- if that

Re: [HACKERS] pg_basebackup stream xlog to tar

On Mon, Oct 17, 2016 at 7:37 AM, Michael Paquier wrote: > On Sat, Oct 15, 2016 at 8:51 AM, Magnus Hagander > wrote: > > Fixed. > > Ok, I had a extra look on the patch: > + The transactionn log files are written to a separate file > +

Re: [HACKERS] pg_basebackup stream xlog to tar

On Fri, Oct 21, 2016 at 2:02 PM, Michael Paquier wrote: > On Mon, Oct 17, 2016 at 2:37 PM, Michael Paquier > wrote: > > Except that it looks in pretty good to me, so I am switching that as > > ready for committer. > > + /* > +*

Re: [HACKERS] pg_basebackup stream xlog to tar

On Mon, Oct 17, 2016 at 2:37 PM, Michael Paquier wrote: > Except that it looks in pretty good to me, so I am switching that as > ready for committer. + /* +* Create pg_xlog/archive_status (and thus pg_xlog) so we can write to +* basedir/pg_xlog as

Re: [HACKERS] pg_basebackup stream xlog to tar

On Sat, Oct 15, 2016 at 8:51 AM, Magnus Hagander wrote: > Fixed. Ok, I had a extra look on the patch: + The transactionn log files are written to a separate file +called pg_xlog.tar. + s/transactionn/transaction/, and the markup should be on

Re: [HACKERS] pg_basebackup stream xlog to tar

On Tue, Oct 4, 2016 at 12:05 AM, Michael Paquier wrote: > (Squashing two emails into one) > > On Fri, Sep 30, 2016 at 11:16 PM, Magnus Hagander > wrote: > > And here's yet another version, now rebased on top of the fsync and > nosync > > changes

Re: [HACKERS] pg_basebackup stream xlog to tar

(Squashing two emails into one) On Fri, Sep 30, 2016 at 11:16 PM, Magnus Hagander wrote: > And here's yet another version, now rebased on top of the fsync and nosync > changes that got applied. My fault :p > In particular, this conflicted with pretty much every single

Re: [HACKERS] pg_basebackup stream xlog to tar

On Thu, Sep 29, 2016 at 12:44 PM, Magnus Hagander wrote: > On Mon, Sep 5, 2016 at 10:01 AM, Michael Paquier < > michael.paqu...@gmail.com> wrote: > >> On Sat, Sep 3, 2016 at 10:35 PM, Magnus Hagander >> wrote: >> > Ugh. That would be nice to have, but I

Re: [HACKERS] pg_basebackup stream xlog to tar

On Mon, Sep 5, 2016 at 10:01 AM, Michael Paquier wrote: > On Sat, Sep 3, 2016 at 10:35 PM, Magnus Hagander > wrote: > > Ugh. That would be nice to have, but I think that's outside the scope of > > this patch. > > A test for this patch that could

Re: [HACKERS] pg_basebackup stream xlog to tar

On Sep 28, 2016 19:11, "Robert Haas" wrote: > > On Mon, Sep 5, 2016 at 4:01 AM, Michael Paquier > wrote: > > [ review comments ] > > This thread has been sitting idle for more than 3 weeks, so I'm > marking it "Returned with Feedback" in the

Re: [HACKERS] pg_basebackup stream xlog to tar

On Mon, Sep 5, 2016 at 4:01 AM, Michael Paquier wrote: > [ review comments ] This thread has been sitting idle for more than 3 weeks, so I'm marking it "Returned with Feedback" in the CommitFest application. Magnus, Michael's latest round of comments seem pretty

Re: [HACKERS] pg_basebackup stream xlog to tar

On Sat, Sep 3, 2016 at 10:35 PM, Magnus Hagander wrote: > Ugh. That would be nice to have, but I think that's outside the scope of > this patch. A test for this patch that could have value would be to use pg_basebackup -X stream -Ft, then untar pg_xlog.tar and look at the

Re: [HACKERS] pg_basebackup stream xlog to tar

On Thu, Sep 1, 2016 at 2:39 PM, Michael Paquier wrote: > On Thu, Sep 1, 2016 at 5:13 PM, Magnus Hagander > wrote: > > We don't seem to check for similar issues as the one just found in the > > existing tests though, do we? As in, we don't actually

Re: [HACKERS] pg_basebackup stream xlog to tar

On Thu, Sep 1, 2016 at 5:13 PM, Magnus Hagander wrote: > We don't seem to check for similar issues as the one just found in the > existing tests though, do we? As in, we don't actually verify that the xlog > files being streamed are 16Mb? (Or for that matter that the tarfile

Re: [HACKERS] pg_basebackup stream xlog to tar

On Thu, Sep 1, 2016 at 9:43 AM, Michael Paquier wrote: > On Thu, Sep 1, 2016 at 4:41 PM, Magnus Hagander > wrote: > > That's definitely not intended - it's supposed to be 16Mb. And it > actually > > writes 16Mb to the tarfile, it's the extraction

Re: [HACKERS] pg_basebackup stream xlog to tar

On Thu, Sep 1, 2016 at 4:41 PM, Magnus Hagander wrote: > That's definitely not intended - it's supposed to be 16Mb. And it actually > writes 16Mb to the tarfile, it's the extraction that doesn't see them. That > also means that if you get more than one member of the tarfile

Re: [HACKERS] pg_basebackup stream xlog to tar

On Thu, Sep 1, 2016 at 9:19 AM, Michael Paquier wrote: > On Thu, Sep 1, 2016 at 6:58 AM, Magnus Hagander > wrote: > > Attached patch adds support for -X stream to work with .tar and .tar.gz > file > > formats. > > Nice. > > > If tar mode is

Re: [HACKERS] pg_basebackup stream xlog to tar

On Thu, Sep 1, 2016 at 6:58 AM, Magnus Hagander wrote: > Attached patch adds support for -X stream to work with .tar and .tar.gz file > formats. Nice. > If tar mode is specified, a separate pg_xlog.tar (or .tar.gz) file is > created and the data is streamed into it. Regular