Re: [HACKERS] Add -c to rsync commands on SR tutorial wiki page

[Robert Haas]

On Thu, Aug 18, 2016 at 2:27 PM, Jim Nasby  wrote:
> I don't think it's any great leap for someone to think they can use those
> commands incrementally. It's certainly one of the first things you think of
> when using rsync. AFAIK there's no downside at all to using -c when it is a
> brand new copy, so I'm thinking we should just put it in there, especially
> considering what the potential downside is.

I think that's right.

The suggestion that people might use some backup tool is a good one,
but that's not a reason not to add -c here.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Add -c to rsync commands on SR tutorial wiki page

[Jim Nasby]

On 8/18/16 1:31 PM, Stephen Frost wrote:

To have proper incremental backups done requires a lot more than just
throwing "-c" into the rsyncs.  For my 2c, I'm at the point where I'd
prefer we discourage people from using rsync, cp, or generally try to
set up their own hand-rolled backup system with PG.  Those examples
simply encourage poor setups that risk losing data and introducing
corruption.


I agree (and the only reason I was looking at that page is because I 
client was following the directions on it!)


So, what would this mean in terms of the wiki? Should we nuke that page 
and point at something in the docs? Something else?

--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)   mobile: 512-569-9461


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Add -c to rsync commands on SR tutorial wiki page

[Stephen Frost]

* Jim Nasby (jim.na...@bluetreble.com) wrote:
> On 8/17/16 9:46 PM, Stephen Frost wrote:
> >* Jim Nasby (jim.na...@bluetreble.com) wrote:
> >>> https://wiki.postgresql.org/wiki/Binary_Replication_Tutorial does
> >>> not specify -c for any of the rsync commands. That's maybe safe for
> >>> WAL, but I don't think it's safe for any of the other uses, right?
> >>> I'd like someone to confirm before I just change the page... my
> >>> intention is to just stick -c in all the commands.
> >-c is only relevant when you are doing an incremental copy, but on a
> >quick look, all those rsync commands appear to be doing full copies?
> >
> >You would want -c if you were taking a backup and then doing an update
> >of it using rsync. or something along those lines, as you can't really
> >trust rsync's time/size based comparison as it only has a 1 second level
> >granularity.
> 
> I don't think it's any great leap for someone to think they can use
> those commands incrementally. It's certainly one of the first things
> you think of when using rsync. AFAIK there's no downside at all to
> using -c when it is a brand new copy, so I'm thinking we should just
> put it in there, especially considering what the potential downside
> is.

To have proper incremental backups done requires a lot more than just
throwing "-c" into the rsyncs.  For my 2c, I'm at the point where I'd
prefer we discourage people from using rsync, cp, or generally try to
set up their own hand-rolled backup system with PG.  Those examples
simply encourage poor setups that risk losing data and introducing
corruption.

Thanks!

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Add -c to rsync commands on SR tutorial wiki page

[Jim Nasby]

On 8/17/16 9:46 PM, Stephen Frost wrote:

* Jim Nasby (jim.na...@bluetreble.com) wrote:

> https://wiki.postgresql.org/wiki/Binary_Replication_Tutorial does
> not specify -c for any of the rsync commands. That's maybe safe for
> WAL, but I don't think it's safe for any of the other uses, right?
> I'd like someone to confirm before I just change the page... my
> intention is to just stick -c in all the commands.

-c is only relevant when you are doing an incremental copy, but on a
quick look, all those rsync commands appear to be doing full copies?

You would want -c if you were taking a backup and then doing an update
of it using rsync. or something along those lines, as you can't really
trust rsync's time/size based comparison as it only has a 1 second level
granularity.


I don't think it's any great leap for someone to think they can use 
those commands incrementally. It's certainly one of the first things you 
think of when using rsync. AFAIK there's no downside at all to using -c 
when it is a brand new copy, so I'm thinking we should just put it in 
there, especially considering what the potential downside is.



--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)   mobile: 512-569-9461


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Add -c to rsync commands on SR tutorial wiki page

[Stephen Frost]

* Jim Nasby (jim.na...@bluetreble.com) wrote:
> https://wiki.postgresql.org/wiki/Binary_Replication_Tutorial does
> not specify -c for any of the rsync commands. That's maybe safe for
> WAL, but I don't think it's safe for any of the other uses, right?
> I'd like someone to confirm before I just change the page... my
> intention is to just stick -c in all the commands.

-c is only relevant when you are doing an incremental copy, but on a
quick look, all those rsync commands appear to be doing full copies?

You would want -c if you were taking a backup and then doing an update
of it using rsync. or something along those lines, as you can't really
trust rsync's time/size based comparison as it only has a 1 second level
granularity.

Thanks!

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Add -c to rsync commands on SR tutorial wiki page

[Peter Eisentraut]

On 8/17/16 5:34 PM, Jim Nasby wrote:
> https://wiki.postgresql.org/wiki/Binary_Replication_Tutorial does not 
> specify -c for any of the rsync commands. That's maybe safe for WAL, but 
> I don't think it's safe for any of the other uses, right? I'd like 
> someone to confirm before I just change the page... my intention is to 
> just stick -c in all the commands.

Why do you think that?

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers