Re: [HACKERS] pg_upgrade and rsync

2015-03-18 Thread Bruce Momjian
On Fri, Mar 6, 2015 at 12:19:36PM -0500, Bruce Momjian wrote: On Fri, Mar 6, 2015 at 10:50:27AM +0300, Vladimir Borodin wrote: What I have done is to update the pg_upgrade instructions to add this required step. Updated doc patch attached. (I also added the --delete flag to

Re: [HACKERS] pg_upgrade and rsync

2015-03-06 Thread Bruce Momjian
On Fri, Mar 6, 2015 at 10:50:27AM +0300, Vladimir Borodin wrote: What I have done is to update the pg_upgrade instructions to add this required step. Updated doc patch attached. (I also added the --delete flag to rsync.) Thanks so much for your detailed report. It seems to

Re: [HACKERS] pg_upgrade and rsync

2015-03-05 Thread Bruce Momjian
On Thu, Mar 5, 2015 at 10:55:28AM +0300, Vladimir Borodin wrote: You are correct that a pg_controldata file is copied over that has wal_level=minimal, but that should not be a problem. I suppose, this is the root cause of why replica does not start as hot standby. It it enough

Re: [HACKERS] pg_upgrade and rsync

2015-03-05 Thread Vladimir Borodin
6 марта 2015 г., в 6:11, Bruce Momjian br...@momjian.us написал(а): On Thu, Mar 5, 2015 at 10:55:28AM +0300, Vladimir Borodin wrote: You are correct that a pg_controldata file is copied over that has wal_level=minimal, but that should not be a problem. I suppose, this is the

Re: [HACKERS] pg_upgrade and rsync

2015-03-04 Thread Bruce Momjian
On Wed, Mar 4, 2015 at 01:53:47PM +0300, Vladimir Borodin wrote: After running initdb to create the new master, but before running pg_upgrade, modify the new master's postgresql.conf and change wal_level = hot_standby. (Don't modify pg_hba.conf at this stage.) That does not

Re: [HACKERS] pg_upgrade and rsync

2015-03-04 Thread Vladimir Borodin
4 марта 2015 г., в 19:28, Bruce Momjian br...@momjian.us написал(а): On Wed, Mar 4, 2015 at 01:53:47PM +0300, Vladimir Borodin wrote: After running initdb to create the new master, but before running pg_upgrade, modify the new master's postgresql.conf and change wal_level =

Re: [HACKERS] pg_upgrade and rsync

2015-03-04 Thread Vladimir Borodin
3 марта 2015 г., в 18:01, Bruce Momjian br...@momjian.us написал(а): On Tue, Mar 3, 2015 at 04:55:56PM +0300, Vladimir Borodin wrote: OK, hmmm. Thanks for testing. It feels like you didn't have your new master set up for streaming replication when you ran pg_upgrade. Is that

Re: [HACKERS] pg_upgrade and rsync

2015-03-03 Thread Bruce Momjian
On Tue, Mar 3, 2015 at 04:55:56PM +0300, Vladimir Borodin wrote: OK, hmmm. Thanks for testing. It feels like you didn't have your new master set up for streaming replication when you ran pg_upgrade. Is that correct? Should I specify that specifically in the instructions?

Re: [HACKERS] pg_upgrade and rsync

2015-03-03 Thread Vladimir Borodin
2 марта 2015 г., в 21:28, Bruce Momjian br...@momjian.us написал(а): On Tue, Feb 24, 2015 at 12:13:17PM +0300, Vladimir Borodin wrote: 20 февр. 2015 г., в 18:21, Bruce Momjian br...@momjian.us написал(а): On Fri, Feb 20, 2015 at 09:45:08AM -0500, Bruce Momjian wrote:

Re: [HACKERS] pg_upgrade and rsync

2015-03-03 Thread Bruce Momjian
On Tue, Mar 3, 2015 at 08:38:50AM -0500, Bruce Momjian wrote: 2015-02-24 11:47:22.861 MSK WARNING: WAL was generated with wal_level= minimal, data may be missing 2015-02-24 11:47:22.861 MSK HINT: This happens if you temporarily set wal_level=minimal without taking a new base backup.

Re: [HACKERS] pg_upgrade and rsync

2015-03-03 Thread Bruce Momjian
On Tue, Mar 3, 2015 at 11:38:58AM +0300, Vladimir Borodin wrote: No, you would not need to take a full backup if you use these instructions. Although it would be applied to documentation for 9.5 only, are these instructions applicable for upgrading from 9.3.6 to 9.4.1? Yes. They

Re: [HACKERS] pg_upgrade and rsync

2015-03-03 Thread Vladimir Borodin
3 марта 2015 г., в 16:38, Bruce Momjian br...@momjian.us написал(а): On Tue, Mar 3, 2015 at 11:38:58AM +0300, Vladimir Borodin wrote: No, you would not need to take a full backup if you use these instructions. Although it would be applied to documentation for 9.5 only, are these

Re: [HACKERS] pg_upgrade and rsync

2015-03-02 Thread Bruce Momjian
On Tue, Feb 24, 2015 at 12:13:17PM +0300, Vladimir Borodin wrote: 20 февр. 2015 г., в 18:21, Bruce Momjian br...@momjian.us написал(а): On Fri, Feb 20, 2015 at 09:45:08AM -0500, Bruce Momjian wrote: #3 bothered me as well because it was not specific enough. I like

Re: [HACKERS] pg_upgrade and rsync

2015-02-20 Thread Bruce Momjian
On Thu, Feb 19, 2015 at 09:35:02PM -0500, David Steele wrote: On 2/19/15 11:57 AM, Bruce Momjian wrote: On Wed, Jan 28, 2015 at 09:26:11PM -0800, Josh Berkus wrote: 3. Check that the replica is not very lagged. If it is, wait for traffic to die down and for it to catch up. Now that

Re: [HACKERS] pg_upgrade and rsync

2015-02-20 Thread Bruce Momjian
On Fri, Feb 20, 2015 at 09:45:08AM -0500, Bruce Momjian wrote: #3 bothered me as well because it was not specific enough. I like what you've added to clarify the procedure. Good. It took me a while to understand why they have to be in sync --- because we are using rsync in

Re: [HACKERS] pg_upgrade and rsync

2015-02-19 Thread David Steele
On 2/19/15 11:57 AM, Bruce Momjian wrote: On Wed, Jan 28, 2015 at 09:26:11PM -0800, Josh Berkus wrote: 3. Check that the replica is not very lagged. If it is, wait for traffic to die down and for it to catch up. Now that 9.4.1 is released, I would like to get this doc patch applied --- it

Re: [HACKERS] pg_upgrade and rsync

2015-02-19 Thread Bruce Momjian
On Wed, Jan 28, 2015 at 09:26:11PM -0800, Josh Berkus wrote: So, for my 2c, I'm on the fence about it. On the one hand, I agree, it's a bit of a complex process to get right. On the other hand, it's far better if we put something out there along the lines of if you really want to, this

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Bruce Momjian
On Wed, Jan 28, 2015 at 09:26:11PM -0800, Josh Berkus wrote: 3. Check that the replica is not very lagged. If it is, wait for traffic to die down and for it to catch up. Is this necessary. It seems quite imprecise too. 4. Shut down the master using -m fast or -m smart for a clean shutdown.

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Bruce Momjian
On Thu, Jan 29, 2015 at 10:21:30AM -0500, Andrew Dunstan wrote: On 01/29/2015 12:26 AM, Josh Berkus wrote: So, for my 2c, I'm on the fence about it. On the one hand, I agree, it's a bit of a complex process to get right. On the other hand, it's far better if we put something out there

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Bruce Momjian
On Tue, Jan 27, 2015 at 10:16:48PM -0500, David Steele wrote: This is definitely an edge case. Not only does the file have to be modified in the same second *after* rsync has done the copy, but the file also has to not be modified in *any other subsequent second* before the next incremental

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Andrew Dunstan
On 01/29/2015 12:26 AM, Josh Berkus wrote: So, for my 2c, I'm on the fence about it. On the one hand, I agree, it's a bit of a complex process to get right. On the other hand, it's far better if we put something out there along the lines of if you really want to, this is how to do it than

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Andrew Dunstan
On 01/29/2015 11:34 AM, Bruce Momjian wrote: On Thu, Jan 29, 2015 at 10:21:30AM -0500, Andrew Dunstan wrote: On 01/29/2015 12:26 AM, Josh Berkus wrote: So, for my 2c, I'm on the fence about it. On the one hand, I agree, it's a bit of a complex process to get right. On the other hand, it's

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Bruce Momjian
On Thu, Jan 29, 2015 at 12:09:58PM -0500, Andrew Dunstan wrote: 7. shut down postgres on the replica. 8. rsync both the old and new data directories from the master to the replica, using the --size-only and -H hard links options. For example, if both 9.3 and 9.4 are in /var/lib/postgresql,

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Josh Berkus
On 01/29/2015 09:11 AM, Bruce Momjian wrote: On Thu, Jan 29, 2015 at 12:09:58PM -0500, Andrew Dunstan wrote: Then step 2 should specify that it's for the master. Right. Josh is just listing all the steps --- the pg_upgrade docs already have that spelled out in detail. What I'm also saying

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 12:42 PM, Josh Berkus wrote: On 01/29/2015 09:11 AM, Bruce Momjian wrote: On Thu, Jan 29, 2015 at 12:09:58PM -0500, Andrew Dunstan wrote: Then step 2 should specify that it's for the master. Right. Josh is just listing all the steps --- the pg_upgrade docs already have that

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Jim Nasby
On 1/29/15 7:02 PM, David Steele wrote: On 1/29/15 7:55 PM, Jim Nasby wrote: On 1/29/15 6:25 PM, David Steele wrote: Safe backups can be done without LSNs provided you are willing to trust your timestamps. Which AFAICT simply isn't safe to do at all... except maybe with the manifest stuff

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Jim Nasby
On 1/29/15 5:53 PM, David Steele wrote: On 1/29/15 12:42 PM, Josh Berkus wrote: On 01/29/2015 09:11 AM, Bruce Momjian wrote: On Thu, Jan 29, 2015 at 12:09:58PM -0500, Andrew Dunstan wrote: Then step 2 should specify that it's for the master. Right. Josh is just listing all the steps --- the

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Jim Nasby
On 1/29/15 6:25 PM, David Steele wrote: Safe backups can be done without LSNs provided you are willing to trust your timestamps. Which AFAICT simply isn't safe to do at all... except maybe with the manifest stuff you've talked about? FWIW, I personally am very leery of relying on

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 11:34 AM, Bruce Momjian wrote: 3. Check that the replica is not very lagged. If it is, wait for traffic to die down and for it to catch up. I think I'd want a something a bit more specific here. When the primary shuts down it will kick out one last WAL. The filename should be

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 7:55 PM, Jim Nasby wrote: On 1/29/15 6:25 PM, David Steele wrote: Safe backups can be done without LSNs provided you are willing to trust your timestamps. Which AFAICT simply isn't safe to do at all... except maybe with the manifest stuff you've talked about? Yes - that's what

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 10:13 AM, Bruce Momjian wrote: Agreed. I have update the two mentions of rsync in our docs to clarify this. Thank you. The patch also has pg_upgrade doc improvements suggested by comments from Josh Berkus. It's very good to see this. Mentions of this rsync vulnerability are few

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 7:07 PM, Jim Nasby wrote: Ultimately, there is no single best method. It depends a lot on your environment. I would prefer the official documents to contain very safe methods. How do we define safe though? Your method leaves you without a backup server until your base backup

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 8:09 PM, Jim Nasby wrote: On 1/29/15 7:02 PM, David Steele wrote: On 1/29/15 7:55 PM, Jim Nasby wrote: On 1/29/15 6:25 PM, David Steele wrote: Safe backups can be done without LSNs provided you are willing to trust your timestamps. Which AFAICT simply isn't safe to do at all...

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Stephen Frost
* Jim Nasby (jim.na...@bluetreble.com) wrote: On 1/27/15 9:29 AM, Stephen Frost wrote: My point is that Bruce's patch suggests looking for remote_dir in the rsync documentation, but no such term appears there. Ah, well, perhaps we could simply add a bit of clarification to this: for details

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: On Tue, Jan 27, 2015 at 09:36:58AM -0500, Stephen Frost wrote: The example listed works, but only when it's a local rsync: rsync --archive --hard-links --size-only old_dir new_dir remote_dir Perhaps a better example (or additional one)

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: Interesting problem, but doesn't rsync use sub-second accuracy? No. Simple test will show: touch xx/aa ; rsync -avv xx yy ; sleep 0.5 ; touch xx/aa ; rsync -avv xx yy Run that a few times and you'll see it report xx/aa is uptodate sometimes, depending

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
Bruce, Stephen, etc.: So, I did a test trial of this and it seems like it didn't solve the issue of huge rsyncs. That is, the only reason to do this whole business via rsync, instead of doing a new basebackup of each replica, is to cut down on data transfer time by not resyncing the data from

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
On 01/28/2015 02:10 PM, Josh Berkus wrote: So 390MB were transferred out of a possible 474MB. That certainly seems like we're still transferring the majority of the data, even though I verified that the hard links are being sent as hard links. No? Looks like the majority of that was pg_xlog.

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
On 01/28/2015 02:28 PM, Josh Berkus wrote: On 01/28/2015 02:10 PM, Josh Berkus wrote: So 390MB were transferred out of a possible 474MB. That certainly seems like we're still transferring the majority of the data, even though I verified that the hard links are being sent as hard links. No?

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: On 01/28/2015 02:28 PM, Josh Berkus wrote: On 01/28/2015 02:10 PM, Josh Berkus wrote: So 390MB were transferred out of a possible 474MB. That certainly seems like we're still transferring the majority of the data, even though I verified that the

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
So, for my 2c, I'm on the fence about it. On the one hand, I agree, it's a bit of a complex process to get right. On the other hand, it's far better if we put something out there along the lines of if you really want to, this is how to do it than having folks try to fumble through to find

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Robert Haas
On Sat, Jan 24, 2015 at 10:04 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Jan 23, 2015 at 02:34:36PM -0500, Stephen Frost wrote: You'd have to replace the existing data directory on the master to do that, which pg_upgrade was designed specifically to not do, in case things went

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread David Steele
On 1/27/15 6:09 PM, Jim Nasby wrote: The whole remote_dir discussion made me think of something... would --link-dest be any help here? I'm pretty sure --link-dest would not be effective in this case. The problem exists on the source side and --link-dest only operates on the destination. -- -

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jan 23, 2015 at 1:48 PM, Andres Freund and...@2ndquadrant.com wrote: I don't understand why that'd be better than simply fixing (yes, that's imo the correct term) pg_upgrade to retain relfilenodes across the upgrade. Afaics there's no conflict

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Robert Haas
On Tue, Jan 27, 2015 at 9:36 AM, Stephen Frost sfr...@snowman.net wrote: The example listed works, but only when it's a local rsync: rsync --archive --hard-links --size-only old_dir new_dir remote_dir Perhaps a better example (or additional one) would be with a remote rsync, including

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Robert Haas
On Tue, Jan 27, 2015 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jan 23, 2015 at 1:48 PM, Andres Freund and...@2ndquadrant.com wrote: I don't understand why that'd be better than simply fixing (yes, that's imo the correct term) pg_upgrade

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Sat, Jan 24, 2015 at 10:04 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Jan 23, 2015 at 02:34:36PM -0500, Stephen Frost wrote: You'd have to replace the existing data directory on the master to do that, which pg_upgrade was designed

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Jim Nasby
On 1/27/15 9:29 AM, Stephen Frost wrote: My point is that Bruce's patch suggests looking for remote_dir in the rsync documentation, but no such term appears there. Ah, well, perhaps we could simply add a bit of clarification to this: for details on specifying optionremote_dir/ The whole

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Bruce Momjian
On Mon, Jan 26, 2015 at 05:41:59PM -0500, Stephen Frost wrote: I've thought about it a fair bit actually and I agree that there is some risk to using rsync for *incremental* base backups. That is, you have a setup where you loop with: pg_start_backup rsync - dest pg_stop_backup without

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread David Steele
On 1/27/15 9:51 PM, Bruce Momjian wrote: According to my empirical testing on Linux and OSX the answer is no: rsync does not use sub-second accuracy. This seems to be true even on file systems like ext4 that support millisecond mod times, at least it was true on Ubuntu 12.04 running ext4.

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Robert Haas
On Fri, Jan 23, 2015 at 1:48 PM, Andres Freund and...@2ndquadrant.com wrote: On 2015-01-22 20:54:47 -0500, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: On Fri, Jan 23, 2015 at 01:19:33AM +0100, Andres Freund wrote: Or do you - as the text edited in your patch, but not the

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 27, 2015 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: That's certainly impossible for the system catalogs, which means you have to be able to deal with relfilenode discrepancies for them, which means that maintaining the same relfilenodes

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Andres Freund
On 2015-01-27 10:20:48 -0500, Robert Haas wrote: On Tue, Jan 27, 2015 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jan 23, 2015 at 1:48 PM, Andres Freund and...@2ndquadrant.com wrote: I don't understand why that'd be better than simply

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 27, 2015 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: That's certainly impossible for the system catalogs, which means you have to be able to deal with relfilenode discrepancies for them, which

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Tue, Jan 27, 2015 at 9:36 AM, Stephen Frost sfr...@snowman.net wrote: The example listed works, but only when it's a local rsync: rsync --archive --hard-links --size-only old_dir new_dir remote_dir Perhaps a better example (or additional

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Bruce Momjian
On Tue, Jan 27, 2015 at 09:36:58AM -0500, Stephen Frost wrote: The example listed works, but only when it's a local rsync: rsync --archive --hard-links --size-only old_dir new_dir remote_dir Perhaps a better example (or additional one) would be with a remote rsync, including clarification

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Bruce Momjian
On Tue, Jan 27, 2015 at 09:44:51PM -0500, David Steele wrote: On 1/27/15 9:32 PM, Bruce Momjian wrote Now, this isn't actually a problem for the first time that file is backed up- the issue is if that file isn't changed again. rsync won't re-copy it, but that change that rsync missed won't

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread David Steele
On 1/27/15 9:32 PM, Bruce Momjian wrote Now, this isn't actually a problem for the first time that file is backed up- the issue is if that file isn't changed again. rsync won't re-copy it, but that change that rsync missed won't be in the WAL history for the *second* backup that's done (only

Re: [HACKERS] pg_upgrade and rsync

2015-01-26 Thread Jim Nasby
On 1/23/15 12:40 PM, Stephen Frost wrote: That said, the whole timestamp race condition in rsync gives me the heebie-jeebies. For normal workloads maybe it's not that big a deal, but when dealing with fixed-size data (ie: Postgres blocks)? Eww. The race condition is a problem for

Re: [HACKERS] pg_upgrade and rsync

2015-01-26 Thread David Steele
On 1/26/15 5:11 PM, Jim Nasby wrote: The race condition is a problem for pg_start/stop_backup and friends. In this instance, everything will be shut down when the rsync is running, so there isn't a timestamp race condition to worry about. Yeah, I'm more concerned about people that use rsync

Re: [HACKERS] pg_upgrade and rsync

2015-01-26 Thread Stephen Frost
Jim, * Jim Nasby (jim.na...@bluetreble.com) wrote: On 1/23/15 12:40 PM, Stephen Frost wrote: That said, the whole timestamp race condition in rsync gives me the heebie-jeebies. For normal workloads maybe it's not that big a deal, but when dealing with fixed-size data (ie: Postgres blocks)?

Re: [HACKERS] pg_upgrade and rsync

2015-01-26 Thread Jim Nasby
On 1/26/15 5:08 PM, David Steele wrote: I've written tests to show the rsync vulnerability and another to show that this can affect a running database. However, to reproduce it reliably you need to force a checkpoint or have them happening pretty close together. Related to this and Stephen's

Re: [HACKERS] pg_upgrade and rsync

2015-01-24 Thread Bruce Momjian
On Fri, Jan 23, 2015 at 02:34:36PM -0500, Stephen Frost wrote: You'd have to replace the existing data directory on the master to do that, which pg_upgrade was designed specifically to not do, in case things went poorly. Why? Just rsync the new data directory onto the old directory

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Jim Nasby
On 1/22/15 7:54 PM, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: On Fri, Jan 23, 2015 at 01:19:33AM +0100, Andres Freund wrote: Or do you - as the text edited in your patch, but not the quote above - mean to run pg_upgrade just on the primary and then rsync? No, I was

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-23 13:52:54 -0500, Stephen Frost wrote: That wouldn't actually help with what Bruce is trying to do, which is to duplicate the results of the pg_upgrade from the master over to the standby. Well, it'd pretty much obliviate the

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-23 14:27:51 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-23 14:05:10 -0500, Stephen Frost wrote: If I follow what you're suggesting, pg_upgrade would need a new 'in-place' mode that

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Stephen Frost
* Jim Nasby (jim.na...@bluetreble.com) wrote: On 1/22/15 7:54 PM, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: On Fri, Jan 23, 2015 at 01:19:33AM +0100, Andres Freund wrote: Or do you - as the text edited in your patch, but not the quote above - mean to run pg_upgrade just

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Andres Freund
On 2015-01-22 20:54:47 -0500, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: On Fri, Jan 23, 2015 at 01:19:33AM +0100, Andres Freund wrote: Or do you - as the text edited in your patch, but not the quote above - mean to run pg_upgrade just on the primary and then rsync?

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Andres Freund
On 2015-01-23 13:52:54 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-22 20:54:47 -0500, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: On Fri, Jan 23, 2015 at 01:19:33AM +0100, Andres Freund wrote: Or do you - as the text

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Andres Freund
On 2015-01-23 14:05:10 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-23 13:52:54 -0500, Stephen Frost wrote: That wouldn't actually help with what Bruce is trying to do, which is to duplicate the results of the pg_upgrade from the master over to

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-22 20:54:47 -0500, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: On Fri, Jan 23, 2015 at 01:19:33AM +0100, Andres Freund wrote: Or do you - as the text edited in your patch, but not the quote above - mean to

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-23 14:05:10 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-23 13:52:54 -0500, Stephen Frost wrote: That wouldn't actually help with what Bruce is trying to do, which is to duplicate the

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Andres Freund
On 2015-01-23 14:27:51 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-23 14:05:10 -0500, Stephen Frost wrote: If I follow what you're suggesting, pg_upgrade would need a new 'in-place' mode that removes all of the catalog tables from the old

Re: [HACKERS] pg_upgrade and rsync

2015-01-23 Thread Bruce Momjian
On Fri, Jan 23, 2015 at 02:34:36PM -0500, Stephen Frost wrote: Why? Just rsync the new data directory onto the old directory on the standbys. That's fine and simple. That still doesn't address the need to use --size-only, it would just mean that you don't need to use -H. If anything the

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: On Fri, Jan 23, 2015 at 01:19:33AM +0100, Andres Freund wrote: Or do you - as the text edited in your patch, but not the quote above - mean to run pg_upgrade just on the primary and then rsync? No, I was going to run it on both, then rsync. I'm

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread David Steele
On 1/22/15 8:54 PM, Stephen Frost wrote: The problem, as mentioned elsewhere, is that you have to checksum all the files because the timestamps will differ. You can actually get around that with rsync if you really want though- tell it to only look at file sizes instead of size+time by

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread Stephen Frost
* David Steele (da...@pgmasters.net) wrote: On 1/22/15 8:54 PM, Stephen Frost wrote: The problem, as mentioned elsewhere, is that you have to checksum all the files because the timestamps will differ. You can actually get around that with rsync if you really want though- tell it to only

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread David Steele
On 1/22/15 10:05 PM, Stephen Frost wrote: In addition, there is a possible race condition in rsync where a file that is modified in the same second after rsync starts to copy will not be picked up in a subsequent rsync unless --checksum is used. This is fairly easy to prove and is shown here:

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread Bruce Momjian
On Thu, Jan 22, 2015 at 10:48:37PM +0200, Heikki Linnakangas wrote: * If we need to protect hint bit updates from torn writes, WAL-log a * full page image of the page. This full page image is only necessary * if the hint bit update is the first change to the

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread Jim Nasby
On 1/22/15 5:43 PM, Bruce Momjian wrote: This brings up the other problem that the mod times of the files are likely to be different between master and slave --- should I recommend to only use rsync --checksum? I don't think so. AIUI if the timestamps are different the very next thing it does

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread Andres Freund
On 2015-01-22 14:20:51 -0500, Bruce Momjian wrote: It is possible to upgrade on pg_upgrade on streaming standby servers by making them master servers, running pg_upgrade on them, then shuting down all servers and using rsync to make the standby servers match the real master. Isn't that a

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread Bruce Momjian
On Thu, Jan 22, 2015 at 06:04:24PM -0600, Jim Nasby wrote: On 1/22/15 5:43 PM, Bruce Momjian wrote: This brings up the other problem that the mod times of the files are likely to be different between master and slave --- should I recommend to only use rsync --checksum? I don't think so. AIUI

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread Bruce Momjian
On Fri, Jan 23, 2015 at 01:19:33AM +0100, Andres Freund wrote: On 2015-01-22 14:20:51 -0500, Bruce Momjian wrote: It is possible to upgrade on pg_upgrade on streaming standby servers by making them master servers, running pg_upgrade on them, then shuting down all servers and using rsync to

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread Heikki Linnakangas
On 01/22/2015 09:20 PM, Bruce Momjian wrote: One question I have is whether hint bits are set by read-only transactions on standby servers. No. See comments in MarkBufferDirtyHint: /* * If we need to protect hint bit updates from torn writes, WAL-log a

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread Jim Nasby
On 1/22/15 2:19 PM, Heikki Linnakangas wrote: On 01/22/2015 09:20 PM, Bruce Momjian wrote: One question I have is whether hint bits are set by read-only transactions on standby servers. No. See comments in MarkBufferDirtyHint: /* * If we need to protect hint bit updates

Re: [HACKERS] pg_upgrade and rsync

2015-01-22 Thread Heikki Linnakangas
On 01/22/2015 10:34 PM, Jim Nasby wrote: On 1/22/15 2:19 PM, Heikki Linnakangas wrote: On 01/22/2015 09:20 PM, Bruce Momjian wrote: One question I have is whether hint bits are set by read-only transactions on standby servers. No. See comments in MarkBufferDirtyHint: /*