Re: [osmosis-dev] Wrapper scripts for streaming replication

2013-07-19 Thread Brett Henderson
Hi Paul,

On 25 May 2013 15:15, Paul Norman penor...@mac.com wrote:

  From: Brett Henderson [mailto:br...@bretth.com]
  Sent: Friday, April 19, 2013 4:12 AM
  Subject: Re: [osmosis-dev] Wrapper scripts for streaming replication
 
  It'd be great to see the replication streaming getting some usage.  It
 doesn't seem to be used at the moment.  In fact it's been down for a couple
 of weeks and it's trying to catch up again now.

 Do you think it's stable enough to base a service on, or am I better off
 sticking with minutely diffs?


 The minutely diffs are used by enough services including OSMF hosted ones
 that you can be fairly confident that any stoppages will be found and fixed
 quickly, but I'm not sure the same is true for streaming diffs.


That's right, we have a chicken and egg problem here.  Minutely diffs are
more stable because they've been worked on for several years with a large
number of consumers.  Replication streaming is fairly new and likely to
take a while to iron out the issues, but that won't occur unless people use
it.

However I'm largely unavailable at the moment with other commitments, so I
don't have much time to help fix things.



  There are a couple of example Osmosis command lines on the following
 wiki page in case you haven't already seen them.
 
 http://wiki.openstreetmap.org/wiki/Osmosis/Replication#Client-side_Streaming

 The pgsnapshot example would work from the shell, but do you have any
 examples for an unattened server?


I don't have any scripts that I can share.  It should be sufficient to
write a wrapper script that redirects output to log files and launches the
osmosis instance.  For extra points it might be worth having it restart
Osmosis if it fails (with some delays included), however Osmosis should
theoretically handle error conditions such as network failures gracefully.



 Also, what's the best way to switch to streaming replication from diffs,
 and the reverse?

 It's a similar question to switching from hourly to minute diffs where
you'll have to figure out the sequence number to begin from, and doing that
will require figuring out an approximate date that is earlier than your
current replication point.  Note that the streaming server does provide
some assistance where it can provide you with a state file matching a
particular timestamp.  A curl example is provided in the URL you've already
read, but more details on all the URLs are available here:
http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage#--send-replication-data_.28--srd.29

Brett
___
osmosis-dev mailing list
osmosis-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/osmosis-dev


Re: [osmosis-dev] Wrapper scripts for streaming replication

2013-05-24 Thread Paul Norman
 From: Brett Henderson [mailto:br...@bretth.com] 
 Sent: Friday, April 19, 2013 4:12 AM
 Subject: Re: [osmosis-dev] Wrapper scripts for streaming replication
 
 It'd be great to see the replication streaming getting some usage.  It 
 doesn't seem to be used at the moment.  In fact it's been down for a couple 
 of weeks and it's trying to catch up again now.

Do you think it's stable enough to base a service on, or am I better off 
sticking with minutely diffs?

The minutely diffs are used by enough services including OSMF hosted ones that 
you can be fairly confident that any stoppages will be found and fixed quickly, 
but I'm not sure the same is true for streaming diffs.

 There are a couple of example Osmosis command lines on the following wiki 
 page in case you haven't already seen them.
 http://wiki.openstreetmap.org/wiki/Osmosis/Replication#Client-side_Streaming

The pgsnapshot example would work from the shell, but do you have any examples 
for an unattened server?

Also, what's the best way to switch to streaming replication from diffs, and 
the reverse?


___
osmosis-dev mailing list
osmosis-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/osmosis-dev