On Fri, Mar 5, 2010 at 2:12 AM, Ian Dees <ian.d...@gmail.com> wrote: > On Thu, Mar 4, 2010 at 9:05 AM, Andy Allan <gravityst...@gmail.com> wrote: > >> On Thu, Mar 4, 2010 at 3:52 AM, Blars Blarson >> <openstreetmap-...@scd.debian.net> wrote: >> >> > This is just to let you all know TRAPI development has been suspended >> > and may never resume. >> >> That's a great shame. Would it be possible for you to commit to svn >> the work-in-progress, in case anyone wants to finish off your hard >> work at some point in the future? >> >> Also, forgive me if this shows my ignorance of TRAPI, but do you think >> it would be feasible to rework the import to use osmosis for the >> replication diff fetching? I was thinking that a custom output plugin >> could be created to write out the osm data in whichever format was >> needed for the rest of the TRAPI update process. >> >> > I started working on a streaming XML output plugin for Osmosis. I was > intending to take advantage of PuSH/PubSubHub messaging and maybe even XMPP > (so that you get a 1-min delayed IM when someone changes something in your > bbox). > > Anyway, TRAPI could use this same plugin to apply updates to their > database. >
I've also spent a fair bit of time thinking about this type of thing. When I first started work on the replication diffs I had in mind a server-side daemon (using Osmosis internally) that would push changes to all connected clients. It would allow a client to connect, specify which replication number it was up to, receive all updates in a single stream, then continue to receive live changes as they occurred. Several issues preventing this are: 1. Writing it. I don't have much time to do it therefore I went with the simplest approach that would work at the time. 2. Cacheability. A push approach is non-cacheable and therefore would increase the amount of bandwidth required. I suspect diff updates are small compared with planet downloads though so perhaps this is a non-issue. 3. TCP port availability. The UCL servers only have limited ports accessible to the outside world. This would probably have to sit behind an Apache web server and therefore be HTTP based. 4. Additional administration overhead. For the last few months I have spent literally zero time looking after the replication updates. I can't remember the last time I logged into the services server. Unless somebody else is stepping in and helping out without me being aware, the current mechanism is very reliable. Occasional db or network errors occur causing the cron-based replication jobs to fail, but they recover and pick up where the left off next time things become available again. In summary, the current approach isn't perfect and could certainly be improved. But unless somebody has the time and inclination to write it themselves, go to the trouble of incorporating it into the existing infrastructure, resolve the inevitable issues that arise, and be prepared to administer it on an ongoing basis, it probably won't change any time soon. But I would really like to see it happen :-) Brett
_______________________________________________ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
