Re: [OSM-dev] [Tilesathome] ANNOUNCEMENT: T@H server will go away end of February
On Wed, Feb 1, 2012 at 6:15 AM, Sebastian Spaeth sebast...@sspaeth.de wrote: Hi all, I was just informed that the central ETH IT took an issue with the constant large bandwidth that the t@h server uses. In addition the server is four years old and getting old and senile... This means that end of February, the T@h server will be shut down and go away. Unless a replacement server (admin) is being found, that will also likely imply that the T@H service will go away (together with the tiles web server) at that point. I have previously argued, why I believe that a t@h service is not that crucial anymore, although I still believe it is good that it was there. I am not sure if and what should replace t@h, but generally speaking there are so many ways now to customize and get rendered tiles, that t@h is a bit of a dinosaur. A fun one though :-) I would like to thank the ETH Zurich, and specifically the Institute for Cartography of Prof. Lorenz Hurni (http://.karto.ethz.ch) that had been sponsoring the server and the bandwidth over the course of four years. They have been very generous, and never complained about the bandwidth/diskspace we were hogging all the time. Finally a special thanks to Claudia Matthys, their IT sysadmin who helped to purchase, setup and administer the thing. Also thanks to the legions of renderers, client developers, and style tweakers, such as Bob Kare, Petschge, Dirk Lüder-Kreie, the ROMA and TRAPI developers, and the people involved in running the read-only mirror architecture. That was and is amazing work that diverse people have been putting together there. I am not that much into mapping any more, but I will hang around and am open to other fun projects. Sebastian Boy, that's a shame. :( Always sad to see the end of a project, even if it has served it's purpose. t@h was the only way to get instant renders after doing mapping work back in the day and really is/was an extremely interesting collection of methods and technologies to make that happen. I've enjoyed all the time I've spent hacking away at it, whether running clients or keeping TRAPI running along. I will truly be sad to see it go. (although it'll be nice to get my 20Mbit of outbound TRAPI traffic back ;)) I'd like to echo Sebastian's statements above and say thanks to everyone who made it happen. There were many hands at work in developing and running the t@h infrastructure before it because as stable as it's been lately and they all desire a heartfelt thank you. -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Minute Diffs broken?
It looks like the minute replicate diffs have stopped at 1071883. http://planet.openstreetmap.org/minute-replicate/001/057/ Could someone please check? Yes they will down until a new osmosis build can be deployed. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ Is that something that's happening the near future? There's a lot of services that use those diffs. Did something unexpected happen that caused this? -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Minute Diffs broken?
On Nov 6, 2011 8:45 AM, Tom Hughes t...@compton.nu wrote: On 06/11/11 13:35, Jeremy Adams wrote: It looks like the minute replicate diffs have stopped at 1071883. http://planet.openstreetmap.org/minute-replicate/001/057/ Could someone please check? Yes they will down until a new osmosis build can be deployed. Is that something that's happening the near future? There's a lot of services that use those diffs. Yes, it will be happening as soon as I get a chance to sort it out. Did something unexpected happen that caused this? No, but the update to rails 3 was done this morning, and that has some schema changes that mean osmosis needs to be updated. I just haven't had time to do that yet. Tom OK, sounds great. Thanks for the update Tom. -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] osmosis stuck, possible to debug?
On Wed, Jun 22, 2011 at 12:51 PM, Stephan Knauss o...@stephans-server.de wrote: Hi, this afternoon my update process got stuck. Running on linux. according to ps it's in state Sl. It's stated with these arguments: osmosis -q --rri --bc --simc --bc --write-xml-change - | I guess killing it is all it takes to recover. But is there any tracing I could enable now to detect why it got stuck and make it possible to fix the issue? Attach a debugger to get a stacktrace or similar? Stephan It appears to have just happened again. At least my TRAPI server did anyway... -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] please help peer a test torrent
my latest experiment, splitting the osm files and sharing via torrent: http://osmopenlayers.blogspot.com/2010/06/first-running-of-osm-gittorrent.html can you please try the torrent? For Kosovo : http://xhema.flossk.org/~mdupont/osm/osmgit-test/kosovo.torrenthttp://xhema.flossk.org/%7Emdupont/osm/osmgit-test/kosovo.torrent For Albania : http://xhema.flossk.org/~mdupont/osm/osmgit-test/albania.torrenthttp://xhema.flossk.org/%7Emdupont/osm/osmgit-test/albania.torrent thanks, mike -- According to my client, your tracker (xhema.flossk.org on port 6969) is refusing connections. -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] TRAPI down?
Hi all, we currently see LOTS of INFO Reset tileset Cause: Download of data failed from API,TRAPI,ROMA in the stats of the ti...@home clients looks like the TRAPI server(s) are currently down and the main API can't feed the resulting flood in batch requests. The logs suggest that this happened about 30 minutes ago. Does anybody know what exactly broke and can perhaps even fit it? Patrick Petschge Kilian My TRAPI server is struggling with what looks like a corrupted DB. It may need to be taken down for a rebuild. The other server is up, but may be slow since it's serving all of the t...@h clients. We could always use another server to spread the load... -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] TRAPI status
On Thu, Mar 4, 2010 at 10:05 AM, Andy Allan gravityst...@gmail.com wrote: On Thu, Mar 4, 2010 at 3:52 AM, Blars Blarson openstreetmap-...@scd.debian.net wrote: This is just to let you all know TRAPI development has been suspended and may never resume. That's a great shame. Would it be possible for you to commit to svn the work-in-progress, in case anyone wants to finish off your hard work at some point in the future? Also, forgive me if this shows my ignorance of TRAPI, but do you think it would be feasible to rework the import to use osmosis for the replication diff fetching? I was thinking that a custom output plugin could be created to write out the osm data in whichever format was needed for the rest of the TRAPI update process. Cheers, Andy This was already done, and both TRAPI servers that are part of the t...@h load balancer use exactly this process. This is the ugly hack that Blars refers to in his first email. I can send more info if anyone is interested in setting it up. I also plan on updating the wiki with this info as soon as I have a chance. -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] TRAPI status
I can send more info if anyone is interested in setting it up. I also plan on updating the wiki with this info as soon as I have a chance. -Jeremy I've gone ahead and updated the wiki. Let me know if anything doesn't make sense. http://wiki.openstreetmap.org/wiki/Trapi -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Removing Minutely and Hourly Changesets
I'm surprised stuff didn't come up sooner too, but yea, unless you broke stuff which you did, unless people found out they wouldn't have noticed most likely... If the maintainer of that site didn't switch over to the replicate diffs in the last months and hasn't fixed the site in the last month that it's been non-functional, then it's hardly Brett's fault. The old diffs were broken, it was wise to force people to the new method. It's not like they weren't given plenty of time to make the switch. I'd be interested in fixing the code for that site if anyone knows where the source can be obtained. I have server space to host it as well. -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Removing Minutely and Hourly Changesets
On Fri, Jan 29, 2010 at 2:21 PM, Stefano Salvador stefano.salva...@gmail.com wrote: Are there any objections to me disabling the old minute and hourly changesets and deleting their files from the planet server? I think that this broke OSM-Live web site: http://datenkueche.com/osmlive/ That looks like a cool site. Is the code to generate the map available anywhere? -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Removing Minutely and Hourly Changesets
How do i init the --rri task (--rrii) to a specific date/time? I am set off at 7th of Jan 0:00 and i'd like to continue with --rri from there. Flo Run ./osmosis --rrii workingDirectory=/work with a suitable directory. Osmosis will create some files in that directory. Open the configuration file and set the url to http://planet.openstreetmap.org/minute-replicate. Set interval to whatever makes sense for your application. Next, find the state.txt file on planet.openstreetmap.org that corresponds to the time you'd like to start at. Download that state.txt file and save it to the osmosis working directory. For instance, the state.txt file for Jan 7 at 00:00 is 167266. Run osmosis with --rri option to start downloading data. -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Removing Minutely and Hourly Changesets
On Fri, Dec 18, 2009 at 11:06 PM, Brett Henderson br...@bretth.com wrote: Are there any objections to me disabling the old minute and hourly changesets and deleting their files from the planet server? http://planet.openstreetmap.org/minute/ http://planet.openstreetmap.org/minute-slow/ http://planet.openstreetmap.org/hourly/ They are replaced by the (hopefully) reliable transaction based replication changesets. http://planet.openstreetmap.org/minute-replicate/ http://planet.openstreetmap.org/hour-replicate/ I haven't heard any complaints about the replication changesets so I'm assuming they're working well. I'll leave the daily changesets running for now because they are running with a much longer delay and shouldn't miss data. Brett There is a fix for the TRAPI servers to use the minute-replicate diffs instead of the regular minute diffs. Both of my servers have been switched over, but I'm not sure about any others. I'd wait to hear from Mathieu Arnold as he's in charge of the other server(s). -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Distributing a VirtualBox HD-File
Hi Ian Peter, I can mirror/host it on my shared server account. I've got unlimited bandwidth and they seem to be pretty good about allowing large file downloads. Of course I'm interested! Next steps will be: 1. complete last tasks 2. make a torrent of it and publish it on osm-talk (soft-opening for technicians to filter out bugs) 3. completion of documentation tutorials for the wiki 4. release - this is when i'll come back to your offer Peter Peter, If needed, I can also put a copy on my planet mirror. Just ping the list once you've got something ready for release. -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Bittorrent mirroring 04-08
-Original Message- From: dev-boun...@openstreetmap.org [mailto:dev- boun...@openstreetmap.org] On Behalf Of Robert Joop Sent: Tuesday, April 14, 2009 8:09 PM To: Stefan de Konink Cc: OSM-Dev Openstreetmap Subject: Re: [OSM-dev] Bittorrent mirroring 04-08 On 09-04-10 23:15:32 CEST, Stefan de Konink wrote: If someone wants to help seeding: http://thepiratebay.org/torrent/4838395 - Download the torrent into the directory where you downloaded the planet - Start rtorrent on the file - Hash is checked, you start seeding :) how about simply starting rtorrent on the file, without downloading the planet via any other method first? (i did this for the first two torrents, btw.) some brief stats from my view: the first planet, -04-01, downloaded pretty quickly. in total, i've uploaded some 15 GB for this file so far, and there's still somebody leeching it from me, at a rate of some 10 KB/s. the second planet, -04-08, started to download very slowly, but after about a day i got some fast seed. up till now, i've uploaded some 7 GB, with one current peer leeching at a rate of up to 115 KB/s. rtorrent reports 4 seeds and 1 leech for each torrent. rj I had added my http mirror in right away on the first planet, but forgot about the second until a day or so went by. When I joined the second planet I was uploading at ~10Mbit for quite some time but has since dropped to nothing. I show one peer for each torrent, but they're not currently connected to me. I've uploaded 26 GB on planet one and 18 GB on planet two. I'll try to remember to jump in on this weeks' planet on Wednesday after the http mirror syncs. -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] openstreetmap could start at user's approximate location using geo-ip
hostip.info shows my IP in the US (VZ DSL), but OSM has always centered me on the UK. It would be a neat feature if this worked. -Jeremy Original Message --- --z4+8/lEcDcG5Ke9S Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jan 18, 2009 at 03:34:14PM +, Tom Hughes wrote: It uses the cookie first, then your home location (assuming you're=20 logged in), then GeoIP via hostip.info, and finally defaults to=20 Europe. Take a look at www.hostip.info if you want to see where it thinks you=20 are. hostip.info locates me at Stuttgart, Germany, but OSM shows Europe=20 centered on UK. Cookies, cache and proxy disabled for testing. CU Sascha --=20 http://sascha.silbe.org/ http://www.infra-silbe.de/ --z4+8/lEcDcG5Ke9S Content-Type: application/pgp-signature; name=signature.asc Content-Description: Digital signature Content-Disposition: inline -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iQEVAwUBSXNh1bpz82VMF3DaAQLA1Af/W621/P/b4FFX9cRQQAi9bIwjasrY5/7J tMIW1qEInLZ0E1F5Z7ewELcogkZLCKPYPbmRSt/oroMR35AhzATrouyi62POUh23 dl+JstVR8r91D8JLg01+FwKJVCnqyOwKjr4rw8hkQlXWSs4zfE/zA4Ull7cCBRHf kpxfey7Viv+CSzezs3fbZrxwdRE6QqF5uZJIjIWScJxrlLWUqdWXAgMObYVgf7Iu MImKqBSFP1qpCSm6NMeGNv6wjerdeogZZP4+1ASNDG6iBJ2tInH18/vVUgP1LA5F RVqjm8OvGOpq0CdxPGx9MHB4C/Q1vLZBvGLcq/JwItQhvU9b8aUI1A== =MEe7 -END PGP SIGNATURE- --z4+8/lEcDcG5Ke9S-- ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Diffs are empty
Hey all, Looks like there's something wrong with diff generation. For the last three hours or so all of the diffs generated by the API server are empty. -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] ROMA servers down - osmosis large way problem
Hey all, There's an issue with the minute change file 200812290912-200812290913.osc.gz. It contains a way which has over 40k nodes. It's an administrative boundary which appears to enclose all of Quebec. It's way id is 29309772. When osmosis attempts to process this file it fails. The relevant parts of the error appear to be Unable to insert new way node and a pgsql error ERROR: smallint out of range. I'm sure I can find the way and split it into smaller chunks, but what's the process for getting the ROMA servers running again? Can we just remove the reference to the way in the changeset and continue updating? The way should be added back in once it's split and reuploaded, correct? -Jeremy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ROMA servers down - osmosis large way problem
It's not the ROMA script that's failing - osmosis is failing while it's trying to update the database using changesets from the main API. Whatever db schema the main API uses must allow for this, while the one for osmosis that was used to setup at least my server does not. -Jeremy Original Message --- Jeremy Adams wrote: There's an issue with the minute change file 200812290912-200812290913.osc.gz. It contains a way which has over 40k nodes. It's an administrative boundary which appears to enclose all of Quebec. It's way id is 29309772. When osmosis attempts to process this file it fails. The relevant parts of the error appear to be Unable to insert new way node and a pgsql error ERROR: smallint out of range. I'm sure I can find the way and split it into smaller chunks, but what's the process for getting the ROMA servers running again? Can we just remove the reference to the way in the changeset and continue updating? The way should be added back in once it's split and reuploaded, correct? So what is the fundamental problem that your tools break on 40k of nodes? Bbox them on the request, and return a partial result that will work just as good for rendering. Stefan ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ROMA servers down - osmosis large way problem
I ran the same command on my box about 4 hours ago and it's still cranking away. Should it really take that long to change a column type? It's got one of the CPU cores pegged, so I assume it's still doing something. -Jeremy Original Message --- On Mon, Dec 29, 2008 at 03:43:27AM +0800, D Tucny wrote: osm= select max(sequence_id) from way_nodes; max --- 39767 (1 row) osm= select * from way_nodes where sequence_id = 39767; way_id | node_id | sequence_id --+---+- 28098452 | 308532457 | 39767 I converted the smallint to int ... Flo So, perhaps following Flo's lead and changing the smallint to int would be the best approach for dealing with getting the ROMA servers back up and running... Both of my databases broke down again as i dropped the databases and recreated them - right now converting again ... I have no clue if this solves it - forgot already what i did last time ;) ALTER TABLE way_nodes ALTER COLUMN sequence_id TYPE integer; Hopefully postgres is clever enough to take over the old content and recreate the index ... Flo -- Florian Lohoff f...@rfc822.org +49-171-2280134 Those who would give up a little freedom to get a little security shall soon have neither - Benjamin Franklin ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
