[OSM-dev] [Nominatim] Import hangs inside virtualized host
Hi,
hope this is the right/best place to ask.
I am trying to import the mapquest npi data inside a virtualized
environment and it gets stuck all the time after a while. Latest svn
version tried is 27508 and last time it got stuck after 38765(k) features.
I don't think the problem is related to nominatim directly but rather to a
problem with inter process communication in the virtualized environment
(afaics nominatim uses pipes to connect to postgres). Probably, the
problem would not occur in a non virtualized environment. Since I still
would like to see it running virtualized, I am trying to understand the
cause of the problem, so I run it straced. Here is the output of the
last few lines (of ~160GB in total) before it got stuck:
recvfrom(3, 2\0\0\0\4n\0\0\0\4C\0\0\0\rDELETE 0\0Z\0\0\0\5I, 16384, 0,
NULL, NULL) = 30
sendto(3, B\0\0\0.\0search_name_delete\0\0\0\0\1\0\0\0..., 69,
MSG_NOSIGNAL, NULL, 0) = 69
poll([{fd=3, events=POLLIN|POLLERR}], 1, -1) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, 2\0\0\0\4n\0\0\0\4C\0\0\0\rDELETE 0\0Z\0\0\0\5I, 16384, 0,
NULL, NULL) = 30
sendto(3, B\0\0\0004\0place_addressline_delete\0\0..., 75, MSG_NOSIGNAL,
NULL, 0) = 75
poll([{fd=3, events=POLLIN|POLLERR}], 1, -1) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, 2\0\0\0\4n\0\0\0\4C\0\0\0\rDELETE 0\0Z\0\0\0\5I, 16384, 0,
NULL, NULL) = 30
sendto(3, B\0\0\0\301\0placex_insert\0\0\0\0\16\0\0\0\n2060..., 216,
MSG_NOSIGNAL, NULL, 0) = 216
poll([{fd=3, events=POLLIN|POLLERR}], 1, -1) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, 2\0\0\0\4n\0\0\0\4C\0\0\0\17INSERT 0 1\0Z\0\0\0\5I, 16384,
0, NULL, NULL) = 32
sendto(3, B\0\0\0q\0place_addressline_insert\0\0..., 136, MSG_NOSIGNAL,
NULL, 0) = 136
poll([{fd=3, events=POLLIN|POLLERR}], 1, -1) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, 2\0\0\0\4n\0\0\0\4C\0\0\0\17INSERT 0 1\0Z\0\0\0\5I, 16384,
0, NULL, NULL) = 32
sendto(3, B\0\0\0
Could somebody tell me what the problem is?
Thanks
fatzopilot
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] [Tilesathome] ANNOUNCEMENT: T@H server will go away end of February
On Wed, 1 Feb 2012 19:44:47 -0500, Jeremy Adams mile...@king-nerd.com wrote: Also thanks to the legions of renderers, client developers, and style tweakers, such as Bob Kare, Petschge, Dirk Lüder-Kreie, the ROMA and TRAPI developers, and the people involved in running the read-only mirror architecture. That was and is amazing work that diverse people have been putting together there. Hi Jeremy, I forgot to mention you by name. Without a read-only API, there would have been no T@H, and although many were involved in creating and keeping it running, special thanks to you for keeping it up and running. Official OSM.org servers had our clients blocked, so were needed something that took over, and you had been doing a great job. The whole thing was pretty stable in the end, I logged into the t@h server only once during the last 9 months and that was after a reboot. :-) Best, Sebastian pgpY0hEf6vgqx.pgp Description: PGP signature ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ANNOUNCEMENT: T@H server will go away end of February
Would it not be possible to have clients render tiles and then share them? mike On Mon, Feb 6, 2012 at 2:06 PM, Simon Poole si...@poole.ch wrote: Am 06.02.2012 10:29, schrieb Matthias Meißer: Am 06.02.2012 09:54, schrieb Sven Geggus: rambling about doing more of the same old stuff deleted If we want to do anything at all officially as a replacement, I would cast my vote clearly in favour of a data tile service with client side rendering. Simon __**_ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.**org/listinfo/devhttp://lists.openstreetmap.org/listinfo/dev -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ANNOUNCEMENT: T@H server will go away end of February
Possible? Sure. Good use of resources? Probably not. Simon Am 12.02.2012 10:00, schrieb Mike Dupont: Would it not be possible to have clients render tiles and then share them? mike On Mon, Feb 6, 2012 at 2:06 PM, Simon Poole si...@poole.ch mailto:si...@poole.ch wrote: Am 06.02.2012 10 tel:06.02.2012%2010:29, schrieb Matthias Meißer: Am 06.02.2012 09 tel:06.02.2012%2009:54, schrieb Sven Geggus: rambling about doing more of the same old stuff deleted If we want to do anything at all officially as a replacement, I would cast my vote clearly in favour of a data tile service with client side rendering. Simon ___ dev mailing list dev@openstreetmap.org mailto:dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ANNOUNCEMENT: T@H server will go away end of February
Well my idea was that people could render on the fly and if they like share those rendered tiles, because those users who willing to donate will help others. Of course there is an issue how to manage the uploads and distribution. Of course your point of why to store tiles at all is valid, but lets assume that some people will still want tiles. mike On Sun, Feb 12, 2012 at 10:09 AM, Simon Poole si...@poole.ch wrote: Possible? Sure. Good use of resources? Probably not. Simon Am 12.02.2012 10:00, schrieb Mike Dupont: Would it not be possible to have clients render tiles and then share them? mike On Mon, Feb 6, 2012 at 2:06 PM, Simon Poole si...@poole.ch wrote: Am 06.02.2012 10:29, schrieb Matthias Meißer: Am 06.02.2012 09:54, schrieb Sven Geggus: rambling about doing more of the same old stuff deleted If we want to do anything at all officially as a replacement, I would cast my vote clearly in favour of a data tile service with client side rendering. Simon ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ANNOUNCEMENT: T@H server will go away end of February
Am 06.02.2012 09:54, schrieb Sven Geggus: To be serious, I consider distributed rendering broken by design. I agree. Here is why: Distributed computing is usually a good idea when computing takes more time than data transmission. Otherwise it just does not make sense to transmit the data rather then just computing it right at the place where it is needed. Additionally to that you cannot render on request, and have to prerender the entire world, which is wasteful in any respect, (Rendering time, transmission volume and disk space) TilesAtHome was successful because of three main factors: The styles were easily modifiable, The Client was relatively easy to set up and when it started it was the only method to get reasonably up-to date Maps. The rendering ruleset became ever more complicated over time, and Mapnik developers added the support for almost instantaneous updates, so only the easy to install part of the client remained, but with that came specialisation on the sole purpose of providing tiles for The TilesAtHome server, and less an easy rendering tool for a Mapper at home. So one possible future of the software would be to transform it into a quick Tiles prerenderer for home use, where a power user could distribute the work on their machines, for their area only, so the server simply goes away. However due to the ever more dense maps this is not something that easily runs on a home pc anymore, because the process of OSM-XML to SVG to PNG tiles is very very inefficient. All in all it was worthwile to let the project run on its own until something broke, but in my opinion it is not really worth to resurrect full scale after the Server goes away. -- Dirk-Lüder Deelkar Kreie Bremen - 53.0901°N 8.7868°E signature.asc Description: OpenPGP digital signature ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Need advice on my software stack: How to display a small/medium mapping project in OpenLayers?
Hi, I'm doing a project on open aviation data. I am using a custom modified rails port for this. Until now I've shown the map this way: * A Google physical as a basemap that shows terrain, borders etc. * My own WFS layer above that shows aerodromes, runways, airspaces, etc. * Both these layers are combined and shown in OpenLayers. The project is starting to grow on me. I want to import all the aviation data from OSM, mainly runways, helipads and terminals/taxiways; Things on the ground. I believe that if I try to show all the new data via the WFS layer it's going to kill performance. I don't think OpenLayers can cope with all those vector features. Therefore I'm thinking I have to do some modifications to my software stack. What do you think is the best solution? My requirements are: * Possibility to show the layer as an overlay, that is, it has to support transparency. This is because I use the Google physical (or possibly other) map as base topographic map. * Some way to show what tags a feature has when you click it. This is important when people use the map to plan flights. * Not require me to render the whole map in all zoom levels, I don't have the hardware to do this. If you want, you can take a look at the page: http://openaviationmap.org Click one of the features to see the functionality. Basically I need something that has the same functionality, but scales a lot better when loads of data is added. Any advice/experiences are greatly appreciated! -- Morten :) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ANNOUNCEMENT: T@H server will go away end of February
That would make one large distributed tile server, but the issue is getting inbound connections to those T@H users running behind firewalls. They can connect OUT to get work and again to push the results out to a central server, but anyone that needs a tile would a) have to know who has it (central server index hit most likely), and b) get a connection INTO the renderer that has the tile. Lynn (D) - KJ4ERJ - Watching for a distributed tile server solution On 2/12/2012 4:09 AM, Simon Poole wrote: Possible? Sure. Good use of resources? Probably not. Simon Am 12.02.2012 10:00, schrieb Mike Dupont: Would it not be possible to have clients render tiles and then share them? mike On Mon, Feb 6, 2012 at 2:06 PM, Simon Poole si...@poole.ch mailto:si...@poole.ch wrote: Am 06.02.2012 10 tel:06.02.2012%2010:29, schrieb Matthias Meißer: Am 06.02.2012 09 tel:06.02.2012%2009:54, schrieb Sven Geggus: rambling about doing more of the same old stuff deleted If we want to do anything at all officially as a replacement, I would cast my vote clearly in favour of a data tile service with client side rendering. Simon ___ dev mailing list dev@openstreetmap.org mailto:dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Toronto Hack Weekend
Dear All, The Toronto Hack Weekend[1] is shaping up nicely for mid-March. You can see that some have registered already on the wiki page. I have verbal confirmation from three others who are each flying in for the event. There will certainly be more. It is critically important to the event organizers that you RSVP early. I have a potential venue sponsor who will make the event even better. I hope to confirm and announce those details this week. I would welcome, in addition to the venue sponsor, additional sponsors to help defray the travel costs for those visiting Toronto for the event, and / or for contributions to defray the food and beverage costs. If you find Toronto to be too far from the US East Coast for your preference or too far in the future, remember that the DC Hack Weekend (Arlington VA) is this weekend. Don't miss it. Best regards, Richard [1] http://wiki.openstreetmap.org/wiki/Toronto_Hack_Weekend_March_2012 [2] http://wiki.openstreetmap.org/wiki/DC_Hack_weekend_Feb_2012 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Toronto Hack Weekend
On Sun, Feb 12, 2012 at 9:47 AM, Richard Weait rich...@weait.com wrote: If you find Toronto to be too far from the US East Coast for your preference or too far in the future, remember that the DC Hack Weekend (Arlington VA) is this weekend. Don't miss it. By this, Richard means this upcoming, not this current. - Serge ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ANNOUNCEMENT: T@H server will go away end of February
I would like to point out this proposal from agron, a fellow flossk member : http://www.flossk.org/en/blog/project-proposal-map-tile-distribution-and-master-directory-map-tile-distributors It shows that tiles are still very interesting for end users. Also the point is that many people have small geographic needs, it would be fine to have a small host that only hosts one city if that server is in a directory mike On Sun, Feb 12, 2012 at 2:51 PM, Lynn W. Deffenbaugh (Mr) ldeff...@homeside.to wrote: That would make one large distributed tile server, but the issue is getting inbound connections to those T@H users running behind firewalls. They can connect OUT to get work and again to push the results out to a central server, but anyone that needs a tile would a) have to know who has it (central server index hit most likely), and b) get a connection INTO the renderer that has the tile. Lynn (D) - KJ4ERJ - Watching for a distributed tile server solution On 2/12/2012 4:09 AM, Simon Poole wrote: Possible? Sure. Good use of resources? Probably not. Simon Am 12.02.2012 10:00, schrieb Mike Dupont: Would it not be possible to have clients render tiles and then share them? mike On Mon, Feb 6, 2012 at 2:06 PM, Simon Poole si...@poole.ch wrote: Am 06.02.2012 10:29, schrieb Matthias Meißer: Am 06.02.2012 09:54, schrieb Sven Geggus: rambling about doing more of the same old stuff deleted If we want to do anything at all officially as a replacement, I would cast my vote clearly in favour of a data tile service with client side rendering. Simon ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org ___ dev mailing listdev@openstreetmap.orghttp://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [josm-dev] Creating a user-friendly changelog for tested release
On Sat, 11 Feb 2012, Simon Legner wrote: Do you think we can merge both lists, at least internally? We'd have one wiki page ChangelogSource and each entry is marked either for changelog or for startup note or for both. This would remove redundancy and make live easier for the translators. Removing duplication is a good idea. However, in my opinion, this only makes sense if the changelog messages are translated to a decent number of languages. Is it common to translate the changelog of a software to several languages? I think the common parts between these two aren't large enough to try to think about joining them. Concerning a ChangelogSource, I don't know whether this is possible from a technical point of view as I'm not familiar with the internals of Trac. A negative aspect (in my opinion) of StartupPageSource is that This is not trac, but JOSM Trac. ChangelogSource could be setup similar to VersionHistory or StartupPage. the translation strings somehow clutters the text which makes it harder to just quickly find a news item or edit the wiki page. This translation system was designed to translate the single-line startup screen. Especially the fact that translated and untranslated strings are mixed in final page was important for this. The drawback is that layout is broken into multiple pieces. Today we have a second system with the language specific pages and the Diff-display showing differences to last translation. I basically developed this to improve translation update process (which works, as translations are updated a lot better than before) and now maintain the relevant Trac plugin. But for the StartupPage I still think the existing line-based system fits better. For Changelog I don't see a big need for translations, so I would think the page based translation system fits better for this one. What I would like to do is to display the translation diff in the edit screen of a translated page, but I was not yet able to do so. Ciao -- http://www.dstoecker.eu/ (PGP key available) ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
