Re: [OSM-dev] unable to create jar file from the OSM-binary
Hi On 03.06.2015 16:49, Holger Jeromin wrote: Laura Barroso wrote on 02.06.2015 20:22: Hi, currently Im trying to compile the OSM-binary library however theres an error that keeps bugging me: I think the reason why no one is helping you (on list) is, that nobody knows was you are doing. What is the the OSM-binary library ? I guess he/she's talking about https://github.com/scrosby/OSM-binary This text looks like Java stuff... Which is why I can't help either. Regards, Peter signature.asc Description: OpenPGP digital signature ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Fwd: [Feature suggestion] importing bitmap/vectir gfx fort easing map editing
Hi JOSM is already capable of doing this using the PicLayer Plugin: http://wiki.openstreetmap.org/wiki/JOSM/Plugins/PicLayer I'm not sure what you mean with in OSM. Regards, Peter On 02.06.2015 10:56, Andréa Loyer wrote: Hello, I juste started editing some laps, ans if I think osm édit interface prety good, IT would be a killer feature to be able to load a picture (bitmap or vector gfx) to rotate it and scale it to fit the map in order to place installations and roads precisely. It seems quit simple to add, at least in JOSM, but even in OSM. The displayed local picture should be set with a reasonable transparency to ensure correct visibility. Rotation and scaling should be available through mouse pointing/drag/wheel. Best regards, -- A. ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev signature.asc Description: OpenPGP digital signature ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Leaflet Plugin: Interactive Country-Sized Vector Layer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi my Employer allowed me to publish a Leaflet-Plugin I've been working on some time and he also gave me the time to write some Documentation around it. I'd like to show it to you, hoping that it may help you some day. Often our customers want Maps where they can hover over Countries and Click on them. Often they want us to paint the Countries depending on some current data (ie: Do we have Jobs in this country right now?) Both is not satisfyable with pre-rendered Tiles. Rendering Vector-Maps in the Browser is nothing new, but I have not seen an Implementation which can render the whole World in good enough Quality to show small Countries like Singapore [1]. For this reason I implemented a Canvas-Based Vector-Layer that can render a complete Worldmap in the Browser and uses some Canvas-Tricks to provide a Hitmap to change the look of Countries in Realtime while the Mouse-Cursor is hitting them. We published it on Github: https://github.com/Personalwerk/Leaflet.InteractiveCountryVectorLayer I'll be happy to accept Pull-Requests for things that are missing or nonfunctional. Regards, Peter [1] http://www.openstreetmap.org/#map=10/1.3340/103.8510 -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJUyQUMAAoJECqyljcQiW2wCfwH/ipjhLWlNhUK1mY1edLomOwM HXqnUzXSprqWyk2yDm3l7+AMEsd2/kJKmpSYF/2nD38Jav6uZ7y1PLnx28eLlCSS YykBCg1+sprvy/7os2dsnw5V3ncOrjoPZPezxV5tgjMCNcgg9eqHBicseRExDYPd GdWWKwKKCNm5BxDr269xso0VZtzCQdvF2YgNQAq3ZP6sSOWJ+72nnzy3N+iGU/sr +PE+aurov99KK/zTRiTmo86923AQTjxolmtiAIXa4ywwPKp4XQLULQC/ezmVgwRB t3HLTv0305Dr0PK4BI6xemxji17JaDUr2joAxiaWN08AYcAcgq2zk87Je56Lv+E= =9Q+h -END PGP SIGNATURE- ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] using Osmium to filter osh files
Hi joost, hi Abhishek, I've mailed with both of you because you asked me questuons on history processing. Abhishek is trying to process a much larger amount of data that can't easily be imported into a database, like joost did. See https://wiki.openstreetmap.org/wiki/OSM_History_Renderer for some numbers on large (Abhishek = Dalek2point3). Mainland USA did, according to Abhishek, not import successfully on a 120GB-RAM Amazon instance. At this point, importing into a database does not seem suitable. As Abhishek does not need minutely accuracy, jochens advice given on the #osm-dev irc channel may make things a lot easier: use the osmium_range_from_history-tool or the underlying RangeFromHistory-handler to extract all objects valid at a given point in time and do your analysis on top of that. then continue to the next point in time and read the input file again with a new filter. Reading the files multiple times to avoid the need for massive amounts of memory often works well. This technique is also used in the history-splitter for example. Regards, Peter Am 24.05.2014 23:45, schrieb joost schouppe: Hi Abichek, I've been using Mazderminds History Splitter and Importer to create a Postgres database to do similar analysis. See https://github.com/MaZderMind/osm-history-renderer/blob/master/TUTORIAL.md and my diary for how far I've gotten: http://www.openstreetmap.org/user/joost%20schouppe/diary/21826 I'm a complete newbee, but from what I hear, it sounds like this would give you an easier framework than working directly with Osmium? I'll send you a personal mail too, because I'm working on a somewhat similar project. Joost ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] existing object not found, 404 error
Hi Am 31.03.2014 11:18, schrieb amrit karmacharya: i am trying to query this node* http://www.openstreetmap.org/node/2400031584*. it opens up in browser but on accessing with script at the url* http://api.openstreetmap.org/api/0.6/node/24000315844/* returns object not found. http://www.openstreetmap.org/node/2400031584 http://api.openstreetmap.org/api/0.6/node/24000315844/ The API-URL has an excess 4. Regards, Peter ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Full history dump
Hi weekls -experimental- dumps here: http://planet.osm.org/planet/experimental/ Regards Am 22.03.2014 11:29, schrieb yvecai: It's about time to renew yearly stats of ski mapping in OSM. I can find Mazdermind history-dump extracts from 03.03.2014, but no complete file ? Before I use those, is the complete dump available somewhere, or a new dump scheduled ? Yves - opensnowmap.org ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Full History Extracts
Hi Am 26.02.2014 11:56, schrieb bibekshres...@gmail.com: May be someone from Geofabrik can help do a regular dump here http://download.geofabrik.de/asia.html that would be big help. I'm trying to do weekly extracts. Just check out http://osm.personalwerk.de/full-history-extracts/. Regards, Peter ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Full History Extracts
Am 02.03.2014 07:23, schrieb amrit karmacharya: Thanks for the extract, I just went through http://osm.personalwerk.de/clipbounds/asia/ would like to share that india and pakistan also lie in southern-asia *g thank you. fixed that. Next week's extract will show the repaired directory structure. Regards, Peter On Wed, Feb 26, 2014 at 1:43 PM, Peter Körner osm-li...@mazdermind.de mailto:osm-li...@mazdermind.de wrote: Hi Am 25.02.2014 07:45, schrieb amrit karmacharya: Thank you very much, this is a big help to developers here. Np. If you need other extracts - smaller or bigger - just drop me a line at my private address: o...@mazdermind.de mailto:o...@mazdermind.de I'd be gladfully using the processing resources available to me to help people with thin internet connectivity. Regards, Peter On Mon, Feb 24, 2014 at 5:03 PM, Peter Körner osm-li...@mazdermind.de mailto:osm-li...@mazdermind.de mailto:osm-li...@mazdermind.de mailto:osm-li...@mazdermind.de wrote: Hi Am 14.02.2014 05:56, schrieb amrit karmacharya: Can you make a extract for Nepal? As of now, I need to download the whole asia and extract nepal which covers only 0.3% of asia. There are no other options available. Here you go: http://osm.personalwerk.de/full-history-extracts/latest/asia/southern-asia/ I'll try to update them weekly. Regards, Peter ___ dev mailing list dev@openstreetmap.org mailto:dev@openstreetmap.org mailto:dev@openstreetmap.org mailto:dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Full History Extracts
Hi Am 25.02.2014 07:45, schrieb amrit karmacharya: Thank you very much, this is a big help to developers here. Np. If you need other extracts - smaller or bigger - just drop me a line at my private address: o...@mazdermind.de I'd be gladfully using the processing resources available to me to help people with thin internet connectivity. Regards, Peter On Mon, Feb 24, 2014 at 5:03 PM, Peter Körner osm-li...@mazdermind.de mailto:osm-li...@mazdermind.de wrote: Hi Am 14.02.2014 05:56, schrieb amrit karmacharya: Can you make a extract for Nepal? As of now, I need to download the whole asia and extract nepal which covers only 0.3% of asia. There are no other options available. Here you go: http://osm.personalwerk.de/full-history-extracts/latest/asia/southern-asia/ I'll try to update them weekly. Regards, Peter ___ dev mailing list dev@openstreetmap.org mailto:dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Full History Extracts
Hi Am 14.02.2014 05:56, schrieb amrit karmacharya: Can you make a extract for Nepal? As of now, I need to download the whole asia and extract nepal which covers only 0.3% of asia. There are no other options available. Here you go: http://osm.personalwerk.de/full-history-extracts/latest/asia/southern-asia/ I'll try to update them weekly. Regards, Peter ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Full History Extracts
Hi Am 14.02.2014 05:56, schrieb amrit karmacharya: Can you make a extract for Nepal? As of now, I need to download the whole asia and extract nepal which covers only 0.3% of asia. There are no other options available. I can add Nepal to the list of extracts, if you supply me with a bbox or an outline of the area you want. I'll take tome time though, because the server's harddisk have an outage an we're preparing a repair atm. While it's in this degraded state I'll not run the extract service. Peter ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Experimental history files too
Am 28.01.2014 03:18, schrieb Matt Amos: On Fri, Jan 24, 2014 at 2:56 PM, Peter Körner osm-li...@mazdermind.de wrote: What do you think about adopting the osmium-naming-scheme for history files? personally, i think it's misleading [...] in the case of .osm files, they're all potentially history files, and the file format does not change depending on whether multiple versions are present for a single ID or not. History-Files carry an extra attribute (bool visible) that distinguishes them from regular osm files. Also it's not only about the parser. osm2pgsql's parser is absolutely capable of parsing files with multiple versions of an object, but the whole processing chain will crash with wired errors. Same for osmosis. Its parser will work but only a small fraction of tasks will. So from a data-user point of view it does not matter if the format is actually-quite-similar, as long as there are separate programs and tools required to handle the two types of file, they are actually different. I'd compare it more to tiff/aiff. While they are actually quite similar from a file-format point of view, no one would argue that audio-files and image-files should be handled as if there were no difference. whether something is a history osm file or a current osm file is a matter of the content - so wanting a different extension is a bit like wanting .png for truecolour images and .pgr for greyscale images (in the same PNG format). The main difference here is that all applications capable of reading png MUST be capable of reading pgr as well. That's not true for osm/osh applications. Also the tasks you can perform on both file formats are the same (display, crop, combine, ...). This is also not true for osm/osh files. One can import both into a database but the database-format required and the actions possible on those databases are quite different. having said that, it would seem reasonable to add a flag to the document element to indicate whether the .osm file is a special case, having a single version for each ID, as many programs seem to rely on this assumption and it would be better to be able to check it. Isn't that already implemented via the required_features header of pbf-files? See https://github.com/joto/osmium/blob/master/include/osmium/output/pbf.hpp#L880 for a reference. the generation is synced to the backup database dumps, so the clock starts running early Tuesday, when Monday's backup is complete. they seem to be fairly reliably finished by Wednesday morning, so it's probably safe to start looking for them then - although they'll be named for Monday's date. Thank you for that info, I'll see when I can setup my regular splitting task and announce sepeately. I xml-writing takes only half as long as xml-reading I'd double-think about supplying xml-based files. nobody really has fun reading such huge files with expat. And if it's really neccessary, there's always osmium_convert which will generate xmls from pbf-dumps or -extracts locally. this is a discussion which could probably continue forever. my opinion is that it's worthwhile distributing files which are sort-of human-readable, in a well-known format/markup for which many libraries exist in many languages, compressed with standard tools, and in the same format as the API. got your point. Peter ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Experimental history files too
Hi Am 23.01.2014 18:28, schrieb Matt Amos: i encourage everyone to take a look and report back any problems you find. my thanks to Peter Körner, who seems to already be doing this - with no problems? No problems yet. Have run two splits of those files already (http://osm.personalwerk.de/full-history-extracts/) What do you think about adopting the osmium-naming-scheme for history files? .osm.[bz2|gz|pbf] - regular osm files .osh.[bz2|gz|pbf] - history files .osc.[bz2|gz]- changeset-files That would make detecting the kind of file at first glance more easy and it also fits nicely int othe .osc-file nameing convention. I'm going to implement a regular run that generates fresh extracts every week from the available file. Is there any note on which weekday the full-history-dumps are generated, so I can loosely sync my split-script to that rhythm? I xml-writing takes only half as long as xml-reading I'd double-think about supplying xml-based files. nobody really has fun reading such huge files with expat. And if it's really neccessary, there's always osmium_convert which will generate xmls from pbf-dumps or -extracts locally. Regards, Peter ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Full History Extracts
Hi I made fresh history extracts [1] from the new full-history dumps [2]. If you miss your region or want a small custom tailored area, just drop me a line or follow the instructions at [3] to split the file yourself. Regards, Peter [1| http://osm.personalwerk.de/full-history-extracts/2014/2014-01-06/ [2] http://planet.osm.org/planet/experimental/ [3] https://github.com/MaZderMind/osm-history-renderer/blob/master/TUTORIAL.md ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] History Extracts available
Hi The Splitter has finished anbd here are the cut-down files: http://osm.personalwerk.de/full-history-extracts/planet-history-2013-11-02/ Have fun! Peter Am 05.11.2013 13:54, schrieb Peter Körner: Hi Jochen, thank you very much, the splitting process is running: http://osm.personalwerk.de/full-history-extracts/planet-history-2013-11-02/ Regards, Peter Am 04.11.2013 09:26, schrieb Jochen Topf: Due to popular demand I have made a planet file with full history available at http://data.openstreetmapdata.com/planet-history-2013-11-02.osh.pbf The file has about 37 GB, it's MD5 is 22169f0f997391b090df546769eb2357. This was generated from the last official history and all the daily diffs until two days ago. THIS IS NOT AN OFFICIAL HISTORY FILE. I haven't tested it or verified that my code that generates this is doing the right thing, so no guarantees. (On the other hand there is not much to it, really, just collect all the objects in the planet and change files and put them in the right order.) Apply reasonable caution when using it. If there is demand I can make a osm.bz2 version available, too. Jochen ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Planet History file available
Hi Jochen, thank you very much, the splitting process is running: http://osm.personalwerk.de/full-history-extracts/planet-history-2013-11-02/ Regards, Peter Am 04.11.2013 09:26, schrieb Jochen Topf: Due to popular demand I have made a planet file with full history available at http://data.openstreetmapdata.com/planet-history-2013-11-02.osh.pbf The file has about 37 GB, it's MD5 is 22169f0f997391b090df546769eb2357. This was generated from the last official history and all the daily diffs until two days ago. THIS IS NOT AN OFFICIAL HISTORY FILE. I haven't tested it or verified that my code that generates this is doing the right thing, so no guarantees. (On the other hand there is not much to it, really, just collect all the objects in the planet and change files and put them in the right order.) Apply reasonable caution when using it. If there is demand I can make a osm.bz2 version available, too. Jochen ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] GraphHopper Maps 0.1
Hi Am 24.07.2013 06:52, schrieb Kai Krueger: I was curious to see how it compares in speed and quality of calculated routes to the other engines, so I took the liberty to add it to http://apmon.dev.openstreetmap.org/routing It seems the labeling is wrong. When using the Grasshopper Router it always prints Route was calculated by Cloudmade. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Plotting data on an equirectangular map, distorted for sphere projection
Hi Am 17.07.2013 15:41, schrieb Christopher Stevens: Hello gurus, I'm thinking that I may need to find some alternative data visualization software to make this work, but I love mapnik and thought it would be good to start here first. Mapnik is capable of rendering data in nearly any possible projection - and it can be done with osm-data, too. On http://polar.openstreetmap.de/ we do a rendering in EPSG:3031 Antarctic Polar Stereographic projection. Code and documentaztion can be found here: https://github.com/MaZderMind/mapnik-stylesheets-polar However those projections are applied to the geometries *before* rendering out a planar, rectangular image. I'm hoping to create a basic themed map of different types of data and plot it on a full-Earth image (I can figure this part out, starter examples cover this as well). The challenge is that I'm then projecting this on Science On a Sphere http://www.spacefoundation.org/visit/northrop-grumman-science-center-featuring-science-sphere/science-sphere%C2%AE, which requires equirectangular images that distort more towards the poles. Here's a quick example of something I made in a 3D program to show latitude and longitude lines (note how the lines and labels get fuzzy and stretched near the poles). http://www.flickr.com/photos/8139783@N08/9305646391/ This however looks like an post-render distortion, so I'd guess you'd have to apply some filter to the rendered mapnik images. Is there a way to add this type of distortion in mapnik? I'm guessing not as mapnik is optimized for other uses (but who knows). I'm guessing that it would work as is, but labels would get distorted (there are work-arounds), and lines would get very thin near the poles. I'd bet that it would be better to use sth. like graphicsmagick or a custom program with libpng to do the pixel-transformations after the rendering finished. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] projection of the tables in the database
Hi by default they are in EPSG:900913, which is the deprecated name of EPSG:3857: http://spatialreference.org/ref/sr-org/6864/ Peter Am 11.07.2013 09:59, schrieb natalia aznar nuñez: Good morning. I am having troubles with the osm2pgsql. I would like to use the tables of nodes, ways and relations, and I don't know in which projection they are. I hope you can help me, I will be very thankful for your help. Natalia. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile on windows?
Am 07.06.2013 17:29, schrieb Vince Berubey: Hi Sven, I was a looking for a way to generate tiles on the fly on windows, so I would not have to use the generate_tiles.py. You could try to use - http://blog.jochentopf.com/2011-03-03-a-nodejs-tileserver-for-tirex.html - http://lists.openstreetmap.org/pipermail/dev/2011-April/022186.html - or the module included in TileMill http://www.mapbox.com/tilemill/ Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [osmosis-dev] Usage of EntityMerger
Hi just by looking at the code https://github.com/openstreetmap/osmosis/blob/master/osmosis-set/src/main/java/org/openstreetmap/osmosis/set/v0_6/EntityMerger.java?source=cc it looks like instanciating your EntityMerger creates two internal sinks (sortedEntityValidator0 and sortedEntityValidator1) which are queried for data. It looks like you can access them using getSink(0) and getSink(1). So I'd guess you could do sth. like this: XmlReader xmlReader0 = new XmlReader(inputFile0, true, CompressionMethod.None); XmlReader xmlReader1 = new XmlReader(inputFile1, true, CompressionMethod.None); EntityMerger merger = new EntityMerger( ConflictResolutionMethod.Timestamp, 1000, BoundRemovedAction.Ignore); xmlReader0.setSink(merger.getSink(0)); xmlReader1.setSink(merger.getSink(1)); MyOwnSink mySink = new MyOwnSink(...); merger.setSink(mySink); xmlReader0.run(); xmlReader1.run(); The question is: will xmlReader0.run() block or not? If it blocks, you'd probably have to start two threads and run one xmlReader per thread, joining them back when both are finished. Lg, Peter Am 28.05.2013 16:13, schrieb Richard Redweik: Hi list, I am trying to implement my own process to read osm data. Therefore I am using the osmosis jars from the maven repository. Currently I am connecting my tasks by hand. For example: XmlReader xmlReader = new XmlReader(inputFile, true, CompressionMethod.None); PolygonFilter polygonFilter = new PolygonFilter(IdTrackerType.Dynamic, polygonInputFile, false, false, false, false); xmlReader.setSink(polygonFilter); MyOwnSink mySink = new MyOwnSink(...); polygonFilter.setSink(mySink); xmlReader.run(); Here MyOwnSink is a Sink written by myself. This works as expected. Now I want to read two OSM files and merge them with the EntityMerger. But I do not know how to let the EntityMerger know about the two OSM files. Calling xmlReader.setSink(merger) is not possible since EntityMerger is no Sink, but a MultiSink. Does someone know, how to use the EntityMerger with two or multiple OSM files? Can I add the EntityMerger into my workflow, which I described above? Thanks in advance, Richard ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [OSM-dev] Import a huge amount of GPS data
Am 23.05.2013 16:00, schrieb Jingmin Chen: Hi Martin, Thank you for your comments. Although 30 seconds are large interval, but when I aggregate all data in a period from 20,000 vehicles, the trajectory is very dense, and cover almost all the roads. For legal issue, I will investigate that. But definitely I am not going to upload the original GPS data. Would uploading only the points in a randomized order be okay for you? The actual track (the connection between the points) is - for a 30s interval, nearly useless for us. There is a whole bunch of tools that yould achieve this: http://wiki.openstreetmap.org/wiki/User_talk:DerKuchen#GPX_editing_tools http://wiki.openstreetmap.org/wiki/GPSBabel Ans possibly more tools on http://wiki.openstreetmap.org/wiki/Edit_GPS_tracks At the very last it should not be too hard to write a small script to do that work for you. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] osm2pgsql: support for local route names
Am 11.04.2013 19:57, schrieb RainerU: As I am not familiar with github, I would need some hints how to proceed. The usual approach would be fo fork the osm2pgsql repo, create a feature-branch, check out that branch, change your code inside, test, commit and push the new branch to your fork of the repo. Then you can issue a pull-request from your branch to the main-master. It may sound a little scary when written down, but it is a relativly straightforward process, once you know the steps involved. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Tiled Vector Data for Mapnik!
Hi looks interesting! Are the tiles updated? Peter Am 16.03.2013 21:41, schrieb Michal Migurski: Hi, I've released two experimental layers of tiled vector data on the OSM US server, and I'm looking for feedback. Documentation for layers: http://www.openstreetmap.us/~migurski/vector-datasource/ Meandering explanatory blog post: http://mike.teczno.com/notes/postgreslessness-mapnik-vectiles.html Other OSM stuff I've been working on: http://www.openstreetmap.us/~migurski/ If these prove popular and useful, I hope to transition them from experimental service to reliable, speedy service. Please let me know what you think! -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Zeroing or Extending a Hillshade-Tiff
Am 11.03.2013 21:10, schrieb Sven Geggus: Peter Körner osm-li...@mazdermind.de wrote: what way would you choose and wich tools could be used to go that way? Use the Water-Polygons from openstreetmapdata.com instead of land polygons. That's not that easy for the polar regions, because transforming the splittet polygons to the destination projection results in gaps because of straigt lines not beeing curved. For the land there are complete polygons which are used on the Antarctica map but there are no complete water polygons, because they would be too complex (a huge polygon with 400.000+ holes). So in order to get this right one would need to modify jochend osmcoastline tool to support EPSG:3031 natively, to all corrdinates are transformed before splitting and calculating the water polygons. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Zeroing or Extending a Hillshade-Tiff
Hi I have a Hillshading-Tiffs (a GeoTIFF with a single Greyscale-Band) in the correct projection and correctly aligned. My problem is, that it's slightly smaller then the area I'd like to render using mapnik: http://imageshack.us/photo/my-images/202/bildschirmfotovom201303.png/ There are options i'm aware of: a) set the water level (its values 180-182 in the greyscale-band) to transparent, so there's no visible border at the end of the image b) extend the image with value 181 at the borders to fit the size of the image i'm going to render My question is: what way would you choose and wich tools could be used to go that way? Regards, Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] osm-history-renderer now for bigger areas
Hi all I added a new feature to my osm-history-renderer/-importer that allows you to handle much larger areas, by using the RAM more efficiently. In theoriy it should be possible wo import a fill history planet with ~32GB of RAM. On my Notebook I imported rheinland-pfalz.osh.pbf (308M) with the sparse nodestore. It took around 1.2 GB of RAM from which apparently ~700M was taken by the nodestore and 400M by the pbf reader. Process Runtime was around 30 Minutes. The generated Tables on disk took ~14 GB including indexes. In comparison, africa.osh.pbf and north-america.osh.pbf are of a similar size and I guess one could even import netherlands.osh.pbf on a home notebook with 4 GB of RAM. Maybe even germany.osh.pbf or france.osh.pbf would work out with 6 GB or 8 GB. I'd love to hear feedback about this. Current source and instructions can be found on https://github.com/MaZderMind/osm-history-renderer Current Extracts can be found at http://osm.personalwerk.de/full-history-extracts/history_2013-02-05_1701/ Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Reloading data without effect
Hi Am 13.02.2013 12:29, schrieb Krzysztof Rutkowski: Should I do something else after import data to have updated tiles? How about rerendering the tiles? You didn't tell us much about your setup but I'm guessing you're going the postgres/osm2pgsql/mapnik/mod_tile route with either renderd or tirex. After reloading (or updating) your data, you got two options: - rerender all tiles in background use render_list or one of its companions (for renderd) use tirex-batch (for tirex) - let mod_tile request the rerender on-access touch /var/lib/mod_tile/planet-import-complete to tell mod_tile the new date of your database, so it can recognize old tiles and request a rerender on access Regards, Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Latest full-history planet
Hi I re-cuttet one using the geofabrik poly file, you can find it here: http://osm.personalwerk.de/full-history-extracts/history_2013-02-05_1701/europe/france/ Its extend won't be identical to the geofabrik, because there they first extract europe from the world, then france from europe and then the regions from france, so in the end the resulting polygon would be the union of the three. I for my part usually do the extracts directly from the world file, so it may contain more data then the geofabrik files. Peter Am 15.02.2013 12:21, schrieb kimaidou: Thanks a lot ! I have seen the extract for Provences Alpes Côte d'Azur in France [1] is very small (smaller than Paris). Is it possible to use the clipbound from geofabrick [2] instead of the bbox ? The corresponding file is attached. Thanks in advance. This will help me testing the updated scripts [3] Kimaidou [1] http://osm.personalwerk.de/full-history-extracts/history_2013-02-05_1701/europe/france/provence-alpes-cote-d-azur.osh.pbf [2] http://download.geofabrik.de/clipbounds/clipbounds.tgz [3] https://github.com/MaZderMind/osm-history-renderer ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Latest full-history planet
Hi here it is http://osm.personalwerk.de/full-experimental/ Regards, Peter Am 14.02.2013 19:34, schrieb Peter Körner: Hi Am 09.02.2013 11:08, schrieb Paweł Paprota: Is anyone planning to produce a full history PBF like the last time in October? It's on the run, as well as the extracts. Regards, Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Latest full-history planet
Hi Am 09.02.2013 11:08, schrieb Paweł Paprota: Is anyone planning to produce a full history PBF like the last time in October? It's on the run, as well as the extracts. Regards, Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Planet file date unclear at planet.osm.org
Am 11.01.2013 15:17, schrieb sly (sylvain letuffe): bzcat planet-130102.osm.bz2 | head -n 2 Or without downloading the whole file: curl http://planet.openstreetmap.org/planet/planet-latest.osm.bz2 | bzcat | head -n 2 Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Is YOUR code 64 bit proof?
Am 30.12.2012 21:26, schrieb Andrew: Could you store node numbers as strings? You could do so and use bcmath for all your calculations, but depending on the size of the region you're working with it would take a whole lot of time and memory to do so. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Is YOUR code 64 bit proof?
Hi mine is (although not yet in the master branch): https://github.com/MaZderMind/osm-history-splitter/compare/64bit https://github.com/MaZderMind/osm-history-renderer/compare/64bit I have a pull request for osmium pending which is required for these branches to be merged into master: https://github.com/joto/osmium/pull/60 Am 27.12.2012 14:55, schrieb Frederik Ramm: There's a README there. A little thing to add: the bbox of your full-64.osm is: minlon=-61.8114 minlat=17.1255 maxlon=-61.7733 maxlat=17.1542 This info is helpful for testing your renderers. Thank you for providing this really helpful test-files! Have a nice new year! Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Snapshot Server
Hi Rob, Am 27.12.2012 15:41, schrieb Rob Nickerson: However, when I run rake db:migrate I get the following error: PG::Error: ERROR relation “geometry_columns” does not exist LINE 1: SELECT * FROM geometry_columns WHERE f_table_name='schema_migrations' Sounds like your database is missing postgis. Take a look at the database section on my history-rendering tutorial: https://github.com/MaZderMind/osm-history-renderer/blob/master/TUTORIAL.md#creating-the-database the other parts of the document are not relevant for the snapshot server. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] making osm2pgsql import relations?
Hi Am 07.11.2012 22:07, schrieb Ákos Maróy: my relations are a bit different - I'm collecting all data related to an airport in a relation. this includes a number of polygons, lines points. I'd like to see the same relation in the database as described in the OSM XML file Well, you usually use a common tag for that instead of a relation. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Compressed vector tiles with mod_tile?
Hi Am 04.11.2012 05:06, schrieb Kai Krueger: Overall this works very well in combination with e.g. KothicJS[2]. However, as text based json is not very space efficient, Cover stores the json compressed in the metatiles. Mod_tile can't yet natively deal with this and doesn't automatically set the HTTP header Content-Encoding: gzip and I'd like to fix this. Just a thing I stumbled across the other day: http://code.google.com/p/protobuf-js/ It's a really small library but it could save us, together with some tricks, a lot of the GeoJSON-overhead. Possible tricks could be the mentioned Transfer-Encoding: gzip, using Integers for the Coordinates, store only the difference to the prev. coordinate, using a stringtable -- pretty much what the osm.pbf format useses to make our files smaller. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Increase max zoom past 16
Am 01.11.2012 21:03, schrieb Sam Rose: !ENTITY minscale_zoom18 MinScaleDenominator1000/MinScaleDenominator however this did not change anything. Can anyone help? Are you using the renderd or the tirex stack? Check their configs. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ODbL full history planet
Am 25.10.2012 17:09, schrieb kimaidou: Hi Peter, I would like to know if you could build a small extract for me (or point me to a tutorial to do so myself) . I need to run a test for the europe/france/provence-alpes-cote-d-azur.poly contained in the geofabrik zip here : http://download.geofabrik.de/clipbounds/clipbounds.tgz Or the bbox if easier : europe/germany/rheinland-pfalz.osh.pbf BBOX 4.303,43.847,5.375,44.493 Here you go: http://osm.personalwerk.de/full-history-extracts/history_2012-10-13_13:35/europe/france/ Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ODbL full history planet
Am 23.10.2012 10:13, schrieb kimaidou: Could you give the github repository url for the osm2pgsql-style table tool ? https://github.com/MaZderMind/osm-history-renderer To start I'd recommend following the Tutorial: https://github.com/MaZderMind/osm-history-renderer/blob/master/TUTORIAL.md I'd say just take the osmium lib and the postgres lib and write yourself an importer (and put it on github). This is what I would do I think. I just need time to get used to osmium... I usually code in python, not C++ ... And obviously to find enough time. I will sure put it on github if I managed to developp this tool Sound's nice. Doing the processing with C++ gains you a lt of speed over python. (a lot as in: 1 hour for a planet instead of 1 day) Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ODbL full history planet
Hi Am 23.10.2012 23:26, schrieb Pedro Larroy: Do you have a pointer to the pbf definition files? It's in the usual osm.pbf-File-Format: https://github.com/scrosby/OSM-binary The only difference is, that the visible-field is filled and there are more then one entity per id (one for each version). See: https://github.com/scrosby/OSM-binary/blob/master/src/osmformat.proto#L130 Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ODbL full history planet
Am 22.10.2012 00:43, schrieb Stephan Knauss: On 19.10.2012 16:53, Matt Amos wrote: now available from: http://planet.openstreetmap.org/planet/full-history/ only limited testing has been done on this file (checking for XML well-formed-ness and various spot-checks on elements), so please let me know if you find any errors with it. I have not yet downloaded it, hoping Peter will make the pbf conversion available which is a lot faster to process and saving my time to do the conversion by myself. Here we go: http://osm.personalwerk.de/full-experimental/ And the generated extracts are here: http://osm.personalwerk.de/full-history-extracts/history_2012-10-13_13:35/ As always, if you need another extract (and don't want/can't to generate it yourself), drop me a line. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ODbL full history planet
Hi Am 21.10.2012 20:42, schrieb Martijn van Exel: Also, I was going to make US state and county cuts of this file and possibly make them available on our OSM US server - Peter, my osm-history-splitter is a few months old, any reason to pull and recompile? Just get a fresh osmium and splitter just to be on the safe side. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ODbL full history planet
Am 19.10.2012 16:53, schrieb Matt Amos: now available from: http://planet.openstreetmap.org/planet/full-history/ only limited testing has been done on this file (checking for XML well-formed-ness and various spot-checks on elements), so please let me know if you find any errors with it. Thanks for this, I just kicked on the pbf-converter-and-split-process and on Monday we'll see if it worked out. Will report back, then. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] History statistics
Hi no it was a mistake. Basicly atm there is one huge nested map, that holds all required information: nodemap[Objekt-ID][Timestamp] = (lat, lon) lat/lon is atm stored as double. The suggested memory layout is as follows: 1. An Array with a 32bit Pointer for each Node-ID 2. A Series of Timestamp-Lat-Lon Triplets, each stored as Integers, all together on one row for all versions, ending with a 0-byte Memory-Usage would be 8*(max. node-id) for 1. 4*(number of used node-ids) + 12*(number of used node-version) for 2. Maximum memory usage for a complete planet would be ~40GB. With a normal Array for 1. the minimum memory usage would be ~10GB. This could be reduced by using a sparsetable http://google-sparsehash.googlecode.com/svn/trunk/doc/sparsetable.html Peter Am 16.10.2012 10:50, schrieb Ab_fab: Thanks for your answer Peter. You did not answer to the list, was it intentional ? I forwarded your reply to the French list thread [1], where the initial discussion with Aurelien and others took place, back in july. I totally miss the skills required to implement anything ... Anyway, it could be of interest for some people, in particular for stats purpose on the evolution of the database contents (not only the number of buildings from cadastre ;-) ). French community has also (I think) enough material ressources to setup such low activity database, once this RAM issue for the import is solved. Regards Fabien [1] http://lists.openstreetmap.org/pipermail/talk-fr/2012-July/045337.html http://lists.openstreetmap.org/pipermail/talk-fr/2012-October/049849.html 2012/10/16 Peter Körner pe...@mazdermind.de mailto:pe...@mazdermind.de Hi Well, osm-history-renderer is - after all - a proof of concept. It's impossible atm. to import a full country without having tons of RAM. If you are interested in optimizing the memory usage I could provide you with plans about a memory layout frederik ramm and I once talked about; I never came around to implement it, while the necessary interfaces are already in place. Regards, Peter Am 15.10.2012 15:08, schrieb Ab_fab: I think the memory problems occured during the osm-history-importer [1] process Some people were successfull in building the database for limited areas, but France is extract is huge in comparison. Is there a way to limit RAM usage for such large files during import ? [1] https://github.com/MaZderMind/__osm-history-renderer https://github.com/MaZderMind/osm-history-renderer 2012/7/16 Jochen Topf joc...@remote.org mailto:joc...@remote.org mailto:joc...@remote.org mailto:joc...@remote.org On Mon, Jul 16, 2012 at 01:28:50PM +0200, Aurélien FILEZ wrote: I'm searching history statistics of the french territory contributions in order to make some graphics about : - Number of named roads - Numbers of house numbers - Number of kilometers of different types of roads (highway, primary, secondary, tertiary, residential). I tried OSMIUM with the last avalable france.osh.pbf, but data are too many data and cause bad allocs errors on a 8 Gb memory server. What exactly did you do? If you use the RangeFromHistory handler to filter out one specific date in the history and then the Statistics handler, you should be able to generate some statistics, though not exactly the kind of statistics you are asking for. But you can use the Statistics handler as a model on how to write your own. You'd have to tun the thing several times, each time filtering out a different date with RangeFromHistory. Or you write your own handler that directyl works on history data, but thats a bit more tricky. In any case it should not need 8GB for all of this. You might have used the MmapAnon node storage instead of SparseTable handler. This explains the different options: http://wiki.openstreetmap.org/__wiki/Osmium/Storing_Node___Positions http://wiki.openstreetmap.org/wiki/Osmium/Storing_Node_Positions Jochen -- Jochen Topf joc...@remote.org mailto:joc...@remote.org mailto:joc...@remote.org mailto:joc...@remote.org http://www.remote.org/jochen/ +49-721-388298 tel:%2B49-721-388298 tel:%2B49-721-388298 _ dev mailing list dev@openstreetmap.org mailto:dev@openstreetmap.org mailto:dev
Re: [OSM-dev] OSM Wishlist
Hi Am 12.10.2012 16:14, schrieb Serge Wroclawski: I made a second issue at the same time regarding gathering histrorical data around way geometry. Right now, due to the way OSM stores its geometry data, the only way to know about a way's geometry history is to call the history call for the way, then make a history call for each and every node that was ever in the way. That's something I ussed about 4 years ago when I stared working with the history and someone pointed me to an important fact: The API is to support mappers. Using it for small tools/experiments is ok, as long as they dont't put noticable load on the api. For everything else, there are planet/history dumps. With today's tools it's quite simple to extract your area of interest from a history dump and work on that. In fact it's much much easier to use those dumps to fetch data then to crawl the api, once ypou get the algorithms in place. Talking with Matt about this a month ago on IRC, he suggested a new call could be made, one that would do the above work in a single call. This would allow the same work to be done with less server overhead, and greater resource management. Tom closed it, saying he didn't like expensive calls. I don't like expensive calls *on the main api* either. But why not build a history api? It could feed itsself from the history dumps and kept up to date from the replication diffs. It's all there, just assemble it. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] [GSoC] Improvements to Vespucci
Hi Am 28.07.2012 09:03, schrieb Jan Schejbal: I have rewritten the GPS tracking part to use a Service, allowing tracks to be reliably recorded even when the application is not active. Tracks are now only recorded by user request. The GPS location can be shown or hidden independently from track recording. Recorded tracks are automatically and continuously saved (an 8k buffer from a BufferedWriter and OS buffers apply). GPS location updates are only requested when needed. This means that exitting the app with the home button will disable location updates (if no track is being recorded) to avoid draining the battery. Just want to drop a line to say what a great work you are doing. I just tested a set of OSM-Editors for iOS and there's nothing that's even worth beeing called like that. Thank you for your great work! Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Augmented Diffs
Hi Wouldn't it be simpler to augment all changed elements with finished WKT-Descriptions? It would mean even less work for Clients and smaller Extracts, though it would not help clients needing topology (ie routing). Peter Am 27.08.2012 20:14, schrieb Roland Olbricht: Dear all, Overpass API now offers Augmented Minute Diffs. The idea goes back to a talk of Matt Amos at SOTM 2010. These diffs allow e.g. to keep a geographically limited database extract up to date. The details are documented in the wiki http://wiki.openstreetmap.org/wiki/Overpass_API/Augmented_Diffs However, these diffs address developer, not end users, because there is no software yet that can process these diffs. Therefore, I would be happy for comments and suggestions on this matter that arise when writing such a software. If there are good reasons to change the format, I'll try to do it only once, at the same time like the database reimport after the license change. Happy updating, Roland ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New proposed directory layout for planet.openstreetmap.org
Am 05.09.2012 18:13, schrieb Jochen Topf: On Wed, Sep 05, 2012 at 08:58:30AM -0700, Paul Norman wrote: /full-experimentalFull history dumps Can't we loose the experimental? +1 for that Also full isn't the best name. How about /with-history or /planet-with-history actually it's 1% planet and 99% history, so maybe /history-planet would be better ;) /users_agreed Users agreed /gps GPS dumps /cc-by-sa Old CC BY-SA content However the last few turn out, I vote for either using - or using _ everywhere. Consistency is important, so +1 on this, too. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Augmented Diffs
Am 10.09.2012 12:10, schrieb Robert Scott: On Monday 10 September 2012, Peter Körner wrote: Hi Wouldn't it be simpler to augment all changed elements with finished WKT-Descriptions? It would mean even less work for Clients and smaller Extracts, though it would not help clients needing topology (ie routing). This makes some massive assumptions about what the data is being used for and in the process makes many clever/unexpected things impossible. But it would also free you from the burdon of having n+1 implementations of the multipolygon code. I admit that it would be a specialized augmentation that probably should a) be called differently and b) be coded deployed independently. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] osm-tools (alpha) for windows [was: How to compile osm2pgsql without persistent node cache?]
Hi Am 02.09.2012 19:47, schrieb Kai Krueger: OK, so now osm2pgsql builds on Linux, FreeBSD, Mac OSX and Solaris. If only someone could figure out how to build osm2pgsql on Windows... I built osm2pgsql under cygwin as well as all the gdal and osmium tools. I planned to distribute them soon but I faced a SegFault when running tools that used geos or gdal outside of cygwin. Tools (like osmium_convert) are running fine. I also did not get osmjs to compile under cygwin. Because I don't know how to fix this, I uploaded the compiled files here: http://osm4windows.org/downloads/ Maybe some one has an Idea on how to fix that. I'd love to distribute 100% running binaries of those OpenSource-Tools. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] (Multi)Polygon handling
Hi Am 13.07.2012 14:41, schrieb nimix: [1] https://github.com/nimix/osm_conv_tests good start but I'd remove the .wkb files in favor of the wkt ones. You shouldn't have two versions of the expected output or you will get cases where the WKB and WKT differs. And it's unclear what the real expected result is. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Retina tiles - best way to support them?
On 17.07.2012 20:25, Robert Joop wrote: But I’ve copied your page, adjusted the style and tiles URIs and added a viewport. As expected, it doesn’t make any visible difference on an iPhone 3 (pixel ratio 1), but the difference is remarkable on e.g. an HTC Sensation (pixel ratio 1.5) or an iPhone 4 (pixel ratio 2). See https://mlist.timesink.de/osm-geofabrik-highres/ for some screen shots. That looks amazing! Can the index.html be reached somehow to see the result on my own devices? Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [osmosis-dev] Problem with PBF reader
Hi Am 29.06.2012 22:01, schrieb WanMil: I started an investigation to find the problem. It is currently unknown if the problem is located on osmupdate or on osmosis site or if that's a different understanding of the PBF format. Maybe this could be clarified by verifying the obf's with osmium. If osmium and osmupdate produces the same results, it's probably osmosis that get's it wrong. The current status is that a planet file updated with osmupdate can be read by osmsosis without any error message but osmosis reads different longitude values than osmconvert/osmupdate. It would be very helpful if you could supply a minimal testcase (ha hand full of nodes) that have the same problem. Peter ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [OSM-dev] Same node shows up multiple times in changeset
Am 13.06.2012 18:35, schrieb Eric Wolf: I am using a dated fork of the Rails Port, so I am hoping this has been fixed already. I am seeing occasional changesets with the same node showing up multiple times. The Api does nor prevent this. If an Editor chooses to upload a new Version of a Node without Changing anything, then it is like that. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] GSoC - Data Tile Service
Hi Am 10.06.2012 18:03, schrieb Michael Daines: I have been documenting the data tile format from the Kothic project (including the details of how map features are cropped, etc.) which is at present what the data tile format I am working on will be based on. I am also working on describing other schemes for cropping OSM data, and other JSON representations of OSM data. Is this documentation-draft published somewhere? Would really like to take a view into it. I will post this work (the documentation and the tile production script and web testbed) on the wiki and in a GitHub repository in the coming week. Ahh, forget my question from above ;) Finally, I have been working on a script for producing augmented diffs (as suggested a little while ago on the list by my mentor) by querying the Overpass API, meant to make keeping a replica of the planet up to date with minute diffs more tenable in the face of data processing speed and the possibility of missing data and broken geometry. An initial version is here: Isn't this a totaly unreleated project? Although its more interesting then a Tiled-Data-Service to me :) Despite that I'd strongly vote for NOT using Pverpass-API, because of multiple problems that could arise from this dependency: - Overpass API is a important tool for Individuums to get specific data they're interested in. It's no good idea to occupy that api with another service - It's no good idea to provide a service that depends on another service that is not maintained by you. - Your tool yould be a lot faster when you base it on a specialized database and not on a generic api I think it would be far better for an diff-augmentation-server to hold its own specialized database, optimized for the single job of augmenting diffs. (remember: preprocessing is everything in the osm universe) Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Map rendering issue
Hi Jason, is the tile-server public? Can you share an url with us? If not, maybe a screenshot? Did you do an import in the close past or are you running on minutely updates? Peter Am 07.06.2012 19:31, schrieb Jason Clark: I posted earlier about what I thought was a problem with Australia data, but it isn't It looks like all data below -35 lat is missing... Anyone know what might cause this? My tile server is built from http://switch2osm.org/serving-tiles/building-a-tile-server-from-packages/ and had no issues during install. The import of the planet was fine, no errors. Thanks! ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [osmosis-dev] Tagfilter plugin: filter by key-values read from a file instead of comma separated values
Hi Thank you for your contribution! Am 01.06.2012 15:05, schrieb Raluca Martinescu: For the*--node-key-value*and*--way-key-value*command line options I added also the possibility to read the key.value combinations from an external file, No bad idea. specifying*keyValueList=file=file_path*(if the keyValueList does not start with file=, the option works as before). I think a separate option would be a nicer way and would follow the conventions of the other calls more closely: keyValueListFile=file_path I've attached the code changes I've made (very small): It's unusual to attach files on mails to mailinglists, as with 2000 subscribers it produces 2000x the size of your attachment as traffic on the server and on each client. That's a problem when you're in a low-bandwith location somewhere on this planet without VDSL. The better approach would be to clone the repositroy on github https://github.com/openstreetmap/osmosis create a working branch in youer fork and send a pull request. This way it's easy to attach comments to the lines of your code and the merging for the maintainers is a one-click operation. Regards, Peter ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [OSM-dev] osm-history-splitter quirk
Hi using softcut gives better results in most cases which is why I made id default some months ago. The split-all-clipbounds script was hack which I don't use at all. It tries to optimize the splitting-process by creating country-extracts, then state extracts from them and so one. This fails if the inner polygons are not fully containes by the outer ones. Does the same problem occur when you cut your state-polygon straight from the source file? If yes, could you tell me which poly-file you are running in which source? Peter Am 26.05.2012 18:29, schrieb Martijn van Exel: Hi, I'm using the split-all-clipbounds script often to generate US state and county sized pbf files using osm-history-splitter. I used to use the hardcut method, but this generates false positives in the analysis I am running currently (detecting disconnected _link roads) so I switched to the softcut method. I did a file size check to see if all went well. It turns out four state PBF files were empty (or rather, 133 bytes). I checked the POLY files used for the cut but they seem fine (no self-intersection, closed polygons). I am running the four states again now so I can have a closer look at what's going on. Any ideas up front about where to look for the cause of this? ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [osmosis-dev] maybe error in osmosis!?
Hi Am 16.05.2012 19:22, schrieb Masi Master: An area is an 'outer'-element from a multipolygon, and the 'outer' OR the multipolgon is tagged as *=residential, *=riverbank or something other. If someone delete the multipolygon or relevant tags (and tag the area with the old multipolygon-tags, if it was untagged), then Mapnik render it as an untagget area. Maybe osmosis create a wrong diff-file, or the osmosis version, which Mapnik use on openstreetmap.org, is old. It really sounds like an osm2pgsql glich. Could you please verify that this problem still exists with the current osm2pgsql-version. Peter ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [OSM-dev] Non-ASCII characters in XML generated from PostGIS
Am 18.05.2012 09:48, schrieb Andy Allan: I don't understand why you're trying to force the UTF-8 characters into an ISO-8859-1 encoding - which has barely enough code points to cover Western European languages, never mind Greek, Russian or any other OSM data. Stick to UTF-8 encoding in your XML, and you should be fine. Provided, of course, you can persuade PHP that's it's an XML rather than an HTML document, to sort out the entity encoding as I explained above. Don't use htmlentities, better use htmlspecialchars and set the charset of your XML to UTF-8. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New version of MapOSMatic, a city map rendering service
Hi Am 18.04.2012 08:58, schrieb Thomas Petazzoni: * The availability of several rendering styles. For now, we provide the standard OpenStreetMap.org style, several styles provided by MapQuest, and a custom style more suitable for printing. In the future, we expect to extend those styles, or even to let users provide their own styles. Do not hesitate to contact us about custom styles; How about the fabules toner style of stamen? http://maps.stamen.com/toner/#12/37.7706/-122.3782 Thanks a lot for the great service! Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Coastline by Name
Hi I was looking for Coastline-Polygons by Country-Name, that would be the coastline intersected with the administrative border and saved under that name. Before I start to build that on myself I'd like to ask if there's already Code to do that. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] different rendering with mapnik based on area?
Am 31.03.2012 21:12, schrieb Phil! Gold: * Sven Geggusli...@fuchsschwanzdomain.de [2012-03-31 16:06 +]: is it possible to have different rendering rules based on the area where rendring takes place using an ordinary mapnik rendering stack (osm2pgsql+mod_tile+tirex)? Supposedly yes. MapQuest uses has three different stylesheets that they use for their tiles and the appropriate one is selected based on where the rendered data is. It won't work without adapting the source. You can choose which source you want to adapt: - The Style: use different layers for the areas whith 'SELECT WHERE ST_Inside(way, ...)'-style queries - simple but slow - mapnik: somehow.. - mod_tile: switch between different maps depending on the requested z/x/y-tile - The Client (OpenLayers/Leaflet): enable/disable base-layers depending on the visible area Choose whichever way fits your needs best - and tell us about. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] CT-incompatible planet file?
Am 28.03.2012 11:28, schrieb David Groom: I'm not sure that is a help I'm sorry, you're right. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile Developer list?
Am 28.03.2012 19:58, schrieb Lynn W. Deffenbaugh (Mr): However, if you use the -a option (along with -z/Z/x/X/y/Y), every tile in the specified range is requested with NO date check and the -f option is completely unused. Sounds like a bug for me, too. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] CT-incompatible planet file?
Am 27.03.2012 22:32, schrieb David Groom: Is there anywhere a CT-incompatible planet file? i.e a file containing all the ways and nodes which are likely to be dropped during the rebuild. Take a look at http://cleanmap.poole.ch/ Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Directionality of lanes
Am 27.03.2012 12:14, schrieb Dietrich Opitz: Hello Thank you for the information. I didn't wanted to add the obivous to the database. I just missed the default settings per country and was wondering how to get this type of information from the database. I don't think you can get it out of the osm-database as it isn't something that belongs to osm. Better build a custom database (or better: list) of it or query wikipedia or some other ressource. Not every information has to be stored in the osm database. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Fwd: Postcode to LonLat tool
Am 25.03.2012 01:31, schrieb Richard Ive: Goodness, I got so excited I didn't mention how to use it! http://postcode.nodester.com/ is the test page. (There's an input box bottom right). It's a pitiy that it's UK-Only. A majority of the OSM-Community is outside the UK. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Overpass-Query: Complete Relations in BBox
Hi I tried to fiddle an overpass-query together that would give me: - all relations tagged with type=boundary or type=multipolygon - their way-members - nodes used by thode way-members but I was unable to get this result. Regarding the docs: they consist of a large list of examples with very minimal explanations that leave open a lot of questions: - which query-language is suited for what? - what's this .-x or .-_ about? - are two queries in a union get ANDed or ORed? - will a print-command print all results of all unions or only the last one? I'd love to see a walktrgough or a tutorial that explains the building blocks and how they interact in a chronological order without adding stuff that isn't explained. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Test API minutely diffs
Am 13.03.2012 08:47, schrieb Paul Norman: If not, does anyone have suggestions for generating minutely diffs? You should be able to generate .osm-Fiels with JOSM locally, do some changes and save those changes as .osc -- also locally. No need to involve the API. If you'd like to go for live-testing, best would be to do some real mapping (go out and find the opening-hours of some shops or add that new footway) and check with those changes. You can save an .osm-file of the state itthe database was before your edits, upload the changes, identify the diff-sequence they're in and test your app against those. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] How to get all members of relations into GIS?
Am 07.03.2012 17:27, schrieb Ramas: Hi, i have made some processing scripts for public transport relations. Daily dumps you can get here - http://osm.ramuno.lt/transport/ And working example - http://openmap.lt/#l=60.16671,24.93903,15,MT Would you mind sharing the sourcode of those processing scripts? Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] How to apply OSM Official Application for Android GSOC project
Am 11.03.2012 12:18, schrieb Gihan De Silva: HI, I'm Gihan De Silva and I'm interested in OSM Official Application for Android GSOC project. And is this project supposed to be done with HTML5+JavaScript or Java? If anyone can show me a good tutorial on OSM API, it would be highly appreciated. The won't be *the* osm application. Depending on what you want to do you'll need several different applications. - One for mapping POIs - One for mapping all OSM objects (ways, areas, route-relations) - One for watching pixel-tile-maps, possibly with server-side routing - One for watching Vector-Maps and routing locally - One for recording GPX-Traces and upload them directly to the API The technology to be used depends highly on what you're familar with. If you know on how to develop rich UIs for mobile devices with HTML5, do that. If you're better with Java, go that route. But which ever way you choose, first define what exactly what you want to do, then choose the tools you're planning to use. Regarding the OSM Api you need to first check, what you want to do, because there are several APIs for diffentent tasks to perform. - The Main-API for Editors and GPX-Uploaders - The XAPI and the Overpass-API for specific Data-Consumers - The Tiles-Server for Pixel-Map-Viewer But in most cases you'll need to define set up a custom API dedicated at the use case of your app. This offloads the main apis which are dedicated to mappers and allows you to optimize the communication between your app an the API by shifting processing requirements away from the mobile device. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSMCoastline
Am 08.03.2012 00:00, schrieb Frederik Ramm: Hi, On 03/07/2012 04:05 PM, Tom MacWright wrote: Whoah! Nice work. All of these osmium-related projects are really exciting, can't wait to have more performant and efficient parts of the OSM stack. Plus, they're well-written and hosted on GitHub :) Github is a third-party, proprietary, commercial software hosting platform which as part of its marketing strategy makes certain services available free of charge for certain users - currently. That's ok, and every developer is free to chose where they want to host their project. But I wouldn't go so far as to applaud someone for that choice. Well positive side of git is, that once github goes crazy every user of the cloned repo can do an add remote push and switch to another hosting location without loosing any history. This would not be possible with eg. svn on sourceforge. On the other hand github gives with its forks and pull-requests neat tools to make software better in a community. Once there's OpenGithub or sth. like that it would not be problematic to switch over - or use both systems in parallel. So this applause is for using git: *applause* Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Possible GSoC project: tag/area monitoring service
Am 07.03.2012 05:11, schrieb Michael Daines: First, a longstanding wishlist item for OSM has been data tiles, that is the API data, split into preset sized areas (eg z14), which a client could call. This may not seem reelvant to your project but you'll see why it is soon. This was actually part of my original motivation for proposing this project -- in my 2010 GSoC project, I used bbox queries to load data in tile-like sections, but as I mentioned this turned out to be very slow. Data tiles seem like they could speed things up for that sort of use. Ideally, the work involved in accessing a data tile would be comparable to accessing an image tile. Also, it seems easier to cache data addressed by tile than it is to cache the results of arbitrary bbox queries. I'd also be interested in working on data tiles -- is that in itself a reasonable project idea? My hope is that if either of these ideas are things people have been wanting for a while, they'll want to use them, and that if a project has people using it, it would be more likely to be around after the summer. A Service that is able to provide 1. fast and scalable 2. tiled access to 3. updated data 4. around the world with a constant tile size (eg z12 or z14) 5. together with formulars to calculate the tile coordinate from lat/lon and 6. complete documentation would be project of reasonable complexity and usefulness. The most complex part here is 3. If you have further questions on possible implementations or use-cases don't hesitate to contact me directly: pe...@mazdermind.de One thing I was wondering about -- how do you choose a tile size to minimize both the number of accesses (larger tiles) and the byte size of tiles (smaller tiles)? Some areas have a much higher density of data than others. Perhaps some kind of quadtree-type approach could be used, where tiles are split if they have high density? This could be a Project for the next GSoC, but calculating the tile sizes is in itsself so complex, that it would fill a complete GSoC-Project, leaving no room for the project outlined above. But a Tiling-Algorithm without a service implementing it would not be of great use for the community, would it? Despite that there are already tools that are dedicated to this kind of computation: http://www.mkgmap.org.uk/page/tile-splitter The ideas you suggest for streaming-type updates on data tiles are very interesting. If you were writing an editor, you could be more certain that you were displaying the most recent data without having to reload all of it. But you would not want the editor to display those changes without user interaction. Imagine you are drawing a road and around your cursor everything changes shapes the whole time. You would not call this a good user experience, would you? Also a streaming editor is nothing the community is requesting, editing works good (enough) the way it is. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSMCoastline
Am 07.03.2012 15:57, schrieb Jochen Topf: Hi! I have been working on writing a substitution for the aging coastcheck program. It is not finished yet, but maybe somebody wants to play around with it. Whoo would have needed that just 2 Weeks ago. What do you think about adding osmium as submodule, like I did it with the history splitter/importer (https://github.com/MaZderMind/osm-history-splitter/). The benefit is, that with git submodule init git submodule update you'll get the last version of osmium the coastcheck was tested with. After that, osmium is in the right location and there's no need to edit the Makefile. See my renderer-Tutorial for how it works: https://github.com/MaZderMind/osm-history-renderer/blob/master/TUTORIAL.md The drawback is, that when a general update to osmium happens, you have to update the reference in the coastcheck repo. Thank you very much! Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] replicate-sequences tool bust?
Hi I'm currently switching to another mail-provider, sorry for the bounces. In the toolserver the server running my cronjob went down during the last weeks and has not been fixed. I just ran the job and the database is up now. I activated the job on another server in the cluster, so that the database should stay updated now. Peter Am 03.03.2012 07:28, schrieb Stephan Knauss: On 03.03.2012 03:32, Lynn W. Deffenbaugh (Mr) wrote: I tried the contact the tool owner link (maz...@toolserver.org), but my e-mail bounced. Is this the place to report this issue or is there a better place? I have an alternative Mail contact of Peter and forwarded the problem description. I also think he is active on this list. Guess he will answer soon... Stephan ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] generate image failed
Am 02.02.2012 10:56, schrieb Anwar Azulfa: It's not working on me, what repository which i must add to install mapnik2 or libmapnik2 ? Because this question was not answered in that thread I'd like to point to my tutorial which includes installing mapnik2 from the testing-repo under ubuntu and debian: https://github.com/MaZderMind/osm-history-renderer/blob/master/TUTORIAL.md Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] producing historical snapshots for an area
Am 04.02.2012 17:35, schrieb Martijn van Exel: Hi, I want to create historical snapshots of small (county-sized) areas from the full history planet for analysis purposes. Say, one file for every three months since 2007. My current approach is to first create the full planet snapshots from the full history planet using osmium, Which tool are you using for this? and then creating the smaller files from that using osmhistorysplitter (I'm thinking of using the hierarchical splitting feature to first create state-sized files and county-sized files from that). But: how hard would it be to adapt osmhistorysplitter to do this all in one go? Or maybe there already is a smarter way to do this that I don't know about. My C++ skills are all but nonexistant, but I'm not afraid of python and only a little of perl. It's of course possible to do that in C++ but I tend to use a higher level programming language for those not-so-critical tasks. You may take a look at split-all-clipbounds.py, which is a script that takes all clipbounds from the clipbounds folder ans tries to sort them in order to call the splitter and generate extracts in a hierarchical manner. [1] https://github.com/MaZderMind/osm-history-splitter/blob/master/split-all-clipbounds.py Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Nested relations and generic implementation of geometry constructs - Was :osm2pgsql patch for nested relations
Am 01.03.2012 15:26, schrieb sly (sylvain letuffe): By meannings I mean that some relations are made to build a bigger geometry (type=multipolygon is one), some to records facts unreleated to geometry (i.e type=restriction ) The question is how do we improve that, while keeping free tagging system ? By defining algorithms to work with relations in one place and mapping relation-types to that algorithms in another place. For example: type=multipolygon -- build-polygon type=route-- build-linestring type=waterway -- build-polygon type=restriction -- discard type=windmils -- collect The goal should be reducing the number of algorithms and re-using them as much as possible. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] release of full-history extracts
Hi I generated a new charge of history extracts, based on the 120213 full history dump. They have been created from the latest full-experimental-dump [1] using my history splitter [2], based on Jochen Topfs really great osmium framework [3]. They contain multiple versions of an object. If you just want the map-data as it is today, use the Geofabrik-Extracts [4]. The extracts can be downloaded from a server of my employer: http://osm.personalwerk.de/history-extracts/ Their size ranges from very small (a village) via medium (Berlin) to large (Germany), touching various countries. They only cover a very, very small part of the world and are currently targeted at application developers that are looking for data to test their history analysis apps. Most extracts are delivered as .osh.pbf files, readable with all history-enabled pbf parsers (eg. osmium). Some extracts are also available in the .osh.bz2 format (xml-basesd). Some common programs like JOSM can open them them when you rename them to .osm, but the produced output is not very useful in most cases. In contrast to the last set of extracts, the new ones are now cuttet using the softcut algorithm [6] using simple bounding-boxes [5]. Dumps created using that algorithm have the following characteristics: - ways are complete (as they are in the api-database) - ways are reference-complete (all referenced points are included) - relations are complete (as they are in the api-database) - relations are NOT reference-complete (relations may reference ways or nodes that are not in the extract) - relations referring to relations that come later in the file are included - all versions of an object of which one version touched the bbox are included Those files can eg. be used to create nice animations of your home-town like this: http://mazdermind.github.com/osm-history-renderer/karlsruhe.html Take a look at the Tutorial to see how it goes: https://github.com/MaZderMind/osm-history-renderer/blob/master/TUTORIAL.md As always, I first created a .pbf version of the full-planet file, which can be downloaded, too: http://osm.personalwerk.de/full-experimental-pbf/full-planet-120213-1231.osh.pbf Peter [1] http://planet.osm.org/full-experimental/full-planet-120213-1231.osm.bz2 [2] https://github.com/MaZderMind/osm-history-splitter [3] https://github.com/joto/osmium/ [4] http://download.geofabrik.de/ [5] http://osm.personalwerk.de/history-extracts/split.config [6] https://github.com/MaZderMind/osm-history-splitter/blob/master/softcut.hpp ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Using custom renderer with mod_tile
Hi take a look at tirex, it's mod_tile's successor and it has a general interface to renderers (via UDP or Domain Sockets, afair). Peter Am 29.02.2012 20:14, schrieb Skye Book: Hi all, We're looking to setup mod_tile to serve arbitrary tiles (i.e: not OpenStreetMap). Is there a way to change out the use of Mapnik in favor of running another application/script/whatever to render the tile to disk? Thanks, -Skye ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] What can we do to get a tile.openstreetmap.org contributable CDN within a month?
Am 30.12.2011 19:17, schrieb Stefan de Konink: I do not have yet investigated what the options are of CoralCDN, but the main problem is to get the *alternative used*. I guess switching to a system that is partly in our own hands results in a higher quality, thus we must have ability to invalidate data quickly. Tirex is able to be used as cache against a wms. It should be able to work as a cache against another tile server, supports expiring and rerendering rules and stores content as metatiles. I guess a cache could be implemented as tirex renderer. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [osmosis-dev] [OSM-talk] Osmosis replication fails
Am 19.12.2011 13:19, schrieb Brett Henderson: You need to pick a sequence number by examining the timestamp property in the state files. Pick a timestamp in the hour replication state files that is earlier than the last timestamp received from the minute replication. It's a bit tedious. I seem to remember somebody creating a web-based too to assist with this but I don't have a link handy. http://toolserver.org/~mazder/replicate-sequences/ I just updated it to include the hourly and daily files. Peter ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [OSM-dev] IP limit issue when doing mapping parties on a network with NAT
Am 16.12.2011 09:39, schrieb Andreas Hammershøj: Well, you could download the area of interest via the homepage export function as .osm file and place it on a network share / usb stick. The Editors of the mapping party would open that file, do their changes and upload from there. The solution with a local copy sounds interesting, but because of the time available and the average computer skills of the course participants, we teach Potlatch2. Is it possible to do this with Potlatch? I don't think so, but maybe then Potlatch is the wrong tool for your task. If you get to that conclusion, there are two possibilities: modify the tool or choose one that suits your needs better. Maybe you should give JOSM a try ;) Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] speeding up loading an OSM dump into PostGIS?
Am 16.12.2011 11:09, schrieb Hartmut Holzgraefe: So even a public PG replication master would only make sense for those who run exactly the same architecture, or multiple masters for different architectures would be needed ... :/ At one of the Hack-Weekends someone played around with distributing the SQL-Commands issued by osm2pgsql via XMPP. Such a server could distribute full-import-sql-dumps in a weekly shedule as files and the minutely updates as XMPP messages. Not sure if it would sove any real problems. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] IP limit issue when doing mapping parties on a network with NAT
Am 16.12.2011 11:21, schrieb Andreas Hammershøj: Hi Peter, we're aware that using JOSM would counter this problem, but we have deemed JOSM too complicated. Many of our participants are recently retired, with limited computer skills. They have enough of a challenge just learning to use P2. Also, we normally have only around 7 hours at our disposal to get people editing safely, without being menaces to the map;) I think Mercator is also able to do offline editing: http://wiki.openstreetmap.org/wiki/Mercator I don't want to vote for the one or the other, just point out that if a tool doesn't solve your problem, your possibly better off with trying another tool. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] IP limit issue when doing mapping parties on a network with NAT
Am 16.12.2011 11:33, schrieb Peter Körner: Am 16.12.2011 11:21, schrieb Andreas Hammershøj: Hi Peter, we're aware that using JOSM would counter this problem, but we have deemed JOSM too complicated. Many of our participants are recently retired, with limited computer skills. They have enough of a challenge just learning to use P2. Also, we normally have only around 7 hours at our disposal to get people editing safely, without being menaces to the map;) Correct link: http://wiki.openstreetmap.org/wiki/Merkaartor I don't want to vote for the one or the other, just point out that if a tool doesn't solve your problem, your possibly better off with trying another tool. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] speeding up loading an OSM dump into PostGIS?
Am 16.12.2011 12:47, schrieb Hartmut Holzgraefe: On 16.12.2011 11:21, Peter Körner wrote: At one of the Hack-Weekends someone played around with distributing the SQL-Commands issued by osm2pgsql via XMPP. with the SQL command execution, especially the index creation, being the most expensive part of an osm2pgsql run this would onyl save about 1/4 to 1/3 of the total planet import execution time i'm afraid ... It would help with keeping updated: you would not need the --slim tables anymore (on each server - only on the master) Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Integrate Mapnik 2.0.0 with svn.openstreetmap.org/applications/rendering/mapnik/
Am 13.12.2011 14:49, schrieb Oon Arfiandwi: After struggling with the new syntax of Mapnik2: TextSymbolizer[label]/TextSymbolizer A lots of change must be applied on the osm.xml and *.xml.inc With mapnik2 there comes a utility upgrade_map_xml.py which ports your maonik0.x xml files to the new mapnik2 syntax. apply that after the generate_xml.py step. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Writing PBF
Am 13.12.2011 15:04, schrieb Jochen Topf: Has anybody any data on how much smaller the files get with sorted vs. unsorted string table? Sorting the string table means there is a lot more work and a lot more main memory needed for the PBF writer. I encountered shrinkage of around 1-2% of the overall block size while comparing the intermediate string ids with the sorted ones. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] IP limit issue when doing mapping parties on a network with NAT
Am 12.12.2011 20:49, schrieb Serge Wroclawski: HOT gets around issues by downloading an area ahead of time and loading it into the editor. Well, you could download the area of interest via the homepage export function as .osm file and place it on a network share / usb stick. The Editors of the mapping party would open that file, do their changes and upload from there. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] White squares in sea inEurope
Am 08.12.2011 17:49, schrieb yvecai: On 07. 12. 11 20:26, yvecai wrote: I've already seen here and there the effect of a broken coastline, but we have something different in that server: http://osmati.net/osmand/map?zoom=3lat=53.98194lon=23.90625layers=B I hope somebody has already seen something like this. Have you tried disabling the shapefiles and re-rendering some tiles? This could help to locate the problem in the db or in the shapefiles. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] osm2pgsql not appending changeset?
Am 06.12.2011 07:55, schrieb Ákos Maróy: $ osm2pgsql -d osm -S /usr/share/osm2pgsql/default.style --append -G -v -m -K changesets-16.osm.bz2 changesets*.osm.bz2 only contains changeset meta information, no content. You'll need one of the replicate-series http://planet.osm.org/day-replicate/ http://planet.osm.org/hour-replicate/ http://planet.osm.org/minute-replicate/ which are usually downloaded and simplified using osmosis and then fed to osm2pgsql. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] osm2pgsql not appending changeset?
Am 06.12.2011 11:30, schrieb Ákos Maróy: on the OSM wiki, I found a number of update options, and I have to say I'm a bit confused. which is the best option if I want to get all the updates for the whole planet file, say, each week? Call osmosis once a week to download the last 7 day-replicate files and push them through osm2pgsql. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] idea: before-after map
Am 05.12.2011 18:08, schrieb Martijn van Exel: Hi, Now that we have the history files and a history renderer in the making [1] it would be possible to create a Before / After service: user would select one or a few concurrent changesets comprising an edit session, and the service would churn out a pretty HTML page with a static map of the area just prior to the changesets affecting the area and just after the last ones were closed. I was just wondering how difficult / feasible this would be to implement. Do we even need Peter's renderer for it? You could use the database and the valid_from / valid_to processing used in the views to get the before/after state and do whatever you want (ie compare version minor version to find affected items). Just use this clause in your where-statement to get all items at this point in time: '2010-01-01' BETWEEN valid_from AND COALESCE(valid_to, '-12-31') Take a look at render.py to see how it's used. [1] https://github.com/MaZderMind/osm-history-renderer Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] idea: before-after map
Hi Am 05.12.2011 19:11, schrieb Martijn van Exel: For it to work (for current changesets) though, we would need a continuously updated full history database.. Doing the import is one thing, doing the update is a whle different thing. Both being mixed up into the same application results in a lot of not-so-nice and not-so-well-optimized code (see osm2pgsql). For updates you'll need a different cache strategy, a different level of access to the database and different algorithms. For me, cleaning up the importer, adding multipolygon and route-support as well as optimizing the importer to be able to handle full-planet-full-history imports is higher on the priority list then an updater, and you'll, have to live with that ;) I'd love to see those tools sketched out on the data we have already. Developing a nice before/after/diff app is a really complex task and maybe if the app is finished, the updater is, too. Don't wait for other tasks beeing completed if you could work with the things that are in place already. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Memory-Leak in osmium
Hi I found a memory-leak in osmium-master. I compiled the osmium_progress example and fed germany.osh.pbf http://ftp5.gwdg.de/pub/misc/openstreetmap/osm-full-history-extracts/110919/pbf/europe/germany.osh.pbf into it. During the nodes processing the program takes 70 MB of RAM. When it starts to process Ways, the required memory raises constantly up to 300 MB. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Memory-Leak in osmium
Hi Am 21.11.2011 12:07, schrieb Peter Körner: I found a memory-leak in osmium-master. Digging into it, it seems that the pbf reader is leaking memory. I attached valgrind-massif output for running osmium_progess against mainz.osh.pbf and mainz.osh. The pbf-run took nearly 44x the memory of the xml-run: http://daten.marvinstudio.de/osmium_progress-on-pbf.txt http://daten.marvinstudio.de/osmium_progress-on-xml.txt Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Memory-Leak in osmium
Am 21.11.2011 15:36, schrieb Jochen Topf: I don't think that there is a memory leak. Valgrind tells me that all is ok. A program can leak memory without valgrind showing it. I already guessed that this data is not completely useless. The behaviour you see comes from the PBF reader and especially the Google Protobuf stuff. Protobuf does not free memory but keeps it around until the next time it is needed. So every time you hit a PBF block with more data in it than all the other blocks before (for instance more nodes per way), PBF will allocated more memory and not de-allocate it. So for a while you'll see the process slowly using more and more memory until it hits the peak. 260 MB for a city-sized file (mainz) and 500 MB for a country (germany) is a lot. That explains, why my memory calculations always were wrong. Thanks for that explanation. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
