Re: [OSM-dev] Processing dual carriageway highways into one linestring?`
https://www.openstreetmap.org/user/Tomas%20Straupis/diary/390267 But it is one of cartographic generalisation operators requiring preprocessing and quite some resources. OSM carto by definition is not doing cartographic generalisation (except very minor ones). When done on global scale you would probably have to think about segmentation as well. ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] dev Digest, Vol 158, Issue 3
Hello 2018-05-16 10:01 GMT+03:00 SandorS wrote: > The statement was not quite wise and in some aspects it is wrong. Blindly > trusting open source solutions is not the best thing for a newcomer > especially not for a developer. By experience I know that sometimes a hint, > a simple warning may help a developer to change his way of thinking. > Besides, reading someone’s complex and complicated source (like the > generalisation related source) is not just a simple exercise. If you ever > wrote a complex basic sw and tried to read it after several months of brake > then you understand what is my point. I think there was some misunderstanding here. I only stated that using commercial (or to be more precise - closed) software in my opinion is not the way to go for an open project like OSM. Results of generalisation using commercial software can be used to refine the approach (by looking, comparing), but final software will probably be open (and open gis software is gaining fast anyway). For example Netherlands example of using commercial software shows that generalisation is successfully(?) done on PARTS of data, so now we know that we do not have to think of ways to generalise whole world at once. Open data entered by volunteers does have a higher probability of errors, but generalisation could be just one of many ways of detecting such errors (and so fixing them). Regarding other points. There are a lot of different operations done on a lot of different types of objects in different sequences. But in my opinion it is not "all or nothing". You can start with some generalisation and progress with time. For example doing simple polygon aggregation/amalgamation is doable now with good results but should of course be improved with proper polygon conflict resolution methods (already described in numerous scientific papers, f.e. "Detecting and resolving size and proximity conflicts in the generalization of polygonal maps", Bader and Weibel 1997). Transport network collapsing (multi-lines to one line) can already be done with standard PostGIS functions (buffer, approximatemedialaxis). Building simplification is already in testing stage. GRASS provides functionality for transport network displacement and simplification/selection. And these already give good and noticeable results. Of course a lot of other things must be done, but we have a greatest luxury - time - there are no deadlines. This is why some operations could be implemented with even better results than closed implementations. -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Generalisation
2018-05-03 1:05 GMT+03:00 Marco Boeringa wrote: > You do realize the 1-2 years is well after the 2013 date that the Dutch > Kadastre started to publish their work? Lithuania was given as contra to "the only". Savino (Italy) was given as contra to "the first" (as his work was published in 2011). (It is also a "must read" for anyone interested in generalisation) Anyways, there is no point of talking about who first, last, only etc. All approaches using closed commercial software are pointless for OSM - it cannot be reused. Everything can be done with open source so that all code/algorithms are open and clear and there is no need to pay piles of money for nothing. -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Generalisation
Hello 2018-05-02 19:33 GMT+03:00 Marco Boeringa wrote: > The generalization I wrote about was just a crude basic generalization of > vector (building) data of OSM using a default tool of ESRI's ArcGIS. The > specific tool used (Simplify Polygon), has more advanced settings than > standard Douglas Peucker, but by itself does nothing really special other > than weeding out vertices / nodes. I just attempted to use it with different > tolerances to see what the results would be, and concluded the resulting > defects in building topology, were not worth the reduction in file size. You've probably used this: http://desktop.arcgis.com/en/arcmap/10.3/tools/cartography-toolbox/simplify-polygon.htm And I'm talking about this: http://desktop.arcgis.com/en/arcmap/10.3/tools/coverage-toolbox/simplify-building.htm But both of these are closed and tied to their proprietary architecture. And there is even less information on implementation than in first(?) scientific paper about actual building generalisation algorithms: Sester M. ( 2000 ) Generalization Based on Least Squares Adjustment In: ISPRS (ed.) > However, as to interesting stuff to read: our national Kadaster of the > Netherlands, is actually the very first and only national mapping > organization world wide, that has successfully managed to implement a fully > automated generalization work flow for generating 1:50k maps from 1:10k > maps, including landuses, waterways, highways, but also generalizing > build-up areas and buildings. They used a range of cartographic > generalization tools from ArcGIS (that I didn't use...). Congratulations to national Kadaster, but I'm not sure you're correct about "first and only". Our local (Lithuanian) land agency (or to be more specific gis-centras) has completed automated generalisation 1-2 years ago (using esri tools as well). As far as I know fully automated and done in ~day. Most GIS people use a work by Sandro Savino "A solution to the problem of the generalization of the Italian geographical databases from large to medium scale: approach definition, process design and operators implementation". Author claims to have completed automated generalisation for Italy and it dates to 2011. This work is very interesting because instead of referring to closed commercial tools it has a very detailed description of how to actually do this and that. Also Swiss Topo is known to be doing a very high quality generalisation for years(?). But thank you for your links, it is interesting to learn how different countries handle generalisation. -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Generalisation
SwissTopo maps are some of the best examples of generalisation (as well as other cartography principles/techniques) https://map.geo.admin.ch/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Generalisation
2018-04-16 19:34 GMT+03:00 Marco Boeringa wrote: > No, buildings are not the most interesting. I once generalized all buildings > in Denmark. It only reduced the storage by maybe 5%, at the high cost of > heavily distorting a large number of them. Most buildings in OSM are in fact > already in their most generalized state: just 4 nodes. Unless you think > triangles is a suitable representation ;-) Interesting, what algorithm did you use? I'm playing around in Vilnius which has urban houses, big block houses, industrial zones and old town with lots of connected buildings of very irregular shapes. In Vilnius there are 54267 buildings tagged with 366979 vertexes. Clustering them with distance of 5m gets 45810 objects (of course with the same number of vertexes). Removing buildings with area < 100 and having neighbours in < 500 meters I'm left with 28974 buildings with 299224 vertexes. Simplification (amalgamating buildings in the cluster and trying to remove edges < 20m) reduces the number of vertexes to 117108. So this is much more than 5%. There are still a lot of problems (no triangles:), but I do not expect number of vertexes to rise considerably. Even "dumb" generalisation (st_buffer+- with join=mitter) reduces vertex count by ~25%. Reducing storage/tile size is not the only/main purpose of generalisation. > Besides, buildings are only shown at high zoom, > while generalization is most needed and beneficial at > low zoom. Lastly, most vector generalization algorithms are primarily > designed and effective for rather smooth and node rich data, like a > stream-digitized feature, neither of which relates to square buildings. > Hence i consider generalizing buildings largely senseless. I suspect you're talking about st_simplify(preservetopology) use in vector tile generators. Which as mentioned earlier is only technically a "generalisation". http://www.gitta.info/Generalisati/en/html/unit_GenProcedure.html -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Generalisation
2018-04-16 11:34 GMT+03:00 Martin Koppenhoefer wrote: > There are some precomputed / extracted data files though, some of which > contain generalized (simplified) data. These are all "external" sources: > <...> Ok, so this is natural polygon generalisation. Looking at https://github.com/imagico/coastline_gen the method used is to rasterise, process and then vectorise back. I wonder if that is better/faster than full vector way: st_clusterwithin, st_union, st_buffer(positiveN), st_buffer(negativeN+M), st_buffer(positiveM) with a seasoning of st_simplifypresevetopology according to taste. > Another aspect is filtering: osm-carto removes features when they would be > very small (pixels at a given zoom level) and lead to "noise". Filtering (selection) is technically also a generalisation. But you need to group and probably amalgamate them before deciding that it is "too small". For example if we have a lot of small patches of forest close together (say 1000 patches of 10x10m with distance between patches of 1m) you would want to amalgamate them to one large forest, not to get rid of them all. Ways and especially buildings are the most interesting (difficult) part :-) P.S. GRASS claims to be doing displacement and way selection (https://grasswiki.osgeo.org/wiki/V.generalize_tutorial) -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Generalisation
Hello 2018-04-15 22:54 GMT+03:00 Frederik Ramm wrote: > The most likely location for this to be discussed is probably within the > openstreetmap-carto developer community as they would benefit most from > such approaches. I don't follow their work closely though so couldn't > say if the issues have been discussed in the past. Good point. I'll watch openstreetmap-carto. >> Also, maybe there are ideas to translate the thesis mentioned above >> to English? > I'm not aware of any plans, but I do know that Arne has quoted similar > work done by others, and there were many English-language works among > that, so perhaps if you skim through his literature list at the end of > the PDF you'll find interesting articles. I'm already buried in pdf's from google scholar :-) You start with one paper, then open references, then references in references and here you are the "wikipedia effect". The point is that Arne's work looks (from what I did understand) at generalising openstreetmap data, while most papers look at general data. This is important because OSM tagging does help a lot. Some generalisation can be done much easier with OSM data because of additional tags and you can add additional tags describing some real properties (not designed for the sole purpose of generalisation) of an objects which could help generalisation. For example with road generalisation, a lot is written on intersection generalisation. Having *_link ways or ref tags in OSM helps this effort a lot. Railway network also has different subtags like main/siding/spur etc. which also helps a lot. So while generalisation is generalisation anywhere, but generalisation of OSM data does have some important and useful differences/advantages. Another interesting thing is recalculating/updating generalised data. "OSM way" is to update very often. So it is important to be able to recalculate generalised data only on impacted part and it is not as trivial as calculating "dirty tiles". I do not know how much of that is covered in Arne's thesis. Will try to read/translate German version if there are no plans for translation. Thank you! -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Generalisation
Hello In last weeklyOSM (http://www.weeklyosm.eu/archives/10227) there was a very interesting point: "Arne Johannessen has published his diploma thesis (PDF) with the title "Algorithms for automated generalization by combining polylines in OpenStreetMap for specific special cases" and he also released the corresponding Java source code." As OSM is mature enough to start generalisation (more than "selection" operator), maybe there is some place where such topics (in OSM context) are discussed in English? Also, maybe there are ideas to translate the thesis mentioned above to English? -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] History feed stopped working
Hello For two days history feed is throwing 500 error: http://www.openstreetmap.org/history/feed/?bbox=20.9,53.95,26.83,56.45 Is this intended? Maybe url has changed? -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] PostGIS query "Crossing ways"
2016-09-11 18:20 GMT+03:00 Mike N : > Given a PostGIS database populated from OSM data by osm2pgsql, and 2 sets of > lines (such as the selection of all footways and the selection of all > roads) what function or series of functions will result in a list of > locations where footways cross roads without any OSM connecting node? (the > equivalent of JOSM's "Crossing Ways" warning) The closest I could get to this was by splitting way (planet_osm_lines) geometry into their segments. I was creating a new table say „segments“ with identical attributes as planet_osm_lines in which the same osm_id would be repeated as many times as many segments that way has. (You could also do it without actually creating a new table and letting postgresql process everything on the fly). If you have a way with three nodes, you would get two new lines (two line geometries, two records in the new table). Then you can use PostGIS functions to find intersecting ways. But you will not find intersection problems where two ways cross each other on the same x/y, but they have two different points in OSM (two different points with the same position one for way1, another for way2). -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Reverting changesets
2016-01-25 19:58 GMT+02:00 Andy Townsend:
> If it helps, I saw the same problem yesterday ("does not apply any
> changes"). What fixed it for me was to update JOSM to the latest stable
> version (the monthly plugin check had occurred earlier that morning, but I
> was running the previous stable version of JOSM). After doing that I was
> able to do a normal "full changeset revert" without problems.
I have a wget of josm-tested.jar in my JOSM launch script, so I am
always using the latest version.
But I managed to revert changes using perl scripts. I just had to
create a changeset separately and then provide its number to revert.pl
- then it worked ok.
Thank you for help.
--
Tomas
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Reverting changesets
Hello Three changesets were found to be accidentally deleting quite large amounts of correct data. I have informed the user and asked him to wait until revert is completed. Changesets in question are: 36781121, 36781218 and 36781956. I'm starting with the last one and intend to revert all three. But there I have a problem, JOSM reverter plugin does not work (it fetches info but does not apply any changes and does not complain about anything) and there was info on weeklyosm that reverter plugin is broken at this time. I then tried reverter scripts (as per http://wiki.openstreetmap.org/wiki/Revert_scripts). But running: ./revert.pl 36781956 fetches objects, identifies correct delete/modify operations to be done but then fails with: PUT changeset//close Use of uninitialized value $body in concatenation (.) or string at OsmApi.pm line 227. I'm still running in dry mode. I wonder if I could proceed anyway or should I ask somebody else (DWG?) for help? Thank you P.S. I'm running scripts on fedora 23. -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] osm.org website translations
Hello How often are translations loaded from translatewiki.org to openstreetmap.org? Or should I somehow request that? Currently month old translations (Lithuanian ones) are not loaded yet. Thank you -- Tomas ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] osm2pgsql invalid polygons
So what is the general rule? Inner polygons in multipolygon should not have shared vectors (or in "human readable format": inner polygons should not "touch" each other)? -- Tomas Straupis ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Info request on: additional GIS layers
2012-03-07 Andre Joost wrote: > Another question is how they keep their basic database in sync with ours? That is a different question with a lot of different approaches ranging from using only OSM data up to simple validation-comparing of two available datasets for error identification. > I think > http://wiki.openstreetmap.org/wiki/The_Rails_Port > is what you are looking for. Yes, that is it! Thank you > Or they can use the UMN Mapserver to combine Data from several sources. Interesting. Thank you for pointing at it. -- Tomas Straupis ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Info request on: additional GIS layers
Hello 2012-03-07 Andre Joost wrote: > This is a point which breaks the idea: > > If *they* want to be responsible for the data and making updates, no other > user should have access to it. True. But this would be PART of the data they provide. And that part would barely be interesting to the rest of OSM community (both in terms of filling/taking care of the data and using it). To rephrase this: they do not want to HIDE some data (they are actually happy with adding that data directly to OSM), just some data is NOT interesting to OSM (f.e. land ownership, self-government community information, problems with infrastructure, illegal structures etc.). > So the only way is to add a separate layer with openlayers. Or use a WMS > layer with OSM data in the governmental Web-GIS. Point is to move government GIS to OSM software stack. They would then partly use OSM main db (where data is usable for OSM) and for other specific data they could use their own server (DB). This would be kind of "GIS layering". That is software/skills required would be the same (one stack, no zoo). People would have to learn to use "osm" and then there would simply be "more than one osm" :-) So both sides win: * OSM gets some additional data source (and more publicity) * government gets free software stack, provides better/more services to public etc. The part of creating separate layers (say with mapnik) and presenting them (say with openlayers) is clear. Uncertain part (for me) is just the database part because I only have a general understanding of what would be required (I guess database and "something" which gives API to access that database from JOSM/Mapnik etc.). Therefore my question is: where can I get more information about creating "own" database/api and setting up all infrastructure around it (f.e. authorisation, editors etc.). Or if there is any other approach of achieving this goal of "GIS layers"? Thank you -- Tomas Straupis ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Info request on: additional GIS layers
Hello Some governmental institutions are willing to use OSM software stack and share their information. For example they could use a base map and share information they have. Some information they want to collect and give out to public is perfectly "fit" for OSM (say max speeds) while others are purely "their" thing which should not go to main OSM database (say information like which "community" a specific house belongs to). (So they want to share all information with public, but some information should be captured on a separate storage/db) They would like to have a separate layer to hold such "non-OSM" information. This would keep OSM database clean and it would allow government to take responsibility for quality assurance and updates of that data. How should this be implemented? Separate osm database (with data of those additional layers only) and then different editors could be used to fill that database by pointing to api url of this extra server? If I need more layers, more databases have to be created etc.? (presentation part of mapnik/openlayers is clear) I was looking for information about this on osm wiki but didn't find anything useful so maybe you can give me some urls/keywords where I can read about it? Thank you -- Tomas Straupis ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
