Re: [OSM-dev] Restarting the EWG
Replying to Jochen and Frederik together here for compacting things. Thanks for the replies, although i would still be very much interested in the board's thoughts on the matter these comments already provide some valuable views. Since neither you nor me are pure hobby volunteers in OSM, us discussing how convincing your replies to my first question are is pretty academic. Actual hobby volunteers will ultimately have to answer that. Regarding the second question both of you seem to see no problem with the preference for "people whose work we know and enjoy" paradigm regarding money spending and personnel selection choices in an organization like the OSMF. That is where our views seem to fundamentally differ. A deeper open discussion on the matter within the OSM community would be fundamentally important (and i tried to incite such a discussion in the past on several occasions) but this list is certainly not the right place for that. Since Jochen brought the discussion a bit on personal motivation for volunteer work a few notes on that from my side - to maybe make it easier for others to understand why i choose to engage in certain volunteer activities but not in others. For volunteer contributions in OSM i chose those fields where i can make difference through the quality of my work - either craftsmanship in practical work or solid analysis, arguments and reasoning in more abstract matters. I deliberately and fairly categorically stay out of matters where i would have to negotiate with the interests of others independent of arguments and reason w.r.t. the common good. If that disqualifies me for a leadership position or even fully in having a substantial influence on decisions in the prevailing organizational culture of the OSMF i am fine with that. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Restarting the EWG
> Paul Norman via dev hat am 19.11.2020 17:09 > geschrieben: > > - managing development to be paid by the OSMF by contracts and grants, and For better understanding of how the board envisions this to work, could you explain: * why i as a pure hobby OSM contributor with experience in the field of development should volunteer my time to manage the paid development work of others on my own unpaid time. * how i as someone with a business or professional career interest in the OSM context would be able to contribute to this work without universally having a massive conflict of interest with every decision of substance that is being made. Please note although these might sound like rhetorical questions they are not, i am honestly interested in how the board envisions this to work. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Standardized feature/tag database idea & proposal
> Simon Poole hat am 16.11.2020 10:21 geschrieben: > > [...] On the other hand, not just because of the demographics, but also > because of the OSMFs involvement, like it or not, the iD presets are the law > of the land and anything else is simply irrelevant. From my experience with OSM-Carto that is not really accurate. Whenever iD presets suggest something different from established practice or community consensus the result typically is not the meaning of the tag changing to what iD says. The result tends to be a huge mess of inconsistent uses. In other words: iD's power over the development tagging and tag meaning in OSM is primarily either destructive or affirmative. iD has the power to wreak havoc and destroy a tag or it can affirm and support a trend in already predominantly consistent use. Same is the case with OSM-Carto. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Carto stylesheet vs Release osm2pgsql 1.3.0
On Wednesday 29 July 2020, Lynn W. Deffenbaugh (Mr) wrote:
> >
> > * The pgsql output now looks for lua script relative to the
> >
> > |style.json| file.
> >
> > This is a breaking change. Users might have to change the file
> > names of
> > their lua scripts in the style files.
>
> Does anyone know if changes need to be made to the OpenStreetMap
> Carto stylesheet or is it already compatible with this breaking
> change?
OSM-Carto has no style.json file, you specify the lua script directly at
the command line - i think the quoted change applies to the multi/flex
backends only, though the documentation is not very clear in the
terminology used ('pgsql output' vs. 'pgsql backend').
--
Christoph Hormann
http://www.imagico.de/
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
On Tuesday 26 May 2020, nd...@redhazel.co.uk wrote: > > Rasterization is a data generalization technique too. No it isn't. Generalization is the most frequently abused term in cartography. Calling ST_Simplify and way_area filtering on OSM landcover polygon data generalization is a bit like calling JPEG compression noise reduction. But we are drifting off topic here. > Yes, in some > cases it leaves no artifacts at all Who would say that? In fact sampling artefacts (a.k.a. aliasing) are the mother of all artefacts in computer graphics. The point is that sampling artefacts are very well understood and it is known how to minimize them. > I've been looking into the code and I couldn't find anything related > to polygon generalization. Where is it done? Lossy geometry compression is happening in ST_AsMVTGeom - which is not just the inevitable coordinate rounding to the vector tile coordinate grid but also the subjective choice of per feature Douglas-Peucker line compression. See https://github.com/postgis/postgis/pull/463 for the well argued suggestion not to do that. The way_area filtering is explicit in the SQL code. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
On Tuesday 26 May 2020, Tom Hughes via dev wrote: > More specifically what you are seeing is that it will let you zoom in > beyond zoom 8 but you are seeing data that was simplified on the > basis that it would be shown at zoom 8 where those artefacts would > not be visible. As said that is not quite true - but of course attributing artefacts, noise and rendering distortions that are visible to specific data processing steps is not easy. As Joseph has hinted when it comes to accurate rendering for precise mapper feedback without unintentional incentives at the lower zoom levels there are other options based on pre-rasterizing the data which would still allow using the benefits of client side styling. I have demonstrated that in the past. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
On Monday 25 May 2020, Paul Norman via dev wrote: > https://github.com/pnorman/openstreetmap-cartographic. In general this looks like a good approach to evaluate where the practical issues are without being bogged down by bloated legacy tools (well - except for the client side of course). > Landuse, vegetation, and other natural features are not rendered > until zoom 7. This is the scale of OpenStreetMap Carto zoom 8, and > these features first appear at zoom 5. There are numerous problems > with unprocessed OpenStreetMap data at these scales. OpenStreetMap > Carto gets a result that looks acceptable but is poor at conveying > information by tweaking Mapnik image rasterizing options. I'm looking > for better options here involving preprocessed data, but haven't > found any. Yes, i think this style as is is a good demonstration that the current approach used universally in the industry (and that is used in ST_AsMVT() as well) of performing per feature lossy vector geometry compression (usually in combination with way_area filtering) is a dead end. You can see that nicely if you overzoom the demo map: http://imagico.de/files/screenshot_cartographic.png http://imagico.de/files/screenshot_carto.png This creates a lot more artefacts and noise in the rendering than the AGG rendering issues in Mapnik typically do even at the resolution it is intended for - including counterproductively adding a lot of high frequency noise in many cases. > Often forgotten is the development requirements. The style needs to > support multiple developers working on similar areas, git merge > conflicts while maintaining an easy development workflow. I'm still > figuring this out. Mapbox GL styles are written in JSON and most of > the tools overwrite any formatting. This means there's no way to add > comments to lines of codes. Comments are a requirement for a style > like this, so I'm investigating minimal pre-processing options. The > downside to this will make it harder to use with existing GUI editors > like Fresco or Maputnik. I think this is a point that cannot be overstated. Mapbox GL JSON (or FWIW JSON in general) in contrast to CartoCSS is a software developer centered format and not a cartographer centered format. Many mainstream low sophistication users of client side vector tile frameworks focus exclusively on interactive editors and not working on the code level - which as you explained is not an option for a cooperative community project or for styles with a higher level of sophistication. Just re-casting the JSON data structures in a different format easier to edit by hand, suitable for comments and leading to well readable diffs is IMO not enough for a truly cartographer centered approach. But this is definitely a hard problem because there are so many in parts opposite requirements. Having a reference renderer that does not build on an extremely complex and platform dependent framework (a.k.a. web browser) would be important for style development - i think Joseph also hinted in that direction. It would also be important for things like automated tests etc. One other thing to keep in mind - OSM-Carto uses polygon fill patterns a lot for differentiating different types of polygons. While it is in principle possible to show fill patterns in continuous zooming in an ergonomic fashion i have so far never seen maps in the wild doing that. This would be something to consider if you want to create a style similar in the richness of landcover rendering to OSM-Carto. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Freely accessible aerial imagery tileserver?
On Wednesday 19 February 2020, Simon Poole wrote: > The very short answer, if you are referring to a global "high > resolution" imagery mosaic, is "no". Depending on your definition of "global" that answer might also apply to any (even non-freely available) imagery services and to a much broader understanding of "high resolution". ;-) -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Issue with https://www.naturalearthdata.com
On Wednesday 17 April 2019, Eric Gorr wrote: > > I am following the instructions at > https://switch2osm.org/manually-building-a-tile-server-18-04-lts/ and > got to the part where it wants me to run 'scripts/get-shapefiles.py'. > The problem is that when the script tries to download: > > http://www.naturalearthdata.com/http//www.naturalearthdata.com/downlo >ad/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip > Seems to be down. See: https://www.naturalearthdata.com/downloads/110m-cultural-vectors/ which uses https but is otherwise the same link. You can find the files as well on: https://github.com/nvkelso/natural-earth-vector/tree/v4.0.0/110m_cultural -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile in other projections? (2018)
On Friday 26 October 2018, Eduardo Ramalho wrote: > > Now I am trying to only show a country (Portugal) > with its specific projection EPSG:3763. > > I changed the "mapnik.xml" to have my target "srs" and specific > cartesian bounds, also changed renderd.conf to have aspectx=1 and > aspecty=2, > and finally removed all cached tiles. > > When I check the request url for /0/0/0.png I get a world view bitmap > (but very distorced). > I was expecting to get only the half north of Portugal on zoom level > 0. Something like this is to be expected. Mapnik supports rendering in arbitrary projections but renderd and mod_tile are likely hardcoded to Mercator and what you probably get is your local coordinate system but rendered with the mercator bounds - which of course does not make any sense. For local coordinate systems there is no natural tile system so what tile 0/0/0 shows exactly depends on the projection *and* the configured bounds. The bounds you define in mapnik are just the limits of the area mapnik renders. But mapnik does not know anything about tiling, this is done by renderd. The funny thing is that the tile manager could be completely projection agnostic, it would not even need to know what projection the map is in, it would just need to know the bounds of tile 0/0/0 in the desired projection - and divide these bounds as needed to generate the rendering requests for mapnik. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
On Saturday 23 June 2018, Christoph Hormann wrote: > [...] > > I still hope Lucas will write up a few details about the technical > approach that would enable you to more easily re-implement this in a > different style. By the way Lucas has already done so, you can find it here: https://github.com/sommerluk/roadpatternrendering Apart from the 2d pattern rendering of lines this also nicely explains how the drawing order works in Mapnik/CartoCSS in general. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
On Sunday 24 June 2018, Sven Geggus wrote: > > May I mention the fact, that Mapserver is still actively maintained > and that there even is a carto preprocessor called magnacarto which > is able to produce mapserver style files. Yes, Mapserver has some very powerful features - including even the ability to code styling logic in javascript: http://mapserver.org/de/mapfile/styleitem.html The problem i have with Mapserver from a designer point of view is that it offers a complex high level feature set with many functions clearly intended for direct use. But at the same time the language to formulate this in (the mapfiles) is not really suited to write and manage more complex stuff directly. And Magnacarto only supports the basic features offered by both Mapserver and Mapnik which does not really work for Mapserver in the end since it does not make use of what Mapserver can but Mapnik cannot. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
On Sunday 24 June 2018, Mateusz Konieczny wrote: > > > It is all open source of course so anyone can pick up development > > but you need to ask yourself if this would be worthwhile of course. > > Both the technological and cartographic environment have changed a > > lot since these programs were originally designed. > > What would you recommend for someone starting a new project? That's a really tough call. The only real recommendation i can give is a quite abstract one: Go small, concentrate on doing one thing and one thing only and avoid the impetus to go broader and then get bogged down by conflicting goals and the complexity of the whole thing. It is interesting you mentioned 3d rendering since i think (and also have frequently written in the past) map rendering can learn a lot from the historic development of general 3d rendering (which has a much longer and more faceted history than digital map production) For a map design project with similar scope and goals as OSM-Carto the situation today is quite impossible indeed. But you need to keep in mind the OSM standard style (combining the old XML style and OSM-Carto here) and Mapnik essentially grew up together so to speak. You can't expect today to be able to find a framework that can directly take over this role Mapnik essentially grew into over many years. So the goal would be to look for a toolchain that has the potential to grow into such a role over time which is both a technical requirement and one regarding the developer community. But it is also possible that what the OSM standard style represents, the one size fits all map for OpenStreetMap, will end in the near future - as a dinosaur incapable to adjust to a changing world. And that my recommendation to go small and concentrate on doing one thing and one thing only not only is to be applied to software development but also map design projects. But i don't see this happening at the moment yet because i don't see the open, non-commercial limited scope map design projects flourishing. How likely this is to happen in the future depends largely on the burden of suffering felt by the OSM community because of OSM-Carto not being able to deliver certain things. And to get back to the original subject of this thread - the issue regarding rendering of paved/unpaved was open for nearly five years, the desire to have this rendered had existed probably already significantly longer (surface tag reached one million uses in 2009). And despite this pressure and the incapability of OSM-Carto to deliver on this it has none the less not led to the development of a specialized roads map style to fill this gap. So the inertia in the social environment surrounding the OSM standard style is immense. It is quite possible that from this point of view OSM-Carto could continue more or less indefinitely and the only contraint would be to mitigate the scaling problems (as it is already done with aggressive way_area filtering for example). > Recently Mapzen ecosystem looked interesting for me (though > making maps good from cartographic design viewpoint was not one of > its strength), but I picked poor moment to start with Mapzen :) Well - as i wrote some time ago (http://blog.imagico.de/on-permanence-in-it-and-cartography/) Mapzen did not have any truly revolutionary and innovative ideas in their portfolio. They had the advantage of not being as dominated by immediate business needs as some of the other commercial players which allowed them to think things through a bit better in some cases and make tools a bit more generic which also makes them look more attractive to third parties obviously. Still there is a lot of 'just another version of the same thing essentially' in what they did. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
On Saturday 23 June 2018, Yves wrote: > > Which is not going to change if everybody scripts its own way :) > I did not followed the discussion around that change, but it would be > a good idea to make changes upstream to come up with an equivalent > style without special code. Yves The question is how long would you be committed to wait for an upstream change to be implemented? Fixes for major bugs like here: https://github.com/mapnik/mapnik/issues/3558 can take a year. Not sure how requests for feature additions fare on average. But i think without an existing implementation being submitted as a pull request the chances are relatively small. It is commendable that Mapbox still does some basic maintainance and bugfixing there but this is hardly more than keeping the lights on. And any feature addition would also need support in Carto which - as Daniel already mentioned - is definitely and officially not maintained any more. It is all open source of course so anyone can pick up development but you need to ask yourself if this would be worthwhile of course. Both the technological and cartographic environment have changed a lot since these programs were originally designed. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
On Saturday 23 June 2018, Sven Geggus wrote: > > I'm doing this for German style for more than 2 years now without > that much of a hassle up till this release. > > This will now kill my ability to keep on doing this at least as > roads.mss is concerned! This is to be expected. You are kind of spoiled because during the last year this is about the only larger technologically more sophisticated change in the style. Discussion of this change can be found in: https://github.com/gravitystorm/openstreetmap-carto/pull/2640 and discussion leading up to this in: https://github.com/gravitystorm/openstreetmap-carto/issues/110 The reason for the technological complexity of this and the resulting mangling of the code is the lack of support to render lines with a 2d pattern in Mapnik. I still hope Lucas will write up a few details about the technical approach that would enable you to more easily re-implement this in a different style. Note there have been voices to choose a more conservative approach to rendering unpaved roads in OSM-Carto - like yet another form of dashing in addition to the various dashing types already used in road rendering. There are very good cartographic reasons not to do that though. All in all this is a good example for OSM-Carto being at a crossroads (and having been for quite some time) between staying avant-garde and pushing the boundaries of cartographic design and technology or being satisfied with shuffling the options offered by the cartographic mainstream within the technological framework used - and which, due to Mapnik and CartoCSS being essentially unmaintained, becomes more narrow and limiting every year. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] GDPR implementation on planet.osm.org
On Wednesday 20 June 2018, Roland Olbricht wrote: > [...] > Taking GDPR serious means every data processor must decide which use > cases they make simple, which use cases they make hard, and tailor > the documentation according to that. For example, for that reason > Overpass API has no feature to track all actions of a single user. I > have proposed a declaration tailored to Overpass API on the FOSSGIS > list (the FOSSGIS is sponsoring the server operations), and I would > prefer to go forward with that one. A central ToU does not help, > hence having it ticked or not is of no interest to the data > processor. Since not everyone knows the draft you suggested in FOSSGIS - the plans you sketch there (correct me please if i am wrong) essentially say that you intend to continue distributing geodata and timestamps without access restrictions but plan to manage restricted access to other data (changesets and user identities) using your own mechanism and own criteria of approval (which are not completely finalized yet). As i understand your mail here you think this clashes with the OSMF plans because these will require you - for accessing the raw data to feed into the Overpass API - to accept the OSMF ToU which likely will a) not allow you to distribute data with timestamps without access restrictions b) require you to implement access restrictions using OAuth I assume if this is actually the case will depend on the specifics of the OSMF ToU. I would also assume that (b) most likely would not require you to use OAuth with every request, you probably could just use OAuth when people register with you for an API key. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] GDPR implementation on planet.osm.org
On Wednesday 20 June 2018, Frederik Ramm wrote: > > In my view, this is not "cargo cult". If someone comes to us, today, > and complains that their OSM contributions are being used to stalk > them, then we cannot even point to a rule that says you cannot do > this. The stalker is, as far as OSMF is concerned, 100% within their > rightful use of the data. This is something that needs to stop - even > if, in the future, it only becomes marginally more difficult for the > stalker to use OSM data, at least we clearly say that (a) this use is > not allowed, and (b) the stalker knows it. I am no legal expert on this matter but as i understand it you do not need a contractual agreement to restrict what others can do with data you give to them. You simply put this in the terms of use and the data user can either use the data under these terms or not at all. If people use or distribute the data for other purposes they are in violation of copyright. According to my layman understanding of the law here a copyright violation is much less troublesome to prove in a legal struggle than a contract violation (which would require you to prove the existence of a valid contract which is likely almost always impossible practically in this case). > Yes. This also requires the delicate distinction that not everything > in a .osm file is necessarily under ODbL. Note you can - based on the current contributor terms - only do this if you declare the sensitive metadata to not be data contributed by the users because for contributed data the OSMF only has the mandate to use or sub-license under an open license. But if you declare the metadata not ODbL you run into the problem that if you combine it with ODbL geodata you have to consider the share-alike requirements of the ODbL. And at least for history data where correct interpretation of the geodata depends on the timestamps you will have a hard time interpreting this as a collective database. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] GDPR implementation on planet.osm.org
On Tuesday 19 June 2018, Frederik Ramm wrote: > [...] > 3b. ensure that everyone who has an OSM account agrees to these > guidelines one way or the other, This is the point that looks very fuzzy to me. Could someone point out the legal concept behind this idea for me? Such agreement would not be an agreement to process your own data given by individuals to the OSMF (which is the kind of agreement you would normally expect in the GDPR context). You probably mean some kind of contractual agreement about what can be done with the data. But to be honest i don't really see the point in that. People who download the data can easily create an ad hoc account every time they download data. The OSMF does not verify the identity of who is behind a user account created. So what do you expect to gain from such an agreement? Is there any reason to assume that in a case of such data being released in a way that is not according to the legal requirements by a third party the agreement can be used to avoid legal responsibility for the OSMF it would otherwise need to face? To me this looks more like cargo cult actionism, doing something that communicates being a serious measure at the surface but which is a hollow promise at a closer look. Note these concerns are not about the idea of restricting access to sensitive data to logged in users, it is about requiring some kind of agreement from these users. What i can understand is giving people a simple selection option between [ ] i want to be safe w.r.t. personal data and not being provided potentially sensitive information when logged in. [ ] i want to have the possibility to access potentially sensitive data when logged in. which would mainly be a service to the user - kind of like the sensitive content switch on youtube. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Loc2Vec
On Monday 18 June 2018, Mateusz Konieczny wrote: > > One interesting thing about this kind of work is the legal side and > > the question if a process as described there is a derivative work > > of the training data used. > > I am surprised that somebody may think that it is not a derivative > work of training dataset. Given how rarely you see attribution for data sources in the 'big data' sector and how widespread attribution requirements are in the open data world OTOH it is pretty likely that use as training data in adaptive algorithms is widely not considered to be copyright relevant. If this view is correct or not is a whole other story. The topic is also further complicated because if there is a relevance of copyright/database rights here there are still two options: * The trained algorithm is considered to be a derivative of the training data but the output of applying the trained algorithm is not. * The output of the trained algorithm is considered to be a derivative of the training data as well. I think it is probably possible to create scenarios where all three viewpoints (the two above and the 'no copyright issues involved' case) are plausible interpretations. So there are no simple yes/no answer to the whole subject - you would need to draw a line somewhere - preferably one that, as i phrased it in legal-talk recently, "supports desirable and harmless use cases but does not create loopholes against the spirit of the license." -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Loc2Vec
On Saturday 16 June 2018, Michał Brzozowski wrote: > I'm quite surprised this went past the radar, because it's very > interesting research and abuses openstreetmap-carto style in a novel > way: > > https://www.sentiance.com/2018/05/03/loc2vec-learning-location-embedd >ings-w-triplet-loss-networks/ > > [...] One interesting thing about this kind of work is the legal side and the question if a process as described there is a derivative work of the training data used. If you are tempted to answer that with a 'no' you should keep in mind that while most work on so called AI methods these days concentrates on data processing after training you can equally try to use such methods more like a memory device. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Generalisation
On Wednesday 02 May 2018, Marco Boeringa wrote: > > "And also it is ultimately: Demo or it didn't happen - at the moment > the only thing you can get at 1:50k is the old style map: > https://www.pdok.nl/nl/producten/pdok-downloads/basisregistratie-topo >grafie/topraster/topraster-actueel/top50raster" > > This is a serious misunderstanding. This is the a rasterized version > of the new style 1:50k vector map based on generalization, that just > happens to look very close to the originally manually generalized > one: that was the whole target of the effort. They make available the > rasters for clients, as it is just an easy way to consume the data, > styled and all. If that is the result of what you so boldly described as the first fully automated generalization workflow of a national mapping organization that would be fairly underwhelming - both in terms of "fully automated" and "successful". -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Bolder - Starting a new client-side OpenStreetMap style
On Monday 30 April 2018, Paul Norman wrote: > > I didn't find noticeable differences between the Natural Earth and > OSM data I'm using at these very low zooms I find this kind of hard to believe since the differences are so obvious and characteristic. In most cases when i ask people why they use Natural Earth in web maps it is not that they don't think there is a visual difference but because of indifference, i.e. people don't care about the differences. Of course most of the differences are in areas 90 percent of mappers and map users never consciously look at. That's the difference between noticeable and noticed. ;-) And of course Natural Earth ocean polygons are not equivalent to OSM water polygons definition wise - you need to cut out the Antarctic ice shelves for that: http://www.naturalearthdata.com/downloads/50m-physical-vectors/50m-antarctic-ice-shelves/ Almost everyone gets this wrong. > I've been using the same script with my server demoing the new WMF > style work. It doesn't run automatically there, but i have run it a > lot. It takes about 30 seconds to load and process the water polygons > for the coastline on my home server, and that includes clustering and > index building. It takes longer to download the file. > > The script makes sure to create a table optimised for rendering, with > the contents clustered, indexes created for a read-only case, and > some other tweaks. This avoids bloated tables. > > How do you consider this case extreme? Well - >99 percent of the coastline data stays identical from one day to the other typically. So you are replacing several hundred MB of data in the database every day with overwhelmingly exactly the same data. In absolute numbers this might not be too significant but relatively speaking this is like doing a new full planet import every day. When creating the regular split coastline polygons for openstreetmapdata.com (which are currently only produced in geographic coordinates, not in mercator) i contemplated the idea that it would actually be nice to create a way for data users to only get those tiles where something has changed. This would be possible but it would be quite a bit of work to set up and it would add quite a bit of complexity for the data user so it is unlikely many would use this. But it would be a much more elegant way to perform coastline updates when you have the data in a PostGIS database. > I've been experimenting with different colour selection methods, and > have found using a limited colour palette helpful. I want to specify > colours in LCH, not sRGB, and Tangram scene files, like most style > files, only allow direct input of the latter. The problem (apart from a somewhat awkward syntax of color specification) is you unnecessarily limit the accuracy at which you can specify colors at in a quite severe way (if i saw correctly you have a fixed color palette of about 1700 colors). You probably say you don't need more than 1700 distinct colors but if you for example want to create a harmonically stepped color sequence for roads those colors will usually not contain the ones you would ideally want for such purpose. And in terms of sampling desity of color space 1700 distinct colors is very coarse. I don't know if Tangram allows defining custom color conversion functions in some way - i am not even sure how the native color format is defined, i.e. if that is sRGB or linear color space as you would normally expect with OpenGL: https://mapzen.com/documentation/tangram/draw/#color -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Bolder - Starting a new client-side OpenStreetMap style
On Monday 30 April 2018, Paul Norman wrote: > > I’ve started work on a new client-side style for OpenStreetMap data, > and feel it’s reached the point where I can release it to the public. I think that is a good idea - i in particular like that the toolchain used seems to be free of NodeJS. ;-) > The style should still be useful for mapper feedback, and some ways > will become more useful. Vector tiles can associate OSM feature IDs > with objects in many cases, helping debugging “where did that label > come from”. As you know i doubt serious overall mapper feedback is possible at the moment with client side rendered vector tiles but the possibility of additional QA functionality based on data in the vector tiles is an interesting option. I have not actually tested it yet but you seem to be using stone age Natural Earth data at z0-z5 - which is a big step back from OSM-carto IMO. Do you have practical experience in deployments with loading external data, in particular the coastlines, in PostGIS with regular (daily) updates? Certainly this is possible but it is also kind of and extreme case of using PostGIS for something it is not intended for. In the client style the method to specify colors seems rather odd to me - is this some kind of standard in this field or is this just an invention of you? -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Generalisation
On Monday 16 April 2018, Martin Koppenhoefer wrote: > AFAIK, in the osm-carto style there is no generalization on live data > for performance reasons (because of continuous data updates via > minutely diffs). There are some precomputed / extracted data files > though, some of which contain generalized (simplified) data. These > are all "external" sources: I think this requires some clarification. There are different definitions of what kind of operations and processes you call "generalization". But typically selection is considered a generalization operation meaning that most of what is shown at all but the highest zoom levels is shown with some level of generalization. Most selection is rather primitive - based on fixed zoom level thresholds, the dreaded way_area filtering etc. There are very limited attempts at some additional sophistication, like for populated place rendering at low to mid zoom levels. When talking about explicit geometric generalization while specifically exclusing lossy vector data compression (which is widely mischaracterized as generalization) the Natural Earth boundaries at z=1-3 is currently the only case. Lossy vector data compression in addition is currently/historically used in the following places: * Coastlines at z8-9 - the visual impact of this was rather small but visible in some areas (in particular with small islands) if you kept an eye on it at the transit from z9 to z10. This has been the case for a long time but is essentially superseded by: * Coastlines at z0-9 since https://github.com/gravitystorm/openstreetmap-carto/pull/3065 * The administrative boundaries at z4+ since https://github.com/gravitystorm/openstreetmap-carto/pull/3103 You need to keep in mind that both geometric generalization and lossy vector data compression operations are largely incompatible with the current goals of OSM-Carto: https://github.com/gravitystorm/openstreetmap-carto/blob/master/CARTOGRAPHY.md -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] [OSM-talk] Nominatim - censorship at github pages
On Monday 26 March 2018, cm-sani...@wp.pl wrote: > [...] In my opinion the fact I am not > liked by the maintainers, shouldnt result in deleting my on > topic comments. Is there some oversight over moderation? Can I > somehow repeal the decision? Or at least get some explanation why my > comment was removed? I can't comment on the comment deletion without knowing the content of the comment in question. But on a more general note - project maintainers on github are autonomous in how they manage discussions and development. OpenStreetMap is not a centralized project and there is no overall authority that controls software development. So if you are dissatisfied on a fundamental level with how Nominatim (or any other similar project) is run (you write "maintained in a way which makes it quite impossible to contribute to") the way to go is to create a fork or to develop an alternative from scratch according to your ideas. If this turns out to be better or offer advantages over Nominatim you can try to get it integrated into the OSM website in addition or instead of Nominatim. This approach (having alternatives and fostering diversity in tools and development goals) is generally superior for innovation and progress in general compared to trying to be universally inclusive w.r.t. feature and design ideas or communication and cooperation styles within projects. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Working with OSM data with less or no metadata
On Wednesday 14 February 2018, Darafei "Komяpa" Praliaskouski wrote: > > > > While this seems a useful test to do i wonder how the timestamp and > > version fields are relevant regarding privacy and personal data > > protection? > > If OSM API were fast enough, it would allow to rather easily group > changes back to changesets. Number of changesets gets you back number > of mappers. Classifying a returning mapper by edit pattern would > allow to get back the geometric median of their edits, which brings > you to knowing where they live. No, even if the API was infinitely fast limited bandwidth and the possibility to append new edits to existing open changesets would make this impossible. And even if you could identify the changeset a certain feature was last modified in that would still not allow you to conclude which changesets are created by the same user. If you practically want to reverse engineer user identities from a planet file with user info stripped looking at the data itself and mapper specific data charateristics (tag combinations, the way geometries are drawn) would likely be more useful than versions and timestamps. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Working with OSM data with less or no metadata
On Wednesday 14 February 2018, Michael Reichert wrote: > > All challenges are run four times, one iteration with full metadata, > one with timestamp and version fields, one with version field only > and one without any metadata. [...] While this seems a useful test to do i wonder how the timestamp and version fields are relevant regarding privacy and personal data protection? I know a possible answer could be that in combination with personal data (like user names or ids) it would provide additional information on people. But this argument applies to *any data* including the geometry (i.e. coordinates) and tags. So what is the special thing from a legal standpoint about versions and timestamps compared to geometries and tags? -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Extra-large metatiles
On Monday 15 January 2018, Paul Norman wrote: > > Latency does not matter with a pre-render task, so I've been > considering using extra-large metatiles. Has anyone tried this > before? Not for normal rendering but for raster based data pre-processing/generalization tasks i am usually working with 16k metatiles. There are mostly two things that limit the usefulness of increasing tile size in this field - which might to some extent also apply to normal rendering: * memory footprint: Since the processing itself is usually single threaded you run several tiles in parallel and the memory demands for this must stay within the available memory. * processing time granularity: Since different parts of Earth take very different times to process it can happen that the slowest processing tiles are processed last leading to less than optimal parallelity in processing overall (i.e. you have one or two tiles dragging on in the end using only part of the ressources available while the rest is already finished). > Based on experience from metatiles 1-8 tiles across, as size > increases I expect query time and time to render to increase, but > time per tile to decrease. I don't know at what point this stops > being true. I would expect this to happen pretty fast, possibly 8x8 is already at or near the top. But it will likely depend quite a bit on the hardware used and the specific rendering task. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] looking for some spline code to adapt
On Sunday 02 July 2017, Rich Morin wrote: > I'm generating sets of tactile maps for a blind friend, as follows: > > OSM export files -(Ruby)-> PostScript -(lpr)-> swell paper > > I'm using straight lines for the roads, using the PostScript moveto > and lineto operators. However, I'd like to use the curveto operator: > > x1 y1 x2 y2 x3 y3 curveto - > > This operator draws a curve from the current point to the point > (x3, y3) using points (x1, y1) and (x2, y2) as control points. The > curve is a Bézier cubic curve. In such a curve, the tangent of the > curve at the current point will be a line segment running from the > current point to (x1, y1) and the tangent at (x3, y3) is the line > running from (x3, y3) to (x2, y2). > > [...] You can certainly do so but you need to expect that your results are largely wrong. The problem is that in OSM we do not record any supplemental information on ways regarding if the corners at the nodes are actual corners or if they are just approximations for a smooth form of some sort - what you would usually do with a kind of tension parameter. Neither can you rely on the fact that when two roads connect this is not a smooth transit. It is not even defined if when a way approximates a smooth form the nodes of the way are supposed to be on the curve or if they should be placed so the average distance of the way to the curve is minimized. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v3.3.1
On Thursday 25 May 2017, Julio Costa Zambelli wrote: > Dear Matthijs, > > I am seeing the waterway=stream+intermittent=yes ways, as black > lines. Is this the intended result of this change? This change has not been rolled out on openstreetmap.org yet: https://github.com/openstreetmap/chef/issues/124 -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Planet file with preprocessed lines/polygons
On Monday 15 May 2017, Jochen Topf wrote: > [...] What about non-multipolygon relations like routes > and turn restrictions? How to represent them? A general format should > probably allow different options here. But if you want to make this > is available for download, which variant will it be? I would like to emphasize this point. We currently have a very limited set of generally established relation types - essentially only multipolygons, routes and turn restrictions. The hurdles to introduce something new here are already high due to the difficulty of getting application support even though file formats used at the moment do not yet limit this. If we would have in the future a widely used standard format for data users that is fixed to multipolygons and routes that would essentially freeze our open relation type data model into the currently established subset. In addition to relations (which are not very popular with many mappers anyway) there are also cases where geometry information is encoded in tags for which an universal OSM geometry data format might need to include support. Practically the best known case is the 3d buildings. This might not be overly relevant for a purely 2d format. But there are also possible uses of this concept in 2d. I have not seen this being used but it would for example be possible to introduce a new circle geometry based on nodes (like with tags type=circle and radius=x) that would allow mapping circular features more precisely and more compact than approximated with a polygon. Of course this would not be lost in a format allowing freely tagged nodes but at least if you add indices or would allow such geometries to be part of multipolygons you would need explicit support for this. Ok, at the moment this is all fairly theoretical but i none the less wanted to mention this as a suggestion to keep this in mind when contemplating ideas on file formats. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Natural Earth
On Wednesday 29 March 2017, Sebastian Kürten wrote: > Can somebody tell me whether the map on openstreetmap.org still > relies on Natural Earth data for rendering of lower zoom levels? Natural Earth data at the moment is used for the lowest zoom level boundaries (z1/2/3) and for the builtup areas at z8/9. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Removing functionality and giving just No as answer
On Friday 24 February 2017, Jóhannes Birgir Jensson wrote: > As evident in the discussion on issue > https://github.com/openstreetmap/openstreetmap-website/issues/1446 > there are several people interested in maintaining functionality that > existed only a few days ago - being able to view single tiles just as > easily as before. > > [...] I think the key here is to have more diversity and not insist on every functionality that could be desirable to be present on openstreetmap.org. Certainly this would seem convenient but it ultimately will never be possible and it would be extremely hard to maintain. I for example really miss a coordinate display for the mouse position but there are plenty of other map interfaces that offer this so no big deal. When closing the discussion Tom clearly stated what kind of solution he considers acceptable: https://github.com/openstreetmap/openstreetmap-website/issues/1446#issuecomment-282233549 Everyone is entitled to disagree with that but you'd also have to ask yourself why the outlined solution is not acceptable for you. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v3.0.0
On Thursday 22 December 2016, Sven Geggus wrote: > > Unfortunately my german style fork is based on a modified > project.yaml not project.mml (which I always ignored) thus your > change will now basically kill my fork because I did not maintain a > forked .mml :( No, the new project.mml is just a renamed project.yaml. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Turning boundaries to linestrings
On Tuesday 13 December 2016, Paul Norman wrote: > > Feedback from anyone interested in using this output is welcome, as > well as any additional information that should be added to the > linestrings. You have some conflicting goals here - you want to maintain the OSM tagging information on the way level so you need to maintain the individual ways - which however limits what you can do in terms of rendering - dashing is not consistent for example where a boundary is split and you have problems doing geometry processing (like the all popular ST_Simplify()). IMO a process that merges the ways where it is non-ambiguous would be more useful. And most cases where tagging varies in an admin boundary without there being a junction that is a tagging error. The maritime attribute based on tagging is currently of very limited use since it is not consistently applied. On admin_level 2 a better way would be to classify this based on topology - i.e. all boundaries that are only part of one admin_level 2 relation are maritime boundaries. On the higher admin levels this is more complicated of course and not easy to solve. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] A PostgreSQL extension for real-time simplification of objects in OSM API database
On Wednesday 16 November 2016, RTOSM DOOPAS wrote: > > In short, rtosm is about to extend the OSM API to make it support > viewing and editing data at any scale directly from database. The OSM API's main purpose is two way data transfer in small chunks. Since you modify the geometry this rules out any editing and transferring data back. For rendering/viewing your concept hinges on the idea that dropping nodes is a good way to simplify geometries for rendering purposes which is not generally the case. Or in other words - you assume that your error metric accurately describes the perceptual error when viewing the rendered geometry (which is evidently not the case). -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Tile usage without proper identification
On Tuesday 23 August 2016, Frederik Ramm wrote: > > I'm in favour of privacy too but please spend a moment to think what > would happen if everyone did what you do - we'd never know who uses > our tiles, at all, and hence we'd be unable to allocate our resources > for the maximum benefit of all. The responsible and effective implementation of privacy here would be to configure your webbrowser to prevent websites to load external data like images. This is course fairly theoretical since a lot of websites would become unusable. If you turn off referrer transmission w.r.t. tiles the only effect this has is it prevents the tile provider from knowing what website you were requesting the tiles from. It does not prevent the tile provider or the website operator from learning about your map browsing activity. In case of non-free tile providers by the way use of API keys and tokens means the website is known to the tile provider even without a referrer. So you'd just punish the free services by withholding them information others get despite these measures. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Tile usage without proper identification
On Friday 12 August 2016, Andy Allan wrote: > > We've always allowed other people to use our spare capacity on the > tileservers, but recently it's got completely out of hand. Most of > the use of our tileservers has become developers looking for free > maps, nothing to do with the rest of the project. Often these are > commercial companies who are using our tileservers and selling their > apps. Subsidising commercial companies isn't the best use of > community donations and volunteer sysadmin time, when there are many > alternative services (such as those run by CartoDB, Stamen, etc) that > provide zero-cost map layers based on OSM data anyway. It would of course also be possible to be more strict in the limits of using the tiles in general on websites etc. but care would need to be taken to not scare users away from OSM into the arms of Google etc. By the way currently http://wiki.openstreetmap.org/wiki/Tile_usage_policy does not list any companies that provide drop in replacement services for the standard style. Using the services listed there websites would be forced to switch to a different style. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM-Carto style, ported to vector tiles!
On Monday 25 July 2016, Rory McCann wrote: > > > That however would also mean you would need to remove all the > > > > way_area > 0.01*!pixel_width!::real*!pixel_height!::real > > > > filters from SQL for z14 - otherwise you get missing geometries at > > the higher zooms (not a real lot - that limit is about one Mercator > > square meter at z14 - but when this occurs it would be visible and > > possibly surprising). > > Oops. I will fix that. Do you know of an example of where this > filtering applies in the normal OSM tileserver? It's helpful to have > something to compare to? That filter is primarily for the lower zooms for performance reasons, in particular z6-z9. You should be able to simply remove it for z14 without negative effects. > Yes, you can't use SQL to filter different things after z14. You have > to use CartoCSS filters. A z14 tile needs to have all the data a z19 > tile needs to render. The problem is not really filtering - this would be simple using additional attributes generated from SQL. The real difficulty is geometry processing. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM-Carto style, ported to vector tiles!
On Sunday 24 July 2016, Rory McCann wrote: > > The problem (correct me if I'm wrong!) is that with vector tiles it's > common to generate them only down to a certain zoom level (e.g. z14), > and for lower zoom levels (z15+) to use the z14 tile, and just crop > out the area you want, i.e. you are using the result of z14 SQL > queries for z15+. > > The SQL queries are run when generating the .pbf vector tiles, and > only go down to z14. For generating images, SQL isn't run and instead > have to use the pregenerated pbf, and hence pregenerated way_pixels > value. Hence you have to change the style, not the SQL. > > If you don't change it, areas that should appear at z15+ won't, since > the way_pixels will be wrong for that zoom. Ok - so you essentially say that use of !pixel_width! and !pixel_height! is - in the vector tiles system used - limited to the lower zoom levels. That however would also mean you would need to remove all the way_area > 0.01*!pixel_width!::real*!pixel_height!::real filters from SQL for z14 - otherwise you get missing geometries at the higher zooms (not a real lot - that limit is about one Mercator square meter at z14 - but when this occurs it would be visible and possibly surprising). And if i understand you correctly this also means you cannot render any features differently on z14 and any higher zoom level based on processing done in SQL - unless you duplicate them into different separate layers for different zoom levels. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM-Carto style, ported to vector tiles!
On Friday 22 July 2016, Rory McCann wrote: > Hello all, > > Over the past while, I've been working on porting the > openstreetmap-carto style to vector tiles. I'm happy to have > something of beta quality to release to you all: > > https://github.com/geofabrik/openstreetmap-carto-vector-tiles > Looking at your changes a significant part seems to be dealing with way_pixels and indicates that the meaning of way_pixels in your version is quite different from that in the normal standard style. Could you explain that? You do not seem to have changed the SQL. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Storing node locations on ways
On Wednesday 20 April 2016, Jochen Topf wrote: > > Adding node locations to ways is a pain (and needs more than 32 GB > RAM these days for a full planet if you want to get it done in > reasonable time). So I experimented with adding the node locations to > the ways once and storing it in the OSM files. Works for XML, PBF, > and OPL. See my blog post at > https://blog.jochentopf.com/2016-04-20-node-locations-on-ways.html > for the full story. From perspective of database design this of course introduces redundancy in the data which in turn bears the risk of introducing inconsistencies. For storing data temporarily during processing this is not generally a big deal because the processes writing and reading this data are well defined but as a general data exchange and storage format this would be highly problematic - depending on how a program reads the file it will potentially get different results if there are inner inconsistencies in the file. On a more general note: for the purpose of storing the OSM planet data in a form that is efficient to use and update wouldn't it make sense to store the node locations in a flat file separately from the rest instead of storing everything in a dynamically structured format like XML or PBF that can essentially only be read sequentially? Another thing: you say you throw away untagged nodes that are not member of any way - don't forget nodes can also be relation members. I don't know any practical case where an untagged node is a relation member but in principle this is possible and could make sense. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Upcoming change in files on openstreetmapdata.com
In the processed coastlines available for download on openstreetmapdata.com we are planning a change in the files that could affect some data users. Specifically we intend to move the split polygons in EPSG:4326 projection from the current dynamic splitting to a regular 1x1 degree grid. I described the reasons for this in more detail here: http://blog.imagico.de/on-slicing-the-world-news-from-openstreetmapdata-com/ The new style files are temporarily available on http://data.openstreetmapdata.com/land-polygons-split-regular-4326.zip http://data.openstreetmapdata.com/water-polygons-split-regular-4326.zip I assume this change will be an improvement for the majority of users but if there are data users for whom this might cause problems please let us know. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] changes in coastline are not rendered
On Monday 08 February 2016, Gerd Petermann wrote: > > It was changed more than 3 days ago and the change is not rendered. > > I understand that coastline ways are special, but that seems too long > for me. > > Any hints why this takes so long ? Coastline processing on openstreetmapdata.com is stuck for more than a month now, mostly because a lot of back and forth with tagging of large lakes as coastline. For information: to avoid major disruptions of map rendering due to data errors the coastline is not updated if there are larger changes in the geometry compared to the last time it was successfully processed. Any addition or removal of a lake with coastline tag will require manual intervention and we do not have the time to manually check the data every day because some mapper somewhere wants to scratch an itch and decides to tag a lake outline with natural=coastline. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Why is ford=yes not rendered in the default style?
On Thursday 26 November 2015, Gerd Petermann wrote: > > I know we should not map for the renderer, but why is > the deprecated and rarely used highway=ford > rendered with a nice icon on https://www.openstreetmap.org > while the much more often and documented > ford=yes is not ? Because the ford key is currently not in the rendering database. See: https://github.com/gravitystorm/openstreetmap-carto/issues/267 -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Migrating osm.org to vectors/Kartotherian
On Saturday 31 October 2015, Yuri Astrakhan wrote: > > Christoph, re multiple projections - if Mapnik implements vector tile > merging, we could in theory dynamicaly recombine tiles and change > projections, would't we? At a performance cost of course. First of all if you don't use the tiles as tiles but merge them into a full data set again there is no point in generating them in the first place of course. Apart from that regarding the abstract possibility: probably no for a number of reasons: - since mercator tiles don't include areas near the poles you will not be able to handle any projections that include the poles although better display of polar regions is of course one of the primary reasons for using other projections. - in general the tile edges in mercator will not be straight lines in other projections. You would have to specifically take this into account when generating and processing the tiled data, otherwise you can't reproject the tiles without generating artefacts or invalid geometries. - you cannot map the zoom levels in one projection to those in another, especially not with the strongly variable scale of the mercator projection so unless your tiles are not zoom level specific you are screwed. These points apply to all generic geometry data in the tiles, any data specifically generated for rendering like label positions etc. is normally projection specific anyway. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Migrating osm.org to vectors/Kartotherian
On Friday 30 October 2015, Jochen Topf wrote: > > This is not a failure of the vector tile approach in general. It > might be a problem of how the vector tiles are generated today, but > not a general problem of the vector tile approach. A vector tile can > contain any data that you need for rendering any kind of style. The > question is how the data will get into that vector tile. Currently > the tool chain that does this is not mature enough, I agree. It > doesn't take into account different sources of data (needed for > special maps) and doesn't do generalization well enough. But this is > something we can and should work on. We need to distinguish a number of different things here: 1. the idea of storing and transporting geometries in a tiled form in general 2. the concept of current vector tile stacks 3. the idea of processing geometry on a level that is not generally possible to do on the fly during rendering I was exclusively talking about 2. here, in particular because in contrast to the classical non-vector-tile stacks theys are not suited to efficiently deal with the geometric complexity issues without 3. As i have mentioned many times i think 3. is important and grossly neglected. Vector tiles might bring an additional incentive to address this of course. The one important thing speaking against tiled geometry storage and transport in general is handling of different map projections. Any such system will require you to select one projection from the very beginning and you stay bound to it or need fully separate setups for every projection you want to offer. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Migrating osm.org to vectors/Kartotherian
On Friday 30 October 2015, Martin Koppenhoefer wrote: > > > > - improved tile serving efficiency > > - a larger bandwidth of style variations > > - tighter contraints in basic styling decisions beyond what is > > already imposed by the OSM data model > > some additional points that come into mind: > > [...] Note these are issues with client side rendering, not with vector tiles - those are ultimately independent things. That client side rendering is often not very efficient if you consider overall costs is a given. Especially when serving different styles from the same infrastructure (as most vector tiles map providers do) there is certainly potential for energy saving by caching the vector representations on the server side. If the way this is done with current vector tiles stacks is the most efficient approach for that is a different matter of course. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Migrating osm.org to vectors/Kartotherian
On Friday 30 October 2015, Yuri Astrakhan wrote: > > Kartotherian <https://github.com/kartotherian/kartotherian>, the > Mapnik+Mapbox-based vector service has been implemented and > trial-launched <https://maps.wikimedia.org/> at Wikipedia. The > service itself is fairly stable, but the styles can use some > improvements - both the sql->vtile > <https://github.com/kartotherian/osm-bright.tm2source> and > vtile->image <https://github.com/kartotherian/osm-bright.tm2>. > Hopefully this work can be used as the basis for the osm.org style. > Once the vtiles are ready, we can easily move to client side WebGL > rendering. From my perspective this, i.e. imposing a certain technological framework on designers based on technological considerations, is the wrong approach. I wrote about this on my blog recently from a slightly different angle[1]. For a high quality style, design development has to mandate the technological framework, not the other way round. If you look at design problems recently discussed in the osm-carto style development[2] you will see most of them have nothing to do with vector tiles, they would not be made any easier to address with such an approach. On the other hand there are a multitude of things the current style handles fairly gracefully, especially the problem of reducing geometric complexity, that would be much harder to deal with in a vector tiles system[3]. In general it seems to me vector tiles are today often carried as some kind of religious mantra promising to be the solution of all problems while in reality they certainly are not. It is better to look for and identify actual design problems and see what technological means are available to solve them. So far use of vector tiles seem to primarily have lead to the following effects: - improved tile serving efficiency - a larger bandwidth of style variations - tighter contraints in basic styling decisions beyond what is already imposed by the OSM data model In short: from a design perspective vector tiles so far brought more variety in map styles but they ultimately all look very similar beyond superficial aspects. Nearly all of the more unusual maps that currently exist are not vector tile based. [1] http://blog.imagico.de/map-design-economics/ [2] https://github.com/gravitystorm/openstreetmap-carto [3] For another example of where wikimedia maps fails miserably here see: https://maps.wikimedia.org/#18/47.99579/7.85194 http://www.openstreetmap.org/#map=18/47.99579/7.85194 -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] SRTM GL1 contour and hillshade available
On Monday 21 September 2015, Gmail wrote: > Due to the use of ASTER and CGIAR, for non commercial use, of course. ASTER is not limited to non-commercial. Your use of the CGIAR data seems limited to areas with neither ASTER nor SRTM data so you could get rid of it quite easily without significant losses. Your method by the way simply overlays the different data sets which will lead to significant steps at the edges due to differences in height calibration. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Extracting data for testing low scale rendering
On Monday 13 July 2015, Daniel Koć wrote: I'd like to know how to extract from planet.osm only the features visible on 1-6 or even 1-7 levels in osm-carto (default OSM style). Mateusz Konieczny has already tried to do something like this, but hit the wall at the moment: http://forum.openstreetmap.org/viewtopic.php?pid=514998#p514998 If anyone can help us with it in any way, we would appreciate it a lot. It would be very useful for massive testing of proposed highway rendering changes and general low scale rendering problems. With the default style for z=1-5 this should be fairly easy, for z=6 and above the volume of data required will increase massively since you need all water areas. Filtering for the following will probably get you all you need for z=1-5 and will not get you too much you do not need: admin_level=* highway=motorway, highway=trunk place=city, capital=* -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM for Retina and high-pixel density devices
On Wednesday 28 January 2015, Michael Meier wrote: [...] However, in addition to the problem of limited resources, I don't think the style is ready yet. Though there has been a lot of progress lately, the openstreetmap-carto-style has many symbols, and many still aren't SVG and thus are not HD ready - they look really ugly in HD tiles. Non-scalable elements in the style are only part of the problem. In general nearly all maps rely on the rasterization to perform necessary spatial frequency filtering or in other words: they cannot change output resolution independent from the level of detail. If you for example compare https://c.osm.rrze.fau.de/osmhd/6/35/18.png https://c.osm.rrze.fau.de/tiles/7/71/36.png or https://c.osm.rrze.fau.de/osmhd/8/133/85.png https://a.osm.rrze.fau.de/tiles/9/267/171.png or https://a.osm.rrze.fau.de/osmhd/8/132/75.png https://c.osm.rrze.fau.de/tiles/9/265/151.png You can see that apart from the styling differences between the zoom levels and the differences in label sizes these look identical. But for good readability on high resolution devices you do not want the additional details which are too small to be properly read leading to mushy results, you want better acuity with the same amount of detail. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] OSM based replacement for Natural Earth national boundaries
Most OpenStreetMap maps use the Natural Earth boundary data for showing national boundaries at the lowest zoom levels. On http://www.imagico.de/map/boundaries_download_en.php you can now find a replacement for that based on OSM data and simplified specifically for the common web map zoom levels. Also the maritime boundaries are trimmed to where they are actually useful. Some more background information is available on http://blog.imagico.de/rendering-boundaries/ and a simple map showing the data on http://maps.imagico.de/#map=3/55.0/0.0r=coastlineo=7ui=0 -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Release openstreetmap-carto v2.25.0
On Thursday 11 December 2014, Andy Allan wrote: From that it would be a reasonable conclusion to think that I'm being a bottleneck on the development - well, perhaps I am. I don't think there is a serious bottleneck in the merging of changes to go active in the main map. The problem is not bandwidth but latency. This might be to some part due to the order in which you work through changes. You could try doing it the opposite way, working on the oldest changes first. This would ensure things do not get stuck at the bottom of the pile for a long time (there are quite a few pull requests active at the moment for example that have been waiting for at least the past three releases). So I pose a question that's most pressing on my mind - should the other maintainers be merging PRs without me reviewing them first? Will this lead to a better result? Well - the job of the gatekeeper of the rolled out style is essentially an administrative task and having an own opinion on styling is probaly more a disadvantage for this. ;-) Another side of the matter i already pointed out occasionally is that everyone has specific priorities and no matter how well meaning you are these go into decisions when making judgements about merging a change or not. No matter how you put it the style still has a strong European and even British focus which is an issue with OSM being an international project IMO. Mainly for these reasons i would much support opening up the active style for more people to immediately commit to although i also see the risk of a significant loss in style coherence and ultimately map quality over time with that approach. The dominance of technical considerations over design aspects and cartographical arguments is also likely an issue. Having a two stage process with a separate map for testing before deployment in the main map would really help here i think. The lack of a central test environment could be mitigated by trying to get a larger number of people from different regions to test changes in their local test environment on areas they are familiar with and weigh in with their assessment of the changes. But people will only be motivated to do this if they see their contributions have an effect. And i would like to emphasize again the need for a more systematic followup on rolled out changes - quite independent from the exact mode of operation. For a developer it is always more interesting to work on something new but it would be good to establish that the work on a style change does not end with it being deployed. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Release openstreetmap-carto v2.25.0
On Thursday 11 December 2014, Matthijs Melissen wrote: Andy mentions that he thinks that the pull requests he has merged are not the most important ones. [...] I think Andy's point was more that considering the amount of work that went into it the overall state of the style is not really that much better than it was two years ago. Some things are improved but these improvements also introduced a lot of new issues. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Release openstreetmap-carto v2.25.0
On Wednesday 10 December 2014, Matthijs Melissen wrote: Today, v2.25.0 of the openstreetmap-carto stylesheet has been released [...] First of all these remarks don't have much to do with the changes of this release in particular, it is mostly that this has confirmed a number of observations i have made in the last months. It seems to me that with the more active development of the style in the past year or so with frequent changes with high visual impact the mode of development of the style as it is currently practiced has reached a limit where it becomes very difficult to develop well composed changes to the style at all. I am somewhat reluctant to bring up this problem since i think the more active development is a good thing in total and i don't want to badmouth this. I have not done a precise statistical analysis but the time for a change from being designed to actually being deployed is fairly long, usually there are at least 1-2 releases in between, often several months. This does not only mean changes need to be adapted to a changed codebase, it also means that the basis in terms of visual design is often changing significantly while a change is waiting to be merged. As a result most changes which are deployed in recent releases are not really very well aimed. They are meant to address certain issues but usually introduce at least as many new issues. Other changes are then made later to address some of these with a similar result. To some extent this is inevitable due to the complex interactions within the style but it is also largely a result of 'shooting at a moving target'. I don't have a real solution for this problem but there are a number of things that IMO could much improve the situation: (1) Systematic followups on the success and side effects of changes. The only case where i remember this happening is with the landcover labeling [1] but note that the majority of the issues pointed out there are not yet addressed now, two month later. IMO development ressources should in most cases focus on bringing a change to a satisfying overall conclusion before working on new changes. For most of the recent high impact changes this is not the case. (2) Stricter queuing of changes for deployment. The manner in which pull requests deemed ready for merging are selected for deployment seems quite random at least to the casual observer. Some are merged within days, others take several months (the oldest one currently seems to be from July [2]) Followups on previous changes should probably be given priority. (3) A test environment with worldwide data. I know this is not a new suggestion and mostly a problem of ressources. What would already help, especially for systematic followups on changes and would probably require much less ressources is a frozen tile cache that allows better before-after comparisons for changes already deployed. [1] https://github.com/gravitystorm/openstreetmap-carto/pull/941#issuecomment-58766694 [2] https://github.com/gravitystorm/openstreetmap-carto/pull/725 -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Nominatim 2.3 release announcement
On Sunday 05 October 2014, Sarah Hoffmann wrote: There are two new features that are most notable for users of the search engine: there is now support for waterway relations, which improves searching for rivers, [...] Nice. It seems though they are reported in the same way in the results lists as river lines with 'River' which makes them hard to identify. In principle i would have four major classes for water features: * standing water areas (natural=water + water!=river/channel) * flowing water areas (waterway=riverbank or natural=water + water=river/channel) * waterways (waterway tag but not waterway=riverbank) * waterway relations -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Nominatim 2.3 release announcement
On Sunday 05 October 2014, Sarah Hoffmann wrote: I figured that intention from the results i saw for various rivers. Still it would be good i think to distinguish between waterway relations and ways in the results. For larger rivers there are very often individual ways missing in the relation, most frequently side streams of course, so you rarely only get a sigle result with type=waterway:river. These are mapping errors, aren't they? As a rule, Nominatim does not try to cover such errors, precisely so that mappers can easily spot them and fix them. That is exactly my point, if you can distinguish between the waterway relation and such stray individual waterways this is much easier. But of course if the relation turns up reliably at the first place in the list this is much less critical. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Simplifying glacier rendering in Antarctica
Since hardly any OSM map correctly renders glaciers in Antarctica at the moment despite the current tagging scheme [1] being used for more than a year now i have put together a processing script to simplify this for map developers. It allows you to render the Antarctic ice the same way as normal glaciers despite the fact it is mapped differently. Some more background information can be found in a blog post [2], the script on github [3] and the resulting shapefiles on [4]. [1] http://wiki.openstreetmap.org/wiki/Antarctica/Tagging [2] http://blog.imagico.de/easier-rendering-of-the-antarctic-land-ice/ [3] https://github.com/imagico/icesheet_proc [4] http://www.imagico.de/map/icesheet_download_en.php -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] OSM based replacement for builtup_area.shp in standard style
Hello Everyone, it has been pointed out occasionally (see for example [1]) that current use of external data sources in the OSM standard style is somewhat ugly. Back in April i made some tests how to process OSM data to generate simplified polygons for urban landuse/builtup areas to be used as replacement for builtup_area.shp (which is the most significant outside source currently in the standard style). This has not been mentioned here, you can read the details on [2]. I have now improved this technique sufficiently to process the whole planet and you can find a shapefile for testing on [3]. This is of course a fairly ressource intensive process with all building and road data in the database being used as basis. But in principle it would be possible to run this with periodic updates in a similar way as currently done for the coastline. Everyone feel free to try out these polygons in your maps. I have not tried this myself in the standard style yet, it is likely that adaptations in either the generalization process or the style (or both) are necessary for best results. Comments and examples on that are welcome. Greetings, Christoph [1] http://wiki.openstreetmap.org/wiki/User:Mrwojo/builtup_area.shp [2] http://www.imagico.de/map/osm_builtup_en.php [3] http://www.imagico.de/map/osm_builtup_en.php#download -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM based replacement for builtup_area.shp in standard style
On Tuesday 03 June 2014, Martin Koppenhoefer wrote: As a sidenote I wanted to point out that there are currently already 281547 ways (I suspect most of them to represent areas) tagged with place=* (not much compared to a total of 3.1M places in OSM) which could serve as an alternative to your (preprocessing intensive) process if more mappers could be convinced for this concept of mapping settlement extensions. Note many of those are islands, see: http://taginfo.openstreetmap.org/keys/place?filter=ways#values And the others - like Frederik said - are most likely not really describing the actual builtup area. The main problems of your approach --- it is still only guessing and the derived dataset might have a high probability to identify built-up areas but it will not in all cases be able to tell which settlement an area belongs to and it is a ressource intensive process that reasonably cannot be done on the fly --- could be overcome with explicit place polygons. On the downside it will of course take us quite some time to manually map all those settlements, and it is not even clear if people are interested in general to map this kind of feature (in addition to landuse etc.). If the main style picked this up and rendered areas with place=(city,town,village,hamlet,isolated_dwelling) as built-up-area this would surely help promoting the cause ;-) I think i'd disagree here. place=city|town|... is simply not a well defined area. You might think you can draw a line around any settlement in the world but there is no way to say where exactly this line is to be in a verifiable way. So if you start mapping this everyone will draw it differently and nothing is gained in the end. What i do is essentially defining the builtup area as the area where the density of roads and buildings exceeds a certain limit. This in itself is no guesswork at all - the base data is of course very variable and there are a lot of subjective choices involved in the process that follows. You might feel inclined to adapt this density measurement as a rule for manually mapping areas with place=* but keep in mind measuring object density is always specific to the scale you look at and there is no natural scale of settlements you can base this on. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Polar Projection question...
On Wednesday 02 April 2014, David Van Gilst wrote: I'm interested in showing Openstreetmap data in a polar projection for use in research near the north and south poles. The wiki ( http://wiki.openstreetmap.org/wiki/Antarctica/Creating_a_map) indicates that there was a set of instructions for doing this at https://github.com/MaZderMind/mapnik-stylesheets and an example at http://polar.openstreetmap.de/antarctica/, but both of these links seem to be broken. Links need to be updated: https://github.com/MaZderMind/mapnik-stylesheets-polar http://polar.openstreetmap.de/ I you are going to adapt the standard mapnik style you also might want to read http://wiki.openstreetmap.org/wiki/Polar_Regions_Rendering_Issues Would be great to see more polar maps by the way. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New OSM Test Data Repository
On Friday 21 February 2014, Jochen Topf wrote: I have started to work on a common OSM test data repository that can be used to test all sorts of software that works with OSM data. See my blog post at http://blog.jochentopf.com/2014-02-21-osm-test-data-repository.html for details. Nice. Of course I could not resist creating another test: http://www.imagico.de/files/data.osm where the result would be 'valid but invalid in plate carree or web mercator'. 'invalid but valid in plate carree and web mercator' is also possible of course, just like all other combinations. ;-) In case you wonder about the coordinates - that's (50 900) and (55 905) in UTM zone 47. And of course this does not work near the equator. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New OSM Test Data Repository
On Friday 21 February 2014, you wrote: I am not sure yet what exactly the valid and invalid mean and whether we need different outcomes. For the moment valid means more something like: The data looks good, could exist in a real database, and everybody would agree on how to interpret it. While invalid is everything else. I suppose there are many cases where it would make sense to allow an assessment somewhere between 'fully valid no matter how hard you look at it' and 'invalid beyond any consistent interpretation of the data'. It might make sense to say robust programs should be able to deal with certain data while data producing progams should avoid generating such data. I haven't figured out all the details yet, for the moment I am trying to get the ball rolling, get started writing test data and tests programs that use them. We'll figure out the details along the way. It will be interesting to see results for multipolygons in various programs. I have long suspected the JOSM validator is fairly 'loose' in this regard (which is a bad thing - validation in the editor should ideally be the strictest check of all) but never got around to pinning this down. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] [OSM-talk] Assembling imagery: to mosaic or not
On Friday 03 January 2014, Paul Norman wrote: [...] The two options are to add a number of discrete sources, or to add a layer mosaicking them together, and displaying what the best source is at any one point. I think this is the primary issue you need to consider. If you are sure you know what the 'best source' is at any point (which in the most general case requires in depth knowledge of the area and of the image data in question) mosaicking might be the best solution (but can be a real lot of work). If on the other hand you plan to just layer the image sources by formal quality measures (like resolution and age) to create the mosaic you will most likely end up with less than optimal quality in some areas. In addition there is of course a significant value of having multiple images available in itself since it allows the mapper to get an idea on how variable images can be. A frequent mistake in common practice armchair mapping is that people regard the image as the truth rather than a most likely inaccurate depiction of the truth. Having several images available can help here. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] SotM-EU 2014 - Call for Presentations
Hello everyone, i am pleased to announce the Call for Presentations for the SOTM-EU 2014 in June next year. Please communicate this to anyone who might might be interested in presenting something at the conference. The full text and links can also be found at the conference website (http://sotm-eu.org/en/pages/cfp) Call for Presentations Now as the cold time of the year is approaching many of you are spending the long and dark evenings working on some OpenStreetMap related projects like some tool, some mapping project or some creative use of the OpenStreetMap data. Maybe you think it would be nice to share your work with a larger European community next summer? Then we invite you to submit a presentation to the State of the Map Europe (SOTM-EU) conference in Karlsruhe, Germany in June next year. Where and when The SOTM-EU 2014 will take place in Karlsruhe from June 13-15, 2014. The conference will be hosted at Hochschule Karlsruhe. On Friday and Saturday there will be talks, and Sunday will be a hack day for practical work and discussion. Your presentation We would like SOTM-EU to be a platform for OpenStreetMap communities from across Europe, as well as for geodata professionals, cartographers and researchers, for sharing experiences, stories and knowledge around the OpenStreetMap project. In case * you are developing a tool related to OpenStreetMap for mapping, data processing, visualization or other applications * you have some mapping project, ideas on tagging or an innovative mapping technique * you are using OpenStreetMap data in a business project * you are doing research based on OpenStreetMap data * you are working on something else related to OpenStreetMap or that will be of interest for the community we invite you to submit a presentation proposal to the SOTM-EU programme committee. Are you involved with a local project anywhere in Europe? SOTM-EU is a great opportunity to present it to a Europe-wide audience. Regular talks will be 20 minutes long with five minutes for discussion. In addition we will offer the opportunity for shorter five minute lightning talks. You can submit both types of presentation in advance on the website. However, for the lightning talks there will also be the possibility of spontaneous registration at the conference. The conference language is English. Your submissions will be reviewed by a programme committee consisting of OpenStreetMap community members from various parts of Europe as well from the Hochschule Karlsruhe. Talk submission Talks can be submitted on the SOTM-EU web site (http://sotm-eu.org/en/presentations/new) where you will also find more information and updates on the conference. You should submit by February 28. We're looking forward to seeing you in Karlsruhe in June! On behalf of the SOTM-EU 2014 program committee, -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Default map style on osm.org
On Thursday 29 August 2013, Kai Krueger wrote: [...] It really is a rather powerful reward to see your effort being directly incorporated into the in production map within minutes which can and _is_ viewed by millions of people. This i can very much agree with. My idea of a separate data verification map was not meant as a purely technical display without any concern for practical usability. I could also phrase it differently: Having two different but both real time updated maps would help communicating to the mapper that the purpose of the data is not just to produce a single map. Seeing how the same features and tags affect the result in the two maps in different ways could make people more aware of this. And for this purpose it would not matter too much in what ways the two map styles differ. Imho, the solution of cleaning up the low zoom tiles to look good for general use and then increasing or at least maintaining the detail on z18, z19 (and z20) is perhaps the best compromise we can achieve. One problem with having a style with strong differences between zoom levels is that the map scale is strongly varying across the map. z18 at the equator is the same as z15 in northern Canada. So when peaks for example get displayed starting with z11 and get labels at z13 this works nicely at low to moderate latitudes but is pointless at high latitudes (once you see the peak you usually see nothing but the peak) meaning the reward of seeing your efforts incorporated into the production map is much less in those areas. Greetings, -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Default map style on osm.org
On Wednesday 28 August 2013, Andy Allan wrote: I think that being pretty is not the goal of the default map style at all. The primary goal of the default style is to expose as much of the OSM data as possible. That's widely held opinion, but there's an equally sized opinion that would heartily disagree. That's probably to some extent because the main map serves a double purpose: For the user of the map and for the mapper. Both these uses have their own requirements which are not always identical. It always amazes me how much the map style influences the mapping practice, not only by obvious tagging for the renderer but also in the form of subtle priorities. Having two separate styles - a 'data verification' style and a 'presentation style' would help reducing this effect. Another problem is that a consistent and asthetic map design is difficult to achieve with a large number of people working on it together. On the other hand with few people making the design decisions there is inevitably a bias in decisions what is prominently visible and what not (which as said above has a significant influence back on the mapping practice). Greetings, -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Coastline, lakes, rivers
On Tuesday 16 April 2013, Pieren wrote: I'm not sure that the New tagging of riverbanks has been really adopted, excepted by the 16 who approved it on the wiki (is it in editors presets ?). Actually the situation is even worse - current practice in tagging water areas is a wild mix of applying waterway=riverbank and/or natural=water to multipolygons, member ways and individual ways not part of a multipolygon. I doubt this is going to change as long as the common map render style simply renders everything with either of these tags in plain blue. Cases like the linked one are situations where even this basic handling fails (mostly due to broken multipolygons or inner rings tagged waterway=riverbank) And that's only for the water areas - i will refrain from getting into the water line features here. My conclusion on this matter is that the only thing you can currently do with the OSM water data is to render it - or in other words: treat it as mere information 'here is water'. And even that can be unreliable since intermittent=yes/water=intermittent are not consistently applied (not astonishing considering they are not rendered either). All information beyond that can - if at all - only be relied on locally if consistency has been maintained by local mappers or if most of the data is based on a well structured import. Greetings, Christoph -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Zeroing or Extending a Hillshade-Tiff
On Wednesday 13 March 2013, Jochen Topf wrote: Another option would be to split up the long straight lines resulting from the splitting into shorter pieces. If the pieces are short enough, it should be okay. Maybe there is some tool around that can do that easily? I don't want to put more and more of these things into OSMCoastline, because they are not really related to coastlines, but general issues with large objects. Not quite actually - in case of splitting the problem does not stem from the long segments but from the fact that you split one polygon but do not split the edge of the adjacent one at the same time. If you'd do that the edges could have any length, they would still fit after the reprojection (although you could get intersections between the actual coastline and the splitting lines). -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Coastline changes Antarctica
On Tuesday 12 March 2013, Jochen Topf wrote: OSMCoastline first generates the land polygons, them splits them, then creates the water polygons as inverse from the lang polygons. If the land polygons are broken, so are the water polygons. So I don't think you can reduce the chance of breakage that way. Possibly a better way to prevent large errors in the coastline would be to maintain a simplified version known to be free of large errors and use it to create bounds for the real up-to-date coastline. As you said the quality of the OSM coastline is quite good these days globally so at least on the land side it can be said with very high confidence that a few kilometers from the OSM coastline there is only land. So using a correspondingly buffered old and verified land polygon in addition to the real one would prevent large errors either by accident or by vandalism and would not hamper real improvements of the data. I don't know how often you discard the new data based on comparison with previous day data but the above would be an alternative to that. Greetings, -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Coastline changes Antarctica
On Monday 11 March 2013, Jochen Topf wrote: I have described the details in this blog post: http://blog.jochentopf.com/2013-03-11-state-of-the-osm-coastline.html Have a look at the chapter about Antarctica. (The other stuff might be interesting, too, though.) Concerning the 180-degree-line - for that a solution that would allow to process the coastline in a pacific centered coordinate system (or with any other cutting meridian) would be best. And the obvious way for that is to simply draw the coastline ways indiscriminately across the line. I think the API and database would have no problem with that but OSMCoastline would need to be able to close the coastline in arbitrary configurations at the edges and the editors would need to support it. I suppose especially the latter means it won't happen but still it would be the most elegant solution i think. Note doing the same also for other geometries would be problematic since on a sphere you have no reliable way to decide about inside and outside of a (large) polygon without the orientation convention of the coastlines. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Coastline changes Antarctica
On Monday 11 March 2013, Paul Norman wrote: OSM data is in WGS84 and the line going from -180 to +180 goes across the entire world in this projection. This is how osm2pgsql interprets it and I believe most other tools. I know - but i was just musing about coastlines in an isolated manner. And since coastlines are not rendered from the database but from a separate shapefile this would not be an issue. But of course coastline ways can have other tags or be part of relations and then will possibly be rendered from the database. For this to work it would need to be ensured that the ways crossing the line are tagged natural=coastline only and are not part of a relation. And rather than enforcing such a complex rule it would probably be much easier to simply introduce a new type of relation containing the two corresponding nodes on both sides or even simpler: have OSMCoastline determine the matching nodes on its own and make the connection itself (which is exactly what Jochen did - although at the moment only for the special case of the southmost line). -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Zeroing or Extending a Hillshade-Tiff
On Thursday 07 March 2013, Peter Körner wrote: b) extend the image with value 181 at the borders to fit the size of the image i'm going to render My question is: what way would you choose and wich tools could be used to go that way? for b: gdalwarp -dstnodata 181 Note it is usually best to only hillshade the land and not the water so if you draw the water above the hillshade you won't have this edge problem (it is of course even better to modify the DEM before shading to match the coastline but that's quite a bit more complex). I am currently preparing rendering polar maps myself so if you need hillshade images for the lower zoom levels i could provide them - for zoom=2 with 6000km map area (i think that's what you are using as well): http://www.imagico.de/files/gshade_2_0_0.tif.bz2 Greetings, -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Coastline generalization tool and data
On Tuesday 19 February 2013, Michal Migurski wrote: Christoph, this is so cool. Excellent work! Thanks. I've been hammering away at an OSM generalization effort of my own, focused on the global preparation of simplified linework for major roads and route relations: http://www.openstreetmap.us/~migurski/streets-and-routes/ For generalizing roads I think there are four major components: 1. removing smaller roads where there is no room to show them properly 2. summarize dense networks of roads of similar size 3. move roads to be able to distinctly show them without overlap with other features or themselves 4. simplify the individual road segments From your description I have the impression that a major part of your effort is in 2. although I could not clearly identify cases of summarization in your examples. From the samples it seems your technique sometimes produces gaps between roads which are in fact connected. Apart from that it looks quite good - for curved roads the simplification seems a bit strong unless you mean to use some kind of smooth spline rendering. By the way for the coastlines I also looked into the idea of using vector skeletons but I soon realized this would be prohibitively slow on a global scale. From the pure program design point this would still be an interesting idea. I did find one weird part of your data, around Boston where the peninsula appears detached from the mainland. That's another canal tagged as coastline: http://www.openstreetmap.org/browse/way/22721484 -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Coastline generalization tool and data
On Tuesday 19 February 2013, Michal Migurski wrote: No rendering at all. The intent of this work is to provide labeling hints, so you'd still render the regular roads from OSM and use this dataset to place labels over them. We did this for the US when I worked on the Stamen Terrain map, visible here: http://maps.stamen.com/terrain/#13/37.7786/-122.4408 So rendering of the roads themselves is based on the original OSM data. This explains the gaps - for labeling they do not matter. So a re-tag would fix this? How long does the data take to re-run? I added it to the list: http://wiki.openstreetmap.org/wiki/User:Imagico/coastline_problems_for_generalization The run for the whole earth takes about 6 hours (just the generalization, osmcoastline not included). Greetings, -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Coastline generalization tool and data
On Thursday 14 February 2013, Stefan Ziegler wrote: If you look at the island Usedom in the Baltic Sea (Northeast Germany, near Poland): it is connected to mainland. Usedom is really difficult with small stripes of water between island and mainland. That's right. Since it is a raster based process only structures large enough to be resolved in the raster will be taken into account. The Peenestrom misses the cutoff by a small distance here. A possible solution would be to distinctly tag such waterways, in this case: http://www.openstreetmap.org/browse/way/102832340 with something like waterway=strait and then consider those in addition to the coastline itself during generalization. Usedom however is currently no real island in OSM anyway since only one side is tagged as coastline: http://www.openstreetmap.org/browse/way/174336199 http://www.openstreetmap.org/browse/way/7724629 -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Coastline generalization tool and data
in the demo map. In my opinion, it is hard to believe that this tool/technology may compete with other similar technologies for creating zoom-levels in digital cartography. Yet, it may be of certain value in the field of illustrations among many other options. About the coastlines you use for comparison your write they are created using a 'proprietary vector smoothing and data reduction algorithm' - it would be important to know what exactly this does of course. And as said, my technique is not about data reduction so the assessment based on data sizes is somewhat missing the point here. But of course there is much room for improvements with my technique and of course generalization is always a lot about subjective choices so it is perfectly fine to prefer a different approach. If you can point out disadvantages of my files when compared to alternative techniques in actual appearance in rendering that would be very useful. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Coastline generalization tool and data
Hello everyone, some time ago i introduced a technique to generalize OSM coastline data for better readable map rendering [1] on talk-de and i now release the tool used for this [2] as well as a set of generalized coastline files for zoom levels 1 to 8 [3]. There is also a demo map [4] - this uses a slightly different coastline than the one for download (in particular Antarctica data is replaced by a different source). Feedback on both the tool and the files is welcome. It would be interesting to see how use of this would look like in the standard mapnik rendering, especially how much mismatch the moving of the coastline during generalization causes and how troublesome this is when viewing the map. You might notice various strange forms in the coastline at some places, most of them are caused by features being tagged as coastline which in fact should probably not be. I put up a wiki page [5] to collect these problems. If you find more or fix them feel free to edit that. [1] http://www.imagico.de/map/coastline_en.php [2] http://www.imagico.de/coastline_gen/index.php [3] http://www.imagico.de/map/coastline_download_en.php [4] http://www.imagico.de/map/map_en.php [5] http://wiki.openstreetmap.org/wiki/User:Imagico/coastline_problems_for_generalization Greetings, -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
