Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
On 2020-05-26 06:24, Yves wrote: Le 26 mai 2020 03:58:50 GMT+02:00, Paul Norman via dev a écrit : Performance with normal basemaps and small stylesheets should be acceptable. Reading this, I don't think I understand. However it makes me think of a raster basemap plus a vector overlay for labels to solve internationalization issue. How lightweight such an overlay could be? You might run into problems when you have to place labels very exactly. For low zoom that is not an issue, if a place name is a bit left or right than that's no problem, but when you get to high zoom levels and want to place lets say street names, than you have to know where to place it and that means you have to know how the raster tile behind it is rendered. Regards, Maarten ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
It seems like the biggest problem with the vector rendering, for performance and cartography, is rendering areas such as water and landcover, while the big benefit is with text labels, SVG icons, and linear features. Is it possible to continue rendering the landcover and water areas from a pre-rendered raster file with this toolchain? – Joseph Eisenberg On Mon, May 25, 2020 at 9:27 PM Yves wrote: > > > Le 26 mai 2020 03:58:50 GMT+02:00, Paul Norman via dev < > dev@openstreetmap.org> a écrit : > > >Performance with normal basemaps and small stylesheets should be > >acceptable. > > Reading this, I don't think I understand. However it makes me think of a > raster basemap plus a vector overlay for labels to solve > internationalization issue. How lightweight such an overlay could be? > Yves > > ___ > dev mailing list > dev@openstreetmap.org > https://lists.openstreetmap.org/listinfo/dev > ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
Le 26 mai 2020 03:58:50 GMT+02:00, Paul Norman via dev a écrit : >Performance with normal basemaps and small stylesheets should be >acceptable. Reading this, I don't think I understand. However it makes me think of a raster basemap plus a vector overlay for labels to solve internationalization issue. How lightweight such an overlay could be? Yves ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
On 2020-05-25 9:25 a.m., Jason Remillard wrote: Hi Paul, The Mapbox GL styles are very low level, and are like the mapnik XML input files. The Mapbox GL editing tools available don't seem to provide abstractions over the underlying specification. Yes, the specification and tools are quite clearly not designed for human readability. There has been some work at languages that compile to Mapbox GL styles, but nothing that supports expressions yet. Also, rewriting from scratch the main OpenStreetMap Carto style sheet is not appealing. There has been a lot of work put into it, and whatever happens next, everybody is going to be living with it for a long time. I'm a maintainer of OpenStreetMap Carto so I'm familiar with its development. I don't think this is a problem. It's not from scratch, it's implementing existing cartography in a new language. I have done some projects with Mapbox GL with client side rendering, and the performance on my tiny style sheet was unacceptable on mobile. Having the main style sheet use client side rendering will need to wait for newer and faster phones. Performance with normal basemaps and small stylesheets should be acceptable. The question is, how do you get normal sized tiles at low zooms from OSM data, how big will the tiles be at high zooms, and will the stylesheet be too complex. Going back to the problems. Long term it is clear that it would be a good thing if OSM had a client side rendering option. Also, right now, the existing tile servers are overloaded. A victim of the projects ongoing success and CartoCSS baked in performance issues. No, the issues have been with the tile CDN, not the tile servers. CartoCSS has never been a performance issue. ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
Hi Paul, The Mapbox GL styles are very low level, and are like the mapnik XML input files. The Mapbox GL editing tools available don't seem to provide abstractions over the underlying specification. Also, rewriting from scratch the main OpenStreetMap Carto style sheet is not appealing. There has been a lot of work put into it, and whatever happens next, everybody is going to be living with it for a long time. I have done some projects with Mapbox GL with client side rendering, and the performance on my tiny style sheet was unacceptable on mobile. Having the main style sheet use client side rendering will need to wait for newer and faster phones. Going back to the problems. Long term it is clear that it would be a good thing if OSM had a client side rendering option. Also, right now, the existing tile servers are overloaded. A victim of the projects ongoing success and CartoCSS baked in performance issues. My ambitious thought is that the equivalent of the CartoCSS needs to be invented to provide a human friendly interface to both the Mapbox GL and mapnik rendering engines. The language should be at a higher level of abstraction. For example, it should just handle the "layer" key. To broaden the pool of contributors, it should directly use the OSM key and values, and not a foreign intermediate SQL database or tile schema. Lastly, accept that the language is just for rendering normal maps, and assume things like labels are always on top. I am not a fan of CartoCSS, but in theory, we could write a new CartoCSS processor to target Mapbox GL to save the main style sheet. Jason ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
On 25/05/2020 01:34, Paul Norman via dev wrote: Landuse, vegetation, and other natural features are not rendered until zoom 7. This is the scale of OpenStreetMap Carto zoom 8, and these features first appear at zoom 5. There are numerous problems with unprocessed OpenStreetMap data at these scales. OpenStreetMap Carto gets a result that looks acceptable but is poor at conveying information by tweaking Mapnik image rasterizing options. I'm looking for better options here involving preprocessed data, but haven't found any. I think we will need fairly heavy pre-processing (filtering and simplification/generalization) of data before they hit the tile and the client. Carto is doing some of it already, even though data stay on the server side at all times. With the client-side rendering this is much more important for three reason - real time rendering, network traffic, limited resources on client side. Some ideas: - Tiles should contains only objects to be rendered, so all the filtering currently done in Carto should be applied at the tile construction time. - Currently Carto produces a lot of data that are later filtered out in Mapnik (via spatial occlusion) - this also should be brought back to a tiling script. - Tiles should be split into layers (similar to layers in Carto). Then the user would have an option to select layers they are interested in (think language versions, POIs). With more, smaller tiles there is also a potential for distributing the server side load more uniformly. - For z5-z12 we will need some kind of polygon generalization/simplification. For example, at z7 I can see all landuses in England, with some towns occupying only a couple of px but but still being comprised of hundreds of landuse polygons. I'm not talking about anything fancy, but merging anything sub-1px into "pixel rectangles" could go a long way. (Edit: I see you are already doing it - I can see some simplified landuse polygons when zooming in further). The technology choices are designed to be suitable for a replacement for tile.osm.org. This means minutely updates, high traffic, high reliability, and multiple servers. Do we need minutely updates for vector tiles? Sure, that's a nice feature to have but with client-side rendering any performance issues will be directly exposed to the users. And performance is a big issue in most client-side rendering solutions. If we can keep the current raster backend with minutely updates for mapping feedback, I have no problem with hourly or daily updates for the vector version. We would need a raster option for browsers/devices not supporting WebGL anyway. ndrw ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
On Monday 25 May 2020, Paul Norman via dev wrote: > https://github.com/pnorman/openstreetmap-cartographic. In general this looks like a good approach to evaluate where the practical issues are without being bogged down by bloated legacy tools (well - except for the client side of course). > Landuse, vegetation, and other natural features are not rendered > until zoom 7. This is the scale of OpenStreetMap Carto zoom 8, and > these features first appear at zoom 5. There are numerous problems > with unprocessed OpenStreetMap data at these scales. OpenStreetMap > Carto gets a result that looks acceptable but is poor at conveying > information by tweaking Mapnik image rasterizing options. I'm looking > for better options here involving preprocessed data, but haven't > found any. Yes, i think this style as is is a good demonstration that the current approach used universally in the industry (and that is used in ST_AsMVT() as well) of performing per feature lossy vector geometry compression (usually in combination with way_area filtering) is a dead end. You can see that nicely if you overzoom the demo map: http://imagico.de/files/screenshot_cartographic.png http://imagico.de/files/screenshot_carto.png This creates a lot more artefacts and noise in the rendering than the AGG rendering issues in Mapnik typically do even at the resolution it is intended for - including counterproductively adding a lot of high frequency noise in many cases. > Often forgotten is the development requirements. The style needs to > support multiple developers working on similar areas, git merge > conflicts while maintaining an easy development workflow. I'm still > figuring this out. Mapbox GL styles are written in JSON and most of > the tools overwrite any formatting. This means there's no way to add > comments to lines of codes. Comments are a requirement for a style > like this, so I'm investigating minimal pre-processing options. The > downside to this will make it harder to use with existing GUI editors > like Fresco or Maputnik. I think this is a point that cannot be overstated. Mapbox GL JSON (or FWIW JSON in general) in contrast to CartoCSS is a software developer centered format and not a cartographer centered format. Many mainstream low sophistication users of client side vector tile frameworks focus exclusively on interactive editors and not working on the code level - which as you explained is not an option for a cooperative community project or for styles with a higher level of sophistication. Just re-casting the JSON data structures in a different format easier to edit by hand, suitable for comments and leading to well readable diffs is IMO not enough for a truly cartographer centered approach. But this is definitely a hard problem because there are so many in parts opposite requirements. Having a reference renderer that does not build on an extremely complex and platform dependent framework (a.k.a. web browser) would be important for style development - i think Joseph also hinted in that direction. It would also be important for things like automated tests etc. One other thing to keep in mind - OSM-Carto uses polygon fill patterns a lot for differentiating different types of polygons. While it is in principle possible to show fill patterns in continuous zooming in an ergonomic fashion i have so far never seen maps in the wild doing that. This would be something to consider if you want to create a style similar in the richness of landcover rendering to OSM-Carto. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
On 2020-05-25 09:59, Paul Norman via dev wrote: On 2020-05-24 10:26 p.m., Joseph Eisenberg wrote: Thank you for making this, it looks like a lot of work! These client-side vector tiles at z8? (https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html) My laptop (2015 Macbook Pro, 16 GB RAM, 2.2 GHz Intel Core i7) appears to use some effort to show areas with a large amount of data, For example if I view England + Wales on z8 and then move over to the Netherlands and then to Germany, at z7 and z8, CPU usage goes up to >100% for a time, and there is a noticeable delay. Perhaps part of this is a delay in serving the tiles? Some of the tiles are decently large so could be slow to download, but based on your CPU it's not that. More likely it's the high number of vegetation and landuse features at those zooms. There are some things I might be able to do reduce the feature count. I'm not noticing much slowness on my computer, but having a lot of cores does help here I think. I won't notice it when my browser is using one core (out of six) at 100%. I did get some delay when zooming in, at first I thought "oh, only motorways, that's not much" but after some 10 seconds I got landuse and other roads. Still, it looks to be a very simplified subset of the complete data. So I'd be interested to see how this works on a full dataset. Regards, Maarten ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto
On 2020-05-24 10:26 p.m., Joseph Eisenberg wrote: Thank you for making this, it looks like a lot of work! These client-side vector tiles at z8? (https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html) My laptop (2015 Macbook Pro, 16 GB RAM, 2.2 GHz Intel Core i7) appears to use some effort to show areas with a large amount of data, For example if I view England + Wales on z8 and then move over to the Netherlands and then to Germany, at z7 and z8, CPU usage goes up to >100% for a time, and there is a noticeable delay. Perhaps part of this is a delay in serving the tiles? Some of the tiles are decently large so could be slow to download, but based on your CPU it's not that. More likely it's the high number of vegetation and landuse features at those zooms. There are some things I might be able to do reduce the feature count. It would be nice to see a demonstration of this applied server-side to produce raster tiles. Are there any samples of that yet? There are a few examples of producing raster tiles from vector tiles and a Mapbox GL style. Mapbox does this with Mapbox GL Native on the server, but I don't believe they've released the exact software they use. https://github.com/maptiler/tileserver-gl/ is open-source software that generates raster tiles on demand. ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
