Will have a look... I hadn’t spent much time on optimization at this point. I wouldn’t doubt there might be improvements to be made. The simplification code would be the spot to look.
Extent represents the resolution of the tile and guides the simplification done as part of the conversion from world to mvt coordinates. Edge buffer defines tile overlap. By chance have you done output comparisons at all? That is, tile to tile in terms of size: feature counts, point counts/feature, etc... That would help tune the format parameters and make sure MapServer is returning reasonable output. One quick note on the SQL (2) is to be explicit with returned columns, avoid *. The t-rex SQL hints at zoom-related attributes in the layer? Again, will have to find time to checkout the benchmark stuff. ________________________________ From: karsten <kars...@terragis.net> Sent: Monday, March 11, 2019 5:53:35 PM To: 'karsten'; mapserver-users@lists.osgeo.org Cc: 'Pirmin Kalberer'; Lime, Steve D (MNIT); l...@lingner.eu; jmcke...@gatewaygeomatics.com; thomas.bonf...@gmail.com Subject: RE: MapServer derived MVTiles Benchmark test vs. T-Rex tile server Corrected map file link: https://github.com/karstenv/mvt-benchmark/blob/master/MapServer/ne.map<https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkarstenv%2Fmvt-benchmark%2Fblob%2Fmaster%2FMapServer%2Fne.map&data=02%7C01%7Csteve.lime%40state.mn.us%7Ca811ae33be2d46f58eed08d6a67469eb%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636879416281231683&sdata=tjeEJeuolr3ihw9fj%2B2ejyrDb46FJRR3haXMjhRMO1I%3D&reserved=0> ________________________________ From: karsten [mailto:kars...@terragis.net] Sent: Monday, March 11, 2019 23:15 To: 'mapserver-users@lists.osgeo.org' Cc: 'Pirmin Kalberer'; 'Lime, Steve D (MNIT)'; 'l...@lingner.eu'; 'jmcke...@gatewaygeomatics.com'; 'thomas.bonf...@gmail.com' Subject: MapServer derived MVTiles Benchmark test vs. T-Rex tile server Dear All, I am working on optimizing a MapServer /MapCache configuration of a benchmark for creating MVT tiles (seeding) of MapServer - versus T-Rex. This will be featured in a presentation at the German FOSS4g ('FOSSGIS') 2019 in Dresden at the end of this week. The whole thing is based on Pirmin Kalberer's automated vector tile benchmark see - http://blog.sourcepole.ch/2018/09/06/the-tale-of-a-vector-tile-benchmark/<https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fblog.sourcepole.ch%2F2018%2F09%2F06%2Fthe-tale-of-a-vector-tile-benchmark%2F&data=02%7C01%7Csteve.lime%40state.mn.us%7Ca811ae33be2d46f58eed08d6a67469eb%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636879416281241691&sdata=Rq7vxQk62vpPNJ4qjeq4fkLSuvSJ46t3Bicj760mTCU%3D&reserved=0> . Thus I created a similar configuration of the Natural Earth Data based world map for MapServer (my favorite rendering engine !) to run it against T-REX.. Now Pirmin has added this into the GitHUB repository as a 'MapServer' branch see - https://github.com/pka/mvt-benchmark/tree/mapserver<https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpka%2Fmvt-benchmark%2Ftree%2Fmapserver&data=02%7C01%7Csteve.lime%40state.mn.us%7Ca811ae33be2d46f58eed08d6a67469eb%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636879416281251699&sdata=0j4MMJBWx5IYNszsZPrSY1RRTGzvicRUoPw1BZGGcqk%3D&reserved=0> Results of one of my first trial benchmarks can be found here https://github.com/karstenv/mvt-benchmark/tree/master/firsttests<https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkarstenv%2Fmvt-benchmark%2Ftree%2Fmaster%2Ffirsttests&data=02%7C01%7Csteve.lime%40state.mn.us%7Ca811ae33be2d46f58eed08d6a67469eb%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636879416281261701&sdata=cu3VQvVDV2ieGjZs6Lld9kLa7SK0W%2FJe27Ak%2BJqcZSc%3D&reserved=0> . Unfortunately my favorite software (MapServer /MapCache.fcgi) is falling pretty far behind in seeding speed in these tests see e.g. in https://github.com/karstenv/mvt-benchmark/blob/master/firsttests/http.jpg<https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkarstenv%2Fmvt-benchmark%2Fblob%2Fmaster%2Ffirsttests%2Fhttp.jpg&data=02%7C01%7Csteve.lime%40state.mn.us%7Ca811ae33be2d46f58eed08d6a67469eb%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636879416281271709&sdata=K7o4vKUwAbL79z%2BPMElpMwMkPXtRNr%2BQO708LgTJN%2Fg%3D&reserved=0> Thus, I wanted to call in the experts to find out what configuration tweaks will help to increase the MapServer / Mapcache performance. So far my next steps I will be working on -------------------------------------------------------------------------------------------------- 1.) Setting the correct map file parameters to match the T-Rex configuration for the FORMATOPTIONS "EXTENT" and "EDGE_BUFFER" in the OUTPUTFORMAT bloc. OUTPUTFORMAT NAME "mvt" DRIVER MVT FORMATOPTION "EXTENT=512" # default is 4096. # Comment by Pirmin for t-rex this is 'tile_size' (per Layer): Width and height of the tile (Default: 4096. Grid default size is 256) FORMATOPTION "EDGE_BUFFER=20" # Comment by Pirmin - seems to be the 'buffer_size' in mvtbench.toml , is set to 3 for the Country layer, for all others END Any suggestions wthat my best bet settings are in the map file is for MapServer for "EXTENT" and "EDGE_BUFFER" ? -------------------------------------------------------------------------------------------------- 2.) Tweaking the data statement of the map file to match the T-Rex config So far I have e.g. for the countries layer DATA "wkb_geometry from (SELECT * FROM ne_10m_admin_0_countries) as temp using unique ogc_fid using SRID=3857" while T-Rex has sql = """SELECT wkb_geometry, adm0_a3, mapcolor7 FROM ne_10m_admin_0_countries WHERE min_zoom::integer <= !zoom! AND wkb_geometry && !bbox!""" see https://github.com/karstenv/mvt-benchmark/blob/master/t-rex/mvtbench.toml<https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkarstenv%2Fmvt-benchmark%2Fblob%2Fmaster%2Ft-rex%2Fmvtbench.toml&data=02%7C01%7Csteve.lime%40state.mn.us%7Ca811ae33be2d46f58eed08d6a67469eb%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636879416281281722&sdata=nWLjUb0zFYZZuNFq6FGc4aaQsF9bhJJnD5Zlnpoheos%3D&reserved=0> I am still trying to match the T-Rex statement above but did not come up with a suitable comparable MapServer data statement so far. Any suggestions for this anyone ? -------------------------------------------------------------------------------------------------- 3.) Does anyone have any other/ additional suggestions what else might help to increase MapServer /MapCache performance ? Thanks Karsten PS: The map file used is here https://github.com/karstenv/mvt-benchmark/blob/master/MapServer/mapcache_mvt.xml<https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkarstenv%2Fmvt-benchmark%2Fblob%2Fmaster%2FMapServer%2Fmapcache_mvt.xml&data=02%7C01%7Csteve.lime%40state.mn.us%7Ca811ae33be2d46f58eed08d6a67469eb%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636879416281291730&sdata=TIhHGj9EjFXidv6BvhbAloVb9IYYcM3bnOhzd5xiq18%3D&reserved=0>, the MapCache config here https://github.com/karstenv/mvt-benchmark/blob/master/MapServer/mapcache_mvt.xml<https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkarstenv%2Fmvt-benchmark%2Fblob%2Fmaster%2FMapServer%2Fmapcache_mvt.xml&data=02%7C01%7Csteve.lime%40state.mn.us%7Ca811ae33be2d46f58eed08d6a67469eb%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636879416281301738&sdata=VEuw6Lx8Me8jdU16oTOKQv17CCOJJ3mLTrPxh%2FJ9pu4%3D&reserved=0>
_______________________________________________ mapserver-users mailing list mapserver-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
