Did you have a look at https://blog.cleverelephant.ca/2022/04/coshp.html?
On Mon, Dec 18, 2023 at 2:39 PM Matthew Graber via MapServer-users < [email protected]> wrote: > Hi Jukka, > > Thanks for the advice. I was able to confirm through local testing that > using "DATA" is consistently faster than using the OGR connection (was > getting over 500 ms for OGR vs just over 300 ms for DATA). My results are > also suggesting that performing the shapefile optimizations may make my > request several hundredths of a second faster, I'm a little less sure on > that one. But regardless, it appears that the bulk of the problem with the > slow S3 connection is likely S3's overhead, so I don't think there's > anything more I can do with MapServer to speed it up. > > Thanks again, > Matt > > On Fri, Dec 8, 2023 at 11:58 AM Rahkonen Jukka < > [email protected]> wrote: > >> Hi, >> >> >> >> You should test the same shapefiles locally for understanding where the >> time gets spent. Normal time with simple styles is perhaps up to 300 ms. >> The optimization hints were written long before there were S3 or Amazon >> (the company). >> >> Unfortunately I don’t know how to optimize access to S3. >> >> >> >> -Jukka Rahkonen- >> >> >> >> *Lähettäjä:* MapServer-users <[email protected]> >> *Puolesta >> *Matthew Graber via MapServer-users >> *Lähetetty:* perjantai 8. joulukuuta 2023 21.47 >> *Vastaanottaja:* [email protected] >> *Aihe:* [MapServer-users] Speeding up msDrawMap() for shapefiles >> >> >> >> Hi, >> >> >> >> I'm trying to speed up the time required for msDrawMap() to execute for >> rendering shapefiles. Currently, according to the mapserver logs, it's >> taking 2.5-3.5 seconds, sometimes up to 5 seconds for it to execute: >> >> >> >> [Wed Dec 6 18:43:47 2023].699791 CGI Request 53 on process 2641 >> [Wed Dec 6 18:43:47 2023].717216 msDrawMap(): rendering using >> outputformat named png (AGG/PNG). >> [Wed Dec 6 18:43:52 2023].299580 msDrawMap(): Layer 1010 >> (Shapefile_Layer), 4.582s >> [Wed Dec 6 18:43:52 2023].299792 msDrawMap(): Drawing Label Cache, 0.000s >> [Wed Dec 6 18:43:52 2023].299803 msDrawMap() total time: 4.584s >> [Wed Dec 6 18:43:52 2023].310853 msSaveImage(stdout) total time: 0.011s >> >> [Wed Dec 6 18:43:52 2023].310883 mapserv request processing time >> (msLoadMap not incl.): 4.611s >> >> >> >> The mapfile is currently set up using "CONNECTIONTYPE OGR" and >> "CONNECTION" to access the shapefiles from an S3 bucket as follows: >> >> >> >> CONNECTIONTYPE OGR >> CONNECTION '/vsis3/s3_bucket_path/my_shapefile%.shp' >> >> >> >> I noticed at https://mapserver.org/input/vector/shapefiles.html that it >> says "Shapefile access is built directly into MapServer. It is also >> available through OGR, but direct access without OGR is recommended and >> discussed here." >> >> So I tried removing "CONNECTIONTYPE OGR" and changing "CONNECTION" to >> "DATA" as follows: >> >> >> >> DATA '/vsis3/s3_bucket_path/my_shapefile%.shp' >> >> >> >> I haven't been able to observe any definitive speed improvements by doing >> this yet, although I can't say for sure that there aren't any. >> >> >> >> Based on https://mapserver.org/optimization/vector.html, I also tried >> using the shptree command to create a spatial index and the coshp command >> to sort the shapefile and index. Even after doing that, I still see load >> msDrawMap() times of up to 5 seconds. >> >> >> >> Does anyone know of any other optimizations that could be made here? Or >> if not, would using shptree and coshp on the shapefile and then accessing >> it in the mapfile using "DATA" to take it from S3 be the optimal way to set >> things up? >> >> >> >> Thanks! >> >> Matt >> > _______________________________________________ > MapServer-users mailing list > [email protected] > https://lists.osgeo.org/mailman/listinfo/mapserver-users >
_______________________________________________ MapServer-users mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/mapserver-users
