You can also use something like goofys which is a fuse provider for s3 which provides s3 access while using normal file path access. It’s useful when you have to deploy to cloud and non cloud.
Michael Smith RSGIS Center > On Aug 30, 2019, at 7:31 PM, Jan Hartmann <j.l.h.hartm...@uva.nl> wrote: > > Impressive, looks like viscurl is a real alternative to local files. Thanks > for the information, I'm certainly going to look further on this. > > Jan > >> On 8/30/2019 6:15 PM, Peter Schmitt wrote: >>> On Fri, Aug 30, 2019 at 9:48 AM Jan Hartmann <j.l.h.hartm...@uva.nl> wrote: >>> Thanks Peter, this is really useful. Do you have any real-world benchmarks >>> for MapServer that compare regular file access with vsicurl access, using >>> optimized Geotifs? I've seen the tests for GDAL at >>> https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF, but what about >>> servering large map-sets over the web? >>> >> I don't have any sort of formal benchmarks... but I can provide anecdotes >> from my experiences. Random access to a 256x256 block of extremely large >> COGs served from fast SSD would have maybe 100ms response. The same COG >> accessed with /vsicurl/: First time the tile is requested about 400ms. >> GDAL has a least recently used curl cache. When a tile is cached, >> subsequent requests can be served maybe about 100ms. My cloud provider is >> AWS... so access from s3 using MapServer running in a Docker container on an >> AWS EC2 instance in the us-east-1 region... and the tile is requested from >> my local wireless network in Colorado. >> >> When the data is a COG in s3 and MapServer runs near the cloud storage, >> performance is quite good. I've scaled only to dozens of users. >> Allegedly s3 can scale to 5,500 requests per second per prefix in s3. >> https://docs.aws.amazon.com/AmazonS3/latest/dev/optimizing-performance.html >> Presumably it would scale fairly well by adding more MapServer processes. >> > > _______________________________________________ > mapserver-users mailing list > mapserver-users@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________ mapserver-users mailing list mapserver-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
