There are multiple advantages to using s3 over ebs. First is availability. S3 is highly available (and replicated). EBS is limited to the single volume and is much more fragile. EBS is also more expensive for storage than S3. And S3 is available to multiple instances, EBS is not, it can only be mounted to one EC2 instance.
Mike -- Michael Smith Remote Sensing/GIS Center US Army Corps of Engineers From: gdal-dev <[email protected]> on behalf of Richard Greenwood <[email protected]> Date: Monday, March 4, 2019 at 9:21 AM To: Jeremy Palmer <[email protected]> Cc: gdal dev <[email protected]> Subject: Re: [gdal-dev] GDAL VRT with Cloud Optimized GeoTIFFs and AWS What are the advantages of storing the imagery in S3 as opposed to EBS? I'm using the throughput optimized magnetic EBS which I is a little less expensive than S3. I did some testing a couple years ago and didn't see enough performance gain to justify SSD. Have you tested S3 against any of the EBS options? Rich On Sat, Mar 2, 2019 at 7:58 AM Jeremy Palmer <[email protected]> wrote: I've now found this useful Mapserver wiki page https://github.com/mapserver/mapserver/wiki/Render-images-straight-out-of-S3-with-the-vsicurl-driver. It seems to imply that it's better to merge a collection of dataset tiles into a single big Geotiff rather than create a VRT due to the repeated HTTP calls. I'm still interested in people's experiences using imagery direct from S3 and it's performance. Note some of my largest datasets have a total size of 1.5TBs uncompressed Geotiff with about 8000 tiles @ 175mb per tile. I'll likely compress the uncompressed Geotiffs and use JPEG compression, which I estimate can bring the size down to about 200GB total. Cheers Jeremy On Sat, Mar 2, 2019 at 9:15 AM Jeremy Palmer <[email protected]> wrote: Hi All, Does anyone have any tips or experience in trying to serve RGB large imagery multi file datasets hosted on S3 to application servers/containers for bulk tile rendering? Is this possible using VRTs and is the performance manageable when compared to other mounted storage options? Thanks, Jeremy _______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev -- Richard W. Greenwood, PLS www.greenwoodmap.com _______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
