Re: [gdal-dev] Serve COG images - GEE & Google Cloud Storage

2020-09-04 Thread Frank Warmerdam
Arun,

For what it's worth, I routinely do this by preparing signed urls to the
objects in GCS and just using normal /vsicurl/ references in mapfiles,
etc.  If you happen to be using boto libraries in Python signed urls can be
created with the generate_url() call on "key" objects.   Of course if you
data is publish, you can just use the direct GCS urls without signing them.

Best regards,
Frank






On Fri, Sep 4, 2020 at 8:38 AM Michael Smith 
wrote:

> Another option is a fuse level driver that maps cloud storage to a drive
> path.
>
>
>
> Mike
>
>
>
>
>
> --
>
> Michael Smith
>
> US Army Corps of Engineers
>
> Remote Sensing/GIS Center
>
>
>
>
>
> *From: *gdal-dev  on behalf of Travis
> Kirstine 
> *Date: *Friday, September 4, 2020 at 8:35 AM
> *To: *gdal dev 
> *Subject: *Re: [gdal-dev] Serve COG images - GEE & Google Cloud Storage
>
>
>
>
>
> There is a good article here on how to do this using MapServer and S3,
> this may work for Google as well
>
>
>
>
> https://github.com/mapserver/mapserver/wiki/Render-images-straight-out-of-S3-with-the-vsicurl-driver
>
>
>
> You can configure MapServer as a WMS server and add the layers to Open
> Layers or take the extra step and configure MapCache to generate a tiled
> output and caching using your WMS as source
>
>
>
>
>
>
>
> On Fri, 4 Sep 2020 at 03:01, Even Rouault 
> wrote:
>
> Arun,
>
>
>
> Not sure this completely answer your question, but GDAL has gained a
>
> Google Cloud Storage virtual file system handler similar
>
> to the AWS S3 one since the post you mention.
>
> See
> https://gdal.org/user/virtual_file_systems.html#vsigs-google-cloud-storage-files
>
>
>
> Even
>
>
>
> > Hi
>
> > In this article, I came across about servicing rasters on AWS S3:
>
> >
> https://www.azavea.com/blog/2019/04/23/using-cloud-optimized-geotiffs-cogs/
>
> > , https://lists.osgeo.org/pipermail/gdal-dev/2015-October/042975.html
>
> >
>
> > I am trying to see if something similar exists for Google? I'm exporting
>
> > COG from Google Earth Engine (GEE) to a Google Cloud Storage bucket in
> the
>
> > hopes to access and serve it with other layers using GeoServer or create
>
> > tile map services. I've many dates and many products (true color, false
>
> > color, ndvi) for many sites. So I'm not sure what's the best way to serve
>
> > them. I tried researching online, asking in StackExchange, GeoServer and
>
> > GEE forums and reaching out to a few people, but I couldn't get specific
>
> > info... Most info is on AWS and S3...
>
> >
>
> > My requirement is to somehow get the images of "GEE to bucket" exported
>
> > images into Openlayers (web mapping) and JavaScript (frontend). GEE can
>
> > export to my Drive or a GCP bucket. So serving from GEE to GCP to S3
> means
>
> > some download, unzip, zip and upload - so more time consuming and manual
>
> > steps. In my experience, AWS was costly (as my EC2 runs all the time)
> than
>
> > a VPS server...
>
> >
>
> > I'll appreciate your help. Thank you!
>
>
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> ___ gdal-dev mailing list
> gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev



-- 
---+--
I set the clouds in motion - turn up   | Frank Warmerdam,
warmer...@pobox.com
light and sound - activate the windows | +1 650-701-7823
and watch the world go round - Rush| Geospatial Software Developer
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Serve COG images - GEE & Google Cloud Storage

2020-09-04 Thread Michael Smith
Another option is a fuse level driver that maps cloud storage to a drive path. 

 

Mike

 

 

-- 

Michael Smith

US Army Corps of Engineers

Remote Sensing/GIS Center

 

 

From: gdal-dev  on behalf of Travis Kirstine 

Date: Friday, September 4, 2020 at 8:35 AM
To: gdal dev 
Subject: Re: [gdal-dev] Serve COG images - GEE & Google Cloud Storage

 

 

There is a good article here on how to do this using MapServer and S3, this may 
work for Google as well

 

https://github.com/mapserver/mapserver/wiki/Render-images-straight-out-of-S3-with-the-vsicurl-driver

 

You can configure MapServer as a WMS server and add the layers to Open Layers 
or take the extra step and configure MapCache to generate a tiled output and 
caching using your WMS as source 

 

 

 

On Fri, 4 Sep 2020 at 03:01, Even Rouault  wrote:

Arun,

 

Not sure this completely answer your question, but GDAL has gained a

Google Cloud Storage virtual file system handler similar

to the AWS S3 one since the post you mention.

See 
https://gdal.org/user/virtual_file_systems.html#vsigs-google-cloud-storage-files

 

Even

 

> Hi

> In this article, I came across about servicing rasters on AWS S3:

> https://www.azavea.com/blog/2019/04/23/using-cloud-optimized-geotiffs-cogs/

> , https://lists.osgeo.org/pipermail/gdal-dev/2015-October/042975.html

> 

> I am trying to see if something similar exists for Google? I'm exporting

> COG from Google Earth Engine (GEE) to a Google Cloud Storage bucket in the

> hopes to access and serve it with other layers using GeoServer or create

> tile map services. I've many dates and many products (true color, false

> color, ndvi) for many sites. So I'm not sure what's the best way to serve

> them. I tried researching online, asking in StackExchange, GeoServer and

> GEE forums and reaching out to a few people, but I couldn't get specific

> info... Most info is on AWS and S3...

> 

> My requirement is to somehow get the images of "GEE to bucket" exported

> images into Openlayers (web mapping) and JavaScript (frontend). GEE can

> export to my Drive or a GCP bucket. So serving from GEE to GCP to S3 means

> some download, unzip, zip and upload - so more time consuming and manual

> steps. In my experience, AWS was costly (as my EC2 runs all the time) than

> a VPS server...

> 

> I'll appreciate your help. Thank you!

 

 

-- 

Spatialys - Geospatial professional services

http://www.spatialys.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

___ gdal-dev mailing list 
gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Serve COG images - GEE & Google Cloud Storage

2020-09-04 Thread Travis Kirstine
There is a good article here on how to do this using MapServer and S3, this
may work for Google as well

https://github.com/mapserver/mapserver/wiki/Render-images-straight-out-of-S3-with-the-vsicurl-driver

You can configure MapServer as a WMS server and add the layers to Open
Layers or take the extra step and configure MapCache to generate a tiled
output and caching using your WMS as source



On Fri, 4 Sep 2020 at 03:01, Even Rouault 
wrote:

> Arun,
>
>
>
> Not sure this completely answer your question, but GDAL has gained a
>
> Google Cloud Storage virtual file system handler similar
>
> to the AWS S3 one since the post you mention.
>
> See
> https://gdal.org/user/virtual_file_systems.html#vsigs-google-cloud-storage-files
>
>
>
> Even
>
>
>
> > Hi
>
> > In this article, I came across about servicing rasters on AWS S3:
>
> >
> https://www.azavea.com/blog/2019/04/23/using-cloud-optimized-geotiffs-cogs/
>
> > , https://lists.osgeo.org/pipermail/gdal-dev/2015-October/042975.html
>
> >
>
> > I am trying to see if something similar exists for Google? I'm exporting
>
> > COG from Google Earth Engine (GEE) to a Google Cloud Storage bucket in
> the
>
> > hopes to access and serve it with other layers using GeoServer or create
>
> > tile map services. I've many dates and many products (true color, false
>
> > color, ndvi) for many sites. So I'm not sure what's the best way to serve
>
> > them. I tried researching online, asking in StackExchange, GeoServer and
>
> > GEE forums and reaching out to a few people, but I couldn't get specific
>
> > info... Most info is on AWS and S3...
>
> >
>
> > My requirement is to somehow get the images of "GEE to bucket" exported
>
> > images into Openlayers (web mapping) and JavaScript (frontend). GEE can
>
> > export to my Drive or a GCP bucket. So serving from GEE to GCP to S3
> means
>
> > some download, unzip, zip and upload - so more time consuming and manual
>
> > steps. In my experience, AWS was costly (as my EC2 runs all the time)
> than
>
> > a VPS server...
>
> >
>
> > I'll appreciate your help. Thank you!
>
>
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Serve COG images - GEE & Google Cloud Storage

2020-09-04 Thread Even Rouault
Arun,

Not sure this completely answer your question, but GDAL has gained a
Google Cloud Storage virtual file system handler similar
to the AWS S3 one since the post you mention.
See 
https://gdal.org/user/virtual_file_systems.html#vsigs-google-cloud-storage-files

Even

> Hi
> In this article, I came across about servicing rasters on AWS S3:
> https://www.azavea.com/blog/2019/04/23/using-cloud-optimized-geotiffs-cogs/
> , https://lists.osgeo.org/pipermail/gdal-dev/2015-October/042975.html
> 
> I am trying to see if something similar exists for Google? I'm exporting
> COG from Google Earth Engine (GEE) to a Google Cloud Storage bucket in the
> hopes to access and serve it with other layers using GeoServer or create
> tile map services. I've many dates and many products (true color, false
> color, ndvi) for many sites. So I'm not sure what's the best way to serve
> them. I tried researching online, asking in StackExchange, GeoServer and
> GEE forums and reaching out to a few people, but I couldn't get specific
> info... Most info is on AWS and S3...
> 
> My requirement is to somehow get the images of "GEE to bucket" exported
> images into Openlayers (web mapping) and JavaScript (frontend). GEE can
> export to my Drive or a GCP bucket. So serving from GEE to GCP to S3 means
> some download, unzip, zip and upload - so more time consuming and manual
> steps. In my experience, AWS was costly (as my EC2 runs all the time) than
> a VPS server...
> 
> I'll appreciate your help. Thank you!


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Serve COG images - GEE & Google Cloud Storage

2020-09-03 Thread Arun Govind
Hi
In this article, I came across about servicing rasters on AWS S3:
https://www.azavea.com/blog/2019/04/23/using-cloud-optimized-geotiffs-cogs/
, https://lists.osgeo.org/pipermail/gdal-dev/2015-October/042975.html

I am trying to see if something similar exists for Google? I'm exporting
COG from Google Earth Engine (GEE) to a Google Cloud Storage bucket in the
hopes to access and serve it with other layers using GeoServer or create
tile map services. I've many dates and many products (true color, false
color, ndvi) for many sites. So I'm not sure what's the best way to serve
them. I tried researching online, asking in StackExchange, GeoServer and
GEE forums and reaching out to a few people, but I couldn't get specific
info... Most info is on AWS and S3...

My requirement is to somehow get the images of "GEE to bucket" exported
images into Openlayers (web mapping) and JavaScript (frontend). GEE can
export to my Drive or a GCP bucket. So serving from GEE to GCP to S3 means
some download, unzip, zip and upload - so more time consuming and manual
steps. In my experience, AWS was costly (as my EC2 runs all the time) than
a VPS server...

I'll appreciate your help. Thank you!
-- 
Sincerely yours,

*Arun Govind, PhD*
*Email*: arun.gov...@usask.ca
*Ph*: +16394714525
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev