Re: [GRASS-dev] Cloud optimized GeoTIFF support

2017-10-15 Thread Markus Metz 
On Sun, Oct 15, 2017 at 9:28 PM, Markus Neteler  wrote:
>
> On Sun, Oct 15, 2017 at 7:47 PM, Markus Metz
>  wrote:
> > What do you mean with "a more direct GRASS GIS integration" regarding
cloud
> > storage and/or Cloud Optimized GeoTIFF?
>
> Well, I tought of r.external and r.external.out.

OK, rephrasing my question:

What do you mean with "a more direct GRASS GIS integration" regarding cloud
storage and/or Cloud Optimized GeoTIFF, and regarding reading and writing
such data?

Markus M
>
> Markus Metz wrote:
> > Have you tried GDAL's virtual network based file systems [0]?
>
> ... I didn't yet since traveling all the time. Maybe simple and solved
> but I have no stable connections at time to try out.
>
> Will study it soon :-)
>
> markusN
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Cloud optimized GeoTIFF support

2017-10-15 Thread Even Rouault 
On dimanche 15 octobre 2017 21:28:24 CEST Markus Neteler wrote:
> On Sun, Oct 15, 2017 at 7:47 PM, Markus Metz
> 
>  wrote:
> > What do you mean with "a more direct GRASS GIS integration" regarding
> > cloud
> > storage and/or Cloud Optimized GeoTIFF?
> 
> Well, I tought of r.external and r.external.out.
> 
> Markus Metz wrote:
> > Have you tried GDAL's virtual network based file systems [0]?
> 
> ... I didn't yet since traveling all the time. Maybe simple and solved
> but I have no stable connections at time to try out.
> 

Also note that the performance of network accesses depends a lot of how close 
the machine is from the server. For example if you use the VMs of a cloud 
provider in the same region as the bucket you access, the timing measuremetns 
will be 10 times or more better than if you try from a consumer ADSL 
connection.

For Linux & Mac, there are also various projects that use FUSE to mount 
buckets in the file system
https://github.com/kahing/goofys
https://github.com/googlecloudplatform/gcsfuse

I tried quickly goofys and it worked pretty well. It tends to have a more 
aggressive read-ahead strategy than GDAL /vsis3/, which makes reading a lot of 
sequential data faster than with /vsis3/. In latest GDAL trunk, the GeoTIFF 
driver is aware of the network nature of the files and thus can better 
optimize windowed access, so this makes it a bit faster in cases where you 
only access part of the imagery (for example with MapServer that must satisfy 
a WMS request on a BBOX, which was the use case for this recent round of 
optimization, or if you have a processing spread over several VMs with a 
spatial subdivision)

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Cloud optimized GeoTIFF support

2017-10-15 Thread Markus Neteler 
On Sun, Oct 15, 2017 at 7:47 PM, Markus Metz
 wrote:
> What do you mean with "a more direct GRASS GIS integration" regarding cloud
> storage and/or Cloud Optimized GeoTIFF?

Well, I tought of r.external and r.external.out.

Markus Metz wrote:
> Have you tried GDAL's virtual network based file systems [0]?

... I didn't yet since traveling all the time. Maybe simple and solved
but I have no stable connections at time to try out.

Will study it soon :-)

markusN
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Cloud optimized GeoTIFF support

2017-10-15 Thread Markus Metz 
On Sun, Oct 15, 2017 at 6:25 PM, Markus Neteler  wrote:
>
> On Fri, Oct 13, 2017 at 2:45 PM, Stefan Blumentrath
>  wrote:
> > Hi,
> >
> > I assume you have seen this:
> > https://lists.osgeo.org/pipermail/gdal-dev/2017-October/047349.html
> >
> > and subsequently this:
> > https://erouault.blogspot.no/2017/10/gdal-and-cloud-storage.html
>
> Sure, that's where I got it from and got interested in a more direct
> GRASS GIS integration.

What do you mean with "a more direct GRASS GIS integration" regarding cloud
storage and/or Cloud Optimized GeoTIFF?

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Cloud optimized GeoTIFF support

2017-10-15 Thread Markus Neteler 
On Fri, Oct 13, 2017 at 2:45 PM, Stefan Blumentrath
 wrote:
> Hi,
>
> I assume you have seen this:
> https://lists.osgeo.org/pipermail/gdal-dev/2017-October/047349.html
>
> and subsequently this:
> https://erouault.blogspot.no/2017/10/gdal-and-cloud-storage.html

Sure, that's where I got it from and got interested in a more direct
GRASS GIS integration.

Markus
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] community sprint before Xmas? :)

2017-10-15 Thread Markus Neteler 
On Sat, Oct 14, 2017 at 11:35 PM, Helmut Kudrnovsky  wrote:
>
>>very good idea! we are accumulating things to discuss like
>>
>>- specs for GRASS GIS 8
>>- what's missing for 7.4.0
>>
>>Outreach:
>>- maybe run a survey to know who uses GRASS GIS
>>- offer more GRASS GIS courses
>>- more online material, use Sphinx
>>- set up a new web site (asap) with responsive design
>>- generate tons of 1min videos
>
> what about review and (possible) integration of GSoC 2017 work?

Good idea! I'd suggest to start a Wiki page for the meeting...

Markus (in a train with occasional net connection)
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Landsat MTL files

2017-10-15 Thread Nikos Alexandris 

Dear all, in improving https://github.com/NikosAlexandris/i.landsat.import,
new is, for example (assuming a 'Landsat8' directory that
contains 87 decompressed and unpacked Landsat acquisitions),

i.landsat.import pool=Landsat8 -t

which will return 87 timestamps

LO81920282015061LGN00|2015-03-02 09:58:07.5566043 +
LC81920282013135LGN01|2013-05-15 10:00:20.1682888 +
..
LC81920282013279LGN00|2013-10-06 10:00:15.3022235 +
LC81920282014218LGN00|2014-08-06 09:58:23.0169101 +

useful to feed t.register (a suffix option yet to implement so as to
define the  part).


Not all MTL files have identical strings for what concerns the DATE and
TIME of acquisition. So far, the module treats

TIME_STRINGS = ['SCENE_CENTER_TIME', 'SCENE_CENTER_SCAN_TIME']
DATE_STRINGS = ['DATE_ACQUIRED', 'ACQUISITION_DATE']

Would you know of other strings, let me please know.
Any module-related ideas welcome.

Thank you, Nikos


signature.asc
Description: PGP signature
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] community sprint before Xmas? :)

2017-10-15 Thread Paulo van Breugel 


Hi Vero


On October 12, 2017 1:42:58 PM Veronica Andreo  wrote:


Hi devs,

What do you think about a short community sprint before Xmas? Well, indeed
before December 17th would be better because I travel to Argentina for Xmas
:)

Ok seriously, even though there's the OSGeo code sprint in March, that's
still a looong way to go and there are topics that have been on the agenda
for quite some time already...e.g. new website, git migration, include more
core devs and so on... we should also keep planning ahead for GRASS 8...
The idea of meeting is not only because of that, but above all because it
is always so nice and fun to meet :D



Great idea



Anyway, I have created a doodle [0] with friday+weekends until December
10th inclusive. Other options are also possible, I just put that because
friday+weekend(+monday alternatively) is the most convenient in my case. If
you are interested/available, please fill in the doodle or suggest other
preferred dates. The idea is to see if we may coincide in our availability
and meet.

Regarding the place, I can offer my house to accomodate 5-6 persons (we
would need 2-3 inflatable mattresses and sleeping bags). Or we can rent an
airbnb here or somewhere else (btw, I live in Enschede, Netherlands).

First, let's see if we can coincide in timing.

What do you say?



I'm interested to participate. Not yet sure what dates I'll have time 
(probably end of November or in December.




Cheers,
Vero

[0] https://beta.doodle.com/poll/p4a8wmtcpw6c9m9s



--
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev