Re: [gdal-dev] vsicurl configuration design decisions

[Frank Warmerdam]

Sean,

The obvious answer is that interfaces that are organized around the
dataset name do not make it easy to transport other parameters in a
way that is specific to one dataset.

While I worry a bit about complex dataset syntaxes I do think there is
a power to embedding these options in the dataset name.  I sometimes
wish we had a more generalized way of wrapping options into dataset
names with clear escaping rules, and a way to avoid interference
between different levels of wrapping and virtualization.

I would add that I requested a mechanism to control /vsicurl/ retry
strategies as we this machinery widely and failure to do "normal
retries" in /vsicurl was a problem for us.

Best regards,
Frank


On Mon, Oct 9, 2017 at 7:18 PM, Sean Gillies  wrote:
> Hi all,
>
> It's written in
> http://gdal.org/gdal_virtual_file_systems.html#gdal_virtual_file_systems_vsicurl:
>
>> Starting with GDAL 2.3, options can be passed in the filename with the
>> following syntax: /vsicurl/option1=val1[,optionN=valN]*,url=http://...
>
> I'd like to discuss the design decisions that are being made here before
> this gets out into the world.
>
> I'm uncomfortable with the way configuration is spread between environment
> variables, config options that surface in the API, and also in identifiers.
> I don't think it's a great idea to that expand the amount of configuration
> in dataset identifiers. It's redundant, the syntax is complicated, and it
> dilutes the network effects of reusing identifiers in our applications.
>
> Are there specific advantages to this
>
>   ogrinfo -so /vsicurl/max_retry=10,url=https://example.com/poly.shp
>
> that we can't also have with a curl-style
>
>   ogrinfo -so --max-retry=10 /vsicurl/https://example.com/poly.shp
>
> or, better yet, in my opinion
>
>   ogrinfo -so --max-retry=10 https://example.com/poly.shp
>
> on the command line?
>
> --
> Sean Gillies
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev



-- 
---+--
I set the clouds in motion - turn up   | Frank Warmerdam, warmer...@pobox.com
light and sound - activate the windows |
and watch the world go round - Rush| Geospatial Software Developer
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] vsicurl configuration design decisions

[Sean Gillies]

Hi all,

It's written in
http://gdal.org/gdal_virtual_file_systems.html#gdal_virtual_file_systems_vsicurl
:

> Starting with GDAL 2.3, options can be passed in the filename with the
following syntax: /vsicurl/option1=val1[,optionN=valN]*,url=http://...

I'd like to discuss the design decisions that are being made here before
this gets out into the world.

I'm uncomfortable with the way configuration is spread between environment
variables, config options that surface in the API, and also in identifiers.
I don't think it's a great idea to that expand the amount of configuration
in dataset identifiers. It's redundant, the syntax is complicated, and it
dilutes the network effects of reusing identifiers in our applications.

Are there specific advantages to this

  ogrinfo -so /vsicurl/max_retry=10,url=https://example.com/poly.shp

that we can't also have with a curl-style

  ogrinfo -so --max-retry=10 /vsicurl/https://example.com/poly.shp

or, better yet, in my opinion

  ogrinfo -so --max-retry=10 https://example.com/poly.shp

on the command line?

-- 
Sean Gillies
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL Cubic Interpolation

[Joaquim Luis]

Alternatively,  use GMT's grdsample with -fg option to force the knowledge  
(if it's not already in the file) that the Earth is round.


http://gmt.soest.hawaii.edu/doc/5.4.2/grdsample.html





Harvey,






I subsampled a GLOBE image of elevation data by a subsampling factor of  
12


to a resolution of 0.1 deg x 0.1 deg on integer multiples of 0.1 deg.  
The



original GLOBE image is: Latitude from +90-1/240 deg to -90+1/240 deg in



1/120 deg steps, and Longitude from -180+1/240 deg to +180-1/240 deg in



1/120 deg steps.






I understand that the cubic interpolation method in gdalwarp accounts  
for


the subsampling factor, so 48x48 adjacent pixels influence the target  
pixel



for a subsampling factor of 12.






Does gdalwarp assume the original image is a sphere, where 48x48  
adjacent


pixels influence all target pixels, or a rectangle, where a limited  
number


of pixels influence the target pixels at the top, bottom, left, and  
right



edges of the subsampled image?



A rectangle, with fallback to bilinear resampling at edges



If gdalwarp assumes the original image is a


rectangle and uses a modified interpolation method at the four edges of  
the


subsampled image, the modified method affects two rows and two columns  
at


the top, bottom, left and right edges of the subsampled image. If  
gdalwarp



assumes the original image is a rectangle, is there a command option so



gdalwarp considers spherical wrapping when interpolating?



Nothing directly straightforward. Your below plan is the a reasonable  
solution.




Alternatively, I



assume I could augment the original GLOBE image with the appropriate


additional rows and columns on all four edges. If I did that, how do I  
tell



gdalwarp the extent of the augmented input image?



You'll have to recreate a properly referenced input image, for example  
with



gdal_translate -ullr upperleftx upperlefty lowerleftx lowerlefty


Even


--
Spatialys - Geospatial professional services

http://www.spatialys.com___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL Cubic Interpolation

[Even Rouault]

Harvey,

> 
> I subsampled a GLOBE image of elevation data by a subsampling factor of 12
> to a resolution of 0.1 deg x 0.1 deg on integer multiples of 0.1 deg. The
> original GLOBE image is: Latitude from +90-1/240 deg to -90+1/240 deg in
> 1/120 deg steps, and Longitude from -180+1/240 deg to +180-1/240 deg in
> 1/120 deg steps.
> 
> I understand that the cubic interpolation method in gdalwarp accounts for
> the subsampling factor, so 48x48 adjacent pixels influence the target pixel
> for a subsampling factor of 12.
> 
> Does gdalwarp assume the original image is a sphere, where 48x48 adjacent
> pixels influence all target pixels, or a rectangle, where a limited number
> of pixels influence the target pixels at the top, bottom, left, and right
> edges of the subsampled image?

A rectangle, with fallback to bilinear resampling at edges

> If gdalwarp assumes the original image is a
> rectangle and uses a modified interpolation method at the four edges of the
> subsampled image, the modified method affects two rows and two columns at
> the top, bottom, left and right edges of the subsampled image. If gdalwarp
> assumes the original image is a rectangle, is there a command option so
> gdalwarp considers spherical wrapping when interpolating? 

Nothing directly straightforward. Your below plan is the a reasonable solution.

> Alternatively, I
> assume I could augment the original GLOBE image with the appropriate
> additional rows and columns on all four edges. If I did that, how do I tell
> gdalwarp the extent of the augmented input image?

You'll have to recreate a properly referenced input image, for example with

 gdal_translate -ullr upperleftx upperlefty lowerleftx lowerlefty

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL Cubic Interpolation

[Harvey Berger]

Hi,

I subsampled a GLOBE image of elevation data by a subsampling factor of 12
to a resolution of 0.1 deg x 0.1 deg on integer multiples of 0.1 deg. The
original GLOBE image is: Latitude from +90-1/240 deg to -90+1/240 deg in
1/120 deg steps, and Longitude from -180+1/240 deg to +180-1/240 deg in
1/120 deg steps.

I understand that the cubic interpolation method in gdalwarp accounts for
the subsampling factor, so 48x48 adjacent pixels influence the target pixel
for a subsampling factor of 12.

Does gdalwarp assume the original image is a sphere, where 48x48 adjacent
pixels influence all target pixels, or a rectangle, where a limited number
of pixels influence the target pixels at the top, bottom, left, and right
edges of the subsampled image? If gdalwarp assumes the original image is a
rectangle and uses a modified interpolation method at the four edges of the
subsampled image, the modified method affects two rows and two columns at
the top, bottom, left and right edges of the subsampled image. If gdalwarp
assumes the original image is a rectangle, is there a command option so
gdalwarp considers spherical wrapping when interpolating? Alternatively, I
assume I could augment the original GLOBE image with the appropriate
additional rows and columns on all four edges. If I did that, how do I tell
gdalwarp the extent of the augmented input image?

Kind regards,

Harvey
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Problem with merging SHAPEs into SpatiaLite

[Casper Børgesen]

Hi Even

Thanks for your reply, I will try the option. I just wonder, if the driver 
performs automatical laundering, shouldn't it automatically transform any 
requests too? So when asked if a layer exists, launder the incoming name and 
compare to that or does it have other complications down the road?

Regards, Casper

From: Even Rouault [mailto:even.roua...@spatialys.com]
Sent: 5. oktober 2017 13:15
To: Casper Børgesen 
Cc: gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] Problem with merging SHAPEs into SpatiaLite


On jeudi 5 octobre 2017 10:52:42 CEST Casper Børgesen wrote:

> Hi Even

>

> I have prepared a very small test data set:

>

> 2 shape files

> 1 BAT file

> 1 screenshot of my output

>

> You can find it here: https://ufile.io/ravt2

>

> I don't have access to a Linux machine at the moment, so I cannot test it on

> Linux. But I am curious to know if it works on Linux :)



Oh I wrongly assumed you used geopackage whereas you use sqlite



The difference is that the sqlite driver applies by default name laundering 
which will turn "a-b" into a_b in the creation stage. So your append -nln "a-b" 
will match a non existing layer



Add -lco LAUNDER=NO to the creation stage and that will solve it.



>

> Regards, Casper

>

> From: Even Rouault [mailto:even.roua...@spatialys.com]

> Sent: 5. oktober 2017 12:34

> To: gdal-dev@lists.osgeo.org

> Cc: Casper Børgesen >

> Subject: Re: [gdal-dev] Problem with merging SHAPEs into SpatiaLite

>

> On mercredi 4 octobre 2017 08:58:25 CEST Casper Børgesen wrote:

> > Hi Jukka

> >

> >

> >

> > I tried only to discover that the error is probably about an invalid (?)

> >

> > layer name (-nln). ogr2ogr doesn't complain if I give it a layer name with

> >

> > a dash in the middle, like 'some-thing'. When appending data to the layer

> >

> > using the same layer name, I get the error message.

> >

> > I don't know if this type of layer names are indeed invalid and if so, the

> >

> > current behaviour of ogr2ogr not complaining is intended.

>

> I cannot reproduce on Linux (and can't see why that would be Windows

> specific)

>

>

>

> The following works fine for me:

>

> ogr2ogr -f gpkg test.gpkg ../autotest/ogr/data/poly.shp -nln "a-b"

>

> ogr2ogr -gt 10 -update -append test.gpkg ../autotest/ogr/data/poly.shp

> -nln "a-b"

>

>

>

> Even

>

>

>

> --

>

> Spatialys - Geospatial professional services

>

> http://www.spatialys.com





--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev