[gdal-dev] [GDAL-dev] convert rgba alpha layer to rgb white

[Pierre CARREAUD]

Hi list,

I’m trying to assembly some RGBA tiff in one with gdalbuildvrt and then to
translate them to a RGB TIFF tiled file YCBCR jpeg compressed.

No problem… But now, I would like to get a white color in the background
instead of the black.

 

The following commands work but I get nodata (and so, black) on any white
pixels whereas I just wanted to convert alpha to white… The problem in on
the vrt.

 

gdalbuildvrt assembly.vrt -srcnodata b4 –hidenodata -vrtnodata 255 *.tif

gdal_translate -of GTIFF -b 1 -b 2 -b 3 -co COMPRESS=JPEG -co
JPEG_QUALITY=95 -co TFW=YES -co BIGTIFF=YES -co ALPHA=NO -co TILED=YES -co
PHOTOMETRIC=YCBCR -co PROFILE=BASELINE assembly.vrt assembly_JPG.tif

gdaladdo -r gauss -ro --config COMPRESS_OVERVIEW JPEG --config
JPEG_QUALITY_OVERVIEW 95 --config BIGTIFF_OVERVIEW YES --config
PHOTOMETRIC_OVERVIEW YCBCR --config INTERLEAVE_OVERVIEW PIXEL --config
NODATA_VALUES 255 assembly_JPG.tif 2 4 8 16 32 64 128 256

 

Do you have an idea? The problem is the is not –init option, just -vrtnodata
“255 255 255” to fix both background color but also the nodata bandper band…

 

Pierre CARREAUD

Spécialiste en imagerie numérique

 


carte_voeux_2014

L’INGÉNIERIE DE LA DURABILITÉ

 

SITES – Entité Île-de-France

Division du Développement Technique

2 bis avenue du Centre

92500 Rueil-Malmaison, France

Tel : 01 41 39 02 00 / Fax : 01 41 39 02 01

  www.sites.fr

 

CARRES  EXPERTISE  cid:image002.jpg@01CB7A74.CDE1E680  INSTRUMENTATION /
MESURES  cid:image002.jpg@01CB7A74.CDE1E680  INGENIERIE

 

<><><>___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Fast Pixel Access

[David Baker (Geoscience)]

Klokan,

This looks like it could work.  This would be accessed with the GDAL_WCS 
driver?  If so, I see that there it requires Curl, and I have a Curl directory 
in my install, but I get an error when I try and access the data.

David

David M. Baker
Senior Advisor - Geoscience Technology
Chesapeake Energy Corporation
405-935-3715 office
405-496-5373 cell
david.m.ba...@chk.com

From: klo...@gmail.com [mailto:klo...@gmail.com] On Behalf Of Klokan Petr Pridal
Sent: Monday, February 03, 2014 1:38 PM
To: David Baker (Geoscience)
Cc: Rahkonen Jukka (Tike); gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] Fast Pixel Access

Hi Jukka, David and others,

We were rendering in the past the elevation data into Mercator tiles with 
http://www.maptiler.com/, the follower of my GDAL2Tiles.py script - for 
extremely fast pixel access to the elevation values at given geographic 
location - without a need for server software for hosting such data. We made 
the tiles for CGIAR 90m DEM for whole world.

In fact the raw elevation was mapped into RGB space for this purpose. Decoding 
on the client side is then very easy and you have precise elevation values for 
the whole near area preloaded.

You may need to do something similar if you want to implement very fast client 
side hill-shading in web browser canvas (similar to 
http://dev.klokantech.com/klokan/hillshading/), or dynamic elevation profile 
drawn while moving the mouse over a map, and many other tasks with data loaded 
in the client directly.

The visualisation of the tiles is possible also in 3D, see: 
http://vimeo.com/29605292
For WebGL there are now more efficient ways (direct binary data structures 
instead of images).

See:
http://dev.klokantech.com/srtm/srtm_decode.html
http://www.klokantech.com/labs/dem-color-encoding/
http://dev.klokantech.com/srtm/googlemaps.html

There is a source code of the GDAL utility for encoding elevation data into RGB 
available online:
https://github.com/webglearth/gdaldem_web
but you may not need it, if direct reading via JavaScript in the web browser is 
not required for your application.

Regards,

Klokan Petr Pridal

On Mon, Feb 3, 2014 at 7:48 PM, David Baker (Geoscience) 
mailto:david.m.ba...@chk.com>> wrote:
Jukka,

No matter the endpoint the user uses to access the data, behind the scenes, 
there must be fast pixel access, correct?  Or are you saying that at WFS would 
do it quickly out of the box?

David

-Original Message-
From: gdal-dev-boun...@lists.osgeo.org 
[mailto:gdal-dev-boun...@lists.osgeo.org]
 On Behalf Of Rahkonen Jukka (Tike)
Sent: Monday, February 03, 2014 11:37 AM
To: gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] Fast Pixel Access
Hi,

Perhaps, but in this game the rule was not to have any GIS servers. Myself I 
would rather consider WFS. It could send heights from single points but also a 
profile along a line or all values within a polygon.

-Jukka-

Brian Case [r...@winkey.org] wrote:

> -Jukka

> tileindex, mapserver, and the gdal wms driver



> On Mon, 2014-02-03 at 17:20 +, Jukka Rahkonen wrote:
>> Luke Roth  gmail.com> writes:
>
> >
>> > Another thing that might speed up access is setting the config
>> option GDAL_DISABLE_READDIR_ON_OPEN = TRUE, either as an environment
>> variable or on the command line.  That should help with GDAL reading the
>> directory each time it opens a dataset.  I have an application which reads
>> one value from each of a large number of datasets and setting this option
>> made it run about 3 times faster.
>
>
> Hi,
>
> You are right. This config option makes GDAL to skip the reading of the
> remote directory and saves a lot of bandwidth:
>
> VRT case:
> Bytes Received:  4 244 509 (of which the vrt file: 4 192 577)
> Sequence (clock) duration:00:00:09.9996000
> Was:
> Bytes Received:  6 459 443
> Sequence (clock) duration:00:00:37.813
>
> BigTIFF case:
> Bytes Received:  2 158 917
> Sequence (clock) duration:00:00:04.4368000
> Was:
> Bytes Received:  4 374 137
> Sequence (clock) duration:00:00:30.9192000
>
>
> Conclusion:
> Both options are unsuitable for serious use while amusing to play with.
> Reading the BigTIFF tile offset index (or whatever it is) seems to mean
> about 2 MB of compultory payload traffic. Reading the VRT file means in this
> example 4 MB of payload. If this sort of net access to a large directory of
> raster files should be important for someone there should be a way to find
> the right raster file and righ data range in that file with minimum amount
> of bytes. Perhaps some kind of rtree indexed vrt file? First aid might be to
> keep the vrt file on the client side.
>
> -Jukka Rahkonen-
>
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://l

Re: [gdal-dev] How Can I gdalwarp From One Image to a Larger Spatial Area While Leaving Missing Data Blank in the Destination?

[Peter Willis]

Hello,

Thank you Jukka, that worked.
I just shifted the coordinate of the origin to another location and
everything worked.

Peter

-Original Message-
From: gdal-dev-boun...@lists.osgeo.org
[mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Jukka Rahkonen
Sent: February-05-14 5:03 AM
To: gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] How Can I gdalwarp From One Image to a Larger
Spatial Area While Leaving Missing Data Blank in the Destination?

Peter Willis  aslenv.com> writes:

> 
> Hello,
> 
> Sorry about the title but it's a bit of a bear finding answers if the 
> headers don't show the actual topic.
> But I digress.
> 
> My Issue:
> I have an input image that covers coordinates 0,0 to 450,350 UTM zone 
> 13 North and the pixel size is 1 meter.
> 
> I want to mosaic pixels 0,0 to 25,25 into a coverage of UTM 
> coordinates
> -25,-25 : 25, 25 at the output in 1 meter pixels.
> This should show a 25x25 square portion of the image in one corner of 
> the
> 50x50 output image.
> 
> 'gdal_translate' will not do this because  -25,-25  are outside the 
> bounds of the original image.
> 
> I have tried the following with gdalwarp:
> 
> gdalwarp -overwrite -te -25 -25 25 25 -tr 1 1  ./input.tif 
> ./output.tif
> 
> This does not work. I get the following error:
> 
> ERROR 1: Unable to compute a transformation between pixel/line and 
> georeferenced coordinates for ./input.tif.
> There is no affine transformation and no GCPs.
> 
> There is geography applied to the input file.
> 
> What am I doing wrong ?

Hi,

This is funny. GDAL seems not to believe in images with origo at 0,0.
I tested with some dummy png file
gdal_translate -of gtiff -co tfw=yes -co profile=baseline test.png test.png
test.tif Input file size is 588, 321
0...10...20...30...40...50...60...70...80...90...100 - done.

It does not create a tfw file. Now this fails gdalwarp test.tif test2.tif
ERROR 1: Unable to compute a transformation between pixel/line and
georeferenced coordinates for test.tif.
There is no affine transformation and no GCPs.

I made tfw file test.tfw by hand as
1
0
0
-1
0
0

and then this is successful
gdalwarp test.tif test2.tif
Creating output file that is 588P x 321L.
Processing input file test.tif.
0...10...20...30...40...50...60...70...80...90...100 - done.

and also what you want to do
gdalwarp -te -1000 -1000 1000 1000  test.tif testi3.tif Creating output file
that is 2000P x 2000L.
Processing input file test.tif.
0...10...20...30...40...50...60...70...80...90...100 - done.

So I feel there is a bug if corner coordinates are 0,0. Please verify with a
handwritten tfw file and images with origo at, let's say, 1,1.

-Jukka Rahkonen-



> Thanks for any help.
> 
> Peter
> 




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Fast Pixel Access

[David Baker (Geoscience)]

Luke,

Thank you for this suggestion.  This too the access times from 15-20 seconds 
down to 1 to 3 seconds.  The majority of the time seems to be spent on the 
initial read of the vrt as subsequent piped locations after the first are 
returned sub-second.  For my current application, this should be okay.

David


From: gdal-dev-boun...@lists.osgeo.org 
[mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Luke Roth
Sent: Monday, February 03, 2014 8:11 AM
To: Jukka Rahkonen
Cc: gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] Fast Pixel Access

Another thing that might speed up access is setting the config option 
GDAL_DISABLE_READDIR_ON_OPEN = TRUE, either as an environment variable or on 
the command line.  That should help with GDAL reading the directory each time 
it opens a dataset.  I have an application which reads one value from each of a 
large number of datasets and setting this option made it run about 3 times 
faster.
Luke
On Sun, Feb 2, 2014 at 2:12 PM, Jukka Rahkonen 
mailto:jukka.rahko...@mmmtike.fi>> wrote:
Hi,

I made a few tests and here comes my conclusions. Hypothesis is that someone
wants to make a DEM query service which is using gdallocationinfo for
queries and DEM data is to be accessed as files from a standard web site. I
compared three alternatives:
1) There are thousands of DEM files on the server and they are combined
together with a VRT file.
2) There is only one DEM file as BigTIFF.
3) DEM is split into tiles into x/y/z tile directory structure like in
Google maps or OpenStreetMap tiles.

My test data covers Finland with 10 m grid size and as deflate compressed
tiffs they make about 10 GB together.

Before going on, keep in mind that the speed needs indexes. The better
index, the less unnecessary data to read. In case 1) the first level index
is the VRT file. The second level index, if it exists, is in the headers of
the real DEM files. It may be possible to jump to a correct offset from the
beginning of the DEM data and read only a part of the file.  In case 2) the
index is in the internal TIFF directory. If the BigTIFF is tiled the access
to tiles should be rather effectice. And finally in case 3) the index is
built into directory structure and tiling schema that is used for saving the
tiles. The schema is no well known that tile map service clients can
directly ask for a certain file name if they know the coordinates and scale.

Conclusions:

1)
- The whole VRT file must be readed. Caching the vrt file would make next
requests faster.
- For some reason gdallocationinfo wants to get the directory list of the
directory where the vrt file is. This is slow and generates lots of traffic
if the thousands of DEM files are in the same directory. Probably it would
be faster to have them in another dierectory.

2)
- BigTIFF route is more straight forward but gdallocationinfo needs still to
do many big range reads.
- Also in this case gdallocationinfo reads the target file directory. It
would be good to keep this directory small. Don't do like I did with having
in the directory the BigTIFF DEM file that was the only file needed, but
also the vrt and thousands of original DEMs from the previuos test -> but at
least this is a know this issue now and know how to avoid it. In my case
reading the directory made 2.2 MB of web traffic and all or most for wain.

3)
- I used OpenStreetMap tile service as the test data for the third test. In
this case gdallocationinfo knows exactly which tile to request and it is
making only one request. It also seems to cache some tiles on the client
side which means that queries for close locations may hit the cached tile
and be very fast.

Summary statistics:

1) Gdallocationinfo makes 6 requests and reads 6.4 MB of data
2) Gdallocationinfo makes 8 requests and reads 4.3 MB of data
3) Gdallocationinfo makes 1 requests and reads 10 kB of data

Requests I used are these:

1)
gdallocationinfo /vsicurl/http://latuviitta.kapsi.fi/data/
dem10m/dem_10m.vrt -geoloc  389559 6677412
2)
gdallocationinfo /vsicurl/http://latuviitta.kapsi.fi/data/
dem10m/dem_10m.tif -geoloc  389559 6677412
3)
gdallocationinfo  frmt_wms_openstreetmap_tms.xml -geoloc  389559 6677412

I know that the queried place in 3) is not the same because SRIDs of data
differ nor does OSM return 16-bit DEM heights but 3-band RGB values instead
but it does not matter here, the idea is what is important.

My conclusion is that you should cut your DEM into tiles with for example
gdal2tiles or MapTiler and the resuld could actually be quit speedy and
perhaps using 126x126 tiles could make it still a bit faster. Hope that they
can create tiles as 16-bit tiffs.

 I am sure that these results are not scientifically sound but I am also
sure that the difference between 6.4 MB/4.3 MB/10 kB is something to think
about especially if you dream about a mobile service.

I placed the requests which gdallocationinfo made during these tests into
http://latuviitta.org/documents/gdallocationinfo_requests.txt

-Jukka Rahko

Re: [gdal-dev] How Can I gdalwarp From One Image to a Larger Spatial Area While Leaving Missing Data Blank in the Destination?

[Jukka Rahkonen]

Peter Willis  aslenv.com> writes:

> 
> Hello,
> 
> Sorry about the title but it's a bit of a bear finding answers if the
> headers don't show the actual topic.
> But I digress.
> 
> My Issue:
> I have an input image that covers coordinates 0,0 to 450,350 UTM zone 13
> North and the pixel size is 1 meter.
> 
> I want to mosaic pixels 0,0 to 25,25 into a coverage of UTM coordinates
> -25,-25 : 25, 25 at the output in 1 meter pixels.
> This should show a 25x25 square portion of the image in one corner of the
> 50x50 output image.
> 
> 'gdal_translate' will not do this because  -25,-25  are outside the bounds
> of the original image.
> 
> I have tried the following with gdalwarp:
> 
> gdalwarp -overwrite -te -25 -25 25 25 -tr 1 1  ./input.tif ./output.tif
> 
> This does not work. I get the following error:
> 
> ERROR 1: Unable to compute a transformation between pixel/line
> and georeferenced coordinates for ./input.tif.
> There is no affine transformation and no GCPs.
> 
> There is geography applied to the input file.
> 
> What am I doing wrong ?

Hi,

This is funny. GDAL seems not to believe in images with origo at 0,0.
I tested with some dummy png file
gdal_translate -of gtiff -co tfw=yes -co profile=baseline test.png test.png
test.tif
Input file size is 588, 321
0...10...20...30...40...50...60...70...80...90...100 - done.

It does not create a tfw file. Now this fails
gdalwarp test.tif test2.tif
ERROR 1: Unable to compute a transformation between pixel/line
and georeferenced coordinates for test.tif.
There is no affine transformation and no GCPs.

I made tfw file test.tfw by hand as
1
0
0
-1
0
0

and then this is successful
gdalwarp test.tif test2.tif
Creating output file that is 588P x 321L.
Processing input file test.tif.
0...10...20...30...40...50...60...70...80...90...100 - done.

and also what you want to do
gdalwarp -te -1000 -1000 1000 1000  test.tif testi3.tif
Creating output file that is 2000P x 2000L.
Processing input file test.tif.
0...10...20...30...40...50...60...70...80...90...100 - done.

So I feel there is a bug if corner coordinates are 0,0. Please verify with a
handwritten tfw file and images with origo at, let's say, 1,1.

-Jukka Rahkonen-



> Thanks for any help.
> 
> Peter
> 




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Problems linking 1.10 with CURL support

[Jeff McKenna]

Hi Andy,

For MS4W I currently build GDAL 1.10.1 against curl-7.34.0, with no
issues.  My CURL_DIR setting points to the curl root directly (not "src"
as yours does).

-jeff


-- 
Jeff McKenna
MapServer Consulting and Training Services
http://www.gatewaygeomatics.com/



On 2/3/2014, 10:24 AM, Andy Cheetham wrote:
> Hi All
> 
> I'm have trouble linking GDAL 1.10 with CURL support enabled.
> 
> I can and have been building GDAL 1.9 with CURL so I'm sure I have all the 
> settings within the nmake.opt file correct, unless I have to do something 
> different for the latest version.
> 
> I'm on Windows 7 and have tried using both VS2005 and VS2010.
> 
> The linking error I'm getting is:
> 
> error LNK2019: unresolved external symbol "void __cdecl 
> CPLHTTPSetOptions(void *,char * *)" (?CPLHTTPSetOptions@@YAXPAXPAPAD@Z) 
> referenced in function "void __cdecl CPLHTTPInitializeRequest(struct 
> CPLHTTPRequest *,char const *,char const * const *)" (?
> CPLHTTPInitializeRequest@@YAXPAUCPLHTTPRequest@@PBDPBQBD@Z)   gdalhttp.obj
> 
> My nmake.opt section looks like the following
> 
> # Uncomment to use libcurl (DLL by default)
> # The cURL library is used for WCS, WMS, GeoJSON, SRS call importFromUrl(), 
> WFS, GFT, CouchDB, /vsicurl/ etc.
> CURL_DIR=W:\3rdpartyLibs\curl\7_35\src\
> CURL_INC=-I$(CURL_DIR)/include
> # Uncoment following line to use libcurl as dynamic library
> CURL_LIB = w:/lib/libcurl.lib wsock32.lib wldap32.lib winmm.lib
> # Uncoment following two lines to use libcurl as static library
> #CURL_LIB = $(CURL_DIR)/libcurl.lib wsock32.lib wldap32.lib winmm.lib
> #CURL_CFLAGS = -DCURL_STATICLIB   
> 
> The file cpl_http.cpp does have an implimentation but VS2005 would appear to 
> indicate that the #define HAVE)CURL is not defined???
> 
> Any help would be appreciated.
> 
> Cheers
> Andy
> 

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev