[gdal-dev] What Statistics Are Returned By GDAL.Dataset.GetRasterBand(...).GetStatistics(...) ?
Hello,
I am using osgeo.gdal python module to load and get stats from image
bands.
While working with the, python, 'GetStatistics(...)' function I noticed that
the returned values seemed a bit off.
A snippet of code is shown below.
I compared the statistics returned in python by the function to band
statistics in ENVI image processing package for the same input file.
The results are shown below.
I am wondering if I am missing information regarding some key data filtering
feature in GDAL GetStatistics.
What statistics are GDAL returning?
#
## PYTHON CODE
#
src_ds = gdal.Open(image_path )
for bnd in range(1, src_ds.RasterCount+1 ):
item = {}
print("[ GETTING BAND ]: ", bnd)
srcband=None
srcband = src_ds.GetRasterBand(bnd)
if srcband is None:
continue
bnddata=srcband.ReadAsArray()
stats = srcband.GetStatistics( 0, 1 )
if stats is None:
continue
else:
print('put the values in a dictionary')
##GDAL stats##
#item['min'] = stats[0]
#item['max'] = stats[1]
#item['mean'] = stats[2]
#item['stdev'] = stats[3]
##numpy array stats##
#item['min'] = bnddata.min()
#item['max'] = bnddata.max()
#item['mean'] = bnddata.mean()
#item['stdev'] = bnddata.std()
###
#END PYTHON CODE
###
/*..
..*/
ENVI Statistics test returns these vales for each channel
ChannelMinMax MeanStdev
Band 1 0.00 0.441641 0.135938 0.095007
Band 2 0.00 0.42 0.134556 0.096385
Band 3 0.00 0.512614 0.143145 0.108702
Band 4 0.00 0.574203 0.159381 0.128212
Band 5 0.00 1.286870 0.190917 0.159562
Band 6 0.00 1.368695 0.218191 0.191100
Band 7 0.00 1.208142 0.179098 0.158407
/*..
..*/
GDAL GetStatistics in Python 3.0 Returns
[
{'min': 0.10543035715818, 'max': 0.35029646754265, 'mean': 0.19733288299107,
'stdev': 0.03141020073449},
{'min': 0.087364979088306, 'max': 0.36481207609177, 'mean':
0.19531263034662, 'stdev': 0.040193729911813},
{'min': 0.066563792526722, 'max': 0.40562310814857, 'mean':
0.20773722116063, 'stdev': 0.060965329408999},
{'min': 0.057971999049187, 'max': 0.49459338188171, 'mean':
0.23126556845189, 'stdev': 0.084854378172706},
{'min': 0.052771702408791, 'max': 0.57949388027191, 'mean':
0.27695694029464, 'stdev': 0.11419731959916},
{'min': 0.033055797219276, 'max': 0.67045384645462, 'mean':
0.31645853188616, 'stdev': 0.14756091787453}
]
/*..
..*/
Using NumPy Stats Functions in Python 3.0
np.min()
np.max()
np.mean()
np.std()
Return the following values after converting the previous GDAL bands to
'NumPy.Array' and using the NumPy functions
(Values look similar to ENVI)
[
{'min': 0.0, 'max': 0.44164082, 'mean': 0.13593808, 'stdev': 0.09500719},
{'min': 0.0, 'max': 0.4158, 'mean': 0.13455594, 'stdev': 0.09638489},
{'min': 0.0, 'max': 0.51261353, 'mean': 0.14314489, 'stdev': 0.108702265},
{'min': 0.0, 'max': 0.57420313, 'mean': 0.15938139, 'stdev': 0.12821245},
{'min': 0.0, 'max': 1.2868699, 'mean': 0.19091722, 'stdev': 0.15956147},
{'min': 0.0, 'max': 1.3686954, 'mean': 0.21819061, 'stdev': 0.1911003}
]
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] How Can I gdalwarp From One Image to a Larger Spatial Area While Leaving Missing Data Blank in the Destination?
Hello, Thank you Jukka, that worked. I just shifted the coordinate of the origin to another location and everything worked. Peter -Original Message- From: gdal-dev-boun...@lists.osgeo.org [mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Jukka Rahkonen Sent: February-05-14 5:03 AM To: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] How Can I gdalwarp From One Image to a Larger Spatial Area While Leaving Missing Data Blank in the Destination? Peter Willis pwillis at aslenv.com writes: Hello, Sorry about the title but it's a bit of a bear finding answers if the headers don't show the actual topic. But I digress. My Issue: I have an input image that covers coordinates 0,0 to 450,350 UTM zone 13 North and the pixel size is 1 meter. I want to mosaic pixels 0,0 to 25,25 into a coverage of UTM coordinates -25,-25 : 25, 25 at the output in 1 meter pixels. This should show a 25x25 square portion of the image in one corner of the 50x50 output image. 'gdal_translate' will not do this because -25,-25 are outside the bounds of the original image. I have tried the following with gdalwarp: gdalwarp -overwrite -te -25 -25 25 25 -tr 1 1 ./input.tif ./output.tif This does not work. I get the following error: ERROR 1: Unable to compute a transformation between pixel/line and georeferenced coordinates for ./input.tif. There is no affine transformation and no GCPs. There is geography applied to the input file. What am I doing wrong ? Hi, This is funny. GDAL seems not to believe in images with origo at 0,0. I tested with some dummy png file gdal_translate -of gtiff -co tfw=yes -co profile=baseline test.png test.png test.tif Input file size is 588, 321 0...10...20...30...40...50...60...70...80...90...100 - done. It does not create a tfw file. Now this fails gdalwarp test.tif test2.tif ERROR 1: Unable to compute a transformation between pixel/line and georeferenced coordinates for test.tif. There is no affine transformation and no GCPs. I made tfw file test.tfw by hand as 1 0 0 -1 0 0 and then this is successful gdalwarp test.tif test2.tif Creating output file that is 588P x 321L. Processing input file test.tif. 0...10...20...30...40...50...60...70...80...90...100 - done. and also what you want to do gdalwarp -te -1000 -1000 1000 1000 test.tif testi3.tif Creating output file that is 2000P x 2000L. Processing input file test.tif. 0...10...20...30...40...50...60...70...80...90...100 - done. So I feel there is a bug if corner coordinates are 0,0. Please verify with a handwritten tfw file and images with origo at, let's say, 1,1. -Jukka Rahkonen- Thanks for any help. Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] How Can I gdalwarp From One Image to a Larger Spatial Area While Leaving Missing Data Blank in the Destination?
Hello, Sorry about the title but it's a bit of a bear finding answers if the headers don't show the actual topic. But I digress. My Issue: I have an input image that covers coordinates 0,0 to 450,350 UTM zone 13 North and the pixel size is 1 meter. I want to mosaic pixels 0,0 to 25,25 into a coverage of UTM coordinates -25,-25 : 25, 25 at the output in 1 meter pixels. This should show a 25x25 square portion of the image in one corner of the 50x50 output image. 'gdal_translate' will not do this because -25,-25 are outside the bounds of the original image. I have tried the following with gdalwarp: gdalwarp -overwrite -te -25 -25 25 25 -tr 1 1 ./input.tif ./output.tif This does not work. I get the following error: ERROR 1: Unable to compute a transformation between pixel/line and georeferenced coordinates for ./input.tif. There is no affine transformation and no GCPs. There is geography applied to the input file. What am I doing wrong ? Thanks for any help. Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Is it possible to ehance input channels separately with gdal_translate ?
Hello, I have some input data where the dynamic range and offset of each channel are distinct. This means that applying the same linear enhancement based on data range to my selected RGB output bands will not produce an optimal visual enhancement. -scale smin smax dmin dmax only provides the same linear enhancement for all selected input channels (bands). Also the default causes gdal_translate to calculate the enhancement from the data which is also the wrong thing to do owing to the relative changeability of albedo from one scene to another due to the possible presence or absence of cloud, snow, or sand, vs. water or coal piles (light vs. dark) etc.. and also the relative difference in dynamic range between channels. QUESTION: Is there a way to enhance my input channels separately in a single gdal_translate call, or do I need to make multiple passes? Thanks Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Hello,
I have a curious problem with 'gdalinfo' (gdal version 1.9.0 ).
I have an input ENVI file with the following .HDR entry:
data ignore value = -9.9900e+002
gdalinfo for the IMG file provides stats that exclude all values -999.0 as
expected.
I then create a masked output file using 'gdal_rasterize' from that input
file once again using -999.0 as the mask value.
'gdal_rasterize' produces an ENVI format output with an accompanying .HDR
file.
The resulting 'gdal_rasterize' HDR *does not* contain the previous 'data
ignore value' entry. This is something that should
be fixed in future versions of that utility. The work around is, of course,
to simply concatenate the 'data ignore value'
to the end of the HDR file. This is a quick step:
echomyfile.hdr
echo data ignore value = -9.9900+e2 myfile.hdr
echomyfile.hdr
[NOTE: There are some other issues with ENVI HDR generation in GDAL that
should also be addressed but I won't go into those here.]
PROBLEM:
gdalinfo does not parse the value for the concatenated header entry if it is
placed at the end of the HDR file.
In fact, the HDR file may be edited to move the entry to a location near the
top of the file whereby it remains unacknowledged by gdalinfo.
QUESTION:
Does gdalinfo expect the header entries to be in a specific order?
I have appended some parsing information to the end of this email.
You are probably aware of most, if not all, of this information.
Thanks,
Peter
--ENVI TEXT HEADERS---
The ENVI header file is not an 'ordered' data file.
The entries may appear in any order.
The generic format for entries is:
NAME = VALUE[EOL]
Equals '=' normally appears on the same line as NAME without intervening
carriage return characters.
It is normal for '=' to be preceded by one space character although not
necessary.
Equals '=' is considered to be the primary delimiter between NAME and
VALUE.
---NAMES---
'Name' may be any string without carriage returns, and the NAME may
include space characters.
All names (and values) are trimmed of preceding and trailing white space
characters at parse time.
TAB'\t' is not normally used as whitespace but is parsed as though it
were a space ' ' character.
---VALUES---
'VALUE' may be a singular value (ie: a number or a string with no
whitespace ) or a 'list' bounded by curly braces { }.
Examples of singular string values would be:
Interleave = BSQ
Sensor type = Unknown
Wavelength units = nanometers
where 'BSQ' , 'Unknown', and 'nanometers' are singular strings containing no
whitespace characters.
In the context of the VALUE strings containing necessary whitespace should
be encoded as a list. String values may contain carriage return characters.
The comma ',' is considered to
be the default delimiter for list type data.
Ie:
My List = { some string data , another string data , yet another data
string ,
etc data strings ad infinitum
}
Number formats associated with VALUE
Singular Integers:
My Integer = 9
My Integer = -99
My Integer = 0
Singular Reals:
My Real = 0.9
My Real = .9
My Real = -9.0+e8
Lists of numbers:
My Integer List = {
1 , 2 , 3, 4 , 5 , 6
}
My Real List = {
1.0 , -2.00e+2 , 3.0e-4, 4.0 , .5 , 6.1
}
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
-98_float.img):LSatThe
rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1998_212 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThe
rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1998_221 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThe
rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1998_260 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThe
rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1998_269 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThe
rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1998_292 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThe
rmalCube_96-98_float_cmsk.img)}
coordinate system string = {
PROJCS[UTM_Zone_10N, GEOGCS[GCS_WGS_1984, DATUM[D_WGS_1984,
SPHEROID[WGS_1984, 6378137.0, 298.257223563]], PRIMEM[Greenwich, 0.0],
UNIT[Degree, 0.0174532925199433]], PROJECTION[Transverse_Mercator],
PARAMETER[False_Easting, 50.0], PARAMETER[False_Northing, 0.0],
PARAMETER[Central_Meridian, -123.0], PARAMETER[Scale_Factor, 0.9996],
PARAMETER[Latitude_Of_Origin, 0.0], UNIT[Meter, 1.0]]}
-Original Message-
From: Even Rouault [mailto:even.roua...@mines-paris.org]
Sent: October-19-12 11:53 AM
To: gdal-dev@lists.osgeo.org
Cc: Peter Willis
Subject: Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI
header
Le vendredi 19 octobre 2012 20:41:16, Peter Willis a écrit :
Hello,
I have a curious problem with 'gdalinfo' (gdal version 1.9.0 ).
I have an input ENVI file with the following .HDR entry:
data ignore value = -9.9900e+002
gdalinfo for the IMG file provides stats that exclude all values
-999.0 as expected.
I then create a masked output file using 'gdal_rasterize' from that
input file once again using -999.0 as the mask value.
'gdal_rasterize' produces an ENVI format output with an accompanying
.HDR file.
There's indeed no support currently in the ENVI driver to write the data
ignore value field.
The resulting 'gdal_rasterize' HDR *does not* contain the previous
'data ignore value' entry. This is something that should be fixed in
future versions of that utility. The work around is, of course, to
simply concatenate the 'data ignore value'
to the end of the HDR file. This is a quick step:
echomyfile.hdr
echo data ignore value = -9.9900+e2 myfile.hdr echo
myfile.hdr
I've tried that and this worked for me. Perhaps you could send your
myfile.hdr ?
LSatThermalCube_96-98_float_cmsk_landmasked.hdr
Description: Binary data
LSatThermalCube_96-98_float_cmsk_landmasked_ll.hdr
Description: Binary data
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Hello,
The one listed as 'failed' actually fails for me as it is sent .
I was experimenting by moving the value near the top of the file.
It continues to fail whether the entry is at the bottom of the file
Or where it appears in the sample that I sent to you.
Could it be a line feeds issue?
I haven't tried running unix2dos/dos2unix against the HDR files.
Does the program mind if the EOL are \13\10 vs. \10 ?
Best Regards,
Peter
-Original Message-
From: Even Rouault [mailto:even.roua...@mines-paris.org]
Sent: October-19-12 1:44 PM
To: gdal-dev@lists.osgeo.org
Cc: Peter Willis
Subject: Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Le vendredi 19 octobre 2012 22:23:49, Peter Willis a écrit :
Hello,
I am providing the failed example followed by a working example.
Let me know if there is any additional information you need.
With both headers, gdalinfo correctly reports the nodata value... But you
didn't send the .hdr files you've generated since the data ignore value is
not at their end.
Thanks,
Peter
--The Failed Header Sample--
ENVI
description = {
/web2-disk1/PR762/landsat/cubes/96_98/LSatThermalCube_96-98_float_cmsk
_land
m asked_ll.img}
samples = 2178
lines = 1841
bands = 49
header offset = 0
file type = ENVI Standard
data type = 4
interleave = bsq
byte order = 0
map info = {Geographic Lat/Lon, 1, 1, -124.258818822249,
51.6708415299984, 0.000337173258151718, 0.000337173258151718,WGS-84}
wavelength units = Unknown data ignore value = -9.9900e+002 band
names = { Band 1, Band 2, Band 3, Band 4, Band 5, Band 6, Band 7, Band
8, Band 9, Band 10, Band 11, Band 12, Band 13, Band 14, Band 15, Band
16, Band 17, Band 18, Band 19, Band 20, Band 21, Band 22, Band 23,
Band 24, Band 25, Band 26, Band 27, Band 28, Band 29, Band 30, Band
31, Band 32, Band 33, Band 34, Band 35, Band 36, Band 37, Band 38,
Band 39, Band 40, Band 41, Band 42, Band 43, Band 44, Band 45, Band
46, Band 47, Band 48, Band 49}
--The Working Header Sample--
ENVI
description = {
Band Math Result, Expression = [(float(b2 gt 0.0) * b1) + (float(b2
eq
0.0)*
(-999.0))] B1:LSatThermalCube_96-98_float_cmsk.img
B2:LSatThermalCube_96-98_float_cmsk_multi_channel_mask.img [Thu Oct 18
10:12:05 2012]}
samples = 1677
lines = 2281
bands = 49
header offset = 0
file type = ENVI Standard
data type = 4
interleave = bsq
sensor type = Unknown
byte order = 0
map info = {UTM, 1.000, 1.000, 412940.000, 5724563.000,
3.00e+001, 3.00e+001, 10, North, WGS-84, units=Meters}
wavelength units = Unknown data ignore value = -9.9900e+002 pixel
size = {3.e+001, 3.e+001, units=Meters} band names = {
Band Math (Band Math (Band Math (1996_088 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_104 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_111 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_120 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_136 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_152 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_159 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_168 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_191 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_200 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_207 Landsat-4/5 TM :Surface
Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L
SatTh
e rmalCube_96-98_float_cmsk.img),
Band Math (Band Math (Band Math (1996_216 Landsat-4/5 TM
Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Hello,
When I run :
Gdalinfo -stats FILENAME
One file (success) provides statistics calculated by excluding the no data
values.
Ie: min=0.25 max=1234.0 mean=12.0
The other file (failed) provides statistics with -999 calculated into the stats.
Ie: min = -999.0 max=1234.0 mean=-993.0
I have looked at both binary files. Each of the two files contain the mask
value.
ENVI loads both files and headers fine and reports the same no data value, for
each file, in header info just fine.
I think you could probably model the problem by generating a much smaller file
with multiple channels.
The steps happening are as follows:
1.) Original file (IN_IMG) is UTM projection ENVI format 49 bands BSQ, float32
2.) gdalwarp -of ENVI -t_srs +proj=latlong +datum=WGS84 ${IN_IMG} ${OUT_IMG}
3.) put the data ignore value back in the file:
echo $OUT_HDR
echo data ignore value = -9.9900e+002 $OUT_HDR
4.) Call gdalinfo for number of bands in original file = $BANDS
5.) loop through bands seq 1 $BANDS = $BAND
gdal_rasterize -b $BAND -burn -999 -l ${LAYER_NAME} ${MASK_SHP}
${OUT_IMG}
6.) gdalinfo -stats ${OUT_IMG}
At step #6, gdalinfo -stats reports data stats with the -999 values calculated
into the min/max/mean/stdev.
Band 2 Block=2178x1 Type=Float32, ColorInterp=Undefined
Description = Band 2
Min=-999.000 Max=12.250
Minimum=-999.000, Maximum=12.250, Mean=-946.817, StdDev=222.404
NoData Value=-999
Metadata:
STATISTICS_MINIMUM=-999
STATISTICS_MAXIMUM=12.25
STATISTICS_MEAN=-946.81678963104
STATISTICS_STDDEV=222.4036660914
If I perform gdalinfo -stats on the original UTM file the min/max/mean/stdev
exclude -999 from calculations.
Band 2 Block=1677x1 Type=Float32, ColorInterp=Undefined
Description = Band Math (Band Math (Band Math (1996_104 Landsat-4/5 TM
:Surface Temperature
Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThermalCube_96-98_float_cmsk.img)
Min=0.250 Max=12.250
Minimum=0.250, Maximum=12.250, Mean=1.379, StdDev=0.995
NoData Value=-999
Metadata:
STATISTICS_MINIMUM=0.25
STATISTICS_MAXIMUM=12.25
STATISTICS_MEAN=1.3787828985307
STATISTICS_STDDEV=0.99545771926881
I hope this helps.
Let me know if you need any more information.
Peter
-Original Message-
From: Even Rouault [mailto:even.roua...@mines-paris.org]
Sent: October-19-12 2:08 PM
To: gdal-dev@lists.osgeo.org
Cc: Peter Willis
Subject: Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Le vendredi 19 octobre 2012 22:57:13, Peter Willis a écrit :
Hello,
The one listed as 'failed' actually fails for me as it is sent .
Just to be sure we are talking about the same thing, what do you call fail
exactly ? Is it that gdalinfo doesn't report NoData Value=-999 for the bands ?
Just for reference, I've create a fake
LSatThermalCube_96-98_float_cmsk_landmasked_ll.img and gdalinfo on it
reports :
Driver: ENVI/ENVI .hdr Labelled
Files: LSatThermalCube_96-98_float_cmsk_landmasked_ll.img
LSatThermalCube_96-98_float_cmsk_landmasked_ll.hdr
Size is 2178, 1841
Coordinate System is:
GEOGCS[WGS 84,
DATUM[WGS_1984,
SPHEROID[WGS 84,6378137,298.257223563,
AUTHORITY[EPSG,7030]],
TOWGS84[0,0,0,0,0,0,0],
AUTHORITY[EPSG,6326]],
PRIMEM[Greenwich,0,
AUTHORITY[EPSG,8901]],
UNIT[degree,0.0174532925199433,
AUTHORITY[EPSG,9108]],
AUTHORITY[EPSG,4326]]
Origin = (-124.258818822248998,51.670841529998398)
Pixel Size = (0.000337173258152,-0.000337173258152)
Metadata:
[ cut ]
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left (-124.2588188, 51.6708415) (124d15'31.75W, 51d40'15.03N) Lower
Left (-124.2588188, 51.0501056) (124d15'31.75W, 51d 3' 0.38N) Upper Right
(-123.5244555, 51.6708415) (123d31'28.04W, 51d40'15.03N) Lower Right
(-123.5244555, 51.0501056) (123d31'28.04W, 51d 3' 0.38N)
Center (-123.8916371, 51.3604735) (123d53'29.89W, 51d21'37.70N)
Band 1 Block=2178x1 Type=Float32, ColorInterp=Undefined
Description = Band 1
NoData Value=-999
Band 2 Block=2178x1 Type=Float32, ColorInterp=Undefined
Description = Band 2
NoData Value=-999
[... cut ... ]
Band 49 Block=2178x1 Type=Float32, ColorInterp=Undefined
Description = Band 49
NoData Value=-999
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Hello, I am seeing that you are correct that the 'no data value' is being read in each case. Perhaps my interpretation of the problem appears incorrect. Look, however at the 'Metadata:' statistics. Both files have assigned -999 to 'No Data Value' but'STATISTICS_MINIMUM' for one file shows -999 (??). This means the values are not being ignored for the purpose of calculating statistics. --THE BAD FILE-- Band 2 Block=2178x1 Type=Float32, ColorInterp=Undefined Description = Band 2 Min=-999.000 Max=12.250 Minimum=-999.000, Maximum=12.250, Mean=-946.817, StdDev=222.404 NoData Value=-999 Metadata: STATISTICS_MINIMUM=-999 STATISTICS_MAXIMUM=12.25 STATISTICS_MEAN=-946.81678963104 STATISTICS_STDDEV=222.4036660914 ---THE GOOD FILE--- Band 2 Block=1677x1 Type=Float32, ColorInterp=Undefined Description = Band Math (Band Math (Band Math (1996_104 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThermalCube_96-98_float_cmsk.img) Min=0.250 Max=12.250 Minimum=0.250, Maximum=12.250, Mean=1.379, StdDev=0.995 NoData Value=-999 Metadata: STATISTICS_MINIMUM=0.25 STATISTICS_MAXIMUM=12.25 STATISTICS_MEAN=1.3787828985307 STATISTICS_STDDEV=0.99545771926881 Note the difference in stats calculations. Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Hello, Good news. Removing the '.aux.xml ' file and again running gdalinfo -statsindeed fixes the problem. I now get the expected statistics values. Thank you for your help. Best Regards, Peter Le vendredi 19 octobre 2012 23:57:01, Peter Willis a écrit : Hello, At step #6, gdalinfo -stats reports data stats with the -999 values calculated into the min/max/mean/stdev. Band 2 Block=2178x1 Type=Float32, ColorInterp=Undefined Description = Band 2 Min=-999.000 Max=12.250 Minimum=-999.000, Maximum=12.250, Mean=-946.817, StdDev=222.404 NoData Value=-999 Metadata: STATISTICS_MINIMUM=-999 STATISTICS_MAXIMUM=12.25 STATISTICS_MEAN=-946.81678963104 STATISTICS_STDDEV=222.4036660914 Ok, indeed that's not expected. Well, perhaps you could first check if there is no .aux.xml file that would contain bad values. And if so, delete it before running again gdalinfo -stats. Other than that, I've no more idea without having access to the .img file itself. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector [SEC=UNCLASSIFIED]
Hello,
Thanks, that worked.
Looks like I missed the in-place band number selection flag ( -b ).
Does anyone know if I can direct output to another file, or do
I always need to copy the original file and 'burn' the values into an
existing duplicate file.
I guess it's just as easy either way.
Thanks again,
Peter
From: Pinner, Luke [mailto:luke.pin...@environment.gov.au]
Sent: October-17-12 4:11 PM
To: Peter Willis; gdal-dev@lists.osgeo.org
Subject: RE: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI
SHP format Vector [SEC=UNCLASSIFIED]
Perhaps something like
for i in {1..49}; do gdal_rasterize -burn -999 -b $i mask.shp
49bandenvi.dat; done
Luke
From: gdal-dev-boun...@lists.osgeo.org
[mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Peter Willis
Sent: Thursday, 18 October 2012 9:43 AM
To: gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI
SHP format Vector
Hello,
On second thoughts, this is not really what I want.
I have a SHP format polygon vector file already.
I want to use that existing vector file to mask the ENVI format BSQ file
through all (49) channels.
(ie: 'mask' meaning set any values inside vector polygons to a specific
value within the output ENVI BSQ file )
Perhaps I'm missing something.
Peter
From: fwarmer...@gmail.com [mailto:fwarmer...@gmail.com] On Behalf Of Frank
Warmerdam
Sent: October-17-12 3:22 PM
To: Peter Willis
Cc: gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI
SHP format Vector
Peter,
I'm guessing you are using the gdal_polygonize.py script
for masking. Is that right? I am not aware of any particular
reason this shouldn't work for any update in place format
(ie. shows rw+ in the gdalinfo --formats list). ENVI is in
this list.
BTW, before we do a lot of work to investigate this you might
want to see if the problem persists with GDAL 1.9. GDAL 1.6.3
is getting pretty antique.
Best regards,
Frank
On Wed, Oct 17, 2012 at 2:59 PM, Peter Willis pwil...@aslenv.com wrote:
Hello,
Is it possible to use ESRI SHP polygon file to mask
ENVI BSQ img format files with more than 1 band?
I can mask a GTiff file using the SHP but ENVI file does not
appear to work. Gdal version is 1.6.3 .
The documentation appears unclear for any raster that is not
specifically GTiff.
Thanks,
Peter
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
--
---+
--
I set the clouds in motion - turn up | Frank Warmerdam,
warmer...@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush| Geospatial Software Developer
If you have received this transmission in error please notify us immediately
by return e-mail and delete all copies. If this e-mail or any attachments
have been sent to you in error, that error does not constitute waiver of any
confidentiality, privilege or copyright in respect of information in the
e-mail or attachments. Please consider the environment before printing this
email.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector
Hello, Is it possible to use ESRI SHP polygon file to mask ENVI BSQ img format files with more than 1 band? I can mask a GTiff file using the SHP but ENVI file does not appear to work. Gdal version is 1.6.3 . The documentation appears unclear for any raster that is not specifically GTiff. Thanks, Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector
Hello, I was using gdal_rasterize directly rather than gdal_polygonize.py . .No gdal_poygonize.py found for my revision.. I was forgetful of -formats via gdalinfo. This shows rw+ for ENVI format which is what I need. Upon looking I find that I have already installed revision 1.9 in another /usr/local directory. Perhaps I should use the more recent revision for attempting this. Thanks for the pointers, Peter From: fwarmer...@gmail.com [mailto:fwarmer...@gmail.com] On Behalf Of Frank Warmerdam Sent: October-17-12 3:22 PM To: Peter Willis Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector Peter, I'm guessing you are using the gdal_polygonize.py script for masking. Is that right? I am not aware of any particular reason this shouldn't work for any update in place format (ie. shows rw+ in the gdalinfo --formats list). ENVI is in this list. BTW, before we do a lot of work to investigate this you might want to see if the problem persists with GDAL 1.9. GDAL 1.6.3 is getting pretty antique. Best regards, Frank On Wed, Oct 17, 2012 at 2:59 PM, Peter Willis pwil...@aslenv.com wrote: Hello, Is it possible to use ESRI SHP polygon file to mask ENVI BSQ img format files with more than 1 band? I can mask a GTiff file using the SHP but ENVI file does not appear to work. Gdal version is 1.6.3 . The documentation appears unclear for any raster that is not specifically GTiff. Thanks, Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- ---+ -- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector
Hello, On second thoughts, this is not really what I want. I have a SHP format polygon vector file already. I want to use that existing vector file to mask the ENVI format BSQ file through all (49) channels. (ie: 'mask' meaning set any values inside vector polygons to a specific value within the output ENVI BSQ file ) Perhaps I'm missing something. Peter From: fwarmer...@gmail.com [mailto:fwarmer...@gmail.com] On Behalf Of Frank Warmerdam Sent: October-17-12 3:22 PM To: Peter Willis Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector Peter, I'm guessing you are using the gdal_polygonize.py script for masking. Is that right? I am not aware of any particular reason this shouldn't work for any update in place format (ie. shows rw+ in the gdalinfo --formats list). ENVI is in this list. BTW, before we do a lot of work to investigate this you might want to see if the problem persists with GDAL 1.9. GDAL 1.6.3 is getting pretty antique. Best regards, Frank On Wed, Oct 17, 2012 at 2:59 PM, Peter Willis pwil...@aslenv.com wrote: Hello, Is it possible to use ESRI SHP polygon file to mask ENVI BSQ img format files with more than 1 band? I can mask a GTiff file using the SHP but ENVI file does not appear to work. Gdal version is 1.6.3 . The documentation appears unclear for any raster that is not specifically GTiff. Thanks, Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- ---+ -- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
