[gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector
Hello, Is it possible to use ESRI SHP polygon file to mask ENVI BSQ img format files with more than 1 band? I can mask a GTiff file using the SHP but ENVI file does not appear to work. Gdal version is 1.6.3 . The documentation appears unclear for any raster that is not specifically GTiff. Thanks, Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector
Hello, I was using gdal_rasterize directly rather than gdal_polygonize.py . .No gdal_poygonize.py found for my revision.. I was forgetful of -formats via gdalinfo. This shows rw+ for ENVI format which is what I need. Upon looking I find that I have already installed revision 1.9 in another /usr/local directory. Perhaps I should use the more recent revision for attempting this. Thanks for the pointers, Peter From: fwarmer...@gmail.com [mailto:fwarmer...@gmail.com] On Behalf Of Frank Warmerdam Sent: October-17-12 3:22 PM To: Peter Willis Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector Peter, I'm guessing you are using the gdal_polygonize.py script for masking. Is that right? I am not aware of any particular reason this shouldn't work for any update in place format (ie. shows rw+ in the gdalinfo --formats list). ENVI is in this list. BTW, before we do a lot of work to investigate this you might want to see if the problem persists with GDAL 1.9. GDAL 1.6.3 is getting pretty antique. Best regards, Frank On Wed, Oct 17, 2012 at 2:59 PM, Peter Willis pwil...@aslenv.com wrote: Hello, Is it possible to use ESRI SHP polygon file to mask ENVI BSQ img format files with more than 1 band? I can mask a GTiff file using the SHP but ENVI file does not appear to work. Gdal version is 1.6.3 . The documentation appears unclear for any raster that is not specifically GTiff. Thanks, Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- ---+ -- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector
Hello, On second thoughts, this is not really what I want. I have a SHP format polygon vector file already. I want to use that existing vector file to mask the ENVI format BSQ file through all (49) channels. (ie: 'mask' meaning set any values inside vector polygons to a specific value within the output ENVI BSQ file ) Perhaps I'm missing something. Peter From: fwarmer...@gmail.com [mailto:fwarmer...@gmail.com] On Behalf Of Frank Warmerdam Sent: October-17-12 3:22 PM To: Peter Willis Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector Peter, I'm guessing you are using the gdal_polygonize.py script for masking. Is that right? I am not aware of any particular reason this shouldn't work for any update in place format (ie. shows rw+ in the gdalinfo --formats list). ENVI is in this list. BTW, before we do a lot of work to investigate this you might want to see if the problem persists with GDAL 1.9. GDAL 1.6.3 is getting pretty antique. Best regards, Frank On Wed, Oct 17, 2012 at 2:59 PM, Peter Willis pwil...@aslenv.com wrote: Hello, Is it possible to use ESRI SHP polygon file to mask ENVI BSQ img format files with more than 1 band? I can mask a GTiff file using the SHP but ENVI file does not appear to work. Gdal version is 1.6.3 . The documentation appears unclear for any raster that is not specifically GTiff. Thanks, Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- ---+ -- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector [SEC=UNCLASSIFIED]
Hello, Thanks, that worked. Looks like I missed the in-place band number selection flag ( -b ). Does anyone know if I can direct output to another file, or do I always need to copy the original file and 'burn' the values into an existing duplicate file. I guess it's just as easy either way. Thanks again, Peter From: Pinner, Luke [mailto:luke.pin...@environment.gov.au] Sent: October-17-12 4:11 PM To: Peter Willis; gdal-dev@lists.osgeo.org Subject: RE: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector [SEC=UNCLASSIFIED] Perhaps something like for i in {1..49}; do gdal_rasterize -burn -999 -b $i mask.shp 49bandenvi.dat; done Luke From: gdal-dev-boun...@lists.osgeo.org [mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Peter Willis Sent: Thursday, 18 October 2012 9:43 AM To: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector Hello, On second thoughts, this is not really what I want. I have a SHP format polygon vector file already. I want to use that existing vector file to mask the ENVI format BSQ file through all (49) channels. (ie: 'mask' meaning set any values inside vector polygons to a specific value within the output ENVI BSQ file ) Perhaps I'm missing something. Peter From: fwarmer...@gmail.com [mailto:fwarmer...@gmail.com] On Behalf Of Frank Warmerdam Sent: October-17-12 3:22 PM To: Peter Willis Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Using gdal_rasterize to mask ENVI file Using ESRI SHP format Vector Peter, I'm guessing you are using the gdal_polygonize.py script for masking. Is that right? I am not aware of any particular reason this shouldn't work for any update in place format (ie. shows rw+ in the gdalinfo --formats list). ENVI is in this list. BTW, before we do a lot of work to investigate this you might want to see if the problem persists with GDAL 1.9. GDAL 1.6.3 is getting pretty antique. Best regards, Frank On Wed, Oct 17, 2012 at 2:59 PM, Peter Willis pwil...@aslenv.com wrote: Hello, Is it possible to use ESRI SHP polygon file to mask ENVI BSQ img format files with more than 1 band? I can mask a GTiff file using the SHP but ENVI file does not appear to work. Gdal version is 1.6.3 . The documentation appears unclear for any raster that is not specifically GTiff. Thanks, Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- ---+ -- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer If you have received this transmission in error please notify us immediately by return e-mail and delete all copies. If this e-mail or any attachments have been sent to you in error, that error does not constitute waiver of any confidentiality, privilege or copyright in respect of information in the e-mail or attachments. Please consider the environment before printing this email. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Hello, I have a curious problem with 'gdalinfo' (gdal version 1.9.0 ). I have an input ENVI file with the following .HDR entry: data ignore value = -9.9900e+002 gdalinfo for the IMG file provides stats that exclude all values -999.0 as expected. I then create a masked output file using 'gdal_rasterize' from that input file once again using -999.0 as the mask value. 'gdal_rasterize' produces an ENVI format output with an accompanying .HDR file. The resulting 'gdal_rasterize' HDR *does not* contain the previous 'data ignore value' entry. This is something that should be fixed in future versions of that utility. The work around is, of course, to simply concatenate the 'data ignore value' to the end of the HDR file. This is a quick step: echomyfile.hdr echo data ignore value = -9.9900+e2 myfile.hdr echomyfile.hdr [NOTE: There are some other issues with ENVI HDR generation in GDAL that should also be addressed but I won't go into those here.] PROBLEM: gdalinfo does not parse the value for the concatenated header entry if it is placed at the end of the HDR file. In fact, the HDR file may be edited to move the entry to a location near the top of the file whereby it remains unacknowledged by gdalinfo. QUESTION: Does gdalinfo expect the header entries to be in a specific order? I have appended some parsing information to the end of this email. You are probably aware of most, if not all, of this information. Thanks, Peter --ENVI TEXT HEADERS--- The ENVI header file is not an 'ordered' data file. The entries may appear in any order. The generic format for entries is: NAME = VALUE[EOL] Equals '=' normally appears on the same line as NAME without intervening carriage return characters. It is normal for '=' to be preceded by one space character although not necessary. Equals '=' is considered to be the primary delimiter between NAME and VALUE. ---NAMES--- 'Name' may be any string without carriage returns, and the NAME may include space characters. All names (and values) are trimmed of preceding and trailing white space characters at parse time. TAB'\t' is not normally used as whitespace but is parsed as though it were a space ' ' character. ---VALUES--- 'VALUE' may be a singular value (ie: a number or a string with no whitespace ) or a 'list' bounded by curly braces { }. Examples of singular string values would be: Interleave = BSQ Sensor type = Unknown Wavelength units = nanometers where 'BSQ' , 'Unknown', and 'nanometers' are singular strings containing no whitespace characters. In the context of the VALUE strings containing necessary whitespace should be encoded as a list. String values may contain carriage return characters. The comma ',' is considered to be the default delimiter for list type data. Ie: My List = { some string data , another string data , yet another data string , etc data strings ad infinitum } Number formats associated with VALUE Singular Integers: My Integer = 9 My Integer = -99 My Integer = 0 Singular Reals: My Real = 0.9 My Real = .9 My Real = -9.0+e8 Lists of numbers: My Integer List = { 1 , 2 , 3, 4 , 5 , 6 } My Real List = { 1.0 , -2.00e+2 , 3.0e-4, 4.0 , .5 , 6.1 } ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
-98_float.img):LSatThe rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1998_212 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThe rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1998_221 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThe rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1998_260 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThe rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1998_269 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThe rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1998_292 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThe rmalCube_96-98_float_cmsk.img)} coordinate system string = { PROJCS[UTM_Zone_10N, GEOGCS[GCS_WGS_1984, DATUM[D_WGS_1984, SPHEROID[WGS_1984, 6378137.0, 298.257223563]], PRIMEM[Greenwich, 0.0], UNIT[Degree, 0.0174532925199433]], PROJECTION[Transverse_Mercator], PARAMETER[False_Easting, 50.0], PARAMETER[False_Northing, 0.0], PARAMETER[Central_Meridian, -123.0], PARAMETER[Scale_Factor, 0.9996], PARAMETER[Latitude_Of_Origin, 0.0], UNIT[Meter, 1.0]]} -Original Message- From: Even Rouault [mailto:even.roua...@mines-paris.org] Sent: October-19-12 11:53 AM To: gdal-dev@lists.osgeo.org Cc: Peter Willis Subject: Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header Le vendredi 19 octobre 2012 20:41:16, Peter Willis a écrit : Hello, I have a curious problem with 'gdalinfo' (gdal version 1.9.0 ). I have an input ENVI file with the following .HDR entry: data ignore value = -9.9900e+002 gdalinfo for the IMG file provides stats that exclude all values -999.0 as expected. I then create a masked output file using 'gdal_rasterize' from that input file once again using -999.0 as the mask value. 'gdal_rasterize' produces an ENVI format output with an accompanying .HDR file. There's indeed no support currently in the ENVI driver to write the data ignore value field. The resulting 'gdal_rasterize' HDR *does not* contain the previous 'data ignore value' entry. This is something that should be fixed in future versions of that utility. The work around is, of course, to simply concatenate the 'data ignore value' to the end of the HDR file. This is a quick step: echomyfile.hdr echo data ignore value = -9.9900+e2 myfile.hdr echo myfile.hdr I've tried that and this worked for me. Perhaps you could send your myfile.hdr ? LSatThermalCube_96-98_float_cmsk_landmasked.hdr Description: Binary data LSatThermalCube_96-98_float_cmsk_landmasked_ll.hdr Description: Binary data ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Hello, The one listed as 'failed' actually fails for me as it is sent . I was experimenting by moving the value near the top of the file. It continues to fail whether the entry is at the bottom of the file Or where it appears in the sample that I sent to you. Could it be a line feeds issue? I haven't tried running unix2dos/dos2unix against the HDR files. Does the program mind if the EOL are \13\10 vs. \10 ? Best Regards, Peter -Original Message- From: Even Rouault [mailto:even.roua...@mines-paris.org] Sent: October-19-12 1:44 PM To: gdal-dev@lists.osgeo.org Cc: Peter Willis Subject: Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header Le vendredi 19 octobre 2012 22:23:49, Peter Willis a écrit : Hello, I am providing the failed example followed by a working example. Let me know if there is any additional information you need. With both headers, gdalinfo correctly reports the nodata value... But you didn't send the .hdr files you've generated since the data ignore value is not at their end. Thanks, Peter --The Failed Header Sample-- ENVI description = { /web2-disk1/PR762/landsat/cubes/96_98/LSatThermalCube_96-98_float_cmsk _land m asked_ll.img} samples = 2178 lines = 1841 bands = 49 header offset = 0 file type = ENVI Standard data type = 4 interleave = bsq byte order = 0 map info = {Geographic Lat/Lon, 1, 1, -124.258818822249, 51.6708415299984, 0.000337173258151718, 0.000337173258151718,WGS-84} wavelength units = Unknown data ignore value = -9.9900e+002 band names = { Band 1, Band 2, Band 3, Band 4, Band 5, Band 6, Band 7, Band 8, Band 9, Band 10, Band 11, Band 12, Band 13, Band 14, Band 15, Band 16, Band 17, Band 18, Band 19, Band 20, Band 21, Band 22, Band 23, Band 24, Band 25, Band 26, Band 27, Band 28, Band 29, Band 30, Band 31, Band 32, Band 33, Band 34, Band 35, Band 36, Band 37, Band 38, Band 39, Band 40, Band 41, Band 42, Band 43, Band 44, Band 45, Band 46, Band 47, Band 48, Band 49} --The Working Header Sample-- ENVI description = { Band Math Result, Expression = [(float(b2 gt 0.0) * b1) + (float(b2 eq 0.0)* (-999.0))] B1:LSatThermalCube_96-98_float_cmsk.img B2:LSatThermalCube_96-98_float_cmsk_multi_channel_mask.img [Thu Oct 18 10:12:05 2012]} samples = 1677 lines = 2281 bands = 49 header offset = 0 file type = ENVI Standard data type = 4 interleave = bsq sensor type = Unknown byte order = 0 map info = {UTM, 1.000, 1.000, 412940.000, 5724563.000, 3.00e+001, 3.00e+001, 10, North, WGS-84, units=Meters} wavelength units = Unknown data ignore value = -9.9900e+002 pixel size = {3.e+001, 3.e+001, units=Meters} band names = { Band Math (Band Math (Band Math (1996_088 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_104 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_111 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_120 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_136 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_152 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_159 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_168 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_191 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_200 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_207 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):L SatTh e rmalCube_96-98_float_cmsk.img), Band Math (Band Math (Band Math (1996_216 Landsat-4/5 TM
Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Hello, When I run : Gdalinfo -stats FILENAME One file (success) provides statistics calculated by excluding the no data values. Ie: min=0.25 max=1234.0 mean=12.0 The other file (failed) provides statistics with -999 calculated into the stats. Ie: min = -999.0 max=1234.0 mean=-993.0 I have looked at both binary files. Each of the two files contain the mask value. ENVI loads both files and headers fine and reports the same no data value, for each file, in header info just fine. I think you could probably model the problem by generating a much smaller file with multiple channels. The steps happening are as follows: 1.) Original file (IN_IMG) is UTM projection ENVI format 49 bands BSQ, float32 2.) gdalwarp -of ENVI -t_srs +proj=latlong +datum=WGS84 ${IN_IMG} ${OUT_IMG} 3.) put the data ignore value back in the file: echo $OUT_HDR echo data ignore value = -9.9900e+002 $OUT_HDR 4.) Call gdalinfo for number of bands in original file = $BANDS 5.) loop through bands seq 1 $BANDS = $BAND gdal_rasterize -b $BAND -burn -999 -l ${LAYER_NAME} ${MASK_SHP} ${OUT_IMG} 6.) gdalinfo -stats ${OUT_IMG} At step #6, gdalinfo -stats reports data stats with the -999 values calculated into the min/max/mean/stdev. Band 2 Block=2178x1 Type=Float32, ColorInterp=Undefined Description = Band 2 Min=-999.000 Max=12.250 Minimum=-999.000, Maximum=12.250, Mean=-946.817, StdDev=222.404 NoData Value=-999 Metadata: STATISTICS_MINIMUM=-999 STATISTICS_MAXIMUM=12.25 STATISTICS_MEAN=-946.81678963104 STATISTICS_STDDEV=222.4036660914 If I perform gdalinfo -stats on the original UTM file the min/max/mean/stdev exclude -999 from calculations. Band 2 Block=1677x1 Type=Float32, ColorInterp=Undefined Description = Band Math (Band Math (Band Math (1996_104 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThermalCube_96-98_float_cmsk.img) Min=0.250 Max=12.250 Minimum=0.250, Maximum=12.250, Mean=1.379, StdDev=0.995 NoData Value=-999 Metadata: STATISTICS_MINIMUM=0.25 STATISTICS_MAXIMUM=12.25 STATISTICS_MEAN=1.3787828985307 STATISTICS_STDDEV=0.99545771926881 I hope this helps. Let me know if you need any more information. Peter -Original Message- From: Even Rouault [mailto:even.roua...@mines-paris.org] Sent: October-19-12 2:08 PM To: gdal-dev@lists.osgeo.org Cc: Peter Willis Subject: Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header Le vendredi 19 octobre 2012 22:57:13, Peter Willis a écrit : Hello, The one listed as 'failed' actually fails for me as it is sent . Just to be sure we are talking about the same thing, what do you call fail exactly ? Is it that gdalinfo doesn't report NoData Value=-999 for the bands ? Just for reference, I've create a fake LSatThermalCube_96-98_float_cmsk_landmasked_ll.img and gdalinfo on it reports : Driver: ENVI/ENVI .hdr Labelled Files: LSatThermalCube_96-98_float_cmsk_landmasked_ll.img LSatThermalCube_96-98_float_cmsk_landmasked_ll.hdr Size is 2178, 1841 Coordinate System is: GEOGCS[WGS 84, DATUM[WGS_1984, SPHEROID[WGS 84,6378137,298.257223563, AUTHORITY[EPSG,7030]], TOWGS84[0,0,0,0,0,0,0], AUTHORITY[EPSG,6326]], PRIMEM[Greenwich,0, AUTHORITY[EPSG,8901]], UNIT[degree,0.0174532925199433, AUTHORITY[EPSG,9108]], AUTHORITY[EPSG,4326]] Origin = (-124.258818822248998,51.670841529998398) Pixel Size = (0.000337173258152,-0.000337173258152) Metadata: [ cut ] Image Structure Metadata: INTERLEAVE=BAND Corner Coordinates: Upper Left (-124.2588188, 51.6708415) (124d15'31.75W, 51d40'15.03N) Lower Left (-124.2588188, 51.0501056) (124d15'31.75W, 51d 3' 0.38N) Upper Right (-123.5244555, 51.6708415) (123d31'28.04W, 51d40'15.03N) Lower Right (-123.5244555, 51.0501056) (123d31'28.04W, 51d 3' 0.38N) Center (-123.8916371, 51.3604735) (123d53'29.89W, 51d21'37.70N) Band 1 Block=2178x1 Type=Float32, ColorInterp=Undefined Description = Band 1 NoData Value=-999 Band 2 Block=2178x1 Type=Float32, ColorInterp=Undefined Description = Band 2 NoData Value=-999 [... cut ... ] Band 49 Block=2178x1 Type=Float32, ColorInterp=Undefined Description = Band 49 NoData Value=-999 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Hello, I am seeing that you are correct that the 'no data value' is being read in each case. Perhaps my interpretation of the problem appears incorrect. Look, however at the 'Metadata:' statistics. Both files have assigned -999 to 'No Data Value' but'STATISTICS_MINIMUM' for one file shows -999 (??). This means the values are not being ignored for the purpose of calculating statistics. --THE BAD FILE-- Band 2 Block=2178x1 Type=Float32, ColorInterp=Undefined Description = Band 2 Min=-999.000 Max=12.250 Minimum=-999.000, Maximum=12.250, Mean=-946.817, StdDev=222.404 NoData Value=-999 Metadata: STATISTICS_MINIMUM=-999 STATISTICS_MAXIMUM=12.25 STATISTICS_MEAN=-946.81678963104 STATISTICS_STDDEV=222.4036660914 ---THE GOOD FILE--- Band 2 Block=1677x1 Type=Float32, ColorInterp=Undefined Description = Band Math (Band Math (Band Math (1996_104 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThermalCube_96-98_float_cmsk.img) Min=0.250 Max=12.250 Minimum=0.250, Maximum=12.250, Mean=1.379, StdDev=0.995 NoData Value=-999 Metadata: STATISTICS_MINIMUM=0.25 STATISTICS_MAXIMUM=12.25 STATISTICS_MEAN=1.3787828985307 STATISTICS_STDDEV=0.99545771926881 Note the difference in stats calculations. Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header
Hello, Good news. Removing the '.aux.xml ' file and again running gdalinfo -statsindeed fixes the problem. I now get the expected statistics values. Thank you for your help. Best Regards, Peter Le vendredi 19 octobre 2012 23:57:01, Peter Willis a écrit : Hello, At step #6, gdalinfo -stats reports data stats with the -999 values calculated into the min/max/mean/stdev. Band 2 Block=2178x1 Type=Float32, ColorInterp=Undefined Description = Band 2 Min=-999.000 Max=12.250 Minimum=-999.000, Maximum=12.250, Mean=-946.817, StdDev=222.404 NoData Value=-999 Metadata: STATISTICS_MINIMUM=-999 STATISTICS_MAXIMUM=12.25 STATISTICS_MEAN=-946.81678963104 STATISTICS_STDDEV=222.4036660914 Ok, indeed that's not expected. Well, perhaps you could first check if there is no .aux.xml file that would contain bad values. And if so, delete it before running again gdalinfo -stats. Other than that, I've no more idea without having access to the .img file itself. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Is it possible to ehance input channels separately with gdal_translate ?
Hello, I have some input data where the dynamic range and offset of each channel are distinct. This means that applying the same linear enhancement based on data range to my selected RGB output bands will not produce an optimal visual enhancement. -scale smin smax dmin dmax only provides the same linear enhancement for all selected input channels (bands). Also the default causes gdal_translate to calculate the enhancement from the data which is also the wrong thing to do owing to the relative changeability of albedo from one scene to another due to the possible presence or absence of cloud, snow, or sand, vs. water or coal piles (light vs. dark) etc.. and also the relative difference in dynamic range between channels. QUESTION: Is there a way to enhance my input channels separately in a single gdal_translate call, or do I need to make multiple passes? Thanks Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] How Can I gdalwarp From One Image to a Larger Spatial Area While Leaving Missing Data Blank in the Destination?
Hello, Sorry about the title but it's a bit of a bear finding answers if the headers don't show the actual topic. But I digress. My Issue: I have an input image that covers coordinates 0,0 to 450,350 UTM zone 13 North and the pixel size is 1 meter. I want to mosaic pixels 0,0 to 25,25 into a coverage of UTM coordinates -25,-25 : 25, 25 at the output in 1 meter pixels. This should show a 25x25 square portion of the image in one corner of the 50x50 output image. 'gdal_translate' will not do this because -25,-25 are outside the bounds of the original image. I have tried the following with gdalwarp: gdalwarp -overwrite -te -25 -25 25 25 -tr 1 1 ./input.tif ./output.tif This does not work. I get the following error: ERROR 1: Unable to compute a transformation between pixel/line and georeferenced coordinates for ./input.tif. There is no affine transformation and no GCPs. There is geography applied to the input file. What am I doing wrong ? Thanks for any help. Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] How Can I gdalwarp From One Image to a Larger Spatial Area While Leaving Missing Data Blank in the Destination?
Hello, Thank you Jukka, that worked. I just shifted the coordinate of the origin to another location and everything worked. Peter -Original Message- From: gdal-dev-boun...@lists.osgeo.org [mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Jukka Rahkonen Sent: February-05-14 5:03 AM To: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] How Can I gdalwarp From One Image to a Larger Spatial Area While Leaving Missing Data Blank in the Destination? Peter Willis pwillis at aslenv.com writes: Hello, Sorry about the title but it's a bit of a bear finding answers if the headers don't show the actual topic. But I digress. My Issue: I have an input image that covers coordinates 0,0 to 450,350 UTM zone 13 North and the pixel size is 1 meter. I want to mosaic pixels 0,0 to 25,25 into a coverage of UTM coordinates -25,-25 : 25, 25 at the output in 1 meter pixels. This should show a 25x25 square portion of the image in one corner of the 50x50 output image. 'gdal_translate' will not do this because -25,-25 are outside the bounds of the original image. I have tried the following with gdalwarp: gdalwarp -overwrite -te -25 -25 25 25 -tr 1 1 ./input.tif ./output.tif This does not work. I get the following error: ERROR 1: Unable to compute a transformation between pixel/line and georeferenced coordinates for ./input.tif. There is no affine transformation and no GCPs. There is geography applied to the input file. What am I doing wrong ? Hi, This is funny. GDAL seems not to believe in images with origo at 0,0. I tested with some dummy png file gdal_translate -of gtiff -co tfw=yes -co profile=baseline test.png test.png test.tif Input file size is 588, 321 0...10...20...30...40...50...60...70...80...90...100 - done. It does not create a tfw file. Now this fails gdalwarp test.tif test2.tif ERROR 1: Unable to compute a transformation between pixel/line and georeferenced coordinates for test.tif. There is no affine transformation and no GCPs. I made tfw file test.tfw by hand as 1 0 0 -1 0 0 and then this is successful gdalwarp test.tif test2.tif Creating output file that is 588P x 321L. Processing input file test.tif. 0...10...20...30...40...50...60...70...80...90...100 - done. and also what you want to do gdalwarp -te -1000 -1000 1000 1000 test.tif testi3.tif Creating output file that is 2000P x 2000L. Processing input file test.tif. 0...10...20...30...40...50...60...70...80...90...100 - done. So I feel there is a bug if corner coordinates are 0,0. Please verify with a handwritten tfw file and images with origo at, let's say, 1,1. -Jukka Rahkonen- Thanks for any help. Peter ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] What Statistics Are Returned By GDAL.Dataset.GetRasterBand(...).GetStatistics(...) ?
Hello, I am using osgeo.gdal python module to load and get stats from image bands. While working with the, python, 'GetStatistics(...)' function I noticed that the returned values seemed a bit off. A snippet of code is shown below. I compared the statistics returned in python by the function to band statistics in ENVI image processing package for the same input file. The results are shown below. I am wondering if I am missing information regarding some key data filtering feature in GDAL GetStatistics. What statistics are GDAL returning? # ## PYTHON CODE # src_ds = gdal.Open(image_path ) for bnd in range(1, src_ds.RasterCount+1 ): item = {} print("[ GETTING BAND ]: ", bnd) srcband=None srcband = src_ds.GetRasterBand(bnd) if srcband is None: continue bnddata=srcband.ReadAsArray() stats = srcband.GetStatistics( 0, 1 ) if stats is None: continue else: print('put the values in a dictionary') ##GDAL stats## #item['min'] = stats[0] #item['max'] = stats[1] #item['mean'] = stats[2] #item['stdev'] = stats[3] ##numpy array stats## #item['min'] = bnddata.min() #item['max'] = bnddata.max() #item['mean'] = bnddata.mean() #item['stdev'] = bnddata.std() ### #END PYTHON CODE ### /*.. ..*/ ENVI Statistics test returns these vales for each channel ChannelMinMax MeanStdev Band 1 0.00 0.441641 0.135938 0.095007 Band 2 0.00 0.42 0.134556 0.096385 Band 3 0.00 0.512614 0.143145 0.108702 Band 4 0.00 0.574203 0.159381 0.128212 Band 5 0.00 1.286870 0.190917 0.159562 Band 6 0.00 1.368695 0.218191 0.191100 Band 7 0.00 1.208142 0.179098 0.158407 /*.. ..*/ GDAL GetStatistics in Python 3.0 Returns [ {'min': 0.10543035715818, 'max': 0.35029646754265, 'mean': 0.19733288299107, 'stdev': 0.03141020073449}, {'min': 0.087364979088306, 'max': 0.36481207609177, 'mean': 0.19531263034662, 'stdev': 0.040193729911813}, {'min': 0.066563792526722, 'max': 0.40562310814857, 'mean': 0.20773722116063, 'stdev': 0.060965329408999}, {'min': 0.057971999049187, 'max': 0.49459338188171, 'mean': 0.23126556845189, 'stdev': 0.084854378172706}, {'min': 0.052771702408791, 'max': 0.57949388027191, 'mean': 0.27695694029464, 'stdev': 0.11419731959916}, {'min': 0.033055797219276, 'max': 0.67045384645462, 'mean': 0.31645853188616, 'stdev': 0.14756091787453} ] /*.. ..*/ Using NumPy Stats Functions in Python 3.0 np.min() np.max() np.mean() np.std() Return the following values after converting the previous GDAL bands to 'NumPy.Array' and using the NumPy functions (Values look similar to ENVI) [ {'min': 0.0, 'max': 0.44164082, 'mean': 0.13593808, 'stdev': 0.09500719}, {'min': 0.0, 'max': 0.4158, 'mean': 0.13455594, 'stdev': 0.09638489}, {'min': 0.0, 'max': 0.51261353, 'mean': 0.14314489, 'stdev': 0.108702265}, {'min': 0.0, 'max': 0.57420313, 'mean': 0.15938139, 'stdev': 0.12821245}, {'min': 0.0, 'max': 1.2868699, 'mean': 0.19091722, 'stdev': 0.15956147}, {'min': 0.0, 'max': 1.3686954, 'mean': 0.21819061, 'stdev': 0.1911003} ] ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev