Ivan, This sounds very much like the classic problem with HDF and netCDF where the formats are extremely flexible and allow data to be stored in a variety of ways, but the data providers could not all agree on the same way of doing it. Different providers adopted different conventions such as HDF-EOS. GDAL is designed to work with some products but not all of them (see http://www.gdal.org/frmt_hdf4.html, look under "Georeference").
The solution to this might be to modify the HDF driver to recognize and read the particular HDF "attributes" that are used in this particular file. For example, it could look for dimensions called "longitude" and "latitude" and then assume they are x and y respectively. But this approach can get very tedious: do you look for "longitude", "Longitude", "lon", "LON", "long", etc? I have seen at least all of those. Where do you stop? Perhaps the maintainers of the GDAL HDF driver could comment on their general philosophy of recognizing different attributes or conventions. Because this can be very tedious, they might not want to implement a solution for every HDF that someone happens to find, but perhaps if you prepared a patch that works for your particular HDFs, they would accept it. I have no idea and am curious about their strategy. pyhdf can work with any HDF but that is because it delegates the job of recognizing how to work with the data to you, the programmer. You have to recognize that "longitude" is the x dimension, for example. Thus it is not really fair to compare pyhdf with GDAL because GDAL attempts to do that job itself. Hope this helps... (Sorry if this is all obvious...) Best, Jason -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Ivan Lucena Sent: Tuesday, May 25, 2010 11:41 AM To: Ivan Lucena; Joaquim Luis Cc: [email protected] Subject: Re: [gdal-dev] Can't read HDF4 - TRMM It look like pyHDF know more about the HDF SD dimensions than the other software, including GDAL: >>> from pyhdf.SD import * >>> d = SD('D:\\tmp\\3B430501016.HDF') >>> d.datasets() {'precipitation': (('scan', 'longitude', 'latitude'), (1, 1440, 400), 5, 0), 'relativeError': (('scan', 'longitude', 'latitude'), (1, 1440, 400), 5, 1)} >>> ds = d.select('precipitation') >>> ds.dimensions() {'latitude': 400, 'longitude': 1440, 'scan': 1} >>> That means 1440 columns, 400 rows, and 1 band. But GDAL, Idrisi, ArcGIS, Erdas and Envi reads it as 400 columns, 1440 rows, and 1 band. D:\BugsJar>gdal_translate HDF4_SDS:UNKNOWN:"3B430501016.HDF":0 out.tif Input file size is 400, 1440 0...10...20...30...40...50...60...70...80...90...100 - done. $ gdalinfo out.tif Driver: GTiff/GeoTIFF Files: out.tif Size is 400, 1440 Coordinate System is `' Image Structure Metadata: INTERLEAVE=BAND Corner Coordinates: Upper Left ( 0.0, 0.0) Lower Left ( 0.0, 1440.0) Upper Right ( 400.0, 0.0) Lower Right ( 400.0, 1440.0) Center ( 200.0, 720.0) Band 1 Block=400x5 Type=Float32, ColorInterp=Gray There is something wrong with the internal documentation of that file but it seems to be possible to find the dimensions from other HDF (not EOS) API functions. I would try to include calls to SDdiminfo() or SDgetdimstr() perhaps. Regards, Ivan > -------Original Message------- > From: Ivan Lucena <[email protected]> > To: Joaquim Luis <[email protected]> > Cc: [email protected] <[email protected]> > Subject: Re: [gdal-dev] Can't read HDF4 - TRMM > Sent: May 24 '10 13:51 > > Hi Joaquim, > > It seems to me that the HDF4 driver is taking the liberty of using the EOS API to read that file that is actually in HDF format, more specifically a SD (Scientific Dataset). > > http://trac.osgeo.org/gdal/browser/trunk/gdal/frmts/hdf4/hdf4dataset.cpp# L762 > > That is why it doesn't go to that call to SDfileinfo(): > > http://trac.osgeo.org/gdal/browser/trunk/gdal/frmts/hdf4/hdf4dataset.cpp# L991 > > If change poDS->bIsHDFEOS to 0 in debugging mode and the subdatasets showed up: > > Driver: HDF4/Hierarchical Data Format Release 4 > Files: 3B430501016.HDF > Size is 512, 512 > Coordinate System is `' > Subdatasets: > SUBDATASET_1_NAME=HDF4_SDS:UNKNOWN:"3B430501016.HDF":0 > SUBDATASET_1_DESC=[1x1440x400] precipitation (32-bit floating-point) > SUBDATASET_2_NAME=HDF4_SDS:UNKNOWN:"3B430501016.HDF":1 > SUBDATASET_2_DESC=[1x1440x400] relativeError (32-bit floating-point) > Corner Coordinates: > Upper Left ( 0.0, 0.0) > Lower Left ( 0.0, 512.0) > Upper Right ( 512.0, 0.0) > Lower Right ( 512.0, 512.0) > Center ( 256.0, 256.0) > > Now I can gdal_translate it to GTIFF but the image comes rotated by 90 degrees just like in other software. > > I think we can file a ticket to deal with the miss identification, at least. > > Obrigado pela dica. > > Thanks for the tip, > > Ivan > > > -------Original Message------- > > From: Joaquim Luis <[email protected]> > > To: Ivan Lucena <[email protected]> > > Cc: [email protected] <[email protected]> > > Subject: Re: [gdal-dev] Can't read HDF4 - TRMM > > Sent: May 24 '10 10:46 > > > > Ivan, > > > > Since nasa is now imitating me (http://mirador....) I gave it a shot > > with Mirone, which obviously also failed because because it relies > > mostly in gdal to read the hdf files. > > My experience with the nasa satellite files is that they are a big mess > > with the metadata written in endless different ways which makes the > > parsing very painful. There is some reference info in the files. See > > what I got using Matlab (look at the end of this long list) > > > > Joaquim > > > > > > >> info.Attributes(1).Value > > > > ans = > > > > OBJECT=OrbitNumber; > > Value=-9999; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=OrbitNumber; > > > > OBJECT=RangeBeginningDate; > > Value=2010/04/01; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=RangeBeginningDate; > > > > OBJECT=RangeBeginningTime; > > Value=00:00:00; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=RangeBeginningTime; > > > > OBJECT=RangeEndingDate; > > Value=2010/05/01; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=RangeEndingDate; > > > > OBJECT=RangeEndingTime; > > Value=00:00:00; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=RangeEndingTime; > > > > OBJECT=GranulePointer; > > Value="3B43.100401.6A.HDF"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=GranulePointer; > > > > OBJECT=ShortName; > > Value="Surface Rain from all Satellite and Surface"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=ShortName; > > > > OBJECT=SizeMBECSDataGranule; > > Value=4.394; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=SizeMBECSDataGranule; > > > > OBJECT=LongitudeOfMaximumLatitude; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=LongitudeOfMaximumLatitude; > > > > OBJECT=SpatialCoverageType; > > Value="Horizontal"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=SpatialCoverageType; > > > > OBJECT=EllipsoidName; > > Value="NULL"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=EllipsoidName; > > > > OBJECT=EquatorialRadius; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=EquatorialRadius; > > > > OBJECT=DenominatorFlatteningRatio; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=DenominatorFlatteningRatio; > > > > OBJECT=OrbitalModelName; > > Value="NULL"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=OrbitalModelName; > > > > OBJECT=SemiMajorAxis; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=SemiMajorAxis; > > > > OBJECT=MeanAnomaly; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=MeanAnomaly; > > > > OBJECT=RightAscensionNode; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=RightAscensionNode; > > > > OBJECT=ArgumentOfPerigee; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=ArgumentOfPerigee; > > > > OBJECT=Eccentricity; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=Eccentricity; > > > > OBJECT=Inclination; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=Inclination; > > > > OBJECT=EpochTime; > > Value=99:99:99; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=EpochTime; > > > > OBJECT=EpochDate; > > Value=9999/99/99; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=EpochDate; > > > > OBJECT=EpochMillisec; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=EpochMillisec; > > > > OBJECT=WestBoundingCoordinate; > > Value=-180; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=WestBoundingCoordinate; > > > > OBJECT=EastBoundingCoordinate; > > Value=180; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=EastBoundingCoordinate; > > > > OBJECT=NorthBoundingCoordinate; > > Value=50; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=NorthBoundingCoordinate; > > > > OBJECT=SouthBoundingCoordinate; > > Value=-50; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=SouthBoundingCoordinate; > > > > OBJECT=CenterLatitude; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=CenterLatitude; > > > > OBJECT=CenterLongitude; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=CenterLongitude; > > > > OBJECT=RadiusValue; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=RadiusValue; > > > > OBJECT=LatitudeResolution; > > Value=".25deg"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=LatitudeResolution; > > > > OBJECT=LongitudeResolution; > > Value=".25deg"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=LongitudeResolution; > > > > OBJECT=GeographicCoordinateUnits; > > Value="Decimal Degrees"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=GeographicCoordinateUnits; > > > > OBJECT=TemporalRangeType; > > Value="Continuous Range"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=TemporalRangeType; > > > > OBJECT=QualityAssuranceParameterName; > > Value="ScienceQualityFlag"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=QualityAssuranceParameterName; > > > > OBJECT=QualityAssuranceParameterValue; > > Value="NOT BEING INVESTIGATED"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=QualityAssuranceParameterValue; > > > > OBJECT=ReprocessingActual; > > Value="NULL"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=ReprocessingActual; > > > > OBJECT=BrowsePointer; > > Value="3B43_BR.100401.6A.PNG"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=BrowsePointer; > > > > OBJECT=ScienceContact; > > Value="George Huffman"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=ScienceContact; > > > > OBJECT=MeanMotion; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=MeanMotion; > > > > OBJECT=OrbitAdjustFlag; > > Value=-9999; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=OrbitAdjustFlag; > > > > OBJECT=AttitudeModeFlag; > > Value=-9999; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=AttitudeModeFlag; > > > > OBJECT=SolarBetaAngleAtBeginningOfGranule; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=SolarBetaAngleAtBeginningOfGranule; > > > > OBJECT=SolarBetaAngleAtEndOfGranule; > > Value=-9999.9; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=SolarBetaAngleAtEndOfGranule; > > > > OBJECT=SensorAlignment; > > Value="NULL"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=SensorAlignment; > > > > OBJECT=SensorAlignmentChannelOffsets; > > Value="NULL"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=SensorAlignmentChannelOffsets; > > > > OBJECT=ScanPathModel; > > Value="NULL"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=ScanPathModel; > > > > OBJECT=ScanPathModelParam; > > Value="NULL"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=ScanPathModelParam; > > > > OBJECT=EphemerisFileID; > > Value="NULL"; > > Data_Location=PGE; > > Mandatory=FALSE; > > END_OBJECT=EphemerisFileID; > > > > > > > > > > > Hi, > > > > > > I ran gdalinfo on some HDF4 files and no subdataset shows up. > > > > > > The data is available for download at this website: http://mirador.gsfc.nasa.gov/cgi-bin/mirador/presentNavigation.pl?tree=proje ct&project=TRMM&dataGroup=Gridded&dataset=3B43:%20Monthly%200.25%20x%200.25% 20degree%20merged%20TRMM%20and%20other%20sources%20estimates&version=006 > > > > > > I tried reading those some file on: > > > > > > - Idrisi > > > - ArcGIS 9.3 > > > - ArcGIS 10 (RC) > > > - Erdas > > > > > > They do identify the internal datasets but they cannot identify the reference system and the images they are flipped by 90' clockwise. But that is not due to rotation. I would guess. It is more like a misinterpretation of the dimension parameter, taking rows as columns and columns as rows. It looks like the documentation for that particular product is missing here: http://mirador.gsfc.nasa.gov/cgi-bin/mirador/servcoll.pl?helpmenuclass=inven tory&SearchButton=Search%20GES-DISC > > > > > > Does anybody has a clue? > > > > > > BTW. There is a netCDF version of that data that also has some issues. > > > > > > Regards, > > > > > > Ivan > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > gdal-dev mailing list > > > [email protected] > > > http://lists.osgeo.org/mailman/listinfo/gdal-dev > > > > > > > > > > > > > > _______________________________________________ > gdal-dev mailing list > [email protected] > http://lists.osgeo.org/mailman/listinfo/gdal-dev > _______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev _______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev
