Hi list, Premise: I’m tinkering to add support for datasets from multiple sources (like different subdatasets) in EOxServer (Python, MapScript, …)
Versions: GDAL 1.9.2, MapServer 6.2.2
The sample dataset I try visualize is a MODIS/HDF 4 file with 7+ subdatasets
(see [1] for a description for the whole file HDF 4 and [2] for a description
of one of the subdatasets).
The subdatasets are not georeferenced, but are supplied with GCPs (and
geolocation arrays, which I have not yet tried).
The bands from the dataset are coming from different subdatasets, so I create a
VRT where I collect all the subdatasets and add them as bands to the VRT [3].
Since MapServer is not capable of dealing with GCPs I need to create a
rectified VRT (created with the CreateWarpedVRT function) [4].
This works quite well, unless I’m in a multithreaded environment (I tested with
both apache mod_wsgi with 1 process and 10 threads and Djangos dev server). In
this case, when I request more than one WMS request at once I get the error
message:
`HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:Emis_32'
does not exist in the file system, and is not recognised as a supported
dataset name.
Once I get this error message, ALL further requests are also running into this
error and I don’t get any reasonable response unless I restart the server.
Further details: We used the WarpedVRT method earlier to great success when all
the sources come from a single dataset (TIFF with GCPs) and I never ran into
this problem.
My questions:
* Is this approach feasible?
* Is it possible to just have a single VRT for both collecting the
datasets into a single VRT and having the warping instructions?
* Any ideas why I get the error message? Any chance to get around this?
Regards,
Fabian
[1]
$ gdalinfo /var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf
Driver: HDF4/Hierarchical Data Format Release 4
Files: /var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf
Size is 512, 512
Coordinate System is `'
Metadata:
...
Subdatasets:
SUBDATASET_1_NAME=HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:LST
SUBDATASET_1_DESC=[2030x1354] LST MOD_Swath_LST (16-bit unsigned integer)
SUBDATASET_2_NAME=HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:QC
SUBDATASET_2_DESC=[2030x1354] QC MOD_Swath_LST (16-bit unsigned integer)
SUBDATASET_3_NAME=HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:Error_LST
SUBDATASET_3_DESC=[2030x1354] Error_LST MOD_Swath_LST (8-bit unsigned integer)
SUBDATASET_4_NAME=HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:Emis_31
SUBDATASET_4_DESC=[2030x1354] Emis_31 MOD_Swath_LST (8-bit unsigned integer)
SUBDATASET_5_NAME=HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:Emis_32
SUBDATASET_5_DESC=[2030x1354] Emis_32 MOD_Swath_LST (8-bit unsigned integer)
SUBDATASET_6_NAME=HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:View_angle
SUBDATASET_6_DESC=[2030x1354] View_angle MOD_Swath_LST (8-bit unsigned
integer)
SUBDATASET_7_NAME=HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:View_time
SUBDATASET_7_DESC=[2030x1354] View_time MOD_Swath_LST (8-bit unsigned integer)
SUBDATASET_8_NAME=HDF4_SDS:UNKNOWN:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":0
SUBDATASET_8_DESC=[406x271] Latitude (32-bit floating-point)
SUBDATASET_9_NAME=HDF4_SDS:UNKNOWN:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":1
SUBDATASET_9_DESC=[406x271] Longitude (32-bit floating-point)
SUBDATASET_10_NAME=HDF4_SDS:UNKNOWN:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":2
SUBDATASET_10_DESC=[2030x1354] LST (16-bit unsigned integer)
...
Corner Coordinates:
Upper Left ( 0.0, 0.0)
Lower Left ( 0.0, 512.0)
Upper Right ( 512.0, 0.0)
Lower Right ( 512.0, 512.0)
Center ( 256.0, 256.0)
[2]
$ gdalinfo
HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:Emis_32
Driver: HDF4Image/HDF4 Dataset
Files: /var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf
/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf.aux.xml
Size is 1354, 2030
Coordinate System is `'
GCP[ 0]: Id=, Info=
(2.5,2.5) -> (-17.7197341918945,16.5706329345703,0)
GCP[ 1]: Id=, Info=
(122.5,2.5) -> (-21.4628753662109,16.2001628875732,0)
GCP[ 2]: Id=, Info=
(242.5,2.5) -> (-23.6760387420654,15.9492874145508,0)
...
GCP[143]: Id=, Info=
(1322.5,1982.5) -> (-43.3567504882812,30.9182910919189,0)
Metadata:
...
Geolocation:
LINE_OFFSET=2
LINE_STEP=5
PIXEL_OFFSET=2
PIXEL_STEP=5
X_BAND=1
X_DATASET=HDF4_EOS:EOS_SWATH_GEOL:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:Longitude
Y_BAND=1
Y_DATASET=HDF4_EOS:EOS_SWATH_GEOL:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:Latitude
Corner Coordinates:
Upper Left ( 0.0, 0.0)
Lower Left ( 0.0, 2030.0)
Upper Right ( 1354.0, 0.0)
Lower Right ( 1354.0, 2030.0)
Center ( 677.0, 1015.0)
Band 1 Block=1354x738 Type=Byte, ColorInterp=Gray
Description = Band 32 emissivity
NoData Value=0
Offset: 0.49, Scale:0.002
[3]
<VRTDataset rasterXSize="1354" rasterYSize="2030">
<GCPList>
<GCP Id="" Pixel="2.5000" Line="2.5000" X="-1.771973419189E+01"
Y="1.657063293457E+01" />
<GCP Id="" Pixel="122.5000" Line="2.5000" X="-2.146287536621E+01"
Y="1.620016288757E+01" />
...
<GCP Id="" Pixel="1322.5000" Line="1982.5000" X="-4.335675048828E+01"
Y="3.091829109192E+01" />
</GCPList>
<VRTRasterBand dataType="UInt16" band="1">
<SimpleSource>
<SourceFilename
relativeToVRT="0">HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:LST</SourceFilename>
<SourceBand>1</SourceBand>
<SourceProperties RasterXSize="1354" RasterYSize="2030" DataType="UInt16"
BlockXSize="1354" BlockYSize="738" />
</SimpleSource>
</VRTRasterBand>
<VRTRasterBand dataType="UInt16" band="2">
<SimpleSource>
<SourceFilename
relativeToVRT="0">HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:QC</SourceFilename>
<SourceBand>1</SourceBand>
<SourceProperties RasterXSize="1354" RasterYSize="2030" DataType="UInt16"
BlockXSize="1354" BlockYSize="738" />
</SimpleSource>
</VRTRasterBand>
<VRTRasterBand dataType="Byte" band="3">
<SimpleSource>
<SourceFilename
relativeToVRT="0">HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:Error_LST</SourceFilename>
<SourceBand>1</SourceBand>
<SourceProperties RasterXSize="1354" RasterYSize="2030" DataType="Byte"
BlockXSize="1354" BlockYSize="738" />
</SimpleSource>
</VRTRasterBand>
<VRTRasterBand dataType="Byte" band="4">
<SimpleSource>
<SourceFilename
relativeToVRT="0">HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:Emis_31</SourceFilename>
<SourceBand>1</SourceBand>
<SourceProperties RasterXSize="1354" RasterYSize="2030" DataType="Byte"
BlockXSize="1354" BlockYSize="738" />
</SimpleSource>
</VRTRasterBand>
<VRTRasterBand dataType="Byte" band="5">
<SimpleSource>
<SourceFilename
relativeToVRT="0">HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:Emis_32</SourceFilename>
<SourceBand>1</SourceBand>
<SourceProperties RasterXSize="1354" RasterYSize="2030" DataType="Byte"
BlockXSize="1354" BlockYSize="738" />
</SimpleSource>
</VRTRasterBand>
<VRTRasterBand dataType="Byte" band="6">
<SimpleSource>
<SourceFilename
relativeToVRT="0">HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:View_angle</SourceFilename>
<SourceBand>1</SourceBand>
<SourceProperties RasterXSize="1354" RasterYSize="2030" DataType="Byte"
BlockXSize="1354" BlockYSize="738" />
</SimpleSource>
</VRTRasterBand>
<VRTRasterBand dataType="Byte" band="7">
<SimpleSource>
<SourceFilename
relativeToVRT="0">HDF4_EOS:EOS_SWATH:"/var/eoxserver/autotest/MOD11_L2.A2015001.0015.005.2015005162430.hdf":MOD_Swath_LST:View_time</SourceFilename>
<SourceBand>1</SourceBand>
<SourceProperties RasterXSize="1354" RasterYSize="2030" DataType="Byte"
BlockXSize="1354" BlockYSize="738" />
</SimpleSource>
</VRTRasterBand>
</VRTDataset>
[4]
<VRTDataset rasterXSize="2029" rasterYSize="1820" subClass="VRTWarpedDataset">
<GeoTransform> -4.2266212922257814e+01, 1.1322693568940058e-02,
0.0000000000000000e+00, 3.4413385891072409e+01, 0.0000000000000000e+00,
-1.1322693568940058e-02</GeoTransform>
<VRTRasterBand dataType="UInt16" band="1" subClass="VRTWarpedRasterBand">
<Metadata />
</VRTRasterBand>
<VRTRasterBand dataType="UInt16" band="2" subClass="VRTWarpedRasterBand">
<Metadata />
</VRTRasterBand>
<VRTRasterBand dataType="Byte" band="3" subClass="VRTWarpedRasterBand">
<Metadata />
</VRTRasterBand>
<VRTRasterBand dataType="Byte" band="4" subClass="VRTWarpedRasterBand">
<Metadata />
</VRTRasterBand>
<VRTRasterBand dataType="Byte" band="5" subClass="VRTWarpedRasterBand">
<Metadata />
</VRTRasterBand>
<VRTRasterBand dataType="Byte" band="6" subClass="VRTWarpedRasterBand">
<Metadata />
</VRTRasterBand>
<VRTRasterBand dataType="Byte" band="7" subClass="VRTWarpedRasterBand">
<Metadata />
</VRTRasterBand>
<BlockXSize>512</BlockXSize>
<BlockYSize>128</BlockYSize>
<GDALWarpOptions>
<WarpMemoryLimit>6.71089e+07</WarpMemoryLimit>
<ResampleAlg>NearestNeighbour</ResampleAlg>
<WorkingDataType>UInt16</WorkingDataType>
<SourceDataset
relativeToVRT="1">ff71b2c70ab940d6b35ecaa6c4b100e6</SourceDataset>
<Transformer>
<ApproxTransformer>
<MaxError>0.125</MaxError>
<BaseTransformer>
<GenImgProjTransformer>
<SrcGCPTransformer>
<GCPTransformer>
<Order>2</Order>
<Reversed>0</Reversed>
<GCPList>
<GCP Id="" Pixel="2.5000" Line="2.5000"
X="-1.771973419189E+01" Y="1.657063293457E+01" />
<GCP Id="" Pixel="122.5000" Line="2.5000"
X="-2.146287536621E+01" Y="1.620016288757E+01" />
...
<GCP Id="" Pixel="1322.5000" Line="1982.5000"
X="-4.335675048828E+01" Y="3.091829109192E+01" />
</GCPList>
</GCPTransformer>
</SrcGCPTransformer>
<DstGeoTransform>-42.26621292225781,0.01132269356894006,0,34.41338589107241,0,-0.01132269356894006</DstGeoTransform>
<DstInvGeoTransform>3732.876162806413,88.31820749288477,0,3039.328555660446,0,-88.31820749288477</DstInvGeoTransform>
</GenImgProjTransformer>
</BaseTransformer>
</ApproxTransformer>
</Transformer>
<BandList>
<BandMapping src="1" dst="1" />
<BandMapping src="2" dst="2" />
<BandMapping src="3" dst="3" />
<BandMapping src="4" dst="4" />
<BandMapping src="5" dst="5" />
<BandMapping src="6" dst="6" />
<BandMapping src="7" dst="7" />
</BandList>
</GDALWarpOptions>
</VRTDataset>
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev
