Just in case someone is interested in a solution for this issue:
The most efficient setting was to set the gdal option
GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR
This can reduce the file stat/open calls noticeably, depending on the
raster format. The worst format I found was JP2, which is now
unfortunately more often used like e.g. in Sentinel-2 data. For a JP2
image, a gdal Open() request for a single JP2 file makes 46 file
stat/open calls... With the above setting this is reduced to 6. In order
to set this as default, one needs to change this in the source code in
file gcore/gdalopeninfo.cpp.
If in addition the option GDAL_PAM_ENABLED is set to NO, then another 5
stat/open calls are eliminated. Here this setting can be put as default
at compile time by adding to the "configure" script the option
--without-pam
Cheers
Armin
On 21/03/2019 19:54, Armin Burger wrote:
On 21/03/2019 19:09, Even Rouault wrote:
On jeudi 21 mars 2019 18:34:11 CET Armin Burger wrote:
Hi all
When gdal opens a file, it checks for various sibling files, like *.MTL,
*.aux.xml, *.RPC, etc (this is the case when the full directory scanning
is deactivated). This can be ~30 stat or open requests when opening a
single image file. Is there a way to define the file types that gdal
checks for their existence, thus allowing to reduce this to the pure
minimum, like *.ovr files only? Is this check driver dependent?
If using /vsicurl/ and derived file systems, you can use
CPL_VSIL_CURL_ALLOWED_EXTENSIONS=".tif,.ovr" for example
Even
Hi Even
Thanks, but we're using a POSIX file system (mounted via FUSE client),
so I assume this will not work then.
It would also be fine for us to reduce the number of extensions at
compile time by modifying configuration or source code files. But so far
I haven't found any appearances in the source code files for file
extensions that are obviously by default checked for their existence,
like e.g. "RPC.TXT". So any hints if/how this is possible would be great.
Cheers
Armin
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
