Hi, Hard to say if such use case exists. Perhaps if .ovr is actually not subsampled from the main data at all but tweaked from let's say 1:50000 scale raster maps when the main maps are 1:20000. I have created such overview stacks sometimes through .vrt. I am not sure if the automatic invalidating would have any effect in that case.
-Jukka Rahkonen- Even Rouault-2 wrote > Julien, > >> >> Thanks for your response. I understand all your points, and that Gdal >> requirements usually fall far beyond our limited use cases ... > > I was mostly playing the devil advocate. I don't have in mind real world > use > cases that would rely on .ovr being potentially older that the main file, > while > still considering them valid (anyone having such use cases ?). And I'd > suspect > they are probably very niche use cases, compared to the cases where the > main > image has been refreshed but an outdated overview file is hanging around. > On the other hand, my main fear with your proposal would be timestamp > issues > when copying datasets around that could make GDAL think that an .ovr is > outdated whereas it is not. But that's perhaps a unusual case too. > (That said we have already timestamp based logic in the GML driver where > the > OGR generated .gfs file is only taken into account if more recent than the > .gml. I can remember having had issues with that with the autotest suite > that > have required adding an explicit touch of the .gfs file before running the > test. Might be just a SVN / git thing, or perhaps the content of the .gml > was > indeed minorly edited after the .gfs had been created.) > > Thinking loud: would a USE_EVEN_IF_OLDER_THAN_MAIN_FILE=YES/NO metadata > item > added to the .ovr would make sense to determine the later behaviour ? I > dunno. > If folks are sufficiently aware to set this metadata to YES (assuming the > new > default would be NO), then they might as well touch the .ovr file after > modifying the main file. > > Anyway if you feel strong about changes in that area that have > implications > regarding backward compatibile behaviour, a RFC would be appropriate. > > Even > > >> We can of >> course do a lot in the client code (starting with comparing timestamps), >> but this is not specific to our software, I bet Qgis for instance will >> happily use outdated overviews without notice. This is why I am >> wondering if it could not be handled more cleanly at the Gdal level. I >> understand though that this is not an easy problem to fix. >> >> Regards, >> >> Julien >> >> Le 07/10/2016 à 16:46, Even Rouault a écrit : >> > Hi Julien, >> > >> >> I recently came accross an issue in our software which uses overviews >> to >> >> speed-up navigation in image : the image can change after an external >> >> overview has been generated (for instance it has been re-generated by >> an >> >> upstream processing chain with different parameters), this can lead to >> >> display bugs or even crash in client code (if for instance the number >> of >> >> bands has changed) >> > >> > Are the crashes you mention occuring in GDAL ? If so that should be >> > fixed. From a quick try, RasterIO() requests that involve overviews >> > trigger a proper error if the .ovr has less bands than the full >> > resolution dataset. Of course if client code directly uses >> > GetOverview(), it must be careful to check for NULL pointer. >> > >> >> . Of course this is not a problem for the user that >> >> knows what she is doing : simply removing and generating the overview >> >> again fixes the problem. But for software that offers overview >> >> generation to the end user, this might become an issue, as the >> software >> >> has no clue wether the ovr file is outdated or not. >> >> >> >> Possible mecanism to prevent this would be : >> >> >> >> - Check file last modification time between external overview and >> image >> >> file. If image file is newer than overview files, it is probably >> >> outdated. >> > >> > "Probably" yes :-) But I can imagine workflows where the overviews are >> > not necessarily a subsampling of the full resolution dataset, in which >> > case such behaviour wouldn't be desirable. I'm also wondering if that >> > wouldn't cause problems when datasets are copied. Hopefully timestamps >> > should be preserved most of the time, but I guess you could find >> > situations where they are not and where the overview could be copied >> > before the main dataset. >> > >> >> - Encode image checksum in ovr file, and compare it upon loading >> (might >> >> be a bit intensive). >> > >> > We definitely don't want to do that on gigabyte sized datasets... >> > >> >> With those checks, gdal could detect that the ovr file is outdated and >> >> simply ignore it. Then client code could be aware that there are >> >> actually no overviews for this image and take actions to generate new >> >> ones. >> > >> > The client code can also use the GDALDataset::GetFileList() API and see >> > if there's a .ovr file listed in there. And thus decide to apply a >> > timestamp based logic if it whishes. >> > >> > Or one could imagine to put that logic into GDAL itself, but I think >> that >> > should be an option explicitly set. >> > >> > There would be a subtelty also. Imagine that the external overview is >> > outdated, and you want to update it, but not recreate it from scratch. >> > For example if you know which area has been updated in full resolution >> > dataset. Then you'd want the overview to be still accessible. So hiding >> > the overview should only apply for datasets opened in read-only mode. >> > >> >> PS: This of course does not apply if overviews are internal >> > >> > You probably meant that detecting that overviews are outdated is even >> > less doable than for the external case, but that situation can still >> > happen. >> > >> > Even > > -- > Spatialys - Geospatial professional services > http://www.spatialys.com > _______________________________________________ > gdal-dev mailing list > [email protected] > http://lists.osgeo.org/mailman/listinfo/gdal-dev -- View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-Outdated-external-overviews-tp5289792p5290252.html Sent from the GDAL - Dev mailing list archive at Nabble.com. _______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev
