Hi, I observed something weird in one of our applications using geotools after upgrading: feature ids get inconsistent, when using removes. Background knowledge is, that the FID-Indexer tries to keep feature ids stable, even if you remove some features from the shapefile from inbetween. But if you query the datastore for a reader, it only cares about the feature ids in the index, if a feature id filter is passed along.
My minmimal test setting: 1. create a shapefile datastore on an existing shapefile "test", having 8 features, without initial fid index 2. remove feature 7 (test.7) from the shapefile 3. query all features of the datastore, returning 7 features (test.1,...,test.7) 4. now query test.7 with feature id filter from the datastore, resulting in null, although I saw a feature with exactly that feature id, when iterating over all store features After step 2, the fid index (.fid file) knows that test.7 was removed and tries to keep feature ids stable, thus saving links for test.1,...,test.6,test.8. When querying all features in step 3, the fid index is not cared about, and especially the left out feature id (see https://github.com/geotools/geotools/blob/master/modules/plugin/shapefile/src/main/java/org/geotools/data/shapefile/ShapefileFeatureSource.java, getReaderInternal(Query q), IndexedFidReader is only used if the filter contains an fid-filter, which is not the case if I query all features). That is the reason, why I get a feature with a feature id test.7 here. In step 4 we pass a fid filter, so that the fid index is read and no feature is returned, because the index only knows about test.1,...,test.6 and test.8. Since this seems to be the case since GeoTools 10, I suspect, that we are some of the last ones, writing to shapefiles. :) As a fix, I propose to drop the feature id stability on the shapefile datastore, since it is only kept until half of the features were deleted (see https://github.com/geotools/geotools/blob/master/modules/plugin/shapefile/src/main/java/org/geotools/data/shapefile/fid/IndexedFidReader.java, getHeader(ShpFiles shpFiles)). Since, in my opinion, keeping them stable in the long run on a shapefile can not be done with resonable effort. I would like to see that feature being dropped. What is your opinion? What other possibilities are there? I could create a test, taken from the minimal example described above. Regards, Hendrik ________________________________ IBYKUS AG f?r Informationstechnologie, Erfurt / HRB 108616 - D-Jena / Vorstand: Helmut C. Henkel, Dr. Lutz Richter, Eckehart Klingner Vorsitzender des Aufsichtsrates: Dr. Helmut Vorndran
_______________________________________________ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
