During today's bugstomp I picked up GEOT-5830 <https://osgeo-org.atlassian.net/projects/GEOT/issues/GEOT-5830>to look at as it seemed a pretty easy bug to start with, looked like a simple case of the FIDindex not being updated to allow for the fact a feature had been deleted.
Boy, was I wrong there, this looks like a whole nasty can of worms instead. To start with it looked like we already had a test covering this issue <https://github.com/geotools/geotools/blob/master/modules/plugin/shapefile/src/test/java/org/geotools/data/shapefile/fid/FidQueryTest.java#L155> (but then how could the user be seeing it?) - so I poked around a little and scratched my head, I came up with the following test to see what was happening: /** * Attempt to test GEOT-5830 User reports that deleting a feature, re-requesting the data gives * a duplicate FID and the subsequent attempt to delete fails due to corrupt FIX * * @throws Exception */ @Test public void testDeleteCloseAndRerequestFID() throws Exception { SimpleFeature feature; SimpleFeatureCollection allfeatures = featureStore.getFeatures(); int size = allfeatures.size(); try (SimpleFeatureIterator features = allfeatures.features()) { feature = features.next(); } FilterFactory2 ff = CommonFactoryFinder.getFilterFactory2(null); Id fidFilter = ff.id (Collections.singleton(ff.featureId(feature.getID()))); featureStore.removeFeatures(fidFilter); fids.remove(feature.getID()); assertEquals((size - 1), featureStore.getCount(Query.ALL)); assertEquals(fids.size(), featureStore.getCount(Query.ALL)); featureStore.getDataStore().dispose(); URL url = backshp.toURI().toURL(); ds = new ShapefileDataStore(url); numFeatures = 0; featureStore = (SimpleFeatureStore) ds.getFeatureSource(); assertEquals((size - 1), featureStore.getCount(Query.ALL)); SimpleFeatureCollection features2 = featureStore.getFeatures(fidFilter); assertEquals("wrong number of features", 0, features2.size()); try (SimpleFeatureIterator features = features2.features()) { assertFalse("found fid", features.hasNext()); } // when we fetch the features again the first feature is archsites.1 again! // yet the IndexedShapeReader tries very hard to avoid this! try (SimpleFeatureIterator features = allfeatures.features()) { SimpleFeature f = (SimpleFeature) features.next(); assertFalse(fidFilter.evaluate(f)); } // but not if we use the fid filter features2 = featureStore.getFeatures(fidFilter); assertEquals("wrong number of features", 0, features2.size()); try (SimpleFeatureIterator features = features2.features()) { assertFalse("found fid", features.hasNext()); } this.assertFidsMatch(); } And this one fails "nicely" in exactly the way the user reports. Unless we also make our request using a FIDFilter again in which case it does work (actually this test is the reverse of the users use case but you see what I mean) - so it seems that a Shapefile IndexedFidWriter and a IndexedFIDReader keeps track of deleted features and doesn't reuse the FID (which seems like a good thing) while a "normal" ShapefileFeatureReader doesn't know anything about the missing FID and returns a new list of fids which include the deleted ones (pointing to the next feature). So, now I'm stuck trying to work out which is the "correct" behaviour - should deleted FIDs be discarded? and if so for how long? just until the next time the .fix is created? or forever? Or should deleting a feature renumber all the features immediately? Still reading? cool feel free to chip in here, on the PR <https://github.com/geotools/geotools/pull/2338> or the issue <https://osgeo-org.atlassian.net/projects/GEOT/issues/GEOT-5830>. Thanks Ian -- Ian Turton
_______________________________________________ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
