Hello Andrea
OK, did some testing around, and managed to make things work really fast.
I found out that the spatial query being performed was actually a "not
disjoint" query. Changing that to the equivalent "intersects" spatial
query boosts everything, and the query is completed almost instantaneously!
Now, I'm not entirely sure where exactly the slowdown takes place. At
first, I thought maybe the composite "not disjoint" was making the
ShapefileDataStore think that this was not a strictly spatial filter.
However, it seems that just using "disjoint" is also really slow, so I'm
a little confused. In all cases, a proper indexed reader seems to be in
use by the filtering feature reader.
Anyways: this is not critical in any sense, since it's working
brilliantly using "intersects". But it still leaves me unsure as to when
using shapefiles will be blazingly fast and when not.
Thanks
Milton
PS: I send attached a simple program for testing this. Unfortunately, as
I said before, I'm not allowed to send you the big shapefile we're using
for the tests..
Andrea Aime wrote:
Milton Jonathan wrote:
Hey there Andrea
Thanks for the tip. Found the ShapeFileIndexer main() and ran it.
By the way, the little program worked like a breeze, very easy to use,
fast and with nice feedback. The normal QIX tree was being generated
with a depth of 10 for my shapefile. I then forced it to generate a
tree with a depth 20, which gave a 64MB QIX file instead of an 8MB one.
Unfortunately, I got NO performance increase with all that.. :(
It seems to me that the issue is that, when performing a spatial
query, GeoTools ends up creating a FilteringFeatureReader. And merely
asking featureCollection.isEmpty() leads to a call to
reader.hasNext(), which ends up in the loop at
FilteringFeatureReader.java, lines 127-134 - basically, calling the
internal featureReader's next() and checking to see if the filter
applies.
That needs to be done in any case. The QIX index just returns a set of
candidate, by the way quadtree indexing works some will lie outside
of the requested area.
The question is, what is below the filtering feature reader? An indexed
reader, or a plain one?
I don't have big shapefiles with me now so I cannot check. If you have
a sample app with your data and could made them available to me it
would speed up looking into this
Cheers
Andrea
--
Milton Jonathan
Grupo GIS e Meio Ambiente
Tecgraf/PUC-Rio
Tel: +55-21-3527-2502
package org.tecgraf.tdk.example.geotools.spike;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.HashMap;
import java.util.Map;
import org.geotools.data.DataStore;
import org.geotools.data.FeatureSource;
import org.geotools.data.shapefile.indexed.IndexedShapefileDataStore;
import org.geotools.factory.CommonFactoryFinder;
import org.geotools.feature.FeatureCollection;
import org.geotools.filter.AttributeExpressionImpl;
import org.geotools.geometry.jts.JTS;
import org.geotools.geometry.jts.ReferencedEnvelope;
import org.geotools.referencing.crs.DefaultGeographicCRS;
import org.opengis.feature.simple.SimpleFeature;
import org.opengis.feature.simple.SimpleFeatureType;
import org.opengis.filter.Filter;
import org.opengis.filter.FilterFactory2;
import org.opengis.filter.expression.Expression;
import org.opengis.geometry.BoundingBox;
import com.vividsolutions.jts.geom.Geometry;
/**
* Hello world!
*
*/
public class ShapefileIntersects
{
public static void main( String[] args )
{
try
{
URL shapeUrlSrc = new URL("file:///D:/Milton/data/big/SEGMENTS.shp");
Map<Object,Object> connect = new HashMap<Object,Object>();
connect.put("url", shapeUrlSrc);
DataStore dataStoreSrc = new IndexedShapefileDataStore(shapeUrlSrc);
FilterFactory2 ff = CommonFactoryFinder.getFilterFactory2(null);
String[] typeNames = dataStoreSrc.getTypeNames();
for (String typeName : typeNames)
{
SimpleFeatureType ftype = dataStoreSrc.getSchema(typeName);
BoundingBox area = new ReferencedEnvelope(-80, -70, -30, -20, DefaultGeographicCRS.WGS84);
Geometry areaGeom = JTS.toGeometry(area);
Expression geomPropertyExpr = new AttributeExpressionImpl(ftype.getGeometryDescriptor().getLocalName());
Expression areaGeomExpr = ff.literal(areaGeom);
// Filter filter = ff.not(ff.disjoint(geomPropertyExpr, areaGeomExpr));
// Filter filter = ff.disjoint(geomPropertyExpr, areaGeomExpr);
Filter filter = ff.intersects(geomPropertyExpr, areaGeomExpr);
FeatureSource<SimpleFeatureType,SimpleFeature> featSource = dataStoreSrc.getFeatureSource(typeName);
FeatureCollection<SimpleFeatureType,SimpleFeature> featSrcCollection = featSource.getFeatures(filter);
String result;
if (featSrcCollection.isEmpty())
result = "nothing was found!";
else
result = "SOMETHING WAS FOUND!";
System.out.println("Result: " + result);
}
System.out.println( "DONE" );
}
catch (MalformedURLException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
------------------------------------------------------------------------------
Return on Information:
Google Enterprise Search pays you back
Get the facts.
http://p.sf.net/sfu/google-dev2dev
_______________________________________________
Geotools-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geotools-devel
