Thanks for your replies.
Gabriel, I've attached the full class now, so that you can get a
better idea of the code.
The "create" is called just once by Sextante framework on object
initialization. This method defines one transaction, which will be
used all the time.
The "addFeature" method is called from a loop, as much times as
features are going to be added to the file.
I'd prefer to don't build a feature collection containing all the
features as this will create the full layer in memory, which is a bad
idea for really big layers (as it is the case). I guess I'll need to
use the FeatureWriter, as you suggest.
Thanks for your time.
César
El día 8 de enero de 2010 20:42, Gabriel Roldan <[email protected]> escribió:
> I don't see the loop in the sample code. It certainly looks like if all that
> code is inside a control loop executed 14k times it might perform badly.
>
> Also, it doesn't look like you're using a transaction. Though I'm not
> completely sure about shapefile datastore performances it could be that
> working in auto commit mode is a performance waste.
>
> You can try two things:
> - build the feature collection with all the features to add first and then
> make a single call to store.addFeatures
> - Acquire a FeatureWriter out of the DataStore instead.
>
> in either case try to set a transaction before adding all the features and
> commit it when done.
>
> Transaction tr = new DefaultTransaction();
> FeatureStore store = ...
> store.setTransaction(tr);
> try{
> ... add features...
> tr.commit();
> }catch(Exception e){
> tr.rollback();
> }
>
> sort of..
>
> Cheers,
> Gabriel
>
> César Martínez Izquierdo wrote:
>>
>> Some additional notes on this topic. The reported time (~63 minutes)
>> happens when the FeatureStore is a ShapefileDataStore.
>> When using a MemoryDataStore the process is much faster (~5 minutes).
>>
>> However, I still think it is a lot of time, as the same algorithm
>> execution takes about 3 seconds when using the gvSIG-Sextante
>> bindings. Moreover, the MemoryDataStore is not a suitable solution for
>> big layers.
>>
>> What's wrong with the code bellow?
>>
>> Regards,
>>
>> César
>>
>> El día 21 de diciembre de 2009 16:19, César Martínez Izquierdo
>> <[email protected]> escribió:
>>>
>>> Hello list,
>>>
>>> I'm using Sextante through the GeoTools-Sextante bindings to create a
>>> Graticule (using CreateGraticuleBuilder algorithm form Sextante).
>>> This creates a shapefile containing a graticule of rectangles.
>>>
>>> However, performance is really poor: ~ 63 minutes to just write around
>>> 14000 features.
>>> The relevant part of code is:
>>> FeatureStore<SimpleFeatureType,
>>> SimpleFeature> store =
>>> ((FeatureStore<SimpleFeatureType, SimpleFeature>) getFeatureSource());
>>> List<Object> attributes = new
>>> ArrayList<Object>();
>>> attributes.add(geom);
>>> attributes.addAll(Arrays.asList(values));
>>> SimpleFeatureType ft = store.getSchema();
>>> FeatureCollection<SimpleFeatureType,
>>> SimpleFeature> collection =
>>> FeatureCollections
>>> .newCollection();
>>>
>>> SimpleFeature feature =
>>> SimpleFeatureBuilder.build(ft, attributes,
>>> SimpleFeatureBuilder.createDefaultFeatureId());
>>> collection.add(feature);
>>> store.addFeatures(collection);
>>> Note that "geom" is a JTS geometry, and "values" is an Object[].
>>>
>>> I've been measuring times, and 99% of the time is spent in the last line:
>>>
>>> store.addFeatures(collection);
>>>
>>> Is there a better way to achieve this with GeoTools or should I bypass
>>> the FeatureStore layer in GeoTools and go directly to
>>> ShapefileWriter?
>>> I would really appreciate any clue on this.
>>>
>>> Regards,
>>>
>>> César.
>>>
>>>
>>>
>>> --
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>> César Martínez Izquierdo
>>> GIS developer
>>> - - - - - - - - - - - - - - - - - - - -
>>> ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>> Universitat Autònoma de Barcelona (SPAIN)
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>
>>
>>
>>
>
>
> --
> Gabriel Roldan
> OpenGeo - http://opengeo.org
> Expert service straight from the developers.
>
--
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
César Martínez Izquierdo
GIS developer
- - - - - - - - - - - - - - - - - - - -
ETC-LUSI: http://etc-lusi.eionet.europa.eu/
Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
package es.unex.sextante.geotools;
import java.awt.geom.Rectangle2D;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.geotools.data.DataStore;
import org.geotools.data.DefaultQuery;
import org.geotools.data.DefaultTransaction;
import org.geotools.data.FeatureSource;
import org.geotools.data.FeatureStore;
import org.geotools.data.Query;
import org.geotools.data.Transaction;
import org.geotools.data.shapefile.ShapefileFeatureStore;
import org.geotools.data.shapefile.ShapefileFeatureWriter;
import org.geotools.feature.FeatureCollection;
import org.geotools.feature.FeatureCollections;
import org.geotools.feature.simple.SimpleFeatureBuilder;
import org.geotools.geometry.jts.ReferencedEnvelope;
import org.opengis.feature.simple.SimpleFeature;
import org.opengis.feature.simple.SimpleFeatureType;
import org.opengis.feature.type.AttributeType;
import org.opengis.filter.Filter;
import org.opengis.referencing.crs.CoordinateReferenceSystem;
import com.vividsolutions.jts.geom.Geometry;
import com.vividsolutions.jts.geom.GeometryFactory;
import com.vividsolutions.jts.geom.LineString;
import com.vividsolutions.jts.geom.MultiLineString;
import com.vividsolutions.jts.geom.MultiPolygon;
import com.vividsolutions.jts.geom.Polygon;
import es.unex.sextante.core.Sextante;
import es.unex.sextante.dataObjects.AbstractVectorLayer;
import es.unex.sextante.dataObjects.IFeatureIterator;
import es.unex.sextante.dataObjects.IVectorLayer;
public class GTVectorLayer extends AbstractVectorLayer {
// Factory methods:
/**
* Creates a vector layer from the FeatureCollection resulting from applying
* the query to the Datastore.
* <p>
* Note throws an exception if the query's typename is not one of the
* datastores featuretypes
* </p>
*
* @param source
* the DataStore to query
* @param query
* the query to use for obtaining the data
* @throws IOException
*/
public static GTVectorLayer createLayer(DataStore source, Query query)
throws IOException {
if (!Arrays.asList(source.getTypeNames()).contains(query.getTypeName())) {
throw new IllegalArgumentException(
query.getTypeName()
+ " is not one of the FeatureTypes contained by the Datasource. Options are: "
+ Arrays.asList(source.getTypeNames()));
}
FeatureSource<SimpleFeatureType, SimpleFeature> featureSource = source.getFeatureSource(query
.getTypeName());
GTVectorLayer layer = new GTVectorLayer();
layer.create(featureSource, query);
return layer;
}
/**
* Creates a vector layer from the datasource.
*
* @param source
* the source to query
* @param typename
* the typename (must be one of the datasources typenames) to use
* for the layer
* @param filter
* a filter to apply to the features that are used in the layer
* @param crs
* the crs to reproject the features to (if necessary). If null
* no reprojection will be applied
*/
public static GTVectorLayer createLayer(DataStore source, String typename,
Filter filter, CoordinateReferenceSystem crs) throws IOException {
DefaultQuery query = new DefaultQuery(typename, filter);
if (crs != null) {
query.setCoordinateSystemReproject(crs);
}
return createLayer(source, query);
}
/**
* Creates a vector layer from the datasource.
*
* @param source
* the source to query
* @param typename
* the typename (must be one of the datasources typenames) to use
* for the layer
* @param crs
* the crs to reproject the features to (if necessary). If null
* no reprojection will be applied
*/
public static GTVectorLayer createLayer(DataStore source, String typename,
CoordinateReferenceSystem crs) throws IOException {
return createLayer(source, typename, Filter.INCLUDE, crs);
}
/**
* Creates a vector layer from the datasource.
*
* @param source
* the source to query
* @param typename
* the typename (must be one of the datasources typenames) to use
* for the layer
*/
public static GTVectorLayer createLayer(DataStore source, String typename)
throws IOException {
return createLayer(source, typename, Filter.INCLUDE, null);
}
private String m_sFilename;
private String m_sName;
private int m_iCount;
private CoordinateReferenceSystem m_CRS;
private Query m_Query;
private PostProcessStrategy m_PostProcessStrategy;
private long timePrepareGeom = 0, timeWriteGeom = 0, timeTransaction = 0;
private Transaction transaction = null;
private long transactionFeatures;
private SimpleFeatureBuilder m_featureBuilder = null;
public void create(FeatureSource<SimpleFeatureType, SimpleFeature> featureSource, Query query) {
try {
this.m_Query = query;
m_BaseDataObject = featureSource;
m_iCount = featureSource.getFeatures(query).size();
m_CRS = featureSource.getSchema().getCoordinateReferenceSystem();
m_sName = query.getTypeName();
if (featureSource instanceof FeatureStore) {
transaction = new DefaultTransaction("addFeature");
transactionFeatures = 0;
FeatureStore<SimpleFeatureType, SimpleFeature> store = (FeatureStore<SimpleFeatureType, SimpleFeature>) featureSource;
store.setTransaction(transaction);
m_featureBuilder = new SimpleFeatureBuilder(store.getSchema());
}
} catch (Exception e) {
e.printStackTrace();
}
}
public void open() {
}
public void close() {
System.out.println("time prepare geom: "+timePrepareGeom+ " ms.");
System.out.println("time write geom: "+timeWriteGeom+ " ms.");
if (this.getFeatureSource() != null) {
this.getFeatureSource().getDataStore().dispose();
}
}
public void addFeature(Geometry g, Object[] values) {
long time1, time2, time3, time4;
time1 = System.currentTimeMillis();
if (getFeatureSource() instanceof FeatureStore) {
Geometry geom;
GeometryFactory gf = new GeometryFactory();
if (g instanceof Polygon) {
geom = gf.createMultiPolygon(new Polygon[] { (Polygon) g });
} else if (g instanceof LineString) {
geom = gf.createMultiLineString(new LineString[] { (LineString) g });
} else {
geom = g;
}
try {
FeatureStore<SimpleFeatureType, SimpleFeature> store = ((FeatureStore<SimpleFeatureType, SimpleFeature>) getFeatureSource());
FeatureCollection<SimpleFeatureType, SimpleFeature> collection = FeatureCollections
.newCollection();
m_featureBuilder.add(geom);
m_featureBuilder.addAll(values);
SimpleFeature feat = m_featureBuilder.buildFeature(null);
collection.add(feat);
time2 = System.currentTimeMillis();
store.addFeatures(collection);
m_iCount++;
time3 = System.currentTimeMillis();
timePrepareGeom += time2 - time1;
timeWriteGeom += time3 - time2;
if (++transactionFeatures%4000 == 0) {
outputTimes();
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
public IFeatureIterator iterator() {
if (m_BaseDataObject != null) {
try {
FeatureCollection<SimpleFeatureType, SimpleFeature> features = getFeatureSource().getFeatures(
m_Query);
return new GTFeatureIterator(features);
} catch (Exception e) {
e.printStackTrace();
return null;
}
} else {
return null;
}
}
public String getFieldName(int i) {
if (m_BaseDataObject != null) {
try {
SimpleFeatureType ft = getFeatureSource().getSchema();
AttributeType at = ft.getType(i + 1);
return at.getName().getLocalPart();
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
return null;
}
public Class<?> getFieldType(int i) {
if (m_BaseDataObject != null) {
try {
SimpleFeatureType ft = getFeatureSource().getSchema();
AttributeType at = ft.getType(i + 1);
return at.getBinding();
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
return null;
}
public int getFieldCount() {
if (m_BaseDataObject != null) {
try {
SimpleFeatureType ft = getFeatureSource().getSchema();
return ft.getAttributeCount() - 1;
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
return 0;
}
public int getShapesCount() {
return m_iCount;
}
public int getShapeType() {
if (m_BaseDataObject != null) {
try {
SimpleFeatureType ft = getFeatureSource().getSchema();
Class<?> type = ft.getGeometryDescriptor().getType().getBinding();
if (type.isAssignableFrom(Polygon.class)
|| type.isAssignableFrom(MultiPolygon.class)) {
return IVectorLayer.SHAPE_TYPE_POLYGON;
} else if (type.isAssignableFrom(LineString.class)
|| type.isAssignableFrom(MultiLineString.class)) {
return IVectorLayer.SHAPE_TYPE_LINE;
} else {
return IVectorLayer.SHAPE_TYPE_POINT;
}
} catch (Exception e) {
e.printStackTrace();
return IVectorLayer.SHAPE_TYPE_POLYGON;
}
}
return IVectorLayer.SHAPE_TYPE_POLYGON;
}
public String getName() {
return m_sName;
}
public void postProcess() {
long time1, time2, time3;
time1 = System.currentTimeMillis();
if (transaction!=null) {
try {
transaction.commit();
transaction.close();
} catch (IOException e) {
Sextante.addErrorToLog(e);
}
if (getFeatureSource() instanceof FeatureStore) {
transactionFeatures = 0;
((FeatureStore)getFeatureSource()).setTransaction(Transaction.AUTO_COMMIT); // restore default transaction
}
}
time2 = System.currentTimeMillis();
m_PostProcessStrategy.postProcess(getFeatureSource());
time3 = System.currentTimeMillis();
timeTransaction += time2 - time1;
System.out.println("postprocess time: "+(time3-time2)+ " ms.");
outputTimes();
}
private void outputTimes() {
System.out.println("time prepare geom: "+timePrepareGeom+ " ms.");
System.out.println("time write geom: "+timeWriteGeom+ " ms.");
System.out.println("transaction time: "+timeTransaction+ " ms.");
}
public Rectangle2D getFullExtent() {
if (m_BaseDataObject != null) {
try {
ReferencedEnvelope bounds = getFeatureSource().getFeatures(
m_Query).getBounds();
return new Rectangle2D.Double(bounds.getMinX(), bounds
.getMinY(), bounds.getWidth(), bounds.getHeight());
} catch (Exception e) {
e.printStackTrace();
return null;
}
} else {
return null;
}
}
public String getFilename() {
return m_sFilename;
}
public Object getCRS() {
return m_CRS;
}
public void setName(String name) {
m_sName = name;
}
@SuppressWarnings("unchecked")
private FeatureSource<SimpleFeatureType, SimpleFeature> getFeatureSource() {
return (FeatureSource<SimpleFeatureType, SimpleFeature>) m_BaseDataObject;
}
public void setPostProcessStrategy(
PostProcessStrategy strategy) {
this.m_PostProcessStrategy = strategy;
}
}
------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Geotools-gt2-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
