Re: [Geotools-gt2-users] [udig-devel] Poor performance using Filter with shapefile...

Thu, 21 Sep 2006 09:37:08 -0700

Sebastian,

I am not sure what your filter actually is, but I can tell you a few things for sure:
- if it is an attribute filter then it will be slow, because there is no leverage at all available from indexing attributes in shape files
- if it is  a spatial filter, then you might need to reconfigure it a bit to ensure that it makes use of the spatial index on the shape file, at which point it should work a little better
- in general, even a spatially indexed filter query on shape file might underperform a non-indexed query in memory, if only due to the massive disparity in access time overhead, as Andrea has discussed (though filesystem caching at the operating system level might make second and third runs through the filter faster as portions of the index and file get pulled into the filesystem memory cache)

I am guessing that since your use case is geocoding you are doing lots of attribute comparisons.  In order to make those fast, if you insist on keeping your data in shape files, you will need to enhance the shapefile datastore to support attribute indexes.  I am pretty sure that this is what ArcView does, for example, in its geocoding routines.

Paul

On 21-Sep-06, at 8:13 AM, [EMAIL PROTECTED] wrote:

Hi all!

We have created a new Udig's wizard which allow us to geocode data coming from xls files. This wizard allow us to :

- choose excel file,
- choose layer from the active map,
- choose an attribute from the layer and a column from the xls file in order to map data,
- then launch our "mapper" to match data.

Our problem coming from the map action in which we use a filter :

             for(int iRow = bUtiliserLaPremiereLignePouTitre?1:0 ; iRow < iNbLigneAImporter; iRow++){
                leRight.setLiteral(sheet.getCell(iColonneNumber, iRow).getContents());

                debug("Start "+ iRow +" check feature collection here");
                fcResultat = fsLayerSelectionne.getFeatures(cf);
                if (!fcResultat.isEmpty()){
                    debug("End "+ iRow +" check feature collection here");

                    Feature f  = null;
                    Iterator iFeature = fcResultat.iterator();
                    f = (Feature) iFeature.next();
                    clCoucheResultat.createCustomFeature(f.getDefaultGeometry().getCentroid(), sheet.getCell(0,iRow).getContents(),sheet.getCell(iColonneNumber, iRow).getContents());
                }
            }

We have to wait a "long" time during the "fcResultat.isEmpty()" when using a shapefile. (it's a long time because we may have thousands of lines)

See the debug info :

[DEBUG_GeoCodageExcelWizard_16:47:04 453] Start 338 check feature collection here
[DEBUG_GeoCodageExcelWizard_16:47:04 484] End 338 check feature collection here
=> 31ms to apply the filter and query.


When I put this layer in Memory (By creating a a new Layer in udig with MemoryDataStore) performances are much better!!

[DEBUG_GeoCodageExcelWizard_16:51:40 187] Start 338 check feature collection here
[DEBUG_GeoCodageExcelWizard_16:51:40 203] End 338 check feature collection here
=> 16ms

I would like to know if such a difference is normal, and how way could we reduce calculation time by other way than creating a new 'in memory" layer?
I haven't trying this using other data source (postgis for example) do you think performances will be better?


Thanks in advance,

Sebastien

_______________________________________________
User-friendly Desktop Internet GIS (uDig)

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geotools-gt2-users mailing list
Geotools-gt2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users

Reply via email to