Access to spatial index is very inefficient
-------------------------------------------
Key: GEOT-1490
URL: http://jira.codehaus.org/browse/GEOT-1490
Project: GeoTools
Issue Type: Bug
Components: data shapefile
Affects Versions: 2.4-RC0
Reporter: Andrea Aime
Assignee: Jesse Eichar
Fix For: 2.4-RC1, 2.5-M1
Accessing the spatial index is done allocating and reading raft of minuscule
buffers in FileSystemNode.
Profiling shows that this is a very expensive operation, both when accessing a
very zoomed in area and when reading the whole file, and that accessing the
index is always more expensive than accessing the shapefile itself.
The fix is to replicate what the shapefile reader does: instead of allocating a
million small buffers, allocate just a direct one, eventually grow it on need,
and have it read significant chunks of the file at a time, starting with 8
kilobytes and doubling that size on demand.
As usual, I'm assigning this to you for review.
I already implemented the solution, all tests are passing, and GeoServer
benchmark results are astounding:
* a bench with 60 threads asking for a small area of a 400MB shapefile shows a
drop in average response time from 1181ms to 895ms
* a request of the whole above file to be rendered into a single 8000x8000
pixels map dropped down from somewhat above a minute
to 17seconds (13seconds if the file is already stored in the file system
cache).
If you're curious to see the map, it's here (rendered in a high compression
1bit png, only 1MB):
http://geo.openplans.org/aaime/wmsTexasHuge.png
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geotools-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geotools-devel
