Ciao Mark, I did a quick scan on your email and therefore I am dumping some thoughts. Before moving forward with helping with your approach, I would try to understand if a different approach would be better.
Can you provide me the output of the gdalinfo on 1 of your raster? How big is each one on average? I am asking this, since I am not so sure that the right approach is to create 900 pyramid and serve them with a layer group. We might use other approaches that should be prove to be much simpler to set up and provide much more performance like a single imagepyramid and/or a single mosaic with larget/fewer tiles. Anyway, before moving forward I would need to get some more info about your data. Regards, Simone Giannecchini ------------------------------------------------------- Ing. Simone Giannecchini GeoSolutions S.A.S. Founder Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 962313 mob: +39 333 8128928 http://www.geo-solutions.it http://geo-solutions.blogspot.com/ http://www.youtube.com/user/GeoSolutionsIT http://www.linkedin.com/in/simonegiannecchini http://twitter.com/simogeo ------------------------------------------------------- On Wed, Aug 24, 2011 at 6:41 PM, Mark Peterson <[email protected]> wrote: > Greetings, > > I am trying to set up a large collection of image pyramids to be served > via GeoServer 2.1.1 WMS and am having a few issues. > > Background: > > The source data is 2010 NAIP satellite imagery consisting of about 900 > images (roughly corresponding to a US county) processed into 900 > pyramids with the GDAL tools. We added these as 900 image pyramid > stores/900 layers in GeoServer and I am currently testing with a layer > group consisting of about half of these layers. The processed imagery is > stored on a 20 TB SAN. > > Our test server is a 64 bit Windows 2008 box running 64 bit Tomcat > 7.0.20/Geoserver 2.1.1 with 4 GB of memory available to it (-Xmx4096M). > The 64 bit Java version is 1.6.0_27. We do not have GDAL or any native > JAI code installed at the moment. We are not using GeoWebCache yet. The > client software we are using is Nasa Worldwind. > > Issues: > > The main issue we are having is with GeoServer not releasing memory. > After a fresh restart of Tomcat/GeoServer, when repeatedly running the > same request (which consists of requesting a set of tiles for a view > with the eye point about 500 miles above the surface), the performance > is acceptable for about the first 7 or so transactions. After this the > performance degrades significantly until it reaches the point that the > requests just time out. > > In looking at the performance monitors, GeoServer keeps grabbing more > and more memory with every request and the performance degradation > occurs once the maximum heap size has been reached. I've tried tweaking > the various performance settings and saw the largest improvement in > memory usage when I turned off JAI Tile Recycling. > > When our GeoServer has reached maximum heap size, the first few lines of > jmap -histo are: > > num #instances #bytes class name > ---------------------------------------------- > 1: 31288522 3271712504 [B > 2: 22653 166615128 [[B > 3: 496962 33813848 [C > 4: 228869 17929440 [I > 5: 512471 16399072 java.lang.String > 6: 507465 16238880 java.util.HashMap$Entry > 7: 171957 14780896 [Ljava.util.HashMap$Entry; > 8: 95652 14719808 <constMethodKlass> > 9: 95652 13024320 <methodKlass> > 10: 228519 10997352 [Ljava.lang.Object; > 11: 9251 10996496 <constantPoolKlass> > 12: 312358 9995456 java.util.Hashtable$Entry > 13: 145426 9855192 [Ljava.util.Hashtable$Entry; > 14: 4462 9602224 > [Lit.geosolutions.imageio.plugins.tiff.TIFFField; > 15: 154694 8479584 <symbolKlass> > > Other issues (possibly related): > > 1. Certain of our stores/layers cause the following warning: "WARN > [imagemosaic.catalog] - BBOXFilterExtractor::extractBasicProperties(): > passed typename is null, using: 7". The number at the end varies. > 2. 99% of our stores/layers load in OpenLayers almost instantaneously > when loaded individually. A few generate a large amount of disk activity > and take much longer to load, these have been removed for now and we > will reprocess them later. > > Sorry for the long winded message, but I thought it was important to be > somewhat detailed. At the moment, I am in the process of building the > source code so that I can do some of my own analysis. Any insights any > of you may have would be welcome. > > Thank you for your time, > > - Mark > > ------------------------------------------------------------------------------ > EMC VNX: the world's simplest storage, starting under $10K > The only unified storage solution that offers unified management > Up to 160% more powerful than alternatives and 25% more efficient. > Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev > _______________________________________________ > Geoserver-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/geoserver-users > ------------------------------------------------------------------------------ Why Cloud-Based Security and Archiving Make Sense Osterman Research conducted this study that outlines how and why cloud computing security and archiving is rapidly being adopted across the IT space for its ease of implementation, lower cost, and increased reliability. Learn more. http://www.accelacomm.com/jaw/sfnl/114/51425301/ _______________________________________________ Geoserver-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-users
