Hi all, I'm trying to use Geoserver with large NetCDF files and have been encountering some issues.
When working with large rasters, indexing takes a long time and causes the front end and REST api to become unresponsive. Some of the climate data I work with is ~500*1000*55000*32bit*3variables (150 years of downscaled daily data) = ~330GB per file. After ~3 hours of indexing however, it stops processing and errors out: 20 Jul 15:38:16 WARN [netcdf.NetCDFFormat] - Unable to connect org.geotools.data.DataSourceException: Unable to connect Caused by: org.geotools.data.DataSourceException: java.io.IOException: Error occured on rollback Caused by: java.lang.RuntimeException: java.io.IOException: Error occured on rollback Caused by: java.io.IOException: Error occured on rollback Caused by: org.h2.jdbc.JdbcSQLException: IO Exception: java.io.IOException: Negative seek offset; /var/lib/tomcat7/webapps-available/geoserver2711/geoserver_data/climate-data/.pr+tasmax+tasmin_day_BCCAQ+ANUSPLIN300+ACCESS1-0_historical+rcp45_r1i1p1_19500101-21001231_57daab961b0813f95a47bf4e3312d0f79cde766b/pr+tasmax+tasmin_day_BCCAQ+ANUSPLIN300+ACCESS1-0_historical+rcp45_r1i1p1_19500101-21001231.7212500018817963839.temp.db; SQL statement: ROLLBACK [90031-119] Caused by: java.io.IOException: Negative seek offset 20 Jul 15:38:16 INFO [geoserver.web] - Getting list of coverages for saved store /var/lib/tomcat7/webapps-available/geoserver2711/geoserver_data/climate-data/pr+tasmax+tasmin_day_BCCAQ+ANUSPLIN300+ACCESS1-0_historical+rcp45_r1i1p1_19500101-21001231.nc java.lang.RuntimeException: Could not list layers for this store, an error occurred retrieving them: Failed to create reader from /var/lib/tomcat7/webapps-available/geoserver2711/geoserver_data/climate-data/pr+tasmax+tasmin_day_BCCAQ+ANUSPLIN300+ACCESS1-0_historical+rcp45_r1i1p1_19500101-21001231.nc and hints null Caused by: java.io.IOException: Failed to create reader from /var/lib/tomcat7/webapps-available/geoserver2711/geoserver_data/climate-data/pr+tasmax+tasmin_day_BCCAQ+ANUSPLIN300+ACCESS1-0_historical+rcp45_r1i1p1_19500101-21001231.nc and hints null At that point it had created 78 <fname>.<fnum>.log.db files at 33MB each, with the last one (fnum=78) being 26MB, and a single ~2.3GB *.temp.db file. I have measured read speeds on this source and they run ~350MB/s. We should be able to read (and index) the entire file in at most 15 minutes single threaded. Does anyone else have experience using Geoserver with large NetCDF files? Geoserver 2.7.1.1 Tomcat 7.0.42 Java 1.7.0_65-b32 NetCDF plugin from: http://ares.opengeo.org/geoserver/2.7.x/community-latest/ Thanks, Basil -- Basil Veerman Web Application Developer Pacific Climate Impacts Consortium http://www.pacificclimate.org/ Tel: (250) 721-6395 ------------------------------------------------------------------------------ _______________________________________________ Geoserver-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-users
