Thanks Jody,
Sorry for the delayed response, but I wanted to run an actual test on our
server before I reported back. We are using the database to index .nc files
based on time & elevation. I was hoping to combine parameters into that single
store as well, but after testing the jdbcconfig plugin we may not need it.
After the initial dataload (importing all 5000+ layers into the DB) our server
now boots in approximately 2 minutes which is a monumental improvement from the
original 2 hours (and is going to look good on my year end review :P ). This
allows us to continue our approach of one store/layer per unique datatype.
Thanks to Kevin Smith, as well as everyone else on the project, for their work!
Kevin M. Weiss
Software Engineer
HARRIS IT Services
From: Jody Garnett [mailto:[email protected]]
Sent: Wednesday, June 18, 2014 10:24 PM
To: Weiss, Kevin
Cc: [email protected]; Kevin Smith
Subject: Re: [Geoserver-users] Consolidation of netCDF layers
On Thu, Jun 19, 2014 at 1:14 AM, Weiss, Kevin
<[email protected]<mailto:[email protected]>> wrote:
Hello all,
Is there a way to consolidate netcdf files with multiple parameters into a
single layer? We have requirements to display large amounts of model data from
multiple sources. Currently, this involves one CoverageStore/Layer per unique
data type. However this results in thousands of layers which causes the
GeoServer boot very slowly. What we’d like to do is create one layer per model
(GFS/NAM/etc) and use custom dimensions to specify the variable name. It
doesn’t appear this is currently possible, but I figured I’d check here if I
was just missing something. For reference, our data comes in as a single
variable and time per netcdf file so our database would need to specify the
correct file and variable name.
There are a few "nD" solutions in the codebase (using a database used to index
which file to use on disk in response to time or elevation changing).
Haver a look at:
- http://docs.geoserver.org/stable/en/user/data/raster/imagemosaicjdbc.html
- http://docs.geotools.org/stable/userguide/library/coverage/jdbc/index.html
Alternatively, if there isn’t any way to combine the layers, is there any way
to optimize the boot time? We currently have 5000+ layers and our last boot
took 5673031 ms (~90 minutes). If anyone has any optimization tips or tricks I
would greatly appreciate it.
Kevin has been working on optimising the JDBC catalog implementation for large
number of layers similar to what you are working with. The "catalog " is what
geoserver uses to store all the layer names and bounds.
- http://docs.geoserver.org/stable/en/user/community/jdbcconfig/index.html
- https://github.com/geoserver/geoserver/pull/592
You could ask Kevin if you can help by testing a nightly build. You may also
wish to at some of the support<http://geoserver.org/support/> options
available.
Cheers
--
Jody
------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
_______________________________________________
Geoserver-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
