Hi Sylvia,
please read below...
On Tue, Jan 20, 2015 at 7:33 PM, Sylvia Charchut <
[email protected]> wrote:
> Hey Daniele,
>
>
>
> I have not looked at the imageMosaic, I’m fairly new to working with this
> data, but this sounds like what I am trying to do.
>
>
>
> I am creating a StructuredGridCoverage2DReader for the netcdf files I have
> instead of using the netcdf plugin because the data I have doesn’t work as
> I need it to with the geotools plugin.
>
>
>
> I have multiple netcdf files that are separated based on time (
> meteo_january_t001.nc, meteo_january_t002.nc etc…) but aggregated on
> elevation (Some are the opposite, individual elevations but aggregated
> time). And each file has multiple coverages as in your example with
> “air_humidity” etc… The issue I was having was when I published each file
> separately by the multiple coverages it contained, it was around 4,000
> layers and took way too long for the project I am working on.
>
This is why you should consider using the ImageMosaic: it will allow to
configure a single layer per coverage so that if all your netCDF files
have, as an instance, air_temperature and air_humidity, you will have only
2 layers (instead of 4000) and all the dimensions and their values
aggregated to the same single coverage will be handled by the indexing
capabilities (so that you can query on TIME=A_TIMESTAMP_OR_A_RANGE,
ELEVATION=AN_ELEVATION_VALUE_OR_A_RANGE...)
>
> To reduce the layers, I’ve aggregated the files based on time. I was
> trying to see if I could use the granules to represent the coverages in
> this new aggregation of files but I didn’t have a clear understanding of
> granules or when that method was ever invoked.
>
>
>
> So if I have a coverage “air_humidity” and some of the data for
> air_humidity comes from meteo_january_t001.nc and the rest comes from
> meteo_january_t002.nc, can I separate this coverage into granules even
> though it overlaps multiple files? (This was also why I asked about NcML’s
> because it’s a similar structure to what I have)
>
>
I have a couple of questions for you:
Is it the time information somehow contained in the NetCDF file too (such
as a time dimension with a single value) or is it this information only
contained in the filename instead (january_t002)?
Do you have any chance to share a netCDF file for analysis or a NCDump
output?
What do you means when you say "can I separate the coverage"?
Here below, some additional chapters of the training you may check about
imageMosaic multidimensional data management and NetCDF topic. Note that it
contains sections about NetCDF data served as a single store through the
self contained plugin as well as NetCDF data served through the
imageMosaic. I'm adding "[topic]" to the list below).
[General info on ImageMosaic]:
http://geoserver.geo-solutions.it/multidim/en/imagemosaic/index.html
[NetCDF plugin (single file management)]:
http://geoserver.geo-solutions.it/multidim/en/netcdf/index.html
[NetCDF mosaicking (multiple files as a single store)]:
http://geoserver.geo-solutions.it/multidim/en/mosaic_config/netcdf_mosaic.html
Hope this helps.
Cheers,
Daniele
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==
Ing. Daniele Romagnoli
Senior Software Engineer
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.
>
> Hope that all made sense!
>
>
>
> Thank you for your response, it definitely helped clear up a lot of
> questions I’ve had.
>
>
>
> Best,
>
> Sylvia
>
>
>
> *From:* [email protected] [mailto:[email protected]] *On
> Behalf Of *Daniele Romagnoli
> *Sent:* Tuesday, January 20, 2015 10:48 AM
> *To:* Sylvia Charchut
> *Cc:* [email protected]
> *Subject:* Re: [Geotools-devel] Question on granules in
> StructuredGridCoverage2DReader
>
>
>
> Hi Sylvia,
> I would take a look to the proposal related to the
> StructuredGridCoverage2DReader:
> http://docs.codehaus.org/display/GEOTOOLS/Structured+grid+coverage+readers
>
> Note that imageMosaic already implements StructuredGridCoverage2DReader
> and it also allows to "mosaic"/"expose" set of netcdf files as a single
> coverageStore.
> What is your use case? Did you consider using ImageMosaic on top of NetCDF
> files instead of using NetCDF self-contained plugin to access multiple
> NetCDF files?
>
> Long story short a StructuredGridCoverage2DReader may be used to provide
> access to a coverageStore containing multiple coverages where we may think
> about an "air_temperature" coverage, an "air_humidity" coverage, an
> "air_current_magnitude" coverage and an "air_current_direction" coverage
> containing values at different times and elevations for air temperature,
> humidity and current...
>
> You can also think that all of these values are contained, let say into a
> single meteo-january.nc file.
>
>
>
> The StructuredGridCoverage2DReader.getGranules method is used to get a
> GranuleSource for a specific coverage (referred by the coverage name such
> as "air_temperature", as an instance) which provides access to all the
> granules of that coverage (in NetCDF, you can think about coverage =
> variable).
>
>
> A granule may be considered as an elementary 2D slice of the coverage.
> So, when you deal as an instance with a mosaic made of multiple single
> banded TIFFs where each TIFF contains data for a specific time, a specific
> elevation and a specific variable, then each TIFF will represent a granule.
>
> In case of a bigger single NetCDF file containing multiple times,
> elevations and variables, that file will provide access to multiple
> granules.
>
> Some more concepts are available as part of a training multidim.You may
> take a look to this section:
>
> http://geoserver.geo-solutions.it/multidim/en/imagemosaic/mosaic_basics.html
>
> Once you get a GranuleSource (A single GranuleSource to refer all possible
> times, elevations, values of the same Coverage) you may restrict the
> collections of granules to be returned by that source using the
> GranuleSource.getGranules(Query q) method. It returns a
> SimpleFeatureCollection where each element of the collection is a granule.
>
>
>
> In terms of NCML support, the NetCDF plugin is built on top of the
> NetCDF-java library which supports NCML. So you can do variable rename,
> dimension subsetting, variable delete through NCML and the NetCDF-java
> library will properly parse the "rules" contained in the NCML to the NetCDF
> plugin.
> Do you have some specific questions on that topic?
>
> Best Regards,
> Daniele
>
>
>
>
>
>
>
>
>
>
>
> ==
> GeoServer Professional Services from the experts! Visit
> http://goo.gl/NWWaa2 for more information.
> ==
>
> Ing. Daniele Romagnoli
> Senior Software Engineer
>
> GeoSolutions S.A.S.
> Via Poggio alle Viti 1187
> 55054 Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> -------------------------------------------------------
>
> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
> darcene notizia via e-mail e di procedere alla distruzione del messaggio
> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
> principi dettati dal D.Lgs. 196/2003.
>
>
>
> The information in this message and/or attachments, is intended solely for
> the attention and use of the named addressee(s) and may be confidential or
> proprietary in nature or covered by the provisions of privacy act
> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
> Code).Any use not in accord with its purpose, any disclosure, reproduction,
> copying, distribution, or either dissemination, either whole or partial, is
> strictly forbidden except previous formal approval of the named
> addressee(s). If you are not the intended recipient, please contact
> immediately the sender by telephone, fax or e-mail and delete the
> information in this message that has been received in error. The sender
> does not give any warranty or accept liability as the content, accuracy or
> completeness of sent messages and accepts no responsibility for changes
> made after they were sent or for other risks which arise as a result of
> e-mail transmission, viruses, etc.
>
>
>
>
>
> On Tue, Jan 20, 2015 at 4:18 PM, Sylvia Charchut <
> [email protected]> wrote:
>
> Hello dev-list,
>
>
>
> I am currently trying to implement the StructuredGridCoverage2DReader to
> use for publishing aggregated NetCDF files.
>
>
>
> What exactly is the getGranules method used for and when is this method
> called in Geoserver?
>
>
>
> I’ve tried looking through the NetCDF plugin and this is my understanding
> of a granule, it is one coverage/variable in a netcdf file (i.e water temp)
> and a GranuleSource is a list of these variables in one netcdf file. If
> this is not correct, can someone clarify?
>
>
>
> Also, can anyone point me in the right direction with how NcML files are
> handled through the NetCDF plugin?
>
>
>
> Thank you!
>
>
>
>
>
> Sylvia Charchut
>
>
>
>
>
> ------------------------------------------------------------------------------
> New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
> GigeNET is offering a free month of service with a new server in Ashburn.
> Choose from 2 high performing configs, both with 100TB of bandwidth.
> Higher redundancy.Lower latency.Increased capacity.Completely compliant.
> http://p.sf.net/sfu/gigenet
> _______________________________________________
> GeoTools-Devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/geotools-devel
>
>
>
>
> ------------------------------------------------------------------------------
> New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
> GigeNET is offering a free month of service with a new server in Ashburn.
> Choose from 2 high performing configs, both with 100TB of bandwidth.
> Higher redundancy.Lower latency.Increased capacity.Completely compliant.
> http://p.sf.net/sfu/gigenet
> _______________________________________________
> GeoTools-Devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/geotools-devel
>
>
------------------------------------------------------------------------------
New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet
_______________________________________________
GeoTools-Devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geotools-devel
