Yes this helps tremendously!
Thank you!
From: dany.geoto...@gmail.com [mailto:dany.geoto...@gmail.com] On Behalf Of
Daniele Romagnoli
Sent: Thursday, January 22, 2015 2:44 AM
To: Sylvia Charchut
Cc: geotools-devel@lists.sourceforge.net
Subject: Re: [Geotools-devel] Question on granules in
StructuredGridCoverage2DReader
Hi Sylvia,
my answers inline...
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==
Ing. Daniele Romagnoli
Senior Software Engineer
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo
è consentito esclusivamente al destinatario del messaggio, per le finalità
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro
sistema. Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the
attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act (Legislative
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in
accord with its purpose, any disclosure, reproduction, copying, distribution,
or either dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the intended
recipient, please contact immediately the sender by telephone, fax or e-mail
and delete the information in this message that has been received in error. The
sender does not give any warranty or accept liability as the content, accuracy
or completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.
On Wed, Jan 21, 2015 at 3:39 PM, Sylvia Charchut
<sylvia.charc...@nrlssc.navy.mil<mailto:sylvia.charc...@nrlssc.navy.mil>> wrote:
Daniele,
Hope this answers your questions.
Is it the time information somehow contained in the NetCDF file too (such as a
time dimension with a single value) or is it this information only contained in
the filename instead (january_t002)?
The time values are stored in the file as well. Some of the netcdf files I have
only have one tau/time value and multiple elevation/depth values. Some will
have multiple tau/time values and a single elevation/depth value.
Do you have any chance to share a netCDF file for analysis or a NCDump output?
Unfortunately I cannot share any of the data but the basic setup of most of my
data is as such:
Lon (dimension)
Lat (dimension)
Time (dimension)
Depth (dimension)
Tau (time)
Variable1 (lon,lat,time,depth)
Ok. So the time dimension information is available in the dataset.
I will definitely try to use the imageMosaic in that case. You may see the
sections of the training I have referred the past days about how to configure
the coverages and the indexer.
In some cases I will have a variable named surf_variable (lon,lat,time) in
which the assumption is it has a depth value of 0.0.
For that variable you can consider that there will not be an elevation
dimension when configuring the layer.
What do you means when you say "can I separate the coverage"?
You had said granules are a 2D slice of a coverage with one time and one
elevation value.
Can I have granules such as:
Granule1
Coverage: air_humidity
Time/Tau: 0.0
Elevation: 0.0
File: meteo_january_t001.nc<http://meteo_january_t001.nc>
Granule2
Coverage: air_humidity
Time/Tau:0.0
Elevation:3.0
File: meteo_january_t002.nc<http://meteo_january_t002.nc>
So one granule is referencing
meteo_january_t001.nc<http://meteo_january_t001.nc> and the other granule for
that coverage references a different netcdf file.
When creating an imageMosaic on top of multiple NetCDF files, an INDEX of all
the available granules is created into a DB (PostGIS, H2, Oracle, ...) with
information about their dimension values as well as their file location.
Moreover, you can add more datasets (and therefore, granules) to the INDEX,
afterwards (supposing you have a model which generates data day by day...
Therefore, when you perform a read operation or a query by specifying proper
parameters values (such as TIME = T1, ELEVATION = E1) the INDEX will be scan to
return the granules satisfying these values. So depending on the specified
dimension values the involved granules may come from different files.
Does it helps? (Sorry but I think I haven't fully understood your question
about granules separation. Not sure if you asking for "Is the imageMosaic able
to deal with granules living into different files?" ---> YES. ).
Please, let me know.
Best Regards,
Daniele
Best,
Sylvia
From: dany.geoto...@gmail.com<mailto:dany.geoto...@gmail.com>
[mailto:dany.geoto...@gmail.com<mailto:dany.geoto...@gmail.com>] On Behalf Of
Daniele Romagnoli
Sent: Wednesday, January 21, 2015 3:33 AM
To: Sylvia Charchut
Cc:
geotools-devel@lists.sourceforge.net<mailto:geotools-devel@lists.sourceforge.net>
Subject: Re: [Geotools-devel] Question on granules in
StructuredGridCoverage2DReader
Hi Sylvia,
please read below...
On Tue, Jan 20, 2015 at 7:33 PM, Sylvia Charchut
<sylvia.charc...@nrlssc.navy.mil<mailto:sylvia.charc...@nrlssc.navy.mil>> wrote:
Hey Daniele,
I have not looked at the imageMosaic, I’m fairly new to working with this data,
but this sounds like what I am trying to do.
I am creating a StructuredGridCoverage2DReader for the netcdf files I have
instead of using the netcdf plugin because the data I have doesn’t work as I
need it to with the geotools plugin.
I have multiple netcdf files that are separated based on time
(meteo_january_t001.nc<http://meteo_january_t001.nc>,
meteo_january_t002.nc<http://meteo_january_t002.nc> etc…) but aggregated on
elevation (Some are the opposite, individual elevations but aggregated time).
And each file has multiple coverages as in your example with “air_humidity”
etc… The issue I was having was when I published each file separately by the
multiple coverages it contained, it was around 4,000 layers and took way too
long for the project I am working on.
This is why you should consider using the ImageMosaic: it will allow to
configure a single layer per coverage so that if all your netCDF files have, as
an instance, air_temperature and air_humidity, you will have only 2 layers
(instead of 4000) and all the dimensions and their values aggregated to the
same single coverage will be handled by the indexing capabilities (so that you
can query on TIME=A_TIMESTAMP_OR_A_RANGE,
ELEVATION=AN_ELEVATION_VALUE_OR_A_RANGE...)
To reduce the layers, I’ve aggregated the files based on time. I was trying to
see if I could use the granules to represent the coverages in this new
aggregation of files but I didn’t have a clear understanding of granules or
when that method was ever invoked.
So if I have a coverage “air_humidity” and some of the data for air_humidity
comes from meteo_january_t001.nc<http://meteo_january_t001.nc> and the rest
comes from meteo_january_t002.nc<http://meteo_january_t002.nc>, can I separate
this coverage into granules even though it overlaps multiple files? (This was
also why I asked about NcML’s because it’s a similar structure to what I have)
I have a couple of questions for you:
Is it the time information somehow contained in the NetCDF file too (such as a
time dimension with a single value) or is it this information only contained in
the filename instead (january_t002)?
Do you have any chance to share a netCDF file for analysis or a NCDump output?
What do you means when you say "can I separate the coverage"?
Here below, some additional chapters of the training you may check about
imageMosaic multidimensional data management and NetCDF topic. Note that it
contains sections about NetCDF data served as a single store through the self
contained plugin as well as NetCDF data served through the imageMosaic. I'm
adding "[topic]" to the list below).
[General info on ImageMosaic]:
http://geoserver.geo-solutions.it/multidim/en/imagemosaic/index.html
[NetCDF plugin (single file management)]:
http://geoserver.geo-solutions.it/multidim/en/netcdf/index.html
[NetCDF mosaicking (multiple files as a single store)]:
http://geoserver.geo-solutions.it/multidim/en/mosaic_config/netcdf_mosaic.html
Hope this helps.
Cheers,
Daniele
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==
Ing. Daniele Romagnoli
Senior Software Engineer
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313<tel:%2B39%200584%20962313>
fax: +39 0584 1660272<tel:%2B39%200584%201660272>
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo
è consentito esclusivamente al destinatario del messaggio, per le finalità
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro
sistema. Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the
attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act (Legislative
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in
accord with its purpose, any disclosure, reproduction, copying, distribution,
or either dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the intended
recipient, please contact immediately the sender by telephone, fax or e-mail
and delete the information in this message that has been received in error. The
sender does not give any warranty or accept liability as the content, accuracy
or completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.
Hope that all made sense!
Thank you for your response, it definitely helped clear up a lot of questions
I’ve had.
Best,
Sylvia
From: dany.geoto...@gmail.com<mailto:dany.geoto...@gmail.com>
[mailto:dany.geoto...@gmail.com<mailto:dany.geoto...@gmail.com>] On Behalf Of
Daniele Romagnoli
Sent: Tuesday, January 20, 2015 10:48 AM
To: Sylvia Charchut
Cc:
geotools-devel@lists.sourceforge.net<mailto:geotools-devel@lists.sourceforge.net>
Subject: Re: [Geotools-devel] Question on granules in
StructuredGridCoverage2DReader
Hi Sylvia,
I would take a look to the proposal related to the
StructuredGridCoverage2DReader:
http://docs.codehaus.org/display/GEOTOOLS/Structured+grid+coverage+readers
Note that imageMosaic already implements StructuredGridCoverage2DReader and it
also allows to "mosaic"/"expose" set of netcdf files as a single coverageStore.
What is your use case? Did you consider using ImageMosaic on top of NetCDF
files instead of using NetCDF self-contained plugin to access multiple NetCDF
files?
Long story short a StructuredGridCoverage2DReader may be used to provide access
to a coverageStore containing multiple coverages where we may think about an
"air_temperature" coverage, an "air_humidity" coverage, an
"air_current_magnitude" coverage and an "air_current_direction" coverage
containing values at different times and elevations for air temperature,
humidity and current...
You can also think that all of these values are contained, let say into a
single meteo-january.nc<http://meteo-january.nc> file.
The StructuredGridCoverage2DReader.getGranules method is used to get a
GranuleSource for a specific coverage (referred by the coverage name such as
"air_temperature", as an instance) which provides access to all the granules of
that coverage (in NetCDF, you can think about coverage = variable).
A granule may be considered as an elementary 2D slice of the coverage.
So, when you deal as an instance with a mosaic made of multiple single banded
TIFFs where each TIFF contains data for a specific time, a specific elevation
and a specific variable, then each TIFF will represent a granule.
In case of a bigger single NetCDF file containing multiple times, elevations
and variables, that file will provide access to multiple granules.
Some more concepts are available as part of a training multidim.You may take a
look to this section:
http://geoserver.geo-solutions.it/multidim/en/imagemosaic/mosaic_basics.html
Once you get a GranuleSource (A single GranuleSource to refer all possible
times, elevations, values of the same Coverage) you may restrict the
collections of granules to be returned by that source using the
GranuleSource.getGranules(Query q) method. It returns a SimpleFeatureCollection
where each element of the collection is a granule.
In terms of NCML support, the NetCDF plugin is built on top of the NetCDF-java
library which supports NCML. So you can do variable rename, dimension
subsetting, variable delete through NCML and the NetCDF-java library will
properly parse the "rules" contained in the NCML to the NetCDF plugin.
Do you have some specific questions on that topic?
Best Regards,
Daniele
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==
Ing. Daniele Romagnoli
Senior Software Engineer
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313<tel:%2B39%200584%20962313>
fax: +39 0584 1660272<tel:%2B39%200584%201660272>
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo
è consentito esclusivamente al destinatario del messaggio, per le finalità
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro
sistema. Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the
attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act (Legislative
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in
accord with its purpose, any disclosure, reproduction, copying, distribution,
or either dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the intended
recipient, please contact immediately the sender by telephone, fax or e-mail
and delete the information in this message that has been received in error. The
sender does not give any warranty or accept liability as the content, accuracy
or completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.
On Tue, Jan 20, 2015 at 4:18 PM, Sylvia Charchut
<sylvia.charc...@nrlssc.navy.mil<mailto:sylvia.charc...@nrlssc.navy.mil>> wrote:
Hello dev-list,
I am currently trying to implement the StructuredGridCoverage2DReader to use
for publishing aggregated NetCDF files.
What exactly is the getGranules method used for and when is this method called
in Geoserver?
I’ve tried looking through the NetCDF plugin and this is my understanding of a
granule, it is one coverage/variable in a netcdf file (i.e water temp) and a
GranuleSource is a list of these variables in one netcdf file. If this is not
correct, can someone clarify?
Also, can anyone point me in the right direction with how NcML files are
handled through the NetCDF plugin?
Thank you!
Sylvia Charchut
------------------------------------------------------------------------------
New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net<mailto:GeoTools-Devel@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geotools-devel
------------------------------------------------------------------------------
New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net<mailto:GeoTools-Devel@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geotools-devel
------------------------------------------------------------------------------
New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
