Hi,
I think Jim is talking about NCML (virtual) aggregation. THREDDS can
aggregate even when the variable is a scalar using "joinNew". But I
think THREDDS can aggregate only over regular arrays. That is, all the
dimensions other than over which a variable is aggregated should be of
same size across all the files. This is possible, for example, only if
all the profiles have same depth( or pressure) levels. Which in general
is not true. NCML aggregation I guess is of limited use when dealing
with jagged arrays. But I agree with Jim in that ability to aggregate
should be an important consideration when coming up with a
representation. Physical aggregation is still possible. But I prefer
virtual aggregation, since letting data to be stored in individual files
in often operationally convenient, but users benefit from aggregated
views. I wonder if NCML could be extended to be able to aggregate jagged
arrays into incomplete multidimensional array representations. I heard
that ERDDAP has similar capability though I don't think it has anything
like NCML where users can create views over remote data, not sure though.
Upendra
On 4/3/2012 1:27 PM, Jim Biard wrote:
Hi.
A colleague of mine has told me that if you use size-1 array for a
variable, then data servers such as THREDDS can aggregate the variable
over multiple files and deliver a single file in which the variables
will be arrays of size > 1. He said that if a variable is a scalar,
THREDDS won't do this. (I don't mess with THREDDS, so I am just
parroting what he said.)
If this is correct, then you might want to consider this point when
deciding which way to represent coordinates.
Grace and peace,
Jim
On Tue, Apr 3, 2012 at 1:02 PM, Upendra Dadi <[email protected]
<mailto:[email protected]>> wrote:
Andrew,
However I have another small concern about all of this. Given
there is more than
1 way to treat dimensions does the netCDF plotting and
visualising sofware out
there understand both approaches. That is can we expect something
like
ncBrowse, IDV or ODV? give me a plot of say salinity vs. pressure
and work OK with BOTH approaches to coding the netCDF?
Software like ncBrowse do not understand CF conventions. They
can read netCDF files, but wouldn't know how to interpret the
attributes like "coordinates". So if you want to plot the profile
location on a map, for example, it wouldn't be able to do it by
itself. It would need to be told how to interpret the
coordinates. A CF based software wouldn't have to be supplied
with the additional semantics since it can read from the file by
itself. But if you want to plot salinity vs pressure, software
like ncBrowse can already do it since lot of these software make
it easy to make plots based on a shared dimension. And here
salinity and pressure share the same dimension. So I guess, the
correct answer to your question is - it depends on what kind of
plot or task you want to do and if the software can understand CF
conventions.
Upendra
Thanks and Regards All,
Andrew Walsh
----- Original Message -----
*From:* Lowry, Roy K. <mailto:[email protected]>
*To:* Upendra Dadi <mailto:[email protected]> ; andrew
walsh <mailto:[email protected]>
*Cc:* Luke Callcut <mailto:[email protected]> ;
[email protected] <mailto:[email protected]> ;
[email protected] <mailto:[email protected]>
*Sent:* Tuesday, April 03, 2012 9:31 AM
*Subject:* RE: [CF-metadata] Ocean CTD data following CF
Conventions v1.6
Hi Upendra,
I like the idea of a station dimension. It goes a long way
to resolving the issue raised in my response to Jim which was
based on the tunnel vision of having pressure/depth as a
dimension. I have yet to look at the recently published NODC
NetCDF templates. Is this CTD encoding included in them? If
so, I'll bump up looking at them on my 'todo' list. I'd also
recommend that Andrew and my colleagues in SeaDataNet take a
look.
Cheers, Roy.
------------------------------------------------------------------------
*From:* [email protected]
<mailto:[email protected]>
[[email protected]
<mailto:[email protected]>] On Behalf Of
Upendra Dadi [[email protected]
<mailto:[email protected]>]
*Sent:* 02 April 2012 17:21
*To:* andrew walsh
*Cc:* Luke Callcut; [email protected]
<mailto:[email protected]>
*Subject:* Re: [CF-metadata] Ocean CTD data following CF
Conventions v1.6
Hi Andrew,
Either way it should be okay as far as CF compliance is
concerned. But the dimensions - latitude, longitude and time
are not really required. If it is required to indicate that
there is only one station(profile) in the file, there could
be a dimension for number of stations instead, with a value
of 1. Also, using a station dimension is the way to go if
storing a collection of profiles in a single file. Here at
NODC, we took the approach that we would use the same
consistent representation whether there is a single instance
or a collection in a file.
Upendra
On Mon, Apr 2, 2012 at 2:51 AM, andrew walsh
<[email protected] <mailto:[email protected]>> wrote:
Hi CF lis
We are working on coding up some 1000's netCDF files off CTD
instruments and want to make usre we are following the
latest netCDF conventions (v1.6) OK. As background the
CTD records
a profile pressure, temperature and salinity.
Here is a summarised CDL version (not all
attributes+variables+qc flags there, just majors for now)
of what we propose:
dimensions:
TIME=1
PRESSURE=729
LATITUDE=1
LONGITUDE=1
variables:
double TIME(TIME) ;
TIME:standard_name = "time" ;
TIME:units = "days since 1950-01-01 00:00:00Z" ;
TIME:axis = "T" ;
TIME:valid_min = 0. ;
TIME:valid_max = 999999. ;
double LATITUDE(LATITUDE) ;
LATITUDE:standard_name = "latitude" ;
LATITUDE:units = "degrees_north" ;
LATITUDE:axis = "Y" ;
LATITUDE:valid_min = -90. ;
LATITUDE:valid_max = 90. ;
double LONGITUDE(LONGITUDE) ;
LONGITUDE:standard_name = "longitude" ;
LONGITUDE:units = "degrees_east" ;
LONGITUDE:axis = "X" ;
LONGITUDE:valid_min = -180. ;
LONGITUDE:valid_max = 180. ;
double PRESSURE(PRESSURE) ;
PRESSURE:standard_name = "sea_water_pressure" ;
PRESSURE:units = "decibars" ;
PRESSURE:axis = "Z" ;
PRESSURE:valid_min = 0. ;
PRESSURE:valid_max = 12000. ;
PRESSURE:positive = "down" ;
double TEMPERATURE(PRESSURE) ;
TEMPERATURE:standard_name = "sea_water_temperature" ;
TEMPERATURE:units = "degrees_C" ;
TEMPERATURE:_FillValue = -99.99 ;
TEMPERATURE:valid_min = -2. ;
TEMPERATURE:valid_max = 40. ;
TEMPERATURE:coordinates="TIME LATITUDE LONGITUDE PRESSURE"
double SALINITY(PRESSURE) ;
SALINITY:standard_name = "sea_water_salinity" ;
SALINITY:units = "psu" ;
SALINITY:_FillValue = -99.99 ;
SALINITY:valid_min = 0. ;
SALINITY:valid_max = 40. ;
SALINITY:coordinates="TIME LATITUDE LONGITUDE PRESSURE"
// global attributes:
:conventions = "CF-1.6" ;
:featureType = "profile"
:cdm_data_type = "profile"
+ several other attributes later for ISO19115 metadata
generation
I am not sure if I should have TEMPERATURE and SALINITY
arrays with 4 dimensions
like TEMPERATURE(TIME,LATITUDE,LONGITUDE,PRESSURE) or
just 1 dimension
like I have above i.e. TEMPERATURE(PRESSURE). ?
Any feedback on the above is greatly appreciated.
Andrew Walsh
_______________________________________________
CF-metadata mailing list
[email protected] <mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
This message (and any attachments) is for the recipient only.
NERC
is subject to the Freedom of Information Act 2000 and the
contents
of this email and any reply you make may be disclosed by NERC
unless
it is exempt from release under the Act. Any material
supplied to
NERC may be stored in an electronic records management system.
_______________________________________________
CF-metadata mailing list
[email protected] <mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
Jim Biard
Research Scholar
Cooperative Institute for Climate and Satellites
Remote Sensing and Applications Division
National Climatic Data Center
151 Patton Ave, Asheville, NC 28801-5001
[email protected] <mailto:[email protected]>
828-271-4900
_______________________________________________
CF-metadata mailing list
[email protected]
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
[email protected]
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
