Here's the documentation from the DX user's guide (Appendix B), that describes the support within the Import module for netCDF.
(See attached file: netcdf2.htm)(See attached file: netcdf1.htm) Having said that, there are limitations in what's implemented in Import. It's not a limitation of DX itself, since the DX data model handles a lot more types of data than netCDF can. One example is that not all attributes that might be in a netCDF file are imported as queryable as DX attributes. You'll need to have that specified at a field level, if I recall correctly, to get them Import'd, as you apparently have seen. If you or someone else is interested in some of the key issues that should be addressed in the netCDF support, I can provide some details. However, consistent with netCDF itself, there is no direct support of connection-dependent data when reading netCDF. The assumption is that the data are position-dependent. Like netCDF, neither CDF or HDF make a distinction between what DX what view as position or connection dependence. Hence, the DX Import treats them all as position-dependent. One solution within DX after Import'g is to use Post. But this changes the data (interpolates), which is often not desirable. As an alternative, here's a macro that will convert from position-dependent to connection-dependent for regular connections without transforming the data. (See attached file: ConvertToCellCentered.net) -------------------------- Lloyd A. Treinish Deep Computing Institute IBM Thomas J. Watson Research Center P. O. Box 704 Yorktown Heights, NY 10598 914-784-5038 (voice) 914-784-7667 (facsimile) [EMAIL PROTECTED] http://www.research.ibm.com/people/l/lloydt/ http://www.research.ibm.com/weather Erik Engquist <[EMAIL PROTECTED]>@opendx.watson.ibm.com on 04/14/2000 08:35:37 AM Please respond to [email protected] Sent by: [EMAIL PROTECTED] To: [email protected] cc: Subject: [opendx-users] Importing NetCDF files with cell centered data Hi, I've been looking at importing geometry data from netCDF files into DX. It seems to work fine except for the case where the field data is connection dependent. I can add a data component to the field and make this component connection dependent, but any dependancy attributes added to the field itself are ignored by the importer. Is this a weakness of the netCDF importer, or am I missing something? As an example, the following works: dimensions: npoints = 10 ; npecs = 20 ; ndim = 3 ; quad = 4 ; variables: float coord(npoints, ndim) ; int connect(npecs, quad) ; float filler(npoints) ; filler:field = "filler, scalar" ; filler:connections = "connect, quads" ; filler:positions = "coord" ; filler:component = "surfcurr, data, scalar" ; float surfcurr(npecs) ; surfcurr:attributes = "dep, connections" ; But this does not: dimensions: npoints = 10 ; npecs = 20 ; ndim = 3 ; quad = 4 ; variables: float coord(npoints, ndim) ; int connect(npecs, quad) ; float surfcurr(npecs) ; surfcurr:field = "surfcurr, scalar" ; surfcurr:connections = "connect, quads" ; surfcurr:positions = "coord" ; surfcurr:attributes = "dep, connections" ; -Erik -- Erik Engquist [EMAIL PROTECTED] NADA KTH tel: +46 8 790 7819 100 44 Stockholm fax: +46 8 790 0930 Sweden www.nada.kth.se/~erikeTitle: IBM Visualization Data Explorer User's Guide
IBM Visualization Data Explorer User's Guide[ Bottom of Page | Previous Page | Next Page | Table of Contents | Partial Table of Contents | Index ]
B.5 netCDF Files: Complex FieldsFor data with more complex structure, conventions have been established for netCDF variable attributes, as described in the format below. The notation used corresponds to that of the netCDL "language."
Irregular ArraysThis section describes how to specify netCDF variables for components with irregular values.
DataTo indicate that a netCDF variable contains values corresponding to the data component, it must have the following attribute:
variable1:field = "fieldname";
Variable1 is the name of the netCDF variable containing data
values to be imported. fieldname is the name
of the Data Explorer field by which the
user refers to the data
(for example, "temperature," "pressure,"
"wind").
If more than one variable is
tagged with the same field name, each
variable is read into a field, and the fields are collected
into a group.
The data are read in as an array of values, one number per grid point. If the data are actually a vector or a matrix at each grid point, use one of the following modifiers:
If the data have both regular connections and regular positions, no other attributes are required. A regular grid is assumed, with the origin at 0.0, and a spacing of 1.0 along each axis. The number of axes will be determined from the number of dimensions in the data array.
PositionsIf the locations of the data values in variable1 do not form a regular lattice (with origins at 0.0 and spacings of 1.0), the name of a netCDF variable that contains the position information must be specified as an attribute for variable1. There are five different types of position specifications: none, completely regular, completely irregular, and two types of partially regular. Completely irregular is assumed if the following attribute is specified:
variable1:positions = "variable2";
where variable2 is an array of vectors, one for each grid point,
defining its location.
The dimensionality of the data space is
determined by the number of items in a vector.
Regular positions can be specified with just the origin and spacing between grid points along each axis in compact form. The following attribute is used:
variable1:positions = "variable2, compact";
where variable2 is the name of a n×2
array containing
origin, delta pairs for the spacing and
location of positions along each axis.
The number of positions
along each axis is determined from the
shape of variable1.
Positions that can be specified as the product of arrays containing the location of points along each axis can be input in product form. Use the following attribute:
variable1:positions = "variable2a, product;
variable2b, product;
.
.
.
variable2x, product";
where the variable2's are each the name of an array containing
a list of positions along that axis.
The number of items in each
array must match the length of the corresponding axis in the original
variable1 data array.
If any of the axes in an partially regular product array are actually regular, they can be specified in compact form:
variable1:positions = "variable2a, product, compact;
variable2b, product;
.
.
.
variable2x, product";
where variable2a is the name of an origin, delta array, and the rest
are position lists as before.
ConnectionsIf the connections between positions is a regular lattice, no additional attributes are necessary. For 1-D data, connections of "lines" is assumed. 2-D data implies "quads," 3-D data implies "cubes" and for higher dimensions, "hypercubes" is assumed. If the connections are irregular, use one of the following attributes:
Additional ComponentsIf additional component information is present in the file, the following attributes are valid:
variable1:component = "variable4, componentname, scalar;
variable5, componentname, vector;
variable6, componentname, matrix";
and
variable4:attributes = "ref, componentname;
dep, componentname";
Series DataThere are three ways to specify the import of datasets that should be treated as series. They are:
Single VariableWhen all data values are defined as a single netCDF variable, and the unlimited dimension of the variable is to be interpreted as the series dimension, then use one of the following forms of the field attribute:
The position and connection information is assumed to be constant for all members of the series. If the positions or connections change for each step of the series, then the variables used for those arrays must also have an unlimited dimension that corresponds one-for-one with the data array. An example using this method is provided in "Partially Regular Grids and Time Series".
Separate VariablesWhen there are separate netCDF variables defined for each step in the series, but all variables are in the same file, use the following global attribute tags:
:seriesxxx = "fieldname;
variable1a;
variable1b;
.
.
.
variable1x";
or
:seriesxxx = "fieldname;
variable1a, float_value;
variable1b, float_value;
.
variable1x, float_value";
where the global tag must have the first 6
characters series.
Global tags must be unique,
so additional characters can be added
to distinguish them.
Each variable1x is the name array containing the data for that step. In the first format, the spacing of the steps is assumed to be 1.0. In the second format, the float_value is the value of each step. All other specifications are the same as for simple fields.
Separate FilesWhen there are netCDF variables in separate files that make up the steps of a series, use the following global attribute tags:
:seriesxxx = "fieldname, files;
filename1;
filename2;
.
.
.
filenameN";
or
:seriesxxx = "fieldname, files;
filename1, float_value;
filename2, float_value;
.
.
.
filenameN, float_value";
where the global tag must have the first 6
characters series. Global tags must be unique,
so additional characters can be added
to distinguish them.
Each filenameN is the name of the netCDF file that contains the data variables for that step. In the first format, the spacing of the steps is 1.0. In the second format, the float_value is the value of each step. All other specifications are the same as for simple fields. This format can be used to create short term series within a file, and then have a series of these smaller series.
ExamplesThis section shows examples of netCDF files in the netCDL description language. See the documentation supplied by UCAR for more information on netCDL and the ncgen and ncdump utilities.
Compact Specifications of Regular DimensionsThis example describes a single two-dimensional scalar field on a latitude-longitude, regular, rectangular grid. The example data are temperature on a one-degree grid with global coverage. Because Data Explorer array objects can be specified compactly, you can use this method to specify a netCDF with regular dimensions. For each dimension, you need to specify its value at the origin and its spacing along the dimension.In this example, two variable attributes are defined for the netCDF variables. field specifies the rank of the parameter, and positions specifies where the information containing the locations of the data is space is located.
dimensions:
lon = 360;
lat = 180;
naxes = 2;
ndeltas = 2;
variables:
float locations(naxes, ndeltas);
float temperature(lat, lon);
temperature:field = "temperature, scalar";
temperature:positions = "locations, regular";
data:
locations = 89.5, -1., // compact specification, origin and
-179, 1.; // spacing for lat and lon
temperature = ... // Data for temperature
Partially Regular Grids and Time SeriesThis example describes an ocean circulation model that consists of a time series of four three-dimensional scalars (temp, sali, wata, and conv) and one three-dimensional 3-vector (vel). netCDF typically requires seven variables, all scalars (the vector counting as three scalars). The coordinate system for the velocity vectors corresponds to that of the grid (that is, +u implies north, +v implies east, and +w implies down). These grids are partially regular in that the time, tlat, and tlon portions (three out of the four dimensions) are all regularly spaced. time is to be mapped to members of a series group. The fourth dimension, tlvl, is irregularly spaced. The compact notation can be used for the regular notation, while the all values along the irregular dimension must be specified; a product is formed from the dimensions. Here is the specification in netCDL notation:
dimensions:
time = UNLIMITED;
tlat = 30;
tlon = 50;
tlvl = 30;
vsize = 3; // At each grid cell for variable vel, there are
// three floats for the u, v, and w components of the
// vector field.
naxes = 3;
ndeltas = 2;
variables:
float lat_axis(ndeltas, naxes);
float lon_axis(ndeltas, naxes);
float level_axis(tlvl, naxes);
float temp(time, tlat, tlon, tlvl);
temp:field = "temperature, scalar, series";
temp:positions = "lat_axis, product, compact; lon_axis,
product, compact; level_axis, product";
float sali(time, tlat, tlon, tlvl);
sali:field = "salinity, scalar, series";
sali:positions = "lat_axis, product, compact; lon_axis,
product, compact; level_axis, product";
float wata(time, tlat, tlon, tlvl);
wata:field = "water parage, scalar, series";
wata:positions = "lat_axis, product, compact; lon_axis,
product, compact; level_axis, product";
float conv(time, tlat, tlon, tlvl);
conv:field = "covective index, scalar, series";
conv:positions = "lat_axis, product, compact; lon_axis,
product, compact; level_axis, product";
float vel(time, tlat, tlon, tlvl, vsize);
vel:field = "velocity, vector, series";
vel:positions = "lat_axis, product, compact; lon_axis,
product, compact; level_axis, product";
data:
lat_axis = -14.667, 0., 0.,
0.333, 0., 0.;
lon_axis = 0.0, -99.8, 0.0,
0.0, 0.5, 0.0;
level_axis = 0.0, 0.0, 17.5,
0.0, 0.0, 53.425,
.
:
0.0, 0.0, 5374.98;
temp = ... ;
sali = ... ;
wata = ... ;
conv = ... ;
vel = ... ;
Irregular SurfaceThis example is the netCDL description of a netCDF for an irregular surface, that of the classic teapot. It has precomputed normals, which are imported as the "normals" component, in addition to positions and connections.
netcdf teapot8 { // name of datafile is "teapot8.ncdf"
// name of field is "surface"
dimensions:
pointnums = 2268;
trinums = 3584;
axes = 3;
sides = 3;
variables:
float locations(pointnums, axes);
float normalvect(pointnums, axes);
long tris(trinums, sides);
float surfacedata(pointnums);
// global attributes:
:source = "Classic Teapot, data from Turner Whitted";
// specific attributes:
surfacedata:field = "surface";
surfacedata:connections = "tris, triangles";
surfacedata:positions = "locations";
surfacedata:component = "normalvect, normals, vector";
normalvect:attributes = "dep, positions";
// This is the start of a large data section
data:
·
}
[ Top of Page | Previous Page | Next Page | Table of Contents | Partial Table of Contents | Index ] [Data Explorer Documentation | QuickStart Guide | User's Guide | User's Reference | Programmer's Reference | Installation and Configuration Guide ] [Data Explorer Home Page | Contact Data Explorer | Same document on Data Explorer Home Page ] [IBM Home Page | Order | Search | Contact IBM | Legal ] |
IBM Visualization Data Explorer User's Guide[ Bottom of Page | Previous Page | Next Page | Table of Contents | Partial Table of Contents | Index ]
B.4 netCDF FilesData Explorer supports the importation of data in netCDF format, a data abstraction for self-describing multidimensional Arrays. It represents a simpler data model than that of Data Explorer, one similar to that of the Array Object. Data are accessed in netCDF through an application programming interface (available in C and FORTRAN libraries from the Unidata Program Center--in Boulder, Colorado). Scalar data on a regular grid can be imported from a standard netCDF file. To import vector data, data on irregular grids, or time series data, additional attributes must be added to the netCDF file. These attributes allow you to specify the data, positions, and connections components of your data set. See B.5 , "netCDF Files: Complex Fields" for more information about these attributes.
Regular GridsTo import scalar data on a regular grid, specify the netCDF file name as the name parameter. By default, all netCDF variables are imported and collected into a Group. To import one or more particular variables, specify their names as the variable parameter. The format parameter must be "netCDF." Data Explorer automatically constructs positions and connections for each variable, with an origin of 0.0 and spacings of 1.0 along each dimension. For data that is logically a vector Field, but whose values are stored in three separate netCDF variables, each component of the vector can be imported separately; the Compute module can then be used to create a single vector Field. For data that is logically a vector Field, but whose values are stored as an n+1 dimensional regular grid, use the Slice and Compute modules to separate the components of the vector, and then recombine them into a single vector Field.
Example of a Regular GridThe following file describes a 3 × 3 × 3 regular grid at origin 0, 0, 0 with deltas of 1.0 along each axis.
netCDF volume {
dimensions:
nx = 3;
ny = 3;
nz = 3;
variables:
float field_data(nx, ny, nz);
data:
field_data =
0, 0, 0
0, 0, 0
0, 5, 0
0, 0, 5
0, 0, 0
0, 0, 0
5, 0, 0
0, 0, 0
0, 0, 0;
}
netCDF on completely regular grids can be imported directly by Data Explorer without modifying the netCDF file. See B.4 , "netCDF Files" for more information. [ Top of Page | Previous Page | Next Page | Table of Contents | Partial Table of Contents | Index ] [Data Explorer Documentation | QuickStart Guide | User's Guide | User's Reference | Programmer's Reference | Installation and Configuration Guide ] [Data Explorer Home Page | Contact Data Explorer | Same document on Data Explorer Home Page ] [IBM Home Page | Order | Search | Contact IBM | Legal ] |
ConvertToCellCentered.net
Description: Binary data
