Dear CF community, I would like to advertise two new CF command line tools for inspecting and combining datasets:
cfdump - view CF fields cfa - create aggregated CF datasets These utilities accept multiple input files (or URLs if DAP access is enabled) and aggregate their contents according to the CF aggregation rules (ticket #78) before either describing the combined fields (cfdump) or writing them out (cfa). These utilities are useful even if aggregation is not relevant. In that case they simply describe or write out the data variables from the input file(s). Aggregation means that all the input files are treated as though they were a single file, and individual data variables in different files are regarded as parts of a larger data variable if their coordinates and other metadata indicate that this interpretation is possible. For example, each file might contain data for surface temperature on the same lat-lon grid but with a different range of dates. The aggregated dataset then contains a single data variable covering all the dates. The CF aggregation rules are fully general so, for example, multidimensional aggregations are handled as are aggregations of fields with ancillary variables, etc. However, these utilities will not create the aggregated file unless requested; the aggregation is done in memory, using only the metadata, in the first place. cfdump ------ The cfdump tool generates text representations on standard output of the CF fields contained in the input files. It can describe fields in various amounts of detail from one-line summaries, to medium-length and complete dumps. It is complementary to other similar tools, such as ncdump, but tells you about the data from a CF point of view because it knows the CF conventions. For example, a one-line summary of a complete dataset that has been split across two files (file1.nc and file2.nc) might look like: $ cfdump -s file1.nc file2.nc <CF Field: air_temperature(time(1200), latitude(64), longitude(128)) K> See http://www.met.reading.ac.uk/~david/cfdump.1 for more examples. cfa --- The cfa tool creates and writes to disk the aggregated CF fields contained in the input files. For example (using the same files as in the cfdump example): $ cfa -o new_file.nc file1.nc file2.nc $ cfdump -s new_file.nc <CF Field: air_temperature(time(1200), latitude(64), longitude(128)) K> See http://www.met.reading.ac.uk/~david/cfa.1 for more examples. CFA-netCDF format files ----------------------- Both tools accept CFA-netCDF format files as input, and cfa also outputs such files. A CFA-netCDF file is actually a netCDF file following the CF convention with the addition of a private convention (the CFA convention) to allow the data to be omitted, because it resides in the individual files that are being aggregated. CFA-netCDF files are consequently typically rather small. The suffix .nca is suggested for CFA-netCDF files. The CFA convention has been designed specifically to cope with the full generality of aggregations made possible by the CF aggregation rules. For example (using the same files as in the cfdump example): $ cfa -f CFA -o new_file.nca file[12].nc $ cfdump -s new_file.nca <CF Field: air_temperature(time(1200), latitude(64), longitude(128)) K> See http://www.met.reading.ac.uk/~david/cfa/0.3/ for details. Met Office (UK) PP format files ------------------------------- Both utilities accept PP format files as input, but output from cfa is only in CF-netCDF or CFA-netCDF format. Therefore cfa may be used as a PP to CF converter. cf-python --------- Both utilities are built on the cf-python library, which also offers an interactive programming environment in which you may read, write, create and manipulate CF fields, including those which are larger than the available machine memory. How do you get these tools and cf-python? ----------------------------------------- They are automatically installed with the open source cf-python library version 0.9.7 (download from http://cfpython.bitbucket.org/). Currently, this library only works on Linux. I hope that these utilities will be useful. I would welcome any feedback, particularly if you find a dataset for which they don't work as expected, and will be glad to make any improvements to cfdump, cfa or the cf-python library. All the best, David -- David Hassell National Centre for Atmospheric Science (NCAS) Department of Meteorology, University of Reading, Earley Gate, PO Box 243, Reading RG6 6BB, U.K. Tel : +44 118 3785613 E-mail: d.c.hass...@reading.ac.uk _______________________________________________ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata