Hello Richard At present these netCDF4/HDF5 user defined data types are not allowed in CF-netCDF, as you can see from the output of the checker; this is also stated in the CF conventions document.
The same is true for groups, these are not allowed within CF-netCDF at version 1.6. There has been a fair amount of discussion on this topic recently. I attended a GoESSP meeting last year and a Unidata NetCDF meeting this year where the need to understand the path to get netCDF4 features adopted for CF was much in evidence. There is also concern about how to manage the data sets not using these features as this are adopted in new versions of CF. There is a discussion space for a future major version of CF2 https://github.com/cf-convention/CF-2/issues It would be great to capture your requirements in a new issue here. I don't think that this should preclude further discussion on adopting these types in a CF 1.x version, CF1.7 may be able to accommodate such things. It would be interesting to understand reasons why this particular feature may or may not be suitable for CF1.7. Are there further insights from others in the community on whether these user defined data types can be incorporated into CF1.7? all the best mark ________________________________________ From: CF-metadata [[email protected]] on behalf of Richard Olav Rud [[email protected]] Sent: 06 July 2016 14:08 To: [email protected] Subject: Re: [CF-metadata] Support for compound data types Dear Alison and Maarten, Thanks for your quick reply. To be a little more specific, we are not in need of implementing a new standard name. By "Compound data types", I mean the netCDF-4 user defined data type (http://www.unidata.ucar.edu/software/netcdf/docs/user_defined_types.html). Using the netcdf-4 format you have support for this data type, but I can not seem to find any information about this subject related to the CF-convention. Since we have metadata per-variable that changes over time, we were thinking about packing it in a nested structure (compound type). We split up our variable in multiple instances, where each instance could potentially hold a new set of metadata. This could be the case when for example a new data-submitter has taken over the responsibility of reporting data from a specific station or that the instrument-model has changed etc. So for a given instance of a variable, we could easily have over 100 metadata fields related to stations, data originators, instruments etc. Basically I have played around creating metadata as compound types, but this does not pass the CF compliance checker (http://puma.nerc.ac.uk/cgi-bin/cf-checker.pl). >From my understanding Maarten, you use groups to separate your metadata from >your product? I will look more into this. So I was wondering if there are any plans to allow for this custom data type in future revisions of the CF-convention, or if there are any other approaches that you might suggest regarding this issue. All the best, Richard Rud Atmosphere and Climate Department (ATMOS) ________________________________ NILU - Norsk institutt for luftforskning Norwegian Institute for Air Research PO Box 100, 2027 KJELLER Besøk/visit: Instituttveien 18, 2007 Kjeller E-mail: [email protected] www.nilu.no ________________________________________ Fra: CF-metadata <[email protected]> på vegne av Maarten Sneep <[email protected]> Sendt: 4. juli 2016 18:45 Til: [email protected] Emne: Re: [CF-metadata] Support for compound data types On 07/04/2016 12:29 PM, Richard Olav Rud wrote: > Dear CF community, > > We are currently working on exporting our NASA Ames formated data to > cf-compliant > netCDF for our atmospheric composition database. > > Since our metadata changes over time, for most of our variables, we were > thinking > about using compound data types. > Most of our metadata is of little interest to our average user and therefore > we would > avoid listing them as variables. Even so, we are still obligated to present > this > information. > > So I am wondering if there any plans in the near future to implement compound > data > types for the CF metadata conventions? What level of metadata? per-variable or per-granule or even per-dataset? If the latter (2): take a look at the methods employed for Sentinel 5 precursor. http://www.tropomi.eu/documents/level-2-products http://www.tropomi.eu/science/level-2-products (the first link contains documentation, the second has (pre-launch) sample products). Best, Maarten Sneep -- KNMI T: 030 2206747 E: [email protected] R: A2.14 _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata P Please consider the environment before printing this email and attachments _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
