Hi David, Thanks for pointing me to the Trac ticket. I knew I had read this before somewhere, but now when the issue resurfaced in a very concrete sense I could not find it when searching the email archive, and I never thought of the Trac.
First of all, I do agree that the convention should not be linked (restrained) to a specific file format or the concept of a file. And I do not want to retrace the discussion from 6 years ago. However, given the outcome back then I have two suggestions: 1. I would wish that the convention would express more clearly what ".... has precedence" means, i.e. that it implies that the information held in the global attribute may not persist if duplicated by the same variable attribute. Indeed, your own inlined comment "...(whatever that means!)" suggests that something more is needed. The same goes for the language in the specification of the data model, i.e. I do not think that the word "superseded" is clear enough (but that might be due to limitations of my English language skills). Making this clear in both text I think is essential when incorporating the CF Data Model into the CF Conventions (<https://github.com/cf-convention/cf-conventions/issues/159>). 2. The initial post on this thread was prompted by that I noted that in many CMIP5 files CMOR had written different information (and relevant) in the variable history and the global history. And the global variable was actually not transferred through some processing steps. As CMIP5, and CMIP6, aims at publishing CF compliant data, I suggest that CMOR and/or its usage documentation is updated to prevent, or at least discourage, duplication of attributes. Kind regards, Lars ________________________________ Från: David Hassell [david.hass...@ncas.ac.uk] Skickat: den 27 mars 2019 15:22 Till: Bärring Lars Kopia: cf-metadata@cgd.ucar.edu Ämne: Re: [CF-metadata] same attribute name in variable and in global Hi Lars, Thanks for bringing this up. Here is the background, as I understand it, to the data model description of global attributes. The conventions do not state rules for combining attributes that appear both globally and on variables; rightly so, because there is no generically correct way to do it, even assuming that they are consistent. Which order to you concatenate strings? What if the attributes are numbers? Bearing this in mind, and noting that i) variable attributes take precedence (whatever that means!) and ii) the data model has no concept of a file (this is deliberate, as one of the aims of the data model is to transcend the file format), leads to the data model saying that "netCDF global file attributes apply to every data variable in the file, except where they are superseded by netCDF data variable attributes with the same name." There was some of interesting discussion on this, around 6 years ago, in Trac ticket #95 (starting at https://cf-trac.llnl.gov/trac/ticket/95#comment:59). The current discussion, and the one in the Trac ticket, highlights that there are multiple interpretations of the conventions in common use, and that not all of these interpretations are suitable for all users. Even though the data model interpretation has to be as described, this does not preclude a software implementation of the data model from adding extra functionality to record the provenance of attributes, or even combing global/variable attributes in a particular manner. Software does this sort of thing all the time, for example in remembering netCDF variable names, which are also not part of the data model. For example, the cf-python library will note which field properties were global attributes in a file that it has read, so when you write the field back to disk it can create the same global attributes (assuming there that no inconsistencies have been introduced after the read ...). I hope this is useful, All the best, David On Fri, 15 Mar 2019 at 12:48, Bärring Lars <lars.barr...@smhi.se<mailto:lars.barr...@smhi.se>> wrote: Dear all, I have come across many CMIP5 files that have the same attribute [name] attached to the data variable as found in the globals. In particular it seems that CMOR was writing variable processing history attached to the variable, and more general file processing in the global history. I have also seen that "comment" sometimes occur both as global and as variable attribute. If such a duplication occurs, CF writes: "When an attribute appears both globally and as a variable attribute, the variable’s version has precedence." My interpretation of this is that if there are contradictory information then the variable's attribute has precedence, but otherwise it does not invalidate or overshadows what is in the global attribute. However, the data model presented in the GMD paper (https://www.geosci-model-dev.net/10/4619/2017/ page 4629, [page 11], under "4.1 The field construct") presents a more restrictive interpretation: "In the data model, we consider that netCDF global file attributes apply to every data variable in the file, except where they are superseded by netCDF data variable attributes with the same name." I understand this to mean that if the same attributes (e.g. comment or history) is present both as global and variable attribute, then the global attribute is overshadowed (as if it did not exist at all)? Is this correct? If so, is this the best solution? Would not concatenating the information contained in each (putting the variable text first) be a more appropriate way to resolve the situation (and thus specify in the data model)? Kind regards, Lars -- Lars Bärring FDr, Forskare PhD, Research Scientist SMHI / Swedish Meteorological and Hydrological Institute Rossby Centre SE - 601 76 NORRKÖPING http://www.smhi.se<http://www.smhi.se/> E-post / Email: lars.barr...@smhi.se<mailto:lars.barr...@smhi.se> Tel / Phone: +46 (0)11 495 8604 Fax: +46 (0)11 495 8001 Besöksadress / Visiting address: Folkborgsvägen 17 _______________________________________________ CF-metadata mailing list CF-metadata@cgd.ucar.edu<mailto:CF-metadata@cgd.ucar.edu> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- David Hassell National Centre for Atmospheric Science Department of Meteorology, University of Reading, Earley Gate, PO Box 243, Reading RG6 6BB Tel: +44 118 3785183 http://www.met.reading.ac.uk/
_______________________________________________ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
