I can appreciate Ken's point of view, particularly where the convenience is like the example Jonathan posted of having one scalar coordinate, there's not much added by encoding this as a scalar.
However I support Jonathan's statement about omitting size one dimensions, particularly where there are a number. It is often helpful not to encode information about the ordering of dimensions of size one. In addition to this, I am keen to make sure that data creators are able to encode metadata where the dependency relationship between multiple scalars is not encoded. In numerous cases, the relationship is not uniquely defined and should not be encoded. For example a data creator may have a set of descriptors and identifiers which define how a particular data set is defined with respect to a larger study, such as a multi-model analysis meta-experiment. Each data set is produced by a model with a collection of scalar coordinates from that model run, e.g. 'ensemble member number', 'experiment id', 'perturbation scheme', 'forcing parameter a', forcing parameter b', ... etc. Given different collections of such data sets, different relationships may emerge from the collection, enabling different types of analysis. It is a really useful approach to encode all these quantities as scalars, and interpret these scalars as potential degrees of freedom with potential inter-relations; the degrees of freedom and inter-relationships are then emergent properties of the collection, not defined in any individual member data set. This seems to me to be the logical conclusion of the use of the term 'scalar coordinate' (as contrasted to 'vector coordinate', i assume) and it is really useful. I think it would be a regressive step for CF to limit or complicate this facet of data comprehension by constraining the meaning of 'scalar coordinate' in the way that has been suggested. I don't think such a constraint has been clear up to now in the conventions and people are making real use of the perceived flexibility. mark ________________________________________ From: CF-metadata [[email protected]] on behalf of Jonathan Gregory [[email protected]] Sent: 21 May 2013 18:42 To: [email protected] Subject: Re: [CF-metadata] scalar coordinates Dear Ken Your argument is one that we should not have scalar coordinates at all. You could propose that they should be removed from the next version of CF (and that would certainly simplify this discussion, if agreed :-). They have been in there for several years now, and I guess they are quite widely used, because it is convenient for data-writers. I agree, it does require a bit more work for data-readers. However, CF-compliant software should expect to inspect the coordinates attribute in any case, and if it does that it will automatically come across the scalar coordinate variables. I think that an attractive feature of omitting the size-one dimensions is not having to decide on the order of them in the data variable, which is really arbitrary for storage in netCDF files, since it makes no difference to the order of the data elements. Best wishes Jonathan ----- Forwarded message from "Kenneth S. Casey - NOAA Federal" <[email protected]> ----- > From: "Kenneth S. Casey - NOAA Federal" <[email protected]> > Date: Tue, 21 May 2013 07:22:34 -0400 > To: "Hedley, Mark" <[email protected]> > X-Mailer: Apple Mail (2.1503) > CC: "[email protected]" <[email protected]> > Subject: Re: [CF-metadata] scalar coordinates > > Hi Everyone, > > After spending 15 minutes reading this and Jonathan's previous post, and > trying hard to be sure I really understand them, I am left wondering if a > "convenience feature" is really a convenience at all. I know programmers > don't like clutter in their code and have certain aesthetic to uphold (I used > to do much more serious programming in my younger days, but am now just a > lazy Matlaber), but is the added complexity of having these two options, > copied from Jonathan's post: > > float height; // scalar coordinate variable > height: standard_name="height"; > float temp(lat,lon); > temp: standard_name="air_temperature"; > temp: coordinates="height"; > > float height(height); // size-one coordinate var, with dimension height=1 > height: standard_name="height"; > float temp(height,lat,lon); > temp: standard_name="air_temperature"; > > really worth it? The manual must be longer to describe the "convenience", > and the application programmers and downstream users of the data now have to > build complexity into THEIR code to handle both cases. This list has to take > time explaining and debating the options, and I have to take time explaining > it to both data producers and data consumers who have never used netCDF or CF > before. Maybe that is not such a big deal on a case by case basis, but in > the broader picture when folks like me are trying to get everyone and their > brothers and sisters and mothers and fathers using CF-netCDF, it is a real > pain. My vote is to simplify. Use the second example, since it handles the > case where height=1 as easily as the case where height=1000. I say be > explicit. Does the first example gain us any functionality? If not, then > don't use it and stay away from it. Encourage others to do the same. > > Ken > > > On May 21, 2013, at 5:48 AM, "Hedley, Mark" <[email protected]> > wrote: > > > The term 'convenience feature' is mentioned in the conventions document: > > > > 'The new scalar coordinate variable is a convenience feature which avoids > > adding size one dimensions to variables.' > > > > Data creators have seen the benefits in not encoding size one dimensions > > and made use of this feature, it has proved very convenient. The > > conventions go on to say: > > > > 'Scalar coordinate variables have the same information content and can be > > used in the same contexts as a size one coordinate variable.' > > > > But this statement is not quite true: the ordering of dimensions is not > > encoded, and the ability to link many coordinates to the same dimension is > > lost. The assumption in this statement is an aspiration which I think > > cannot be delivered without particularly strict limitations on the use of > > scalars during encoding. > > > > Nowhere in the conventions does it state that if more than one > > single-valued coordinate is related to the same degree of freedom, a > > dimension must be declared for these and this relationship explicitly > > encoded. > > > > Later, the case of character strings is addressed: > > > > 'If a character variable has only one dimension (the maximum length of the > > string), it is regarded as a string-valued scalar coordinate variable, > > analogous to a numeric scalar coordinate variable (see Section 5.7, ?Scalar > > Coordinate Variables?) ' > > > > which is a required feature, but the NUG only allows numerical valued data > > arrays as Coordinate Variables, so a further section is added, in the > > Terminology: > > > > 'scalar coordinate variable > > A scalar variable that contains coordinate data. Functionally equivalent > > to either a size one coordinate variable or a size one auxiliary coordinate > > variable. ' > > > > These statements together provide information on how to write files, but > > they are limited in their assistance to file reading and interpretation. > > > > The conventions are not clear how to, or whether to make a distinction for > > a particular scalar coordinate: it does not say that a scalar coordinate is > > a Coordinate Variable or an Auxiliary Coordinate Variable, it says it is > > functionally equivalent to either one or the other. > > > > I have read these sections to mean that by encoding a scalar coordinate the > > data creator is not providing information about how the coordinate is > > related to the dimensions in the file, other than to say it applies to all > > of the cells currently in the file. > > > > As such, I disagree with the statement that that > > > > 'Scalar coordinate variables have the same information content and can be > > used in the same contexts as a size one coordinate variable.' > > > > In many cases this will turn out to be a valid interpretation but it is not > > the only one, and this nuance is a really useful feature, which many data > > creators have benefited from. > > > > From one point of view, a third type of Coordinate exists in CF, the Scalar > > Coordinate, which is neither a Coordinate Variable, nor an Auxiliary > > Coordinate. From another point of view a Scalar Coordinate is an Auxiliary > > Coordinates which has the potential to be an emergent Coordinate Variable, > > if required and consistent for the data consumer. (I am sure there are > > other useful perspectives we can consider) > > > > We have come across many data sets from other data creators where a > > considered reading of the data suggests that they have taken an > > interpretation such as this as well. No distinction has been made between > > scalars which represent a degree of freedom and scalars which do not. > > > > The scalar coordinate is a convenient feature allowing metadata to be > > simply encoded in a clear manner and I feel that the conventions document > > should adapt to reflect the usage some sections of the community have > > adopted. It is not ambiguous, it provides sufficient information to work > > with the file and the data and metadata are well specified. > > > > Indeed when converting from other formats (such as GRIB and BUFR) to CF it > > is the logical way to encode the available metadata. > > > > I am concerned about the implications for these data sets if the > > interpretation of scalar coordinates is tightened in a future version of > > the conventions document to explicitly disallow this useful and well used > > point of view. I would like to stress again Jonathan's point, that all of > > this data is CF compliant, the question is how consumers interpret the > > semantics of the data set. > > > > I think the utility of the scalar coordinate variable is significantly > > diminished if Option B or some derivative of it is pursued for the next > > version of the conventions. Option A preserves all of the interpretations > > of Option B intact, but with caution needed on loading and interpretation > > not to read too much information into any scalar coordinates present. > > > > _______________________________________________ > > CF-metadata mailing list > > [email protected] > > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > Kenneth S. Casey, Ph.D. > Technical Director > NOAA National Oceanographic Data Center > 1315 East-West Highway > Silver Spring MD 20910 > 301-713-3272 x133 > http://www.nodc.noaa.gov > > > > _______________________________________________ > CF-metadata mailing list > [email protected] > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ----- End forwarded message ----- _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
