Hi Phil, > The final para of section 2.5.1 of the CF conventions document describes > the use of the _FillValue (or missing_value) attribute in the case of > data packed using the scale-and-offset method. What is not clear - at > least to me - is what the preferred application behaviour should be in > the case where the data is unpacked and then written out to a new netCDF > file. In particular, what fill value should be used for the unpacked > data variable? > > I presume that one wouldn't normally want to use the original fill value > since that value (typically an 8- or 16-bit integer) is quite likely to > fall within the normal range of the unpacked data (typically a 32- or > 64-bit float). > > In the absence of explicitly setting a fill value attribute on the > unpacked data variable I assume that the netCDF default fill value will > be used for the data type in question. Which may not always be desirable > (certainly not for 32-bit floats, where the default fill value can give > rise to subtle precision-related problems). > > With this in mind, I was wondering if there is any merit in defining a > new attribute called, say, _UnpackedFillValue (or > unpacked_missing_value)? If client software detected this attribute then > the associated value (same data type as the scale_factor and add_offset > attributes) would be used as the fill value for the unpacked data > variable. > > Alternatively, the names _FillValueUnpacked (missing_value_unpacked) > might be preferable since they would then appear together pair-wise in > CDL-type listings, e.g. > > short pkd_var(z, y, x) : > ... > pkd_var:_FillValue =3D -32768 ; > pkd_var:_FillValueUnpacked =3D -1.0e30 ; > pkd_var:add_offset =3D 42.0 ; > pkd_var:scale_factor =3D 1234.0 ; > ... > > > Any merit/mileage in this idea?
A more detailed recommendation for treating special values such as _FillValue or missing_value for packed data is described, with associated formulas, in the "Packed Data Values" section of a best-practices document that we wrote a few years ago: http://www.unidata.ucar.edu/netcdf/docs/BestPractices.html#Packed%20Data%20Values It provides a recommendation for ensuring the unpacked special value is not in the range of other unpacked data values. If that recommendation is followed, I think there is no need for an additional _FillValueUnpacked (or missing_value_unpacked) attribute. If you agree that is an acceptable approach, perhaps we should add it to CF ... --Russ _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
