Barna, Yes an update to the CF document should follow after the new standard_name is implemented. I think multiple examples are needed since status_flag covers many different types of state variables.
Ken On 2019-7-22 10:35, Andrew Barna wrote: > Hi Martin, Ken, > > Is there anything wrong with including multiple "status_flag" > variables to capture all separate state you wish? The CF document > unfortunately only includes an example of how to encode the status of > a sensor, but the actual meanings of the flag values are entirely up > to you, and this will not change with this proposal. Perhaps the CF > document would benefit from additional examples (e.g. one that only > shows data quality flags). > > -Barna > > > On Mon, Jul 22, 2019 at 9:04 AM Kehoe, Kenneth E. <kke...@ou.edu> wrote: >> Hi Martin, >> >> I see status encompassing multiple metadata pieces of information. For >> example it could be a state of the instrument as it cycles through a >> pre-programed routine (Look at calibration target, look at sky, look at >> ground, look at second calibration target, repeat...). Or the sources of >> the inputs for a model where the availability or some other reason could >> require making a decision on what source(s) to use. For provenance this >> source information is important to report on a time step basis. Or the >> status could be a data providers method to provide uncertainty >> information (I see this as incorrect but some people do see it this >> way). Each of these are important metadata but the method of use is >> different than a strictly quality variable. A quality variable provides >> information indicating if the data should be used or possibly could be >> used in a weighted mean method to favor high quality data over low >> quality data. The way the metadata is used is different depending on the >> metadata type. A state of the instrument would be used for sub-setting >> calibration vs. data. There is no ambiguity in this as data from a >> calibration target is not used in a weather research analysis. But >> quality is more subjective and is decided by the data user. If the >> quality variable has 20 different quality tests the user would need to >> decided if all 20 test results should be used or only a subset. Also, >> the code for applying the quality is different than the state of the >> instrument view (in my example above). >> >> It is possible to have a quality test result from the state of the >> instrument, but not the other way around (typically). So I need a way to >> distinguish the two for automated or semi-automated tools. Hence my >> point of quality_flag essentially being a subset of status_flag >> >> Ken >> >> >> >> On 2019-7-22 02:57, Martin Juckes - UKRI STFC wrote: >>> Dear Ken, >>> >>> >>> Can you expand on the distinction between "quality" and "status"? I >>> understand that they are different in principle, but, in order to support >>> this new standard name I think we need a clear objective statement of how >>> we would want to distinguish between them in CF. >>> >>> The conventions section on flags (3.5) mixes the two up >>> (http://cfconventions.org/cf-conventions/cf-conventions.html#flags ), so >>> some re-wording of the document would also be needed, >>> >>> regards, >>> Martin >>> >>> ________________________________ >>> From: CF-metadata <cf-metadata-boun...@cgd.ucar.edu> on behalf of Kehoe, >>> Kenneth E. <kke...@ou.edu> >>> Sent: 19 July 2019 06:42 >>> To: cf-metadata@cgd.ucar.edu >>> Subject: [CF-metadata] New standard_name of quality_flag for corresponding >>> quality control variables >>> >>> Dear CF, >>> >>> I am proposing a new standard name of "quality_flag" to indicate a variable >>> is purely a quality control variable. A quality control variable would use >>> flag_values or flag_masks along with flag_meanings to allow declaring >>> levels of quality or results from quality indicating tests of the data >>> variable. This variable be a subset of the more general "status_flag" >>> standard name. Currently the definition of "status_flag" is: >>> >>> - A variable with the standard name of status_flag contains an indication >>> of quality or other status of another data variable. The linkage between >>> the data variable and the variable with the standard_name of status_flag is >>> achieved using the ancillary_variables attribute. >>> >>> This definition includes a variable used to define the state or other >>> status information of a variable and can not be distinguished by standard >>> name alone from a state of the instrument, processing decision, source >>> information, needed metadata about the data variable or other ancillary >>> variable type. Since there is no other way to define a purely quality >>> control variable, the use of "status_flag" is too general for strictly >>> quality control variables. By having a method to define a variable as >>> strictly quality control the results of quality control tests can be >>> applied to the data with a software tool based on requests by the user. >>> This would not affect current datasets that do use "status_flag" nor >>> require a change to the definition outside of the indication that >>> "quality_flag" standard name is available and a better use for pure quality >>> control variables. >>> >>> Proposed addition: >>> >>> quality_flag = A variable with the standard name of quality_flag contains >>> an indication of quality information of another data variable. The linkage >>> between the data variable and the variable or variables with the >>> standard_name of quality_flag is achieved using the ancillary_variables >>> attribute. >>> >>> Proposed change: >>> >>> status_flag = A variable with the standard name of status_flag contains an >>> indication of status of another data variable. The linkage between the data >>> variable and the variable with the standard_name of status_flag is achieved >>> using the ancillary_variables attribute. For data quality information use >>> quality_flag. >>> >>> Thanks, >>> >>> Ken >>> >>> >>> >>> -- >>> Kenneth E. Kehoe >>> Research Associate - University of Oklahoma >>> Cooperative Institute for Mesoscale Meteorological Studies >>> ARM Climate Research Facility - Data Quality Office >>> e-mail: kke...@ou.edu<mailto:kke...@ou.edu> | Office: 303-497-4754 >> -- >> Kenneth E. Kehoe >> Research Associate - University of Oklahoma >> Cooperative Institute for Mesoscale Meteorological Studies >> ARM Climate Research Facility - Data Quality Office >> e-mail: kke...@ou.edu | Office: 303-497-4754 >> >> _______________________________________________ >> CF-metadata mailing list >> CF-metadata@cgd.ucar.edu >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- Kenneth E. Kehoe Research Associate - University of Oklahoma Cooperative Institute for Mesoscale Meteorological Studies ARM Climate Research Facility - Data Quality Office e-mail: kke...@ou.edu | Office: 303-497-4754 _______________________________________________ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
