Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-03-11 Thread Micah Wengren
@feggleton @japamment @ngalbraith @roy-lowry and others: thanks for publishing the names, contributing to the discussion, and helping steer this through to acceptance! -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-03-11 Thread japamment
Closed #216. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/216#event-3118570282 This list forwards relevant notifications from Github. It is distinct from

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-03-11 Thread japamment
@feggleton thanks for publishing the names. I'm closing this issue now. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/216#issuecomment-597603118 This list forwards

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-03-11 Thread Francesca
These changes have been published in version 72 of the standard name table. Please close this issue if all discussions are complete. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-02-13 Thread japamment
@jessicaaustin @mwengren @ngalbraith @roy-lowry many thanks for all your comments and responses to my questions. This discussion has helped me to understand much better the ways in which quality information is gathered and used. Regarding the 'non aggregate' names, I think we are all agreed

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-02-05 Thread Micah Wengren
@ngalbraith I think we added the 'both automated and manual' clause to account for the case in Jessica's example in the above [comment](https://github.com/cf-convention/cf-conventions/issues/216#issuecomment-581714109), where there's a test with the generic `quality_flag` name that is manually

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-02-05 Thread Jessica Austin
"e.g. somebody who just wants to know which of a raft of flags to use to filter out problem data" -- Yes, this is our primary use case. I think we all agree that in some cases the user wants to know exactly how the aggregate QC flag was derived. But we may not reach an agreement today on

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-02-05 Thread Roy Lowry
I disagree that not describing the aggregate components through the Standard Name makes it useless. I can think of use cases where such minimal semantics are all that is needed (e.g. somebody who just wants to know which of a raft of flags to use to filter out problem data). There are of

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-02-04 Thread Nan Galbraith
This is all an improvement. I like Roy's change to the definition of the aggregate flag, 'an algorithmic combination of the results of all relevant quality tests', in place of 'set to the highest-level (worst case) flag found'. I DO have an issue with the term 'related ancillary parent data

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-02-03 Thread Jessica Austin
Thank you Nan and Alison for your comments and suggestions. I think we're all still in agreement on the standard name list, including Alison's suggestions for additional text to add to the non-aggregate flag names. The wording and scope for the aggregate flag still needs more clarification

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-01-31 Thread Nan Galbraith
Thank you Alison, this is all good. I DO have a couple of issues with this paragraph: 'This flag is a summary of all quality tests **run for another data variable**, which have standard names of the form X_quality_flag, and is **set to the highest-level (worst case) flag found**. Information

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-01-31 Thread japamment
@jessicaaustin @mwengren many thanks for these standard name proposals and my apologies for the delay in responding. Thank you also to all those who have contributed to this interesting discussion. It seems the discussion has reached consensus on adding the terms as standard names rather than

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-01-17 Thread Micah Wengren
@ngalbraith regarding the `references` attribute, I went back and checked [Appendix A](http://cfconventions.org/cf-conventions/cf-conventions.html#attribute-appendix) and was relieved to see it's valid as both a global and variable attribute. I don't know if there might still be software

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-01-08 Thread Nan Galbraith
The external files are fine, but there's one problem with using the 'references' attribute; that term is defined as a global attribute in CF. Most netCDF software does not handle a situation where a term is used as a global and a variable attribute the way you might expect - that's apparently

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-01-08 Thread Micah Wengren
@ngalbraith for the upcoming version of our 'IOOS Metadata Profile' that incorporates these new standard names into a quality flagging scheme for QARTOD, we decided to leverage the `references` attribute to suggest data providers link to external web pages or web-accessible files (e.g. JSON)

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-01-07 Thread Nan Galbraith
I personally think including this level of detail (number of standard deviations, upper/lower limits, gap length, name of climatology used, distance to 'neighbor' data, etc) is beyond the scope of CF; there are just too many details. Every test has its own inputs, and these may vary with

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2020-01-06 Thread Guilherme Castelão
@jessicaaustin and everyone else involved in this proposal and discussions, thank you very much for your time on this. It will be a great advance. I'm excited to see this concluded and start using it. I think it could be useful to include gradient_quality_flag and

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2019-12-17 Thread Nan Galbraith
I think this is fine. It seems clear, and limits the number of new standard names that will be needed. Of course, if people use these without specifying the vocabulary and/or some reference to a description of the tests, they lose a lot of their meaning. My aggregate_quality_flag might

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2019-12-12 Thread Jessica Austin
Catching up here. Sounds like one change we've decided to make is to remove `status_flag` from the recommended usage, so that ``` sea_water_practical_salinity_qc_flat_line_test:standard_name = "flat_line_test_quality_flag status_flag"; becomes ```

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2019-12-11 Thread Roy Lowry
@graybeal My preference is founded by English grammar in which a qualifying adjective precedes the noun reinforced by unfortunate experiences in the past whilst building systems based on label sorting. Only we metadata geeks think in terms of semantic hierarchies! However, my feelings aren't

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2019-12-10 Thread John Graybeal
I support Roy's other points, but whatever experiences showed Roy the error of his ways regarding name sorting never crossed my path. ;-) I find the wordings beginning with quality_flag considerably more intuitive, because the most important thing is that this is a quality flag, and the second

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2019-12-10 Thread Micah Wengren
@roy-lowry Ah, poor quality control on the quality control examples! I fixed the error in my [example](https://github.com/cf-convention/cf-conventions/issues/216#issuecomment-563339265) above. > Using alphanumeric sorting of labels to establish semantic relationships is a > technique I used

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2019-12-09 Thread Roy Lowry
@mwengren I was referring to removing the Modifier 'status_flag' from the examples in the modified proposal, not from the Conventions Document. To do that would be very much against my understanding of best practice. Deprecation, not deletion is the way to go to deliver transparency. Your

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2019-12-09 Thread Micah Wengren
@roy-lowry I was going to comment about the need to remove the deprecated `status_flag` standard name modifier from the conventions document, but I see you agree that's necessary too. That will help eliminate some confusion for newcomers to CF's ancillary variable/flag syntax and how to

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2019-12-09 Thread Roy Lowry
Many thanks to @lbdreyer for removing my concerns. It would certainly be easier to go down the Standard Names route as it is adding to a controlled vocabulary rather than updating the Conventions Document. Semantic information in Standard Names is also more accessible as they are in vocabulary

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2019-12-05 Thread Roy Lowry
@jessicaaustin I have been hanging fire to see if new watchers coming onto GitHub brings any additional comment. It's the next job on my CF work list. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [CF-metadata] [cf-convention/cf-conventions] Proposal: Add QARTOD quality flag names to standard name list (#216)

2019-11-26 Thread Jessica Austin
We discussed this internally and came up with the following: (1) We agree that removing the QARTOD from the test names is fine, since the result is still perfectly useful for our cases, and in fact makes the names more generic and widely useful. (2) It's clear from this thread that we are in