Dear All: I think Richard and Martin are on the right track in recommending a scheduling cadence, preferably where “time to market” has high priority, as it would provide certainty, and also allow operational program needs to be addressed in a manner that is satisfactorily timely.
Also worth noting, I have been active on message board for about 2 1/2 years and although I remember seeing some convention clarifications (e.g. dependent scalar auxiliary coordinate variables), I do not remember an outright retraction of any convention for which consensus was achieved. Have there been any ? very respectfully, randy On Mar 18, 2014, at 1:55 PM, John Graybeal <[email protected]> wrote: > Concur, Seth and Randy. When I was analyzing CF applicability for > Marinexplore, I wanted to choose the most advanced _finalized_ spec I could. > (Because we wouldn't want to publish data files that became invalid due to a > change.) Although 1.6 wasn't finalized, it had been around for so long and > was so vastly improved over 1.5 that I decided to go with it -- not realizing > the predecessors were never finalized either. > > My conclusion: To be consistent with common practice, a spec should stay beta > until it is finalized. > > I don't see the problem with a spec being released that incorporates a > mistake. ("I think there is a good argument that we should try hard to avoid > making a mistake which we have to reverse, because data lasts forever, and if > data were written with a flawed standard, it would forever be a nuisance.") > All the data ever written has loads of mistakes, including sloppy adherence > to specs, and no spec is perfect either. (Plus, an awful lot of data does not > last forever, sorry to say.) The point is, if the data conforms to its stated > specification, we know what to expect, and that is by far the greatest > accomplishment. Let's not let fear of a mistake slow down progress, > especially when our collective good judgment will prevent most mistakes > anyway. > > My conclusion: Yes try hard to identify issues during discussion of changes, > even do some test cases -- but don't arbitrarily extend the beta period just > to identify more issues. When you have a coherent update that you believe is > consistent, release it, and let's move forward from there. > > John > > On Mar 18, 2014, at 10:30, Seth McGinnis <[email protected]> wrote: > >> Hi all, >> >> I have to agree with Randy. When you're producing data, you generally >> don't have time to wait for the standard to stabilize, you just do your >> best to find whatever elements of CF are a decent match to the problem >> at hand and use them, regardless of whether or not they might be marked >> 'provisional'. (Or even whether they've moved from being accepted in >> online discussion to being written down in an official version of the >> spec document...) >> >> And, to be honest, data that's invalid because of sloppy adherence to >> the spec is far, far more of a nuisance than data that's been >> invalidated by the spec moving out from under it will ever be. If >> you've got a system for dealing with the former, it can handle the >> latter, too. >> >> Cheers, >> >> --Seth >> >> >> On 3/18/14 7:36 AM, [email protected] wrote: >>> Dear Jonathan et al: >>> >>> a data point .... >>> >>> As an engineer working an operational data production system with >>> schedules and deadlines, and the requirement to use NetCDF file format >>> we are leveraging CF conventions wherever consensus has been reached. >>> >>> The benefit of using these conventions (regardless of their provisional >>> status) outweighs any risk of their retraction. >>> >>> I am curious what other folks working operational data production >>> systems are doing in this regard. >>> >>> very respectfully, >>> >>> randy >>> >>> >>> >>> ------------------------------------------------------------------------ >>> *From*: "Jonathan Gregory" <[email protected]> >>> *Sent*: Tuesday, March 18, 2014 7:09 AM >>> *To*: [email protected] >>> *Subject*: Re: [CF-metadata] Editing/publishing workflow >>> >>> Dear Richard >>> >>> That's right. No change since 1.0 has so far passed beyond being >>> "provisional" >>> since we didn't definitely agree how to do that. I am not strongly in >>> favour of >>> provisional status myself, but others have argued strongly for it >>> previously. >>> I think there is a good argument that we should try hard to avoid making a >>> mistake which we have to reverse, because data lasts forever, and if >>> data were >>> written with a flawed standard, it would forever be a nuisance. Of course in >>> principle this can be detected and perhaps worked round using the >>> version from >>> the Conventions attribute, but in practice I suspect this attribute is not >>> normally written scrupulously correctly, nor inspected by analysis software. >>> So we should be careful, and that means a "cooling-off" period during which >>> data is at risk of being invalidated if it uses a provisional convention >>> is a >>> reasonable safeguard, but it should not be a long period - months, not >>> years, >>> I would say. >>> >>> Best wishes >>> >>> Jonathan >>> >>> ----- Forwarded message from "Hattersley, Richard" >>> <[email protected]> ----- >>> >>>> Date: Tue, 18 Mar 2014 09:05:36 +0000 >>>> From: "Hattersley, Richard" <[email protected]> >>>> To: "Gregory, Jonathan" <[email protected]>, >>>> "[email protected]" <[email protected]> >>>> Subject: RE: [CF-metadata] Editing/publishing workflow >>>> >>>>> I'd like to propose changing the rules. That's something the conventions >>>>> committee can agree, I believe. I would suggest the simplest possibility, >>>>> if we wish to retain provisional status, is to specify a time. We could >>>>> say that, after one year from acceptance or when the next version of the >>>>> conventions document is published, whichever is later, a change becomes >>>>> permanent. What do you think? >>>> >>>> The more I consider the concept of "provisional status" the more it >>>> concerns me. What does it actually mean for a netCDF file to use a >>>> particular "Conventions" attribute value? How can one tell what is still >>>> in provisional status? What version should data writers be using? I've >>>> checked back through 1.3, 1.4, 1.5, and 1.6 and they all still contain >>>> sections marked as provisional. Using the analogy to software versions >>>> that has been raised elsewhere, the CF convention versions are essentially >>>> pre-release versions, e.g. 1.6-beta, and are not suitable for production >>>> use. >>>> >>>> I would argue that the simplest possibility is to drop the "provisional >>>> status" concept. Identifying and resolving problems should happen during >>>> the discussion of the modification and its subsequent application to the >>>> conventions document. >>>> >>>> If a further flaw, ambiguity, etc. is subsequently discovered prior to the >>>> publication of the next version of the conventions then it can easily be >>>> resolved at that time. >>>> >>>> If a problem is discovered after the publication of the next version then >>>> a correction must be applied and published in a *new* version. That >>>> version could be a "bug fix" version (e.g. 1.6.1) or it could just wait >>>> for the next normal release, e.g. 1.7. It would help to agree that process >>>> in advance but I have no strong opinion either way. >>>> >>>> >>>> Richard Hattersley >>>> Expert Software Developer >>>> Met Office FitzRoy Road Exeter Devon EX1 3PB United Kingdom >>>> Tel: +44 (0)1392 885702 >>>> Email: [email protected] Web: www.metoffice.gov.uk >>>> >>>> -----Original Message----- >>>> From: CF-metadata [mailto:[email protected]] On Behalf Of >>>> Jonathan Gregory >>>> Sent: 13 March 2014 17:24 >>>> To: [email protected] >>>> Subject: Re: [CF-metadata] Editing/publishing workflow >>>> >>>> Dear Jeff >>>> >>>>> Present CF Conventions policies require that all changes be >>>>> provisional, and marked as such in the document, until determined to >>>>> be permanent at a later time (this determination has never been made). >>>>> That's the meaning of all the pink and yellow highlighting in the >>>>> document at cf-pcmdi.llnl.gov. >>>> >>>> Yes, this is a issue. As Richard said, it doesn't matter how it is marked. >>>> The problem is that all changes, however old, are still marked as >>>> provisional, as you said. This is (a) a bit silly and (b) a nuisance as >>>> regards legibility of the doc. The aim of provisional status was to allow >>>> time for people to try out the change, in case a logical flaw was >>>> discovered which hadn't been fore- seen at the time of the proposal. This >>>> was because of the concern that many or most proposals concern data which >>>> has not yet been written, so the metadata being proposed can't have been >>>> thoroughly tested. It was supposed that some tests, using specified >>>> software, would be used to demonstrate the new feature was "working", but >>>> no-one had time to work out the details for this. >>>> >>>> I'd like to propose changing the rules. That's something the conventions >>>> committee can agree, I believe. I would suggest the simplest possibility, >>>> if we wish to retain provisional status, is to specify a time. We could >>>> say that, after one year from acceptance or when the next version of the >>>> conventions document is published, whichever is later, a change becomes >>>> permanent. What do you think? >>>> >>>> Cheers >>>> >>>> Jonathan >>>> _______________________________________________ >>>> CF-metadata mailing list >>>> [email protected] >>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>> >>> ----- End forwarded message ----- >>> _______________________________________________ >>> CF-metadata mailing list >>> [email protected] >>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>> >>> >>> >>> _______________________________________________ >>> CF-metadata mailing list >>> [email protected] >>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >>> >> _______________________________________________ >> CF-metadata mailing list >> [email protected] >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > John Graybeal > [email protected] > > > > _______________________________________________ > CF-metadata mailing list > [email protected] > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ____________________________________ Randy C. Horne ([email protected]) Principal Engineer, Excalibur Laboratories Inc. voice & fax: (321) 952.5100 url: http://www.excaliburlabs.com _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
