Steve, I agree with you 100% about flagging data with quality flags, and I wasn't suggesting that we throw anything away. I just was wondering whether the title "CF-compliant" might be withheld from dataseta until missing coordinates were supplied.
-Rich On Thu, Mar 29, 2012 at 1:50 PM, Steve Hankin <[email protected]> wrote: > > > On 3/29/2012 10:11 AM, Rich Signell wrote: > > Folks, > > I'm confused now. Are we proposing that we could have CF-compliant > files that have no valid coordinate data, with the justification that > somebody may figure the coordinates out later? > > > Hi Rich, > > I'm afraid that this is the tip of an iceberg as we begin to try to > acknowledge the differing viewpoints of observations versus models and > products. It would clearly be wrong to say to an observation program that > it cannot include temperature and salinity measurements in their CF files, > because the pressure sensor failed at that point. While the resulting > observations have greatly diminished value (and maybe ultimately prove to > have no value at all), it would be wrong to throw the measurements away. > This is in the same spirit that it is preferable to flag QC evaluations of > "bad data" rather than throw them away. > > Should CF approach this head-on? For example, CF could stipulate that IFF a > data provider wants to include valid observations at points where auxiliary > coordinates are missing, they must also include some kind of flag variable > as well. To me this seems unnecessary. It is sufficient simply to state > that under some circumstances there may be valid values at indices where > auxiliary coordinates contain missing values -- perhaps adding that the > circumstances will be rare and application-specific. > > Crystal ball: Meeting the sometimes-conflicting needs of observations and > models/products is going to lead to more of these discussions. > > - Steve > > > > -Rich > > On Thu, Mar 29, 2012 at 1:01 PM, Steve Hankin <[email protected]> > wrote: > > Returning to Nan's valid example, the proposed wording isn't very attuned to > the valid needs of (in situ) observations. If the pressure sensor fails, > while other sensors remain active, then the Z auxiliary coordinate becomes > unknown but other parameters remain valid. The observations have > potential value (though greatly degraded, of course), because a future > investigator may figure out how to estimate the Z position from other > information. For the investigator writing those applications, the > statements below are wrong or misleading. > > I think the right thing to say is something along the lines of > > "Application writers should be aware that under some (rare) circumstances > data auxiliary coordinate values may be missing, while other parameters at > the corresponding indices remain valid. While special purpose applications > may be able to glean useful information at these indices, most applications > will want to regard data as missing where the auxiliary coordinates are > missing " > > > On 3/29/2012 9:05 AM, Jim Biard wrote: > > All, > > For the work I am doing right now, I am required to not fill in missing > values in any variable. I encourage everyone to go with John Caron's idea. > > Grace and peace, > > Jim > > > On Thu, Mar 29, 2012 at 12:01 PM, John Caron <[email protected]> wrote: > > To answer this concern, I would agree to modify the statement > > "Applications are free to assume that data is missing where the auxiliary > coordinates are missing" > > to > > > "Applications should treat the data as missing where the auxiliary > coordinates are missing" > > My concern is that we shouldnt make a file "non CF compliant" just because > the data provider would like to store data values where there arent > coordinate values. But telling them that standard software _will_ ignore > them seems good. > > > > > On 3/29/2012 9:47 AM, Rich Signell wrote: > > Jonathan, > > +1 on your idea of only identifying variables as aux coordinate > variables once they have valid values at valid data locations. > > -Rich > > On Thu, Mar 29, 2012 at 11:32 AM, Jonathan Gregory > <[email protected]> wrote: > > Dear Jim > > We are discussing auxiliary coordinate variables. They do not have to be > 1D or monotonic. Those requirements apply to coordinate variables in the > Unidata sense. CF distinguishes these two concepts in Sect 1.2. > > The point is, the information in the variable *is* coordinate > information, > > I would say, if it's missing, it's not information. > > What if we say something along the lines of, "Applications should treat > the > data as missing where the auxiliary coordinates are missing when > plotting > data."? Would that resolve the problem? > > Plotting is not the only thing that an application might wish to use it > for. > If we said, more generally, "Applications should treat the data as > missing for > all purposes where the aux coord variables are missing", it would be > almost > the same as not allowing missing data in aux coord vars, since there > would be > no point in providing a data value if it was not permitted to use it. > > Although I am arguing one side, I could be convinced either way. But it > does > feel unsafe to me at present. > > Cheers > > Jonathan > _______________________________________________ > CF-metadata mailing list > [email protected] > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > > _______________________________________________ > CF-metadata mailing list > [email protected] > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > > > -- > Jim Biard > Research Scholar > Cooperative Institute for Climate and Satellites > Remote Sensing and Applications Division > National Climatic Data Center > 151 Patton Ave, Asheville, NC 28801-5001 > > [email protected] > 828-271-4900 > > > > _______________________________________________ > CF-metadata mailing list > [email protected] > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > > _______________________________________________ > CF-metadata mailing list > [email protected] > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > > > -- Dr. Richard P. Signell (508) 457-2229 USGS, 384 Woods Hole Rd. Woods Hole, MA 02543-1598 _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
