Chris,
While I agree that the elapsed time *should* be correct, I think it's
highly likely that there are files "in the wild" that don't have correct
elapsed times. I know that until I started digging into this I would
have naively used the *nix timegm function to produce elapsed times from
UTC timestamps.
What I'm advocating is a way to allow the large majority of data
producers who have time resolutions that are coarse enough that leap
seconds don't matter to continue to work without having to do anything
extra, while providing a positive mechanism for those who have fine time
resolutions to signal to their users that they can trust the elapsed
time values. I'd say that what is special about this one is that it
involves a coordinate variable, as opposed to a data variable, and that
the number and kinds of pitfalls in moving between timestamps and
elapsed times warrant the extra attention.
The more I think about 'gregorian_nls' though, the more I think it is
not useful. If you have GPS time values (perhaps as elapsed times since
the GPS epoch of 1980-01-06 00:00:00 UTC) as your input, and you wanted
to use a GPS timestamp in your reference time (which for today is
currently 16 seconds off from UTC), you would need to have a way to
state that the timestamp is in the GPS time system. Similarly, if you
have TAI time values (elapsed times since the TAI epoch of 1958-01-01
00:00:00 UTC) and wanted to use a TAI timestamp in your reference time
(which for today is currently 35 seconds off from UTC), you would need
to have a way to state that the timestamp is in the TAI time system. The
gregorian_nls calendar is too generic. And then there's the Loran time
system (and others).
As an example, the date and time stamps as I'm writing this in the
different systems are:
* TAI 2015-05-20 20:21:00
* GPS 2015-05-20 20:21:16
* UTC 2015-05-20 20:21:35
How about this?
Redefine 'gregorian' to remove reference to UTC as Jonathan has
described, and state that presence or absence of leap second artifacts
in the reference timestamp and elapsed time values is not known for a
time variable that names this calendar. This is backward compatible.
Define 'gregorian_utc' as using the Gregorian calendar and the UTC time
system in the reference date and time in the units attribute. Specify
that the elapsed times *must* be free of leap second offsets or
discontinuities. Data producers that have GPS times as inputs (for
example) must be sure to obtain a proper UTC timestamp, but are
otherwise unconstrained. Data producers that have UTC timestamps as
inputs must be sure to properly handle conversion from UTC timestamps to
elapsed times so that leap second problems are not introduced. Data
consumers, if they read the section on time in the CF Conventions, will
understand how to make proper use of the time variable reference
timestamp and elapsed time values.
And we could also define 'gregorian_gps', 'gregorian_tai', etc. Or we
could define just one of them (it doesn't have to be gregorian_utc) and
say that this is what you use with CF. For folks with input time data
that is free of leap seconds (either time stamps or elapsed times), the
most they will have to do is convert the reference time to the chosen
time system. It's going to be more work for people with UTC timestamps
as inputs no matter what.
Grace and peace,
Jim
On 5/20/15 11:44 AM, Chris Barker wrote:
On Wed, May 20, 2015 at 7:43 AM, John Graybeal
<[email protected] <mailto:[email protected]>> wrote:
Without fully appreciating _all_ the particulars (sorry!), I think
Jonathan's diagnostic (that people would tend to keep using
gregorian) is correct.
agreed.
I like the idea of a warning against that practice, with a
recommendation to use gregorian_nls if that's appropriate (and of
course a pointer to the relevant sections of the standard).
But I'm not sure that we need to lighten the spec. Let's say that
"gregorian" in no CF-1.7+ compliant. Yes, people will still use it, so
they will have slightly non-compliant files, but folks will still be
able to use them.
But people are more likely to do the right thing if is officially
non-=compliant than if it is simply against recommendations.
If the goal is to get as many files as possible to use gregorian_nls
(when appropriate) then it probably should be the standard.
Note that I'm still a bit confused about the particulars here:
It seems to me that the calendar describes what the epoch in a time
variable means.
the elapsed time _should_ be correct. period. If it says X seconds
since the epoch, then it should be X seconds since the epoch.
I understand that someone may have made a mistake/used an
inappropriate library, etc, such that there may be an off
by-a-couple-seconds error in the elapse time. But are we really trying
to include something in the standard to accommodate that?
There could be errors / lack of precision in ANY variable in a file --
what's so special about this one?
Or maybe I'm mis-interpreting the plan here.
-Chris
john
On May 20, 2015, at 07:16, Jonathan Gregory
<[email protected] <mailto:[email protected]>> wrote:
> Dear Jim
>
> If instead we decided not to redefine gregorian, but to replace
it with
> gregorian_nls, that would be a more definite indication that
action had been
> taken to use the right code, wouldn't it. Would you be content
with that?
>
> This isn't my favourite option, as you know. While it doesn't
seem onerous,
> I'm not sure it's realistic. I suspect that people might continue to
> code calendar="gregorian" anyway, even if the CF checker
reported it as an
> error, but perhaps I am too pessimistic! A compromise would be
to *recommend*
> coding gregorian_nls, meaning the same as (redefined) gregorian,
but indicating
> more definitely that there were no leap seconds in the encoding,
in order to
> reassure the user it had been done correctly. If we took that
course, the CF
> checker would give a warning if gregorian was coded, and
recommend that it
> should be changed to gregorian_nls. This is a bit like your idea
of having an
> extra attribute, but it's implied by the existing attribute. I
would be
> comfortable with this compromise. What do you and others think?
>
> Best wishes
>
> Jonathan
>
> ----- Forwarded message from Jim Biard <[email protected]
<mailto:[email protected]>> -----
>
>> Date: Wed, 20 May 2015 09:15:01 -0400
>> From: Jim Biard <[email protected] <mailto:[email protected]>>
>> To: [email protected] <mailto:[email protected]>
>> Subject: Re: [CF-metadata] How to define time coordinate in GPS?
>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0)
>> Gecko/20100101 Thunderbird/31.7.0
>>
>> Jonathan,
>>
>> Given the direction we have taken of using a new calendar name to
>> capture the other aspect, we are limited in our options. We could
>> specify an attribute that, if present, could be taken as a
>> "guarantee" (although people admittedly could set the attribute and
>> still do it wrong). The attribute name could be time_encoding with
>> values of 'true_elapsed' and 'unknown'. The value of 'unknown'
would
>> be the default, so if the attribute was not specified, the
>> assumption would be that you can't be sure. Most data producers
>> would have no need to specify the attribute, since their time
>> resolutions are not so fine as to make them sensitive to the
>> potential problems.
>>
>> Other than your suggestion about 1.7 implying greater care taken,
>> that's what comes to mind.
>>
>> Grace and peace,
>>
>> Jim
>>
>> On 5/19/15 5:23 PM, Jonathan Gregory wrote:
>>> Dear Jim
>>>
>>>> We are definitely getting much closer to full agreement. I
continue
>>>> to think that a separate time_system (or some such) attribute
would
>>>> be a much better way to handle this than modifying the calendar
>>>> attribute, and that space-separated calendar modifiers would
be next
>>>> best after that, but I will bow to the apparent majority and
agree
>>>> to your proposal for modified definitions of the general
reference
>>>> time and Gregorian calendar, the addition of a new gregorian_utc
>>>> calendar, etc, as you just outlined.
>>> That is good news. I am glad that we understand things in a
compatible way now
>>> thanks to this discussion, and I am grateful for your
flexibility and willing-
>>> ness to compromise on the implementation.
>>>
>>>> There are two remaining things that I would like to see.
>>>>
>>>> 1. A section that explains the importance for data producers and
>>>> consumers of using the right time handling routines for the
input
>>>> time data and the calendar chosen if your time resolution
makes you
>>>> sensitive to errors on the order of 1-30 seconds, pointing
out (for
>>>> example) that using the standard *nix time functions with
sets of
>>>> correct UTC timestamps will produce incorrect elapsed time
values
>>>> under the right conditions. Such a section would also point
out to
>>>> data consumers that if errors on this level are significant
to them,
>>>> they shouldn't assume that existing observational datasets
are free
>>>> of these errors.
>>> That's a good idea. I am in favour of it. We can do that in
the section where
>>> we describe the calendars.
>>>
>>>> 2. A standard mechanism for data producers to indicate that
they have,
>>>> in fact, taken the extra care with their time calculations,
>>>> whichever calendar they may have chosen - that is, the
elapsed time
>>>> values are guaranteed to be free of any leap second related
offsets
>>>> or discontinuities. This would give data consumers greater
>>>> confidence in cases where such errors matter.
>>> Yes, I see the value in that. I wonder how we would do it.
What comes to mind
>>> is to suggest that they are making that guarantee by
specifying CF-1.7 (I hope
>>> - or whichever version contains this change) rather than any
earlier version,
>>> which means in particular that the calendar will no longer
have a default. If
>>> they have changed their software to write CF-1.7 in the
Conventions attribute,
>>> we can hope that they have also changed it to be consistent
with the new and
>>> more precise definition of the calendar attribute. The changes
made are listed
>>> in an appendix to the standard (it will be Appendix Z in
CF-1.7) and the
>>> implication of this change should be made clear there. What
else could we do?
>>>
>>> Best wishes
>>>
>>> Jonathan
>>> _______________________________________________
>>> CF-metadata mailing list
>>> [email protected] <mailto:[email protected]>
>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>
>> --
>> CICS-NC <http://www.cicsnc.org/> Visit us on
>> Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
>> *Research Scholar*
>> Cooperative Institute for Climate and Satellites NC
<http://cicsnc.org/>
>> North Carolina State University <http://ncsu.edu/>
>> NOAA National Centers for Environmental Information
<http://ncdc.noaa.gov/>
>> /formerly NOAA?s National Climatic Data Center/
>> 151 Patton Ave, Asheville, NC 28801
>> e: [email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
>> o: +1 828 271 4900 <tel:%2B1%20828%20271%204900>
>>
>> /We will be updating our social media soon. Follow our current
>> Facebook (NOAA National Climatic Data Center
>> <https://www.facebook.com/NOAANationalClimaticDataCenter> and NOAA
>> National Oceanographic Data Center
>> <https://www.facebook.com/noaa.nodc>) and Twitter (@NOAANCDC
>> <https://twitter.com/NOAANCDC> and @NOAAOceanData
>> <https://twitter.com/NOAAOceanData>) accounts for the latest
>> information./
>>
>>
>
>> _______________________________________________
>> CF-metadata mailing list
>> [email protected] <mailto:[email protected]>
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
>
> ----- End forwarded message -----
> _______________________________________________
> CF-metadata mailing list
> [email protected] <mailto:[email protected]>
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
[email protected] <mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
[email protected] <mailto:[email protected]>
_______________________________________________
CF-metadata mailing list
[email protected]
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
CICS-NC <http://www.cicsnc.org/> Visit us on
Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
*Research Scholar*
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/>
/formerly NOAA’s National Climatic Data Center/
151 Patton Ave, Asheville, NC 28801
e: [email protected] <mailto:[email protected]>
o: +1 828 271 4900
/We will be updating our social media soon. Follow our current Facebook
(NOAA National Climatic Data Center
<https://www.facebook.com/NOAANationalClimaticDataCenter> and NOAA
National Oceanographic Data Center <https://www.facebook.com/noaa.nodc>)
and Twitter (@NOAANCDC <https://twitter.com/NOAANCDC> and @NOAAOceanData
<https://twitter.com/NOAAOceanData>) accounts for the latest information./
_______________________________________________
CF-metadata mailing list
[email protected]
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
