Dear David, Lars,
I'm afraid I do have a couple of reservations, though I understand the
motivation for trying to support this kind of information in the convention.
My main concern is that "time" is a very important concept and it is distinct from "date". Time has a very precise
meaning which is recognised across scientific domains. I recognise that it may be used interchangeably with "date" in many
contexts, but the CF convention is built on precisely defined terms, so I feel it would be counter-productive to broaden our concept of
"time" to include "date". On the other hand, introducing a new CF name of "date" would make it possible to
introduce the sort of information that you are talking about.
My 2nd concern is the suggestion that we should relax the current specification of "units", which allows the
comprehensive UDUNITS package to be used when comparing variables without different units. I'm puzzled by your
suggestion the UDUNITS is opaque ... it looks like a pretty well maintained library to me. Their definition of
"year" as a "tropicalyear" agrees with the GNU units library. I can't see why the information you
want has to go in the "units" attribute (I'm sorry if I have missed your explanation on this point). There
are two alternatives which I would prefer:
(1) try to persuade UDUNITS to accept calendar_year and calendar_month as units of "date", or perhaps
"calendar_time". I don't think it would be worth trying to persuade them to add these of units of
"time", because, as Karl has mentioned, they are not convertible in the same way other time units are. As
units of "calendar_time", however, they would fit perfectly well into the UNIDATA and CF model.
(2) place the information in a different attribute. This is clearly a new
concept for the convention, so it would be reasonably to add a new attribute.
I think option (1) is worth trying, it would give a cleaner end result,
regards,
Martin
________________________________
From: CF-metadata <[email protected]> on behalf of David Blodgett
<[email protected]>
Sent: 21 October 2018 13:30
To: Bärring Lars
Cc: [email protected]
Subject: Re: [CF-metadata] [EXTERNAL] 'months since' and 'years since' time
units
Dear Lars,
Yes. I think we are in agreement. It seems odd to me that CF relies so heavily
on UDUNITS given the emphasis on non-opaque metadata in the spec generally.
So the relevant section is here:
http://cfconventions.org/cf-conventions/cf-conventions.html#time-coordinate
I think what we are proposing is to replace the cautionary paragraph about use of year
and month with an addition to the udunits convention that introduces "calendar"
to date/time units.
I think the calendar
list<http://cfconventions.org/cf-conventions/cf-conventions.html#calendar> is
sufficient given the addition of a non-udunits interpretation of a time axis?
Now the question is, what is the additional description? I think we have plenty
of food for that below.
Is there any disagreement with this approach out there? I expect their may be.
Best,
- Dave
On Oct 20, 2018, at 8:27 AM, Bärring Lars
<[email protected]<mailto:[email protected]>> wrote:
Dear David,
It seems that we pretty much agree, the CF Convention should do better than
just refer to the current Udunits for time units.
From a CF Conventions perspective is useful to separate the discussion of the
different year length in various calendars from the issue of uneven length of
calendar months (and Udunits somewhat draconian way of solving this to little
relevance for this community I would argue).
If the solution to both issues is to adopt the solution you refer to, this
would be perfectly fine with me. But to me the important thing is that the
underlying rules should be part of the CF Convention, either by expressing them
in the CF Conventions document, or by making a reference to an external
document (as is now done for Udunits).
Kind regards,
Lars
________________________________
Från: David Blodgett [[email protected]<mailto:[email protected]>]
Skickat: den 20 oktober 2018 13:47
Till: Bärring Lars
Kopia: [email protected]<mailto:[email protected]>
Ämne: Re: [CF-metadata] 'months since' and 'years since' time units
Dear Lars,
Maybe my point is lost in the reference to an outside source. I think CF should
support more than udunits. It should support an extended units attribute for a
variable meant to represent a calendar/time axis.
That extension should be prepending "calendar" to a udunits date string. This is the second
option in the NetCDF-Java Common Data
Model<https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/#home> which was
adopted from the Environmental Data Abstraction
Library<https://www.semanticscholar.org/paper/The-Environmental-Data-Abstraction-Library-(-Edal-)-Griffiths-Haines/77c1018b473dbcee44068075a6458c5ef2c2f5a4>,
both of which support CF but also support non-cf data that exists in the community.
If the developers of udunits want to support the extension, that would be great
-- they would be coming into line with practices implemented by others in the
community.
Regarding the analogy to length units. Let's dig in a bit. Units declarations define the
function that has to be applied to take one quantity and transform it to a reference datum or
some other arbitrary datum. Time is no different -- the function is just a little different.
There are plenty of "transformed" quantities that require a specialized function to
go from the units they are stored in to units that make the data comparable or useful in some
other context (sigma coordinates for
example)<https://www.phy.ornl.gov/csep/om/node24.html>. In this case, what we are saying
is that we have a specialized function for time that has real utility and isn't supported by
the assumptions in udunits. We even have implementations of our specialized function and a way
to describe it!!
All the best,
- Dave
On Oct 20, 2018, at 5:39 AM, Bärring Lars
<[email protected]<mailto:[email protected]>> wrote:
Hi all,
@Jim, I think that even within the domain of months and years, a month is not
necessarily a well behaved unit. Think of precipitation height (mm) or sunshine
hours (h); these quantities depends on the length of the month. Without taking
the month length into account one cannot really compare precipitation height
during January with that of February (10 % difference in accumulation period)
despite that the difference in the time coordinate is 1. If one on the other
hand factor out the the length of the months by using precipitation flux (kg
m-2 s-1), or precipitation rate (mm s-1, mm h-1, mm day-1, but not mm month-1),
and fraction_of_sunshine_hours (1), things suddenly are OK.
@David, I am not familiar with the software you are pointing at, but from your
description (and my rather limited software engineering skills) it certainly
sounds like a good way forward/solution. But what I am after is that the CF
Conventions as such should be more specific and not back away from the issue by
just referring to Udunits, especially as CF has otherwise done such an
excellent work in sorting out the sometimes hairy issues pertaining to
geophysical modelling data and observations. After all, the complications from
having different calendars is specific to the community that CF sets out to
support (and sort out...).
@Chris (your last para in earlier post):
"I know that CF refers to udunits for unit definitions, but we really need to either
allow exceptions or periodically update udunits to meet the needs of CF."
Yes and Yes to this suggestion !
Lars
________________________________
Från: CF-metadata
[[email protected]<mailto:[email protected]>] för Jim
Biard [[email protected]<mailto:[email protected]>]
Skickat: den 19 oktober 2018 22:32
Kopia: [email protected]<mailto:[email protected]>
Ämne: Re: [CF-metadata] 'months since' and 'years since' time units
Chris,
I see what you are saying about category, yet it is metric. Within the domain
of months and years (without days, leap days, etc), it is completely possible
to do exact math, just as we can within the domain of seconds, minutes, hours,
and days. The problem arises when you try to bridge between these two domains.
The current UDUNITS tries to keep it all in one domain, with the result that
values in months and years become problematic.
Grace and peace,
Jim
On 10/19/18 3:53 PM, Chris Barker - NOAA Federal wrote:
I remember this coming up on this list a while back.
Yes, the concept of e.g. monthly averaged data is useful.
But in that case, a “month” is really a category, not a continuous time
variable.
So we need a different way if describing it than a time access with units of
month...
-CHB
Sent from my iPhone
On Oct 19, 2018, at 8:45 AM, Jim Biard
<[email protected]<mailto:[email protected]>> wrote:
I like how you have described the issue, Chris.
Using month in anything except a 360-day calendar (assuming the month is
defined correctly for that calendar) produces erroneous results if you try to
do anything but math that remains in those units - such as convert a month
count to a date. The same goes for physical years.
And yet... If we are writing data collected on monthly, seasonal, or other
large-scale time binnings (when doing climatologies, for example), month and
year become regularized - that is, they become metric within the context of
that data, and it would be so much more human-friendly to be able to work
directly in months, seasons, and years, regardless of whether that is a real
year, a 365 day year, or a 360 day year.
It seems to me that time recorded in "month and larger" terms really represents
a different thing, separate from 'time' as defined by CF. I see two different solutions
that could be based around a new standard name of 'date', as Martin Juckes suggested.
* Declare that the values for a standard name of 'date' are date strings
following a YYYY-MM-DD format (and perhaps something for megayears, etc that
would be useful for paleo people).
* Declare that the values for a standard name of 'date' would use units of
calendar_month, calendar_season, and calendar_year (or whatever names people
like best). These units are entirely convertible between each other, and would
need to be added to UDUNITS. UDUNITS would not convert between these units and
time units. People would be free to write code that could do the conversion
between time and date, but there wouldn't be any particular way that would be
considered standard.
Grace and peace,
Jim
On 10/19/18 11:13 AM, Chris Barker - NOAA Federal wrote:
Calendars are a mess— both because the earth’s rotation is not
human-frendly, and because of human legacy.
So we need to accept that, and not try to use calendars as though they
are logical units for time.
I like to think about it this way: there are time operations: “this
much time has passed”, and there are calendar operations: “this day
next month”.
Calendar operations require a lot of machinations and a well defined
calendar. Time operations are straightforward and behave “sensibly”
with the usual math.
Also — operations belong in libraries, not data files. CF should use
only well defined units.
Personally, I think udunits should never have defined “months” or
“years” as a time unit. But in any case, CF can at least highly
discourage, if not ban, their use.
Possible exception: for “artificial” Calendars, a month can be well
defined, but then the unit should be called something like
“30_day_month”, other well defined name.
I know that CF refers to udunits for unit definitions, but we really
need to either allow exceptions or periodically update udunits to meet
the needs of CF.
-CHB
On Oct 19, 2018, at 5:13 AM, Bärring Lars
<[email protected]><mailto:[email protected]> wrote:
Dear all,
I agree with Jonathan's wish for a more well-behaved Earth in the planetary
system :-)
However, awaiting this I think that we have two issues before us:
1. The fact that different datasets fundamentally are based on different length
of a year, while Udunits defines a year to be the real world tropical year.
2. The common language notion of months of different length (and for February
varying), while Udunits defines the length of a month to be one twelfth of the
real world tropical year.
I fully agree that for datasets from one source (having one calendar) the warning against using
"month_since..." and "year_since..." is very well motivated. But, as Martin
points out, when combining data based on different calendars the easiest /best/most relevant way is
to use one of these. Having said that I will in turn go into some detail with the two issues.
1. Regarding the length of the year, which is what my previous posts concerned,
I think that we have three options:
A) Convince Udunits to change the behaviour of the year unit to depend on the
calendar. To my mind this would actually solve the problem stated in the
initial post of this thread. Whether this option s possible, have a chance to
fly, or is something that the CF community actually want I do not now. But here
is the idea for you to consider. If this solution is implemented, it would
mean changing the text in Section 4.4.
B) In the CF Conventions specify that the definition of the length of a year
deviates from the one used in Udunits. This would mean changing the text in
Section 4.4. Again, once this change has penetrated into software libraries,
this would solve the initial poster's problem.
C) Leave things as they are, preferably with some additional language in
Section 4.4 to explain when unit year might be useful and when not to use it.
Of course, there might be other options that I have not though of. Personally I
advocate A) or B), because to my mind the impact of having defined different
calendars is not fully implemented in CF because the different model calendar
_do_ imply different length of the year. Moreover, at least for the 360_day
calendar the Udunits definition of a month will then coincide with what is
expected from that calendar.
2. The obvious root of the problem here is of course the different months' lengths in the real world calendar. Having
said that, what adds to the confusion is that Udunits has chosen the name "month" for the unit year/12. I
will put aside the problem of different month lengths and only consider the Udunits months, which I here will call
"month12" for clarity, and focus on the implications of having data from different calendars being merged
(think ensemble summary) and described using "month12_since" (where the time coordinate takes on integer
values). For all calendars, but 360_day, this implies that a fractional day goes into the month12ly statistic. If this
is a problem one may one may postulate that the fractional day belongs to the month12 where the larger part belongs. If
this is complemented with the notion of calendar_month it may be reasonable to restrict at least the latter, or both,
to only take on integer values, to avoid hairy questions like ones that Jonathan asked "What does 1
calendar_month since 31 January mean? What does 7.23 calendar_months since 31 January mean?"
Kind regards,
Lars
PS. For those of you without a Scandinavian keyboard; you might relieved to
know that my first name is Lars (and Bärring is family name, despite what the
email alias might suggest)
________________________________________
Från: CF-metadata [[email protected]] för Taylor, Karl E.
[[email protected]]
Skickat: den 19 oktober 2018 06:14
Till: [email protected]
Ämne: Re: [CF-metadata] 'months since' and 'years since' time units
Dear all,
I won't make any recommendations for udunits, but I will comment on the
CF-conventions.
In general, I think we should discourage usage of calendar month as a
unit of measurement because for the real world, these are only defined
for 12 special periods each year (the beginning to the end of each
calendar month) and the
"<mailto:Kindregards,LarsPS.ForthoseofyouwithoutaScandinaviankeyboard;youmightrelievedtoknowthatmyfirstnameisLars%28andB%C3%A4rringisfamilyname,despitewhattheemailaliasmightsuggest%29________________________________________Fr%C3%A5n:CF-metadata[[email protected]]förTaylor,KarlE.[[email protected]]Skickat:den19oktober201806:14Till:[email protected]%C3%84mne:Re:[CF-metadata]%27monthssince%27and%27yearssince%27timeunitsDearall,Iwon%27tmakeanyrecommendationsforudunits,butIwillcommentontheCF-conventions.Ingeneral,Ithinkweshoulddiscourageusageofcalendarmonthasaunitofmeasurementbecausefortherealworld,theseareonlydefinedfor12specialperiodseachyear%28thebeginningtotheendofeachcalendarmonth%29andthe>unit"
is not constant throughout the year.
Nevertheless there are some good arguments for considering adopting a
special calendar month unit by the CF conventions, but only for limited
and very specific purposes. (I'll refer to these new units as
"calendar_months" in the following, with the understanding that the unit
will depend on the calendar adopted and will in general vary for
different months of the year and depend on whether or not the year is a
leap year.) Here are reasons (already articulated by others) why we
might want to adopt "calendar_months" as a unit:
1) there are existing data sets with monthly-mean data and a
time-coordinate that is supposed to indicate how many calendar months
have passed since some base month/year. Such data sets are not easily
accommodated by CF.
2) for some judiciously selected reference times, coordinates expressed
in "calendar_months since" can be easily generated without the help of
calendar-aware time-translation algorithms. For example, given units of
"calendar_months since 2001-01-01" we can trivially generate the
coordinate values for the middle of each month: 0.5, 1.5, 2.5, etc. It
would be much more difficult to generate the coordinate values in units
of "days since ...".
3) monthly mean data sets with time coordinates based on different
calendars can more easily be compared because if all the data sets
adopted the same reference time, then comparable months would have the
same coordinate values, independent of the actual month length defined
by different calendars. [In contrast, if the time coordinate were given
in units of "days since ... ", the coordinate values would depend on the
calendar.]
If we define a unit of "calendar_months since ..." we would need to
address Jonathan's point that it is not immediately obvious how to
handle fractions of months and reference times different from the
beginning of a month. One approach we should consider is to restrict
use of calendar_month units to datasets reporting monthly values only
(not, daily, annual, hourly, etc.) Also for simplicity we could
restrict the reference time to be the beginning of a calendar year
(i.e., January 1 at 0:0:0 for some specified year). If we did this, it
would be relatively easy to define what we mean by fractions of months
and it would also be easy to generate the values needed to define the
time-coordinates and the cell_bounds or the bounds needed to define
climatologies. [It would be almost as easy if we allowed the reference
time to be the beginning of *any* month, but let's consider the more
restrictive "beginning of a year" option first.]
I note that using calendar_month units to report data at intervals other
than monthly intervals offers no advantages. For example different
fractional month increments would have to be used to report data at an
invariant daily interval. This would seem to introduce complexity to a
simple time-coordinate and is why I would restrict use of calendar_month
units to monthly data.
Since months are not all equal in length, the interval of times
represented by the same fraction may differ between months. For a
Gregorian calendar, half-way through the month of January would be noon
on the 16th of January (15.5 days from the beginning of the month),
whereas half-way through the month of June would be the 16th of June at
00:00:00 (15.0 days from the beginning of the month. Thus 0.5 months
since 2001-01-01 would be 2001-01-16 12:00:00 and 5.5 months since
2001-01-01 would be 2001-06-16 00:00:00. Also note that the date
corresponding to the middle of February depends on whether the year is a
leap year or not.
Of course for a different calendar (e.g., 360_day), the dates
corresponding to 0.5 months since 2001-01-01 and 5.5 months since
2001-01-01 would be different from those for the Gregorian calendar.
The bottom line is that under the above restrictions, it is easy to
convert fractions of months to dates (and also to alternative units such
as "days since ...").
Common types of "monthly" data sets are:
1) reports of monthly statistics computed from multiple samples within
each month (e.g., means, standard deviation of daily values, maximum
daily mean; requires cell_bounds)
2) reports of monthly accumulations (e.g. monthly precipitation amount;
requires cell_bounds)
3) monthly climatologies (requiring climatology attribute pointing to
climatology bounds)
For all of the above the bounds coincide with the beginning and end of
each month so with the reference time restricted to the beginning of a
year, the bounds will all be integer values of "months since". The
coordinate value for any of the above can be assigned any value in the
interval defined by the bounds. For monthly statistics one might choose
the middle of each month so coordinate values would be numbers like 0.5,
1.5, 2.5, etc. For monthly accumulations one might prefer to use the
time at the end of each interval to represent the coordinate value
(e.g., 1.0, 2.0, 3.0, etc.).
I understand that defining a unit of calendar_months is not compatible
with udunits, but I think we can rely on tools other than udunits to
handle more generally this new unit and the various CF conventions
calendars to convert coordinate values to other time units like "days
since ...".
Perhaps the biggest argument in favor of introducing a calendar_month
unit is that it should make it much easier for data providers to
generate correct time coordinates for data reported at monthly
intervals. Regardless of calendar, I think it is easy to generate
monthly time coordinates under the current CF standard that are simply
wrong. In contrast, everyone should be able to trivially create a
coordinate axis with values like (1, 2, 3, .... ) or (0.5, 1.5, 2.5,
...) without making a mistake.
best regards,
Karl
On 10/18/18 10:58 AM, Jonathan Gregory wrote:
Dear Martin
The definition of a calendar_month unit would also need rules about calendar
arithmetic e.g. What does 1 calendar_month since 31 January mean? What does
7.23 calendar_months since 31 January mean?
Best wishes
Jonathan
----- Forwarded message from Martin Juckes - UKRI STFC
<[email protected]><mailto:[email protected]> -----
Date: Thu, 18 Oct 2018 16:33:28 +0000
From: Martin Juckes - UKRI STFC
<[email protected]><mailto:[email protected]>
To: Jonathan Gregory <[email protected]><mailto:[email protected]>,
"[email protected]"<mailto:[email protected]>
<[email protected]><mailto:[email protected]>
Subject: Re: [CF-metadata] 'months since' and 'years since' time units
Dear Jonathan,
I think you could go further and disallow the use of "month" or "year" as a
time unit when the calendar is not standard.
If the "ncdump -t" option produces what the user expects when he has units "months
since 1900-01-01" and a 360 day calendar, then it is going to be inconsistent with the current
convention.
I still feel that there is an argument for enabling the storage of information in user months in the files.
E.g. I wish to compare monthly mean data from 20 climate models which use a range of different calendars. The
mean across the models is not on any specific calendar ... I could pretend it is and use units of "days
since ...", but the mappings from input time coordinates to output time coordinates then become rather
complex, when they should be trivial. Having a "date" standard name which allowed the input data to
have a "calendar_month" coordinate would solve this (and I think Klaus's suggestion would also
solve it),
regards,
Martin
________________________________
From: CF-metadata
<[email protected]><mailto:[email protected]> on behalf of
Jonathan Gregory <[email protected]><mailto:[email protected]>
Sent: 18 October 2018 17:10:46
To: [email protected]<mailto:[email protected]>
Subject: Re: [CF-metadata] 'months since' and 'years since' time units
Dear all
This is an interesting discussion, and I agree that's a tricky subject. If only
we could have a well-behaved Earth which orbited the sun in an integral and
easily factorisable number of days!
So far I still think that we should not change the way we interpret the units
string. It's in udunits format, and should be interpreted according to the
calendar attribute. I would suggest that it's helpful to regard time coords as
*encoded* and not necessarily easy for humans to read. It's certainly nice to
see "time=1, 2, 3, ..." months since a reference date - that is easy to read -
but when you get to 747 or 4689 months since a reference date, you don't know
what it means any more (unless you're extremely good at mental arithmetic), and
you might as well encode it as days.
The antidote to inconvenient encoding is convenient software. For example,
could cftime allow the user to construct a time coordinate variable with a
spacing of calendar months, but encode it in the netCDF file in days? Then it's
transparent. Similarly, time coordinate variables can be decoded into human-
readable strings by calendar-aware software. It seems to me that this isn't
different in principle from using ncdump to read a netCDF file, rather than
insisting it should be intelligible when read in hexadecimal. In fact, ncdump
itself has a -t option, which should help, according to the man page:
"-t controls display of time data, if stored in a variable that uses a udunits
compliant time representation such as `days since 1970-01-01' or `seconds since
2009-03-15 12:01:17' .... If this option is specified, time data values are
displayed as human-readable date-time strings rather than numerical values,
interpreted in terms of a `calendar' variable attribute, if specified. ...
Calendar attribute values interpreted with this option include the CF
Conventions values `gregorian' or `standard', `proleptic_gregorian', `noleap'
or `365_day', `all_leap' or `366_day', `360_day', and `julian'."
I agree with comments that if we introduced new units such as calendar_month
or 30day_month, people might well not use them, and would still be disappointed
that "month" wasn't what they expected.
The CF conformance document has a recommendation that "year" and "month" should
be used "with caution". I don't what the CF checker currently does with this.
We could change it to a recommendation that they should *not* be used, in which
case the checker would give a warning if they were.
Best wishes
Jonathan
----- Forwarded message from Bärring Lars
<[email protected]><mailto:[email protected]> -----
Date: Thu, 18 Oct 2018 13:31:10 +0000
From: Bärring Lars <[email protected]><mailto:[email protected]>
To: Martin Juckes - UKRI STFC
<[email protected]><mailto:[email protected]>, David Blodgett
<[email protected]><mailto:[email protected]>, Ryan Abernathey
<[email protected]><mailto:[email protected]>
Cc: "[email protected]"<mailto:[email protected]>
<[email protected]><mailto:[email protected]>
Subject: Re: [CF-metadata] 'months since' and 'years since' time units
Dear Martin, David, all,
As Klaus points out, the aim of my suggestion is to make software using CF aware of the
fact that the unit "year" is different depending on which calendar the model is
implementing. To give an example:
If I want to know when the global average temperature has increased by 1.5C, or
4C, above pre-industrial time in the CMIP5 ensemble I will get answers as a
timedelta in days. As this is not really helpful I might feel inclined to
convert this to years, but now UDUNITS definition of year is not helpful for
those models having a 360_day or 365_day calendar. However, with the
calendar-aware definition of year such a calculation would be supported without
having to deal with it manually.
Now, on to the discussion about months. Before my previous post I quickly read
through extensive exchange on this list back in 2011, so I really appreciate
David's comment that it is a complex subject. And that is the reason for why I
suggested is always month is always a year / 12. So, here is an attempt to
summarise the suggestion in a different way:
* standard and proleptic_gregorian calendars (status quo):
o the number of days in a month is not an integer
o same issue with respect to ordinary (western) world months.
* 365_day calendar:
+ the number of seconds in a month would change from being "ill-defined (?)"
as 84600 * 365.242198781 / 12 = 2574957.50141, to more properly 84600 * 365 / 12 = 2573250
o same issue with respect to ordinary (western) world months.
* 360_calendar:
+ the number of seconds in a month would change from being "very ill-defined
(?)" as 84600 * 365.242198781 / 12 = 2574957.50141, to more properly 84600 * 360 /
12 = 2538000
+ the number of days in a month is an integer; 12 * 30 * 84600 = 2538000
+ the definition of a month is consistent with what is expected in the "360_day
world"
o same issue with respect to ordinary (western) world months.
That is, even though the suggestion certainly do not solve everything (of
course!), the only argument against it, that I can see, is the work to tease
out the details and implement it in software packages. As was extensively
discussed in the 2011 threads, the real problem is the varying length of the
western world calendar months. But that is the topic for another thread.
Kind regards,
Lars
________________________________
Från: CF-metadata
[[email protected]<mailto:[email protected]>] för David
Blodgett [[email protected]<mailto:[email protected]>]
Skickat: den 18 oktober 2018 13:58
Till: Ryan Abernathey
Kopia: [email protected]<mailto:[email protected]>
Ämne: Re: [CF-metadata] 'months since' and 'years since' time units
Dear Ryan, All,
I hesitate to chime in on this thread as I know just how fraught this topic can be, but
then I think I know how fraught it can be so may have something to offer. My experience
is at the intersection of climatological models and landscape models that are calibrated
with "real" data. I've worked with daily and monthly time series model output
and interpolated weather products that needs to match up to observations but uses a
noleap or 360 calendar. It's an enormous pain and we as a community should do better. --
so the business case for taking this complexity head on is there!
One resource I've found useful over the years is the [CDM
implementation](https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/CDM/CalendarDateTime.html)
ut this does not
There are two factors at play.
1) Adding "calendar" to a udunits string avoids conversion to a number of
shorter time increments for long time increments (e.g. seconds per month). It keeps
things in the declared time units so you hit the precise date boundaries you would expect.
2) The "calendar" attribute gets you to how to interpret the datum of the time
axis.
Especially relevant to this thread is:
* uniform30day or 360_day = All years are 360 days divided into 30 day
months.
With these two, I think the problems here are solved. However, inevitably, people will omit the
addition attribute for calendar or fall back on normal "months since ..." when they
actually mean "calendar months since ..." and tell us 'why would you interpret my data
that way it makes no sense?!?' This is perfectly parallel to spatial coordinates where people don't
declare a datum for their latitude/longidute coordinates. Without that information one can not use
the information with a level of precision that some use cases require.
What I'm getting at is that CF should probably:
1) adopt enough attribute precision to fully describe what we are trying to
convey
2) make said attributes required or declare sensible defaults that reduce
ambiguity when users come knocking.
That said, I've had no success pushing the community to accept that there
should be a default lat/lon datum for software developers to go on and I would
not doubt that the same will be true here as ambiguity and uncertainty is
better than dead wrong in many cases. My stance is that we should all be dead
wrong for the same reason rather than each implementor making an arbitrary
decision so we all get different answers (more ambiguity) from our software
du-jour.
All the best,
Dave
On Oct 18, 2018, at 6:08 AM, Martin Juckes - UKRI STFC
<[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>>
wrote:
Hello All,
I think the UNIDATA pull request referenced Jeff (https://github.com/Unidata/cftime/pull/69) is
mis-quoting the CF Convention. As far as I can see, Unidata says that a month is exactly one 12th of
a year, and CF inherits this -- with the statement "For similar reasons the unit month, which is
defined in
udunits.dat<http://www.unidata.ucar.edu/software/udunits/><http://www.unidata.ucar.edu/software/udunits/>
to be exactly year/12, should also be used with caution."
I can't see the difference between Lars's suggestion and the status quo. In UNIDATA a day
is clearly defined as "period of time equal to 24 hours", which gives 84600
seconds.
regards,
Martin
________________________________
From: CF-metadata
<[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>>
on behalf of Bärring Lars
<[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>>
Sent: 18 October 2018 09:29:50
To: Ryan Abernathey;
[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>;
[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>
Subject: Re: [CF-metadata] 'months since' and 'years since' time units
Hi,
I have have come to think about this from a somewhat different perspective. For
some analyses, as well as when calculating certain derived climatological
statistics (aka climate indices), using datasets based on different calendars
the problem becomes obvious.
In the model world of a 365-day GCM one year _is_ 365 days, and in a 360_day
GCM a year _is_ 360 days. In the case of a gregorian/standard calendar GCM I am
not sure whether it is 365.25 or 365.242198781 but this is in practice less of
a problem.
For datasets based non-standard calendars imposing the current UDUNITS
definition of a year leads to complications that require workarounds if one is
interested in for example the time elapsed until something happens or the
duration of some (long-lasting) events. One way to partly mitigate these issues
would be to use the time unit of years_since_START or months_since_START, but
this is warned against in the CF Conventions and may software tools do not
support it .
The fundamental issue is the inconsistency between the GCM year and the UDUNITS
year. So I would like to call on the wisdom of this list to see whether the CF
Convention could include a modification to the definition of a year and month:
* standard calendar (no change)
1 day = 84600 seconds
1 year = 365.242198781 days
1 month = 365.242198781 / 12 days
* 365_day calendar
1 day = 84600 seconds
1 year = 365 days
1 month = 365 / 12 days
* 360_day calendar
1 day = 84600 seconds
1 year = 360 days
1 month = 360 / 12 days
That is, the seconds per day ratio is not changed thus maintaining the
consistency to other SI units. And, for the 360_day calendar month follows the
suggestion by Ryan and Jeffrey.
Kind regards,
Lars
--
Lars Bärring
FDr, Forskare
PhD, Research Scientist
SMHI / Swedish Meteorological and Hydrological Institute
Rossby Centre
SE - 601 76 NORRKÖPING
Tel / Phone: +46 (0)11 495 8604
Fax: +46 (0)11 495 8001
Besöksadress / Visiting address: Folkborgsvägen 17
________________________________
Från: CF-metadata
[[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>]
för Ryan Abernathey
[[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>]
Skickat: den 17 oktober 2018 21:22
Till:
[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>
Kopia:
[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>
Ämne: Re: [CF-metadata] 'months since' and 'years since' time units
Hi everyone,
I am that user, and I'm new to this mailing list. Thank you all for your work
on CF conventions. It's such a valuable effort!
I want to note that this was inspired by the proliferation of datasets in the wild that use
"month" as their units. For example, nearly all of the IRI Data Library does this, in
conjunction with a 3"60_day" calendar (example:
https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP-NCAR/.CDAS-1/.MONTHLY/.Diagnostic/.surface/.temp/).
My impression from talking to data providers is that no one is using "360_day" calendar and
"months" as units, and then expecting "months" to be interpreted as 365.242198781/12 days. They all
expect it to be interpreted as 30 days. While there are various workarounds that can be used at different levels of the
software stack, the best solution, IMHO, is to explicitly allow in CF conventions what Jeff proposed: "months and
years be interpreted as calendar months and years for those calendars where they have a fixed length". I don't
think this will break existing applications.
Thanks,
Ryan
On Wed, Oct 17, 2018 at 3:06 PM Jeffrey Whitaker
<[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>>
wrote:
Hi: I'm a developer of the 'cftime' python package (https://github.com/Unidata/cftime).
A user submitted a pull request (https://github.com/Unidata/cftime/pull/69) that
implements support for a 30-day calendar month time unit for the '360_day' CF calendar.
Although using a 'month' time unit is a tricky proposition in general, for this calendar
it seems straightforward since every month has the same length. However, in the
discussion of the pull request it was pointed out that CF expects that "the value
of the units attribute is a string that can be recognized by UNIDATA's Udunits
package", and that UDUNITS defines a month as 365.242198781/12 days. My question
is this - is it reasonable for our python package to make an exception to this rule for
the 360_day calendar? More generally, can months and years be interpreted as calendar
months and years for those calendars where they have a fixed length, or will this deviate
from the existing CF conventions and break existing applications?
Regards, Jeff
--
Jeffrey S. Whitaker
NOAA/OAR/PSD R/PSD1
325 Broadway, Boulder, CO, 80305-3328
Phone: (303)497-6313
FAX: (303)497-6449
_______________________________________________
CF-metadata mailing list
[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
[email protected]<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
[email protected]<mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
----- End forwarded message -----
_______________________________________________
CF-metadata mailing list
[email protected]<mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
----- End forwarded message -----
_______________________________________________
CF-metadata mailing list
[email protected]<mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
[email protected]<mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
[email protected]<mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
[email protected]<mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
[CICS-NC]<http://www.cicsnc.org/>Visit us on
Facebook<http://www.facebook.com/cicsnc> Jim Biard
Research Scholar
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/>
formerly NOAA’s National Climatic Data Center
151 Patton Ave, Asheville, NC 28801
e: [email protected]<mailto:[email protected]>
o: +1 828 271 4900
Connect with us on Facebook for climate<https://www.facebook.com/NOAANCEIclimate> and ocean and
geophysics<https://www.facebook.com/NOAANCEIoceangeo> information, and follow us on Twitter at
@NOAANCEIclimate<https://twitter.com/NOAANCEIclimate> and
@NOAANCEIocngeo<https://twitter.com/NOAANCEIocngeo>.
_______________________________________________
CF-metadata mailing list
[email protected]<mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
[CICS-NC]<http://www.cicsnc.org/>Visit us on
Facebook<http://www.facebook.com/cicsnc> Jim Biard
Research Scholar
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/>
formerly NOAA’s National Climatic Data Center
151 Patton Ave, Asheville, NC 28801
e: [email protected]<mailto:[email protected]>
o: +1 828 271 4900
Connect with us on Facebook for climate<https://www.facebook.com/NOAANCEIclimate> and ocean and
geophysics<https://www.facebook.com/NOAANCEIoceangeo> information, and follow us on Twitter at
@NOAANCEIclimate<https://twitter.com/NOAANCEIclimate> and
@NOAANCEIocngeo<https://twitter.com/NOAANCEIocngeo>.
_______________________________________________
CF-metadata mailing list
[email protected]<mailto:[email protected]>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
[email protected]
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
