Dear Jonathan and Dan, Jonathan: The delineations of datasets are not yet clear at all, but I can very well imagine that one dataset might contain both types of indices. E.g. it is reasonable to assume that all "heat-related" indices are lumped together. For the other consideration, whether we would like to enable users to compare indices based on strict and non-strict inequalities, I am slightly more ambivalent. On the one hand, for many practical purposes the difference will be small (and should thus be allowed). On the other hand, with the possibility to set the threshold, my argument is that users should be encouraged to compare like with like. Users really keen on comparing the alternative types of indices will of course have the possibility to do so by means of a few extra processing steps.
Prompted by Dan's test case of Heathrow temperature I did a similar thing for Bromma airport in Stockholm (recorded to the nearest 0.1 degC): from 1951-01-01 to yesterday there are 1111 days with Tmax >= 25 degC, and 1073 days with Tmax > 25 degC. To echo Dan's view, I am not sure as to what difference the 38 occasions actual make in practice, but the distinction is more of a principal nature --- compare like with like. Moreover, the possibility for ET-SCI to change the definition of their TXgeNN indices to involve strict inequalities (which is what I personally thought was the cleaner way) have already been discussed (at some length) and ruled out. Thus, there are parties that do think that the distinction is important, and I think that it is relevant for CF to cater for this. Kind regards, Lars -----Original Message----- From: CF-metadata [mailto:[email protected]] On Behalf Of Hollis, Dan Sent: den 29 mars 2017 11:23 To: Gregory, Jonathan; [email protected] Subject: Re: [CF-metadata] Request for new standard names for climatological statistics based on thresholds Hi Jonathan, I find it hard to say whether the distinction is 'critical' - it just seems to me that, given CF goes to great lengths to be as precise as possible, we ought to be similarly precise when describing parameters derived from discretized data. Just out of interest, I ran a query on our database to look at daily max temperatures at Heathrow Airport since 01/01/2000. There are 461 days when the maximum was >= 25.0 deg C. Of these, 12 days are exactly 25.0 deg C and would be excluded if a strict inequality was used. Clearly there are uncertainties associated with observations of temperature, and the values themselves are only reported to the nearest 0.1 deg C. Whether a difference of 12 days is statistically significant I'm not sure. At the very least we know that "greater than" will always produce a lower count than "greater than or equal to" (except for small datasets that do not contain any instances of the threshold value) and it seems reasonable that the definition is clearly stated so that users can make an informed judgement when comparing datasets. Regards, Dan -----Original Message----- From: CF-metadata [mailto:[email protected]] On Behalf Of Jonathan Gregory Sent: 28 March 2017 16:16 To: [email protected] Subject: Re: [CF-metadata] Request for new standard names for climatological statistics based on thresholds Dear Lars and Dan I can't remember what I thought last time this was discussed, so I might be inconsistent! At present, it seems to me the most suitable solution is to have different standard names for strict inequalities and inequalities-allowing- equality, since as Lars says there that means only a dozen new standard names. Even if more are needed in future, it doesn't seem likely that there will be hundreds - but if so, we can adopt another solution when the need arises. But one other consideration is whether, in a given dataset, or when comparing datasets, the distinction is critical. Would you have both kinds in a single dataset, and would you decide that a <= quantity in one dataset should *not* be regarded as comparable to a < quantity from another dataset? If Yes in either case, the distinction is needed; but if No then it may not have to be made in the standard name. It could be recorded in a non-standardised way instead. Best wishes Jonathan ----- Forwarded message from Bärring Lars <[email protected]> ----- > Date: Tue, 28 Mar 2017 15:02:02 +0000 > From: Bärring Lars <[email protected]> > To: "Hollis, Dan" <[email protected]>, "[email protected]" > <[email protected]> > Subject: Re: [CF-metadata] Request for new standard names for climatological > statistics based on thresholds > > Hi Dan, all > > Thanks for these pointers. Obviously I somehow managed to miss this thread > when searching the mail archive (which I actually did try despite all > evidence to the contrary). I am sorry for that. > > Anyway, in an attempt to restart this discussion let me my summarising the > previous thread: > > It started off from the need to distinguish between dry and wet days > according to a threshold of some small precipitation amount (say 0.2 > mm[/day]), that typically had been observed at limited precision (thus > discretized). For floating point data from models this not a problem and the > arguments were that it would be more appropriate to take actual limits > induced by the discretization into account. > > There was a suggestion to use the existing standard name but adding a (non > CF) logical flag attribute to handle non-strict inequality. But this met > opposition as it would split the description of the data into two places (the > standard name and the logical flag attribute). > > It was noted that a strict and a non-strict inequality are indeed two > different things and should not be mixed up in the context of standard names. > The last comment was to suggest that the way forward would be to rename > (redefine) all the relevant standard names to include non-strict > inequalities. But, as was noted, this seems cumbersome. > > I hope this is a reasonable summary of that thread. With this background, let > me add some real use-cases that may shed some new light on the need to make > the distinction. > > I have in previous posts referred to the two WMO/WCRP/etc. expert teams > ETCCDI and ET-SCI defining and producing core sets of climate indices (aka > climate indicators). I am member of neither and have no vested interests, > other than I do see the need to make this kind of data more easily available > in a unified way. > > ETCCDI defines Summer Days ("SU") as the number of days when the daily > maximum temperature is strictly above +25 degC. This perfectly handled by the > standard name number_of_days_with_air_temperature_above_threshold. > > ET-SCI defines Hot Days ("TXge30" previously known as "SU30") as the number > of days when the daily maximum temperature is equal to or above +30 degC. In > structure it similar to SU with the crucial exception of a non-strict > inequality. And, this small but crucial difference in definition cannot be > changed for any number reasons. > > Now, in various user-oriented tools and web services the requirement is to > allow the users to change the threshold value to enable them to explore the > data and carry out simple sensitivity studies on their own. Thus, there is a > concrete need to handle the situation of having the ETCCDI index SU modified > to use the threshold value +30 degC (in which case it might perhaps be be > called "SU30"). This was in fact the very reason for renaming the ET-SCI > index "SU30" to "TXge30" to avoid misunderstandings. Likewise, ET-SCI index > TXge30 can be modified to use the threshold value +25 degC (which then would > be called "TXge25"). > > To facilitate the dissemination of these kinds of datasets to a wider > community it would indeed be helpful to have standard names that reflect the > perhaps small but still crucial difference between strict and non-strict > inequalities. > > I therefore like to again raise the suggestion to introduce the two > constructs "._at_or_above_threshold" and "._at_or_below_threshold" for the > relevant standad names. > > As I have outlined, we have real use-cases that go beyond what was the > driver for the discussion in the previous thread. Moreover, with nine > new standard names requested in my initial post the additional burden > on the standard name space is modest. However, if this is a concern > the use cases I have outlined currently cover the following standard > names number_of_days_with_air_temperature_above_threshold > number_of_days_with_air_temperature_below_threshold > number_of_days_with_lwe_thickness_of_precipitation_amount_above_thresh > old The remaining six I included without having an exhaustive overview > of what climate indices are used out there, and as they are identical in > their structure they could be changed at the same in one go without having to > revisit this issue again every time the need pops up. > > > Regards, > Lars > > > -- > Lars Bärring > > FDr, Forskare > PhD, Research Scientist > > SMHI / Swedish Meteorological and Hydrological Institute Rossby > Centre SE - 601 76 NORRKÖPING http://www.smhi.se > > E-post / Email: [email protected] > Tel / Phone: +46 (0)11 495 8604 > Fax: +46 (0)11 495 8001 > Besöksadress / Visiting address: Folkborgsvägen 17 > > > > > > -----Original Message----- > From: Hollis, Dan [mailto:[email protected]] > Sent: den 27 mars 2017 13:06 > To: Bärring Lars; [email protected] > Subject: RE: Request for new standard names for climatological > statistics based on thresholds > > Hi Lars, > > I raised a very similar question a couple of years ago: > http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2014/057605.html > > The outcome was inconclusive. One suggestion was to add a Boolean attribute > that indicated whether the threshold value was included or not. Others > thought that adding more standard names was the way to go, while others > thought that a single name was sufficient. > > I encourage you to read through the rest of the thread and see if it helps > with your current request: > http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2014/thread.html#576 > 05 > > For info, we continue to use the existing standard names even though they do > not strictly match the definitions of our 'days of rain' statistics. > > Regards, > > Dan > > > Dan Hollis Climatologist > Met Office Hadley Centre FitzRoy Road Exeter Devon EX1 3PB United > Kingdom > Tel: +44 (0)1392 884535 Mob: +44 (0)7342058682 Fax: +44 (0)1392 885681 > E-mail: [email protected] Website: > http://www.metoffice.gov.uk For UK climate and past weather > information, visit http://www.metoffice.gov.uk/climate > > > -----Original Message----- > From: CF-metadata [mailto:[email protected]] On Behalf > Of Bärring Lars > Sent: 24 March 2017 08:10 > To: [email protected] > Subject: [CF-metadata] Request for new standard names for > climatological statistics based on thresholds > > Dear all, > > Several standard names oriented towards climate indices for various impacts > are based on thresholds, and the standard name includes the construct > "..._above_threshold" or "..._below_threshold". However, several > well-established climate indices use non-strict inequalities in their > definition. > > For model output using floating point precision the difference between using > a strict and a non-strict inequality is small or even negligible, but for > observational data discretized to some limited precision (typically one or no > decimal digit) this makes a difference. > > At a workshop last week people involved in WMO/CCl Expert Team on > Sector-specific Climate Indices (ET-SCI) and the joint CCl/WCRP/JCOMM Expert > Team on Climate Change Detection and Indices (ETCCDI), as well as the > European ECA&D programme and several research projects discussed this. > > The outcome of these discussions is to suggest new standard names similar to > the existing ones but using the contructs "..._at_or_above_threshold" and > "..._at_or_below_threshold". In all other respects these new standard names > should be patterned after the following existing ones: > > number_of_days_with_air_temperature_above_threshold > number_of_days_with_air_temperature_below_threshold > number_of_days_with_lwe_thickness_of_precipitation_amount_above_thresh > old number_of_days_with_surface_temperature_below_threshold > number_of_days_with_wind_speed_above_threshold > spell_length_of_days_with_air_temperature_above_threshold > spell_length_of_days_with_air_temperature_below_threshold > spell_length_of_days_with_lwe_thickness_of_precipitation_amount_above_ > threshold > spell_length_of_days_with_lwe_thickness_of_precipitation_amount_below_ > threshold > > The specific use cases for these extension are several ET-SCI defined indices > that involves non-strict inequalities. > > The alternative of changing the ET-SCI definitions to use a strict inequality > is not an option because they have been painstakingly defined in > collaboration with user communities and/or are directly related to > well-established operational usage. > > Likewise, to just adjust the threshold in order to turn the non-strict > inequality to a strict equality (say from 30 C to 29.9 C or 29.99 C or ...) > is not attractive and prone to cause confusion. > > Kind regards, > Lars > > -- > Lars Bärring > > FDr, Forskare > PhD, Research Scientist > > SMHI / Swedish Meteorological and Hydrological Institute Rossby > Centre SE - 601 76 NORRKÖPING http://www.smhi.se > > E-post / Email: [email protected] > Tel / Phone: +46 (0)11 495 8604 > Fax: +46 (0)11 495 8001 > Besöksadress / Visiting address: Folkborgsvägen 17 > > > _______________________________________________ > CF-metadata mailing list > [email protected] > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > _______________________________________________ > CF-metadata mailing list > [email protected] > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ----- End forwarded message ----- _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
