[CF-metadata] Use of axis attribute in an auxillary coordinate

[martin.juckes]

Dear Jim, David,


Sorry for dropping this thread from last August.


I agree that the convention does not have to adhere rigidly to the NUG 
definition of a coordinate variable, but the text of version 1.7 implies that 
it does. David's suggested rewording of the definition of an auxiliary 
coordinate below would begin to address this, but there may be other places 
where a change is needed. E.g.

The current text says that "The value of the coordinates attribute is a blank 
separated list of the names of auxiliary coordinate variables." (3rd paragraph 
in section 5) -- I think you are saying that this is not the meaning you want?

I'm still not entirely clear what you are trying to achieve here .. the 
definition of auxiliary coordinate in version 1.7 looked fine to me (where it 
is clearly stated in relation to the NUG definition of a coordinate variable): 
I'm not objecting to your change, but it might help this conversation if you 
explained the rationale behind the change.

regards,
Martin




Martin,

The height scalar variable you are referring to is numeric, right? It
may not be a coordinate variable in the pure NUG sense, but CF isn't
bound by exact adherence to NUG if it defines its terms, and David has
posted the relevant sections from the Conventions. It is clear to me
that those sections declare that a numeric scalar variable may serve as
a valid 'true' coordinate variable, /as opposed to/ an auxiliary
coordinate variable. The mechanism for relating a scalar coordinate
variable to a data variable is through the 'coordinates' attribute, but
that does not force it to be considered to be an auxiliary coordinate.
Martin is trying to find a way to clarify these concepts, since you seem
to find them unclear.

What would make it clearer to you that CF declares that the height
scalar variable mentioned in this discussion qualifies as a true or full
or complete (or whatever word you prefer) coordinate and is not
relegated to auxiliary coordinate status?

Grace and peace,

Jim

On 7/31/17 10:10 AM, martin.juckes at 
stfc.ac.uk<http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> wrote:
>
> Why?
>
> *From:*David Hassell [mailto:david.hassell at 
> ncas.ac.uk<http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata>]
> *Sent:* 31 July 2017 14:50
> *To:* Juckes, Martin (STFC,RAL,RALSP)
> *Cc:* CF Metadata
> *Subject:* Re: [CF-metadata] Use of axis attribute in an auxillary
> coordinate
>
> Hi Martin,
>
> In #104 <https://cf-trac.llnl.gov/trac/ticket/104> perhaps we should
> have updated the definition of an auxiliary coordinate variable to
> reflect the clarification of scalar cooridnate variables. Something like:
>
>
> auxiliary coordinate variable
>
> Any netCDF variable that contains coordinate data, but is not a
> coordinate variable (in the sense of that term defined by the NUG and
> used by this standard - see below) */nor is functionally equivalent to
> one (such as a numeric scalar coordinate variable)/*. Unlike
> coordinate variables, there is no relationship between the name of an
> auxiliary coordinate variable and the name(s) of its dimension(s).
>
> This could be done with a defect ticket.
>
> All the best,
>
> Diavd

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Definiton of solar_irradiance

[martin.juckes]

Hello Jim,

the usage of solar_irradiance to refer to solar irradiance per unit horizontal 
area was suggested by the RFMIP team, and initially appeared to be supported by 
Roy .. but I'm happy to drop it.

Following the discussion here, which has been very helpful, I think that the 
standard name that RFMIP actually want is toa_incoming_shortwave_flux , and it 
might be helpful to indicate in the definition of this standard name that it is 
a synonym for Global Horizontal Irradiance at the top of the atmosphere. Do you 
agree with this interpretation? (toa_incoming_shortwave_flux carries the 
definition '"shortwave" means shortwave radiation. "toa" means top of 
atmosphere. The TOA incoming shortwave flux is the radiative flux from the sun 
i.e. the "downwelling" TOA shortwave flux. In accordance with common usage in 
geophysical disciplines, "flux" implies per unit area, called "flux density" in 
physics.' )

Your "Total Solar Irradiance" definition below refers to top-of-atmosphere .. 
shouldn't it at "one astronomical unit from the sun"?  The wikipedia page does 
suggest measuring it at the top of the atmosphere, but the current CF 
definition for solar_irradiance implies that it should be measured at a fixed 
distance from the sun.

regards,
Martin




Hi.

I must respectfully disagree with any change to the solar irradiance
definition. The definition is accurate and correct, whether you are a
solar physicist or an oceanographer. The Wikipedia article on solar
irradiance gives good definitions for the various specific terms.

https://en.wikipedia.org/wiki/Solar_irradiance

Notice that none of the various terms ever treat the direct solar power
per unit area of a horizontal surface as a quantity unto itself. The
Vaisala definitions are essentially the same as the ones in the
Wikipedia article. The terms in use are:

  * Total Solar Irradiance - The power per unit area incident on a
surface normal to the sun from direct solar radiation measured at
top of atmosphere.
  * Direct Normal Irradiance - The power per unit area incident on a
surface normal to the sun from direct solar radiation measured at
the surface.
  * Diffuse Horizontal Irradiance - The power per unit area incident on
a horizontal surface from all sources other than direct solar radiation.
  * Global Horizontal Irradiance - The power per unit area incident on a
horizontal surface from all sources, including direct solar radiation.

I found one reference online to Direct Horizontal Irradiance. It does
not appear to be in common use in any community.

I can see defining standard names for the four terms above, but I think
that prior usage and general science usage call for solar_irradiance to
be associated only with TSI. So, we could have standard names

  * total_solar_irradiance
  * direct_normal_irradiance
  * direct_horizontal_irradiance
  * diffuse_horizontal_irradiance
  * global_horizontal_irradiance

with solar_irradiance as an alias for total_solar_irradiance, or vice
versa, but I think we shouldn't alter the definition of solar_irradiance.

Grace and peace,

Jim


On 1/26/18 11:52 AM, martin.juckes at 
stfc.ac.uk<http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> wrote:
> Dear Roy,
>
> If we were starting from scratch I would recommend the following:
>
> Solar irradiance is the power per unit area of the solar radiation received 
> from above at horizontal surface. It is distinct from the Direct Normal 
> Irradiance (which refers to radiation received by a surface perpendicular to 
> rays travelling from the sun) and the Total Solar Irradiance (which is the 
> mean value of Direct Normal Irradiance at a standard distance from the sun).
>
> I am, however, concerned that people may have already, on the basis of the 
> current definition, used the term for Total Solar Irradiance.
>
> regards,
> Martin
>
> 
> From: Lowry, Roy K. [rkl at 
> bodc.ac.uk<http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata>]
> Sent: 26 January 2018 16:03
> To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata at 
> cgd.ucar.edu<http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata>
> Subject: Re: Definiton of solar_irradiance
>
> Dear Martin,
>
>
> I agree that the current definition reflects more an astrophysics textbook 
> rather than common usage of the Standard Name and so I would support tweaking 
> the wording as you suggest. Care to come up with the replacement wording to 
> be used?
>
>
> Cheers, Roy.
>
>
> Please note that I partially retired on 01/11/2015. I am now only working 7.5 
> hours a week and can only guarantee e-mail response on Wednesdays, my day in 
> the office. All vocabulary queries should be sent to enquiries at 
> bodc

Re: [CF-metadata] Definiton of solar_irradiance

[martin.juckes]

Hello Roy, Jim,

OK, I agree with Jim that renaming "solar_irradiance" as 
"total_solar_irradiance" will remove any problems with the definition (the 
definition is fine, but "solar irradiance" has a broader meaning outside CF).

For the horizontal irradiance, there is a specific CMIP6 requirement for a 
variable which is a spatially varying solar horizontal irradiance -- but I've 
just realised that existing name "toa_incoming_shortwave_flux" might cover what 
they want (because "flux" means "flux per unit area" and radiative fluxes in CF 
are always radiative energy fluxes). Is there any difference between TOA 
incoming radiative flux (in W m-2) and horizontal irradiance that I am missing?

regards,
Martin


From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 27 January 2018 07:54
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: Definiton of solar_irradiance


For three, read four!


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: CF-metadata  on behalf of Lowry, Roy K. 

Sent: 27 January 2018 07:51
To: martin.juc...@stfc.ac.uk; cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Definiton of solar_irradiance


Dear Martin,


I think that there has been some misunderstanding on my part, pointed out by 
Jim's e-mail and there is currently no Standard Name for what is often termed 
'solar irradiance' in observation data sets.


Should we proceed with the three new Standard Names suggested by Jim?


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: martin.juc...@stfc.ac.uk 
Sent: 27 January 2018 00:05
To: Lowry, Roy K.; cf-metadata@cgd.ucar.edu
Subject: RE: Definiton of solar_irradiance

Hello Roy,

maybe .. but I was talking about the current definition of the CF standard name 
"solar_irradiance", which, currently, has nothing to do with the vertical,

regards,
Martin


From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 26 January 2018 20:35
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: Definiton of solar_irradiance


Hello Martin,


Isn't that was what cosine collectors on radiometers are for? I thought they 
resolved the vertical component of the radiation within the hemisphere sampled.


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: martin.juc...@stfc.ac.uk 
Sent: 26 January 2018 19:15
To: Lowry, Roy K.; cf-metadata@cgd.ucar.edu
Subject: RE: Definiton of solar_irradiance

Hello Roy,

but the flow of radiant energy isn't necessarily perpendicular to the surface 
of the Earth. At the top of the atmosphere the flow of energy will generally be 
along a ray from the Sun. If you want it to be the irradiance on a horizontal 
surface you really need to take out the bit about being perpendicular to the 
flow of energy.

You can't make the name work for horizontal surfaces without contradicting the 
assertion in the current definition that it applies to TSI,

regards,
Martin


From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 26 January 2018 17:46
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: Definiton of solar_irradiance


Hello again,


What we map to the 'solar_irradiance' Standard name are parameters such as 
'Downwelling vector irradiance as energy of electromagnetic radiation (solar 
wavelengths) in the atmosphere by pyranometer'. So in the use case I was 
thinking of it's both a horizontal surface and a surface perpendicular to the 
direction of flow of the radiant energy i.e. we have made an assumption the the 
direction of flow of the radiant energy is perpendicular to the Earth's surface.


Consequently, I would amend the definition as below rather than redefining from 
scratch with the risk you pointed out of changing the semantics of existing 
data sets.


"Irradiance" means the power per unit area (called radiative flux in other 
standard names), the area being normal to the direction of flow of the radiant 
energy.  The area is a horizontal surface for radiant energy 

Re: [CF-metadata] Definiton of solar_irradiance

[martin.juckes]

Hello Roy,

maybe .. but I was talking about the current definition of the CF standard name 
"solar_irradiance", which, currently, has nothing to do with the vertical,

regards,
Martin


From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 26 January 2018 20:35
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: Definiton of solar_irradiance


Hello Martin,


Isn't that was what cosine collectors on radiometers are for? I thought they 
resolved the vertical component of the radiation within the hemisphere sampled.


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: martin.juc...@stfc.ac.uk 
Sent: 26 January 2018 19:15
To: Lowry, Roy K.; cf-metadata@cgd.ucar.edu
Subject: RE: Definiton of solar_irradiance

Hello Roy,

but the flow of radiant energy isn't necessarily perpendicular to the surface 
of the Earth. At the top of the atmosphere the flow of energy will generally be 
along a ray from the Sun. If you want it to be the irradiance on a horizontal 
surface you really need to take out the bit about being perpendicular to the 
flow of energy.

You can't make the name work for horizontal surfaces without contradicting the 
assertion in the current definition that it applies to TSI,

regards,
Martin


From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 26 January 2018 17:46
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: Definiton of solar_irradiance


Hello again,


What we map to the 'solar_irradiance' Standard name are parameters such as 
'Downwelling vector irradiance as energy of electromagnetic radiation (solar 
wavelengths) in the atmosphere by pyranometer'. So in the use case I was 
thinking of it's both a horizontal surface and a surface perpendicular to the 
direction of flow of the radiant energy i.e. we have made an assumption the the 
direction of flow of the radiant energy is perpendicular to the Earth's surface.


Consequently, I would amend the definition as below rather than redefining from 
scratch with the risk you pointed out of changing the semantics of existing 
data sets.


"Irradiance" means the power per unit area (called radiative flux in other 
standard names), the area being normal to the direction of flow of the radiant 
energy.  The area is a horizontal surface for radiant energy flowing 
perpendicular to the Earth's surface.


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: martin.juc...@stfc.ac.uk 
Sent: 26 January 2018 16:52
To: Lowry, Roy K.; cf-metadata@cgd.ucar.edu
Subject: RE: Definiton of solar_irradiance

Dear Roy,

If we were starting from scratch I would recommend the following:

Solar irradiance is the power per unit area of the solar radiation received 
from above at horizontal surface. It is distinct from the Direct Normal 
Irradiance (which refers to radiation received by a surface perpendicular to 
rays travelling from the sun) and the Total Solar Irradiance (which is the mean 
value of Direct Normal Irradiance at a standard distance from the sun).

I am, however, concerned that people may have already, on the basis of the 
current definition, used the term for Total Solar Irradiance.

regards,
Martin


From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 26 January 2018 16:03
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: Definiton of solar_irradiance

Dear Martin,


I agree that the current definition reflects more an astrophysics textbook 
rather than common usage of the Standard Name and so I would support tweaking 
the wording as you suggest. Care to come up with the replacement wording to be 
used?


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: martin.juc...@stfc.ac.uk 
Sent: 26 January 2018 15:57
To: Lowry, Roy K.; cf-metadata@cgd.ucar.edu
Subject: RE: Definiton of solar_irradiance

Hello Roy,

I suspected that there might be such a usage ... but don't you agree that the 
current CF definition, which I've quoted below, is inconsistent with 

Re: [CF-metadata] Definiton of solar_irradiance

[martin.juckes]

Dear Roy,

If we were starting from scratch I would recommend the following:

Solar irradiance is the power per unit area of the solar radiation received 
from above at horizontal surface. It is distinct from the Direct Normal 
Irradiance (which refers to radiation received by a surface perpendicular to 
rays travelling from the sun) and the Total Solar Irradiance (which is the mean 
value of Direct Normal Irradiance at a standard distance from the sun).

I am, however, concerned that people may have already, on the basis of the 
current definition, used the term for Total Solar Irradiance. 

regards,
Martin


From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 26 January 2018 16:03
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: Definiton of solar_irradiance

Dear Martin,


I agree that the current definition reflects more an astrophysics textbook 
rather than common usage of the Standard Name and so I would support tweaking 
the wording as you suggest. Care to come up with the replacement wording to be 
used?


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: martin.juc...@stfc.ac.uk 
Sent: 26 January 2018 15:57
To: Lowry, Roy K.; cf-metadata@cgd.ucar.edu
Subject: RE: Definiton of solar_irradiance

Hello Roy,

I suspected that there might be such a usage ... but don't you agree that the 
current CF definition, which I've quoted below, is inconsistent with this? But 
it could be adapted with a small change of wording,

regards,
Martin


From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 26 January 2018 15:51
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: Definiton of solar_irradiance


Hello Martin,


>From an oceanographic perspective I had always thought of solar (and other 
>waveband) irradiance as the energy incident on the sea surface, which is a 
>horizontal surface. So, I would like to think of solar_irradiance as being 
>horizontal_solar_irradiance and if any new Standard Name is required to cover 
>the current Vaisala instrumentation use case then that should be 
>normal_standard_irradiance.


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: CF-metadata  on behalf of 
martin.juc...@stfc.ac.uk 
Sent: 26 January 2018 13:37
To: cf-metadata@cgd.ucar.edu
Subject: [CF-metadata] Definiton of solar_irradiance

Hello,

the cf standard name has a definition:

The quantity with standard name solar_irradiance, often called Total Solar 
Irradiance (TSI), is the radiation from the sun integrated over the whole 
electromagnetic spectrum and over the entire solar disk. The quantity applies 
outside the atmosphere, by default at a distance of one astronomical unit from 
the sun, but a coordinate or scalar coordinate variable of distance_from_sun 
can be used to specify a value other than the default. "Irradiance" means the 
power per unit area (called radiative flux in other standard names), the area 
being normal to the direction of flow of the radiant energy.

My question is about the last phrase, which I have highlighted. The flow of 
radiant energy from the sun at the top of the atmosphere is directed away from 
the sun .. so this definition would imply that the irradiance is defined 
relative to a fixed plane in the solar coordinate system. This is OK for solar 
physicists, but atmospheric scientists are sometime interested in irradiance 
relative to a horizontal surface.

Vaisalla distinguish between the two by defining "horizontal solar irradiance" 
to be the irradiance on a horizontal surface and "normal solar irradiance" to 
be irradiance on a surface perpendicular to a line to the sun 
(http://www.3tier.com/en/support/solar-online-tools/what-solar-values-are-shown-map/
 ).

Should "solar_irradiance" apply to both usages, or do we need a new standard 
name, e.g. "horizonatl_solar_irradiance"?

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
CF-metadata Info Page - mailman.cgd.ucar.edu Mailing 
Lists
mailman.cgd.ucar.edu
This is an unmoderated list for discussions about interpretation, 
clarification, and proposals for extensions or change to 

Re: [CF-metadata] Definiton of solar_irradiance

[martin.juckes]

Hello Roy,

I suspected that there might be such a usage ... but don't you agree that the 
current CF definition, which I've quoted below, is inconsistent with this? But 
it could be adapted with a small change of wording,

regards,
Martin


From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 26 January 2018 15:51
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: Definiton of solar_irradiance


Hello Martin,


>From an oceanographic perspective I had always thought of solar (and other 
>waveband) irradiance as the energy incident on the sea surface, which is a 
>horizontal surface. So, I would like to think of solar_irradiance as being 
>horizontal_solar_irradiance and if any new Standard Name is required to cover 
>the current Vaisala instrumentation use case then that should be 
>normal_standard_irradiance.


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: CF-metadata  on behalf of 
martin.juc...@stfc.ac.uk 
Sent: 26 January 2018 13:37
To: cf-metadata@cgd.ucar.edu
Subject: [CF-metadata] Definiton of solar_irradiance

Hello,

the cf standard name has a definition:

The quantity with standard name solar_irradiance, often called Total Solar 
Irradiance (TSI), is the radiation from the sun integrated over the whole 
electromagnetic spectrum and over the entire solar disk. The quantity applies 
outside the atmosphere, by default at a distance of one astronomical unit from 
the sun, but a coordinate or scalar coordinate variable of distance_from_sun 
can be used to specify a value other than the default. "Irradiance" means the 
power per unit area (called radiative flux in other standard names), the area 
being normal to the direction of flow of the radiant energy.

My question is about the last phrase, which I have highlighted. The flow of 
radiant energy from the sun at the top of the atmosphere is directed away from 
the sun .. so this definition would imply that the irradiance is defined 
relative to a fixed plane in the solar coordinate system. This is OK for solar 
physicists, but atmospheric scientists are sometime interested in irradiance 
relative to a horizontal surface.

Vaisalla distinguish between the two by defining "horizontal solar irradiance" 
to be the irradiance on a horizontal surface and "normal solar irradiance" to 
be irradiance on a surface perpendicular to a line to the sun 
(http://www.3tier.com/en/support/solar-online-tools/what-solar-values-are-shown-map/
 ).

Should "solar_irradiance" apply to both usages, or do we need a new standard 
name, e.g. "horizonatl_solar_irradiance"?

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

This message (and any attachments) is for the recipient only. NERC is subject 
to the Freedom of Information Act 2000 and the contents of this email and any 
reply you make may be disclosed by NERC unless it is exempt from release under 
the Act. Any material supplied to NERC may be stored in an electronic records 
management system.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Definiton of solar_irradiance

[martin.juckes]

Hello,

the cf standard name has a definition:

The quantity with standard name solar_irradiance, often called Total Solar 
Irradiance (TSI), is the radiation from the sun integrated over the whole 
electromagnetic spectrum and over the entire solar disk. The quantity applies 
outside the atmosphere, by default at a distance of one astronomical unit from 
the sun, but a coordinate or scalar coordinate variable of distance_from_sun 
can be used to specify a value other than the default. "Irradiance" means the 
power per unit area (called radiative flux in other standard names), the area 
being normal to the direction of flow of the radiant energy.

My question is about the last phrase, which I have highlighted. The flow of 
radiant energy from the sun at the top of the atmosphere is directed away from 
the sun .. so this definition would imply that the irradiance is defined 
relative to a fixed plane in the solar coordinate system. This is OK for solar 
physicists, but atmospheric scientists are sometime interested in irradiance 
relative to a horizontal surface.

Vaisalla distinguish between the two by defining "horizontal solar irradiance" 
to be the irradiance on a horizontal surface and "normal solar irradiance" to 
be irradiance on a surface perpendicular to a line to the sun 
(http://www.3tier.com/en/support/solar-online-tools/what-solar-values-are-shown-map/
 ).

Should "solar_irradiance" apply to both usages, or do we need a new standard 
name, e.g. "horizonatl_solar_irradiance"?

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Use of axis attribute in an auxillary coordinate

[martin.juckes]

rary scalar coordinate 
can be either, but not both at the same time. At version 1.7, section 5.7 
(http://cfconventions.org/cf-conventions/cf-conventions.html#scalar-coordinate-variables)
 now says:

  "A numeric scalar coordinate variable has the same information content and 
can be used in the same contexts as a size one numeric coordinate variable. 
Similarly, a string-valued scalar coordinate variable has the same meaning and 
purposes as a size one string-valued auxiliary coordinate variable"

Thanks,

David

On 30 July 2017 at 07:50, 
<martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk><mailto:martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk>><mailto:martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk><mailto:martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk>>><mailto:martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk><mailto:martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk>><mailto:martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk><mailto:martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk>>>>>
 wrote:
Dear All,

I'm afraid I'm not undrestanding how you are distinguishing between "auxillary" 
and other coordinates. The terminology section of the CF Convention says that a 
variable is an auxillary coordinate if it contains coordinate data and is not 
an NUG coordinate. The definition of a scalar coordinate states that a scalar 
coordinate can be either an auxillary coordinate or a coordinate variable .. so 
the "height" variable here is both a scalar coordinate and an auxillary 
coordinate. It is clearly not a NUG coordinate variable.


regards,
Martin



David,

I'm still not convinced of the utility of axis for coordinate variables
that aren't true coordinate variables, but this case doesn't fit that
one, does it? In this case isn't height a true (scalar) coordinate
variable? Shouldn't this pass the checker, regardless?

Jim


On 5/24/17 3:54 PM, David Hassell wrote:
> Hi Jim, Martin,
>
> I agree - "height" in this case is not an auxiliary coordinate
> variable, rather a scalar coordinate variable (because it doesn't span
> any of the dimensions of "tas").
>
> I also agree that the conformance document needs changing to allow the
> "axis" attribute on auxiliary coordinate variables - this was accepted
> in CF-1.6, I think.
>
> Thanks,
>
> Daivd
>
>
>
> On 24 May 2017 at 19:59, Jim Biard  cicsnc.org<http://cicsnc.org><http://cicsnc.org><http://cicsnc.org><http://cicsnc.org>
> <mailto:jbiard<mailto:jbiard><mailto:jbiard<mailto:jbiard>><mailto:jbiard<mailto:jbiard><mailto:jbiard<mailto:jbiard>>><mailto:jbiard<mailto:jbiard><mailto:jbiard<mailto:jbiard>><mailto:jbiard<mailto:jbiard><mailto:jbiard<mailto:jbiard>>>>
>  at 
> cicsnc.org<http://cicsnc.org><http://cicsnc.org><http://cicsnc.org><http://cicsnc.org>>>
>  wrote:
>
> Martin,
>
> We just had some discussion about the proper use of the axis
> attribute, but this seems to me like it might be a flaw in the
> checker. As a scalar coordinate, height can only be associated
> with the tas variable via the coordinates attribute (per section
> 5.7), but I don't think that makes it an auxiliary coordinate,
> does it?
>
> What do other people think? Chime in!
>
> Grace and peace,
>
> Jim
>
>
> On 5/23/17 10:50 AM, martin.juckes at 
> stfc.ac.uk<http://stfc.ac.uk><http://stfc.ac.uk><http://stfc.ac.uk><http://stfc.ac.uk>
> 
> <mailto:martin.juckes<mailto:martin.juckes><mailto:martin.juckes<mailto:martin.juckes>><mailto:martin.juckes<mailto:martin.juckes><mailto:martin.juckes<mailto:martin.juckes>>><mailto:martin.juckes<mailto:martin.juckes><mailto:martin.juckes<mailto:martin.juckes>><mailto:martin.juckes<mailto:martin.juckes><mailto:martin.juckes<mailto:martin.juckes>>>>
>  at 
> stfc.ac.uk<http://stfc.ac.uk><http://stfc.ac.uk><http://stfc.ac.uk><http://stfc.ac.uk>>
>  wrote:
>> Hello All,
>>
>> I'd just like to check one aspect of the conformanc document, which came 
>> to our attention when somebody ran the CF checker on some CMIP5. If you 
>> check using the convention version declared in the file, 1.4, it will raise 
>> an error if there is a scalar coordinate variable with the axis attribute 
>> set, e.g.
>>
&

Re: [CF-metadata] Use of axis attribute in an auxillary coordinate

[martin.juckes]

David,

I'm still not convinced of the utility of axis for coordinate variables
that aren't true coordinate variables, but this case doesn't fit that
one, does it? In this case isn't height a true (scalar) coordinate
variable? Shouldn't this pass the checker, regardless?

Jim


On 5/24/17 3:54 PM, David Hassell wrote:
> Hi Jim, Martin,
>
> I agree - "height" in this case is not an auxiliary coordinate
> variable, rather a scalar coordinate variable (because it doesn't span
> any of the dimensions of "tas").
>
> I also agree that the conformance document needs changing to allow the
> "axis" attribute on auxiliary coordinate variables - this was accepted
> in CF-1.6, I think.
>
> Thanks,
>
> Daivd
>
>
>
> On 24 May 2017 at 19:59, Jim Biard  cicsnc.org<http://cicsnc.org><http://cicsnc.org><http://cicsnc.org>
> <mailto:jbiard<mailto:jbiard><mailto:jbiard<mailto:jbiard>><mailto:jbiard<mailto:jbiard><mailto:jbiard<mailto:jbiard>>>
>  at cicsnc.org<http://cicsnc.org><http://cicsnc.org><http://cicsnc.org>>> 
> wrote:
>
> Martin,
>
> We just had some discussion about the proper use of the axis
> attribute, but this seems to me like it might be a flaw in the
> checker. As a scalar coordinate, height can only be associated
> with the tas variable via the coordinates attribute (per section
> 5.7), but I don't think that makes it an auxiliary coordinate,
> does it?
>
> What do other people think? Chime in!
>
> Grace and peace,
>
> Jim
>
>
> On 5/23/17 10:50 AM, martin.juckes at 
> stfc.ac.uk<http://stfc.ac.uk><http://stfc.ac.uk><http://stfc.ac.uk>
> 
> <mailto:martin.juckes<mailto:martin.juckes><mailto:martin.juckes<mailto:martin.juckes>><mailto:martin.juckes<mailto:martin.juckes><mailto:martin.juckes<mailto:martin.juckes>>>
>  at stfc.ac.uk<http://stfc.ac.uk><http://stfc.ac.uk><http://stfc.ac.uk>> 
> wrote:
>> Hello All,
>>
>> I'd just like to check one aspect of the conformanc document, which came 
>> to our attention when somebody ran the CF checker on some CMIP5. If you 
>> check using the convention version declared in the file, 1.4, it will raise 
>> an error if there is a scalar coordinate variable with the axis attribute 
>> set, e.g.
>>
>> float tas(time,lat,lon);
>>  ..
>>  tas:coordinates = "height" ;
>> float height ;
>>  
>>  height: axis = "Z";
>>
>> In this case "height" variable is, following the logic of section 1.2 of 
>> the convention, classed as an auxillary coordinate because it is not of the 
>> form "height (height) ; ".
>>
>> The error message appears to relate to a line in the conformance 
>> document saying that "The axis attribute is not allowed for auxiliary 
>> coordinate variables." If the checker is asked to use a later version of the 
>> convention, the error message goes away, but the requirement is still there 
>> in the conformance document.
>>
>> It looks to me as though it should be removed from the conformance 
>> document. The convention document says, in section 4. that "The methods of 
>> identifying coordinate types described in this section apply both to 
>> coordinate variables", referring to the use of the axis attribute, which 
>> appears to directly contradict the line of the conformance document cited 
>> above. But is there another part of the convention that requires some 
>> restriction on the use of the axis attribute?
>>
>> This construction is widely used in CMIP data, so we should get this 
>> point cleared up.
>>
>> regards,
>> Martin
>>
>> ___
>> CF-metadata mailing list
>> CF-metadata at 
>> cgd.ucar.edu<http://cgd.ucar.edu><http://cgd.ucar.edu><http://cgd.ucar.edu> 
>> <mailto:CF-metadata<mailto:CF-metadata><mailto:CF-metadata<mailto:CF-metadata>><mailto:CF-metadata<mailto:CF-metadata><mailto:CF-metadata<mailto:CF-metadata>>>
>>  at 
>> cgd.ucar.edu<http://cgd.ucar.edu><http://cgd.ucar.edu><http://cgd.ucar.edu>>
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata>
> --
> CICS-NC <http://www.cicsnc.org/> 

Re: [CF-metadata] Use of axis attribute in an auxillary coordinate

[martin.juckes]

Hi David,

But it is not a NUG coordinate variable has, by definition, the same name as a 
dimension (see 
http://www.unidata.ucar.edu/software/netcdf/docs/netcdf_data_set_components.html#coordinate_variables
 ) ... so height, in the example below, is not a NUG coordinate variable. It is 
an auxiliary coordinate variable, if you follow the definitions we have in the 
CF convention.

Martin

From: David Hassell [david.hass...@ncas.ac.uk]
Sent: 31 July 2017 09:33
To: Juckes, Martin (STFC,RAL,RALSP)
Cc: CF Metadata
Subject: Re: [CF-metadata] Use of axis attribute in an auxillary coordinate

Hi Martin,

Because it's numeric, I would say that it acting as a coordinate variable in 
the NUG sense.

All the best,

Daivd

On 31 July 2017 at 09:18, 
<martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk>> wrote:
Hello David,

yes, so "height" in the example I gave is clearly an auxiliary coordinate, 
right?

cheers,
Martin


From: David Hassell [david.hass...@ncas.ac.uk<mailto:david.hass...@ncas.ac.uk>]
Sent: 31 July 2017 09:03
To: Juckes, Martin (STFC,RAL,RALSP)
Cc: CF Metadata
Subject: Re: [CF-metadata] Use of axis attribute in an auxillary coordinate

Hello Martin,

This definition was tightened up in ticket 
#104<https://cf-trac.llnl.gov/trac/ticket/104>. An arbitrary scalar coordinate 
can be either, but not both at the same time. At version 1.7, section 5.7 
(http://cfconventions.org/cf-conventions/cf-conventions.html#scalar-coordinate-variables)
 now says:

  "A numeric scalar coordinate variable has the same information content and 
can be used in the same contexts as a size one numeric coordinate variable. 
Similarly, a string-valued scalar coordinate variable has the same meaning and 
purposes as a size one string-valued auxiliary coordinate variable"

Thanks,

David

On 30 July 2017 at 07:50, 
<martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk><mailto:martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk>>>
 wrote:
Dear All,

I'm afraid I'm not undrestanding how you are distinguishing between "auxillary" 
and other coordinates. The terminology section of the CF Convention says that a 
variable is an auxillary coordinate if it contains coordinate data and is not 
an NUG coordinate. The definition of a scalar coordinate states that a scalar 
coordinate can be either an auxillary coordinate or a coordinate variable .. so 
the "height" variable here is both a scalar coordinate and an auxillary 
coordinate. It is clearly not a NUG coordinate variable.


regards,
Martin



David,

I'm still not convinced of the utility of axis for coordinate variables
that aren't true coordinate variables, but this case doesn't fit that
one, does it? In this case isn't height a true (scalar) coordinate
variable? Shouldn't this pass the checker, regardless?

Jim


On 5/24/17 3:54 PM, David Hassell wrote:
> Hi Jim, Martin,
>
> I agree - "height" in this case is not an auxiliary coordinate
> variable, rather a scalar coordinate variable (because it doesn't span
> any of the dimensions of "tas").
>
> I also agree that the conformance document needs changing to allow the
> "axis" attribute on auxiliary coordinate variables - this was accepted
> in CF-1.6, I think.
>
> Thanks,
>
> Daivd
>
>
>
> On 24 May 2017 at 19:59, Jim Biard  cicsnc.org<http://cicsnc.org><http://cicsnc.org>
> <mailto:jbiard<mailto:jbiard><mailto:jbiard<mailto:jbiard>> at 
> cicsnc.org<http://cicsnc.org><http://cicsnc.org>>> wrote:
>
> Martin,
>
> We just had some discussion about the proper use of the axis
> attribute, but this seems to me like it might be a flaw in the
> checker. As a scalar coordinate, height can only be associated
> with the tas variable via the coordinates attribute (per section
> 5.7), but I don't think that makes it an auxiliary coordinate,
> does it?
>
> What do other people think? Chime in!
>
> Grace and peace,
>
> Jim
>
>
> On 5/23/17 10:50 AM, martin.juckes at 
> stfc.ac.uk<http://stfc.ac.uk><http://stfc.ac.uk>
> 
> <mailto:martin.juckes<mailto:martin.juckes><mailto:martin.juckes<mailto:martin.juckes>>
>  at stfc.ac.uk<http://stfc.ac.uk><http://stfc.ac.uk>> wrote:
>> Hello All,
>>
>> I'd just like to check one aspect of the conformanc document, which came 
>> to our attention when somebody ran the CF checker on some CMIP5. If you 
>> check using the convention version declared in the file, 1.4, it will rais

Re: [CF-metadata] Use of axis attribute in an auxillary coordinate

[martin.juckes]

Hello David,

yes, so "height" in the example I gave is clearly an auxiliary coordinate, 
right?

cheers,
Martin


From: David Hassell [david.hass...@ncas.ac.uk]
Sent: 31 July 2017 09:03
To: Juckes, Martin (STFC,RAL,RALSP)
Cc: CF Metadata
Subject: Re: [CF-metadata] Use of axis attribute in an auxillary coordinate

Hello Martin,

This definition was tightened up in ticket 
#104<https://cf-trac.llnl.gov/trac/ticket/104>. An arbitrary scalar coordinate 
can be either, but not both at the same time. At version 1.7, section 5.7 
(http://cfconventions.org/cf-conventions/cf-conventions.html#scalar-coordinate-variables)
 now says:

  "A numeric scalar coordinate variable has the same information content and 
can be used in the same contexts as a size one numeric coordinate variable. 
Similarly, a string-valued scalar coordinate variable has the same meaning and 
purposes as a size one string-valued auxiliary coordinate variable"

Thanks,

David

On 30 July 2017 at 07:50, 
<martin.juc...@stfc.ac.uk<mailto:martin.juc...@stfc.ac.uk>> wrote:
Dear All,

I'm afraid I'm not undrestanding how you are distinguishing between "auxillary" 
and other coordinates. The terminology section of the CF Convention says that a 
variable is an auxillary coordinate if it contains coordinate data and is not 
an NUG coordinate. The definition of a scalar coordinate states that a scalar 
coordinate can be either an auxillary coordinate or a coordinate variable .. so 
the "height" variable here is both a scalar coordinate and an auxillary 
coordinate. It is clearly not a NUG coordinate variable.


regards,
Martin



David,

I'm still not convinced of the utility of axis for coordinate variables
that aren't true coordinate variables, but this case doesn't fit that
one, does it? In this case isn't height a true (scalar) coordinate
variable? Shouldn't this pass the checker, regardless?

Jim


On 5/24/17 3:54 PM, David Hassell wrote:
> Hi Jim, Martin,
>
> I agree - "height" in this case is not an auxiliary coordinate
> variable, rather a scalar coordinate variable (because it doesn't span
> any of the dimensions of "tas").
>
> I also agree that the conformance document needs changing to allow the
> "axis" attribute on auxiliary coordinate variables - this was accepted
> in CF-1.6, I think.
>
> Thanks,
>
> Daivd
>
>
>
> On 24 May 2017 at 19:59, Jim Biard http://cicsnc.org>
> <mailto:jbiard<mailto:jbiard> at cicsnc.org<http://cicsnc.org>>> wrote:
>
> Martin,
>
> We just had some discussion about the proper use of the axis
> attribute, but this seems to me like it might be a flaw in the
> checker. As a scalar coordinate, height can only be associated
> with the tas variable via the coordinates attribute (per section
> 5.7), but I don't think that makes it an auxiliary coordinate,
> does it?
>
> What do other people think? Chime in!
>
> Grace and peace,
>
> Jim
>
>
> On 5/23/17 10:50 AM, martin.juckes at stfc.ac.uk<http://stfc.ac.uk>
> <mailto:martin.juckes<mailto:martin.juckes> at 
> stfc.ac.uk<http://stfc.ac.uk>> wrote:
>> Hello All,
>>
>> I'd just like to check one aspect of the conformanc document, which came 
>> to our attention when somebody ran the CF checker on some CMIP5. If you 
>> check using the convention version declared in the file, 1.4, it will raise 
>> an error if there is a scalar coordinate variable with the axis attribute 
>> set, e.g.
>>
>> float tas(time,lat,lon);
>>  ..
>>  tas:coordinates = "height" ;
>> float height ;
>>  
>>  height: axis = "Z";
>>
>> In this case "height" variable is, following the logic of section 1.2 of 
>> the convention, classed as an auxillary coordinate because it is not of the 
>> form "height (height) ; ".
>>
>> The error message appears to relate to a line in the conformance 
>> document saying that "The axis attribute is not allowed for auxiliary 
>> coordinate variables." If the checker is asked to use a later version of the 
>> convention, the error message goes away, but the requirement is still there 
>> in the conformance document.
>>
>> It looks to me as though it should be removed from the conformance 
>> document. The convention document says, in section 4. that "The methods of 
>> identify

[CF-metadata] Use of axis attribute in an auxillary coordinate

[martin.juckes]

Dear All,

I'm afraid I'm not undrestanding how you are distinguishing between "auxillary" 
and other coordinates. The terminology section of the CF Convention says that a 
variable is an auxillary coordinate if it contains coordinate data and is not 
an NUG coordinate. The definition of a scalar coordinate states that a scalar 
coordinate can be either an auxillary coordinate or a coordinate variable .. so 
the "height" variable here is both a scalar coordinate and an auxillary 
coordinate. It is clearly not a NUG coordinate variable.


regards,
Martin



David,

I'm still not convinced of the utility of axis for coordinate variables 
that aren't true coordinate variables, but this case doesn't fit that 
one, does it? In this case isn't height a true (scalar) coordinate 
variable? Shouldn't this pass the checker, regardless?

Jim


On 5/24/17 3:54 PM, David Hassell wrote:
> Hi Jim, Martin,
>
> I agree - "height" in this case is not an auxiliary coordinate 
> variable, rather a scalar coordinate variable (because it doesn't span 
> any of the dimensions of "tas").
>
> I also agree that the conformance document needs changing to allow the 
> "axis" attribute on auxiliary coordinate variables - this was accepted 
> in CF-1.6, I think.
>
> Thanks,
>
> Daivd
>
>
>
> On 24 May 2017 at 19:59, Jim Biard  <mailto:jbiard at cicsnc.org>> wrote:
>
> Martin,
>
> We just had some discussion about the proper use of the axis
> attribute, but this seems to me like it might be a flaw in the
> checker. As a scalar coordinate, height can only be associated
> with the tas variable via the coordinates attribute (per section
> 5.7), but I don't think that makes it an auxiliary coordinate,
> does it?
>
> What do other people think? Chime in!
>
> Grace and peace,
>
> Jim
>
>
> On 5/23/17 10:50 AM, martin.juckes at stfc.ac.uk
> <mailto:martin.juckes at stfc.ac.uk> wrote:
>> Hello All,
>>
>> I'd just like to check one aspect of the conformanc document, which came 
>> to our attention when somebody ran the CF checker on some CMIP5. If you 
>> check using the convention version declared in the file, 1.4, it will raise 
>> an error if there is a scalar coordinate variable with the axis attribute 
>> set, e.g.
>>
>> float tas(time,lat,lon);
>>  ..
>>  tas:coordinates = "height" ;
>> float height ;
>>  
>>  height: axis = "Z";
>>
>> In this case "height" variable is, following the logic of section 1.2 of 
>> the convention, classed as an auxillary coordinate because it is not of the 
>> form "height (height) ; ".
>>
>> The error message appears to relate to a line in the conformance 
>> document saying that "The axis attribute is not allowed for auxiliary 
>> coordinate variables." If the checker is asked to use a later version of the 
>> convention, the error message goes away, but the requirement is still there 
>> in the conformance document.
>>
>> It looks to me as though it should be removed from the conformance 
>> document. The convention document says, in section 4. that "The methods of 
>> identifying coordinate types described in this section apply both to 
>> coordinate variables", referring to the use of the axis attribute, which 
>> appears to directly contradict the line of the conformance document cited 
>> above. But is there another part of the convention that requires some 
>> restriction on the use of the axis attribute?
>>
>> This construction is widely used in CMIP data, so we should get this 
>> point cleared up.
>>
>> regards,
>> Martin
>>
>> ___
>> CF-metadata mailing list
>> CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>> <http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata>
> -- 
> CICS-NC <http://www.cicsnc.org/> Visit us on Facebook
> <http://www.facebook.com/cicsnc>  *Jim Biard* *Research Scholar*
> Cooperative Institute for Climate and Satellites NC
> <http://cicsnc.org/> North Carolina State University
> <http://ncsu.edu/> NOAA National Centers for Environmental
> Information <ht

[CF-metadata] Another CF complaince checker -- from IOOS --- with some issues

[martin.juckes]

Dear All,

Jonathan makes a fair point that there is no such thing as certified CF 
compliance.

On sample files with interesting errors, the CMIP5 archive provides an 
interesting range of examples. In the process of trying to provide constructive 
feedback to the IOOS team I shrank  (by reducing lat, long and time axis 
lengths) and partially annonymised (by overwriting institute, contact and model 
names) 6 CMIP5 files (  
https://drive.google.com/open?id=0B6AWgiTrQ0xFdmdpczZwQjNMRUU ) . Documenting 
them sufficiently to be useful to users would be more work .. and its not clear 
how best to document them. Putting extended comments in the NetCDF files would 
be possible, but it might be easier to link to a web page with information 
(e.g. http://cfconventions.org/samplefile/notes/ex003.html) explaining the 
example and options. It would be possible to add a list of errors and warnings 
in a global attribute(s).

Perhaps it should be considered standard practise to create a sample file 
implementing each new feature introduced and one or two versions with errors 
which highlight concerns raised in dicussions. Once correct example has been 
ceated, editing one or two attributes to create and invalid version would not 
be hard,

regards,
Martin



Agree with the overall thread. Having well-documented examples in either sense 
is a great step.

The examples on the NOAA site (sorry, imprecise reference without my computer 
handy) showing how to structure and document different kinds of marine 
observations might be a good start for such a collection.

Crowd-sourcing other submissions might be a nice way to build a very useful 
resource.

John

> On Jul 21, 2017, at 14:49, Chris Barker  noaa.gov> wrote:
>
>> On Thu, Jul 20, 2017 at 6:45 AM, Jonathan Gregory > reading.ac.uk> 
>> wrote:
>> perhaps it would be possible for
>> us to maintain a netCDF file containing an example of every possible
>> violation of a requirement or recommendation in the CF conformance document,
>
> Without thinking it through, I suspect that it is literally impossible to put 
> it all in one file -- aren't there examples of "do it this way OR that way", 
> and "fi this, then do that", that you couldn't have data that both does and 
> doesn't conform in one file.
>
> So a set of files would be the way to go -- and would be easier to manage as 
> well.
>
> Also, expecting that we could have a COMPLETE test set is a bit of a fantasy.
>
> However, a most-cases test set of files would be really useful.
>
> I think the "trick" would be to establish a data structure that maps a 
> particular file with a set of what parts of CF are violated, and what 
> recommendations are violated.
>
> The CF-checkers could then check what they find against that data. And it 
> would be easy to simply update a file and/or drop a new one into the set and 
> the checker would update its tests.
>
> A glance at the Python cf-checker code looks like there's a lot of test fies 
> already in there,
>
> https://github.com/cedadev/cf-checker/tree/master/test_files
>
> though it looks to me (at a really quick glance) like the accompanying 
> "check" files are output from cf-checker -- so maybe not a good way to 
> specify what cf-checker SHOULD return.
>
> And if anyone wants to write a cf-checker in another language, it would be 
> nice to have the "errors" specified in a simple to read file format (JSON?)
>
>> and maybe an example of everything which is described as legal e.g. all the
>> actual examples in the conventions document, and some more.
>
> again, hard to put in one file :-) -- but yes, a good set of "perfect" files 
> would be good, too.
>
> -CHB
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> Chris.Barker at 
> noaa.gov
> ___
> CF-metadata mailing list
> CF-metadata at 
> cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
-- next part --
An HTML attachment was scrubbed...
URL: 

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Sign convention of upwelling and downwelling fluxes

[martin.juckes]

Hello All,

there are standard names for a number of upwelling and downwelling fluxes, 
defined clearly as the component of radiation which is travelling upwards or 
downwards. The sign convention of these fluxes is not specified .. CMIP uses 
the convention that downwelling is positive down and upwelling is positive 
upwards, but in the atmospheric literature you can find people using the 
convention that both are positive upwards. Both approaches make sense, but it 
leaves us with some ambiguity in the standard names.

Since 2008 there are a few terms like 
minus_one_times_surface_upwelling_longwave_flux_in_air which are defined as 
having the opposite sign convention to terms without "minus_one_times_". But 
surface_upwelling_longwave_flux_in_air has no explicit sign convention. The 
inclusion of the minus_one_times terms implies that at least some people 
thought that the existing standard names should have a specific sign convention 
... and I hope, if that is the case, that it can be consistent with CMIP. If 
so, can we clear up the ambiguity by adding a line in the description of these 
standard names to say that the sign convention is positive upwards/downwards 
for upwelling/downwelling fluxes?

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Another CF complaince checker -- from IOOS --- with some issues

[martin.juckes]

Hello All,

Following discussions with a colleague (As Stephens) I've taken a look at the 
IOOS compliance-checker, which contains a module for checking how files comply 
with the CF convention. I looked at 4 files with known CF errors, and found an 
average of two erroneous reports per file (listed in an issue which I raised on 
their github site: https://github.com/ioos/compliance-checker/issues/501 ). 
There are also ambiguities arising from the fact that they use priority 1 (low) 
to 3 (high) rather than INFO, WARN, ERROR -- but I haven't gone into all of 
these in the issue raised.

I'm raising it here as well because I'd like to hear other views on the broader 
question of community tools associated with CF. It is good that people are 
getting engaged and working through the details of the convention, not so good 
if they produce and spread misleading information. In its current state, I 
don't think the IOOS compliance checker is one we would want to approve, but if 
they fix the 8 problems identified from a morning looking through the results 
from tests on 4 files, does that make it OK? or, since the 8 problems I've 
raised come from looking at a small set of files, should we assume that there 
are many other problems and ask them to do more?

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Is "psu" a valid cf unit?

[martin.juckes]

Hello All,

This appears to have stirred up quite a lot.

I think Balaji is raising a point that has not been picked up: if we have a 
physical quantity which varies in a small range around 35, then reporting the 
full value (e.g. 35.346) is going to give less precision than reporting an 
anomaly (e.g 0.34689). I think this issue of precision can be covered by the 
use of scale_factor and add_offset as described in section 8.1 of the CF 
Convention, which states that the units declared in the units attribute should 
be for the "unpacked" data .. i.e. the data values after the scaling and offset 
have been inverted to get back the full value. There is a possible confusion 
when the user community actually want to use the scaled and offset version in 
their applications, rather than treating it as a purely technical thing going 
on inside the NetCDF library. In CMIP5, models reported near surface salinity 
with values in the range 0 to 66, so it doesn't look as if there is a case for 
scaling there. Some models reported values a factor of 1000 smaller, so there 
is a need for clearer communication -- but I don't think that needs to be 
addressed by CF: for the CMIP6 data request I can address this by setting an 
"ok_min_mean_abs" value (a CMIP term ... with no CF meaning) of 28.

I'm afraid I don't understand Paul's point. It looks as though the idea of psu 
as salinity units has been discussed by the CF list, the UNIDATA issue tracker, 
and various oceanography standards bodies and there is a broad and clear 
consensus across all these places that psu should not be treated as a unit of 
measure.

regards,
Martin


From: Durack, Paul J. [dura...@llnl.gov]
Sent: 18 July 2017 23:38
To: Pamment, Alison (STFC,RAL,RALSP)
Cc: cf-metadata@cgd.ucar.edu; Juckes, Martin (STFC,RAL,RALSP); 
v.bal...@noaa.gov; r...@bodc.ac.uk; Stephen Griffies - NOAA Federal; Gokhan 
Danabasoglu; simon.marsl...@csiro.au
Subject: Re: [CF-metadata] Is "psu" a valid cf unit?

Hi folks, I’ll chime in here.

There are two issues, labeling salinity in models (which is the focus of much 
of this thread) and observed quantities, which should both be considered when 
standardizing the standard_name definitions and units.

For a trip down memory lane, “salinity” has been discussed repeatedly in the 
CF-list since back to at least 2009 (see 
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2009/047515.html). In 2011, I 
proposed a number of new names that brought the CF names up-to-date with 
TEOS-10 (see 
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2011/055094.html). For those 
interested, search for “salinity” on 
http://cfconventions.org/Data/cf-standard-names/45/build/cf-standard-name-table.html,
 with a particular focus on the names “sea_surface_salinity”, 
“sea_water_absolute_salinity”, “sea_water_cox_salinity”, 
“sea_water_knudsen_salinity” and “sea_water_practical_salinity” and take a look 
at the descriptions for each. The reasons all these names were included was 
more for history than utility, as the evolution of observed “salinity” needs to 
be captured somewhere. It also provided all the required salinity “boxes” in 
the case that older data were digitized.

The “state of the art” oceanographic guidelines (which are very 
observationally-focused) are documented in the TEOS-10 manual (for interested 
folks the description of Absolute Salinity is a good place to start: 
http://teos-10.org/pubs/Getting_Started.pdf#page=7). In this document, the 
summary (spoiler alert) is to maintain consistency with historical databases, 
observed salinity will continue to be measured (using conductivity) on the 
Practical Salinity Scale 1978 (abbreviated to PSS-78). The “unit” PSU is 
strongly encouraged to be dropped (as Martin pointed out below with the 
Millero, 1993 ref), this is also repeated in the TEOS-10 supporting 
documentation. I note that PSS-78, based on conductivity measurements has a 
valid range of 2 to 42.

For models, things are considerably simpler. No model (that I know of) has 
implemented Absolute Salinity, as to do so it would need to consider many more 
tracer fields that are not particularly well observationally constrained, and 
Absolute Salinity is affected by uncertain chemical and biological processes. A 
point to note here is that all ocean models are originally initialized from an 
observational estimate, and most of these are related somehow to one or more 
versions of the World Ocean Atlas (1994 to 2013v2), which have always been 
stored as a “unitless” quantity on the PSS-78 (with values ranging from 5 to 
40). I would also note that most CMIP5 contributing models provided data in the 
range 2-42 (with units of 1e-3, the “unit” prior to changes in 2015), however 
one modeling center took this “unit” literally and provided scaled data in the 
range 0.002-0.042. I do not believe that there are any range limitations for 
modeled salinity, and have seen values in excess of 50 in 

Re: [CF-metadata] Is "psu" a valid cf unit?

[martin.juckes]

Hello All,

After some hunting, I found a copy of a letter from Frank Millero (1993 - "What 
is PSU") ( http://www.marbef.org/wiki/Salinity_sensors  -- box on right) which 
is adamant that PSU is not a unit and the "Joint Panel on Oceanographic Tables 
and Standards" has made this clear in their definition of the international 
equation of state.

There is perhaps a parallel here with "Decibel", which is accepted by cf-python 
and rejected by Udunits. There has been discussion on unidata pages, the 
outcome of which, I think, is that Decibel is a reference to a methodolgy used 
to arrive at a non-dimensional number, not a unit of measure.

The CF Standard Name list now has distinct standard names for salinities 
defined by different algorithms (e.g. sea_water_practical_salinity, 
sea_water_cox_salinity).

One compromise might be to add some recognised standard methodological keywords 
and define a way of placing them in NetCDF files,

regards,
Martin

From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 18 July 2017 14:36
To: V. Balaji - NOAA Affiliate
Cc: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu; 
d.c.hass...@reading.ac.uk; r.s.hatc...@reading.ac.uk
Subject: Re: [CF-metadata] Is "psu" a valid cf unit?


Dear Balaji,


I think there are some crossed wires here. The dimensionless Practical Salinity 
and a Practical Salinity in PSU are exact numeric equivalents. The only 
difference is the name that's given to the unit of measure - that's why to make 
this crystal clear CF includes the scaling factor of 10^-3. So I don't think 
that this can affect storage precision.


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: V. Balaji - NOAA Affiliate 
Sent: 18 July 2017 14:12
To: Lowry, Roy K.
Cc: martin.juc...@stfc.ac.uk; cf-metadata@cgd.ucar.edu; 
d.c.hass...@reading.ac.uk; r.s.hatc...@reading.ac.uk
Subject: Re: [CF-metadata] Is "psu" a valid cf unit?

The ocean modeling community is adamant that they will continue to use PSUs for 
salinity: it's this unit, rather than its SI or dimensionless equivalent, that 
gives the maximum digits of precision in a floating-point representation of 
salinity.

It's a valid concern, and CF should perhaps reconsider, with a new discussion 
giving weight to the views of modelers as well as physical oceanographers.

Thanks,

Lowry, Roy K. writes:

> Hello Martin,
>
>
> This topic has been debated at length in CF. To cut a long story short, the 
> term 'Practical Salinity Unit' was coined when the 1978 Practical Salinity 
> scale was devised. However, the term fell out of favour with the physical 
> oceanographic community whose current recommended practice is that Practical 
> Salinity - a ratio - should be a dimensionless number.  CF followed this 
> recommendation and so PSU is not a part of CF.
>
>
> Have a dig around in the mailing list archive if you want to find out more.
>
>
> Cheers, Roy.
>
>
> Please note that I partially retired on 01/11/2015. I am now only working 7.5 
> hours a week and can only guarantee e-mail response on Wednesdays, my day in 
> the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
> Please also use this e-mail if your requirement is urgent.
>
>
> 
> From: CF-metadata  on behalf of 
> martin.juc...@stfc.ac.uk 
> Sent: 18 July 2017 13:43
> To: cf-metadata@cgd.ucar.edu; d.c.hass...@reading.ac.uk; 
> r.s.hatc...@reading.ac.uk
> Subject: [CF-metadata] Is "psu" a valid cf unit?
>
> Hello David, all,
>
> Is "psu" a valus CF unit? It is not in Udunits, but it is added in cf-python 
> as a unit alias and also appears to be accpeted by the cf-checker. I can't 
> see any mention of it in the CF Convention document: the latter only lists 
> level, layer, and sigma_level permitted departures from Udunits,
>
> regards,
> Martin
>
> ___
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
CF-metadata Info Page - University Corporation for 
...
mailman.cgd.ucar.edu
This is an unmoderated list for discussions about interpretation, 
clarification, and proposals for extensions or change to the CF conventions.



> CF-metadata Info Page - University Corporation for 
> ...
CF-metadata Info Page - University Corporation for 
...
mailman.cgd.ucar.edu
This is an unmoderated list for discussions about interpretation, 
clarification, and 

Re: [CF-metadata] Is "psu" a valid cf unit?

[martin.juckes]

Thanks Roy,

David, Ros: do you accept Roy's answer? If so, cf-python and the cf-checker 
should presumably be updated to flag "psu" as an invalid unit?

regards,
Martin


From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 18 July 2017 14:00
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu; 
d.c.hass...@reading.ac.uk; r.s.hatc...@reading.ac.uk
Subject: Re: Is "psu" a valid cf unit?


Hello Martin,


This topic has been debated at length in CF. To cut a long story short, the 
term 'Practical Salinity Unit' was coined when the 1978 Practical Salinity 
scale was devised. However, the term fell out of favour with the physical 
oceanographic community whose current recommended practice is that Practical 
Salinity - a ratio - should be a dimensionless number.  CF followed this 
recommendation and so PSU is not a part of CF.


Have a dig around in the mailing list archive if you want to find out more.


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: CF-metadata  on behalf of 
martin.juc...@stfc.ac.uk 
Sent: 18 July 2017 13:43
To: cf-metadata@cgd.ucar.edu; d.c.hass...@reading.ac.uk; 
r.s.hatc...@reading.ac.uk
Subject: [CF-metadata] Is "psu" a valid cf unit?

Hello David, all,

Is "psu" a valus CF unit? It is not in Udunits, but it is added in cf-python as 
a unit alias and also appears to be accpeted by the cf-checker. I can't see any 
mention of it in the CF Convention document: the latter only lists level, 
layer, and sigma_level permitted departures from Udunits,

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
CF-metadata Info Page - University Corporation for 
...
mailman.cgd.ucar.edu
This is an unmoderated list for discussions about interpretation, 
clarification, and proposals for extensions or change to the CF conventions.




This message (and any attachments) is for the recipient only. NERC is subject 
to the Freedom of Information Act 2000 and the contents of this email and any 
reply you make may be disclosed by NERC unless it is exempt from release under 
the Act. Any material supplied to NERC may be stored in an electronic records 
management system.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Is "psu" a valid cf unit?

[martin.juckes]

Hello David, all,

Is "psu" a valus CF unit? It is not in Udunits, but it is added in cf-python as 
a unit alias and also appears to be accpeted by the cf-checker. I can't see any 
mention of it in the CF Convention document: the latter only lists level, 
layer, and sigma_level permitted departures from Udunits,

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Use of axis attribute in an auxillary coordinate

[martin.juckes]

Hello All,

I'd just like to check one aspect of the conformanc document, which came to our 
attention when somebody ran the CF checker on some CMIP5. If you check using 
the convention version declared in the file, 1.4, it will raise an error if 
there is a scalar coordinate variable with the axis attribute set, e.g.

float tas(time,lat,lon);
..
tas:coordinates = "height" ;
float height ;

height: axis = "Z";

In this case "height" variable is, following the logic of section 1.2 of the 
convention, classed as an auxillary coordinate because it is not of the form 
"height (height) ; ".

The error message appears to relate to a line in the conformance document 
saying that "The axis attribute is not allowed for auxiliary coordinate 
variables." If the checker is asked to use a later version of the convention, 
the error message goes away, but the requirement is still there in the 
conformance document.

It looks to me as though it should be removed from the conformance document. 
The convention document says, in section 4. that "The methods of identifying 
coordinate types described in this section apply both to coordinate variables", 
referring to the use of the axis attribute, which appears to directly 
contradict the line of the conformance document cited above. But is there 
another part of the convention that requires some restriction on the use of the 
axis attribute?

This construction is widely used in CMIP data, so we should get this point 
cleared up.

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] An area type for non-ocean water surfaces

[martin.juckes]

Hello All,

Stephane Senesi has raised the issue that some CMIP6 models may have a finite 
area of river water, and this should be recorded as an area type 
(https://github.com/cmip6dr/CMIP6_DataRequest_VariableDefinitions/issues/85). 
   
I suspect that such models may also represet lakes.

We could just add a "river" area type. Ideally, we should make clear how this 
relates to other area types. E.g. is "river" part of "land", or separate from 
it? The definition of the CMIP5 residualFrac variable implies that lakes are 
considered as part of land (though it is not clear whether this applies to all 
lakes, or just sub-grid scale lakes -- resolved by perhaps 1 or 2 CMIP5 models).

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] New standard names for OMIP biogeochemistry and chemistry

[martin.juckes]

Hello All,

I'd like to support Alison on point 4 (see 
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2017/059357.html ), regarding 
the proposal for new standard names for near surface tracer fields.

The statement 'The surface called "surface" means the lower boundary of the 
atmosphere' occurs in many standard name definitions: introducing a new set of 
names in which "surface" means "near surface" introduces an unnecessary 
ambiguity.

I also note that some of these variables have already been requested in CMIP5, 
using the approach described by Karl.

regards,
Martin


___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Overstated restriction on references to external variables in version 1.7

[martin.juckes]

Hi David,

my point is indeed that you should be able to put anything in them, so have a 
standards document which says that you can't put references to external 
variables in them would be a mistake .

cheers,
Martin


From: David Hassell [david.hass...@ncas.ac.uk]
Sent: 05 April 2017 11:35
To: Juckes, Martin (STFC,RAL,RALSP)
Cc: CF Metadata
Subject: Re: [CF-metadata] Overstated restriction on references to external 
variables in version 1.7

Hello Martin,

I don't quite see what you mean - such attributes are non-standardised, so you 
can already put anything you like in them, but software won't be able to do 
anything about it. Could you perhaps give a CDL example?

Many thanks,

David

On 5 April 2017 at 09:36, 
> wrote:
Hello All,

Section 2.6.3, External Variables, of version 1.7 of the standard contains the 
statement that:"The only CF standard attribute which is allowed to refer to 
external variables is cell_measures."

I believe this is stronger than intended: it should be possible, for instance, 
to refer to external variables in the comment and history attributes, and any 
other free text attribute, or the comment part of the cell_methods string.

Should it say something along the lines of:
"The only attribute which is allowed to make a CF standard reference to 
external variables is cell_measures"?

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata



--
David Hassell
National Centre for Atmospheric Science
Department of Meteorology, University of Reading,
Earley Gate, PO Box 243, Reading RG6 6BB
Tel: +44 118 378 5613
http://www.met.reading.ac.uk/
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Overstated restriction on references to external variables in version 1.7

[martin.juckes]

Hello All,

Section 2.6.3, External Variables, of version 1.7 of the standard contains the 
statement that:"The only CF standard attribute which is allowed to refer to 
external variables is cell_measures."

I believe this is stronger than intended: it should be possible, for instance, 
to refer to external variables in the comment and history attributes, and any 
other free text attribute, or the comment part of the cell_methods string. 

Should it say something along the lines of:
"The only attribute which is allowed to make a CF standard reference to 
external variables is cell_measures"?

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Silicate vs. dissolved inorganic silicon

[martin.juckes]

Dear Jim,

thanks. I think that means that we need a corrections to the statements, from 
the CF Standard Name list, that:

(1) '"Dissolved inorganic phosphorus" means phosphate ions in solution' in the 
CF Standard Name definition for 
mole_concentration_of_dissolved_inorganic_phosphorus_in_sea_water, and
(2) '"Dissolved inorganic silicon" means silicate ions in solution' in the 
definition of mole_concentration_of_dissolved_inorganic_silicon_in_sea_water

regards,
Martin

From: James Orr [james@lsce.ipsl.fr]
Sent: 24 March 2017 15:46
To: Lowry, Roy K.
Cc: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Silicate vs. dissolved inorganic silicon

Dissolved inorganic phosphorus in seawater takes several forms, with
phosphate (P043-) being only one of them. Furthermore, PO43- is not
even the most abundant form at normal seawater pH. Rather it is HPO42-
(hydrogen phosphate). Oceanographers do often refer to phosphate but
what they really taking about is total dissolved inorganic phosphorus
(the sum of all inorganic forms).

The seawater system for dissolved inorganic silicon is simpler because
we only need to consider two forms: silicic acid (Si(OH)4) and silicate
(SiO(OH)3-). The former is more abundant than the latter in seawater.

It is best then to refer to
- total dissolved inorganic phosphorus rather than phosphate and
- total dissolved inorganic silicon rather than silicate.

For more insight see the last figure in the OMIP-BGC protocols paper
in the CMIP6 special issue at

http://www.geosci-model-dev-discuss.net/gmd-2016-155/

Cheers,

Jim

On Fri, 24 Mar 2017, Lowry, Roy K. wrote:

> Dear All,
>
>
> If one makes the assumption that all the silicon and phosphorus atoms not 
> associated with organic ligands are
> in a single chemical form associated with oxygen in solution then what Martin 
> says is correct. In my
> experience I have never known anybody challenge this assumption and I cannot 
> think of any other anions
> incorporating P and Si. Consequently, I would agree that whilst there is a 
> theoretical semantic difference
> between the members of each Standard Name pair I would agree that this could 
> be ignored and they could be
> considered synonyms.
>
>
> Note, this only holds true as these are MOLE concentrations. The MASS 
> concentration of inorganic phosphorus
> is very different from the MASS concentration of phosphate as the oxygen 
> atoms have mass.
>
>
> If the decision is taken to take action on this then I would recommend that 
> the 'inorganic_silicon' and
> 'inorganic_phosphorus' names be than ones to be converted to aliases. This is 
> based on common terminology
> usage in the oceanographic community.
>
>
> Cheers, Roy.
>
>
> Please note that I partially retired on 01/11/2015. I am now only working 7.5 
> hours a week and can only
> guarantee e-mail response on Wednesdays, my day in the office. All vocabulary 
> queries should be sent to
> enquir...@bodc.ac.uk. Please also use this e-mail if your requirement is 
> urgent.
>
>
>
> _
> From: CF-metadata  on behalf of 
> martin.juc...@stfc.ac.uk
> 
> Sent: 24 March 2017 08:48
> To: cf-metadata@cgd.ucar.edu
> Subject: [CF-metadata] Silicate vs. dissolved inorganic silicon
> Hello Alison, others,
>
> the standard name list includes both
> (1) mole_concentration_of_dissolved_inorganic_silicon_in_sea_water and (2)
> mole_concentration_of_silicate_in_sea_water
>
> The definition of the first says that "dissolved inorganic silicon" means 
> silicate ions in solution. Both
> have units of "mol m-3". It looks to me as though they are describing the 
> same thing. If this is true, should
> one be demoted to the alias of the other? If they are different, what is the 
> difference?
>
> The same question applies to 
> mole_concentration_of_dissolved_inorganic_phosphorus_in_sea_water and
> mole_concentration_of_phosphate_in_sea_water.
>
> regards,
> Martin
>
> ___
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> CF-metadata Info Page - mailman.cgd.ucar.edu Mailing Lists
> mailman.cgd.ucar.edu
> This is an unmoderated list for discussions about interpretation, 
> clarification, and proposals for extensions
> or change to the CF conventions.
>
>
> _
> This message (and any attachments) is for the recipient only. NERC is subject 
> to the Freedom of Information
> Act 2000 and the contents of this email and any reply you make may be 
> disclosed by NERC unless it is exempt
> from release under the Act. Any material supplied to NERC may be stored in an 
> electronic records management
> 

[CF-metadata] SIMIP: ridged_sea_ice area type

[martin.juckes]

Hello Alison,

just following up on some SIMIP issues. In an exchange last year Jonathan 
suggested that we introduce a ridged_sea_ice area type ( 
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2016/058902.html ), instead 
of the new standard name ridged_sea_ice_thickness proposed by SIMIP.

Do you have this on your lists somewhere? Can the new area type be added?

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Silicate vs. dissolved inorganic silicon

[martin.juckes]

Hello Alison, others,

the standard name list includes both
(1) mole_concentration_of_dissolved_inorganic_silicon_in_sea_water and (2) 
mole_concentration_of_silicate_in_sea_water

The definition of the first says that "dissolved inorganic silicon" means 
silicate ions in solution. Both have units of "mol m-3". It looks to me as 
though they are describing the same thing. If this is true, should one be 
demoted to the alias of the other? If they are different, what is the 
difference?

The same question applies to 
mole_concentration_of_dissolved_inorganic_phosphorus_in_sea_water and 
mole_concentration_of_phosphate_in_sea_water.

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Composite area types in CMIP6 data

[martin.juckes]

Hello Karl, Jonathan,

before writing this email, I had a look at a CMIP5 variable using a similar 
coordinate, the variable "treeFracSecEver". That variable was provided by one 
institution (MIROC). I mention that, because I feel we should be cautious about 
extendin the convention in ways which commit us to a certain approach (because 
of the backward compatibility requirement of future changes) for a variable 
which can be described with the existing features and may not be used in the 
same form again.

I agree that there are many possible approaches, but, partly because of that, I 
don't agree that modifying the standard area type list of convention is "not a 
problem". It can be done, but it takes time -- and it is harder when the 
request is coming from outside the board. 

I'm happy to exploit any modifications to the area type list or convention that 
get approved, but, in the meantime I'd like to encode the request for these 
variables in a way which is consistent with the current rules of the CF 
convention. From my perspective, it would make sense to use the existing 
convention and look into a more structured encoding if there is a specific 
requirement for it. 

regards,
Martin


From: Karl Taylor [taylo...@llnl.gov]
Sent: 06 January 2017 17:29
To: Jonathan Gregory; Juckes, Martin (STFC,RAL,RALSP)
Cc: Pamment, Alison (STFC,RAL,RALSP); chris.d.jo...@metoffice.gov.uk; 
cf-metadata@cgd.ucar.edu
Subject: Re: Composite area types in CMIP6 data

Hi Martin and Jonathan,

An alternative would be to (generally) allow elemental area_types to be
combined with logical not/and/or connectors.  area_types are used either
as labels (stored in a variable pointed to by the coordinates attribute)
or in cell_methods.  In both cases parsing combined types would be
straightforward.   In your example their would only be a single
area_type coordinate, and the value of the coordinate could simply be
"pastures .and. c4_plant_functional_types" or even "pastures and
c4_plant_functional_types"

I know this appears like a significant change to the convention, but
since area_types are just strings, I suspect that software won't care
how long the strings are.   I doubt if any existing software would trip
over the compound area types (except perhaps the CF checker?).

best wishes,
Karl

On 1/6/17 8:59 AM, Jonathan Gregory wrote:
> Dear Martin
>
> It's legal to have more than one axis of a given standard_name but it is
> possibly inconvenient, because analysis software could not use the standard_
> name of those two coordinate variables to distinguish them; instead, it would
> have to use the long_name or the variable name, which is not a CF-like method.
>
> Another possibility would be to introduce new standard_names, which are
> more precise, such as area_type_of_land_use and area_type_of_vegetation in 
> this
> case, while also keeping area_type as the union. That would require a
> rearrangement of the area_type table.
>
> On the whole, I think in this case it would be best to introduce a new
> combined area_type of pastures_of_c4_plant_functional_types i.e. flatten it.
> I guess this will also be more convenient for analysts than having to search
> the combination of two dimensions. You are right of course that doing this
> could turn the area_type table into an N^2 or even an N^n problem, which would
> be unmanageable, but if this is the *only* use-case, or even if we had a dozen
> such use-cases, it is not a problem. Thus, I would say we cross this bridge
> when we come to it, rather than anticipating a problem which hasn't arisen
> yet.
>
> Best wishes
>
> Jonathan
>
> On Fri, Jan 06, 2017 at 11:43:57AM +, martin.juc...@stfc.ac.uk wrote:
>> Date: Fri, 6 Jan 2017 11:43:57 +
>> From: martin.juc...@stfc.ac.uk
>> To: j.m.greg...@reading.ac.uk, taylo...@llnl.gov, alison.pamm...@stfc.ac.uk
>> CC: chris.d.jo...@metoffice.gov.uk, cf-metadata@cgd.ucar.edu
>> Subject: Composite area types in CMIP6 data
>>
>> Hello,
>>
>> In CMIP6 we have a request for some data on pasture land with C4 functional 
>> type. We already have CF area types for pasture and c4 plants in general. We 
>> could construct the required information as follows:
>>
>>  float pastureFracC4(time, lat, lon) ;
>>  pastureFracC4:coordinates = "type ftpye" ;
>>  pastureFracC4: standard_name = "area_fraction";
>>  ..
>>  char type(strlen) ;
>>  type:long_name = "Pasture Land" ;
>>  type:standard_name = "area_type" ;
>>  char ftype(strlen) ;
>>  ftype:long_name = "Plant Functional Type" ;
>>  ftype:standard_name = "area_type" ;
>> data:
>>   type = "patsures" ;
>>   ftype = "c4_plant_functional_types";
>>
>> This would provide the information that the variable pastureFracC4 refers to 
>> areas that are both "pastures" and "c4_plant_functional_types". An 
>> altenative would be to add a new area type, but I have the feeling that the 
>> area_type 

Re: [CF-metadata] Composite area types in CMIP6 data

[martin.juckes]

Hi David,

Yes, "type" and "ftype" are conceptually independent. "pasture" refers to a 
land usage category and "C4" refers to a plant functional type which can occur 
in and land usage category.

cheers,
Martin

From: David Hassell [david.hass...@ncas.ac.uk]
Sent: 06 January 2017 12:24
To: Juckes, Martin (STFC,RAL,RALSP)
Cc: Jonathan Gregory; Karl Taylor; Pamment, Alison (STFC,RAL,RALSP); 
chris.d.jo...@metoffice.gov.uk; cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Composite area types in CMIP6 data

Hello Martin,

A potential issue with specifying two scalar coordinate variables is that each 
one must be interpreted as spanning an independent, size 1 dimension (as 
clarified in #104). Is that 
the intention, here? To combine them onto one dimension you would have to make 
the scalar coordinates into 1-d coordinates:

dimensions:
x = 1 ;
variables:
float pastureFracC4(x, time, lat, lon) ;
pastureFracC4:coordinates = "type ftpye" ;
pastureFracC4: standard_name = "area_fraction";
char type(x, strlen) ;
type:long_name = "Pasture Land" ;
type:standard_name = "area_type" ;
char ftype(x, strlen) ;
ftype:long_name = "Plant Functional Type" ;
ftype:standard_name = "area_type" ;
data:
type = "patsures" ;
ftype = "c4_plant_functional_types";

I suppose it's a question of the science that your trying to capture...

All the best,

David

On 6 January 2017 at 11:43, 
> wrote:
Hello,

In CMIP6 we have a request for some data on pasture land with C4 functional 
type. We already have CF area types for pasture and c4 plants in general. We 
could construct the required information as follows:

float pastureFracC4(time, lat, lon) ;
pastureFracC4:coordinates = "type ftpye" ;
pastureFracC4: standard_name = "area_fraction";
..
char type(strlen) ;
type:long_name = "Pasture Land" ;
type:standard_name = "area_type" ;
char ftype(strlen) ;
ftype:long_name = "Plant Functional Type" ;
ftype:standard_name = "area_type" ;
data:
 type = "patsures" ;
 ftype = "c4_plant_functional_types";

This would provide the information that the variable pastureFracC4 refers to 
areas that are both "pastures" and "c4_plant_functional_types". An altenative 
would be to add a new area type, but I have the feeling that the area_type list 
will become unmanagable if we keep adding new terms for combinations of 
existing terms.  Can anyone see a problem with using two area_type coordinates?

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata



--
David Hassell
National Centre for Atmospheric Science
Department of Meteorology, University of Reading,
Earley Gate, PO Box 243, Reading RG6 6BB
Tel: +44 118 378 5613
http://www.met.reading.ac.uk/
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Composite area types in CMIP6 data

[martin.juckes]

Hello,

In CMIP6 we have a request for some data on pasture land with C4 functional 
type. We already have CF area types for pasture and c4 plants in general. We 
could construct the required information as follows:

float pastureFracC4(time, lat, lon) ;
pastureFracC4:coordinates = "type ftpye" ;
pastureFracC4: standard_name = "area_fraction";
..
char type(strlen) ;
type:long_name = "Pasture Land" ;
type:standard_name = "area_type" ;
char ftype(strlen) ;
ftype:long_name = "Plant Functional Type" ;
ftype:standard_name = "area_type" ;
data:
 type = "patsures" ;
 ftype = "c4_plant_functional_types";

This would provide the information that the variable pastureFracC4 refers to 
areas that are both "pastures" and "c4_plant_functional_types". An altenative 
would be to add a new area type, but I have the feeling that the area_type list 
will become unmanagable if we keep adding new terms for combinations of 
existing terms.  Can anyone see a problem with using two area_type coordinates?

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] New Standard Names: frequency_distribution_..... and fractional_X_over_Y

[martin.juckes]

Dear List,

There are two variables from the CMIP5 and 6 data request with standard names 
of the form "histogram_of_X_over_Z". Following some discussion on the thread 
"Usage of histogram_of_X_over_Z 
 " which 
has clarified the definition of these terms and the intended use of 
"histogram_of_..." in CF, the conclusion was reached that the variables in 
question are actually frequency distributions rather than histograms. So, for 
CMIP6, we would like to propose new standard names to be used instead of the 
existing names:
histogram_of_backscattering_ratio_over_height_above_reference_ellipsoid
histogram_of_equivalent_reflectivity_factor_over_height_above_reference_ellipsoid

Proposed new names:
frequency_distribution_of_backscattering_ratio
frequency_distribution_of_equivalent_reflectivity_factor

I've shortened the names by omitting the "over_" component, as this is not 
essential to the definition of the physical quantity. Jonathan has suggested 
that the dimension(s) over which the data is aggregated to create a frequency 
distribution or histogram should be represented in a cell_methods string.

Further: there are two variables in the CMIP6 request which represent 
fractional occurrence of clouds
with a given range of cloud-top-heights and optical depths. Summing over 
cloud-top-heights and optical depths gives the total cloud fraction in a grid 
cell. The CMIP variables are clmisr (new in CMIP6) and clisccp (from CMIP5).  
In CMIP5, clisccp carried the standard name of "isccp_cloud_area_fraction".  
I'm concerned that this standard name does not capture the fact that the 
variables is a distributions, and so would like to propose a new standard name:
fractional_cloud_area_fraction_over_cloud_top_altitude_and_optical_depth

This differs from the histogram and frequency distribution in that it is 
neither counts nor relative frequency. It should carry coordinates 
cloud_top_altitude and optical_depth with bounds to give the bin sizes. The 
CMIP variables will be functions of latitude, longitude, time, cloud-top-height 
and optical-depth. The proposed standard name will clearly indicate the special 
role of the last two dimensions. The generic definition would be 
"fractional_X_over_Y[_and_Z] indicates the amount of X in bins defined by axes 
Y and, if present, Z. The axes must be present as coordinates of the variable 
and must have bounds specified. The variable has the same canonical units as X. 
Summing over Y [and Z] gives the quantity X".

The CMIP5 isccp_cloud_area_fraction variable is specified as a function of 
altitude, which could be interpreted as relating to area fraction of cloud in 
an altitude layer, rather than a fractional distribution with respect to cloud 
top height, and I'd like to remove this ambiguity.

regards,
Martin


___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Usage of histogram_of_X_over_Z

[martin.juckes]

Dear Jonathan,

thanks for that detailed overview. I accept your justification for having the 
the key physical quantity of interest in the standard name.

In broad usage, I have the impression that a "histogram" can be expressed as 
either a count or a percentage, so we should be explicit in the convention if 
we want a narrower definition here. A narrower definition is probably needed, 
as there would otherwise be no way of distinguishing between the two.

Would you support the addition of a paragraph in the convention to explain the 
usage you have described below.

There are two further CMIP variables, both or which are bi-variate 
distributions, with bins of spectral bands and cloud top height ranges, which 
I'd like to bring into the discussion, but it might be useful to transfer the 
conclusions of the exchange so far into a ticket first. I think the two 
additional variables could be covered by a simple extension to 
"probability_density_function_of_X_and_Y" ... though you might want to insert 
"joint_" at the beginning of the term.

regards,
Martin



Dear Martin and Alejandro  (following off-list discussions)

> The CF definitions say ''"histogram_of_X[_over_Z]" means histogram (i.e. 
> number of counts for each range of X) of variations (over Z) of X.'

Yes, that's in the guidelines for construction of standard names, and there
are only two of them at present, as you say. The simplest case is when you
have some quantity Q depending on only one dimension, Q(Z). Then the histogram
H(Q) is the number of values of Q which fall into each interval of Q,
considering variation over Z.  In general there could be more than one
dimension retained, and more than one removed. If the original field was
Q(P,Y,Z,T), we might construct a histogram H(Q,Z,T), for instance, containing
the frequencies of values of Q falling into joint intervals of Q, Z and T, for
variation over P and Y. Following the guideline above, we would call this a
histogram of Q over P and Y, I think.

It is not necessary to indicate in the standard name the dimensions which
the histogram depends on (Z and T in my example) because the coordinate
variables (of Z and T) make that clear. Martin suggests that by this argument
we could also omit Q from the standard name, and just call it a histogram
(or frequency distribution) rather than a histogram of Q, where Q is air
temperature, precipitation amount, backscattering ratio, etc. I think there
are two reasons why we include Q in the standard name,

* I think a histogram of air temperature is not the same geophysical quantity
as a histogram of precipitation amount, for instance, so they should be
distinguished by standard name.

* Although histograms are pure numbers, and so are probabilities, probability
densities are not. Histograms, probability distributions and probability
density functions are all related ways of expressing the same information.
In the guidelines, we foresee that we might need names for all of them (though
so far we have only histograms) and it would make sense to give them consistent
names. The probability density function of air temperature has units of K-1,
and of precipitation amount kg-1 m2, for instance. Because they have different
canonical units, they must have different standard names, so Q needs to be
included in the standard name.

Cell methods describe how the values represent variation within the cells.
The transformation from the values of a quantity to a histogram of the
quantity makes the original quantity into a dimension. This seems more of
a radical transformation than computing a mean or a standard deviation, which
doesn't change the dimensions of the variable, but just reduces their size
(to unity if completely collapsed). A frequency distribution of Q is
regarded as a different geophysical quantity from Q itself, so we have not
used cell methods to describe the relationship. Of course, this is a bit
arbitrary (like everything else in the CF convention!).

I agree with Martin that we could omit the "over" part of the standard name for
histograms, probabilities and probability densities. It is useful to retain the
collapsed dimensions as size-1 dimensions, so that their original range can
be recorded. They could be assigned cell_method of "sum", the default for
extensive quantities, because the histogram applies to their entire range.
The same applies to the variable with has been histogrammed and is now a
dimension; the histogram is a sum for each of its cells.

For example, in the 1D case, suppose the original field is air_temperature
as a function of time only. Then the histogram variable is
  float hair(tair);
hair:standard_name="histogram_of_air_temperature";
hair:units="1";
hair:cell_methods="time: sum tair: sum";
hair:coordinates="time";
  float time; // scalar coordinate variable with bounds
  float tair(tair);
tair:units="K";

As a multidimensional example, suppose the original field is
  float 

[CF-metadata] Metadata/standard names for variables involving thresholds (e.g. climate indices)

[martin.juckes]

Dear Lars,

see comments in line below:

Martin
---
Dear all,

Before the summer I asked in separate emails a few questions related to 
standard names. Based on the responses I have worked on this a bit more and 
made some good progress, but there still are some open issues that I am trying 
to wrap my head around. Most of the issues are related to making use of 
existing standard names involving thresholds to describe well established 
climate indices (aka indices of climate extremes). To get an overview of the 
issues I collect them here (with reference to the previous email threads), but 
if necessary we can again fork them:

1. Canonical units 
(http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2016/058800.html, 3rd point):
"number_of_days_" variables have dimension "1" (dimensionless) and 
"spell_length_of_days_..." variables have dimension "day". This seems somewhat 
inconsistent as both standard names specify "_days_". As both standard names 
contain the unit it may from a formal point of view be more appropriate that 
both would be dimensionless, but from a practical point of view, e.g. when 
designing software, it would be (much) more useful to have the unit ("day") in 
the metadata rather than having to parse it from the standard name for these 
particular variables, when this is mostly not necessary.

It does appear problematic to have units which will be confusing for the 
majority of users who are likely to be interested in these variables. Perhaps 
we could sidestep the issue by adding new standard names of the form 
"time_with_air_temperature_above_threshold" etc. Changing the units of the 
existing standard names would be problematic for existing data files which are 
valid with the current definitions.

2. Climatological time variable 
(http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2016/058800.html, 2nd point):
It is not clear to me why "number_of_days_..." and "spell_length_of_days_..." 
variables must have a climatological time variable. I would suggest that it 
makes sense to have an ordinary time coordinate variable that allows bounds to 
be specified. For example, assume that we have a time-series of daily 
near-surface temperature fields from a CMIP5 model ("tas") and want for each 
month to calculate the number_of_days_with_air_temperature_above_threshold. 
I.e. this is not much different from calculating the monthly mean temperature, 
for which an ordinary time coordinate variable with bounds is used. The same 
line of argument applies for "spell_length_of _days_..." variables and the two 
"integral_of_air_temperature_..." variables.

The climatological time axis specifies what kind of "air_temperature" is being 
compared against the threshold (e.g. daily mean, daily max).

3. Threshold must have a coordinate variable . 
(http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2016/058824.html):
This means that the threshold must be either a constant or several alternative 
constants. As I understand it, it is neither possible to have a two-dimensional 
threshold that varies over the domain, e.g. based on a climatological 
(quantile) value for each gridpoint, nor to have a three-dimensional threshold, 
e.g. a seasonally varying climatological threshold for each gridpoint. I can 
envisage two ways to allow 2-/3-dim thresholds: either 1) allow ancillary 
variables as thresholds. The ancillary variable would have a coordinate 
variable providing the the underlying constant(s), e.g. quantile levels. Or 2) 
allow the threshold to have a different unit from the main variable which then 
enables the underlying quantiles to be used as threshold variable directly 
where the actual 2-/3-dim climatological thresholds are available as an 
ancillary variable.

You could try the following, which, I believe, allows a separate threshold at 
each lat,lon point.
   float myClimateIndex( index )
  coordinates: lat lon threshold
   float lat(index):
  standard_name: latitude
   float lon(index):
  standard_name: longitude
float threshold(index):
  standard_name: air_temperature
  coordinates: lat lon

4. Standard name of the threshold variable 
(http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2016/058856.html):
The standard name air_temperature_threshold is only used in the definition of 
integral_of_air_temperature_deficit_wrt_time and 
integral_of_air_temperature_excess_wrt_time. For other threshold-based standard 
names involving air temperature, wind and lwe thickness of precipitation 
(listed below) the explanation reads ". must have a . variable with the a 
standard name of X to supply the threshold(s)". I.e. the threshold variable 
must have the same standard name as 'main variable' to which the threshold is 
applied.
The latter seems more in line with the view Jonathan expressed in a previous 
email exchange (lower part of 
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2014/057420.html).
Based on this, my suggestion 

[CF-metadata] Usage of histogram_of_X_over_Z

[martin.juckes]

Dear Alejandro,

The two CMIP variables which I'm talking about are cfadDbze94 currently defined 
as "CFAD (Cloud Frequency Altitude Diagrams) are joint height - radar 
reflectivity (or lidar scattering ratio) distributions." and cfadLidarsr532, 
which has the same definition. If they are not joint distributions we clearly 
have a problem with these definitions.

>From your reply I understand now that these are univariate distributions 
>giving the frequency of different radar reflectivities in different height 
>bands. Coming from radar/lidar instruments (or an emulator of these 
>instruments), there are multiple observations in each GCM-scale height band. 
>Presumably, there are also multiple profiles in the GCM-scale grid square, so 
>that we have a frequency distribution over sub-grid scale variability in the 
>vertical and the horizontal? Or is it actually evaluated at a spatial point?

If this is the case, you are right and we just need to correct the definitions 
in the CMIP tables (though there is still a case for introducing a 
frequencs_distribution for other variables, but that should ne another thread). 
I would favour a slightly more verbose and explicit definition, e.g.
"CFAD (Cloud Frequency Altitude Diagrams) are frequency distributions of radar 
reflectivity (or lidar scattering ratio) as a function of altitude. cfadDbze94 
is defined as the simulated relative frequency of radar reflectivity in 
sampling volumes defined by altitude bins and model grid cells."

Note that I'm using "altitude" rather than "height" to match the standard 
names: in the CF Convention, "altitude" means height above the geoid, and 
"height" means height above the surface.

Is that an accurate definition?

regards,
Martin


Dear Martin,

Thanks for your detailed explanation. I'd like to add a bit more information. 
These variables are not joint distributions, they are 1D distributions for 
different ranges of Z. The question is, does "histogram_of_X[_over_Z]" mean 
that the Z coordinate has to be completely collapsed? It is not clear to that 
the current definition implies that. If Z is not completely collapsed, you can 
then end up with a function of the form frequency(lat,lon,X,Z2), where the 
coordinate Z is only partially collapsed into bins described by Z2. I'm using 
here Z2 to explicitly show when the Z coordinate represents bins. This would 
look like a joint histogram, but it is not. I think that your proposal of 
dropping "_over_Z" from the standard name works for a joint distribution, but 
not for a collection of 1D distributions along Z, unless there is a way of 
distinguishing between both cases with the use of attributes.

Another detail is that these histograms provide relative frequencies (values 
between 0 and 1, not counts), not absolute frequencies. Is that inconsistent 
with the current definition of histogram in CF?

Regards,

Alejandro

> -Original Message-
> From: martin.juckes at 
> stfc.ac.uk<http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata> 
> [mailto:martin.juckes at 
> stfc.ac.uk<http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata>]
> Sent: 12 October 2016 19:05
> To: cf-metadata at 
> cgd.ucar.edu<http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata>
> Cc: Bodas-Salcedo, Alejandro
> Subject: Usage of histogram_of_X_over_Z
>
> Hello,
>
> There are two standard names of the form histogram_of_. in the CF Standard
> Name list (at version 36):
> histogram_of_backscattering_ratio_over_height_above_reference_ellipsoid and
> histogram_of_equivalent_reflectivity_factor_over_height_above_reference_ellipsoid
> . Both of these where used in CMIP5 and set to be used in CMIP6, but the usage
> does not appear to match the standard name desecriptions.
>
> The possible confusion is over the role of different coordinates. The CF 
> definitions
> say ''"histogram_of_X[_over_Z]" means histogram (i.e. number of counts for 
> each
> range of X) of variations (over Z) of X.' This implies to me that you start 
> with a
> function of Z and possibly other coordinates and end up with a function of X 
> and the
> other coordinates. E.g. if the source data is X(lat,lon,Z), then the 
> histogram data will
> be of the form frequency(lat,lon,X).
>
> In the two CMIP5/CMIP6 draft variables (cfadLidarsr532, cfadDbze94) using 
> these
> standard names the "Z" coordinate  which is included in the standard name
> ("height_above_reference_ellipsoid") is one of the coordinates of the 
> histogram data
> variable. Both these variables appear to be joint distributions (frequency of 
> X and Y
> values) over sub-grid variability as a function of latitude, longitude and 
> time.
>
> I've been reviewing these existing defi

[CF-metadata] Missing data bins in histograms

[martin.juckes]

Dear Jonathan,

I'm sorry I didn't respond on the point about it being the first bin: I had not 
intended the special value to be restricted to the first bin, so I guess there 
is something ambiguous in my intial formulation which is giving this 
impression. I agree that we should formulate any extension so that it can apply 
to any bin, and I also think it should be possible to label multiple bins in 
this way.

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Usage of histogram_of_X_over_Z

[martin.juckes]

Hello,

There are two standard names of the form histogram_of_. in the CF Standard 
Name list (at version 36): 
histogram_of_backscattering_ratio_over_height_above_reference_ellipsoid and 
histogram_of_equivalent_reflectivity_factor_over_height_above_reference_ellipsoid.
 Both of these where used in CMIP5 and set to be used in CMIP6, but the usage 
does not appear to match the standard name desecriptions. 

The possible confusion is over the role of different coordinates. The CF 
definitions say ''"histogram_of_X[_over_Z]" means histogram (i.e. number of 
counts for each range of X) of variations (over Z) of X.' This implies to me 
that you start with a function of Z and possibly other coordinates and end up 
with a function of X and the other coordinates. E.g. if the source data is 
X(lat,lon,Z), then the histogram data will be of the form frequency(lat,lon,X).

In the two CMIP5/CMIP6 draft variables (cfadLidarsr532, cfadDbze94) using these 
standard names the "Z" coordinate  which is included in the standard name 
("height_above_reference_ellipsoid") is one of the coordinates of the histogram 
data variable. Both these variables appear to be joint distributions (frequency 
of X and Y values) over sub-grid variability as a function of latitude, 
longitude and time. 

I've been reviewing these existing definitions in some detail because there are 
some new distribution variables in the request and I'd like to make sure that 
we have a consistent approach. 

If we need to described a variable which carries a joint distribution of X and 
Y, then the variable will have to use X and Y as coordinates, so perhaps we can 
simplify the process by leaving them out of the standard name. Similarly the 
"over_Z" part of the name would be better expressed as a cell_methods 
construct. This line of reasoning suggests using a new standard name such as 
"frequency_distribution" (units "1"). The only difficulty is that the frequency 
distribution might be a function of the quantities X and Y (scattering ratio 
and cloud top height for cfadLidarsr532) and also of latitude, longitude and 
time. There should be some way of distinguishing the different roles of these 5 
coordinates: is is the distribution of X and Y as a function of latitude, 
longitude and time. I think this could be done conveniently by introducing a 
single new attribute, e.g. "bin_coords: X Y".

"frequency_distribution" could be used for single or joint distributions.

My questions to the list are:
(1) am I missing something in my interpretation of the existing 
histogram_of_... names?
(2) if not, is the adoption of a "frequency_distribution" standard name an 
appropriate way forward?

regards,
Martin

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Missing data bins in histograms

[martin.juckes]

Dear Karl, Jonathan, Jim,

thanks for those comments.

The CMIP6 variable in question is clmisr 
(http://clipc-services.ceda.ac.uk/dreq/u/59151ed6-9e49-11e5-803c-0d0b866b59f3.html)
 with a coordinatte of 16 altitude bins 
(http://clipc-services.ceda.ac.uk/dreq/u/dim:alt16.html ).

I'd be happy with Jim's proposed solution, which does not need any change to 
the convention, though it may be a bit cryptic: all the examples in the 
convention are for cases in which all array values are intended to match one of 
the flag_values. Having an array which is a mixture of flags and "normal" 
values would be a new usage.  We could, perhaps, introduce a consistency 
problem: ticket 151 (http://cf-trac.llnl.gov/trac/ticket/151) explains how, for 
variables with standard_name "area_type", flag_values and flag_meanings can be 
used to encode the data, in which case it is the "flag_meanings" which match 
the requirements of the standard name. Here, on the other hand, we want the 
special bin to be the exception which is not described by the standard name 
(altitude). So .. perhaps it is simpler to introduce a new attribute name?

Concerning Jonathan and Karl's comments, the idea of calling it a 
"missing_value" was a mistake I made, but it actually refers to locations where 
cloud is detected but the height of the cloud cannot be retrieved.

The current proposal is to have a value of 0.0 in the coordinate and 
(-99000.0,0.0) in the bounds of the special value "bin". I imagine these need 
to be present, but I think their values are not going to mean anything.

It is certainly possible to do as Karl suggests and place an explanation in the 
variable description. Having the special status of the first bin explicitly 
flagged in way which can be easily picked up by software brings added value.

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Missing data bins in histograms

[martin.juckes]

Hello,

the CF standard name list has two "histogram_ " entries, and in the CMIP6 
data request we may need to add a third, a histogram_of_cloud_top_height. 
Besides the standard name, we also need, for this new variable, a method of 
encoding the "missing data" bin in the histogram. That is, the histogram should 
record frequency in 16 data bins and one additional bin for the frequency of 
missing data.

Can we define a "missing_data_index" attribute for histogram variables, and use 
this to indicate that the first bin in the array has this special purpose. It 
might be more pythonic to put the _FillValue in the coordinate value for the 
missing data bin, but I suspect that this would cause substantial problems for 
many software packages.

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] CMIP6 Sea Ice MIP: Ice thickness

[martin.juckes]

Dear Dirk,

OK. For the snow thickness, do you want a weighted time mean? For example, if 
there is a 1m snow layer over 50% of a grid cell for half the month and 3m over 
25% for the 2nd half of the month (the rest of the cell being snow free), do 
you want a weighted time mean of 1.6667m or a simple mean of 2m?

regards,
Martin

From: Dirk Notz [dirk.n...@mpimet.mpg.de]
Sent: 02 August 2016 11:59
To: Juckes, Martin (STFC,RAL,RALSP); j.m.greg...@reading.ac.uk; Pamment, Alison 
(STFC,RAL,RALSP)
Cc: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] CMIP6 Sea Ice MIP: Ice thickness

Dear Martin,

> It looks as though you have resolved all the issues nicely, but I have one 
> concern about Jonathan's suggestion for dealing with the thickness of ice 
> floating in melt ponds on the surface of sea ice. The suggestion is to use 
> the existing standard name "floating_ice_thickness" with a cell methods term 
> of the form "area: mean where sea_ice_melt_pond".  My concern is that there 
> is some ambiguity here between the thickness of the ice floating in the melt 
> pond and the thickness of the sea ice underneath the melt pond, which is 
> floating in the sea. This might be resolved if "floating_ice_thickness" was 
> to be defined in such a way as to exclude sea ice, but the current definition 
> does not do this.  (It states that '"Floating ice" means any ice that is 
> floating on water, e.g. on a sea or lake surface.')

This is an important point, thanks for raising it. It might then be
necessary to indeed introduce a new variable as initially suggested to
avoid this ambiguity. This variable was initially called
"thickness_of_sea_ice_melt_pond_refrozen_ice ".

> I have a 2nd comment about the suggested area type of snow_covered_sea_ice:  
> this has been proposed for use with variables which represent the amount of 
> snow on sea ice (m) and the heat content of that snow (J m-2). In both cases 
> these are quantities which can be considered as zero on snow-free sea ice, in 
> which case there is no need to mask them by the area of snow cover (I believe 
> Jonathan has advanced this argument in another thread recently). In this 
> case, amount of snow on sea ice would just be represented by 
> "surface_snow_thickness" with cell methods "area: mean where sea_ice over 
> all_area_types" (or "area: mean where sea_ice over sea").

This is certainly true for the heat content. For snow thickness,
however, we would like to be able to consider partial snow coverage. We
would like to record the thickness of the actual snow, not the average
thickness including those parts of sea ice that are ice free. Hence, for
snow thickness, I believe that the area type "snow_covered_sea_ice"
should still be applied.

Thanks in any case for your helpful feedback,

best,

 Dirk
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] CMIP6 Sea Ice MIP: Ice thickness

[martin.juckes]

Dear Jonathan, Dirk, Alison,

It looks as though you have resolved all the issues nicely, but I have one 
concern about Jonathan's suggestion for dealing with the thickness of ice 
floating in melt ponds on the surface of sea ice. The suggestion is to use the 
existing standard name "floating_ice_thickness" with a cell methods term of the 
form "area: mean where sea_ice_melt_pond".  My concern is that there is some 
ambiguity here between the thickness of the ice floating in the melt pond and 
the thickness of the sea ice underneath the melt pond, which is floating in the 
sea. This might be resolved if "floating_ice_thickness" was to be defined in 
such a way as to exclude sea ice, but the current definition does not do this.  
(It states that '"Floating ice" means any ice that is floating on water, e.g. 
on a sea or lake surface.')

I have a 2nd comment about the suggested area type of snow_covered_sea_ice:  
this has been proposed for use with variables which represent the amount of 
snow on sea ice (m) and the heat content of that snow (J m-2). In both cases 
these are quantities which can be considered as zero on snow-free sea ice, in 
which case there is no need to mask them by the area of snow cover (I believe 
Jonathan has advanced this argument in another thread recently). In this case, 
amount of snow on sea ice would just be represented by "surface_snow_thickness" 
with cell methods "area: mean where sea_ice over all_area_types" (or "area: 
mean where sea_ice over sea"). 

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] CMIP6 Sea Ice MIP: Integrated quantities

[martin.juckes]

Dear Jonathon, Dirk,

I agree with Jonathon that it is a good idea to follow the approach used in 
other names (e.g. number_of_days_with_air_temperature_above_threshold) and use 
a coordinate for the threshold value .. though I would prefer to make it 
obligatory to state the threshold in order to avoid ambiguity.

Another issue is the domain .. the variable is being defined as the total area 
of cells above the threshold within a specific domain (the N. hemisphere of S. 
hemisphere in the CMIP6 data request). The threshold examples that I could find 
in the existing standard names are all temporal statistics (e.g. number of days 
with quantity X above a threshold within a year) and so use a climatological 
time variable. Here we want to specify the area over which cells are being 
summed. Would it make sense to use a 2nd coordinate variable, with standard 
name "region" specifying the region (this would also require adding 
"northern_hemisphere" and "southern_hemisphere" to the region list)? 
Alternatively, we could use latitude and longitude coordinate variables with 
bounds attributes to declare the region.

The region will be in the long name, of course, but it would be good to have an 
agreed approach to encoding it in CF,

regards,
Martin


Dear Dirk

> For sea_ice_extent, there is currently no definition in the CF
> convention. This variable is somewhat odd in a CF context, as it only
> makes sense when applied over several grid cells.
>
> Hence, we suggest to adopt a definition which implies usage over several
> grid cells:
>
> sea_ice_extent: Total area of all grid cells that are covered by a
> minimum of a given area fraction [usually 15 %) of sea ice in a
> specified region".

I agree that it needs a definition. It is like sea ice area (m2) in being an
extensive quantity which is integrated over the area considered, but I agree
that extent is not meaningful unless there is some discretisation of area.
I would suggest that we could allow for the possibility of the threshold
being other than 15%, following practice for other standard names (as in
recent emails of Martin's), like this

sea_ice_extent: Total area of all grid cells in which the sea ice area
fraction equals or exceeds a threshold. By default the threshold is 15%.
The threshold can be specified by supplying a coordinate variable or scalar
coordinate variable with standard_name of sea_ice_area_fraction.

Would something like that be OK?

Best wishes

Jonathan

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Recording requirements expressed in standard name definitions

[martin.juckes]

Hello Roy,

Thanks, in that case I agree completely. It will be an interesting challenge to 
come up with a suitable set of RDF predicates. I hope your optimism about 
reaching agreement on a place for the list in CF is justified,

cheers,
Martin

From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 14 July 2016 16:25
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Recording requirements expressed in standard name 
definitions


Hi Martin,


Slight misunderstanding - I was taking the creation of a structured list and 
its publication through CF for granted. Serving in machine-readable RDF through 
NVS was seen very much as an additional resource.


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: martin.juc...@stfc.ac.uk 
Sent: 14 July 2016 14:10
To: Lowry, Roy K.; cf-metadata@cgd.ucar.edu
Subject: RE: [CF-metadata] Recording requirements expressed in standard name 
definitions

Hi Roy,

I can see how that might work. However, I still think it would still be useful 
to have a clearly structured list as an intermediate step.

After posting the email it occurred to me that it may be better to have such a 
list either as part of the conformance document or as an additional list posted 
in the CF Standard Name page (http://cfconventions.org/standard-names.html) 
rather than adding it to the convention document itself,
Standard Names - CF Conventions
cfconventions.org
Toggle navigation CF MetaData. Conformance; Discussion; Documents; Governance; 
Standard Names; CF Standard Names. Standard Name Table (v34, 13 June 2016)




regards,
Martin

From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 13 July 2016 19:25
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Recording requirements expressed in standard name 
definitions


Hi Martin,


NVS delivers in RDF, so the knowledge would have to be served in the form of 
triples describing required relationships between Standard Names, such as:


area_fraction requires area_type


or between Standard Names and other CF controlled vocabularies in NVS like cell 
methods.  I'll suggest spinning up a project to investigate what is possible 
and serve some examples.


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: martin.juc...@stfc.ac.uk 
Sent: 13 July 2016 16:36
To: Lowry, Roy K.; cf-metadata@cgd.ucar.edu
Subject: RE: [CF-metadata] Recording requirements expressed in standard name 
definitions

Dear Roy,

Having the constraints expressed in a machine readable way in the NVS sounds 
good in principle, but do you have a clear idea about how to make it work in 
practise? The constraints described in ticket 151 are quite complex, so I'm not 
sure how they would be expressed.
e.g.
Either:
  var.type == string AND var.values in standard_region_list
Or:
  flag_values, flag_meanings in var.attributes AND flag_meanings.values in 
standard_region_list AND var.type == flag_values.type

But as the above text is just something I made up now, it would take quite a 
lot of work, I expect, to get a reliable parsing algorithm.

regards,
Martin

From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 13 July 2016 09:38
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: RE: [CF-metadata] Recording requirements expressed in standard name
definitions

Dear Martin,

This sounds an excellent idea. What do you thinks of the idea of building the 
knowledge into NVS to make it accessible in machine-readable form and so the 
requirements could be enforced by the CF checker?

Cheers, Roy.

-Original Message-
From: CF-metadata [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of 
martin.juc...@stfc.ac.uk
Sent: 07 July 2016 12:01
To: cf-metadata@cgd.ucar.edu
Subject: [CF-metadata] Recording requirements expressed in standard name 
definitions

Dear All,

in connection with ticket 151 (http://cf-trac.llnl.gov/trac/ticket/151) I had 
proposed a section in appendix B listing standard names whose definitions 
impose additional constraints on variable attributes etc. Jonathan convinced me 
that ticket 151 was not the right place for this.

The proposal for this new section was motivated by the fact that standard name 
descriptions 

Re: [CF-metadata] Recording requirements expressed in standard name definitions

[martin.juckes]

Hi Roy,

I can see how that might work. However, I still think it would still be useful 
to have a clearly structured list as an intermediate step.

After posting the email it occurred to me that it may be better to have such a 
list either as part of the conformance document or as an additional list posted 
in the CF Standard Name page (http://cfconventions.org/standard-names.html) 
rather than adding it to the convention document itself,

regards,
Martin

From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 13 July 2016 19:25
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Recording requirements expressed in standard name 
definitions


Hi Martin,


NVS delivers in RDF, so the knowledge would have to be served in the form of 
triples describing required relationships between Standard Names, such as:


area_fraction requires area_type


or between Standard Names and other CF controlled vocabularies in NVS like cell 
methods.  I'll suggest spinning up a project to investigate what is possible 
and serve some examples.


Cheers, Roy.


Please note that I partially retired on 01/11/2015. I am now only working 7.5 
hours a week and can only guarantee e-mail response on Wednesdays, my day in 
the office. All vocabulary queries should be sent to enquir...@bodc.ac.uk. 
Please also use this e-mail if your requirement is urgent.



From: martin.juc...@stfc.ac.uk 
Sent: 13 July 2016 16:36
To: Lowry, Roy K.; cf-metadata@cgd.ucar.edu
Subject: RE: [CF-metadata] Recording requirements expressed in standard name 
definitions

Dear Roy,

Having the constraints expressed in a machine readable way in the NVS sounds 
good in principle, but do you have a clear idea about how to make it work in 
practise? The constraints described in ticket 151 are quite complex, so I'm not 
sure how they would be expressed.
e.g.
Either:
  var.type == string AND var.values in standard_region_list
Or:
  flag_values, flag_meanings in var.attributes AND flag_meanings.values in 
standard_region_list AND var.type == flag_values.type

But as the above text is just something I made up now, it would take quite a 
lot of work, I expect, to get a reliable parsing algorithm.

regards,
Martin

From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 13 July 2016 09:38
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: RE: [CF-metadata] Recording requirements expressed in standard name
definitions

Dear Martin,

This sounds an excellent idea. What do you thinks of the idea of building the 
knowledge into NVS to make it accessible in machine-readable form and so the 
requirements could be enforced by the CF checker?

Cheers, Roy.

-Original Message-
From: CF-metadata [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of 
martin.juc...@stfc.ac.uk
Sent: 07 July 2016 12:01
To: cf-metadata@cgd.ucar.edu
Subject: [CF-metadata] Recording requirements expressed in standard name 
definitions

Dear All,

in connection with ticket 151 (http://cf-trac.llnl.gov/trac/ticket/151) I had 
proposed a section in appendix B listing standard names whose definitions 
impose additional constraints on variable attributes etc. Jonathan convinced me 
that ticket 151 was not the right place for this.

The proposal for this new section was motivated by the fact that standard name 
descriptions cannot include examples or markup, and the specification of the 
rules is not as clear as in the convention text. It also appears that the rules 
are not checked by the CF checker (at least not the few that I have looked at 
in detail) and I think the best way to get consistent checking would be to 
first create a well structured summary of these rules in the conventions 
document.

The constraint related to ticket 151 is that variables with standard name 
"region" or "area_type" must be consistent with the standard region and area 
type lists.

I have reviewed the first half of the standard name list (a to m) and found the 
following coordinate constraints expressed:

(1) area_fraction: requires an area_type coordinate;

(2) atmosphere_lifting_condensation_level + 2 others: requires an 
original_air_pressure_of_lifted_parcel coordinate.

(3) Many "layer" quantities (e.g. 
dry_static_energy_content_of_atmosphere_layer): require vertical coordinate 
with bounds.

(4) 
change_in_energy_content_of_atmosphere_layer_due_to_change_in_sigma_coordinate_wrt_surface_pressure:
 must have a vertical coordinate variable (axis=Z)

(5) change_over_time_... and .._displacement: require bounds on time coordinate

(6) downwelling_photosynthetic_photon_radiance_in_sea_water and other radiance 
variables: direction must be specified, e.g. with coordinate of "zenith_angle".

(7) mass_concentration_of_pm..._ambient_aerosol_in_air (and 
mass_fraction_of_pm..): require air_temperature and relative_humidity

(8) 

Re: [CF-metadata] Recording requirements expressed in standard name definitions

[martin.juckes]

Dear Roy,

Having the constraints expressed in a machine readable way in the NVS sounds 
good in principle, but do you have a clear idea about how to make it work in 
practise? The constraints described in ticket 151 are quite complex, so I'm not 
sure how they would be expressed.
e.g.
Either:
  var.type == string AND var.values in standard_region_list
Or:
  flag_values, flag_meanings in var.attributes AND flag_meanings.values in 
standard_region_list AND var.type == flag_values.type

But as the above text is just something I made up now, it would take quite a 
lot of work, I expect, to get a reliable parsing algorithm.

regards,
Martin

From: Lowry, Roy K. [r...@bodc.ac.uk]
Sent: 13 July 2016 09:38
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: RE: [CF-metadata] Recording requirements expressed in standard name
definitions

Dear Martin,

This sounds an excellent idea. What do you thinks of the idea of building the 
knowledge into NVS to make it accessible in machine-readable form and so the 
requirements could be enforced by the CF checker?

Cheers, Roy.

-Original Message-
From: CF-metadata [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of 
martin.juc...@stfc.ac.uk
Sent: 07 July 2016 12:01
To: cf-metadata@cgd.ucar.edu
Subject: [CF-metadata] Recording requirements expressed in standard name 
definitions

Dear All,

in connection with ticket 151 (http://cf-trac.llnl.gov/trac/ticket/151) I had 
proposed a section in appendix B listing standard names whose definitions 
impose additional constraints on variable attributes etc. Jonathan convinced me 
that ticket 151 was not the right place for this.

The proposal for this new section was motivated by the fact that standard name 
descriptions cannot include examples or markup, and the specification of the 
rules is not as clear as in the convention text. It also appears that the rules 
are not checked by the CF checker (at least not the few that I have looked at 
in detail) and I think the best way to get consistent checking would be to 
first create a well structured summary of these rules in the conventions 
document.

The constraint related to ticket 151 is that variables with standard name 
"region" or "area_type" must be consistent with the standard region and area 
type lists.

I have reviewed the first half of the standard name list (a to m) and found the 
following coordinate constraints expressed:

(1) area_fraction: requires an area_type coordinate;

(2) atmosphere_lifting_condensation_level + 2 others: requires an 
original_air_pressure_of_lifted_parcel coordinate.

(3) Many "layer" quantities (e.g. 
dry_static_energy_content_of_atmosphere_layer): require vertical coordinate 
with bounds.

(4) 
change_in_energy_content_of_atmosphere_layer_due_to_change_in_sigma_coordinate_wrt_surface_pressure:
 must have a vertical coordinate variable (axis=Z)

(5) change_over_time_... and .._displacement: require bounds on time coordinate

(6) downwelling_photosynthetic_photon_radiance_in_sea_water and other radiance 
variables: direction must be specified, e.g. with coordinate of "zenith_angle".

(7) mass_concentration_of_pm..._ambient_aerosol_in_air (and 
mass_fraction_of_pm..): require air_temperature and relative_humidity

(8) isotropic_radiance_per_unit_wavelength_in_air (and other 
per_unit_wavelength varables): the definition is slightly ambiguous with the 
sentence  "A coordinate variable for radiation wavelength should be given the 
standard name radiation_wavelength" which, taken literally, means the use of a 
wavelength coordinate is optional: should it be "A coordinate variable for 
radiation wavelength should be given with the standard name 
radiation_wavelength", making the wavelength coordinate required?

Is it worth completing the review of the standard name desriptions and creating 
an appendix section to list these constraints, giving examples (or pointing to 
examples which may be better positioned in relevant portions of the main text)?

regards,
Martin


___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

 This message (and any attachments) is for the recipient only. NERC is subject 
to the Freedom of Information Act 2000 and the contents of this email and any 
reply you make may be disclosed by NERC unless it is exempt from release under 
the Act. Any material supplied to NERC may be stored in an electronic records 
management system.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Recording requirements expressed in standard name definitions

[martin.juckes]

Dear All,

in connection with ticket 151 (http://cf-trac.llnl.gov/trac/ticket/151) I had 
proposed a section in appendix B listing standard names whose definitions 
impose additional constraints on variable attributes etc. Jonathan convinced me 
that ticket 151 was not the right place for this.

The proposal for this new section was motivated by the fact that standard name 
descriptions cannot include examples or markup, and the specification of the 
rules is not as clear as in the convention text. It also appears that the rules 
are not checked by the CF checker (at least not the few that I have looked at 
in detail) and I think the best way to get consistent checking would be to 
first create a well structured summary of these rules in the conventions 
document.

The constraint related to ticket 151 is that variables with standard name 
"region" or "area_type" must be consistent with the standard region and area 
type lists.

I have reviewed the first half of the standard name list (a to m) and found the 
following coordinate constraints expressed:

(1) area_fraction: requires an area_type coordinate;

(2) atmosphere_lifting_condensation_level + 2 others: requires an 
original_air_pressure_of_lifted_parcel coordinate.

(3) Many "layer" quantities (e.g. 
dry_static_energy_content_of_atmosphere_layer): require vertical coordinate 
with bounds.

(4) 
change_in_energy_content_of_atmosphere_layer_due_to_change_in_sigma_coordinate_wrt_surface_pressure:
 must have a vertical coordinate variable (axis=Z)

(5) change_over_time_... and .._displacement: require bounds on time coordinate

(6) downwelling_photosynthetic_photon_radiance_in_sea_water and other radiance 
variables: direction must be specified, e.g. with coordinate of "zenith_angle".

(7) mass_concentration_of_pm..._ambient_aerosol_in_air (and 
mass_fraction_of_pm..): require air_temperature and relative_humidity

(8) isotropic_radiance_per_unit_wavelength_in_air (and other 
per_unit_wavelength varables): the definition is slightly ambiguous with the 
sentence  "A coordinate variable for radiation wavelength should be given the 
standard name radiation_wavelength" which, taken literally, means the use of a 
wavelength coordinate is optional: should it be "A coordinate variable for 
radiation wavelength should be given with the standard name 
radiation_wavelength", making the wavelength coordinate required?

Is it worth completing the review of the standard name desriptions and creating 
an appendix section to list these constraints, giving examples (or pointing to 
examples which may be better positioned in relevant portions of the main text)?

regards,
Martin


___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Use of CF standard name region

[martin.juckes]

Dear Jonathan,

I can understand the argument that the concept of "region" is what you want in 
the standard name, and I'm happy to accept that. The problem that started this 
discussion remains, however. Namely, the current standard name definition for 
region includes a reference to the list of area_types specified within the CF 
Convention and an indication as to how that should be implemented.

regards,
Martin



#



Dear Martin

I agree with adding to the definition of region, and also area_type (for which
this approach has also been advocated), that it may be convenient to store
such variables as numbers with a flag_values and flag_meanings attribute.
However I don't think it should be regarded as a different quantity, so I
don't think it needs a different standard name. I don't think we should define
standard numbers for regions or area_types, because this would be against the
usual CF principle that files should describe their contents without need for
reference to external tables. The representation of strings as numbers is not
standardised, and is defined in the file by the flag attributes. That is why
I regard it as an issue of encoding, more like scale and offset, and not a
different quantity.

Best wishes

Jonathan

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] weighted time mean vs. conditional time mean.

[martin.juckes]

Dear Jonathan,

yes, it is absolutely clear that "where" can only be used with area types. It 
is also clear, I thought, that some of these area types may vary with time: the 
area type list includes "fire" and "cloud", for example.

7.3.3 gives a template for the cell_methods element:
dim1: [dim2: [dim3: ...]] method [where type1 [over type2]] [within|over 
days|years] [(comment)]

and goes on to say: "The valid values for dim1 [dim2[dim3 ...] ] are the names 
of dimensions of the data variable, names ofscalar coordinate variables of the 
data variable, valid standard names,or the word area." There is no stated 
restriction on the values of "dim" which can precede "where ...".

You appear to be taking a geographical interpretation of "where" and assuming 
that it can only apply to spatial information, but have been reading it from a 
mathematical perspective, in which it can refer to any dimension. In 
mathematics, statements of the form "sum of A where condition B" carry no 
implication that "where" has anything to do with area. From this perspective, 
there is no need to introduce "when" ..

The use case that prompted this, from SIMIP, corresponds to your 3rd example, 
in which we are averaging over all points in the cell and time period covered 
for which the area type is valid, giving each point equal weight. This can be 
handled, as you and Karl have pointed out earlier, with a comment in the 
cell_methods string of the form "(weighted by )", but I feel that the use 
case is clear enough that there is a need for it to be treated in the 
conventions.

regards,
Martin

##



Dear Martin

In my reading of 7.3.3 and the conformance document, it seems clear that
"where" is intended to be used with area types.

> There is an issue, it appears, about the use of the "where" modifier for 
> cell_methods elements other than "area:". Jonathan believes "where" should 
> only apply for area on the basis that this where the motivation comes from in 
> the first paragraph of section 7.3.3. The subsequent paragraphs in section 
> 7.3.3. describe the use of "where" with a generic element "name: ". The 
> compliance document clearly states that "where" can be used with any string.

I'm sorry, I can't find that - please could you point it out? In
http://cfconventions.org/Data/cf-documents/requirements-recommendations/requirements-recommendations-1.6.html
regarding
method [where type1 [over type2]]
it says
The valid values for type1 are the name of a string-valued auxiliary or scalar
coordinate variable with a standard_name of area_type, or any string value
allowed for a variable of standard_name of area_type.

We could generalise area_types to mean "states" so they can apply in time as
well as space. I think all the existing ones could be interpreted in this way
i.e. with the sense of "when" rather than "where". Vegetation is sometimes
present and sometimes absent at any given spot, for instance, just as it is
present in some spots and not others at any given time.

Suppose you want to calculate a radiative flux for a grid-box in cloud-free
air. You can do this on each instantaneous timestep for the cloud-free fraction
of the grid-box, and then calculate a time-mean of these timestep values i.e.
"area: mean where clear_sky time: mean". If the input data supplies a higher
spatial resolution than the grid-box, so you have many timeseries, you could
alternatively do it the other way round, and first calculate, for each of the
points, the value of the flux for those timesteps when there is no cloud, then
calculate an area-mean of these local values i.e. "time: mean where clear_sky
area: mean". These aren't the same because they imply different weights.

For example, suppose you have three points within the grid-box and two times,
and the data is as follows:

a X X
b c X

where X means cloudy, and a, b, c are clear-sky values. According to the first
method, the value is a/2 + b/4 + c/4, and according to the second method it is
a/4 + b/4 + c/2, if I've done my sums right. There is a third method, in which
we consider both time and space together: "time: area: mean where clear_sky".
In this case the value is a/3 + b/3 + c/3.

If I'm right about this, I think we could make this generalisation and it would
not be problematic. However, as usual, we should only make the change if there
is a use-case which demands it.

Best wishes

Jonathan




___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] weighted time mean vs. conditional time mean.

[martin.juckes]

Hi David,

that is true. My question was whether there should be, as Jonathan proposes, a 
restriction on the dimensions that "where type1" can be associated with,

cheers,
Martin

From: David Hassell [david.hass...@ncas.ac.uk]
Sent: 20 May 2016 13:03
To: Juckes, Martin (STFC,RAL,RALSP)
Cc: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] weighted time mean vs. conditional time mean.

​Hi Martin,

I think that the compliance document and 7.3.3 consistently force the "type1" 
in "where type1" must ​come from the area_types controlled vocabulary 
(http://cfconventions.org/Data/area-type-table/current/build/area-type-table.html),
 or be the name of a coordinate with area_types as it values:
​
  "The valid values for type1 are the name of a string-valued auxiliary or 
scalar coordinate variable with a standard_name of area_type, or any string 
value allowed for a variable of standard_name of area_type​"

​I've just noticed that the area-types table​ linked above, updated 3 days ago, 
includes some of the things you're interested in: "cloud", "clear_sky", 
"dust_aerosol" - I don't know when these got added, or how they are defined, 
but if they're alright, then I see no reason why "where can not be used.

All the best,

David

On 20 May 2016 at 12:09, 
> wrote:
Hello Jonathan, Jim,

Just picking up the thread again.

There is an issue, it appears, about the use of the "where" modifier for 
cell_methods elements other than "area:". Jonathan believes "where" should only 
apply for area on the basis that this where the motivation comes from in the 
first paragraph of section 7.3.3. The subsequent paragraphs in section 7.3.3. 
describe the use of "where" with a generic element "name: ". The compliance 
document clearly states that "where" can be used with any string.

The specific use case is to specify how the mean should be taken when there is 
a temporarily varying area_type. Jonathan has suggesed using a suitably 
formulated free-text comment in brackets ("time: mean (when cloud present)") 
but I would prefer a formulation with an explicit conventional meaning:  "time: 
mean where cloud".

So there are two questions:
(1) Can "where" be used, as stated by the convention, with any coordinate?
(2) If there is a clear consensus that "where " can only be used for 
"area", then can we update the convention, conformance requirements and CF 
checker to be consistent with this?

regards,
Martin




___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata



--
David Hassell
National Centre for Atmospheric Science
Department of Meteorology, University of Reading, Earley Gate, PO Box 243, 
Reading RG6 6BB
Tel: +44 118 378 5613
http://www.met.reading.ac.uk/
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Use of CF standard name region

[martin.juckes]

Hello All,

In CMIP5 the variable "basin" was used as a fixed spatial field with integer 
values and the CF Standard Name "region", which has the definition "A variable 
with the standard name of region contains strings which indicate geographical 
regions. These strings must be chosen from the standard region list."

The integer valued CMIP5 variable is clearly not consistent with this 
definition. The CMIP5 variable was defined with flag_values and flag_meanings, 
such that the flag_meanings were from the CF standard region list.

The question is, should we redefine the CMIP5 variable somehow, or would it be 
acceptable to adjust the CF Standard Name definition for region to accept this 
usage which appears clear enough and is presumably much easier for plotting 
packages to handle than a spatial array of string values,

regards,
Martin


___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] weighted time mean vs. conditional time mean.

[martin.juckes]

Hello Jonathan, Jim,

Just picking up the thread again.

There is an issue, it appears, about the use of the "where" modifier for 
cell_methods elements other than "area:". Jonathan believes "where" should only 
apply for area on the basis that this where the motivation comes from in the 
first paragraph of section 7.3.3. The subsequent paragraphs in section 7.3.3. 
describe the use of "where" with a generic element "name: ". The compliance 
document clearly states that "where" can be used with any string.

The specific use case is to specify how the mean should be taken when there is 
a temporarily varying area_type. Jonathan has suggesed using a suitably 
formulated free-text comment in brackets ("time: mean (when cloud present)") 
but I would prefer a formulation with an explicit conventional meaning:  "time: 
mean where cloud".

So there are two questions:
(1) Can "where" be used, as stated by the convention, with any coordinate?
(2) If there is a clear consensus that "where " can only be used for 
"area", then can we update the convention, conformance requirements and CF 
checker to be consistent with this?

regards,
Martin




___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Soil litter definitions

[martin.juckes]

Dear All,

I've been checking through the definitions of some of the land variables used 
in CMIP5 and the CMIP6 request, and there may be some ambiguity in the concept 
of "litter".

Two terms (litter_carbon_flux, litter_carbon_content) have definitions which 
include the phrase  '"Litter carbon" is dead inorganic material .', which 
surely should be "dead organic", though "dead plant material", as used in the 
definitions of carbon_mass_flux_into_soil_from_vegetation_excluding_litter, 
carbon_mass_flux_into_soil_from_litter appears to be the more common and 
accurate definition.

There is a further problem in that the above terms are defined as if "litter" 
inlcuded all plant debris, but the definition of "wood_debris_carbon_content " 
asserts, to the contrary, that 'dead organic matter composed of coarse wood .. 
is distinct from litter'.  Do we really want wood debris to be distinct from 
litter, rather than a component of litter?

regards,
Martin


___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] weighted time mean vs. conditional time mean.

[martin.juckes]

Hello,

We have two similar but distinct temporal averaging use cases in the CMIP6 data 
request and I'd be grateful to some advise about CF encoding.

(1) Grid cells have a temporally varying sea-ice fraction, and a mean is wanted 
over the sea-ice area.

Is this a correct cell_methods formulation:
"time: mean where sea_ice area: mean where sea_ice"?

(2) Grid cells which sometimes have convective cloud, and a mean is wanted over 
the times when there is a cloud in the cell. The variable to be averaged is, in 
this case, the minimum over the cell of the convective cloud base.

Is this the correct cell_methods:
"time: mean where convective_cloud area: minimum"?

In the first case the first case "mean where sea_ice" refers to the portion of 
grid cells have sea_ice, in the 2nd "mean where convective_cloud" would refer 
to whole cells ... is this OK?


cheers,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Confusing skin temperature and interface temperature

[martin.juckes]

Hello All,

I'm happy with Karl's suggestion: including "subskin" in the list is a clear 
improvement,

cheers,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Negative emissions?

[martin.juckes]

Dear Karl, Jonathan,

I'm happy with that interpretation. There is a slight problem in the detail: 
standard name definitions containing the word "emission" often have the 
sentence '"Emission" means emission from a primary source located anywhere 
within the atmosphere, including at the lower boundary (i.e. the surface of the 
earth).' In many standard names there are other phrases, but in, for instance 
the definition of 
tendency_of_atmosphere_mass_content_of_ammonia_due_to_emission there is no 
further qualification: if "source" is interpreted as including both positive 
and negative sources (i.e. sinks) and the defnition says that it can be 
anywhere in the atmosphere  ... so "emission" here becomes all sources and 
sinks.

On the other hand, the intention is clear enough, and the specific problem I 
had (about anthropogenic CO2 emissions) is now resolved -- so I won't propose a 
change at this stage,

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Negative emissions?

[martin.juckes]

Hello All,

The CMIP6 data request will, like the CMIP5 data request, include a request for 
a variable defined as "Carbon Mass Flux into Atmosphere Due to All 
Anthropogenic Emissions of CO2" with CF Standard Name 
"tendency_of_atmosphere_mass_content_of_carbon_dioxide_expressed_as_carbon_due_to_anthropogenic_emission".

My question is whether the definition of this term is intended to apply only to 
positive emissions, or should negative values be recorded when, according to 
some scenarios, the net anthropoenic flux is out of the atmosphere? The use of 
the word "Emission" implies some judgement about the sign, but it is not 
spelled out in the definition. 

regards,
Martin


From: CF-metadata [cf-metadata-boun...@cgd.ucar.edu] on behalf of 
cf-metadata-requ...@cgd.ucar.edu [cf-metadata-requ...@cgd.ucar.edu]
Sent: 08 March 2016 10:21
To: cf-metadata@cgd.ucar.edu
Subject: CF-metadata Digest, Vol 155, Issue 14

Send CF-metadata mailing list submissions to
cf-metadata@cgd.ucar.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
or, via email, send a message with subject or body 'help' to
cf-metadata-requ...@cgd.ucar.edu

You can reach the person managing the list at
cf-metadata-ow...@cgd.ucar.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of CF-metadata digest..."


Today's Topics:

   1. Confusing skin temperature and interface temperature
  (martin.juc...@stfc.ac.uk)
   2. Re: Confusing skin temperature and interface  temperature
  (Craig Donlon)


--

Message: 1
Date: Tue, 8 Mar 2016 09:47:40 +
From: 
To: 
Subject: [CF-metadata] Confusing skin temperature and interface
temperature
Message-ID:

Content-Type: text/plain; charset="Windows-1252"

Hello All,

Karl has raised an objection to the wording  " not the skin " which was 
carried over from the current CF Standard Name definition for 
sea_surface_temperature in my suggested update. The update is intended to 
correct a currently erroneous reference to "surface_temperature" as skin 
temperature. Karl's objection, which also applies to the existing definition 
(and appears to date back to v1 fo the list), could be accomodated by a simple 
change:

 ?Sea surface temperature is usually abbreviated as "SST". It
 is the temperature of sea water near the surface (including
 the part under sea-ice, if any). More specific terms 
sea_surface_skin_temperature and surface_temperature
 are available for the skin and interface
 temperature respectively. For the temperature of sea water at a
particular depth or layer, a data variable of
sea_water_temperature with a vertical coordinate axis should
 be used.?

regards,
Martin




From: CF-metadata [cf-metadata-boun...@cgd.ucar.edu] on behalf of 
cf-metadata-requ...@cgd.ucar.edu [cf-metadata-requ...@cgd.ucar.edu]
Sent: 08 March 2016 01:46
To: cf-metadata@cgd.ucar.edu
Subject: CF-metadata Digest, Vol 155, Issue 13

Send CF-metadata mailing list submissions to
cf-metadata@cgd.ucar.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
or, via email, send a message with subject or body 'help' to
cf-metadata-requ...@cgd.ucar.edu

You can reach the person managing the list at
cf-metadata-ow...@cgd.ucar.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of CF-metadata digest..."


Today's Topics:

   1. Re: Confusing skin temperature and interface temperature
  (Karl Taylor)


--

Message: 1
Date: Mon, 7 Mar 2016 17:47:05 -0800
From: Karl Taylor 
To: Peter Minnett ,
alison.pamm...@stfc.ac.uk, craig.don...@esa.int
Cc: cf-metadata@cgd.ucar.edu, kenneth.ca...@noaa.gov,
anne.ocarr...@eumetsat.int, edward.m.armstr...@jpl.nasa.gov
Subject: Re: [CF-metadata] Confusing skin temperature and interface
temperature
Message-ID: <56de2f19.2080...@llnl.gov>
Content-Type: text/plain; charset="utf-8"; Format="flowed"

Dear Peter, Craig and all,

For observations I am not arguing that all the different ocean
temperature definitions aren't needed.  In describing observations I
understand that skin and surface temperature are not identical.  My
statement was that by construction (almost all) current models assume
that the temperature is vertically uniform (i.e., the water is perfectly
mixed and homogeneous) throughout the upper most layer, so in *those*
models the statement that the 

[CF-metadata] Confusing skin temperature and interface temperature

[martin.juckes]

Hello All,

Karl has raised an objection to the wording  " not the skin " which was 
carried over from the current CF Standard Name definition for 
sea_surface_temperature in my suggested update. The update is intended to 
correct a currently erroneous reference to "surface_temperature" as skin 
temperature. Karl's objection, which also applies to the existing definition 
(and appears to date back to v1 fo the list), could be accomodated by a simple 
change:
 
 ‘Sea surface temperature is usually abbreviated as "SST". It
 is the temperature of sea water near the surface (including
 the part under sea-ice, if any). More specific terms 
sea_surface_skin_temperature and surface_temperature 
 are available for the skin and interface
 temperature respectively. For the temperature of sea water at a
particular depth or layer, a data variable of
sea_water_temperature with a vertical coordinate axis should
 be used.’

regards,
Martin




From: CF-metadata [cf-metadata-boun...@cgd.ucar.edu] on behalf of 
cf-metadata-requ...@cgd.ucar.edu [cf-metadata-requ...@cgd.ucar.edu]
Sent: 08 March 2016 01:46
To: cf-metadata@cgd.ucar.edu
Subject: CF-metadata Digest, Vol 155, Issue 13

Send CF-metadata mailing list submissions to
cf-metadata@cgd.ucar.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
or, via email, send a message with subject or body 'help' to
cf-metadata-requ...@cgd.ucar.edu

You can reach the person managing the list at
cf-metadata-ow...@cgd.ucar.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of CF-metadata digest..."


Today's Topics:

   1. Re: Confusing skin temperature and interface temperature
  (Karl Taylor)


--

Message: 1
Date: Mon, 7 Mar 2016 17:47:05 -0800
From: Karl Taylor 
To: Peter Minnett ,
alison.pamm...@stfc.ac.uk, craig.don...@esa.int
Cc: cf-metadata@cgd.ucar.edu, kenneth.ca...@noaa.gov,
anne.ocarr...@eumetsat.int, edward.m.armstr...@jpl.nasa.gov
Subject: Re: [CF-metadata] Confusing skin temperature and interface
temperature
Message-ID: <56de2f19.2080...@llnl.gov>
Content-Type: text/plain; charset="utf-8"; Format="flowed"

Dear Peter, Craig and all,

For observations I am not arguing that all the different ocean
temperature definitions aren't needed.  In describing observations I
understand that skin and surface temperature are not identical.  My
statement was that by construction (almost all) current models assume
that the temperature is vertically uniform (i.e., the water is perfectly
mixed and homogeneous) throughout the upper most layer, so in *those*
models the statement that the "sea_surface_temperature" is "not the skin
or interface temperature" is *wrong*.

The CF standard name description of "sea_surface_temperature" is
somewhat vague by design: "the temperature of sea water near the
surface".  Because it is vague, it *could* defensibly be used to
represent any more precisely defined near-surface temperature, including
"sea_surface_skin_temperature", "sea_surface_subskin_temperature", or
"sea_surface_foundation_temperature".

Even for observations it would be wrong to say  "sea water near the
surface is not the skin temperature".

Since skin temperature is near the surface and sea_surface_temperature
is vague, it might in fact be the same as skin temperature (e.g., if
sea_surface_temperature in fact recorded the conductive
diffusion-dominated sub-layer at a depth of approximately 10-20
micrometers below the air-sea interface). Again, usually in models,
sea_surface_temperature most emphatically does provide the model's best
(only!) estimate of skin temperature.

If the description were changed to read:
"It is the temperature of sea water near the surface (including the part
under sea-ice, if any), and not necessarily the skin temperature".
I would be happy.

Better yet, why not include in the discussion  the following points:

1) surface temperature, sea_surface_temperature,
sea_surface_skin_temperature, sea_surface_subskin_temperature, and
sea_surface_foundation_temperature are all terms that might apply to the
temperature of sea water.
2) When the temperature represents a horizontal spatial average,
surface_temperature represents the mean of the temperature over all
surface types in the domain, whereas the other temperatures do not.
3) The sea_surface_temperature is imprecise because it represents a
near-surface temperature sampled within (or averaged over) the portion
of the column extending from the surface down to perhaps several
meters.  In many ocean models, the temperature does not vary in that
portion of the column so sea_surface_temperature might be the
appropriate standard_name.  Note that in this case, if part of the

[CF-metadata] Confusing skin temperature and interface temperature

[martin.juckes]

Hello All,

The CF Standard Name sea_surface_temperature includes the statement that it is 
" not the skin temperature, whose standard name is surface_temperature". 
The last phrase here is incorrect: the standard name of the skin temperature is 
sea_surface_skin_temperature, not surface_temperature. Can the definition be 
modified to read ".. not the skin or interface temperature, whose standard 
names are sea_surface_skin_temperature and surface_temperature respectively"?

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] On scalar coordinate variables

[martin.juckes]

Hi David,

In case (1) "double dim1" is a an NUG scalar coordinate variable. My point is 
that the #104 text appears ambiguous to me, because it is using the term 
"scalar coordinate variable" in a way which is different from the NUG usage, 
but at the same time relies on reference to the NUG text for definitions of 
coordinate variables,

regards,
Martin

From: David Hassell [d.c.hass...@reading.ac.uk]
Sent: 16 December 2015 23:38
To: Juckes, Martin (STFC,RAL,RALSP)
Cc: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] On scalar coordinate variables

Hi Martin,

I've just come back to this. I think you're right that #104 refers to
your case (2), but is that because in case (1), "double dim1" is a CF
data variable and not a CF scalar coordinate variable?

Thanks,

David

 Original message from martin.juc...@stfc.ac.uk (09AM 09 Dec 15)

> Date: Wed, 9 Dec 2015 09:07:22 +
> From: martin.juc...@stfc.ac.uk
> To: d.c.hass...@reading.ac.uk
> CC: cf-metadata@cgd.ucar.edu
> Subject: RE: [CF-metadata] On scalar coordinate variables
>
> Hi David,
>
> Aren't there two cases here, one in which the scalar coordinate does have the 
> same name as a dimension and one in which it doesn't? i.e.
>
> (1) scalar NUG coordinate variable
> Dimensions:
>dim1 = 1 ;
> variables:
>float myvar(dim1);
>double dim1;
>
> (2) Scalar CF coordinate variable
> variables:
>float myvar;
>   myvar: coordinates= "dim1" ;
>double dim1;
>
> I see that ticket 104 assumes that the term "scalar coordinate variable" only 
> refers to the 2nd example, but example (1) is declares a valid coordinate 
> variable in the NUG sense which is also a scalar. If CF wants to exclude 
> this, it needs to be explicitly stated that it is not allowed (or, if it is 
> already excluded by the convention somehow, this restriction relative to the 
> NUG convention should be clarified).
>
> I'm not sure that the reference to NUG is incorrect .. I certainly didn't 
> mean to assert that.  I have the impression the NUG usage here is what users 
> expect and so it should be in the CF convention and the other parts of the 
> convention should be consistent. In what sense do you think it is incorrect?
>
> Regards,
> Martin
>
> -Original Message-
> From: David Hassell [mailto:d.c.hass...@reading.ac.uk]
> Sent: 08 December 2015 14:19
> To: Juckes, Martin (STFC,RAL,RALSP)
> Cc: cf-metadata@cgd.ucar.edu
> Subject: Re: [CF-metadata] On scalar coordinate variables
>
> Hello Martin,
>
> I think that a CF scalar coordinate variable is not a NUG-defined coordinate 
> variable because it does not have the same name as a dimension.
>
> Nor is it a special type of CF coordinate variable, as was discussed in 
> ticket #104 http://cf-trac.llnl.gov/trac/ticket/104 - it could be 
> functionally equivalent to an auxiliary coordinate variable.
>
> However, section 1.3 makes it clear (in italics, no less) that
>
>   "The use of [NUG-defined] coordinate variables is required for all
>dimensions that correspond to one dimensional space or time
>coordinates"
>
> which as you point out is incorrect. Perhaps that is where a clarification 
> should go, i.e.:
>
>   "The use of coordinate variables or scalar coordinate variables (as
>defined in section 5.7) is required for all dimensions that
>correspond to one dimensional space or time coordinates"
>
> What do you think?
>
> All the best,
>
> David
>
>  Original message from martin.juc...@stfc.ac.uk (09AM 08 Dec 15)
>
> > Date: Tue, 8 Dec 2015 09:58:29 +
> > From: martin.juc...@stfc.ac.uk
> > To: cf-metadata@cgd.ucar.edu
> > Subject: [CF-metadata] On scalar coordinate variables
> >
> > Hello,
> >
> > The CF Convention 1.6 and draft 1.7 both include, in the discussion of 
> > dimensions in Section 2.4,  the statement that:
> > "It is also acceptable to use a scalar coordinate variable which eliminates 
> > the need for an associated size one dimension in the data variable."
> >
> > However, the convention states that coordinate variables should be 
> > interpreted as 'NUG-defined "coordinate variables."'. The NUG is vague 
> > about the definition ( 
> > https://www.unidata.ucar.edu/software/netcdf/docs/coordinate_variables.html 
> > ), but it does say "Current application packages that make use of 
> > coordinate variables commonly assume they are numeric vectors and strictly 
> > monotonic". It also states that "A position along a dimension can be 
> > specified using an index", which is not consistent with the use of a scalar 
> > coordinate variable.
> >
> > One application which appears to assume that coordinate variables are 
> > vectors is the CF Checker, so we need some clarification. I'm not sure how 
> > other applications deal with it.
> >
> > The problem with the current phrasing in the CF Conventions document is 
> > that it suggests the NUG approach is being followed and then introduces a 
> > departure from the NUG approach in a 

Re: [CF-metadata] On scalar coordinate variables

[martin.juckes]

Hi David,

Aren't there two cases here, one in which the scalar coordinate does have the 
same name as a dimension and one in which it doesn't? i.e.

(1) scalar NUG coordinate variable
Dimensions:
   dim1 = 1 ;
variables:
   float myvar(dim1);
   double dim1;

(2) Scalar CF coordinate variable
variables:
   float myvar;
  myvar: coordinates= "dim1" ; 
   double dim1;

I see that ticket 104 assumes that the term "scalar coordinate variable" only 
refers to the 2nd example, but example (1) is declares a valid coordinate 
variable in the NUG sense which is also a scalar. If CF wants to exclude this, 
it needs to be explicitly stated that it is not allowed (or, if it is already 
excluded by the convention somehow, this restriction relative to the NUG 
convention should be clarified).
 
I'm not sure that the reference to NUG is incorrect .. I certainly didn't mean 
to assert that.  I have the impression the NUG usage here is what users expect 
and so it should be in the CF convention and the other parts of the convention 
should be consistent. In what sense do you think it is incorrect?

Regards,
Martin

-Original Message-
From: David Hassell [mailto:d.c.hass...@reading.ac.uk] 
Sent: 08 December 2015 14:19
To: Juckes, Martin (STFC,RAL,RALSP)
Cc: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] On scalar coordinate variables

Hello Martin,

I think that a CF scalar coordinate variable is not a NUG-defined coordinate 
variable because it does not have the same name as a dimension.

Nor is it a special type of CF coordinate variable, as was discussed in ticket 
#104 http://cf-trac.llnl.gov/trac/ticket/104 - it could be functionally 
equivalent to an auxiliary coordinate variable.

However, section 1.3 makes it clear (in italics, no less) that

  "The use of [NUG-defined] coordinate variables is required for all
   dimensions that correspond to one dimensional space or time
   coordinates"

which as you point out is incorrect. Perhaps that is where a clarification 
should go, i.e.:

  "The use of coordinate variables or scalar coordinate variables (as
   defined in section 5.7) is required for all dimensions that
   correspond to one dimensional space or time coordinates"

What do you think?

All the best,

David

 Original message from martin.juc...@stfc.ac.uk (09AM 08 Dec 15)

> Date: Tue, 8 Dec 2015 09:58:29 +
> From: martin.juc...@stfc.ac.uk
> To: cf-metadata@cgd.ucar.edu
> Subject: [CF-metadata] On scalar coordinate variables
> 
> Hello,
> 
> The CF Convention 1.6 and draft 1.7 both include, in the discussion of 
> dimensions in Section 2.4,  the statement that:
> "It is also acceptable to use a scalar coordinate variable which eliminates 
> the need for an associated size one dimension in the data variable."
> 
> However, the convention states that coordinate variables should be 
> interpreted as 'NUG-defined "coordinate variables."'. The NUG is vague about 
> the definition ( 
> https://www.unidata.ucar.edu/software/netcdf/docs/coordinate_variables.html 
> ), but it does say "Current application packages that make use of coordinate 
> variables commonly assume they are numeric vectors and strictly monotonic". 
> It also states that "A position along a dimension can be specified using an 
> index", which is not consistent with the use of a scalar coordinate variable.
> 
> One application which appears to assume that coordinate variables are vectors 
> is the CF Checker, so we need some clarification. I'm not sure how other 
> applications deal with it.
> 
> The problem with the current phrasing in the CF Conventions document is that 
> it suggests the NUG approach is being followed and then introduces a 
> departure from the NUG approach in a separate part of the text.
> 
> I would recommend either adding after 'NUG-defined "coordinate variables"' a 
> clarification '(that is a scalar or vector variable with the same name as a 
> dimension)', or changing the statement about use of scalar coordinate 
> variables.
> 
> regards,
> Martin
> ___
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata


--
David Hassell
National Centre for Atmospheric Science (NCAS) Department of Meteorology, 
University of Reading, Earley Gate, PO Box 243, Reading RG6 6BB, U.K.

Tel   : +44 118 3785613
E-mail: d.c.hass...@reading.ac.uk
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] On scalar coordinate variables

[martin.juckes]

Hello,

The CF Convention 1.6 and draft 1.7 both include, in the discussion of 
dimensions in Section 2.4,  the statement that:
"It is also acceptable to use a scalar coordinate variable which eliminates the 
need for an associated size one dimension in the data variable."

However, the convention states that coordinate variables should be interpreted 
as 'NUG-defined "coordinate variables."'. The NUG is vague about the definition 
( https://www.unidata.ucar.edu/software/netcdf/docs/coordinate_variables.html 
), but it does say "Current application packages that make use of coordinate 
variables commonly assume they are numeric vectors and strictly monotonic". It 
also states that "A position along a dimension can be specified using an 
index", which is not consistent with the use of a scalar coordinate variable.

One application which appears to assume that coordinate variables are vectors 
is the CF Checker, so we need some clarification. I'm not sure how other 
applications deal with it.

The problem with the current phrasing in the CF Conventions document is that it 
suggests the NUG approach is being followed and then introduces a departure 
from the NUG approach in a separate part of the text.

I would recommend either adding after 'NUG-defined "coordinate variables"' a 
clarification '(that is a scalar or vector variable with the same name as a 
dimension)', or changing the statement about use of scalar coordinate variables.

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Weighted means and cell_methods in the CMIP6 data request

[martin.juckes]

Hello All,

There are a number of variables in the CMIP6 data request which are requested 
as weighted means, such as the age of snow as a time mean weighted by mass of 
snow. We also have area weighted means, as in the monthly mean temperature of 
sea-ice weighted by sea ice area. The latter can be handled well enough with 
the existing cell_methods syntax: "time: mean where sea_ice". For the former, 
the CMIP5 approach was to have a comment indicating that weighting should be 
used. As this is a reasonably common operation it would be nice to have 
something more explicit in the cell_methods attribute.

I can think of several possibilities:
(1) add a "weighted-mean" method, and leave it up to the data provider to give 
additional information. This would at least alert the user that they need to 
look for additional information.  This would be an improvement. In the present 
convention "time: mean" can mean either a simple mean or a weighted mean. By 
adding the "weighted-mean" option we would be able to stipulate that "time: 
mean" only be used for non-weighted means, and this would reduce an existing 
ambiguity.

(2) add a "weighted-by: " option in the cell-methods comment 
statement, similar to the "interval: ..." clause, e.g. "time: mean 
(weighted-by: snw)". This would give more information, but if the comment is 
considered as optional it does not remove the ambiguity that "time: mean" can 
apply to either weighted or un-weighted mean. Making the comment obligatory for 
weighted means would blur the status of the comment. There is also the problem 
that since the variable "snw" is not going to be in the file the information 
remains incomplete.

(3) add a weighted clause, e.g. "time: mean [where  ] weighted snm".  The 
main problem here is that parsing cell_methods is already complicated, and this 
would add to that difficulty, though only in a small incremental way.

(4) as (1), but with a additional requirement that the dimension over which the 
weighted mean is being taken carry information about the weighting. The 
information could be attached either as a specified attribute "weighting". 
Because the weighting variable will generally be at a higher frequency than the 
weighted-mean we are trying to describe it will not be sensible to include it, 
so this attribute will at most provide a clue about the provenance. For 
example, it might be of the form " [()]".
e.g. 'weighting: snw (daily snow mass --- archived in the "day" MIP table)'.

The last option appears the cleanest to me, as it does not change the grammar 
of the cell_methods string and adds additional information to the relevant 
dimension in a fairly self-explanatory way.

Perhaps this has been discussed before? Any other thoughts?

regards,
Martin

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Specifying latitude and longitude of transects and regions (Karl Taylor)

[martin.juckes]

Hello Karl,

In your email of 17th July you ask the use case for providing the spatial 
coordinates of data. I was under the impression that this was an accepted 
objective: the proposal here was to close a gap whereby there is a small 
collection of CMIP5 and CMIP6 variables, namely those on corresponding to 
fluxes through transects and basin data, for which the CF Convention provides 
no means of detailed geo-referencing (though we could just use a scalar 
coordinate variable to specify the mid-point of the transect/basin). The use 
case is, as stated in my initial email, geo-referencing the data,

regards,
Martin



From: CF-metadata [cf-metadata-boun...@cgd.ucar.edu] on behalf of 
cf-metadata-requ...@cgd.ucar.edu [cf-metadata-requ...@cgd.ucar.edu]
Sent: 17 July 2015 17:03
To: cf-metadata@cgd.ucar.edu
Subject: CF-metadata Digest, Vol 147, Issue 22

Send CF-metadata mailing list submissions to
cf-metadata@cgd.ucar.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
or, via email, send a message with subject or body 'help' to
cf-metadata-requ...@cgd.ucar.edu

You can reach the person managing the list at
cf-metadata-ow...@cgd.ucar.edu

When replying, please edit your Subject line so it is more specific
than Re: Contents of CF-metadata digest...


Today's Topics:

   1. proposal for new standard names for some cloud quantities
  (Jonathan Gregory)
   2.  How to define time coordinate in GPS? (Jonathan Gregory)
   3. Re: Specifying latitude and longitude of transects and
  regions (Hedley, Mark)
   4. Re: Specifying latitude and longitude of transects and
  regions (Karl Taylor)


--

Message: 1
Date: Fri, 17 Jul 2015 14:49:41 +0100
From: Jonathan Gregory j.m.greg...@reading.ac.uk
To: cf-metadata@cgd.ucar.edu
Subject: [CF-metadata] proposal for new standard names for some cloud
quantities
Message-ID: 20150717134941.ga7...@met.reading.ac.uk
Content-Type: text/plain; charset=us-ascii

Dear Mark

I can't remember whether I found any existing inconsistency. Are there any
names among those which were previously added for CFMIP that have the same
definitions of evaporation and condensation as you used? Thank you for
modifying your definitions and removing problems so that for the moment we
will not be introducing an inconsistency. Using the wider definitions of
evaporation and condensation means that it's simpler to name and discuss
phase transitions gas - liquid or solid, while sublimation and deposition
refer to gas - solid, but as you imply, we would need to find other words or
phrases to refer to gas - liquid. In support of the general definition,
I would note that vapour means gas, and condensed means not gas (as in
https://en.wikipedia.org/wiki/Condensed_matter_physics), so these words are
quite logical - but I would definitely not argue that languages are or should
be always logical! For CF consistency is a very important principle. We could
adopt your preferred definitions, if we renamed quite a few of the existing
standard names.

Best wishes

Jonathan

- Forwarded message from Mark Webb mark.w...@metoffice.gov.uk -

 Date: Wed, 15 Jul 2015 18:36:55 +0100
 From: Mark Webb mark.w...@metoffice.gov.uk
 To: Jonathan Gregory j.m.greg...@reading.ac.uk
 CC: Mark Webb mark.w...@metoffice.gov.uk, cf-metadata@cgd.ucar.edu
 Subject: Re: [CF-metadata] proposal for new standard names for some cloud
   quantities
 User-Agent: Mutt/1.5.20 (2009-12-10)

 Dear Jonathan,

 Thanks for your comments.

  Thanks very much. Having the definitions detail the processes helps a lot.
  I do have a remaining concern about terminology, though, which probably 
  should
  have been noticed earlier. In the guidelines, condensed water means liquid
  or solid (ice), for instance in 
  mass_fraction_of_cloud_condensed_water_in_air,
  which says this explicitly in its definition.

  For consistency, condensation should mean gas - liquid or solid. The
  A Met Soc glossary says in general that's what condensation means, but
  in meteorology it means gas - liquid.
http://glossary.ametsoc.org/wiki/Condensation
  It's unfortunate that it's ambiguous! I think the general definition is
  more satisfactory.

  The same entry says evaporation means liquid or solid - gas i.e. the 
  reverse
  of condensation. That is the sense in which we use it in some other standard
  names e.g. water_evaporation_flux. However the AMS entry for evaporation 
  gives
  this as its first sense, but remarks that it's usually liquid-gas.  
  Again, an
  unsatisfactory ambiguity, and I would prefer the broader definition.  With 
  the
  broader definitions, deposition (gas - solid) is a subset of condensation,
  and sublimation (solid - gas) a subset of evaporation.
 
  It looks like we may have some existing inconsistency 

Re: [CF-metadata] Applying multiple conventions

[martin.juckes]

Continuing thread: 
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2015/058036.html

Hello All,

I'd like to pick up this thread again. The min motivation of raising it was to 
get a change to the following line in the draft 1.7 conformance document (in 
section 2.6.1) 'Files that conform to the CF version 1.5 conventions must 
indicate this by setting the global Conventions attribute to the string value 
CF-1.5'.

As Nan has pointed out, the UNIDATA FTP page 
(ftp://ftp.unidata.ucar.edu/pub/netcdf/Conventions/) is no longer the 
authoritative location for NetCDF conventions, having been superseded by 
http://www.unidata.ucar.edu/software/netcdf/conventions.html : so it would make 
sense to update the reference to the FTP page in the CF Conventions document.

It looks to me as though that page has a clear and well thought out definition 
of the use of the Conventions attribute, and it would make sense to adopt it. 
For the specific cases of UGRID and CMIP it looks as though CF-1.7/CMIP6 or 
CF-1.7/UGRID/CMIP6 would be appropriate.

For UGRID, this approach would be cleaner if the UGRID convention was actually 
registered and referenced on 
http://www.unidata.ucar.edu/software/netcdf/conventions.html -- I'll look into 
that, and the same applies to CMIP conventions.

regards,
Martin












___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Specifying latitude and longitude of transects and regions

[martin.juckes]

Hello Mark,

If the two end points can be specified with bounds within the existing 
convention, it might be simpler to use that. Can you explain to me how this is 
done? The only reference to bounds which I could find in the convention was in 
connection with cell boundaries.

The flow direction does need to be defined .. I suppose that would involve a 
clarification of the standard_name ocean_volume_transport_across_line. As you 
say, this should not be too complicated once we have a definition of the line 
to refer to.

The approach I was thinking of could easily accommodate multiple points on a 
line, though I don't have a use for it at present. e.g.

float mfo(ntransect):
      
   coordinates: passage lon lat
float lat(ntransect):
   
   transect: lat_trans
float lon(ntransect):
   
   transect: lon_trans
float lat_trans(ntransect,npt):
float lon_trans(ntransect,npt):

Where ntransect is the number of transects and npt is the number of points 
use to define each transect (=2).
 
regards,
Martin

From: Hedley, Mark [mark.hed...@metoffice.gov.uk]
Sent: 30 June 2015 16:58
To: Juckes, Martin (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu
Subject: RE: [CF-metadata] Specifying latitude and longitude of transects and   
regions

Hello Martin et al

this sounds an awful lot like an extensive feature definition to me.

such definitions are widely supported in spatial data sets and software.

It would be very useful to be able to define these in CF.  I would not limit 
this (in principal) to defining the start and end of a line, there seems to me 
little added complexity in defining a line of n points rather than a line of 2 
points.  This is already done with bounds

There is lots of prior art, in ISO, GML, OGC so we won't need to invent very 
much.

In the end these are just coordinate collections, we can use lots of what is 
already in CF to define them.

If these are lines which flow is defined through then we'll also need a 
direction of flow to be defined, does this sound correct? There's also prior 
art for this.

This sounds a really useful capability extension to me.  I think the extensions 
you are looking for could be delivered quite neatly.

I expect there will be a few ways to approach encoding this information.  I 
think it might be worth sketching up some options and how they'd look, then 
discussing their strengths and weaknesses.  I'd be happy to contribute to such 
an activity, if that would be useful.

mark



From: CF-metadata [cf-metadata-boun...@cgd.ucar.edu] on behalf of 
martin.juc...@stfc.ac.uk [martin.juc...@stfc.ac.uk]
Sent: 29 June 2015 13:31
To: cf-metadata@cgd.ucar.edu
Subject: [CF-metadata] Specifying latitude and longitude of transects and   
regions

Hello All,

There are some variables in the CMIP5 archive which lack explicit latitude and 
longitude information, such as mfo (standard name 
sea_water_transport_across_line) and msftzzzba 
(ocean_y_overturning_mass_streamfunction_due_to_bolus_advection). The first is 
a mass flux across a transect and the second is a zonally averaged stream 
function, averaged across an ocean basin.

For transects, the mfo variable is encoded with an index dimension and a 
coordinate passage which labels transects by the name of the passage they 
cross (e.g. fram_strait). The information about precisely which part of the 
Fram Strait is used is in a document referred to from the MIP tables. It would 
be nice to have a means of encoding the end points of the transect in the CF 
metadata. I looked into using the bounds attribute, but that is defined to 
represent cell boundaries, so an extension to the convention is, I think, 
needed. It makes sense to follow the pattern used to represent cell boundaries. 
 I propose defining a new attribute transect which, like the bounds 
attribute, can be attached to coordinate variables and takes the name of 
another variable as its value. The named variable should contain the end points 
of the transect.

The same approach could be used for the streamfunction .. which is a mean 
across an east-west transect (this approach would lead to a specifcation of the 
east and west endpoints of each basin at each latitude -- but I'm not sure that 
it would be feasible to collect that much detail in the CMIP data files).

Is there a cleaner way of encoding transect and basin coordinates?

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Specifying latitude and longitude of transects and regions

[martin.juckes]

Hello All,

There are some variables in the CMIP5 archive which lack explicit latitude and 
longitude information, such as mfo (standard name 
sea_water_transport_across_line) and msftzzzba 
(ocean_y_overturning_mass_streamfunction_due_to_bolus_advection). The first is 
a mass flux across a transect and the second is a zonally averaged stream 
function, averaged across an ocean basin.  

For transects, the mfo variable is encoded with an index dimension and a 
coordinate passage which labels transects by the name of the passage they 
cross (e.g. fram_strait). The information about precisely which part of the 
Fram Strait is used is in a document referred to from the MIP tables. It would 
be nice to have a means of encoding the end points of the transect in the CF 
metadata. I looked into using the bounds attribute, but that is defined to 
represent cell boundaries, so an extension to the convention is, I think, 
needed. It makes sense to follow the pattern used to represent cell boundaries. 
 I propose defining a new attribute transect which, like the bounds 
attribute, can be attached to coordinate variables and takes the name of 
another variable as its value. The named variable should contain the end points 
of the transect.

The same approach could be used for the streamfunction .. which is a mean 
across an east-west transect (this approach would lead to a specifcation of the 
east and west endpoints of each basin at each latitude -- but I'm not sure that 
it would be feasible to collect that much detail in the CMIP data files). 

Is there a cleaner way of encoding transect and basin coordinates?

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Specifying latitude and longitude of transects and regions

[martin.juckes]

Hi Jonathon,

I'm afraid I don't have much to add -- the use case is to record endpoints so 
that people can find out what the endpoints are. The transects are requested by 
specification of the endpoints. I'm requesting this for the CMIP6 metadata.  
What would be the use case for more detailed information?

cheers,
Martin


From: CF-metadata [cf-metadata-boun...@cgd.ucar.edu] on behalf of 
cf-metadata-requ...@cgd.ucar.edu [cf-metadata-requ...@cgd.ucar.edu]
Sent: 29 June 2015 17:38
To: cf-metadata@cgd.ucar.edu
Subject: CF-metadata Digest, Vol 146, Issue 55

Send CF-metadata mailing list submissions to
cf-metadata@cgd.ucar.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
or, via email, send a message with subject or body 'help' to
cf-metadata-requ...@cgd.ucar.edu

You can reach the person managing the list at
cf-metadata-ow...@cgd.ucar.edu

When replying, please edit your Subject line so it is more specific
than Re: Contents of CF-metadata digest...


Today's Topics:

   1. Specifying latitude and longitude of transects and regions
  (Jonathan Gregory)
   2. Re: How to define time coordinate in GPS? (Jim Biard)


--

Message: 1
Date: Mon, 29 Jun 2015 17:37:15 +0100
From: Jonathan Gregory j.m.greg...@reading.ac.uk
To: cf-metadata@cgd.ucar.edu
Subject: [CF-metadata] Specifying latitude and longitude of transects
and regions
Message-ID: 20150629163715.ga7...@met.reading.ac.uk
Content-Type: text/plain; charset=us-ascii

Dear Martin

The reason for giving names to the transects and basins is because they are
numerically a bit imprecise - they will differ from model to model - but still
the Atlantic in one model should be regarded as comparable to the Atlantic in
another model, etc. I agree we do not have a way at present to record precisely
what it means in a given case. Since transects might be jagged lines and coasts
always are, a more general solution than endpoints would be a way to associate
a named line or outline (a geographical feature) with a list of coordinates
that trace it out, perhaps. Is there a use-case for this extra metadata? Has it
been requested in CMIP6?

Best wishes

Jonathan

- Forwarded message from martin.juc...@stfc.ac.uk -

 Date: Mon, 29 Jun 2015 12:31:37 +
 From: martin.juc...@stfc.ac.uk
 To: cf-metadata@cgd.ucar.edu
 Subject: [CF-metadata] Specifying latitude and longitude of transects and
   regions

 Hello All,

 There are some variables in the CMIP5 archive which lack explicit latitude 
 and longitude information, such as mfo (standard name 
 sea_water_transport_across_line) and msftzzzba 
 (ocean_y_overturning_mass_streamfunction_due_to_bolus_advection). The first 
 is a mass flux across a transect and the second is a zonally averaged stream 
 function, averaged across an ocean basin.

 For transects, the mfo variable is encoded with an index dimension and a 
 coordinate passage which labels transects by the name of the passage they 
 cross (e.g. fram_strait). The information about precisely which part of the 
 Fram Strait is used is in a document referred to from the MIP tables. It 
 would be nice to have a means of encoding the end points of the transect in 
 the CF metadata. I looked into using the bounds attribute, but that is 
 defined to represent cell boundaries, so an extension to the convention is, I 
 think, needed. It makes sense to follow the pattern used to represent cell 
 boundaries.  I propose defining a new attribute transect which, like the 
 bounds attribute, can be attached to coordinate variables and takes the 
 name of another variable as its value. The named variable should contain the 
 end points of the transect.

 The same approach could be used for the streamfunction .. which is a mean 
 across an east-west transect (this approach would lead to a specifcation of 
 the east and west endpoints of each basin at each latitude -- but I'm not 
 sure that it would be feasible to collect that much detail in the CMIP data 
 files).

 Is there a cleaner way of encoding transect and basin coordinates?

 regards,
 Martin
 ___
 CF-metadata mailing list
 CF-metadata@cgd.ucar.edu
 http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

- End forwarded message -


--

Message: 2
Date: Mon, 29 Jun 2015 12:38:21 -0400
From: Jim Biard jbi...@cicsnc.org
To: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] How to define time coordinate in GPS?
Message-ID: 5591747d.2010...@cicsnc.org
Content-Type: text/plain; charset=windows-1252; Format=flowed

Nan,

GPS time didn't exist, as such, before 1980-01-06 (year-month-day). It
is a continuous count from the reference date and time of 1980-01-06
00:00:00 using the Gregorian calendar and the UTC time system. UTC

[CF-metadata] Conventions global attribute and the conventions playground

[martin.juckes]

Hello All,

I'm going through the draft NetCDF metadata convention for CMIP6, and would 
like to clean up the way the convention is declared in the NetCDF file. The way 
that Unidata advices users to do this is with an appropriate string in the 
Conventions global attribute, using a space or comma separated list if 
multiple conventions are involved. The problem I have is that we also, of 
course, want to use the CF Convention attribute. The CF specification for this 
attribute say that it should be given the string value 'CF-1.6', which 
appears to rule out the use of multiple conventions as envisioned by Unidata 
and CMIP.

Incidentally, the compliance documents for CF-1.7 and CF-1.6 both still insist 
that the attribute should be set to CF-1.5.

Could the relevant section of paragraph 1.3 in CF version 1.7 be modified to 
read 'the NUG defined attribute Conventions include the string value CF-1.7 
to identify datasets that conform to these conventions.If multiple conventions 
apply, these should be included as space or comma separated list as recommended 
by NUG.'?

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] area_type: convention and usage: bringing the checker and convention into line

[martin.juckes]

Hello Ros,

If you have access to JASMIN, the file is at 
/badc/cmip5/data/cmip5/output1/MIROC/MIROC-ESM/historical/mon/land/Lmon/r1i1p1/latest/landCoverFrac/landCoverFrac_Lmon_MIROC-ESM_historical_r1i1p1_185001-200512.nc.
 Otherwise, select model=MIROC-ESM, experiment=historical and 
variable=landCoverFrac in ESGF and you should find it. It is 644Mbytes, so I 
won't send it as an attachment. 

Regards,
Martin

-Original Message-
From: Rosalyn Hatcher [mailto:r.s.hatc...@reading.ac.uk] 
Sent: 23 March 2015 14:59
To: Juckes, Martin (STFC,RAL,RALSP)
Cc: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] area_type: convention and usage: bringing the 
checker and convention into line

HI Martin,

Can you please point me to where I can get hold of this netcdf file?  The 
checker does check against values in the area_type table so I will need to 
investigate why it's not identifying this in these files.

Regards,
Ros.

On 20 Mar 2015, at 10:09, martin.juc...@stfc.ac.uk wrote:

 Hello,
 
 I have a question about the specification of area_type in the CF Convention 
 and its usage in CMIP5 -- motivated by the need to define how it might be 
 used in CMIP6.
 
 The convention document appears clear:  Some standard names (e.g. region and 
 area_type) are used to indicate quantities which are permitted to take only 
 certain standard values. This is indicated in the definition of the quantity 
 in the standard name table, accompanied by a list or a link to a list of the 
 permitted values. (section 3.3) In the case of area_type, values must be 
 taken from the area_type table.
 
 In the CMIP5 variable request, however, the landCoverFrac variable is 
 defined to have a dimension with standard name area_type that takes values 
 corresponding to the model land cover scheme. Consequently, files have been 
 submitted using terminology chosen by the data providers (e.g. 
 Temperate_Evergreen, Temperate_Deciduous in 
 landCoverFrac_Lmon_MIROC-ESM_historical_r1i1p1_185001-200512.nc). Such files 
 are clearly inconsistent with the convention but they appear to be passed by 
 the CF checker (http://puma.nerc.ac.uk/cgi-bin/cf-checker.pl ).
 
 For CMIP6 we want to have compliant files, of course, but in practise we can 
 only hope to have compliance where there is an automated check. So should we 
 treat the rule about area_type only taking values from the approved list as a 
 recommendation, or should the checker and the CMIP request be adjusted to 
 comply with the existing wording? (Or have I completely misread something?)
 
 regards,
 Martin
 
 
 ___
 CF-metadata mailing list
 CF-metadata@cgd.ucar.edu
 http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] area_type: convention and usage: bringing the checker and convention into line

[martin.juckes]

Hello,

I have a question about the specification of area_type in the CF Convention and 
its usage in CMIP5 -- motivated by the need to define how it might be used in 
CMIP6.

The convention document appears clear:  Some standard names (e.g. region and 
area_type) are used to indicate quantities which are permitted to take only 
certain standard values. This is indicated in the definition of the quantity in 
the standard name table, accompanied by a list or a link to a list of the 
permitted values. (section 3.3) In the case of area_type, values must be 
taken from the area_type table.

In the CMIP5 variable request, however, the landCoverFrac variable is defined 
to have a dimension with standard name area_type that takes values 
corresponding to the model land cover scheme. Consequently, files have been 
submitted using terminology chosen by the data providers (e.g. 
Temperate_Evergreen, Temperate_Deciduous in 
landCoverFrac_Lmon_MIROC-ESM_historical_r1i1p1_185001-200512.nc). Such files 
are clearly inconsistent with the convention but they appear to be passed by 
the CF checker (http://puma.nerc.ac.uk/cgi-bin/cf-checker.pl ).

For CMIP6 we want to have compliant files, of course, but in practise we can 
only hope to have compliance where there is an automated check. So should we 
treat the rule about area_type only taking values from the approved list as a 
recommendation, or should the checker and the CMIP request be adjusted to 
comply with the existing wording? (Or have I completely misread something?)

regards,
Martin


___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Applying multiple conventions

[martin.juckes]

Hello,

There is a proposal floating around to use the UGRID conventions for 
unstructured grids in CMIP6 data. One issue is that the CF Convention specifies 
a specific string for the Conventions and the UGRID convention specific a 
different specific string. Is there any chance of modifying CF to accept, e.g. 
Conventions = CF-1.8, UGRID-0.9?

regards,
Martin
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Dealing with large numbers of flag values in netcdf cf -- what are words

[martin.juckes]

Hello Jonathan,

There does not appear to be any objection to the suggestion below. Does
it need a formal proposal, or can it be put in as a minor clarification?

Cheers,
Martin

 -Original Message-
 From: Jonathan Gregory [mailto:jonat...@met.reading.ac.uk] On Behalf
Of
 Jonathan Gregory
 Sent: 28 October 2009 15:14
 To: Juckes, Martin (STFC,RAL,SSTD)
 Cc: cf-metadata@cgd.ucar.edu; r.s.hatc...@reading.ac.uk
 Subject: [CF-metadata] Dealing with large numbers of flag values in
 netcdf cf -- what are words
 
 Dear Martin
 
 It would be more restrictive than your list, but it would be
 consistent, I
 suppose, to use the same list permitted for netCDF variable/dim/attr
 names viz
 
 alphanumeric characters, underscore '_', period '.', plus '+', hyphen
 '-', or
 at sign '@'
 
 Since these are ASCII char arrays, accented etc chars should be
 excluded
 anyway by that, shouldn't they?
 
 Best wishes
 
 Jonathan
-- 
Scanned by iCritical.
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Dealing with large numbers of flag values in netcdf cf -- what are words

[martin.juckes]

Hello,

Rosalyn has very quickly cleared up one problem I was having: I couldn't
get the cf-checker to accept my string of 800 flag_meanings. The problem
lies in the interpretation of Section 3.5 of the CF Conformance
Requirements and Recommendations:

  
 The type of the flag_meanings attribute is a string whose value is a

 blank separated list of words or phrases (words connected by 
 underscores).

The cf-checker currently interprets words as alpha-numeric characters
only. This excludes hyphens and abbreviations (as in Alaska-St. Elias
Range tundra, Trans-Baikal Bald Mountain tundra), and also / and
,, which occurred in my list. My feeling is that the restriction in
the cf-checker should be relaxed and the text clarified -- but it would
be interesting to know what machine-readability criterea people have in
mind for this flag_meanings attribute.

Regards,
Martin 
 
 -Original Message-
 From: V. Balaji [mailto:v.bal...@noaa.gov]
 Sent: 27 October 2009 19:01
 To: Lawrence, Bryan (STFC,RAL,SSTD)
 Cc: cf-metadata@cgd.ucar.edu; Weatherall, Pauline; Juckes, Martin
 (STFC,RAL,SSTD); Ed Hartnett
 Subject: Re: [CF-metadata] Dealing with large numbers of flag
 valuesinnetcdf cf
 
 Bryan Lawrence writes:
 
  My job is to persist data indefinitely, and I'm relying on
vocabulary
  servers ... (and in particular, Roy's ... ) somehow we will persist
  the vocab.ndg.nerc.ac.uk address, regardless of nerc's future, and
of
  ndg's future, although if ac.uk went away we might be in trouble
:-).
 
 Could happen...
 
 http://www.guardian.co.uk/politics/2009/oct/18/conservatives-defence
 
 I see nerc.co.uk coming...
 --
 
 V. Balaji   Office:  +1-609-452-6516
 Head, Modeling Systems Group, GFDL  Home:+1-212-253-6662
 Princeton UniversityEmail: v.bal...@noaa.gov
-- 
Scanned by iCritical.
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Dealing with large numbers of flag values in netcdf cf -- what are words

[martin.juckes]

Hello Jonathan,

That sounds fine to me. 

Concerning non-ascii -- you may be right.

Cheers,
Martin

 -Original Message-
 From: Jonathan Gregory [mailto:jonat...@met.reading.ac.uk] On Behalf
Of
 Jonathan Gregory
 Sent: 28 October 2009 15:14
 To: Juckes, Martin (STFC,RAL,SSTD)
 Cc: cf-metadata@cgd.ucar.edu; r.s.hatc...@reading.ac.uk
 Subject: [CF-metadata] Dealing with large numbers of flag values in
 netcdf cf -- what are words
 
 Dear Martin
 
 It would be more restrictive than your list, but it would be
 consistent, I
 suppose, to use the same list permitted for netCDF variable/dim/attr
 names viz
 
 alphanumeric characters, underscore '_', period '.', plus '+', hyphen
 '-', or
 at sign '@'
 
 Since these are ASCII char arrays, accented etc chars should be
 excluded
 anyway by that, shouldn't they?
 
 Best wishes
 
 Jonathan
-- 
Scanned by iCritical.
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Dealing with large numbers of flag values innetcdf cf

[martin.juckes]

Thanks for the responses.

Seth is right, I was looking for an alternative to having a gigantic 
flag_meanings attribute. 

I have tried Seth's approach, but can’t get it past the cf-checker. There 
appears to be a character set restriction: I can get the first 547 flag_meaning 
strings accepted if I remove all the -, . and , characters. I'm 
reasonably sure there are no control characters in the following strings which 
could be responsible for the error message that I get (ERROR (3.5): Invalid 
syntax for 'flag_meanings' attribute) when I increase the number of 
flag_meanings to 550. Could it be something to do with the string length (16472 
characters)?

I could try abbreviating the flag_meanings, but this would either be tedious 
work by hand or liable to create confusion if done automatically. If you could 
point me to some information about exactly what string format is acceptable, 
that would be very helpful.
 
To answer John's question: I'm currently using netcdf 3, in IDL -- I'm not sure 
what the options for upgrading to netcdf 4 are.

Cheers,
Matin

 -Original Message-
 From: Seth McGinnis [mailto:mcgin...@ucar.edu]
 Sent: 26 October 2009 23:13
 To: John Graybeal; Juckes, Martin (STFC,RAL,SSTD)
 Cc: cf-metadata@cgd.ucar.edu
 Subject: Re: [CF-metadata] Dealing with large numbers of flag values
 innetcdf cf
 
 On Mon, 26 Oct 2009 12:16:17 -0700
  John Graybeal grayb...@marinemetadata.org wrote:
 Love the question! :-
 
 Are you saying every map location has one index value, which is one of
 the
 numbers from 1 to 827?  So these are mutually exclusive?  Or are
 there 827
 flag values assigned independently for each map location?
 
 It sounds to me like it's the former case, and the problem is that
 concatenating 827 flag values into a single string attribute makes
 something that is easy for a machine to interpret, but impenetrable to
 the human reader.
 
 As I read section 3.5, there are no other options for encoding the
 information according to spec; a gigantic flag_meanings attribute is
 the right answer.
 
 But if the issue is really usability, I think the answer is not to do
 it differently, but just to add an extra attribute that presents the
 same information in a more user-friendly format.  You could add an
 attribute named something descriptive, like category_legend, and
 pair the values  meanings up, one per line.  Then you'd end up with
 something that looks like this:
 
 
 float landtype(yc, xc) ;
 landtype:long_name =Land cover type ;
 landtype:standard_name =land_cover ;
 landtype:flag_values = 1.f, 2.f, 3.f, 4.f, 5.f, 6.f, 7.f, 8.f,
 9.f,
 10.f, 11.f, 12.f, 13.f, 14.f, 15.f, 16.f, 17.f, 18.f, 19.f, 20.f, 21.f,
 22.f,
 23.f, 24 .f;
 landtype:flag_meanings =Urban_and_Built-up_Land
 Dryland_Cropland_and_Pasture Irrigated_Cropland_and_Pasture
 Mixed_Dryland/Irrigated_Cropland_and_Pasture Cropland/Grassland_Mosaic
 Cropland/Woodland_Mosaic Grassland Shrubland Mixed_Shrubland/Grassland
 Savanna
 Deciduous_Broadleaf_Forest Deciduous_Needleleaf_Forest
 Evergreen_Broadleaf
 Evergreen_Needleleaf Mixed_Forest Water_Bodies Herbaceous_Wetland
 Wooden_Wetland Barren_or_Sparsely_Vegetated Herbaceous_Tundra
 Wooded_Tundra
 Mixed_Tundra Bare_Ground_Tundra Snow_or_Ice ;
 landtype:category_legend =\n,
1: Urban and Built-up Land \n,
2: Dryland Cropland and Pasture \n,
3: Irrigated Cropland and Pasture \n,
4: Mixed Dryland/Irrigated Cropland and Pasture \n,
5: Cropland/Grassland Mosaic \n,
6: Cropland/Woodland Mosaic \n,
7: Grassland \n,
8: Shrubland \n,
9: Mixed Shrubland/Grassland \n,
10: Savanna \n,
11: Deciduous Broadleaf Forest \n,
12: Deciduous Needleleaf Forest \n,
13: Evergreen Broadleaf \n,
14: Evergreen Needleleaf \n,
15: Mixed Forest \n,
16: Water Bodies \n,
17: Herbaceous Wetland \n,
18: Wooden Wetland \n,
19: Barren or Sparsely Vegetated \n,
20: Herbaceous Tundra \n,
21: Wooded Tundra \n,
22: Mixed Tundra \n,
23: Bare Ground Tundra \n,
24: Snow or Ice ;
 
 (Sorry for the length, but it gives a real sense of what I mean.)
 
 The flag_values and flag_meanings attributes are still a mess to look
 at, but they're easy to deal with programmatically, while end-users
 can easily look at category_legend and figure out what cover type 22
 is if that's all they're after.
 
 Fundamentally, this is the same issue as the question of how to record
 the grid mapping, and the underlying problem is that while attributes
 allow you to associate key-value pairs with a variable, there's no way
 to nest another associative array within an attribute; the best you
 can do is reference a 

Re: [CF-metadata] Dealing with large numbers of flag valuesinnetcdf cf

[martin.juckes]

Hello Roy,

That is an interesting idea. There are definitions of these areas on a WWF 
site, of the form
http://www.worldwildlife.org/wildworld/profiles/terrestrial/nt/nt0908_full.html 
where nt0908 is a tag associated with a particular flag value. 

My first thought was:
URI= 
http://www.worldwildlife.org/wildworld/profiles/terrestrial/[flag_meaning]_full.html;
 with nt/nt0908 in the flag_meanings attribute, but it appears that the 
cf-checker does not like having a / in flag_meanings. This means we need to 
either have a more complex syntax, or set up our definition URLs along the 
lines you suggest.

A more complex syntax might be:
URI= 
http://www.worldwildlife.org/wildworld/profiles/terrestrial/[flag_meaning][0:2]/[flag_meaning]_full.html;

Since the above returns html rather than skos/xml it should perhaps be 
complemented by a uri_content_type attribute, with value 'html/text'.

I'll think about the option of setting up something like your vocab server -- 
or possibly putting some more definitions in the ndg vocab server?

Regards,
Martin 

 -Original Message-
 From: Lowry, Roy K [mailto:r...@bodc.ac.uk]
 Sent: 27 October 2009 11:07
 To: Juckes, Martin (STFC,RAL,SSTD); cf-metadata@cgd.ucar.edu
 Cc: Weatherall, Pauline
 Subject: RE: [CF-metadata] Dealing with large numbers of flag
 valuesinnetcdf cf
 
 Hello Martin,
 
 There is another possible solution to your problem, which we are
 looking at for dealing with a data source flag to be used with the
 GEBCO bathymetric grid.  This is to put a URI base into an attribute
 that when concatenated with a flag values gives the flag definition
 from a vocabulary server.  For example, having a flag_definition
 attribute like:
 
 URI_base='http://vocab.ndg.nerc.ac.uk/term/GGS1/current/'
 
 For a flag value of 1, the resulting URL is:
 
 http://vocab.ndg.nerc.ac.uk/term/GGS1/current/1
 
 which is a live NERC Vocabulary Server URL returning the flag
 definition attributes in a SKOS document.
 
 I appreciate that this moves away from fully self-contained CF files
 and that a standardised syntax for the URI base encoding is required,
 but I think this provides a scalable solution to the flag definition
 problem.
 
 Cheers, Roy.
 
 
 -Original Message-
 From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-
 boun...@cgd.ucar.edu] On Behalf Of martin.juc...@stfc.ac.uk
 Sent: 27 October 2009 10:45
 To: cf-metadata@cgd.ucar.edu
 Subject: Re: [CF-metadata] Dealing with large numbers of flag values
 innetcdf cf
 
 Thanks for the responses.
 
 Seth is right, I was looking for an alternative to having a gigantic
 flag_meanings attribute.
 
 I have tried Seth's approach, but can’t get it past the cf-checker.
 There appears to be a character set restriction: I can get the first
 547 flag_meaning strings accepted if I remove all the -, . and ,
 characters. I'm reasonably sure there are no control characters in the
 following strings which could be responsible for the error message that
 I get (ERROR (3.5): Invalid syntax for 'flag_meanings' attribute) when
 I increase the number of flag_meanings to 550. Could it be something to
 do with the string length (16472 characters)?
 
 I could try abbreviating the flag_meanings, but this would either be
 tedious work by hand or liable to create confusion if done
 automatically. If you could point me to some information about exactly
 what string format is acceptable, that would be very helpful.
 
 To answer John's question: I'm currently using netcdf 3, in IDL -- I'm
 not sure what the options for upgrading to netcdf 4 are.
 
 Cheers,
 Matin
 
  -Original Message-
  From: Seth McGinnis [mailto:mcgin...@ucar.edu]
  Sent: 26 October 2009 23:13
  To: John Graybeal; Juckes, Martin (STFC,RAL,SSTD)
  Cc: cf-metadata@cgd.ucar.edu
  Subject: Re: [CF-metadata] Dealing with large numbers of flag values
  innetcdf cf
 
  On Mon, 26 Oct 2009 12:16:17 -0700
   John Graybeal grayb...@marinemetadata.org wrote:
  Love the question! :-
  
  Are you saying every map location has one index value, which is one
 of
  the
  numbers from 1 to 827?  So these are mutually exclusive?  Or are
  there 827
  flag values assigned independently for each map location?
 
  It sounds to me like it's the former case, and the problem is that
  concatenating 827 flag values into a single string attribute makes
  something that is easy for a machine to interpret, but impenetrable
 to
  the human reader.
 
  As I read section 3.5, there are no other options for encoding the
  information according to spec; a gigantic flag_meanings attribute is
  the right answer.
 
  But if the issue is really usability, I think the answer is not to do
  it differently, but just to add an extra attribute that presents the
  same information in a more user-friendly format.  You could add an
  attribute named something descriptive, like category_legend, and
  pair the values  meanings up, one per line.  Then you'd end up with
  something that looks like this:
 

Re: [CF-metadata] Dealing with large numbers of flag valuesinnetcdf cf

[martin.juckes]

Hello Ed,

To Roy's comment I'll add that I would like to complement in-file metadata with 
additional online info, not replace it. E.g. for the tag NT0908 I have a name 
Paraná flooded savanna which can go in the file, but there is a web page 
giving the full characterisation of what is meant by Paraná flooded savanna, 
and this includes an extended description, references and photographs. It would 
be unreasonable to include all these photos etc in the file, so a convenient 
means of referring to them would be helpful. 

Regards,
Martin

 -Original Message-
 From: Lowry, Roy K [mailto:r...@bodc.ac.uk]
 Sent: 27 October 2009 15:45
 To: Ed Hartnett; Juckes, Martin (STFC,RAL,SSTD)
 Cc: cf-metadata@cgd.ucar.edu; Weatherall, Pauline
 Subject: RE: [CF-metadata] Dealing with large numbers of flag
 valuesinnetcdf cf
 
 Hi Ed,
 
 Establishing on-line metadata resources with permanence is the name of
 my game.  If I didn't have confidence it would be around in 15 years I
 wouldn't be promoting it.
 
 Cheers, Roy.
 
 -Original Message-
 From: Ed Hartnett [mailto:e...@unidata.ucar.edu]
 Sent: 27 October 2009 15:37
 To: martin.juc...@stfc.ac.uk
 Cc: Lowry, Roy K; cf-metadata@cgd.ucar.edu; Weatherall, Pauline
 Subject: Re: [CF-metadata] Dealing with large numbers of flag
 valuesinnetcdf cf
 
 martin.juc...@stfc.ac.uk writes:
 
  Hello Roy,
 
  That is an interesting idea. There are definitions of these areas on
 a WWF site, of the form
 
 http://www.worldwildlife.org/wildworld/profiles/terrestrial/nt/nt0908_f
 ull.html where nt0908 is a tag associated with a particular flag
 value.
 
  My first thought was:
  URI=
 http://www.worldwildlife.org/wildworld/profiles/terrestrial/[flag_meani
 ng]_full.html with nt/nt0908 in the flag_meanings attribute, but it
 appears that the cf-checker does not like having a / in
 flag_meanings. This means we need to either have a more complex syntax,
 or set up our definition URLs along the lines you suggest.
 
  A more complex syntax might be:
  URI=
 http://www.worldwildlife.org/wildworld/profiles/terrestrial/[flag_meani
 ng][0:2]/[flag_meaning]_full.html
 
  Since the above returns html rather than skos/xml it should perhaps
 be complemented by a uri_content_type attribute, with value
 'html/text'.
 
  I'll think about the option of setting up something like your vocab
 server -- or possibly putting some more definitions in the ndg vocab
 server?
 
  Regards,
  Martin
 
 Howdy all!
 
 I really don't think you should store metadata for a file on the
 web. Just put it in the file.
 
 Most scientific data are around a lot longer than most URLs. What
 happens in 15 years when someone wants to understand this data, and the
 URL has vanished? It the metadata were stored in the file, then it
 would
 be there for that programmer of the future. (And that programmer may
 well be you! insert scary Halloween laughter here)
 
 If the metadata seem too large to put in the file, consider storing
 them
 in variables, and then use netCDF-4 with compression turned on for
 those
 variables.
 
 Thanks,
 
 Ed
 
 
 
 --
 Ed Hartnett  -- e...@unidata.ucar.edu
 
 --
 This message (and any attachments) is for the recipient only. NERC
 is subject to the Freedom of Information Act 2000 and the contents
 of this email and any reply you make may be disclosed by NERC unless
 it is exempt from release under the Act. Any material supplied to
 NERC may be stored in an electronic records management system.

-- 
Scanned by iCritical.
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Dealing with large numbers of flag values in netcdf cf

[martin.juckes]

Hello,

 

I wonder if anyone can help with this problem:

 

I have a file with a map of index values, ranging from 1 to 827. The
flag meanings range from Admiralty Islands lowland rain forests to
Zambezian halophytics. I'm reluctant to combine all 827 flag meanings
into a single string for the flag_meanings attribute - is there a
better way?

 

Note that this is not my dataset, so that while suggestions for
presenting the data in a different way may be useful, they won't solve
my current problem. The dataset is a eco-region analysis distributed by
the WWF and has wide usage in its present form. 

 

Sincerely,

Martin Juckes


-- 
Scanned by iCritical.

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata