Re: [CF-metadata] Are ensembles a compelling use case for "group-aware" metadata? (CZ)

2013-09-19 Thread Schultz, Martin
Hi Charlie, very good and extensive explanation of the potential use for groups and group-aware metadata. Yet, I have a few remarks (which may in part reveal that I should probably read the preamble of the CF convention again ;-): > Point 1: How does the user know she has all the realizatio

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Charlie Zender
Hi Corey, Le 19/09/2013 09:05, Corey Bettenhausen a écrit : > > On Sep 19, 2013, at 11:29 AM, Karl Taylor wrote: > >> Hi all, >> >> Again, I may be unaware of all the possible uses of hierarchies, but here's >> our experience with CMIP. >> >> It seems to me if hierarchies are for the purpose of

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Charlie Zender
Hello Nan, Le 19/09/2013 12:16, Nan Galbraith a écrit : > Hi all - >> Did I misunderstand the original proposal? > > Could we possibly have a proposal written up as a ticket on the Trac > system? I'd prefer not to submit anything to the Trac system until/unless there is consensus from many doubt

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread John Graybeal
On Sep 19, 2013, at 13:58, Charlie Zender wrote: > Nothing in our proposal mandates a single way of representing hierarchies. Au > contraire. The concern by the hierarchy-wary is not that you'd mandate a single way, but that by providing an open mechanism, you would encourage a raft of compet

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Charlie Zender
Hi Steve, Your stern but steady caution about the tension between flexibility and interoperability is well-taken here. > What have been the down sides to the > use of groups and hierarchies? Those who respond to this please be careful to distinguish between HDF5 _allowed_ hierarchies and netCDF4

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Charlie Zender
Hi Karl, Please see new thread Are ensembles a compelling use case for "group-aware" metadata? It is a thinly veiled reference to CMIP5, which I don't mention by name so as not to associate it/you with our proposal. Nevertheless, many on the list know what CMIP5 is and it has helped climate mode

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Charlie Zender
Hello Steve, Responses interleaved. Best, cz Le 18/09/2013 09:32, Steve Hankin a écrit : > > > On 9/18/2013 7:56 AM, Roy Mendelssohn - NOAA Federal wrote: >> Hi All: >> >> NASA has used hierarchies for years, and appears committed to them. So, >> either it is done in an ad hoc way, or throug

Re: [CF-metadata] Are ensembles a compelling use case for "group-aware" metadata?

2013-09-19 Thread Russ Rew
Charlie, Great use case, clearly explained ... It also demonstrates a potential need for "Group dimensions", so instead of groups cesm_01 and cesm_02, it would be possible to refer to Groups cesm[1] and cesm[2], supporting loops over contents of closely related groups without having to invent a c

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Charlie Zender
Hi Roy, Thanks for your latest 3 posts. They all make the point that hierarchical files are widely used now. In this post you make the point that maybe having conventions on how to use/interpret hierarchies would be helpful, regardless of its desirability for CF. Yes, part of our motivation is be

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Charlie Zender
Hi Bryan, Responses interleaved. Best, cz Le 18/09/2013 05:57, Bryan Lawrence a écrit : > Hi Charlie > > (Before I disagree with you, like most on the list, I'm glad we're > having the conversation on this topic, it needs to be had, so thanks!) > > I find this particular example completely unh

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Karl Taylor
Hi all, No need to respond further to my post. I missed that the groups were *within* the file. I have now looked at the netCDF documentation about groups and will hopefully contribute more cogent remarks later. cheers, Karl On 9/19/13 9:05 AM, Corey Bettenhausen wrote: On Sep 19, 2013, a

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups

2013-09-19 Thread Charlie Zender
Hi Martin, Interesting point of view. Le 18/09/2013 09:43, Schultz, Martin a écrit : > Dear all, > > maybe we should rephrase the question behind this discussion: How can > hierarchies and/or groups be implemented without breaking CF? CF datasets can certainly be aggregated without loss

[CF-metadata] Are ensembles a compelling use case for "group-aware" metadata?

2013-09-19 Thread Charlie Zender
Hello, Thank you all for input on potentical "group-aware" CF extensions. I do my best to respond to new points and apologies to those I miss. New top-level thread to discuss use cases of, and potential solutions to, research questions relevant to "group-aware" metadata. Some wish to see a compel

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Nan Galbraith
Hi all - Did I misunderstand the original proposal? Could we possibly have a proposal written up as a ticket on the Trac system? I'm finding it difficult to find a description of what's being suggested, in the many emails in this thread. I realize we're very far from a detailed description

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Steve Hankin
On 9/19/2013 9:05 AM, Corey Bettenhausen wrote: On Sep 19, 2013, at 11:29 AM, Karl Taylor wrote: Hi all, Again, I may be unaware of all the possible uses of hierarchies, but here's our experience with CMIP. It seems to me if hierarchies are for the purpose of "organizing" datasets (or orga

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Corey Bettenhausen
On Sep 19, 2013, at 11:29 AM, Karl Taylor wrote: > Hi all, > > Again, I may be unaware of all the possible uses of hierarchies, but here's > our experience with CMIP. > > It seems to me if hierarchies are for the purpose of "organizing" datasets > (or organizing a bunch of files), this should

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Karl Taylor
Hi all, Again, I may be unaware of all the possible uses of hierarchies, but here's our experience with CMIP. It seems to me if hierarchies are for the purpose of "organizing" datasets (or organizing a bunch of files), this should fall outside CF's purview because a single hierarchy is rarel

Re: [CF-metadata] Towards recognizing and exploiting hierarchical groups (Charlie Zender - Steve Hankin - Richard Signell)

2013-09-19 Thread Corey Bettenhausen
On Sep 18, 2013, at 12:32 PM, Steve Hankin wrote: > > > On 9/18/2013 7:56 AM, Roy Mendelssohn - NOAA Federal wrote: >> Hi All: >> >> NASA has used hierarchies for years, and appears committed to them. So, >> either it is done in an ad hoc way, or through a standard. That doesn't >> mean CF i