Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Those changes look fine to me, thanks, David. The clarification is useful. -- Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-1018539726__;Iw!!G2kpM7uM-TzIFchu!n52Q_2vu2MgVKAgGbQHB2qrfDSyveZSnPwKbYTRh97A-h0kxaXmRX0O4K4sV4fCKzPFuKM9w7D4$ You are receiving this because you are subscribed to this thread. Message ID: This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear Jonathan, I have made some commits that hopefully make clearer the relationship between CF-netCDF mesh topology and location index variables, and CF data model Domain and Domain Topology constructs. What do you think? https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/commit/3f9b4d53421f7da8c380c7e489e7713140edc60a?short_path=84395ff*diff-84395ffc95113b0c4c0cd22056e9b7461dc81bec9c89422b66c7b14f73d6c788__;Iw!!G2kpM7uM-TzIFchu!mykjhg-QqYXYxJn78C2m0SdkjwZ5T1TyvH175PbvbVeNaVOWAjxxlzvolSqOOCB_zUzEHYM2NBQ$ https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/commit/c401fae05d15f4aba7df1a2fe69163a8f00d389e?short_path=e4e016c*diff-e4e016c9bc64bd9cea792809a49d188836d6050352f0c24da156b896f8a3b6cf__;Iw!!G2kpM7uM-TzIFchu!mykjhg-QqYXYxJn78C2m0SdkjwZ5T1TyvH175PbvbVeNaVOWAjxxlzvolSqOOCB_zUzEBdX4gJY$ -- Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-1018505924__;Iw!!G2kpM7uM-TzIFchu!mykjhg-QqYXYxJn78C2m0SdkjwZ5T1TyvH175PbvbVeNaVOWAjxxlzvolSqOOCB_zUzE8fWWLY4$ You are receiving this because you are subscribed to this thread. Message ID: This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear Jonathan, Thank you for your comments. > In Appendix I (the data model), you describe the new CF-netCDF element as > "Domain(s) with cell connectivity" and the new CF construct as "Connectivity > of domain cells". I wonder if these should be the same. Good point, In this case I think they should different, though. For instance, in the first new example in the new section 5.9 "Mesh Topology Variables" (which is adapted from https://urldefense.us/v3/__https://ugrid-conventions.github.io/ugrid-conventions/*2d-triangular-mesh-topology__;Iw!!G2kpM7uM-TzIFchu!i3j8zHRk_Fphfr-WluLHOEojaVLHa0uyJCSzU52a9pXQBkO_wevMDymzyrbrREaOM-U9UEXFfeo$ ) , the mesh topology variable `Mesh2` contains nodes, edges and faces, each of which corresponds to a self-contained CF data model domain construct. I.e. this mesh topology variable represents three domains (in the data model sense of "domain"):  (one domain for the 2 triangular faces, one for the 5 edges, and one for the 4 nodes.) By contrast, in the data model we still do not formally recognise the _inter_-domain connectivity (e.g. the relationship between edges and faces; or the relationship between stagger locations on an Arakawa grid), but the existing domain construct does require the new domain topology construct component to represent the _intra_-domain connectivity required by UGRID. I feel that this distinction should be better illuminated in the appendix I text ... I'll see how I can work that in. > In ch01 there is an accidental "accidential". Fixed (I'll push it up with the appendix I changes discussed above). > I note that many of the changes in > https://urldefense.us/v3/__https://github.com/cf-convention/cf-convention.github.io/pull/210/files__;!!G2kpM7uM-TzIFchu!i3j8zHRk_Fphfr-WluLHOEojaVLHa0uyJCSzU52a9pXQBkO_wevMDymzyrbrREaOM-U9DFg5Bug$ > are concerned with respelling github as GitHub. It's fine to put that > right. In the unlikely event that this PR doesn't get accepted, we should > remember to do that anyway. You're quite right. (I shouldn't really do that, but I got carried away during a spell check :)) By the way, I very recently found out that GitHub now has nice image comparison feature when you do a "rich diff" on an image file, that includes side by side, and overlayed views (e.g. https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/pull/353/files?short_path=5a70e4b*diff-5a70e4b347e2935d49b12cfaff78556fb70319f62762e966b1c82042f022d463__;Iw!!G2kpM7uM-TzIFchu!i3j8zHRk_Fphfr-WluLHOEojaVLHa0uyJCSzU52a9pXQBkO_wevMDymzyrbrREaOM-U9bNStxFc$ and https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/pull/353/files?short_path=e4e016c*diff-e4e016c9bc64bd9cea792809a49d188836d6050352f0c24da156b896f8a3b6cf__;Iw!!G2kpM7uM-TzIFchu!i3j8zHRk_Fphfr-WluLHOEojaVLHa0uyJCSzU52a9pXQBkO_wevMDymzyrbrREaOM-U9w0HmcLo$ ). -- Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-1018360298__;Iw!!G2kpM7uM-TzIFchu!i3j8zHRk_Fphfr-WluLHOEojaVLHa0uyJCSzU52a9pXQBkO_wevMDymzyrbrREaOM-U9h1AGcYs$ You are receiving this because you are subscribed to this thread. Message ID: This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear @davidhassell Thanks for summarising the discussion up to now and writing the pull requests. As far as I know, this covers everything. The proposed changes look fine to me from a CF point of view. I noticed three small things: * In Appendix I (the data model), you describe the new CF-netCDF element as "Domain(s) with cell connectivity" and the new CF construct as "Connectivity of domain cells". I wonder if these should be the same. * In ch01 there is an accidental "accidential". * I note that many of the changes in https://urldefense.us/v3/__https://github.com/cf-convention/cf-convention.github.io/pull/210/files__;!!G2kpM7uM-TzIFchu!nGv4oHnK7P-nBQBeVRa-BOwc061GPhN2jyh0ER2XceyotMMWJKk7dTxgFuQ0IAB_YK2w9zkmYzY$ are concerned with respelling github as GitHub. It's fine to put that right. In the unlikely event that this PR doesn't get accepted, we should remember to do that anyway. Cheers Jonathan -- Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-1017713484__;Iw!!G2kpM7uM-TzIFchu!nGv4oHnK7P-nBQBeVRa-BOwc061GPhN2jyh0ER2XceyotMMWJKk7dTxgFuQ0IAB_YK2wQr1B64A$ You are receiving this because you are subscribed to this thread. Message ID: This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Hello all, I have created a couple of pull requests to hopefully finally get UGRID into CF. I've not consulted anyone on the new text yet, so I fully expect some constructive comments! but I thought it a good idea to have something that we can discuss in less abstract terms. I have copied the agreed procedure from previous comments, and mapped them to new parts of the pull requests (one here and one over on https://urldefense.us/v3/__https://github.com/cf-convention/cf-convention.github.io__;!!G2kpM7uM-TzIFchu!lOoPYeWtDSqIZQsKNctTh7mqIX59dfz39Aur1lMQnlR_AKsuhnoT2eRf4jNkNezdlbDu_Y5ms58$ ). Thanks, David ### Procedure for incorporating UGRID into CF 1. UGRID will remain an independent convention, with independent governance. Agreed. 2. Comprehensive conformance rules will be written up for UGRID. These will be maintained alongside UGRID in its repository, and referenced from (not copied into) the CF conformance document. See PR #353 (conformance). The link to the UGRID conformance is a placeholder, as they are still being written. 3. The rules governing the evolution of CF will be modified to ensure that the CF conventions remain consistent with URGID. Whilst UGRID will not be formally constrained by the CF conventions, it will be in everyone’s interest for UGRID changes to be evaluated for compatibility with CF, and vice versa. If desirable changes in one convention would require changes in the other convention, then such changes should be proposed to the other community and discussed together as part of a shared interest. Different time scales of evolution between the two standards are accounted for, as CF will only formally accept a named version (or versions) of UGRID and so is not necessarily obliged to support the latest version of UGRID (although that would generally be the expectation). In the very unlikely event that changes to UGRID can not be accepted by CF then it would be the time to merge the last set of accepted UGRID conventions into the actual CF document, from where the CF representation of unstructured grids would evolve independently to UGRID. See PR https://urldefense.us/v3/__https://github.com/cf-convention/cf-convention.github.io/pull/210__;!!G2kpM7uM-TzIFchu!lOoPYeWtDSqIZQsKNctTh7mqIX59dfz39Aur1lMQnlR_AKsuhnoT2eRf4jNkNezdlbDu1DY2Yv4$ (Additional recommendations relating to UGRID) 4. A subsection will be added to CF section 1: Introduction to introduce UGRID and its purpose, to make clear its special synergy with CF, to remark on the appearance of attributes in CF appendices, and to say that it has its own conformance document which complements the CF conformance document. See PR #353 (section 1) 5. The standardised UGRID attributes will be documented in the CF conventions, thereby making them visible to all users, and it will be mentioned in the CF governance rules that they need maintaining. Only the UGRID attributes which can appear on data variables (currently `mesh`, `location` and `location_index_set`) should be added to CF appendix A: Attributes. All other UGRID attributes (such as those on mesh variables) should go into a new CF appendix specifically about the UGRID mesh topology variable. This would be like the treatment of the attributes of the grid mapping variable, which are in a table in CF appendix F: Grid Mappings, not in appendix A. (It is acknowledged that the geometry variable attributes appear in CF appendix A, but there are only five of them, whereas there are eighteen attributes of the mesh topology variable.) See PR #353 (appendix A, appendix K) 6. Some text will be added to CF section 5.8: Domain Variables to explain the UGRID mesh topology variable and how it relates to a CF domain variable. It may be the case that the occasional note relating to UGRID would be useful in other sections. See PR #353 (section 5). I took the approach of creating a new section (5.9 Mesh Topology Variables) instead, which references the CF domain. 7. It will be decided whether or not UGRID fits into the existing CF data model (defined in CF appendix I: The CF Data Model), and if not then the CF data model will be extended to accommodate UGRID. The understanding of interested parties from both the UGRID and CF communities coalesced on the need for a new CF data model construct that can make explicit the notion of a network topology for CF domain constructs. See PR #353 (appendix I) -- Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-1017626734__;Iw!!G2kpM7uM-TzIFchu!lOoPYeWtDSqIZQsKNctTh7mqIX59dfz39Aur1lMQnlR_AKsuhnoT2eRf4jNkNezdlbDuwa7Jw2U$ You are receiving this because you are subscribed to this thread. Message ID: This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Thanks guys, i think I get in now -- the ocean basins example is a good one, I didn't really know what that meant the first time it was mentioned. "axis" it is :-) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-890001573__;Iw!!G2kpM7uM-TzIFchu!ljc5IpyrRiKWfFzPJJBn7bWUViqc-bLuMmly0RUboD6t2NdakqDYlYeWMUeDVW1h-PMofcuKB4E$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear Chris and David I agree that UGRID uses a discrete axis. There isn't necessarily any ordering implied by such an axis, just as in the case of UGRID. For example, ocean basins or other geographical regions may be arranged along a discrete axis in any order. Best wishes Jonathan -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-889723508__;Iw!!G2kpM7uM-TzIFchu!gmsJ5n97suQ8Qv6JSDVPOtFN9OQ4i7t3fxcUq2w0hmV6LpYm56EByJ8f5D9rcHusEvYZJrIpD1Q$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Hi @ChrisBarker-NOAA, This puzzled me for a bit, until I remembered that in the CF data model, DSGs are _not_ special. This is because what looks like a discrete axis in the encoding of a DSG is a really just a form of lossless packing for an orthogonal multidimensional array (with the data and coordinate arrays padded with missing data if required). This is indeed different to the UGRID mesh, as you rightly picked up on. So bringing DSG's into this was misleading - sorry! The [all that the CF data model says about DSGs](https://urldefense.us/v3/__https://cfconventions.org/cf-conventions/cf-conventions.html*_domain_axis_construct_and_the_data_array__;Iw!!G2kpM7uM-TzIFchu!kpIRMVftXZrAcR1WIvvNZ6ctbN1jxRTEXLA68dt7V_fuRrMa6_J0r1EyT1N_goBO3Cqr0opq5P4$ ) is: _When a collection of discrete sampling geometry (DSG) features has been combined in a data variable using the incomplete orthogonal or ragged representations to save space, the axis size has to be inferred, but this is an aspect of unpacking the data, rather than its conceptual description. In practice, the unpacked data array may be dominated by missing values (as could occur, for example, if all features in a collection of time series had no common time coordinates), in which case it may be preferable to view the collection as if each DSG feature were a separate variable, each one corresponding to a different field construct._ In my other examples (such as data stored on ocean basins) there is no assumption of (spatial) continuity, so I reckon that the CF data model discrete axis does apply to the UGRID case, after all. Thanks, David -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-889701172__;Iw!!G2kpM7uM-TzIFchu!kpIRMVftXZrAcR1WIvvNZ6ctbN1jxRTEXLA68dt7V_fuRrMa6_J0r1EyT1N_goBO3Cqr1s8_kIw$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
thanks @JonathanGregory: that is better. Final note on "axis" -- I think I'm being pedantic here, and it's probably bad to introduce new terms, but: David wrote: > A "discrete axis" in CF is one which does not correspond to a continuous > physical quantity, for example, this is the case for an axis that runs over > ocean basins or area types, or for a domain axis that indexes a time series > at scattered points. This also applies to the axis that stores the nodes of a > UGRID mesh. Does it though? > In many of these cases the discrete axis is sampling a higher dimensional > space, as you say. This where the "discrete sampling geometries" of chapter 9 > get their name (I always imagine a ship sailing along a sinuous course and > recording SSTs at various time intervals). A domain that has such a discrete > domain axis construct can also have other domain axis constructs (such as > ones for time, level, etc.) which are indeed orthogonal to each other and to > the discrete axis. In all of these examples, the axis may be "wandering around" in a higher dimensional space, but as noted, they are still orthogonal to others, and more critically, they represent a continuum of some sort. That is, where along the axis they lie is meaningful -- the fact that a value comes before or after another value, or is "next to" a value is meaningful. But in the case of a unstructured grid's node array, for instance, the order in which the node coordinates are in the array is completely (well, not completely, but ...) arbitrary. It is a mapping between a node index and the coordinates of that node, nothing more -- it could be completely re-ordered without its meaning changing. Which is why I don't think of it as an axis, even if it technically fits the definition. But as I said, probably good enough, and I can't think of a better term. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-889300399__;Iw!!G2kpM7uM-TzIFchu!i2eIVZeGabCzH4ExsKvKNXNdu7uMRyewW_ZrajyYW6qm5tgx1n2Ije2xVYhT4Tx8M_z_er0a42c$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear @JonathanGregory, Thanks for your comments. > I think that "unlike" implies it's somehow inconsistent ... Agreed. With your new text, that last paragraph becomes In CF-netCDF a domain topology can only be provided for a domain defined by a UGRID mesh topology variable. In this case, the connectivity array is supplied by a UGRID connectivity variable, such as a "face_face_connectivity" variable. **The information represented by the symmetrical connectivity array of the domain topology construct in the CF data model is stored in a different but equivalent way in UGRID. The trailing dimension of a UGRID connectivity variable's data records, for each cell, the indices of the other cells to which it is connected (padded with missing data if a cell has fewer connections than some others).** > What role do the edge coordinates have in the domain which refers to the > faces? That sounds like the right question, and I think the answer is "none", which clarifies for me that [my initial concern about "round-tripping"](https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-887768641__;Iw!!G2kpM7uM-TzIFchu!npi0cAUkgsn2Aau2vThp0Uhvt3R2ZkQ5Duq3d1Q2HjJBF0MPtxOnQg8tCC3OtvWiMdmZzePrWEk$ ) is not actually relevant, here. How a software implementation decides, when writing multiple fields to a file, to avoid the duplication and proliferation of domain-related netCDF variables is up to it, as it always has been. All the best, David -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-889263224__;Iw!!G2kpM7uM-TzIFchu!npi0cAUkgsn2Aau2vThp0Uhvt3R2ZkQ5Duq3d1Q2HjJBF0MPtxOnQg8tCC3OtvWiMdmZi84W30Y$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear Chris and David Thanks for this discussion. In David's revised text > Unlike the domain topology construct's connectivity array, a > UGRID connectivity variable's data is not stored as a symmetric matrix that > indicates the connectivity between any two cells. Instead, the trailing > dimension of ... I think that "unlike" implies it's somehow inconsistent. I would suggest something like, "The information represented by the symmetrical connectivity array of the domain topology construct in the CF data model is stored in a different but equivalent way in UGRID. The trailing dimension of ..." David wrote, "If there are edge coordinates as part of a mesh with faces ... the edge coordinates can only be represented by the CF data model in a second domain that comprised 1-d cells defining each edge and how it's connected to others." What role do the edge coordinates have in the domain which refers to the faces? Best wishes Jonathan -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-888999726__;Iw!!G2kpM7uM-TzIFchu!lJmFpLOcoHlaxoUh2SugObq1husFhqGkHu-34GeVsR4zLk-aG3zmozZG6FqkplKpaYXTPyOOlBs$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Hi @ChrisBarker-NOAA, > The whole concept of an "axis" matches orthogonal coordinates of some sort. > (that's kind the definition of orthogonal, yes?). Mapping a Ugrid to the real > world, the real world is 2D (Or 3D, but let's not go there yet) -- and that > 2D world has 2 orthogonal axis -- X, Y, Lat, Long, whatever. So a mesh > represents a 2D space -- but I wouldn't say it has two axes, or a 1-d > discrete axis either, ir is simply something else. Or "axis" means something > different in this context than I think it does. I hope I can clear this up a bit ... A "discrete axis" in CF is one which does not correspond to a continuous physical quantity, for example, this is the case for an axis that runs over ocean basins or area types, or for a domain axis that indexes a time series at scattered points. This also applies to the axis that stores the nodes of a UGRID mesh. In many of these cases the discrete axis is sampling a higher dimensional space, as you say. This where the "discrete sampling geometries" of chapter 9 get their name (I always imagine a ship sailing along a sinuous course and recording SSTs at various time intervals). A domain that has such a discrete domain axis construct can also have other domain axis contructs (such as ones for time, level, etc.) which are indeed orthogonal to each other and to the discrete axis. > That means there could be separate definitions, or data variables could share > the same one, yes? If, for instance, multiple variables in the same file are > on the same mesh, they share a single domain definition, yes? Yes. Ish. Say we have two data variables in a file (temperature and precipitation, say) that are both defined on the faces of the same mesh. The CF data model currently views all field constructs (i.e. data variables in this context) as independent entities, so each of the resulting two field constructs would contain its own domain, and those domains can only, in the data model world, be seen to be equal by inspection. This is different to the [ISO 19123 coverage model](https://urldefense.us/v3/__https://www.iso.org/obp/ui/*iso:std:iso:19123:ed-1:v1:en__;Iw!!G2kpM7uM-TzIFchu!mKc8eKpI4oK_LVEInhm9uLhMqlotAzTZUKB3NH_XFq73vfIuwCvkKsmkysbNhqFoCYv7fWY7bT0$ ) that allows for multiple "features" (i.e. data variables) to be defined at common locations (this is discussed [section 5.1 of the CF data model GMD paper](https://urldefense.us/v3/__https://gmd.copernicus.org/articles/10/4619/2017/gmd-10-4619-2017.pdf__;!!G2kpM7uM-TzIFchu!mKc8eKpI4oK_LVEInhm9uLhMqlotAzTZUKB3NH_XFq73vfIuwCvkKsmkysbNhqFoCYv7lNzAYhM$ )). CF has always had this feature (problem?), even with traditionally defined domains. We're back in SGRID territory here - and one day the CF conventions will formalise domain sharing and linked domains, but until then we're stuck with this theoretical independence. I don't think that this is in any way a blocker for the integration of UGRID into CF, though. Your software can explicitly share domains in the ISO 19123 vein if it wants to. > That makes sense but the language I first noticed said a "symmetric array" -- > I think we need to be careful about the abstraction of a symmetric matrix and > the realization of an "array" in code, or a netcdf file, or ... And it needs > to be clear that it's a abstract (sparse) boolean matrix, rather than an > actual array (necessarily). I hear you, but would like to keep "array", as that is used in the same context throughout the data model description. I think we can improve things by explicitly noting the difference between logical representation and encoding in the text (changes in **bold**, I've also made a few other changes mentioned above). How's this? ### Domain topology construct A domain topology describes the connectivity of domain cells indexed by a subset of the domain axis constructs. When two cells are connected, operations on the data stored on them may be assumed to be continuous across their common boundary. A domain topology construct describes logically and explicitly the domain topology of cells indexed by a single domain axis construct. A domain topology construct contains a connectivity array that spans a single domain axis construct with the addition of an extra dimension of the same size, such that each dimension indexes the cells. The array is symmetrical, and each element indicates whether the pair of cells to which its indices refer are connected. **The connectivity of a cell to itself is undefined, so the diagonal elements of this array are ignored**. A domain construct may contain at most one domain topology construct. For any subset of the domain axis constructs, excluding a domain axis construct for which there is a domain topology construct, there is an implicit domain topology that is defined by a function of the physical contiguousness of the cells, and/or the nature of the real world or simulated processes that produced the
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
I think I'm (slowly) getting this, but a few thoughts on language: "the CF data model is independent of the encoding, and the symmetric matrix is logically what is going on here." That makes sense but the language I first noticed said a "symmetric array" -- I think we need to be careful about the abstraction of a symmetric matrix and the realization of an "array" in code, or a netcdf file, or ... And it needs to be clear that it's a abstract (sparse) boolean matrix, rather than an actual array (necessarily). "The nodes (i.e. bounds) and edges do not have an independent existence - they are elements of the CF cell definition. The new domain topology construct makes explicit the cell connectivities." That makes sense -- that is the whole point after all :-) " A "domain axis" is essentially a dimension of the domain. We restrict the new domain topology constructs to apply to a single domain axis simply because there is no current way of encoding a domain topology construct that applies to multiple domain axes. UGRID only describes a mesh with a 1-d discrete axis." Now I'm getting confused again -- I think this is (at least in my head) a result of a fundamental mismatch between orthogonal meshes and unstructured meshes. The whole concept of an "axis" matches orthogonal coordinates of some sort. (that's kind the definition of orthogonal, yes?). Mapping a Ugrid to the real world, the real world is 2D (Or 3D, but let's not go there yet) -- and that 2D world has 2 orthogonal axis -- X, Y, Lat, Long, whatever. So a mesh represents a 2D space -- but I wouldn't say it has two axes, or a 1-d discrete axis either, ir is simply something else. Or "axis" means something different in this context than I think it does. "separate domain definitions for each data variable is an appropriate view for the CF data model. " That means there *could* be separate definitions, or data variables could share the same one, yes? If, for instance, multiple variables in the same file are on the same mesh, they share a single domain definition, yes? Thanks for taking the time to educate me! -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-887925192__;Iw!!G2kpM7uM-TzIFchu!l25fGL1Spt51msDlCyYHO35_-m5A_TRr12eoj7ksr2j_Dqr1L0uiRCmDLxF85lyapYwtjoW2xeY$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Hi @ChrisBarker-NOAA and @pp-mo, I have been away and am just catching up with the conversation. Thank you for an interesting read! > From the data model perspective, there needs to be SOME way to define the > connectivity. how it's done is a matter of the "encoding", yes? Absolutely. Patrick's description of symmetry (https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-882630042__;Iw!!G2kpM7uM-TzIFchu!lbgXr2bG5DfZVu02KjCqJBqYJUd461YJXHy_PUxq-1YQYcmNceBLkbAVeW4u6ezxu6cK3041iWg$ ) is right - it is symmetric in the square matrix sense. This is indeed not at all how UGRID actually encodes this information, but the CF data model is independent of the encoding, and the symmetric matrix is logically what is going on here. Whilst there is no expectation that anyone should encode it in this manner, it is tempting (to me!) because the square connectivity matrix can be easily updated in subspacing operations. It's a good point about what to do on this matrix's diagonal - I think that the option of these values having 'no meaning regardless of value' is sufficient for the data model. Whether you use booleans, integers, strings, etc. to denote connected/not connected is entirely an encoding choice and has no impact on the data model. > we really do need to capture the concept that the mesh is not unrelated > pieces, it is one thing -- you can't really know what any of the data fully > means without the full mesh description. > (2) likewise, the CF description has no place for the cross-location > connectivities (like face-edge-connectivity) The CF model does connect (e.g.) faces with edges and nodes, but in a different manner to UGRID. In CF, a "cell" is typically defined as the "space" enclosed by bounds, and the edges of the cell are the connections between adjacent cell bounds. This space may be 0-d, in which case it is just a "node"; or 1-d, in which case it is an "edge" connecting two "nodes"; or 2-d, in which case it is a "face" defined by "edges" and "nodes"; (etc, but we don't generalise to 3-d and beyond, yet). The nodes (i.e. bounds) and edges do not have an independent existence - they are elements of the CF cell definition. The new domain topology construct makes explicit the cell connectivities. > (1) Firstly, a UGRID mesh with multiple locations relates to multiple > domain-axes in CF That can "work", because any data-variable can only reference one of them, as described : > > > we have no use case for two or more topology constructs, each of which > > applies to a single unique domain axis, and in fact we have no way of > > encoding it, so that case should indeed be excluded. > > So, the CF 'topology construct' can only be atttached to a single domain axis > of a domain. > That means that UGRID data which maps to a different mesh location is > modelled as belonging to a separate, independent "domain". > This seems okay for now, but it means of course that the CF decsription has > no concept of the intercoupled nature of the different locations. Yes. A CF field construct contains a domain that is limited to describe just the parent field's data. Although, be careful not to confuse "domain axis" and "domain". A "domain axis" is essentially a dimension of the domain. We restrict the new domain topology constructs to apply to a single domain axis simply because there is no current way of encoding a domain topology construct that applies to multiple domain axes. UGRID only describes a mesh with a 1-d discrete axis. If this ever changed, it would be described by a simple generalisation of the data model text. The CF data model mustn't provide capabilities that are not allowed by the CF conventions. If there are no data variables in a file - i.e. just the mesh is stored in a dataset - then the entire mesh definition is captured by a CF domain construct. However, if one or more a data variables are defined on the mesh, then their CF domains are only allowed to be those elements of the mesh that are in use. For example, if a temperature variable is stored on faces and a U-wind is stored at nodes, then the domain of the former will include the UGRID faces, edges and nodes, but the domain of the latter will only know about the nodes. In both cases, a connectivity matrix will retain the required connectivities. I just wrote _"If there are no data variables in a file - i.e. just the mesh is stored in a dataset - then the entire mesh definition is captured by a CF domain construct."_, however I realise that that's not necessarily the case if there are edge coordinates as part of a mesh with faces. In this case, the edge coordinates can only be represented by the CF data model in a second domain that comprised 1-d cells defining each edge and how it's connected to others. I can't decide right now if this is a problem for the data model (i.e. should edge coordinates be a new feature of
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
> From the data model perspective, there needs to be SOME way to define the > connectivity. how it's done is a matter of the "encoding", yes? Yes, that is what I was "trying" to say ! > ?? -- but does X connect to Y is the key concept we are trying to capture > here. > in "lay" terms, I might say something like: ... All agreed, absolutely I was just trying to express that I think the two descriptions **_are_** compatible, and in fact basically mean the same thing. I **_think_** the key point is that how UGRID actually organises things (what I termed "encoding") is **_not_** the same terminology in which the CF datamodel description is decribed. So, we do need to be confident that they are _equivalent_, but that also means exploring the rules and constraints which are part of the description (on either side of the fence). If I read correctly what has been said here ... The CF datamodel only states that the connectivity relation must be symmetrical : "If A connects to B, then B connects to A". It also describes such relations only on one "domain axis". My ouststanding concern is that UGRID does define additional information structures (and governing "rules") which the CF 'domain' model has no interest in. (1) Firstly, a UGRID mesh with multiple locations relates to multiple domain-axes in CF That can "work", because any data-variable can only reference **_one_** of them, as described : > we have no use case for two or more topology constructs, each of which > applies to a single unique domain axis, and in fact we have no way of > encoding it, so that case should indeed be excluded. So, the CF 'topology construct' can only be atttached to a single domain axis of a domain. That means that UGRID data which maps to a different mesh location is modelled as belonging to a separate, independent "domain". This seems okay for now, but it means of course that the CF decsription has no concept of the intercoupled nature of the different locations. (2) likewise, the CF description has no place for the cross-location connectivities (like face-edge-connectivity) So, I'm not sure where any of that might become practically relevant, but I'm still a little worried that they aren't really describing the same ideas with the same range of possibilities. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-884008101__;Iw!!G2kpM7uM-TzIFchu!heMcp7OrldjJzmI0l-rXlmXX4p2RCrOo9OpiNkaQZTwht7eRpskqRdQyuPdtJH2KB__v_BpyEc0$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
> Looking at it again, from the viewpoint of actual UGRID connectivity data, that does make the 'does X connect to X' question seem a rather abstruse and unimportant. ?? -- but does X connect to Y is the key concept we are trying to capture here. >From the data model perspective, there needs to be SOME way to define the >connectivity. how it's done is a matter of the "encoding", yes? > that basic graph theory is written in terms like these. I'll take your work for it :-) But in "lay" terms, I might say something like: """ The connectivity arrays capture how the individual component of the mesh relate to each other. For example, for the face-face connectivity, each face has N neighbors (where N is the order of the cells, e.g. triangles are N=3). So for each face, the indexes of the three connecting faces is provided, resulting in a num_faces X N array. Note that there is duplication where there is symmetry: that is, if face i connects to face j, face j connects to face i. """ For the most trivial example of a two-triangle mesh, you would have, for the face-face connectivity: ``` [[-1, -1, 1], # face 0 connects only to face 1 [-1, -1, 0], # face 1 connects only to face 0 ] ``` where -1 means "nothing" and the faces are zero indexed. So that those two array entries have dublicative information, representing the symmetry NOTE: I realised as I was writing that that the encoding does not have to be duplicative and symmetric. We tend to store it that way because it's easy to ask the question:"what are the neighbors of face i?" -- but we could only store the connections, which would make it harder to interpret, but easy to encode without duplication. The above two-face mesh would simply be: [[1, 0]] that is: face one connects to face zero. (and therefor face zero connects to face 1 as well) In fact, in UGRID, the "edges" connectivity is described exactly like that. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-883550731__;Iw!!G2kpM7uM-TzIFchu!lfKIbGIvZOfhrO6mqfawl6sUjsh9rY0h93P2SjKe2WjT-sCh_-Oh8FGaUW6uAnxaLB6G1Lq7qLE$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
> It sounds like we agree that the domain could have multiple topologies (as > opposed to topology constructs). It's a good point that we have no use case > for two or more topology constructs, each of which applies to a single unique > domain axis, and in fact we have no way of encoding it, so that case should > indeed be excluded. As with @ChrisBarker-NOAA , I'm not truly confident that I know what this really means + intends. However, another thing that has been bothering me in this, is a possible lack of "symmetry" in the relationship between the CF and UGRID views of these definitions. In UGRID, what we might call the "reflexive" connectivities, i.e. face-face / edge-edge / node-node, are only a _part_ of the possible information -- but with the current statement, it seems that these are the **_only_** part of it that the CF datamodel will concern itself with. So, as this only constrains (or describes) properties of a _part_ of the UGRID information, I'm a bit worried if this could be an omission -- which in turn could require further amendments at some future time ? If I've understood the above, then it means we are also proposing a limited _type_ of "topology constructs" : that only relates one of (possibly multiple) toplogies to itself. In which case... what is being "excluded" is (a) any more than one topology-construct, defined on a given domain, and also (b) any (hypothetical) topology-construct that would carry a relationship between two _different_ topologies of a domain. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-882641160__;Iw!!G2kpM7uM-TzIFchu!hUQFGCaTweQZzDhEFeXyOCF-ZM4yCAdi1HdMhZp-d-msusDGMdf9kT9XXJ3IUn35aoT6DXvVkYs$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear David It's a good point that there may be domain axes which aren't involved in the topology. Thanks for the correction. We can supply a ?? topology construct provided it refers to a single domain axis (the UGRID case). I'm not sure there would be more than topology for a domain, although I agree it's conceivable, but I think that's in the realm of speculation, since we don't currently have a use-case for describing the situation with more than one domain axis involved in the topology. Best wishes Jonathan -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-881400525__;Iw!!G2kpM7uM-TzIFchu!jKWZ51HoixEz5djWST4YyfJngt2th3x9y2FGQIgn80lQhtMEjv9yHgvcepMu-iXIBxJkpC0NN7c$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
@davidhassell: Thanks! this looks good to me. Am am curious why you chose "network topology" rather than "mesh topology" -- I"m not sure it matters much but at least that wikipedia article is pretty focused on the topology of communication networks which is pretty similar, but not quite. On the other hand, "mesh topology" is used for computer networks (https://urldefense.us/v3/__https://www.computerhope.com/jargon/m/mesh.htm__;!!G2kpM7uM-TzIFchu!m0ctAe0cUVBEF_t4HBNY8OlXwmbUA3GUCl--BUxUGY5YFNQtVbK-_lz7bnjgYaE0QwYIii3FrK4$ ) I have no idea if there's a standard term for what we mean in this case, but modeling folks do use teh term "mesh" or "grid" in this context, and I've never heard anyone use the term "network". Though the GIS folks use "Triangulated Irregular Network" (https://urldefense.us/v3/__https://en.wikipedia.org/wiki/Triangulated_irregular_network__;!!G2kpM7uM-TzIFchu!m0ctAe0cUVBEF_t4HBNY8OlXwmbUA3GUCl--BUxUGY5YFNQtVbK-_lz7bnjgYaE0QwYIJzB7bx8$ ) -- maybe that's where "network" came from? The computer graphics folks use: "Triangle mesh" (https://urldefense.us/v3/__https://en.wikipedia.org/wiki/Triangle_mesh__;!!G2kpM7uM-TzIFchu!m0ctAe0cUVBEF_t4HBNY8OlXwmbUA3GUCl--BUxUGY5YFNQtVbK-_lz7bnjgYaE0QwYIgNZ3RWk$ ) So there's president for that, too. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-880950086__;Iw!!G2kpM7uM-TzIFchu!m0ctAe0cUVBEF_t4HBNY8OlXwmbUA3GUCl--BUxUGY5YFNQtVbK-_lz7bnjgYaE0QwYI9xf65sc$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Hello @ChrisBarker-NOAA and all, After some very illuminating discussion over at https://urldefense.us/v3/__https://github.com/ugrid-conventions/ugrid-conventions/issues/52__;!!G2kpM7uM-TzIFchu!gsrPpUHyBO1L3RCLlnbI4ykGpE8GyFUQGCnBa97TVwjO6tJnc_z4tfusXiad6JE3NXdsbwwppNw$ , and some offline discussions, @JonathanGregory and I have come up with some suggested text to the data model in [Appendix I](https://urldefense.us/v3/__https://cfconventions.org/cf-conventions/cf-conventions.html*appendix-CF-data-model__;Iw!!G2kpM7uM-TzIFchu!gsrPpUHyBO1L3RCLlnbI4ykGpE8GyFUQGCnBa97TVwjO6tJnc_z4tfusXiad6JE3NXdsLhQqXgs$ ) that we hope will update the CF data model for the mythical connectivity. The new text is hopefully self-explanatory and edits the "Domain construct" section of Appendix I (edits in italics) and provides a new section describing a new construct type - the "network topology construct". Here we are using "network topology" in [this sense](https://urldefense.us/v3/__https://en.wikipedia.org/wiki/Network_topology__;!!G2kpM7uM-TzIFchu!gsrPpUHyBO1L3RCLlnbI4ykGpE8GyFUQGCnBa97TVwjO6tJnc_z4tfusXiad6JE3NXdsctCuFWE$ ), but there may well be better names. Remember that the CF data model is a logical data model that is independent of the encoding, so it not a problem that the connectivity array does not look like a UGRID netCDF connectivity variable. (I took the liberty of rearranging some the network topology text after Jonathan last saw this - I take full responsibility for any negative consequences.) ### Domain construct The domain construct (figure 3) describes a domain comprising measurement locations and cell properties. The domain construct is the only metadata construct that may also exist independently of a field construct. The domain construct contains properties to describe the domain (in the same sense as for the field construct) and relates the following metadata constructs * Domain axis constructs. * Dimension coordinate and auxiliary coordinate constructs. * Coordinate reference constructs. * Domain ancillary constructs. * Cell measure constructs. * _Network topology constructs._ All of the constructs contained by the domain construct are optional (as indicated by "0.." in figure 3). In CF-netCDF, domain information is stored either implicitly via data variable attributes (such as coordinates), or explicitly in a domain variable, _or a UGRID mesh topology variable. In the latter two cases_, the domain exists without reference to a data array. ### Network topology construct A network topology describes the connectivity of cells of domain indexed by a subset of the domain axis constructs. When two cells are connected, operations on the data stored on them may be assumed to be continuous across their common boundary. A network topology construct describes logically and explicitly the network topology of cells indexed by a single domain axis construct. A network topology construct contains a connectivity array that spans a unique domain axis construct with the addition of an extra dimension of the same size, such that each dimension indexes the cells. The array is symmetrical, and each element indicates whether the pair cells to which its indices refer are connected. A network topology that has no corresponding network topology construct, which includes those for multiple domain axis constructs, is nonetheless implicitly defined by a function of both the physical contiguousness of the cells, and the nature of the real world or simulated processes that produced the data. For example, in a field which contains both land and ocean cells, connections between land and ocean cells might be excluded for some physical processes.The description of such an implicit network topology may require metadata that is external to CF. In CF-netCDF a network topology can only be provided for a domain defined by a UGRID mesh topology variable. In this case, the connectivity array is supplied by a UGRID connectivity variable (such as a "face_face_connectivity" variable). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-880923765__;Iw!!G2kpM7uM-TzIFchu!gsrPpUHyBO1L3RCLlnbI4ykGpE8GyFUQGCnBa97TVwjO6tJnc_z4tfusXiad6JE3NXdsqqsaF5E$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear Patrick, Thank you for bringing up "location index set" variables. I agree that in the absence of the `cf_role` attribute it is not always possible to distinguish one from data variable, so I would be happy with retaining it on these variables. By extension, I think that we should drop the suggestion (it is just that at this stage) of removing `cf_role` from the mesh topology variable. This is because a location index set variable is logically identical to a mash variable, so having a common mechanism of identification would be nice. > ( This problem is entirely analogous to an existing problem with > auxiliary-coordinates in standard CF : If the data-variable which references > them is removed, then they are not distinguishable from data-variables -- so > they "become" data-variables ) I wouldn't call this a problem, rather a feature! When we read a dataset, we tend to not cast variables that have been identified as auxiliary coordinates (or other roles) as data variables _as well_, but that is only a default behaviour that is what most of want most of the time. All the best, David -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-872289997__;Iw!!G2kpM7uM-TzIFchu!m3MvmAJbcTaPU6Ztlb1aEbCnoUt3-uDR3R1pG6HQ9fufA_aTTj_7DV37GTYArWT6nxp6IECmCAg$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Hi, sorry to be late to the table with this issue, but I have been listening in here a while, in hopes of understanding better and maybe contributing. I think that I may have spotted a potential problem with the removal of 'cf_role' : As stated (above), the role of a mesh-variable is identifiable by its having a 'topology_dimension' attribute, but **_the same is not true of a location-index-set_** : So, if we include an index set in a "mesh only file" (aka "meshfile" or "gridfile" in some quarters), then we will be unable to distinguish it from a data-variable. This in turn implies that when loading a file, we would need to "know" whether it was intended to be a "mesh file" or a "normal datafile" as you can't determine that by inspection. ( This problem is entirely analagous to an existing problem with auxiliary-coordinates in standard CF : If the data-variable which references them is removed, then they are not distinguishable from data-variables -- so they "become" variables ) Our context : Here at the UK MetOffice, we are working to support unstructured data within Iris, [using UGRID as the template for our internal data-model](https://urldefense.us/v3/__https://scitools-iris.readthedocs.io/en/mesh-data-model/generated/api/iris/experimental/ugrid.html__;!!G2kpM7uM-TzIFchu!gI2poPff_xwfIpnQMVw6F_zs4v2r3s3F4jWKcIS2Wef7scNBlJ8EEWlts5nRZ3b4KwzlqWjLQmM$ ) (as we already do for CF). Locally, we have a particular interest in the use of location-index-sets, and we are also intending to use "mesh files" i.e. files with only the mesh structure and no data. Also, our practical experience in tools development and support shows that files that do not fully comply with conventions are, sadly, just not that uncommon even in the respected international archives. Thus, just as Iris has to somehow handle files with invalid standard-names and units, or mis-specified grid-mappings, so our trial UGRID files suffer from problems like missing optional connectivity links, the odd miss-spelling and so on. What this tells us is, that the "robustness" of the format is also a important consideration. >From the point of view of a generic code library developer, the unambiguous >identification of the 'role' of elements within a file will definitely make >writing parsing code more straightforward -- and not least because dealing >with _incorrect_ input in a helpful way is an important usability factor (just >as it is in compiler design). So, I must confess that I personally was _preferring_ the way that UGRID labels each component unambiguously, instead of relying on links from other components to infer the role of a variable. Solutioneering maybe, but ... could we instead **_allow the attribute to be named 'ugrid_role', and simultaneously deprecate the older 'cf_role' usage ?_** -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-872063284__;Iw!!G2kpM7uM-TzIFchu!gI2poPff_xwfIpnQMVw6F_zs4v2r3s3F4jWKcIS2Wef7scNBlJ8EEWlts5nRZ3b4Kwzl0-z6CLg$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear Klaus, I hope to allay your concerns by noting that all of the UGRID machinery for storing connectivity will certainly be imported into CF unchanged - it is just that in the logical data model we don't need to make special mention of it. This is because it turns out the connectivity variables are just one (very convenient) way of encoding something in CF that can _in principle_ already be done another way. We call the CF data model a "minimal" logical data model, meaning that we keep it simple by not overlapping the contents of one part of the model with another. All the best, David -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-871526998__;Iw!!G2kpM7uM-TzIFchu!nJQQ-SJUjuYRcEafLmhlUyS5zK6UjwoUb96yVMPZZ89A_s5kED9DGNpFVzdetcE__uB-R3l6E6Y$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Note that not having explicit connectivity creates a (potentially) rather large computational overhead for the reconstruction of it. This will be exacerbated with more complicated grids in the future, think time-dependent unstructured grids, perhaps with regionally varying timestep. I think it would be good to provide a standardized way of storing connectivity, though I agree it also makes sense to make it optional, at least for most commonly used classes of grids today. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-871478644__;Iw!!G2kpM7uM-TzIFchu!iWswTipYA5EBK8IzXGOj5Vq_lVnALN16Cgw9Az-6h551zh55jPZldda2WlknBq9UZI98Ip1OvI0$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear Jonathan, That's right. I should have made that clearer, so many thanks for pointing it out! CF provides cell connectivity by inspection of coincident (or possibly overlapping) bounds. UGRID provides an index based encoding for making the connectivity easier to find in many circumstances, but the CF data model does not need to copy this. Not including these connectivity indices in the CF data model in no way prevents a software application from managing, and utilising, them "alongside" the data model representation. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-871270692__;Iw!!G2kpM7uM-TzIFchu!g4yoe64-ExvhkL70aKWrCFJ5lJylfftHb1VyVnOGSA4MW4qFPyII6ukaJPOgR-FUAaXy6R7S_cM$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
I agree that a cell which is an edge bounded by two nodes is fine in the CF data model, yes. (H) is correct that CF doesn't explicitly recognise connectivity, although you could infer it from coincidence - isn't that right? Jonathan -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-871264221__;Iw!!G2kpM7uM-TzIFchu!gpU7DJSuTUOiA4uZ1OyXdF852Z6xqnWdZuZbdo7H0bWNCsz5x9z7MgLRM4JoppL1er5_M60eMp8$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Hello,
In starting to think about:
> (H) Further to the discussion on implications on the CF data model
> (ugrid-conventions/ugrid-conventions/issues/52), the CF data model needs to
> be updated to allow the storage of topological connections between cells
> ("cells" in the CF data model sense). It is not necessary at this stage for
> the connectivity between cell elements (such as the edge of a face) to be a
> part of the CF data model.
For the one case that I thought was not possible in the current CF data model,
i.e. a [1D network
topology](https://urldefense.us/v3/__https://ugrid-conventions.github.io/ugrid-conventions/*1d-network-topology__;Iw!!G2kpM7uM-TzIFchu!gkEbRVWDm8OAd0GAR_p9pl8dtYki7o7-HSGJLjtIHnNlpvKRtYuQk9d3o7xJUf_EfmafcA7vglQ$
),
I now realise the current data model is OK. I was thinking that in this case
the CF domain cells were the nodes, but really a cell is an edge bounded by
two nodes. In this view, the existing CF framework suffices. Remember that the
CF data model is independent of the netCDF encoding, including the UGRID. It
doesn't matter if the CF data model view of the world looks quite different to
that of the UGRID view, as long as it is possible to unambiguously map between
the two.
Thanks,
David
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-871255936__;Iw!!G2kpM7uM-TzIFchu!gkEbRVWDm8OAd0GAR_p9pl8dtYki7o7-HSGJLjtIHnNlpvKRtYuQk9d3o7xJUf_EfmafUs7YDn8$
This list forwards relevant notifications from Github. It is distinct from
cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the
UCAR list will result in a subscription to this list.
To unsubscribe from this list only, send a message to
cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
This is fine, thanks. If we agree, I suppose that we need to write propose some textual changes in the CF documents for some of these points. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-865061871__;Iw!!G2kpM7uM-TzIFchu!lpypouSOBJ9kpFe7Hc9l1U1EX5cNBaR_5LYsVUoJH92cmxK_NXDfnx5MkxpRJdCUY89wiNHJ2-c$ This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Hello,
Following on from some discussions that have been taking place on the [UGRID
issue
tracker](https://urldefense.us/v3/__https://github.com/ugrid-conventions/ugrid-conventions/issues__;!!G2kpM7uM-TzIFchu!jNG-Y16d7c-C88U9VVZoa-yNuIMgPRCdgpg_rDn4glZ59EsZV9mellI5RhDBXtvHkx_max3jEsU$
), a couple more items to the proposal are needed (**G** and **H**). Here is
the new set:
**(A)** The governance is written up along the lines of @hrajagers ideas:
https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-703858946__;Iw!!G2kpM7uM-TzIFchu!jNG-Y16d7c-C88U9VVZoa-yNuIMgPRCdgpg_rDn4glZ59EsZV9mellI5RhDBXtvHkx_mdt2ya0M$
**(B)** Comprehensive conformance rules are written up for UGRID. These should
be maintained alongside UGRID in its repository, and referenced from (not
copied into) the CF conformance document.
**(C)** Document the standardised UGRID attributes in the CF conventions,
thereby making them visible to all users. Mention in the governance rules that
they need maintaining. Only the UGRID attributes which can appear on data
variables (mesh, location and location_index_sets should be added to CF
Appendix A. All other attributes UGRID attributes (such as those on mesh
variables) should go into a new CF appendix specifically about the UGRID mesh
topology variable, consisting of the table with an introductory sentence or
two. This would be like the treatment of the attributes of the grid mapping
variable, which are in a table in Appendix F, not in Appendix A. Note that the
geometry variable attributes appear in Appendix A, but there are only five of
them, whereas there are 18 attributes of the mesh topology variable.
**(D)** Deprecate the cf_role attribute for connectivity variables (but
retaining it on the mesh topology variable). If and only if @hrajagers and
UGRID colleagues are in agreement, also deprecate the cf_role attribute mesh
variables, deferring to the presence of the topology_dimension attribute for
identification. This latter suggestion will make the mesh variable "more
CF-like", but that is not worth at this time if the expense is a lot of broken
existing software.
**(E)** Add some text to CF 5.8 (Domain Variables) (now accepted for CF-1.9) to
explain the UGRID mesh topology variable and how it relates to a domain
variable. It may the case that the occasional note relating to UGRID would be
useful in other sections. I don't propose to review for these, but they could
always be added as when it was felt to be useful.
**(F)** Add a short subsection of CF section 1 to introduce UGRID and its
purpose, to make clear its special synergy with CF, to remark on the appearance
of attributes in CF appendices, and to say that it has its own conformance
document which complements the CF conformance document.
**(G)** Further to the discussion on the order in which the corner nodes of a
volume are specified (ugrid-conventions/ugrid-conventions/issues/53), the UGRID
conventions drops support for the fully 3D meshes (When someone actually needs
it, it can reintroduced without the current ambiguities, which will be fine CF.)
**(H)** Further to the discussion on implications on the CF data model
(ugrid-conventions/ugrid-conventions/issues/52), the CF data model needs to be
updated to allow the storage of topological connections between cells ("cells"
in the CF data model sense). It is not necessary at this stage for the
connectivity between cell _elements_ (such as the edge of a face) to be a part
of the CF data model.
Thanks,
David
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://urldefense.us/v3/__https://github.com/cf-convention/cf-conventions/issues/153*issuecomment-86019__;Iw!!G2kpM7uM-TzIFchu!jNG-Y16d7c-C88U9VVZoa-yNuIMgPRCdgpg_rDn4glZ59EsZV9mellI5RhDBXtvHkx_mW2viD_c$
This list forwards relevant notifications from Github. It is distinct from
cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the
UCAR list will result in a subscription to this list.
To unsubscribe from this list only, send a message to
cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Thank you, @davidhassell . Looking good to me as well. As shown above on the discussion page, I have posted in the UGRID forum (as well as the SGRID forum following [this comment](https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-712262386)) a request for feedback on this proposal. I'll also reach out to a number of people offline. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-785360644 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Thank you, @davidhassell. I think this is fine. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-738180915 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear @JonathanGregory and @hrajagers, Here is a synthesis current proposal. How does it sound? **(A)** The governance is written up along the lines of @hrajagers ideas: #153 (comment) **(B)** Comprehensive conformance rules are written up for UGRID. These should be maintained alongside UGRID in its repository, and referenced from (not copied into) the CF conformance document. **(C)** Document the standardised UGRID attributes in the CF conventions, thereby making them visible to all users. Mention in the governance rules that they need maintaining. Only the UGRID attributes which can appear on data variables (`mesh`, `location` and `location_index_sets` should be added to CF Appendix A. All other attributes UGRID attributes (such as those on mesh variables) should go into a new CF appendix specifically about the UGRID mesh topology variable, consisting of the table with an introductory sentence or two. This would be like the treatment of the attributes of the grid mapping variable, which are in a table in Appendix F, not in Appendix A. Note that the geometry variable attributes appear in Appendix A, but there are only five of them, whereas there are 18 attributes of the mesh topology variable. **(D)** Deprecate the `cf_role` attribute for connectivity variables (but retaining it on the mesh topology variable). If and only if @hrajagers and UGRID colleagues are in agreement, also deprecate the `cf_role` attribute mesh variables, deferring to the presence of the `topology_dimension` attribute for identification. This latter suggestion will make the mesh variable "more CF-like", but that is not worth at this time if the expense is a lot of broken existing software. **(E)** Add some text to CF 5.8 (Domain Variables) (now accepted for CF-1.9) to explain the UGRID mesh topology variable and how it relates to a domain variable. It may the case that the occasional note relating to UGRID would be useful in other sections. I don't propose to review for these, but they could always be added as when it was felt to be useful. **(F)** Add a short subsection of CF section 1 to introduce UGRID and its purpose, to make clear its special synergy with CF, to remark on the appearance of attributes in CF appendices, and to say that it has its own conformance document which complements the CF conformance document. Thanks, David -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-736382330 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear @davidhassell > Is it "Even though UGRID is being included in CF, we don't have to consider > the relationship between domains of different data variables at the moment." Yes, that is what I meant. By "not being included in CF" I meant "in the CF _document_". Sorry to be unclear. Thanks Jonathan -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-712851846 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
(sorry - pressed send too early - I'll try again!) Dear @JonathanGregory, Thanks for these comments I agree with your updated **(C)**. I'm also fine in **(D)** with a mesh topology variable getting its canonical identity from the `topology_dimension` attribute (which is similar in principle to how a domain variable is to be identified). However, it would be good to here from @hrajagers if making this `cf_role` use optional is OK. I agree with **(F)**. > If UGRID is not being included in CF, we don't have to consider it at the > moment. I'm not sure what you mean here. Is it "Even though UGRID is being included in CF, we don't have to consider the relationship between domains of different data variables at the moment.". Apologies if I have misunderstood. Also on the data model, I'd just like to highlight that the integer-valued interconnectivity variables (e.g. `"edge_node_connectivity"`) need not feature in the data model, for the same reasons that a list variable for compression by gathering does appear in the data model. I agree that the issue of SGRID, and interconnected domain/data variables in general, is a conversation for another issue; and now that we are starting to formalise the storage of domains as independent entities, this is a good moment to reignite this conversation. All the best, David -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-712686233 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear @JonathanGregory, Thanks for these comments I agree with your updated **(C)**. I'm also fine in (D) with a mesh topology variable getting its canonical identity from the `topology_dimension` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-712675809 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear @davidhassell Thanks for this summary. I agree with **(A)**, **(B)** and **(E)** as given. Re **(C)**, I would suggest that only the UGRID attributes which can appear on data variables should be added to CF Appendix A. If I have read the document correctly, these are `mesh`, `location` and `location_index_set`. The other attributes belong to mesh variables. To prevent accidental collision with CF, it would nonetheless be useful to tabulate them, but I'd suggest we put them in a new CF appendix specifically about the UGRID mesh topology variable, consisting of the table with an introductory sentence or two. This would be like the treatment of the attributes of the grid mapping variable, which are in a table in Appendix F, not in Appendix A. I note that the geometry variable attributes appear in Appendix A, but there are only five of them, whereas there are 18 attributes of the mesh topology variable. Re **(D)**, if Bert @hrajagers and UGRID colleagues are OK with dropping `cf_role` for connectivity variables, that's good. They could continue to be allowed but deprecated, rather than disallowed. That's a decision to be made when the conformance rules **(B)** are written. It seems to me that `cf_role` is also redundant on the mesh topology variable, because it must also have a `topology_dimension` attribute, it seems. Couldn't that be used as the defining characteristic of a mesh topology variable? If so, this `cf_role` could also be deprecated; it can't be disallowed if current software depends on it, as Bert says. **(F)** I proposed earlier that we could add a short subsection of CF section 1 to introduce UGRID and its purpose, to make clear its special synergy with CF, to remark on the appearance of attributes in CF appendices, and to say that it has its own conformance document which complements the CF conformance document. What do you think? I agree with you that the relationship between domains of different data variables is not currently considered in the CF data model, but not inconsistent with the data model. If UGRID is not being included in CF, we don't have to consider it at the moment. Regarding Ryan @rabernat's comment, I think it would be fine to consider SGRID as well, but let's do it as a separate issue, and perhaps after UGRID, because we may not have enough mental capacity to deal with both at once. Best wishes Jonathan -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-712441568 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Thanks to everyone who is working through this important issues. I strongly support incorporating SGRID into this same framework. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-712262386 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
A note on the CF data model: I think that UGRID need not affect the CF data model at this time. This is because CF does not currently formalise connections between data variables, on the same or different domains. A mesh topology variable collates multiple domains (one for faces, one for edges, etc.), but a given data variable only refers to one of them (e.g. ` data:location = "face" ;`). How you relate a "face" data variable to an "edge" one is moot when you abstract out the netCDF encoding - you get to the same place if you do it by inspection of the coordinate values, or by inspection of the mesh topology and data variable attributes. I realise that you could say that the point of UGRID is to make these relations explicit, but if that is to be the case then it should be propagated to other areas of CF (e.g. as [SGRID](https://sgrid.github.io/sgrid/) proposes), and so should be considered in the round at a later stage. Does sound reasonable? Thanks, David -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-712239331 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear Bert, Jonathan, and all, I would like to try to summarize the ideas that have been discussed in the form of some broad proposals that I hope could be acceptable to allow us conclude this issue. I welcome your feedback. In no particular order: **(A)** The governance is written up along the lines of @hrajagers ideas: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-703858946 **(B)** Comprehensive conformance rules are written up for UGRID. These should be maintained alongside UGRID in its repository, and referenced from (not copied into) the [CF conformance document](http://cfconventions.org/cf-conventions/conformance.html). **(C)** Update the aforementioned [CF Appendix A](https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#attribute-appendix) to include the relevant UGRID attributes, thereby making them visible to all users. Mention in the governance rules that this table needs maintaining. **(D)** Based on @hrajagers [previous comment](https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-703858946), dropping the standardisation of `cf_role` on the "connectivity" variables, but retaining it on the mesh topology variable. This is related to my previous comments [about the use of datasets with meshes but no data](https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-704435832), which I now withdraw. A mesh topology variable can actually contain multiple domains in the CF-sense, one of which can "picked out" by a data variable. This makes it sufficiently different, I realise, to the proposed CF domain variable (#301) that we shouldn't to unify them at this time. **(E)** Add some text to CF 5.8 (Domain Variables) (currently being proposed in #301) to explain the UGRID mesh topology variable and how it relates to a domain variable. It may the case that the occasional note relating to UGRID would be useful in other sections. I don't propose to review for these, but they could always be added as when it was felt to be useful. Thanks, David -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-710122506 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear Bert,
Thank you for describing all of the history that has occurred here - it really
is very helpful, particularly the interactions you have had on the geometries
front.
A summary of my position would be that I would support UGRID being moved into
CF ("chapter 10"), but if this is not possible then I think that we would still
be able to find a way to make things work satisfactorily.
Governance
If UGRID were incorporated into the CF text, there is no governance issue. It's
an issue that only arises if it lives outside.
If UGRID lived outside, I genuinely do not expect any problems in the CF/UGRID
working relationship, for the same reasons described by @hrajagers, but the
part of the point of the governance rules is to prevent problems arising,
however unlikely. We would need to cover situations thorny situations, [such
those
raised](https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-702580499)
by @erget.
One of my own concerns with UGRID not being inside CF is its general
invisibility to users. The proposed introductory text is very short and will
surely not result in most users seeking out and reading the full UGRID
conventions. For instance, what would happen if a user applied `mesh` and
`location` attributes to a non-UGRID data variable in good faith, having
checked in [Appendix
A](https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#attribute-appendix)
that these attributes are not standardised? The UGRID-aware CF checker might
tell them their dataset is broken. OK, they could then rename the attributes
with this new-found knowledge, but they need to be able to ascertain this
_before_ creating datasets, and I don't think that this is easy enough were
UGRID to exist elsewhere.
cf_role
This comes down to variable identification, clearly. From a CF perspective, we
know that a data variable employs UGRIDbecause it has
`location_index_set`, or both `mesh` and `location`, attributes. One of these
attributes identifies the relevant mesh container variable, that in turn
identifies other required variables (such as edge node connectivity variables).
I don't need `cf_role` to make any of these connections, so the use of
`cf_role` comprises a redundancy.
So, if we were starting _ab initio_, I would argue for dropping the `cf_role`
attributes altogether. But we are not! So I can see there is an argument for
keeping it for backwards UGRID-compatibility, i.e. extending the use of
`cf_role` to include UGRID (as well as DSGs). Or perhaps it could be dropped
without problems?
What do others think?
Conformance
I presume from what you say that there are no conformance rules (rather than
there are rules but no existing software)? Is that right. We only need the
rules to get UGRID (in any form) into CF. If it were helpful, I would be happy
to draft some CF-style conformance rules.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-703714811
This list forwards relevant notifications from Github. It is distinct from
cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the
UCAR list will result in a subscription to this list.
To unsubscribe from this list only, send a message to
cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Dear David, Jonathan, Thank you for moving this forward. In the very beginning we were thinking to move the definiiton for unstructured grids into the CF conventions, later the discussion moved more in the direction of a separate but affiliated convention, and in the end the discussion seems to have settled on a separately documented but linked convention. Since documentation for the CF conventions has shifted to GitHub as well I don't know whether the effort of porting the documents over to the CF repository is still a major issue, but there is a bit of a difference in documentation style. Since UGRID has received quite some uptake already, it may be advantagious to keep the separate identy whereas being more integrated into CF may increase the uptake in the wider OGC and GIS domains ... although via [MDAL support](https://www.mdal.xyz/drivers/ugrid.html) (closely linked to CF) we have already made a good step forward in that direction. By keeping the conventions separate it helps to lower the threshold for implementing either convention. The coin can still flip either way for me, but the concept of modularity introduces a slight preference for keeping the conventions somewhat separate. Many of the core developers of the UGRID conventions are well embedded in the CF conventions, so I think it would be quite acceptable to set out some formal rules as you describe to assure that the two conventions develop in a mutually compatible way. The UGRID 1.0 convention has indeed remained quite stable, the main reason for this is that we already had quite some discussion and uptake before we released version 1.0 ... and ... conventions for specific topics can remain stable for much longer periods than the CF conventions that cover a wide range of features. I don't know of any recent changes in CF that would make UGRID less compatible with 1.9 than with 1.6. We use it ourselves in combination with 1.8 on a daily basis. The [geometries](https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#geometries) are the only development closely related to UGRID and at some time it used a very similar definition format, but given the considered use cases it made more sense there to drop the concept of shared nodes in favour of simplicity and consistency with other storage formats for GIS feature sets. At Deltares we have a draft extension to allow the 1D discretisations within UGRID to be defined on a curved 1D space ... for this extension we build on those geometries introduced in CF 1.8 to describe the shape of the space (i.e. river) before descretisation. So, we try to adopt new CF features as much as possible. There is one element in UGRID that is not fully compatible with CF in any version and that's the use of the attribute "cf_role" to identify the "mesh_topology" and various types of connectivity variables such as the "face_node_connectivity". The name of this attribute was chosen when it was anticipated that UGRID was to move into CF, but if remains a separate convention we may have to reconsider this attribute name ... or CF would have to formally permit this type of use. The question about a UGRID checker has popped up twice or so in discussions I had regarding UGRID over the last year. We haven't implemented a formal UGRID checker, but it would indeed be useful to have. Best regards, Bert -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-703216041 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
> It would be great to hear from some folks who work on UGRID. I see that the [UGRID conventions GitHub repository](https://github.com/ugrid-conventions/ugrid-conventions) has not been updated for 2 years, and the version being recommended for CF is `UGRID 1.0`, which was released 4 1/2 years ago and is also the latest version. At this time CF was at CF-1.6. _This is absolutely not a criticism_ (CF has gone for longer periods in the past with no readily available signs of advancement, although progress was always going on), but we do need to be sure that UGRID 1.0 is compatible with the draft of CF-1.9. Has anyone looked at this? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-702607897 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
In general, I'm not a huge fan of tight coupling but I don't have objections to the thrust of these changes - they aren't very onerous and won't force us to increment if we have no reason to. The specifics make me slightly more nervous. My understanding is that, if proposed changes to CF are not compatible with the currently referenced version of UGRID, > then UGRID will need to be modified to accommodate the new features via a proposal to UGRID, but > to facilitate the progress of a proposal that requires UGRID changes, it is > sufficient for the general nature of the UGRID modifications to be > identified, on the understanding that the UGRID conventions will be updated > in detail at a later date, possibly after the proposal has been accepted in > all other aspects. **Final acceptance will always rely on the completion of > changes to the UGRID conventions, which is at the discretion of the UGRID > community.** Emphasis is my own. If this is the case, does that mean that we would accept the proposal formally from CF's side when we have verified that a proposal to UGRID has been made, and then simply wait to merge the CF proposal until the UGRID proposal has been accepted? The danger is that this might take a long time, and then the CF baseline might have drifted, so that the original CF proposal would need revisiting. In all cases I think that this would have to be checked again to ensure that we don't introduce inconsistencies into the Conventions. Alternatively we could merge immediately in the hopes that the UGRID proposal would be accepted - a bit too optimistic for my taste though and the risk would be that the two standards diverge. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-702580499 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Here are my proposed additions to https://github.com/cf-convention/cf-convention.github.io/blob/master/rules.md. I probably haven't quite got it yet, but it's a start. A key thing to note is the second line: _"The assessment will be carried out by a member of the conventions committee or another suitably qualified person. If no-one volunteers, the chairman of the committee will ask someone to do it."_ What this means is that every enhancement proposal must be "signed off" for any UGRID issues. Almost always this will be a trivial task (e.g. the introduction a new grid mapping attribute parameter - no problem!), but it needs to be done. For simple cases, as in the example just mentioned, someone who is familiar (rather than expert) with UGRID could take care of this, but for more complicated proposals, the opinion of an expert from the UGRID community must be available. The first and last sentences covers how to increment the version of UGRID that is acceptable to a give version of CF. - ### Additional rules relating to the UGRID conventions All new proposals will be assessed to see if the new features defined in the proposal are compatible with the named version of UGRID that is defined for the current version of the CF conventions. The assessment will be carried out by a member of the conventions committee or another suitably qualified person. If no-one volunteers, the chairman of the committee will ask someone to do it. If the proposal is deemed to be not compatible with UGRID in some way, then an attempt must be made to modify the proposal so that its new features are compatible with UGRID, and in such a way that the proposal's intent is not compromised. If the proposal cannot be acceptably modified to conform to the UGRID conventions, then UGRID will need to be modified to accommodate the new features. If UGRID is extended or generalized in some way that allows the new features but does not affect its existing structure and functionality, the proposal is considered backwards compatible. This is the preferred solution. Any such changes to UGRID must be defined in general terms, and preferably with a detailed description of the UGRID alterations. However, to facilitate the progress of a proposal that requires UGRID changes, it is sufficient for the general nature of the UGRID modifications to be identified, on the understanding that the UGRID conventions will be updated in detail at a later date, possibly after the proposal has been accepted in all other aspects. Final acceptance will always rely on the completion of changes to the UGRID conventions, which is at the discretion of the UGRID community. The UGRID conventions exist independently from CF and have their own repository and governance. Therefore the acceptance of a new version of UGRID, whether it arises from a change to CF or from an independent change to UGRID itself, must be raised and discussed in its own enhancement proposal in the usual manner. It follows that a change to CF that requires a change to UGRID will be associated with two GitHub issues - one for the change to CF and one for accepting the new version of UGRID. - -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-702313556 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Yes, please. That would be most helpful. Jonathan -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-702146979 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Hello, Some colleagues were asking after the status of this proposal. As far as I'm aware, there are no outstanding objections other than the requirement to spell out some rules for the co-management of the two conventions: CF and UGRID. The CF data model has now been accepted, and the [rules for its management](https://github.com/cf-convention/cf-convention.github.io/blob/master/rules.md) will be in CF-1.9. I think there are some similarities between the requirements rules for evolving the data model, and for evolving UGRID. I am happy to draft some rules for UGRID, if that helps to get the ball rolling. Thanks, David -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-702118667 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
I think that this is essentially correct. The naming issues are also not insurmountable. Finite element has its own conventions, but they are just naming conventions. If UGRID has finite difference naming then this will merely be a bit confusing for finite element users. In the specific cases above, there are two relevant differences to note: 1. Vertex vs node. Node means something quite different in finite element (a node is a basis function in the dual space to the finite element space), which is why finite element codes usually talk about vertices when discussing the mesh topology. 2. face vs element or cell. node, edge, face, volume are names for mesh entities of a given dimension. Finite element is often more concerned with codimension, which counts downwards from the mesh dimension. A cell (or element) is an entity of codimension 0, i.e. an entity of maximal dimension. On a 3D mesh the cells are volumes and on a 2D mesh they are faces (cell is also defined for 2D or 1D meshes). A facet is an entity of codimension 1. Facets form the boundaries between cells. On a 3D mesh, the facets are faces while on a 2D mesh they are edges, and on a 1D mesh they are nodes. Given that a UGRID mesh knows its dimension, it is possible for software to identify the cells or facets so the difference in naming convention is not so significant. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-572056622 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
Re: [CF-metadata] [cf-convention/cf-conventions] Reference UGRID conventions in CF (#153)
Hello, I have been looking at the Finite Element based CF proposal for Unstructured Grid data model (https://publicwiki.deltares.nl/display/NETCDF/Finite+Element+based+CF+proposal+for+Unstructured+Grid+data+model) which was written up some time ago by Bert. This proposes an encoding for the information required for a consistent spatial interpretation of the values. Given that UGRID is going to be incorporated into CF _[1]_, I was looking at this with to see if backward incompatible changes could occur if this new proposal (or something like it) become part of UGRID at a later date. _[1] it seems like this will indeed happen, once the management side is sorted out ..._ My conclusion from a quick read of Bert's document was that adding the "Function Space" capability, would not impact on files encoded using UGRID 1.0 - which would be good. However, the proposal does suggest renaming certain special attributes (e.g. `face_node_connectivity` becomes `element_vertex_connectivity`). This could be bad for CF backwards compatibility, but, I presume, is easily avoided with a little thought at this stage. Does all this sound like a a reasonable assessment? Thanks, David -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cf-convention/cf-conventions/issues/153#issuecomment-572043075 This list forwards relevant notifications from Github. It is distinct from cf-metad...@cgd.ucar.edu, although if you do nothing, a subscription to the UCAR list will result in a subscription to this list. To unsubscribe from this list only, send a message to cf-metadata-unsubscribe-requ...@listserv.llnl.gov.
