Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hi Stefan, On 07/03/2014 10:46 AM, Blumentrath, Stefan wrote: Hi Matej, Thanks for your reply. Metadata is a current topic I my organization at the moment, so your work comes exactly at the right time for me. Here in NINA people are using many different kinds of GIS software and we are considering introducing GeoNetwork as a central metadata storage solution, which makes metadata management to some extend independent from the GIS software used. Among public and environmental organisations, GeoNetwork is quite popular for metadata management (amongst others Mapping authorities from Netherlands, Switzerland, and the Nordic countries are using it, see also: http://geonetwork-opensource.org/gallery/gallery.html). I have no real personal experience with GeoNetwor (yet), but from what I read in the documentation I could imagine two ways how GRASS metadata could make their way into such a central metadata catalogue: 1) XML import (+ this option is in place when GRASS metadata is written to XML, - import requires manual labor) 2) Metadata harvesting (GeoNetwork can harvest metadata from other CSWs, so maybe pycsw already provides automatic an interface in that direction(?). That would be great!) pycsw is an OGC Reference Implementation of CSW, which means it supports all the features in the spec, including the harvesting and transactions. When it comes to consistent metadata, I guess e.g. lists with values for metadata fields will in practice have to be more or less dynamic (as new staff member are being employed, new topics for spatial analysis arise (requiring new keywords), new types of data being developed and so on). Not sure if manipulating the templates is convenient enough for average users (seems to be mainly suitable for users with some degree of programming experience). However, good to know that providing a custom set of values is already possible through the templates. If you start working on a DB for metadata related content, feel free to let me know if I can support you by any means (yet I will be on leave for almost the rest of the year, which means I can spend only a limited amount of time). However, I had a second look on GeoNetwork in this regard, and GeoNetwork is (naturally) using a database for metadata management (one option here is PostgreSQL). If it would be possible to just connect to that same DB (e.g. for fetching and storing key words, data on contact persons...), that would be really, really great, as this ensures that metadata produced in GRASS are 100% compliant to metadata standards used in GeoNetwork (if GRASS users also use GeoNetwork). Anyway, Rome was not built in a day and I am 100% sure that your work will be very, very useful already from what you planned for GSoC. All the best, Stefan ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev Best, Angelos -- Angelos Tzotsos Remote Sensing Laboratory National Technical University of Athens http://users.ntua.gr/tzotsos ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
On 07/03/2014 02:40 PM, Margherita Di Leo wrote: Hi Stefan, On Thu, Jul 3, 2014 at 9:46 AM, Blumentrath, Stefan stefan.blumentr...@nina.no wrote: However, I had a second look on GeoNetwork in this regard, and GeoNetwork is (naturally) using a database for metadata management (one option here is PostgreSQL). If it would be possible to just connect to that same DB (e.g. for fetching and storing key words, data on contact persons...), that would be really, really great, as this ensures that metadata produced in GRASS are 100% compliant to metadata standards used in GeoNetwork (if GRASS users also use GeoNetwork). Almost every CSW server out there has its own implementation of such a database schema (GeoNetwork, pycsw, deegree, MDweb etc...). How are we going to select which is best to support as native? IMHO there is not one solution to this problem. If you take a close look into MDweb for example, there is an almost full implementation of ISO 19115 in a PostgreSQL schema Anyway, Rome was not built in a day and I am 100% sure that your work will be very, very useful already from what you planned for GSoC. All the best, Stefan ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev -- Angelos Tzotsos Remote Sensing Laboratory National Technical University of Athens http://users.ntua.gr/tzotsos ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
On 07/04/2014 10:15 PM, Matej Krejci wrote: Hi Stefan, 2014-07-03 18:22 GMT+02:00 Blumentrath, Stefan stefan.blumentr...@nina.no: Hei Madi, Thanks for your clarification. Very interesting! I see I have to take a closer look on pycsw and it`s front-ends (as a possible alternative to GeoNetwork). The pycsw documentation [1] says that also pycsw Metadata repositories are set up with a database backend (SQLite, PostgreSQL, and even PostGIS support). However, integration with data portal solutions like e.g. GeoNode or Open Data Catalogue is unfortunately read-only meaning that meta-data can be only queried [2]. Would be nice if metadata from GRASS could somehow go directly into a geodata portal (and that the other way around e.g. people, keywords ... could be fetched from there)... (BTW, in the proprietary world such a functionality requires yet another extension: https://www.geocat.net/bridge/) For organization with a geodata portal solution it is maybe possible to just sync pycsw`s DB with a DB from e.g. GeoNode or so, if one wants to have that consistent... If you allow me two more questions in this regard: Will pycsw become a dependency for GRASS (with metadata support)? Yes it is. Currently there are no packages for Debian. Pycsw has dependency on SQLalchemy, shapely and pyproj. I have info that debian package will probably be created during a few months. There are pycsw deb packages for UbuntuGIS and OSGeoLive, but those were not accepted for Debian and need some more work. On a side note, there is work in progress for read/write integration of pycsw with CKAN. We are hopping to have a working demo for FOSS4GE in 2 weeks http://publicamundi.eu/ Angelos And will it be necessary, that pycsw runs on the same computer as GRASS (meaning also one pycsw for each GRASS installation) or could that somehow be centralized (in other words that GRASS connects to a central pycsw)? Yes of course, the pycsw is providing csw catalogue which can run on a local pc (localhost) or on a server. Our vision is to make an interface based on OWSLib and pycsw which produces essential functions like browsing based on filtering, harvesting and publishing metadata to csw catalogue. The user will be able to choose csw catalogue target like a localhost(probably SQLite) or catalogue on internet. This task will part of GSOC term and after... Thank you, Matej ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev -- Angelos Tzotsos Remote Sensing Laboratory National Technical University of Athens http://users.ntua.gr/tzotsos ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hi Stefan, 2014-07-03 18:22 GMT+02:00 Blumentrath, Stefan stefan.blumentr...@nina.no: Hei Madi, Thanks for your clarification. Very interesting! I see I have to take a closer look on pycsw and it`s front-ends (as a possible alternative to GeoNetwork). The pycsw documentation [1] says that also pycsw Metadata repositories are set up with a database backend (SQLite, PostgreSQL, and even PostGIS support). However, integration with data portal solutions like e.g. GeoNode or Open Data Catalogue is unfortunately “read-only” meaning that meta-data can be only queried [2]. Would be nice if metadata from GRASS could somehow go directly into a geodata portal (and that the other way around e.g. people, keywords … could be fetched from there)… (BTW, in the proprietary world such a functionality requires yet another extension: https://www.geocat.net/bridge/) For organization with a geodata portal solution it is maybe possible to just sync pycsw`s DB with a DB from e.g. GeoNode or so, if one wants to have that consistent… If you allow me two more questions in this regard: Will pycsw become a dependency for GRASS (with metadata support)? Yes it is. Currently there are no packages for Debian. Pycsw has dependency on SQLalchemy, shapely and pyproj. I have info that debian package will probably be created during a few months. And will it be necessary, that pycsw runs on the same computer as GRASS (meaning also one pycsw for each GRASS installation) or could that somehow be centralized (in other words that GRASS connects to a central pycsw)? Yes of course, the pycsw is providing csw catalogue which can run on a local pc (localhost) or on a server. Our vision is to make an interface based on OWSLib and pycsw which produces essential functions like browsing based on filtering, harvesting and publishing metadata to csw catalogue. The user will be able to choose csw catalogue target like a localhost(probably SQLite) or catalogue on internet. This task will part of GSOC term and after... Thank you, Matej ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hi Matej, Thanks for your reply. Metadata is a current topic I my organization at the moment, so your work comes exactly at the right time for me. Here in NINA people are using many different kinds of GIS software and we are considering introducing GeoNetwork as a central metadata storage solution, which makes metadata management to some extend independent from the GIS software used. Among public and environmental organisations, GeoNetwork is quite popular for metadata management (amongst others Mapping authorities from Netherlands, Switzerland, and the Nordic countries are using it, see also: http://geonetwork-opensource.org/gallery/gallery.html). I have no real personal experience with GeoNetwor (yet), but from what I read in the documentation I could imagine two ways how GRASS metadata could make their way into such a central metadata catalogue: 1) XML import (+ this option is in place when GRASS metadata is written to XML, - import requires manual labor) 2) Metadata harvesting (GeoNetwork can “harvest” metadata from other CSWs, so maybe pycsw already provides automatic an interface in that direction(?). That would be great!) When it comes to consistent metadata, I guess e.g. lists with values for metadata fields will in practice have to be more or less dynamic (as new staff member are being employed, new topics for spatial analysis arise (requiring new keywords), new types of data being developed and so on). Not sure if manipulating the templates is convenient enough for average users (seems to be mainly suitable for users with some degree of programming experience). However, good to know that providing a custom set of values is already possible through the templates. If you start working on a DB for metadata related content, feel free to let me know if I can support you by any means (yet I will be on leave for almost the rest of the year, which means I can spend only a limited amount of time). However, I had a second look on GeoNetwork in this regard, and GeoNetwork is (naturally) using a database for metadata management (one option here is PostgreSQL). If it would be possible to just connect to that same DB (e.g. for fetching and storing key words, data on contact persons…), that would be really, really great, as this ensures that metadata produced in GRASS are 100% compliant to metadata standards used in GeoNetwork (if GRASS users also use GeoNetwork). Anyway, Rome was not built in a day and I am 100% sure that your work will be very, very useful already from what you planned for GSoC. All the best, Stefan ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hi Stefan, On Thu, Jul 3, 2014 at 9:46 AM, Blumentrath, Stefan stefan.blumentr...@nina.no wrote: Hi Matej, Thanks for your reply. Metadata is a current topic I my organization at the moment, so your work comes exactly at the right time for me. Here in NINA people are using many different kinds of GIS software and we are considering introducing GeoNetwork as a central metadata storage solution, which makes metadata management to some extend independent from the GIS software used. Among public and environmental organisations, GeoNetwork is quite popular for metadata management (amongst others Mapping authorities from Netherlands, Switzerland, and the Nordic countries are using it, see also: http://geonetwork-opensource.org/gallery/gallery.html). I have no real personal experience with GeoNetwor (yet), but from what I read in the documentation I could imagine two ways how GRASS metadata could make their way into such a central metadata catalogue: 1) XML import (+ this option is in place when GRASS metadata is written to XML, - import requires manual labor) 2) Metadata harvesting (GeoNetwork can “harvest” metadata from other CSWs, so maybe pycsw already provides automatic an interface in that direction(?). That would be great!) pycsw is a CSW server that allows to harvest information from other OGC services, as well as Geonetwork. This latter offers also the editor, while pycsw doesn't, but is used internally by other applications which offer full metadata management (and a GUI), such as CKAN, GeoNode, etc. and the idea is to add GRASS to this list as well :-) If you want to stick with geonetwork, you can harvest the metadata from the pycsw, there's no need of manual work. When it comes to consistent metadata, I guess e.g. lists with values for metadata fields will in practice have to be more or less dynamic (as new staff member are being employed, new topics for spatial analysis arise (requiring new keywords), new types of data being developed and so on). Not sure if manipulating the templates is convenient enough for average users (seems to be mainly suitable for users with some degree of programming experience). However, good to know that providing a custom set of values is already possible through the templates. If you start working on a DB for metadata related content, feel free to let me know if I can support you by any means (yet I will be on leave for almost the rest of the year, which means I can spend only a limited amount of time). The idea is to allow user to generate a custom template using the gui, that can be reused for any metadata generated afterwards. Your idea of a database deserves consideration as well Thanks Madi However, I had a second look on GeoNetwork in this regard, and GeoNetwork is (naturally) using a database for metadata management (one option here is PostgreSQL). If it would be possible to just connect to that same DB (e.g. for fetching and storing key words, data on contact persons…), that would be really, really great, as this ensures that metadata produced in GRASS are 100% compliant to metadata standards used in GeoNetwork (if GRASS users also use GeoNetwork). Anyway, Rome was not built in a day and I am 100% sure that your work will be very, very useful already from what you planned for GSoC. All the best, Stefan ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev -- Best regards, Dr. Margherita DI LEO Scientific / technical project officer European Commission - DG JRC Institute for Environment and Sustainability (IES) Via Fermi, 2749 I-21027 Ispra (VA) - Italy - TP 261 Tel. +39 0332 78 3600 margherita.di-...@jrc.ec.europa.eu Disclaimer: The views expressed are purely those of the writer and may not in any circumstance be regarded as stating an official position of the European Commission. Hi Matej, Thanks for your reply. Metadata is a current topic I my organization at the moment, so your work comes exactly at the right time for me. Here in NINA people are using many different kinds of GIS software and we are considering introducing GeoNetwork as a central metadata storage solution, which makes metadata management to some extend independent from the GIS software used. Among public and environmental organisations, GeoNetwork is quite popular for metadata management (amongst others Mapping authorities from Netherlands, Switzerland, and the Nordic countries are using it, see also: http://geonetwork-opensource.org/gallery/gallery.html). I have no real personal experience with GeoNetwor (yet), but from what I read in the documentation I could imagine two ways how GRASS metadata could make their way into such a central metadata catalogue: 1) XML import (+ this option is in place when GRASS metadata is written to XML, - import requires manual labor) 2) Metadata harvesting (GeoNetwork can “harvest”
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hei Madi, Thanks for your clarification. Very interesting! I see I have to take a closer look on pycsw and it`s front-ends (as a possible alternative to GeoNetwork). The pycsw documentation [1] says that also pycsw Metadata repositories are set up with a database backend (SQLite, PostgreSQL, and even PostGIS support). However, integration with data portal solutions like e.g. GeoNode or Open Data Catalogue is unfortunately “read-only” meaning that meta-data can be only queried [2]. Would be nice if metadata from GRASS could somehow go directly into a geodata portal (and that the other way around e.g. people, keywords … could be fetched from there)… (BTW, in the proprietary world such a functionality requires yet another extension: https://www.geocat.net/bridge/) For organization with a geodata portal solution it is maybe possible to just sync pycsw`s DB with a DB from e.g. GeoNode or so, if one wants to have that consistent… If you allow me two more questions in this regard: Will pycsw become a dependency for GRASS (with metadata support)? And will it be necessary, that pycsw runs on the same computer as GRASS (meaning also one pycsw for each GRASS installation) or could that somehow be centralized (in other words that GRASS connects to a central pycsw)? Cheers Stefan [1] http://docs.pycsw.org/en/1.8.3/administration.html#metadata-repository-setup [2] http://docs.pycsw.org/en/1.8.3/geonode.html From: Margherita Di Leo [mailto:dileomargher...@gmail.com] Sent: 3. juli 2014 13:40 To: Blumentrath, Stefan Cc: Matej Krejci; Martin Landa; GRASS-dev Subject: Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6 Hi Stefan, On Thu, Jul 3, 2014 at 9:46 AM, Blumentrath, Stefan stefan.blumentr...@nina.nomailto:stefan.blumentr...@nina.no wrote: Hi Matej, Thanks for your reply. Metadata is a current topic I my organization at the moment, so your work comes exactly at the right time for me. Here in NINA people are using many different kinds of GIS software and we are considering introducing GeoNetwork as a central metadata storage solution, which makes metadata management to some extend independent from the GIS software used. Among public and environmental organisations, GeoNetwork is quite popular for metadata management (amongst others Mapping authorities from Netherlands, Switzerland, and the Nordic countries are using it, see also: http://geonetwork-opensource.org/gallery/gallery.html). I have no real personal experience with GeoNetwor (yet), but from what I read in the documentation I could imagine two ways how GRASS metadata could make their way into such a central metadata catalogue: 1) XML import (+ this option is in place when GRASS metadata is written to XML, - import requires manual labor) 2) Metadata harvesting (GeoNetwork can “harvest” metadata from other CSWs, so maybe pycsw already provides automatic an interface in that direction(?). That would be great!) pycsw is a CSW server that allows to harvest information from other OGC services, as well as Geonetwork. This latter offers also the editor, while pycsw doesn't, but is used internally by other applications which offer full metadata management (and a GUI), such as CKAN, GeoNode, etc. and the idea is to add GRASS to this list as well :-) If you want to stick with geonetwork, you can harvest the metadata from the pycsw, there's no need of manual work. When it comes to consistent metadata, I guess e.g. lists with values for metadata fields will in practice have to be more or less dynamic (as new staff member are being employed, new topics for spatial analysis arise (requiring new keywords), new types of data being developed and so on). Not sure if manipulating the templates is convenient enough for average users (seems to be mainly suitable for users with some degree of programming experience). However, good to know that providing a custom set of values is already possible through the templates. If you start working on a DB for metadata related content, feel free to let me know if I can support you by any means (yet I will be on leave for almost the rest of the year, which means I can spend only a limited amount of time). The idea is to allow user to generate a custom template using the gui, that can be reused for any metadata generated afterwards. Your idea of a database deserves consideration as well Thanks Madi However, I had a second look on GeoNetwork in this regard, and GeoNetwork is (naturally) using a database for metadata management (one option here is PostgreSQL). If it would be possible to just connect to that same DB (e.g. for fetching and storing key words, data on contact persons…), that would be really, really great, as this ensures that metadata produced in GRASS are 100% compliant to metadata standards used in GeoNetwork (if GRASS users also use GeoNetwork). Anyway, Rome was not built in a day and I am 100% sure
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hi Matej, Thanks for working on metadata support in GRASS! I am very much looking forward to having the modules you are developing in core. The documentation about your work which you have on the Wiki looks very, very promising! I was wondering, did you consider storing keyword lists, points of contacts, and so on in a central (SQLite) database (comparable to what the t.*-modules do with time series)? For organizations it would be useful to have tools which produce consistent meta-data (same set of key-words used by all members of staff…). Furthermore, efficiency might be improved by having e.g. table with contact persons within an organization (where all contact details are saved) one can pick from in a combo-box. Such a DB should probably be accessible from all locations and mapsets… Just an idea, and I do not know how much work that would be… BTW, is the metadata publishing functionality based on owslib + pycsw, which you mention on your wiki-page, somehow related to GeoNetwork? Cheers and thanks again for your excellent work! Stefan ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hi Stefan, 2014-07-02 9:35 GMT+02:00 Blumentrath, Stefan stefan.blumentr...@nina.no : Hi Matej, I was wondering, did you consider storing keyword lists, points of contacts, and so on in a central (SQLite) database (comparable to what the t.*-modules do with time series)? For organizations it would be useful to have tools which produce consistent meta-data (same set of key-words used by all members of staff…). Current possibility how to produce consistent metadata (same set of key) is by modifcation jinja template [1]. Values in jinja template are represented by Python objects. You can just replace these objects by desire values for creating pattern. What do you think? Furthermore, efficiency might be improved by having e.g. table with contact pers ons within an organization (where all contact details are saved) one can pick from in a combo-box. Such a DB should probably be accessible from all locations and mapsets… It makes sense to me. This task should not be problem, but is more than essential. Probably I'll do it after the GSOC term. BTW, is the metadata publishing functionality based on owslib + pycsw, which you mention on your wiki-page, somehow related to GeoNetwork? No, I dont know this project in detail... Thank you for useful hints! Cheers Matej [1] https://svn.osgeo.org/grass/sandbox/krejcmat/src/data/grassInspireTemplate.xml ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
On 29 June 2014 21:57, Matej Krejci matejkre...@gmail.com wrote: Hi all, Hi Matej to finish version 1.1 (by wikipage) of g.gui.editor I think that g.gui.editor it is not so clear, could you rename it to g.gui.metadata or g.gui.metaeditor? to connect GUI editor (middle panel) with OWSLib and Jinja template Best, Matej -- ciao Luca http://gis.cri.fmach.it/delucchi/ www.lucadelu.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hi, 2014-06-30 9:37 GMT+02:00 Luca Delucchi lucadel...@gmail.com: I think that g.gui.editor it is not so clear, could you rename it to g.gui.metadata or g.gui.metaeditor? right, `g.gui.metadata` seems to be the most appropriate, Martin -- Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
g.gui.metadata is the most clear to me. Doug On Mon, Jun 30, 2014 at 4:30 AM, Martin Landa landa.mar...@gmail.com wrote: Hi, 2014-06-30 9:37 GMT+02:00 Luca Delucchi lucadel...@gmail.com: I think that g.gui.editor it is not so clear, could you rename it to g.gui.metadata or g.gui.metaeditor? right, `g.gui.metadata` seems to be the most appropriate, Martin -- Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev -- Doug Newcomb USFWS Raleigh, NC 919-856-4520 ext. 14 doug_newc...@fws.gov - The opinions I express are my own and are not representative of the official policy of the U.S.Fish and Wildlife Service or Dept. of the Interior. Life is too short for undocumented, proprietary data formats. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [SoC] gsoc2014 grass metadata week 6
Hi, 2014-06-30 9:37 GMT+02:00 Luca Delucchi lucadel...@gmail.com: On 29 June 2014 21:57, Matej Krejci matejkre...@gmail.com wrote: Hi all, Hi Matej to finish version 1.1 (by wikipage) of g.gui.editor I think that g.gui.editor it is not so clear, could you rename it to g.gui.metadata or g.gui.metaeditor? Thanks for notice. The right/current name is g.gui.metadata. I just wrote name wrongly in weekly report. Sorry for confusion. However, the final name of module is open. Matej ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
