Re: [OSGeo-Discuss] Re: git like for geodata management
This is one of the most needed feature in today's enterprise systems.Great work. Regards Kumaran --- On Sat, 11/27/10, Cameron Shorter cameron.shor...@gmail.com wrote: From: Cameron Shorter cameron.shor...@gmail.com Subject: Re: [OSGeo-Discuss] Re: git like for geodata management To: OSGeo Discussions discuss@lists.osgeo.org Date: Saturday, November 27, 2010, 12:26 AM Awesome! Are there any plans to roll this functionality into the core postgis? On 26/11/10 09:32, Horst Düster wrote: Some times ago the upper subject was discussed on the osgeo.org list. Now I'm able to announce the first release of the pgvs system I developed in the last few month. The aim of pgvs is to offer an option to edit PostGIS layers with CVS, SVN or GIT like support for concurrencing editing of geodata stored in a PostGIS Database. Additionally I've developed a QGIS plugin to support your work with pgvs in conjunction with QGIS. For further information have a look at: http://www.kappasys.ch/cms/index.php?id=23L=5 Regards Horst Düster ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss -- Cameron Shorter Geospatial Director Tel: +61 (0)2 8570 5050 Mob: +61 (0)419 142 254 Think Globally, Fix Locally Geospatial Solutions enhanced with Open Standards and Open Source http://www.lisasoft.com ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Re: git like for geodata management
Hi Noli Of course it is possible to manage more than 2 revisions. On my website its only an example. In the moment the revisions are depending on the user who edits the data. You can define as many users as you want. As I understood right you can solve your problem with introducing several new users for the different periods. I hope that helps. Horst Am 26.11.2010 00:46, schrieb Noli Sicad: Hi Horst, In the example given in your website, you have 2 revisions, could you do more than 2 revisions? For example, in forest management we do plant, thin and harvest the forest for period of time (i.e 100 years planning period). You keep on editing planting, thinning and harvesting plans of the same forest. We like to see the revisions of these 3 plans (3 layers) e.g. 5 years, 10 years or 50 years, the edits that have been in these periods. Could we use this plugin for this purpose? I think this scenario also apply to agricultural management (planting and harvesting of crops) and animal range management - grazing. Thanks. Noli On 11/26/10, Horst Düster horst.dues...@kappasys.ch wrote: Some times ago the upper subject was discussed on the osgeo.org list. Now I'm able to announce the first release of the pgvs system I developed in the last few month. The aim of pgvs is to offer an option to edit PostGIS layers with CVS, SVN or GIT like support for concurrencing editing of geodata stored in a PostGIS Database. Additionally I've developed a QGIS plugin to support your work with pgvs in conjunction with QGIS. For further information have a look at: http://www.kappasys.ch/cms/index.php?id=23L=5 Regards Horst Düster ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Re: git like for geodata management
Hi Kalyan When a conflict is encountered, conflict presented to the user for resolution. At this stage, will all the individual transactions that make up the version be rolled back as well? No of course not. Only the single conflicts are handled. All other individual transactions are not rolled back they persists in an uncommited state. That means when conflicts are encountered nothing will be commited until all conflicts are solved. Than the user is able to commit the individual editings to the DB. Horst ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Re: git like for geodata management
Awesome! Are there any plans to roll this functionality into the core postgis? On 26/11/10 09:32, Horst Düster wrote: Some times ago the upper subject was discussed on the osgeo.org list. Now I'm able to announce the first release of the pgvs system I developed in the last few month. The aim of pgvs is to offer an option to edit PostGIS layers with CVS, SVN or GIT like support for concurrencing editing of geodata stored in a PostGIS Database. Additionally I've developed a QGIS plugin to support your work with pgvs in conjunction with QGIS. For further information have a look at: http://www.kappasys.ch/cms/index.php?id=23L=5 Regards Horst Düster ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss -- Cameron Shorter Geospatial Director Tel: +61 (0)2 8570 5050 Mob: +61 (0)419 142 254 Think Globally, Fix Locally Geospatial Solutions enhanced with Open Standards and Open Source http://www.lisasoft.com ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Re: git like for geodata management
Hi Horst, In the example given in your website, you have 2 revisions, could you do more than 2 revisions? For example, in forest management we do plant, thin and harvest the forest for period of time (i.e 100 years planning period). You keep on editing planting, thinning and harvesting plans of the same forest. We like to see the revisions of these 3 plans (3 layers) e.g. 5 years, 10 years or 50 years, the edits that have been in these periods. Could we use this plugin for this purpose? I think this scenario also apply to agricultural management (planting and harvesting of crops) and animal range management - grazing. Thanks. Noli On 11/26/10, Horst Düster horst.dues...@kappasys.ch wrote: Some times ago the upper subject was discussed on the osgeo.org list. Now I'm able to announce the first release of the pgvs system I developed in the last few month. The aim of pgvs is to offer an option to edit PostGIS layers with CVS, SVN or GIT like support for concurrencing editing of geodata stored in a PostGIS Database. Additionally I've developed a QGIS plugin to support your work with pgvs in conjunction with QGIS. For further information have a look at: http://www.kappasys.ch/cms/index.php?id=23L=5 Regards Horst Düster ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Re: git like for geodata management
Hi Horst, It looks nice and appreciate. Version control must be integral part of a geospatial data transacton management and so this is a good effort. Some more details need to be added to the conflict resolutions at the time of posting -- When a conflict is encountered, conflict presented to the user for resolution. At this stage, will all the individual transactions that make up the version be rolled back as well? Cheers - kalyan -Original Message- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Horst Düster Sent: Friday, 26 November 2010 9:32 AM To: discuss@lists.osgeo.org Subject: [OSGeo-Discuss] Re: git like for geodata management Some times ago the upper subject was discussed on the osgeo.org list. Now I'm able to announce the first release of the pgvs system I developed in the last few month. The aim of pgvs is to offer an option to edit PostGIS layers with CVS, SVN or GIT like support for concurrencing editing of geodata stored in a PostGIS Database. Additionally I've developed a QGIS plugin to support your work with pgvs in conjunction with QGIS. For further information have a look at: http://www.kappasys.ch/cms/index.php?id=23L=5 Regards Horst Düster ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss *** This message is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete it and notify the sender. Views expressed in this message are those of the individual sender, and are not necessarily the views of the Land and Property Management Authority. This email message has been swept by MIMEsweeper for the presence of computer viruses. *** Please consider the environment before printing this email. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Re: git like for geodata management
This is cool! Thanks Noli! Anyone tried this already? (can't compile source while in the middle of a critical project) (cross posted to qgis list) On Thu, Sep 30, 2010 at 12:50 PM, Noli Sicad nsi...@gmail.com wrote: Hi, FYI, there is a new offline plugin for QGIS that synchronizes data source ( e.g. PostGIS or WFS-T) to a spatialite database and storing the offline edits to dedicated tables. http://www.sourcepole.ch/2010/9/29/offline-editing-plugin-for-qgis Screenshot http://www.sourcepole.ch/assets/2010/9/27/screenshot1.png Noli ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss -- cheers, maning -- Freedom is still the most radical idea of all -N.Branden wiki: http://esambale.wikispaces.com/ blog: http://epsg4253.wordpress.com/ -- ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Re: git like for geodata management
Hi, FYI, there is a new offline plugin for QGIS that synchronizes data source ( e.g. PostGIS or WFS-T) to a spatialite database and storing the offline edits to dedicated tables. http://www.sourcepole.ch/2010/9/29/offline-editing-plugin-for-qgis Screenshot http://www.sourcepole.ch/assets/2010/9/27/screenshot1.png Noli ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Re: git like for geodata management
Perhaps the only online service I can think of that is similar to what I mentioned is openstreetmap's API: http://wiki.openstreetmap.org/wiki/API_v0.6 On Mon, Sep 27, 2010 at 3:13 PM, Kalyan Janakiraman kalyan.janakira...@lpma.nsw.gov.au wrote: Hi Versions are also used to demarcate the the geospatial transaction boundary. I didn’t see this point articulated. We have been sucessfully running replication of ArcSDE geodatabase from data maintenance environment to different geodatabase repositories (about 150 repositories) for many years now through event-driven mediation framework. Because we had used the event-driven mediation approach, we could replicate irrespective of the version or the vendor. Because ESRI didn’t support robust replication before, we did this ourselves. In gist the version is boundary of each geospatial transaction. When a version is posted, the transactions in it are picked up and shipped across as XML event feeds. I published this as a paper. I can send it to anyone interested. - Kalyan From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Ragi Burhum Sent: Friday, 24 September 2010 4:06 AM To: discuss@lists.osgeo.org; Noli Sicad Subject: [OSGeo-Discuss] Re: git like for geodata management Hi Noli, thanks for the link. That is definitely a step in the right direction, but it is hardly comparable to git ArcSDE versioning at that. The article and sample code you describe above generates hashes for all rows and tables in the db and compares them to the target db. So 1 million rows in a db, regardless if the two dbs are identical, would cause 1 million hashes to go over the wire. Every single time you ask to sync you pay the price. Git and ArcSDE keep track of changesets, and when it is time to synchronize, they exchange that changeset and apply it. One insert? That is all that needs to be sent. Another issue is that there is nothing about conflict resolution there (what happens when you delete one row in one db and modify it in another one?). There is also the problem of allowing multiple versions of the data in the same db (Like having multiple heads). Regardless, thank you for the link, - Ragi Date: Thu, 23 Sep 2010 13:22:17 +1000 From: Noli Sicad nsi...@gmail.com Subject: Re: [OSGeo-Discuss] Re: git like for geodata management To: OSGeo Discussions discuss@lists.osgeo.org Message-ID: aanlkti=3anc4baand4hk9uuzfsasxn-8ybpnkyong...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 PostgreSQL Synchronization Tool --- psync [1] The article introduces a method of synchronizing two PostgreSQL databases. Although, this seems to be an easy task, no product (slony, londiste, ...) really satisfied the needs within the maps.bremen.de project. Either they have special prerequsits that didn't apply for our problem or they didn't support synchronizing of large objects. Large objects are used to store tiles of a street/aerial map within PostgreSQL. My GIS-server queries the database and gets the tiles out. By using this construction we are getting a flexible infrastructure for updating and maintaining different versions of the maps. Everything was working fine until the service needs to be spread over three servers. How can we easily synchronize the databases? I really found no really working solution that is clean and easy to use. [1]http://www.codeproject.com/KB/database/psync.aspx Noli On 9/23/10, Ragi Burhum r...@burhum.com wrote: Are you looking for an alternative to (1)ESRI's versioning, (2)ESRI's disconnected editing, or a mix of both (3)git like? the scenario that you described first was more like (2), but this one fits (1). I would love to see something like (3), but truth of the matter, AFAIK, there is nothing like that implemented for geo (yet). On Sep 22, 2010, at 9:00 AM, discuss-requ...@lists.osgeo.org wrote: On Wed, 2010-09-22 at 12:10 +0800, maning sambale wrote: Any real world cases for this? Imagine the following scenario: * 50 ~ 70 digitizers * 5 QA * 1 Manager Each QA has 10 digitizers assigned. After all the data is validated, the manager merges it and generates the geodb. All users work against the same DB, most of them linked. This causes disconnections, duplicated data, and lots of random errors. Also, they can't be forced to work on different DB's because they are all working on the same project, at the same time. This is the real scenario of GISWorking (http://www.gisworking.com/), a company we are working with. It would be perfect to have smaller groups (ideally 1 person), working against separated databases, but that can be synchronized with the rest of the data when needed. Then each QA merges data from the people he supervises. After it's validated the manager merges the complete dataset, and generates the final
Re: [OSGeo-Discuss] Re: git like for geodata management [SEC=UNCLASSIFIED]
This might be too far off the original idea... But: It might be interesting to look into a more disconnected way of managing geospatial data with the Web. As in having servers advertize latest changes in geometry and attributes using GeoRSS and clients picking these up and integrating them (potentially including processing, changes) into their cache or copy of the data. Which might automagically turn into a server itself advertizing changes following the same pattern. This idea grew on me over the past years after seeing Ward Cunningham give his presentation at Wikimania 2005 in Fankfurt on bits of information floating around the Web [1]. Maybe now the time is ready to implement some of these thoughts in our domain. I am currently seeding these ideas to the European National Mapping Agencies through the ESDIN project. Yes I know, it sounds a bit remote but hey, it is a nice vision... :-) Have fun, Arnulf. [1] Slides: http://c2.com/doc/wikimania/ Screener: http://www.archive.org/details/WIKIMANIA_TAPE_019_SCREENER.mov Low res video runs 30 minutes On 09/24/2010 01:40 AM, Bruce Bannerman wrote: Ragi, I agree. I think that we have a way to go yet to have something comparable to the ArcSDE / ArcGIS Multi-versioning and version conflict detection functionality. The advantage that the ArcSDE solution has is that edits are made directly within the database. This works well within an Enterprise environment as described by Fabio earlier in this thread. I may be wrong, but I think that git works on files, but I haven’t used it myself. Can git detect changes to the spatial representation of a feature within a binary file? Also, speaking as someone who implemented an ArcSDE/ArcGIS Multi-versioned edit scenario several years ago, the ESRI solution is far from perfect. It imposes very strict environment management on the system managers, e.g.: * All versions of the software used (client and server) must be at precisely the same version, service pack and patch; * The environment can only use software that implements the ArcObjects environment (from experience, this rules out the use of the ArcSDE Java and C API’s); * Editors must be well trained and knowledgeable in using both ArcGIS and Multi-versioned processes; * The Organisation needs to think through their maintenance processes to get best advantage of the functionality; and * It doesn’t remove the need for data maintenance people to talk to each other about work that is going on, as the software cannot resolve all conflicts. For example, if two editors make changes to the spatial representation of a feature, which one is correct? The software will detect the conflict, but the editors (or their managers) will need to resolve the issue of which version of the feature’s spatial representation is correct. Bruce On 24/09/10 4:05 AM, Ragi Burhum r...@burhum.com wrote: Hi Noli, thanks for the link. That is definitely a step in the right direction, but it is hardly comparable to git ArcSDE versioning at that. The article and sample code you describe above generates hashes for all rows and tables in the db and compares them to the target db. So 1 million rows in a db, regardless if the two dbs are identical, would cause 1 million hashes to go over the wire. Every single time you ask to sync you pay the price. Git and ArcSDE keep track of changesets, and when it is time to synchronize, they exchange that changeset and apply it. One insert? That is all that needs to be sent. Another issue is that there is nothing about conflict resolution there (what happens when you delete one row in one db and modify it in another one?). There is also the problem of allowing multiple versions of the data in the same db (Like having multiple heads). Regardless, thank you for the link, - Ragi Date: Thu, 23 Sep 2010 13:22:17 +1000 From: Noli Sicad nsi...@gmail.com Subject: Re: [OSGeo-Discuss] Re: git like for geodata management To: OSGeo Discussions discuss@lists.osgeo.org Message-ID: aanlkti=3anc4baand4hk9uuzfsasxn-8ybpnkyong...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 PostgreSQL Synchronization Tool --- psync [1] The article introduces a method of synchronizing two PostgreSQL databases. Although, this seems to be an easy task, no product (slony, londiste, ...) really satisfied the needs within the maps.bremen.de http://maps.bremen.de http://maps.bremen.de project. Either they have special prerequsits that didn't apply for our problem or they didn't support synchronizing of large objects. Large objects are used to store tiles of a street/aerial map within PostgreSQL. My GIS-server queries the database and gets the tiles out
Re: [OSGeo-Discuss] Re: git like for geodata management [SEC=UNCLASSIFIED]
Hi, Yes Smallworld did a lot of innovation in the area of version management and long transactions some 20 years ago, I was quite involved in all that. The technical paper you mention is at archive.org, a great resource for finding web pages that have disappeared! http://web.archive.org/web/20080519210011/http://cfis.savagexi.com/pages/technical_paper_4 This paper, which I was a co-author of, talks about long transactions as well as query architectures: http://web.archive.org/web/20070828091638/cfis.savagexi.com/pages/technical_paper_8 The whole list of Smallworld technical papers is at http://web.archive.org/web/20080519210016/cfis.savagexi.com/pages/technical_papers ESRI, Oracle and others adopted a number of the version management ideas that Smallworld pioneered, but (IMHO) still don't have nearly as robust an implementation as Smallworld does. If anyone has questions on this area, I'm happy to try to answer them. If you're not familiar with Smallworld, there's a bit more background on my blog at http://bit.ly/cDUid1 Cheers, Peter. On Fri, Sep 24, 2010 at 3:55 AM, STEPHEN STANTON sstan...@btinternet.comwrote: I'm a bit surprised no-one mentions Smallworld. Weren't they at the forefront of version management almost 20 years ago? The following link to an interesting technical paper isn't currently working, but maybe it'll come back later: http://cfis.savagexi.com/pages/technical_paper_4 Steve On 09/24/2010 01:40 AM, Bruce Bannerman wrote: Ragi, I agree. I think that we have a way to go yet to have something comparable to the ArcSDE / ArcGIS Multi-versioning and version conflict detection functionality. The advantage that the ArcSDE solution has is that edits are made directly within the database. This works well within an Enterprise environment as described by Fabio earlier in this thread. I may be wrong, but I think that git works on files, but I haven’t used it myself. Can git detect changes to the spatial representation of a feature within a binary file? Also, speaking as someone who implemented an ArcSDE/ArcGIS Multi-versioned edit scenario several years ago, the ESRI solution is far from perfect. It imposes very strict environment management on the system managers, e.g.: * All versions of the software used (client and server) must be at precisely the same version, service pack and patch; * The environment can only use software that implements the ArcObjects environment (from experience, this rules out the use of the ArcSDE Java and C API’s); * Editors must be well trained and knowledgeable in using both ArcGIS and Multi-versioned processes; * The Organisation needs to think through their maintenance processes to get best advantage of the functionality; and * It doesn’t remove the need for data maintenance people to talk to each other about work that is going on, as the software cannot resolve all conflicts. For example, if two editors make changes to the spatial representation of a feature, which one is correct? The software will detect the conflict, but the editors (or their managers) will need to resolve the issue of which version of the feature’s spatial representation is correct. Bruce ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Re: git like for geodata management [SEC=UNCLASSIFIED]
Arnulf, that's exactly what the OGC Geosynchronization spec describes! Servers advertise their changes in a GeoRSS feed -- where the content element describes the change in WFS-T XML. It's nicely loosely coupled, because as you eloquently state, clients can decide what they want to do with the suggested modifications, accept them all, just use those in a certain area, or those from a trusted agency, etc. --- Raj On Sep 24, at 4:08 AM, Seven (aka Arnulf) wrote: This might be too far off the original idea... But: It might be interesting to look into a more disconnected way of managing geospatial data with the Web. As in having servers advertize latest changes in geometry and attributes using GeoRSS and clients picking these up and integrating them (potentially including processing, changes) into their cache or copy of the data. Which might automagically turn into a server itself advertizing changes following the same pattern. This idea grew on me over the past years after seeing Ward Cunningham give his presentation at Wikimania 2005 in Fankfurt on bits of information floating around the Web [1]. Maybe now the time is ready to implement some of these thoughts in our domain. I am currently seeding these ideas to the European National Mapping Agencies through the ESDIN project. Yes I know, it sounds a bit remote but hey, it is a nice vision... :-) Have fun, Arnulf. [1] Slides: http://c2.com/doc/wikimania/ Screener: http://www.archive.org/details/WIKIMANIA_TAPE_019_SCREENER.mov Low res video runs 30 minutes On 09/24/2010 01:40 AM, Bruce Bannerman wrote: Ragi, I agree. I think that we have a way to go yet to have something comparable to the ArcSDE / ArcGIS Multi-versioning and version conflict detection functionality. The advantage that the ArcSDE solution has is that edits are made directly within the database. This works well within an Enterprise environment as described by Fabio earlier in this thread. I may be wrong, but I think that git works on files, but I haven’t used it myself. Can git detect changes to the spatial representation of a feature within a binary file? Also, speaking as someone who implemented an ArcSDE/ArcGIS Multi-versioned edit scenario several years ago, the ESRI solution is far from perfect. It imposes very strict environment management on the system managers, e.g.: * All versions of the software used (client and server) must be at precisely the same version, service pack and patch; * The environment can only use software that implements the ArcObjects environment (from experience, this rules out the use of the ArcSDE Java and C API’s); * Editors must be well trained and knowledgeable in using both ArcGIS and Multi-versioned processes; * The Organisation needs to think through their maintenance processes to get best advantage of the functionality; and * It doesn’t remove the need for data maintenance people to talk to each other about work that is going on, as the software cannot resolve all conflicts. For example, if two editors make changes to the spatial representation of a feature, which one is correct? The software will detect the conflict, but the editors (or their managers) will need to resolve the issue of which version of the feature’s spatial representation is correct. Bruce On 24/09/10 4:05 AM, Ragi Burhum r...@burhum.com wrote: Hi Noli, thanks for the link. That is definitely a step in the right direction, but it is hardly comparable to git ArcSDE versioning at that. The article and sample code you describe above generates hashes for all rows and tables in the db and compares them to the target db. So 1 million rows in a db, regardless if the two dbs are identical, would cause 1 million hashes to go over the wire. Every single time you ask to sync you pay the price. Git and ArcSDE keep track of changesets, and when it is time to synchronize, they exchange that changeset and apply it. One insert? That is all that needs to be sent. Another issue is that there is nothing about conflict resolution there (what happens when you delete one row in one db and modify it in another one?). There is also the problem of allowing multiple versions of the data in the same db (Like having multiple heads). Regardless, thank you for the link, - Ragi Date: Thu, 23 Sep 2010 13:22:17 +1000 From: Noli Sicad nsi...@gmail.com Subject: Re: [OSGeo-Discuss] Re: git like for geodata management To: OSGeo Discussions discuss@lists.osgeo.org Message-ID: aanlkti=3anc4baand4hk9uuzfsasxn-8ybpnkyong...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 PostgreSQL Synchronization Tool --- psync [1] The article introduces a method of synchronizing two PostgreSQL
Re: [OSGeo-Discuss] Re: git like for geodata management [SEC=UNCLASSIFIED]
Ragi, I agree. I think that we have a way to go yet to have something comparable to the ArcSDE / ArcGIS Multi-versioning and version conflict detection functionality. The advantage that the ArcSDE solution has is that edits are made directly within the database. This works well within an Enterprise environment as described by Fabio earlier in this thread. I may be wrong, but I think that git works on files, but I haven't used it myself. Can git detect changes to the spatial representation of a feature within a binary file? Also, speaking as someone who implemented an ArcSDE/ArcGIS Multi-versioned edit scenario several years ago, the ESRI solution is far from perfect. It imposes very strict environment management on the system managers, e.g.: * All versions of the software used (client and server) must be at precisely the same version, service pack and patch; * The environment can only use software that implements the ArcObjects environment (from experience, this rules out the use of the ArcSDE Java and C API's); * Editors must be well trained and knowledgeable in using both ArcGIS and Multi-versioned processes; * The Organisation needs to think through their maintenance processes to get best advantage of the functionality; and * It doesn't remove the need for data maintenance people to talk to each other about work that is going on, as the software cannot resolve all conflicts. For example, if two editors make changes to the spatial representation of a feature, which one is correct? The software will detect the conflict, but the editors (or their managers) will need to resolve the issue of which version of the feature's spatial representation is correct. Bruce On 24/09/10 4:05 AM, Ragi Burhum r...@burhum.com wrote: Hi Noli, thanks for the link. That is definitely a step in the right direction, but it is hardly comparable to git ArcSDE versioning at that. The article and sample code you describe above generates hashes for all rows and tables in the db and compares them to the target db. So 1 million rows in a db, regardless if the two dbs are identical, would cause 1 million hashes to go over the wire. Every single time you ask to sync you pay the price. Git and ArcSDE keep track of changesets, and when it is time to synchronize, they exchange that changeset and apply it. One insert? That is all that needs to be sent. Another issue is that there is nothing about conflict resolution there (what happens when you delete one row in one db and modify it in another one?). There is also the problem of allowing multiple versions of the data in the same db (Like having multiple heads). Regardless, thank you for the link, - Ragi Date: Thu, 23 Sep 2010 13:22:17 +1000 From: Noli Sicad nsi...@gmail.com Subject: Re: [OSGeo-Discuss] Re: git like for geodata management To: OSGeo Discussions discuss@lists.osgeo.org Message-ID: aanlkti=3anc4baand4hk9uuzfsasxn-8ybpnkyong...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 PostgreSQL Synchronization Tool --- psync [1] The article introduces a method of synchronizing two PostgreSQL databases. Although, this seems to be an easy task, no product (slony, londiste, ...) really satisfied the needs within the maps.bremen.de http://maps.bremen.de http://maps.bremen.de project. Either they have special prerequsits that didn't apply for our problem or they didn't support synchronizing of large objects. Large objects are used to store tiles of a street/aerial map within PostgreSQL. My GIS-server queries the database and gets the tiles out. By using this construction we are getting a flexible infrastructure for updating and maintaining different versions of the maps. Everything was working fine until the service needs to be spread over three servers. How can we easily synchronize the databases? I really found no really working solution that is clean and easy to use. [1]http://www.codeproject.com/KB/database/psync.aspx http://www.codeproject.com/KB/database/psync.aspx Noli On 9/23/10, Ragi Burhum r...@burhum.com wrote: Are you looking for an alternative to (1)ESRI's versioning, (2)ESRI's disconnected editing, or a mix of both (3)git like? the scenario that you described first was more like (2), but this one fits (1). I would love to see something like (3), but truth of the matter, AFAIK, there is nothing like that implemented for geo (yet). On Sep 22, 2010, at 9:00 AM, discuss-requ...@lists.osgeo.org wrote: On Wed, 2010-09-22 at 12:10 +0800, maning sambale wrote: Any real world cases for this? Imagine the following scenario: * 50 ~ 70 digitizers * 5 QA * 1 Manager Each QA has 10 digitizers assigned. After all the data is validated, the manager merges it and generates the geodb. All users work against the same DB, most of them linked. This causes disconnections, duplicated data, and lots of random errors. Also, they can't be forced to work on different DB's because they are all working
Re: [OSGeo-Discuss] Re: git like for geodata management
PostgreSQL Synchronization Tool --- psync [1] The article introduces a method of synchronizing two PostgreSQL databases. Although, this seems to be an easy task, no product (slony, londiste, ...) really satisfied the needs within the maps.bremen.de project. Either they have special prerequsits that didn't apply for our problem or they didn't support synchronizing of large objects. Large objects are used to store tiles of a street/aerial map within PostgreSQL. My GIS-server queries the database and gets the tiles out. By using this construction we are getting a flexible infrastructure for updating and maintaining different versions of the maps. Everything was working fine until the service needs to be spread over three servers. How can we easily synchronize the databases? I really found no really working solution that is clean and easy to use. [1]http://www.codeproject.com/KB/database/psync.aspx Noli On 9/23/10, Ragi Burhum r...@burhum.com wrote: Are you looking for an alternative to (1)ESRI's versioning, (2)ESRI's disconnected editing, or a mix of both (3)git like? the scenario that you described first was more like (2), but this one fits (1). I would love to see something like (3), but truth of the matter, AFAIK, there is nothing like that implemented for geo (yet). On Sep 22, 2010, at 9:00 AM, discuss-requ...@lists.osgeo.org wrote: On Wed, 2010-09-22 at 12:10 +0800, maning sambale wrote: Any real world cases for this? Imagine the following scenario: * 50 ~ 70 digitizers * 5 QA * 1 Manager Each QA has 10 digitizers assigned. After all the data is validated, the manager merges it and generates the geodb. All users work against the same DB, most of them linked. This causes disconnections, duplicated data, and lots of random errors. Also, they can't be forced to work on different DB's because they are all working on the same project, at the same time. This is the real scenario of GISWorking (http://www.gisworking.com/), a company we are working with. It would be perfect to have smaller groups (ideally 1 person), working against separated databases, but that can be synchronized with the rest of the data when needed. Then each QA merges data from the people he supervises. After it's validated the manager merges the complete dataset, and generates the final product. I don't know if this it's the exact same case, but we are working on it with a similar approach. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
