RE: [OSGeo-Discuss] designing databases, organizing data formats to work with open source and proprietary GIS
Hi there. Im my view, there are 2 levels of integration: with and without ArcSDE. If you have ArcSDE then you should follow Simon’s tips. It’s a tougher challenge to try getting all ArcGIS functionality and at the same time sharing the database with other tiers of software. If you don’t then it’s simpler but it’s very likely that you will have to settle with some limitations in ArcGIS, mainly read-only access to the database, and not having the geodb design’s perks, like subtypes, domains, topology, etc. Since 10 you can already load data from a variety of spatial rdbms including pgsql. But not editing. I’m not sure what will change in 10.1. For the Open Source side of things, going the way of pgsql is all you need! All tribes show their love for pgsql so it’s very powerfull in all aspects, desktop and server, C and Java, etc. Also you will be well served in other arenas besides GIS. Just my 2 cents. Duarte De: karsten vennemann [mailto:kars...@terragis.net] Enviada: quarta-feira, 10 de Agosto de 2011 18:43 Para: discuss@lists.osgeo.org Assunto: [OSGeo-Discuss] designing databases, organizing data formats to work with open source and proprietary GIS Hi all, in the near future I will have the opportunity to help design databases, decide on data formats (files data) for an international organization that wishes to be able to use both proprietary and open source based systems, mostly in web mapping solution but also possibly on the desktop. The task will be to design and organize the data stores in a way that both types of systems - open source (e.g. MapServer, OpenLayers) and proprietary systems (ESRI Arc Server) can use them well, and along the way to try to avoid too much data duplication (having to store data in multiple formats just to make them accessible) . This sounds to like a exiting useful, fun task, but given the limitations of both systems (regarding input data that might not work out of the box- namely file Geodatabases in open source solutions, and PostGIS data in ESRI products) might be not totally trivial ;) I was wondering if anybody has done work on this, has implemented systems facing the same issues or knows of projects or reports that have been dealing with similar issues. Also I anybody has comments about what data storage solution you would recommend and comments about the pro and cons of certain storage designs please send it to the list. Looking forward to hear what other have come up with. Thanks a lot Cheers Karsten Karsten Vennemann Principal Terra GIS LTD USA www.terragis.nethttp://www.terragis.net ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
[OSGeo-Discuss] designing databases, organizing data formats to work with open source and proprietary GIS
Hi all, in the near future I will have the opportunity to help design databases, decide on data formats (files data) for an international organization that wishes to be able to use both proprietary and open source based systems, mostly in web mapping solution but also possibly on the desktop. The task will be to design and organize the data stores in a way that both types of systems - open source (e.g. MapServer, OpenLayers) and proprietary systems (ESRI Arc Server) can use them well, and along the way to try to avoid too much data duplication (having to store data in multiple formats just to make them accessible) . This sounds to like a exiting useful, fun task, but given the limitations of both systems (regarding input data that might not work out of the box- namely file Geodatabases in open source solutions, and PostGIS data in ESRI products) might be not totally trivial ;) I was wondering if anybody has done work on this, has implemented systems facing the same issues or knows of projects or reports that have been dealing with similar issues. Also I anybody has comments about what data storage solution you would recommend and comments about the pro and cons of certain storage designs please send it to the list. Looking forward to hear what other have come up with. Thanks a lot Cheers Karsten Karsten Vennemann Principal Terra GIS LTD USA www.terragis.net ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] designing databases, organizing data formats to work with open source and proprietary GIS
Karsten, On Aug 10, 2011, at 11:12 PM, karsten vennemann wrote: Hi all, in the near future I will have the opportunity to help design databases, decide on data formats (files data) for an international organization that wishes to be able to use both proprietary and open source based systems, mostly in web mapping solution but also possibly on the desktop. The task will be to design and organize the data stores in a way that both types of systems - open source (e.g. MapServer, OpenLayers) and proprietary systems (ESRI Arc Server) can use them well, and along the way to try to avoid too much data duplication (having to store data in multiple formats just to make them accessible) . This sounds to like a exiting useful, fun task, but given the limitations of both systems (regarding input data that might not work out of the box- namely file Geodatabases in open source solutions, and PostGIS data in ESRI products) might be not totally trivial ;) I was wondering if anybody has done work on this, has implemented systems facing the same issues or knows of projects or reports that have been dealing with similar issues. Also I anybody has comments about what data storage solution you would recommend and comments about the pro and cons of certain storage designs please send it to the list. Looking forward to hear what other have come up with. Thanks a lot Perhaps other will respond with something helpful, but the above is way too generic. You might have to narrow down the specific issues in order to get more useful responses. Puneet. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] designing databases, organizing data formats to work with open source and proprietary GIS
All, esp Karsten, I wonder if you've given consideration to the feasibility of a solution based on web services; If feasible, this could bypass the issue you correctly raise re what data storage solution you would recommend , since the function of the service wd be to return data in one of the standard/common formats regardless of the origin. It's a non-trivial effort to design a suitable API, but such an implementation cd provide the data to suitably-connected desktop applications, as well as to conventional browsers, of course AS On Wed, Aug 10, 2011 at 2:19 PM, Mr. Puneet Kishor punk.k...@gmail.comwrote: Karsten, On Aug 10, 2011, at 11:12 PM, karsten vennemann wrote: Hi all, in the near future I will have the opportunity to help design databases, decide on data formats (files data) for an international organization that wishes to be able to use both proprietary and open source based systems, mostly in web mapping solution but also possibly on the desktop. The task will be to design and organize the data stores in a way that both types of systems - open source (e.g. MapServer, OpenLayers) and proprietary systems (ESRI Arc Server) can use them well, and along the way to try to avoid too much data duplication (having to store data in multiple formats just to make them accessible) . This sounds to like a exiting useful, fun task, but given the limitations of both systems (regarding input data that might not work out of the box- namely file Geodatabases in open source solutions, and PostGIS data in ESRI products) might be not totally trivial ;) I was wondering if anybody has done work on this, has implemented systems facing the same issues or knows of projects or reports that have been dealing with similar issues. Also I anybody has comments about what data storage solution you would recommend and comments about the pro and cons of certain storage designs please send it to the list. Looking forward to hear what other have come up with. Thanks a lot ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
[OSGeo-Discuss] designing databases, organizing data formats to work with open source and proprietary GIS
Karsten,Interoperability with ESRI GeoDatabases (SQL, not file) have their difficulties but they aren't insurmountable.To address such interop one must make clear distinctions at differing tiers.The first is that the chosen "storage format" is the "native" one for the chosen database product: That is Oracle's SDO_Geometry, Microsoft's geomery/geography, or PostgreSQL/PostGIS's geometry/geography. Interop with ESRI's ST_Geometry is possible butwidespread support across different proprietary and open source product is problematic.The second is the database data model. One should not use ESRI's GeoDatabase Design Tool as the ONLY tool for defining the data model.Use it, yes, for the ESRI side of the data model/access question, but additionally, one should use a horizontal market data modelling tool(like Enterprise Architect which supports ArcGIS data model xml) to QA the model produced by the ESRI tool, to specify more open datatypes, to add additional documentation (that most ESRI customer shops do in Word documents by hand), and to forward engineer those components of the model that ESRI rather likes to keep to itself and not share with others (eg constraints, lookup tables/coded domains, foreign keys etc).The final aspect is ensuring that ESRI software can do what it believes it needs to be able to do if you are using topological rules,versioning etc.In essence the activities are:1. Define the model so that it can serve an intelligent data to multiple clients independent of vendor. This is best practice data management fully in line with database theory.2. Then "configure" the ESRI interface (treat ArcSDE as a JDBC interface for ESRI clients) so that it can use the model as if it created it itself.I have had success with this in the past. It isn't rocket science though the latter can require a bit of clever thinking.regardsSimon Original Message Subject: [OSGeo-Discuss] designing databases, organizing data formats to work with open source and proprietary GIS Date: Wed, 10 Aug 2011 10:42:55 -0700 From: karsten vennemann kars...@terragis.net Reply-To: OSGeo Discussions discuss@lists.osgeo.org To: discuss@lists.osgeo.org Hi all, in the near future I will have the opportunity to help design databases, decide on data formats (files data) for an international organization that wishes to be able to use both proprietary and open source based systems, mostly in web mapping solution but also possibly on the desktop. The task will be to design and organize the data stores in a way that both types of systems - open source (e.g. MapServer, OpenLayers) and proprietary systems (ESRI Arc Server) can use them well, and along the way to try toavoid too much data duplication (having to store data in multiple formats just to make them accessible) . This sounds to like a exiting useful,fun task, but given the limitations of both systems (regarding input data that might not work out of the box- namely file Geodatabases in open source solutions, and PostGIS data in ESRI products) might be not totally trivial ;) I was wondering if anybody has done work on this, has implemented systems facing the same issues or knows of projects or reports that have been dealing with similar issues. Also I anybody has comments about what data storage solution you would recommend and comments about the pro and cons of certain storage designs please send it to the list. Looking forward to hear what other have come up with. Thanks a lot Cheers Karsten-- Holder of "2011 Oracle Spatial Excellence Award for Education and Research."SpatialDB Advice and Design, Solutions Architecture and Programming,Oracle Database 10g Administrator Certified Associate; Oracle Database 10g SQL Certified ProfessionalOracle Spatial, SQL Server, PostGIS, MySQL, ArcSDE, Manifold GIS, FME, Radius Topology and Studio Specialist.39 Cliff View Drive, Allens Rivulet, 7150, Tasmania, Australia.Website: www.spatialdbadvisor.com Email: si...@spatialdbadvisor.com Voice: +61 362 396397Mobile: +61 418 396391Skype: sggreenerLongitude: 147.20515 (147° 12' 18" E)Latitude: -43.01530 (43° 00' 55" S)GeoHash: r22em9r98wgNAC:W80CK 7SWP3___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] designing databases, organizing data formats to work with open source and proprietary GIS
These 2 softwares will help connect / move your ArcGIS project to FOSS. Connect: ArcGIS to PostGIS PgMap - for Postgresql / PostGIS - Free This is the ArcMap Extension for connectinng to PostGIS (PostgreSQL) spatial database. http://st-links.com/default.aspx Move: ArcGIS to QGIS - (Free ArcGIS extension) One needs to export layers in an opened Arcmap document into a QGIS project file Mxd2Qgs Tool http://geoscripting.blogspot.com/2011/05/exporting-current-arcmap-document.html Noli On 8/11/11, Arnie Shore shor...@gmail.com wrote: All, esp Karsten, I wonder if you've given consideration to the feasibility of a solution based on web services; If feasible, this could bypass the issue you correctly raise re what data storage solution you would recommend , since the function of the service wd be to return data in one of the standard/common formats regardless of the origin. It's a non-trivial effort to design a suitable API, but such an implementation cd provide the data to suitably-connected desktop applications, as well as to conventional browsers, of course AS On Wed, Aug 10, 2011 at 2:19 PM, Mr. Puneet Kishor punk.k...@gmail.comwrote: Karsten, On Aug 10, 2011, at 11:12 PM, karsten vennemann wrote: Hi all, in the near future I will have the opportunity to help design databases, decide on data formats (files data) for an international organization that wishes to be able to use both proprietary and open source based systems, mostly in web mapping solution but also possibly on the desktop. The task will be to design and organize the data stores in a way that both types of systems - open source (e.g. MapServer, OpenLayers) and proprietary systems (ESRI Arc Server) can use them well, and along the way to try to avoid too much data duplication (having to store data in multiple formats just to make them accessible) . This sounds to like a exiting useful, fun task, but given the limitations of both systems (regarding input data that might not work out of the box- namely file Geodatabases in open source solutions, and PostGIS data in ESRI products) might be not totally trivial ;) I was wondering if anybody has done work on this, has implemented systems facing the same issues or knows of projects or reports that have been dealing with similar issues. Also I anybody has comments about what data storage solution you would recommend and comments about the pro and cons of certain storage designs please send it to the list. Looking forward to hear what other have come up with. Thanks a lot ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
[OSGeo-Discuss] designing databases, organizing data formats to work with open source and proprietary GIS
Karsten,Interoperability with ESRI GeoDatabases (SQL, not file) have their difficulties but they aren't insurmountable.To address such interop one must make clear distinctions at differing tiers.The first is that the chosen "storage format" is the "native" one for the chosen database product: That is Oracle's SDO_Geometry, Microsoft's geomery/geography, or PostgreSQL/PostGIS's geometry/geography. Interop with ESRI's ST_Geometry is possible butwidespread support across different proprietary and open source product is problematic.The second is the database data model. One should not use ESRI's GeoDatabase Design Tool as the ONLY tool for defining the data model.Use it, yes, for the ESRI side of the data model/access question, but additionally, one should use a horizontal market data modelling tool(like Enterprise Architect which supports ArcGIS data model xml) to QA the model produced by the ESRI tool, to specify more open datatypes, to add additional documentation (that most ESRI customer shops do in Word documents by hand), and to forward engineer those components of the model that ESRI rather likes to keep to itself and not share with others (eg constraints, lookup tables/coded domains, foreign keys etc).The final aspect is ensuring that ESRI software can do what it believes it needs to be able to do if you are using topological rules,versioning etc.In essence the activities are:1. Define the model so that it can serve an intelligent data to multiple clients independent of vendor. This is best practice data management fully in line with database theory.2. Then "configure" the ESRI interface (treat ArcSDE as a JDBC interface for ESRI clients) so that it can use the model as if it created it itself.I have had success with this in the past. It isn't rocket science though the latter can require a bit of clever thinking.regardsSimon Original Message Subject: [OSGeo-Discuss] designing databases, organizing data formats to work with open source and proprietary GIS Date: Wed, 10 Aug 2011 10:42:55 -0700 From: karsten vennemann kars...@terragis.net Reply-To: OSGeo Discussions discuss@lists.osgeo.org To: discuss@lists.osgeo.org Hi all, in the near future I will have the opportunity to help design databases, decide on data formats (files data) for an international organization that wishes to be able to use both proprietary and open source based systems, mostly in web mapping solution but also possibly on the desktop. The task will be to design and organize the data stores in a way that both types of systems - open source (e.g. MapServer, OpenLayers) and proprietary systems (ESRI Arc Server) can use them well, and along the way to try toavoid too much data duplication (having to store data in multiple formats just to make them accessible) . This sounds to like a exiting useful,fun task, but given the limitations of both systems (regarding input data that might not work out of the box- namely file Geodatabases in open source solutions, and PostGIS data in ESRI products) might be not totally trivial ;) I was wondering if anybody has done work on this, has implemented systems facing the same issues or knows of projects or reports that have been dealing with similar issues. Also I anybody has comments about what data storage solution you would recommend and comments about the pro and cons of certain storage designs please send it to the list. Looking forward to hear what other have come up with. Thanks a lot Cheers Karsten-- Holder of "2011 Oracle Spatial Excellence Award for Education and Research."SpatialDB Advice and Design, Solutions Architecture and Programming,Oracle Database 10g Administrator Certified Associate; Oracle Database 10g SQL Certified ProfessionalOracle Spatial, SQL Server, PostGIS, MySQL, ArcSDE, Manifold GIS, FME, Radius Topology and Studio Specialist.39 Cliff View Drive, Allens Rivulet, 7150, Tasmania, Australia.Website: www.spatialdbadvisor.com Email: si...@spatialdbadvisor.com Voice: +61 362 396397Mobile: +61 418 396391Skype: sggreenerLongitude: 147.20515 (147° 12' 18" E)Latitude: -43.01530 (43° 00' 55" S)GeoHash: r22em9r98wgNAC:W80CK 7SWP3___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
