Re: [OSGeo-Discuss] idea for an OSGeo project -- a new, open data format
Puneet: I believe you've just described the SDF format (open, based on SQL lite) that is currently in use by FDO / MapGuide. Some info from a post from Jason Birch a while back: To bring us back to the start of this post, the one item on this list that I think may get overlooked is support for the Spatial Data File (SDF) format. SDF is a single-user database format, similar to ESRI’s personal geodatabase. It is built on top of SQLite, is fully open, and is already supported by MapGuide Open Source and (yay!) Safe Software’s current FME betas. http://www.jasonbirch.com/nodes/2006/03/04/8/closed-and-open-and- better-oh-my/ Dave On 13-Nov-07, at 9:52 AM, P Kishor wrote: So, I am thinking, Shapefile is the de facto data standard for GIS data. That it is open (albeit not Free) along with the deep and wide presence of ESRI's products from the beginning of the epoch, it has been widely adopted. Existence of shapelib, various language bindings, and ready use by products such as MapServer has continued to cement Shapefile as the format to use. All this is in spite of Shapefile's inherent drawbacks, particularly in the area of attribute data management. What if we came up with a new and improved data format -- call it Open Shapefile (extension .osh) -- that would be completely Free, single-file based (instead of the multiple .shp, .dbf, .shx, etc.), and based on SQLite, giving the .osh format complete relational data handling capabilities. We would require a new version of Shapelib, improved language bindings, make it the default and preferred format for MapServer, and provide seamless and painless import of regular .shp data into .osh for native rendering. Its adoption would be quick in the open source community. The non-opensource community would either not give a rat's behind for it, but it wouldn't affect them... they would still work with their preferred .shp until they learned better. By having a completely open and Free single-file based, built on SQLite, fully relational dbms capable spatial data format, it would be positioned for continued improvement and development. Is this too crazy? -- Puneet Kishor ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] idea for an OSGeo project -- a new, open data format
Puneet, You wrote: Is this too crazy? I don't think this idea is crazy at all. In fact, I think it is a very good idea. I do have a couple of comments, which you can read below: You wrote: What if we came up with a new and improved data format -- call it Open Shapefile (extension .osh)... I think we would have to completely avoid the term Shapefile as it is probably an ESRI trademark. You wrote: that would be completely Free, single-file based (instead of the multiple .shp, .dbf, .shx, etc... Is there a problem with a multiple file format? I have tinkered with some different binary file formats, and it seems that separating some information in a spatial data set (like indexes, for example) makes it easier to create programs that parse and write the format. You wrote: ...and based on SQLite, giving the .osh format complete relational data handling capabilities. I would prefer tightly integrating any software package, even if it was FOSS, into a new data format. SQLite is written in the C programming language, as an example, and that doesn't do me a lot of good as a Java programmer. Tightly integrating a Java library or program wouldn't do much good for a programmer using C. That is the real beauty of an open file format: If it is designed properly your programming language doesn't matter! I did a little brainstorming for a binary file format that could replace Shapefiles. Its called BOFF, or Binary Open Feature Format. I talked over small bits of the design for BOFF with Frank Wammerdam, who expressed some small interest in it at the time. Perhaps it would offer some ideas? I'd be very interesting in offering Java support for a shapefile replacement endorsed by the OSGeo. The Sunburned Surveyor -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of P Kishor Sent: Tuesday, November 13, 2007 6:53 AM To: OSGeo Discussions Subject: [OSGeo-Discuss] idea for an OSGeo project -- a new, open data format So, I am thinking, Shapefile is the de facto data standard for GIS data. That it is open (albeit not Free) along with the deep and wide presence of ESRI's products from the beginning of the epoch, it has been widely adopted. Existence of shapelib, various language bindings, and ready use by products such as MapServer has continued to cement Shapefile as the format to use. All this is in spite of Shapefile's inherent drawbacks, particularly in the area of attribute data management. What if we came up with a new and improved data format -- call it Open Shapefile (extension .osh) -- that would be completely Free, single-file based (instead of the multiple .shp, .dbf, .shx, etc.), and based on SQLite, giving the .osh format complete relational data handling capabilities. We would require a new version of Shapelib, improved language bindings, make it the default and preferred format for MapServer, and provide seamless and painless import of regular shp data into .osh for native rendering. Its adoption would be quick in the open source community. The non-opensource community would either not give a rat's behind for it, but it wouldn't affect them... they would still work with their preferred .shp until they learned better. By having a completely open and Free single-file based, built on SQLite, fully relational dbms capable spatial data format, it would be positioned for continued improvement and development. Is this too crazy? -- Puneet Kishor ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss Warning: Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] idea for an OSGeo project -- a new, open data format
this sounds like the SDF format, although I am not sure that SDF is open source - there is an open source implementation as part of FDO (http://fdo.osgeo.org/fdosdf/index.html ) Paul On 13-Nov-07, at 9:52 AM, P Kishor wrote: So, I am thinking, Shapefile is the de facto data standard for GIS data. That it is open (albeit not Free) along with the deep and wide presence of ESRI's products from the beginning of the epoch, it has been widely adopted. Existence of shapelib, various language bindings, and ready use by products such as MapServer has continued to cement Shapefile as the format to use. All this is in spite of Shapefile's inherent drawbacks, particularly in the area of attribute data management. What if we came up with a new and improved data format -- call it Open Shapefile (extension .osh) -- that would be completely Free, single-file based (instead of the multiple .shp, .dbf, .shx, etc.), and based on SQLite, giving the .osh format complete relational data handling capabilities. We would require a new version of Shapelib, improved language bindings, make it the default and preferred format for MapServer, and provide seamless and painless import of regular .shp data into .osh for native rendering. Its adoption would be quick in the open source community. The non-opensource community would either not give a rat's behind for it, but it wouldn't affect them... they would still work with their preferred .shp until they learned better. By having a completely open and Free single-file based, built on SQLite, fully relational dbms capable spatial data format, it would be positioned for continued improvement and development. Is this too crazy? -- Puneet Kishor ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss +-+ |Paul Spencer [EMAIL PROTECTED]| +-+ |Chief Technology Officer | |DM Solutions Group Inchttp://www.dmsolutions.ca/ | +-+ ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] idea for an OSGeo project -- a new, open data format
On Tue, Nov 13, 2007 at 07:42:12AM -0800, Landon Blake wrote: You wrote: that would be completely Free, single-file based (instead of the multiple .shp, .dbf, .shx, etc... Is there a problem with a multiple file format? I have tinkered with some different binary file formats, and it seems that separating some information in a spatial data set (like indexes, for example) makes it easier to create programs that parse and write the format. Although I can see both sides of an argument here, for working over the web, multiple files are *much* more difficult, for the simple reason that you can't have someone download 'a file' and use it in their tool. This doesn't mean that internally, the representation can't be stored in multiple logical groups -- KML, for example, accomplishes multiple file storage via zipping up a directory of files, and some ponderance on .shz (zipped shapefiles, with all three important bits included) has been bounced around in my presence before. I can't speak to implementation details, but for 'moving data around', a single file format is much preferred over the web. Regards, -- Christopher Schmidt Web Developer ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
[OSGeo-Discuss] Re: idea for an OSGeo project -- a new, open data format
Thanks everyone, for responding. Here is my groundwork. The new format -- - Should be fast. SQLite is plenty fast, and anything that simply extends the Shapefile format to inject relational capabilities should be pretty fast. It should definitely be faster than a geodatabase format (such as PostGIS/ArcSDE) and perhaps even faster than Shapefiles especially while accessing attribute data. DBF is sequential, and searching for textual information is particularly expensive. SQLite has been tuned to excellence. I have been working with it for a few years now, and it really is an amazing product, development community, support, and capabilities. That it is in public domain makes for a transfat-free icing on the cake. - Should be unencumbered by licenses and copyrights. Ideally, the new format could also be put back into public domain. We want to remove all encumbrances to encourage rapid and wide adoption. - Should be a single file. Well, some like multiple files and some like single files. We can achieve both objectives by using a tar-gzipped packaging such as Apple tends to use for much of its stuff (for example, its Pages wordprocessor uses a tgzipped xml file along with other resources for icons and pictures and stuff). Or, if speed is going to be affected because of gzipping and gunzipping, just a package format (I have no idea if this is a Unix thing or a Mac OS thing -- we, in the Mac world, call them packages... they appear like files in the Finder, and like directories in the shell). - Should be easy to transition to. By building the new format on the structure of the Shapefile format, and *in fact*, calling it open shapefiles or some such thing, we indicate from its name that the transition is not that revolutionary but is evolutionary. This, hopefully, will bring some name-familiarity, and make the transition less scary. - Frank mentions SQLite's lack of datatypes as an issue -- I guess that is a matter of preference. I personally quite like that freedom as it gives me, the application developer, complete control over what goes where. SQLite actually does have now a few datatypes that it respects, but doesn't complain about. Since all users will be accessing the data via an application, as long as the application is well defined, it should be fine. - SQLite excels at one thing that it has been entrusted to do -- retrieve data that it has been entrusted with at extremely fast speeds, and maintain ACID data integrity in case of a programmatic catastrophe. The transactions themselves are worth their price of admission, which, happily, happens to be zero. - Langdon mentions Java support -- well, yes, use/work on SQLite JDBC. I have been using it for a few days now and find it to be a pretty competent conduit. Extend it, spatialize it. ANSI standard C is still that magic common denominator that compiles and works predictably on most number of systems. I have a lot against Java, but those who love Java should definitely work on tools for accessing and working with this new format as it would only make the format more widely used and adopted. Ok, enough for now. On Nov 13, 2007 8:52 AM, P Kishor [EMAIL PROTECTED] wrote: So, I am thinking, Shapefile is the de facto data standard for GIS data. That it is open (albeit not Free) along with the deep and wide presence of ESRI's products from the beginning of the epoch, it has been widely adopted. Existence of shapelib, various language bindings, and ready use by products such as MapServer has continued to cement Shapefile as the format to use. All this is in spite of Shapefile's inherent drawbacks, particularly in the area of attribute data management. What if we came up with a new and improved data format -- call it Open Shapefile (extension .osh) -- that would be completely Free, single-file based (instead of the multiple .shp, .dbf, .shx, etc.), and based on SQLite, giving the .osh format complete relational data handling capabilities. We would require a new version of Shapelib, improved language bindings, make it the default and preferred format for MapServer, and provide seamless and painless import of regular .shp data into .osh for native rendering. Its adoption would be quick in the open source community. The non-opensource community would either not give a rat's behind for it, but it wouldn't affect them... they would still work with their preferred .shp until they learned better. By having a completely open and Free single-file based, built on SQLite, fully relational dbms capable spatial data format, it would be positioned for continued improvement and development. Is this too crazy? -- Puneet Kishor ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Re: idea for an OSGeo project -- a new, open data format
Puneet, You wrote: Should be easy to transition to. By building the new format on the structure of the Shapefile format, and *in fact*, calling it open shapefiles or some such thing, we indicate from its name that the transition is not that revolutionary but is evolutionary. This, hopefully, will bring some name-familiarity, and make the transition less scary. I really think you are going to run into problems using the Shapefile as part of the trademark or name for any product not sold by ESRI. I strongly recommend against this move. Let people adopt the implementation of your idea for its merits, not for name recognition that comes from another product line. You wrote: ANSI standard C is still that magic common denominator that compiles and works predictably on most number of systems. I have a lot against Java, but those who love Java should definitely work on tools for accessing and working with this new format as it would only make the format more widely used and adopted. It sounds to me like you are really describing a tool. File formats are written in a binary encoding or text, not in a programming language. If you are designing a tool you can choose the programming language of your choice, but be aware that this will limit the developers that adopt the tool. This will be the case no matter what language you choose to use, whether it is C, Java, or something else. If, in contrast, you are creating a file format, then programming languages shouldn't really matter. Binary and text data can be accessed by almost all programming languages. I think you need to decide if you want a tool or a data format. It sounds like you are shooting more for a spatial database written in the C programming language that uses some form of the ESRI Shapefile as its underlying data storage mechanism. To me that is a tool or piece of software, not a format. But maybe I don't completely understand your goal. Landon -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of P Kishor Sent: Tuesday, November 13, 2007 8:35 AM To: OSGeo Discussions Subject: [OSGeo-Discuss] Re: idea for an OSGeo project -- a new,open data format Thanks everyone, for responding. Here is my groundwork. The new format -- - Should be fast. SQLite is plenty fast, and anything that simply extends the Shapefile format to inject relational capabilities should be pretty fast. It should definitely be faster than a geodatabase format (such as PostGIS/ArcSDE) and perhaps even faster than Shapefiles especially while accessing attribute data. DBF is sequential, and searching for textual information is particularly expensive. SQLite has been tuned to excellence. I have been working with it for a few years now, and it really is an amazing product, development community, support, and capabilities. That it is in public domain makes for a transfat-free icing on the cake. - Should be unencumbered by licenses and copyrights. Ideally, the new format could also be put back into public domain. We want to remove all encumbrances to encourage rapid and wide adoption. - Should be a single file. Well, some like multiple files and some like single files. We can achieve both objectives by using a tar-gzipped packaging such as Apple tends to use for much of its stuff (for example, its Pages wordprocessor uses a tgzipped xml file along with other resources for icons and pictures and stuff). Or, if speed is going to be affected because of gzipping and gunzipping, just a package format (I have no idea if this is a Unix thing or a Mac OS thing -- we, in the Mac world, call them packages... they appear like files in the Finder, and like directories in the shell). - Should be easy to transition to. By building the new format on the structure of the Shapefile format, and *in fact*, calling it open shapefiles or some such thing, we indicate from its name that the transition is not that revolutionary but is evolutionary. This, hopefully, will bring some name-familiarity, and make the transition less scary. - Frank mentions SQLite's lack of datatypes as an issue -- I guess that is a matter of preference. I personally quite like that freedom as it gives me, the application developer, complete control over what goes where. SQLite actually does have now a few datatypes that it respects, but doesn't complain about. Since all users will be accessing the data via an application, as long as the application is well defined, it should be fine. - SQLite excels at one thing that it has been entrusted to do -- retrieve data that it has been entrusted with at extremely fast speeds, and maintain ACID data integrity in case of a programmatic catastrophe. The transactions themselves are worth their price of admission, which, happily, happens to be zero. - Langdon mentions Java support -- well, yes, use/work on SQLite JDBC. I have been using it for a few days now and find it to be a pretty competent conduit. Extend it, spatialize it. ANSI standard C is
Re: [OSGeo-Discuss] idea for an OSGeo project -- a new, open data format
On Nov 13, 2007, at 6:42 AM, Frank Warmerdam wrote: o RDBMS style operations like SQL filtering, joins, etc. o Get past all the shapefile limitations related to the .dbf format (very restricted data types, short attribute names, lots of other limits) o Allow storing many layers in one file. o Built in spatial indexing and attribute indexing. o OGC style coordinate system and geometry support. This idea has been kicked around for a long time, in fact I did some preliminary work on it a couple of years ago to help identify the issues/challenges in such an implementation. I'm pretty much aligned with Frank's goals (quoted above). I see it as more than a shapefile replacement. It can be a container for multiple layers with SQL functionality. This allows you to not only package up everything for a project (web or desktop) but also have the power and flexibility of an RDBMS. -gary -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_- Gary Sherman Chair, QGIS Project Steering Committee -Micro Resources: http://mrcc.com *Geospatial Hosting *Web Site Hosting -Desktop GIS Book: *http://desktopgisbook.com We work virtually everywhere -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_- ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
RE: [OSGeo-Discuss] Re: idea for an OSGeo project -- a new, open data format
Landon wrote: --- I really think you are going to run into problems using the Shapefile as part of the trademark or name for any product not sold by ESRI. I strongly recommend against this move. --- I'm not a lawyer, but I really doubt that shapefile is unique enough to be subject to trademark. Initial searches of TESS don't show up anything either. That said, I don't think that shapefile should be used anywhere in an open format name _because_ it's too generic and will cause confusion. As my initial blog entry mentioned, I had (have?) pretty high hopes for SDF. It's extremely fast (native rtree), contains multiple tables in a single file, supports complex geometries (arcs, aggregates, etc), includes native schema definition, etc... However... while some initial marketing materials marked SDF as an open format, I have been unable to obtain any clarification from Autodesk on the file specification or openness of SDF. As well, there are some limitations; in the quest for speed, SDF is only using the SQLite table access mechanisms, not the entire relational layer (or something like that). For reference, most of the interesting parts of the SDF Provider, including the Rtree implementation, are available here: http://tinyurl.com/2b5en2 Jason ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] idea for an OSGeo project -- a new, open data format
On 11/13/07, Frank Warmerdam [EMAIL PROTECTED] wrote: P Kishor wrote: So, I am thinking, Shapefile is the de facto data standard for GIS data. That it is open (albeit not Free) along with the deep and wide presence of ESRI's products from the beginning of the epoch, it has been widely adopted. Existence of shapelib, various language bindings, and ready use by products such as MapServer has continued to cement Shapefile as the format to use. All this is in spite of Shapefile's inherent drawbacks, particularly in the area of attribute data management. What if we came up with a new and improved data format -- call it Open Shapefile (extension .osh) -- that would be completely Free, single-file based (instead of the multiple .shp, .dbf, .shx, etc.), and based on SQLite, giving the .osh format complete relational data handling capabilities. We would require a new version of Shapelib, improved language bindings, make it the default and preferred format for MapServer, and provide seamless and painless import of regular .shp data into .osh for native rendering. Its adoption would be quick in the open source community. The non-opensource community would either not give a rat's behind for it, but it wouldn't affect them... they would still work with their preferred .shp until they learned better. By having a completely open and Free single-file based, built on SQLite, fully relational dbms capable spatial data format, it would be positioned for continued improvement and development. Puneet, I've had a similar idea kicking around in my head for a while, but I think of it as open geodatabase. I see the goals as providing a similar role to the personal geodatabase, including: I should mention that over on the SQLite list every once a eon someone asks the question about spatlializing SQLite a la PostGIS. That definitely could be one way to go as I have a little experience in this. Last year I had the opportunity to tackle a point-in-polygon overlay problem that was seeming to be intractable for ArcGIS. So, I dumped the data into Shapefile, unpacked the coordinates from the Shapefile and stuffed them into SQLite, then used the bounding box method to narrow my searches. With the clever use of indexes, and a bunch of optimizations, I basically wrote a fast and functional overlay program with Perl/DBI. The overlay task now takes 2 days, which is a huge improvement from the earlier 7-8 days that it used to be. One thing this new geo database should not be is that like SQLite it should not require a server. As wonderful as PostGIS is, installing and managing PostGres is a major obstacle to its use. I use SQLite for pretty much everything because of the ease with which I can get started with it. It takes me longer to download it than to start working with it on a new machine. As Larry Wall likes to say, you can write faster programs in C, but you can program faster in Perl. It is kinda like that... on the one hand, highlighting the database-ness of the new format would be a good and powerful thing, but on the other hand, it might lead folks to think that a db server is required. Having a server-less, self-contained, rdbms-capable format would be the key. o RDBMS style operations like SQL filtering, joins, etc. o Get past all the shapefile limitations related to the .dbf format (very restricted data types, short attribute names, lots of other limits) o Allow storing many layers in one file. o Built in spatial indexing and attribute indexing. o OGC style coordinate system and geometry support. I have had some hope that the existing SDF format supported by FDO would be this new format; however, SDF is quite a complicated format, and the only available open source implementation is quite heavily tied to FDO. Once you carry along FDO the whole thing becomes fairly heavy in terms of the amount of code required, and the interface complexity. But (I think) it satisfies most of my goals and already exists. I do feel that we need to be cautious before launching yet another format. I'm also a bit dubious about some aspects of sqlite as a native data store. In particular, it's typeless everything is a string approach strikes me as potentially being a problem. It also remains to be seen whether we could build fast spatial indexing directly in, though I suppose with a fat enough middleware layer it could be done. PS. I'm still doubtful it would be faster than shapefiles+qix for most web mapping needs. We will never know until it is done. And, isn't premature optimization the root of all evil? Best regards, -- ---+-- I set the clouds in motion - turn up | Frank Warmerdam, [EMAIL PROTECTED] light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| President OSGeo, http://osgeo.org
Re: [OSGeo-Discuss] Re: idea for an OSGeo project -- a new, open data format
David, On 11/13/07, David William Bitner [EMAIL PROTECTED] wrote: Part of the mission of the OSGeo Geodata committee (http://www.osgeo.org/geodata) is to promote the use of open geospatial formats. If there is a group that wants to continue pursuing the creation of a new open geodata format, I would like to encourage the use of the geodata mailing list. That being said, I think part of the discussion that needs to be had is whether or not OSGeo should be creating standards in the first place. A couple comments that I have on some of the discussion that has taken place in this thread: Regarding the suggestion that MapServer takes on this new format as the primary format: I think this is way beyond the scope of what OSGeo should be doing. Even if we spec a new standard, we (OSGeo) have no teeth to be able to make any of our projects do any kind of implementation of that standard. The choice of formats that are used by any of our projects is driven by the needs of the users and developers and the resources (time, money) that have been dedicated towards implementing them. If someone takes OpenShape or whatever and decides they have a business need that they can spend the time or money to get it implemented then it will be implemented. Shapefile has and will continue to be an important format for many projects as it is one of, if not the most distributed formats in the GIS world. I respectfully disagree. I think OSGeo has plenty teeth for those who want to believe in it. In the end, yes, just like any real project, it needs a core of committed developer and plenty of time (or money -- usually they are synonymous). This is not something that can happen overnight, but if good, it deserves a start and support. That the long, long-term effects of a solid, relational, transactional, geodata format would be very good is a reasonable assumption for me. Regarding the comments on standards wanking: Standards can get in the way of progress along a straight line, but they can also encourage interoperability that can create better progress for everyone. To get a singular task done, standards often can slow things down, but there *are* gains to be had from playing well with everyone else. Here I totally agree. I am not sure how to interpret the standards wanking statement. On the one hand it is a reasonably accurate assessment of a lot of public hand-wringing and open alliances (for a really funny take on this, read Fake Steve's tirade on the open handset alliance at http://fakesteve.blogspot.com/2007/11/its-not-phone-its-alliance.html). But, on the other hand, it is a pretty damning judgment on any attempt to do things via collaboration, and thus, on OSGeo and such efforts itself. My take is that if I can't do it alone, I will lay it out in the open hoping someone better than me will work on it as well. If I can do it alone, I will do it until I think it is ready to benefit from extra eyeballs. Sometimes getting started is the biggest hurdle. David Bitner OSGeo, Public Geospatial Data Project Chair On Nov 13, 2007 11:40 AM, Allan Doyle [EMAIL PROTECTED] wrote: On Nov 13, 2007, at 12:24 , Steve Coast wrote: OSM: $0 CCBYSA: $0 Donation of entire Netherlands: Priceless Real artists ship. For everyone else there's standards wanking. Perhaps there's an art to wanking standards as well. Seriously though, this is so kafka-esque. When OSM started it was like this: We should have got a committee to design a standard, then we could think about a committee to design an ontology... and choose a name... and on some sunny distant day make a map. On 13 Nov 2007, at 17:09, P Kishor wrote: On 11/13/07, Landon Blake [EMAIL PROTECTED] wrote: Puneet, You wrote: Should be easy to transition to. By building the new format on the structure of the Shapefile format, and *in fact*, calling it open shapefiles or some such thing, we indicate from its name that the transition is not that revolutionary but is evolutionary. This, hopefully, will bring some name-familiarity, and make the transition less scary. I really think you are going to run into problems using the Shapefile as part of the trademark or name for any product not sold by ESRI. I strongly recommend against this move. Let people adopt the implementation of your idea for its merits, not for name recognition that comes from another product line. Good enough point to keep in mind, but not to get hung up over enough to entangle us. Suggestions for names of the data format can be a project in itself. open spatial data format or its variations could be chosen. Still, point taken. You wrote: ANSI standard C is still that magic common denominator that compiles and works predictably on most number of systems. I have a lot against Java, but those who love Java should definitely work on
Re: [OSGeo-Discuss] Re: idea for an OSGeo project -- a new, open data format
I have created a (now empty) space on the OSGeo wiki to start to fill in concrete details that come out of this discussion at http://wiki.osgeo.org/index.php/Geodata_formats. Please use the wiki to put your wishlists for a new open data format, lists of existing data formats with links to their specifications etc in the wiki. Please join the Geodata Mailing list (http://www.osgeo.org/geodata) and continue this thread with debate and discussion relating to a new format on that list as I believe it is a more appropriate venue. David On Nov 13, 2007 12:55 PM, P Kishor [EMAIL PROTECTED] wrote: David, On 11/13/07, David William Bitner [EMAIL PROTECTED] wrote: Part of the mission of the OSGeo Geodata committee (http://www.osgeo.org/geodata) is to promote the use of open geospatial formats. If there is a group that wants to continue pursuing the creation of a new open geodata format, I would like to encourage the use of the geodata mailing list. That being said, I think part of the discussion that needs to be had is whether or not OSGeo should be creating standards in the first place. A couple comments that I have on some of the discussion that has taken place in this thread: Regarding the suggestion that MapServer takes on this new format as the primary format: I think this is way beyond the scope of what OSGeo should be doing. Even if we spec a new standard, we (OSGeo) have no teeth to be able to make any of our projects do any kind of implementation of that standard. The choice of formats that are used by any of our projects is driven by the needs of the users and developers and the resources (time, money) that have been dedicated towards implementing them. If someone takes OpenShape or whatever and decides they have a business need that they can spend the time or money to get it implemented then it will be implemented. Shapefile has and will continue to be an important format for many projects as it is one of, if not the most distributed formats in the GIS world. I respectfully disagree. I think OSGeo has plenty teeth for those who want to believe in it. In the end, yes, just like any real project, it needs a core of committed developer and plenty of time (or money -- usually they are synonymous). This is not something that can happen overnight, but if good, it deserves a start and support. That the long, long-term effects of a solid, relational, transactional, geodata format would be very good is a reasonable assumption for me. Regarding the comments on standards wanking: Standards can get in the way of progress along a straight line, but they can also encourage interoperability that can create better progress for everyone. To get a singular task done, standards often can slow things down, but there *are* gains to be had from playing well with everyone else. Here I totally agree. I am not sure how to interpret the standards wanking statement. On the one hand it is a reasonably accurate assessment of a lot of public hand-wringing and open alliances (for a really funny take on this, read Fake Steve's tirade on the open handset alliance at http://fakesteve.blogspot.com/2007/11/its-not-phone-its-alliance.html). But, on the other hand, it is a pretty damning judgment on any attempt to do things via collaboration, and thus, on OSGeo and such efforts itself. My take is that if I can't do it alone, I will lay it out in the open hoping someone better than me will work on it as well. If I can do it alone, I will do it until I think it is ready to benefit from extra eyeballs. Sometimes getting started is the biggest hurdle. David Bitner OSGeo, Public Geospatial Data Project Chair On Nov 13, 2007 11:40 AM, Allan Doyle [EMAIL PROTECTED] wrote: On Nov 13, 2007, at 12:24 , Steve Coast wrote: OSM: $0 CCBYSA: $0 Donation of entire Netherlands: Priceless Real artists ship. For everyone else there's standards wanking. Perhaps there's an art to wanking standards as well. Seriously though, this is so kafka-esque. When OSM started it was like this: We should have got a committee to design a standard, then we could think about a committee to design an ontology... and choose a name... and on some sunny distant day make a map. On 13 Nov 2007, at 17:09, P Kishor wrote: On 11/13/07, Landon Blake [EMAIL PROTECTED] wrote: Puneet, You wrote: Should be easy to transition to. By building the new format on the structure of the Shapefile format, and *in fact*, calling it open shapefiles or some such thing, we indicate from its name that the transition is not that revolutionary but is evolutionary. This, hopefully, will bring some name-familiarity, and make the transition less scary. I really think you are going to run into problems using the Shapefile as part of
[OSGeo-Discuss] idea for an OSGeo project -- a new, open, data format
From: P Kishor [EMAIL PROTECTED] Subject: Re: [OSGeo-Discuss] idea for an OSGeo project -- a new, open data format To: OSGeo Discussions discuss@lists.osgeo.org Message-ID: [EMAIL PROTECTED] Content-Type: text/plain; charset=ISO-8859-1 On 11/13/07, Frank Warmerdam [EMAIL PROTECTED] wrote: P Kishor wrote: So, I am thinking, Shapefile is the de facto data standard for GIS data. BIG SNIP Having a server-less, self-contained, rdbms-capable format would be the key. No argument per say. However, it's my -very much less than- 2 cents worth, that while this is very useful, I wouldn't do it to the exclusion of server based, distributed database. Meaning, that's nice, but if you can't easily transition the data munging back into a larger shared dataset, then the utility is somewhat less than optimal. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
[OSGeo-Discuss] OSGeo4W - Binary Windows Installer - in development
This is to announce that work has begun on what is being named the OSGeo4W installer. To keep this announcement email short, please see its Wiki page for more installer details (http://wiki.osgeo.org/index.php/OSGeo_Win32_Installer). Also, a new mailing list has been created, osgeo4w-dev (join at http://lists.osgeo.org/mailman/listinfo/osgeo4w-dev), so let's use that list for further discussions please (no need for a long thread on the discuss list). Consider this an invitation to all foss4g software maintainers to jump on board and assist in the creation of this much needed windows installer (currently windows users need to know where to get FWTools, MS4W, QuantumGIS, PostGIS, winGRASS, GeoServerand the list goes on...*and* each contains their own build of shared libraries like GDAL). If you are interested as a foss4g maintainer, to start please login to that wiki page and add your name to the 'Participants' section. One thing is for sure, this is a huge task so it is clearly going to require involvement from lots of communities. So please jump on board! See ya on the osgeo4w-dev list!! -jeff ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] OSGeo4W - Binary Windows Installer - in development
Jeff McKenna wrote: This is to announce that work has begun on what is being named the OSGeo4W installer. To keep this announcement email short, please see its Wiki page for more installer details (http://wiki.osgeo.org/index.php/OSGeo_Win32_Installer). Also, a new mailing list has been created, osgeo4w-dev (join at http://lists.osgeo.org/mailman/listinfo/osgeo4w-dev), so let's use that list for further discussions please (no need for a long thread on the discuss list). Consider this an invitation to all foss4g software maintainers to jump on board and assist in the creation of this much needed windows installer (currently windows users need to know where to get FWTools, MS4W, QuantumGIS, PostGIS, winGRASS, GeoServerand the list goes on...*and* each contains their own build of shared libraries like GDAL). If you are interested as a foss4g maintainer, to start please login to that wiki page and add your name to the 'Participants' section. One thing is for sure, this is a huge task so it is clearly going to require involvement from lots of communities. So please jump on board! Folks, To add a bit of context, OSGeo4W is essentially an effort to broaden the existing MS4W efforts to many OSGeo projects, and to provide a slicker package oriented GUI. It is my hope that this will become the official OSGeo software packaging for windows, eventually becoming a full fledged OSGeo project. Jeff has some funding to work on this - hopefully sufficient to get some of the core aspects working smoothly, but what packages actually make it into the distribution (beyond his requirements from the MS4W point of view) will depend on who else gets involved. Best regards, -- ---+-- I set the clouds in motion - turn up | Frank Warmerdam, [EMAIL PROTECTED] light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| President OSGeo, http://osgeo.org ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
