Hi Paolo, >Hi Michaël, >I'm sorry but I don't have any experience with HSQL, H2, or Derby, >so I can't tell how they would perform in JUMP, both from a speed >and a memory-use point of view. > > At least, your database experience is welcome ;-)
>This aside, I found that having a "real", albeit embedded, database >would be useful for OJ. Shape files are good to exchange data, but they >suffer from the limitations of the DBF format (field name length, data types, >etc.). >Also I consider Shape files only as a _transport_ format, not as something >one should actually _work_ upon. > > Agree with you for the shp limitation remark. More moderate about what is a format like shapefile for : data access / data exchange ? Many softs have a proprietary binary format for efficient data access and use more open text-based formats for exchange (I think that's what is gml good for). Maybe shapefile is in between ; widely used for data exchange, and quite efficient for data access with its indexed shape-files. I'd like to know how an embeded database can compare to a shapefile from a performance point of view. For scalability, interoperability etc., I entirely agree with you. >Instead engines like HSQL and others are able to manage any data type >and can create files that can be exchanged without any loss of information. > >So a PostgreSQL user could save a few layers to the embedded DB, >send the relevant files to an Oracle user and then he can >load those layers in JUMP and save them as native Oracle tables. >You can do the same with Shape files, but you risk to loose data >and/or data structure. > > Agree with you >But what Landon (A.K.A. - The Sunburned Surveyor) is trying >to achieve is probably a different thing. >For sure OJ should be able to, at least, _display_ large Shape files >without consuming too much memory, and this have to be addressed. > >Since those embedded DB engines already have capacities to balance memory use, >they _may_ be a solution, but the requirement of not been dependant >from another piece of software is a good one, and it may justify the burden >of writing such a functionality from scratch. > > >Bye >Paolo Rizzi > > Bye > > > >>Michaël >> >>P.Rizzi Ag.Mobilità Ambiente a écrit : >> >> >> >>>What about using a Java embedded database like HSQL or Derby??? >>>They already can manage in-memory / persistence of any kind of data >>>and they do it very well I believe... >>>You should only keep each Feature's primary keys and, >>> >>> >>probably, it's >> >> >>>bounds. >>>The db engine will take care of everything else. >>>Also this kind of engine usually create simple files that can be >>>copied around >>>so they effectively also are a binary format. >>>Also, beeing used in many places, there're lots of >>> >>> >>utilities and software >> >> >>>able to use them. The Derby engine has been recently integrated into >>>Java 6 (aka Mustang). >>>But maybe there're other aspects that I can't see... >>>Bye >>>Paolo Rizzi >>> >>> >>> >>> >>-------------------------------------------------------------- >>---------- >> >> >>>*Da:* [EMAIL PROTECTED] per conto di >>>Sunburned Surveyor >>>*Inviato:* ven 30/03/2007 19.18 >>>*A:* List for discussion of JPP development and use. >>>*Oggetto:* Re: [JPP-Devel] Fwd: [Freegis-list] Binary >>> >>> >>Format For Features >> >> >>>David, >>>There are some reasons why I would like to stay away from a system >>>based on an existing file format like ESRI's Shapefile, or >>> >>> >>on a system >> >> >>>that requires an external database. I'm looking for a >>> >>> >>simpler solution >> >> >>>that can be used to suck in and store Features from numerous file >>>formats. In essence the binary format for Features will be >>> >>> >>similar to >> >> >>>what is done currently in ESRI's shapefile format, but hopefully it >>>will be a little more elegant. (For example, doesn't a >>> >>> >>Shapefile use >> >> >>>little-endian and big-endian format, while Java natively >>> >>> >>supports only >> >> >>>big-endian binary data?) >>>I'm curious why the OGC didn't define a binary format for features >>>when they were coming up with WKB. (Unless they did, and I >>> >>> >>just missed >> >> >>>it.) >>>The Sunburned Surveyor >>> >>>On 3/30/07, *David Zwiers* <[EMAIL PROTECTED] >>><mailto:[EMAIL PROTECTED]>> wrote: >>> >>> Sounds a lot like some similar problems that have been >>> >>> >>addressed, >> >> >>> and for the most part solved. For most cases you could probably >>> use an indexed shapefile reader (not in OJ) or leverage a >>> container such as Pico or Hibernate (think Geotools has >>> >>> >>some code, >> >> >>> or atleast a design in this space – talk to Jody Garnett). >>> >>> HTH, >>> >>> David >>> >>> >>> >>> >>-------------------------------------------------------------- >>---------- >> >> >>> *From:* [EMAIL PROTECTED] >>> <mailto:[EMAIL PROTECTED]> >>> [mailto:[EMAIL PROTECTED] >>> <mailto:[EMAIL PROTECTED]>] *On >>> Behalf Of *Sunburned Surveyor >>> *Sent:* March 30, 2007 9:16 AM >>> *To:* List for discussion of JPP development and use.; >>> User-friendly Desktop Internet GIS >>> *Subject:* [JPP-Devel] Fwd: [Freegis-list] Binary >>> >>> >>Format For Features >> >> >>> >>> ---------- Forwarded message ---------- >>> From: *Sunburned Surveyor *<[EMAIL PROTECTED] >>> <mailto:[EMAIL PROTECTED]>> >>> Date: Mar 30, 2007 9:15 AM >>> Subject: Re: [Freegis-list] Binary Format For Features >>> To: Frank Warmerdam <[EMAIL PROTECTED] >>> >>> >><mailto:[EMAIL PROTECTED]>> >> >> >>> Frank, >>> >>> Thank you for your comments. I am going to forward your >>> >>> >>e-mail to >> >> >>> the JPP Developer and GeoTools lists. Please see my >>> >>> >>comments below. >> >> >>> You wrote: "For instance, the header appears to be a feature >>> header, rather than file >>> header, is that right? Shouldn't there be some sort of >>> >>> >>file level >> >> >>> header >>> or is it intended that a file is just a collection of features >>> appended?" >>> >>> I should have explained this better. For my immediate purposes I >>> only need to store a single feature in a binary file. >>> >>> >>(This has to >> >> >>> do with a system I'm trying to design for OpenJUMP that will >>> allows us to access features stored in permanent memory >>> >>> >>instead of >> >> >>> in RAM.) However, I realize an important and practical >>> >>> >>use of this >> >> >>> format would be storage of multiple features in a single binary >>> file. I would be more than willing to expand the format >>> specification to include this. In fact, I think it >>> >>> >>might be a good >> >> >>> idea to specify a way to do both. Specify a single feature in a >>> binary file, and multiple features in a single file. (Or perhaps >>> you would use the same format, and would only store a single >>> feature in a file when needed???) >>> >>> You wrote: "I'd like some options for efficient access. The most >>> obvious thing needed >>> for this is an index to the features relating feature id with >>> offset in >>> the file. This also gives the opportunity to add a >>> >>> >>spatial index, >> >> >>> perhaps >>> as a seperate file, at some point in the future." >>> >>> I agree. This is something that would be important for >>> >>> >>storage of >> >> >>> multiple features in a single file format and should be >>> >>> >>included. >> >> >>> I left it out because I was only thinking of storing a single >>> feature initially. >>> >>> You wrote: "This also gives the opportunity to add a spatial >>> index, perhaps >>> as a seperate file, at some point in the future." >>> >>> This would also be a good idea if we are storing >>> >>> >>multiple features >> >> >>> in a single file. An index for attribute values coulb >>> >>> >>be stored in >> >> >>> the same way. These could be future additions to the >>> >>> >>format as you >> >> >>> suggest. I do, however, think there is wisdom in storing the >>> feature geomtries separately. I'll be using JTS in WKB >>> >>> >>format, but >> >> >>> there may be other types of geometry definitions that >>> >>> >>people would >> >> >>> like to use. If we can keep the storage of feature geometries >>> segragrated I think the format will be more flexible. >>> >>> You wrote: "Is there a good place to discuss this?" >>> >>> I think we can have the discussion on the JPP Developer Mailing >>> List for the time being. You can subscribe here: >>> >>> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >>> <https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel> >>> >>> I'm open to other forums for discussion if you have a >>> >>> >>suggestion. >> >> >>> You wrote: "I think it would be great if a Java >>> >>> >>implementation of >> >> >>> this could be done >>> in a way that would easily drop into Geotools without license >>> conflicts. >>> I think Geotools is LGPL, is that right?" >>> >>> I'm open to negotiation on the license for the parser. I don't >>> think I'd have a problem with the LGPL, but I'd need to get a >>> little more familiar with its terms. >>> >>> Landon (A.K.A. - The Sunburned Surveyor) >>> >>> >>> On 3/30/07, *Frank Warmerdam* < [EMAIL PROTECTED] >>> <mailto:[EMAIL PROTECTED]>> wrote: >>> >>> Sunburned Surveyor wrote: >>> >>> >>>>I'm currently talking with some of the other OpenJUMP developers >>>> >>>> >>> about a >>> >>> >>>>simple binary format that we can use to store Features in >>>> >>>> >>a way that >> >> >>>>complements the storage of geometries in the OpenGIS >>>>Well-Known-Binary WKB format. I thought this might be of >>>> >>>> >>interest to >> >> >>>>other open source GIS projects, so I am sending a brief message >>>> >>>> >>> here. >>> >>> >>>>I'd be willing to work with developers from other projects on a >>>> >>>> >>> format >>> >>> >>>>that could be used by other applications. >>>> >>>>I've put a rough draft of the format here: >>>> >>>> >>>> >>http://thejumppilotproject.pbwiki.com/OpenJUMP-Binary-Feature-Format >> >> >>>>Let me know if your interested, or if you have any suggestions. I >>>> >>>> >>> hope >>> >>> >>>>to finalize the format in the next week or two. In the next month >>>> >>>> >>> or two >>> >>> >>>>I hope to have a working parser for the binary format in Java >>>> >>>> >>> that will >>> >>> >>>>be released under the GPL through the SurveyOS SourceForge Project. >>>> >>>> >>> Landon, >>> >>> I am interested in this effort. I think a binary >>> >>> >>"simple features" >> >> >>> file format >>> with the potential for efficient access is interesting, and >>> valuable, and I >>> might like to implement support for it in OGR as well. >>> >>> I reviewed >>> >>> >>> >>http://thejumppilotproject.pbwiki.com/OpenJUMP-Binary-Feature-Format >> >> >>> and what is there looks good, but it seems incomplete. >>> >>> For instance, the header appears to be a feature header, rather >>> than file >>> header, is that right? Shouldn't there be some sort of >>> >>> >>file level >> >> >>> header >>> or is it intended that a file is just a collection of features >>> appended? >>> >>> I'd like some options for efficient access. The most >>> >>> >>obvious thing >> >> >>> needed >>> for this is an index to the features relating feature id with >>> offset in >>> the file. This also gives the opportunity to add a >>> >>> >>spatial index, >> >> >>> perhaps >>> as a seperate file, at some point in the future. >>> >>> Is there a good place to discuss this? >>> >>> PS. I think it would be great if a Java implementation of this >>> could be done >>> in a way that would easily drop into Geotools without license >>> conflicts. >>> I think Geotools is LGPL, is that right? >>> >>> Best regards, >>> -- >>> >>> >>> >>---------------------------------------+---------------------- >>---------------- >> >> >>> I set the clouds in motion - turn up | Frank Warmerdam, >>> [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> >>> light and sound - activate the windows | >>> http://pobox.com/~warmerdam <http://pobox.com/%7Ewarmerdam> >>> and watch the world go round - Rush | President OSGeo, >>> http://osgeo.org <http://osgeo.org/> >>> >>> >>> >>> >>> >>> >>> >>-------------------------------------------------------------- >>----------- >> >> >>> Take Surveys. Earn Cash. Influence the Future of IT >>> Join SourceForge.net's Techsay panel and you'll get the >>> >>> >>chance to >> >> >>> share your >>> opinions on IT & business topics through brief >>> >>> >>surveys-and earn cash >> >> >>> >>> >>> >>http://www.techsay.com/default.php?page=join.php&p=sourceforge >> >> >&CID=DEVDEV > > >> >> <http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV> >> _______________________________________________ >> Jump-pilot-devel mailing list >> Jump-pilot-devel@lists.sourceforge.net >> <mailto:Jump-pilot-devel@lists.sourceforge.net> >> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> <https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel> >> >> >>------------------------------------------------------------------------ >> >>------------------------------------------------------------------------- >>Take Surveys. Earn Cash. Influence the Future of IT >>Join SourceForge.net's Techsay panel and you'll get the chance to share your >>opinions on IT & business topics through brief surveys-and earn cash >>http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> >>------------------------------------------------------------------------ >> >>_______________________________________________ >>Jump-pilot-devel mailing list >>Jump-pilot-devel@lists.sourceforge.net >>https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> >> >> >> > > >------------------------------------------------------------------------- >Take Surveys. Earn Cash. Influence the Future of IT >Join SourceForge.net's Techsay panel and you'll get the chance to share your >opinions on IT & business topics through brief surveys-and earn cash >http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >_______________________________________________ >Jump-pilot-devel mailing list >Jump-pilot-devel@lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > >------------------------------------------------------------------------- >Take Surveys. Earn Cash. Influence the Future of IT >Join SourceForge.net's Techsay panel and you'll get the chance to share your >opinions on IT & business topics through brief surveys-and earn cash >http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >_______________________________________________ >Jump-pilot-devel mailing list >Jump-pilot-devel@lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
