Thank you Andrea, that is a sensible -1 vote with a viable workaround for Alvaro.
Alvaro is there another index format available with an open source implementation (so we could create one). It is hard when working with DBF (a file format from the 1980s) to find any modern custodian. I think Excel and Access can import the format, but I doubt they support an index. Reading OGR docs: *Currently the OGR Shapefile driver only supports attribute indexes for looking up specific values in a unique key column. To create an attribute index for a column issue an SQL command of the form "CREATE INDEX ON tablename USING fieldname". To drop the attribute indexes issue a command of the form "DROP INDEX ON tablename". The attribute index will accelerate WHERE clause searches of the form "fieldname = value". The attribute index is actually stored as a mapinfo format index and is not compatible with any other shapefile applications.* So some kind of mapinfo format index. -- Jody Garnett On 29 November 2015 at 10:36, Andrea Aime <andrea.a...@geo-solutions.it> wrote: > Hi, > regarding this proposal: > > https://github.com/geotools/geotools/wiki/Implement-a-pure-java-Dbase-indexing-to-optimize-shapefile-access > > After reviewing the proposal and checking the current pull request, I'm > going to cast a -1, fair and square. > > I'm quite happy to see work on indexing the DBF files, but the current > implementation choice is not acceptable > for a few reasons: > * It bakes a large-ish amount of new untested code into a core supported > module that's used by many users > * The current implementation causes evident performance regressions if the > CDX file is not there > * The chosen format, CDX, has been created by Microsoft for Visual Foxpro, > and can only be created > by Visual FoxPro > * Visual FoxPro has been completely abandoned by Microsoft, the last > release of it being dated 2007 > https://msdn.microsoft.com/it-it/vfoxpro/bb190225.aspx > * The current implementation of the CDX support cannot create an index, > just use an existing one, so you > either own a Visual FoxPro license and can run windows, or you're toast > > Long story short, the implementation is of interest of a niche within a > niche, the subset of users that > cannot use a proper spatial database, nor H2 with spatial extensions, nor > GeoPackage, > but demand usage of shapefiles, and still own a 8 years old license of > Visual FoxPro to create the > CDX files. > > That said, I don't want to reject the code, just make it manageable so > that only the few interested > parties can be affected by its presence, and without burdening myself (as > the shapefile module > maintainer) with code that in the current state only helps a minuscule > portion of the user base. > > Alvaro already made modifications to the shapefile store to support > generic attribute indexing, > with interfaces to support other index types (e.g., MDX), in order to make > the contribution acceptable all that's required > is to make it actually pluggable, via SPI, so that the CDX index > implementation can reside in another module. > And then create a gt-shapefile-cdx unsupported module that contains the > CDX indexing support, which will > be maintained by Alvaro himself, so that when that jar is plugged in, CDX > files will be used > for attribute indexing. > This will also lower the bar for entry, as an unsupported module does not > have particular requirements > for code quality or testing level, it's there for everybody to try at > their own risk. > > Alvaro, in order to make the plugin system work you'll need to write a > factory class for each index type > and a finder helping to locate the factories. The latest extension point > we added is for projection handlers > in the referencing subsystem, you can find the commit introducing it here: > > https://github.com/geotools/geotools/commit/d206957bc4d8de1f7d33e06501aa3b9c95496d7d > > For reference, the moving parts of the SPI system used by > ProjectionHandler are: > * The object being created, in this case, an implementation of the > ProjectionHandler interface: > https://github.com/geotools/geotools/blob/master/modules/library/render/src/main/java/org/geotools/renderer/crs/ProjectionHandler.java > * The factory that creates it: > https://github.com/geotools/geotools/blob/master/modules/library/render/src/main/java/org/geotools/renderer/crs/ProjectionHandlerFactory.java > * The finder that can be invoked to locate all the factories: > https://github.com/geotools/geotools/blob/master/modules/library/render/src/main/java/org/geotools/renderer/crs/ProjectionHandlerFinder.java > * The registration files listing which factories are available, in > META-INF, using the same name as the factory itself: > https://github.com/geotools/geotools/blob/master/modules/library/render/src/main/resources/META-INF/services/org.geotools.renderer.crs.ProjectionHandlerFactory > > You already have the interface and implementation, and will have to add > the three other bits (for a single implementation > it should be quick). > > The new unsupported module will give the CDX code an occasion to be tested > by the interested user base > without affecting the general user population, and will greatly expedite > the inclusion in the code base (you will > receive a detailed review only of the changes in gt-shapefile, to make > sure there are no functional and performance > regressions for the index-less case). > > Cheers > Andrea > > > > > -- > == > GeoServer Professional Services from the experts! Visit > http://goo.gl/it488V for more information. > == > > *Geosolutions' Winter Holidays from 24/12 to 6/1* > > Ing. Andrea Aime > @geowolf > Technical Lead > > GeoSolutions S.A.S. > Via Poggio alle Viti 1187 > 55054 Massarosa (LU) > Italy > phone: +39 0584 962313 > fax: +39 0584 1660272 > mob: +39 339 8844549 > > http://www.geo-solutions.it > http://twitter.com/geosolutions_it > > *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* > > Le informazioni contenute in questo messaggio di posta elettronica e/o > nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il > loro utilizzo è consentito esclusivamente al destinatario del messaggio, > per le finalità indicate nel messaggio stesso. Qualora riceviate questo > messaggio senza esserne il destinatario, Vi preghiamo cortesemente di > darcene notizia via e-mail e di procedere alla distruzione del messaggio > stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, > divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od > utilizzarlo per finalità diverse, costituisce comportamento contrario ai > principi dettati dal D.Lgs. 196/2003. > > > > The information in this message and/or attachments, is intended solely for > the attention and use of the named addressee(s) and may be confidential or > proprietary in nature or covered by the provisions of privacy act > (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection > Code).Any use not in accord with its purpose, any disclosure, reproduction, > copying, distribution, or either dissemination, either whole or partial, is > strictly forbidden except previous formal approval of the named > addressee(s). If you are not the intended recipient, please contact > immediately the sender by telephone, fax or e-mail and delete the > information in this message that has been received in error. The sender > does not give any warranty or accept liability as the content, accuracy or > completeness of sent messages and accepts no responsibility for changes > made after they were sent or for other risks which arise as a result of > e-mail transmission, viruses, etc. > > ------------------------------------------------------- > > > ------------------------------------------------------------------------------ > > _______________________________________________ > GeoTools-Devel mailing list > GeoTools-Devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/geotools-devel > >
------------------------------------------------------------------------------
_______________________________________________ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
