This sounds great Gabriel, a truly impressive amount of work, congrats on finishing up! Would you like to make an experimental release? Or rather, I can volunteer to make a release for you, since you've been overworked getting this all done. This would probably get a few more people testing out the code. Just let me know, I can try to find the time sometime next week. And I'm excited to check it out, it opens up a lot of exciting doors, like Shapefiles passing cite tests by default.
best regards, Chris On Thu, 17 Nov 2005, GabrielRoldán wrote: > For those interested: > > Dear lists, This is an status update for the job done on the ComplexDataStore > project encouraged by Social Change Online. > > The first phase of the project is almost complete, and a > working example is provided on the geoserver's complex-features branch. > > The functional goal of this project's initial phase was to serve spatial data > through WFS, which is stored in an organization's internal RDBMS, conforming > to an externally defined schema, or "community schema". > > > A number of actions had to be made in order to seamlessly integrate this > functionality into the GeoServer product. Lets review them in a bottom-up > scenario: > > 1). Feature instances have to be served through a non spatial RDBMS, where > the spatial attribute is constructed from a pair of table fields holding X > and Y ordinates. > > 2). For technical and/or business reasons, the dataset exposed has to be > derived from an SQL query, which provides the full power of the SQL language > for creating a "view" of the dataset that better represents the featureset to > serve, and allows native RDBMS optimizations. > > 3). Input queries against the FeatureType derived > from an SQL query must be translated to the correct backend SQL query. > > 4). As the output schema differs from the input one, it must be possible to > define the attribute mappings from the input FeatureType to the output > FeatureType, which involves not only direct mappings or "aliases", but the > ability to derive an output schema property from a combination of input > schema properties. > > 5). As the output schema defines a complex FeatureType, "complex" meaning > that attributes may have multiplicity other than 0..1 or 1..0, and they may > have nested properties of any level of deepness, the GeoTools feature model > must support this kind of feature attributes. > > 6). Having support for complex attributes, and being the geotools restriction > model being based on OGC Filter 1.0 spec, the geotools Filter implementation > must be able of defining dataset restrictions using XPath attribute > expressions and correctly evaluate them. > > 7). For performance reasons, filters made against the complex output schema > must be "unrolled" to its equivalent construct against the input schema, > avoiding a full run-time evaluation and allowing the backend datastore to > optimize the query. > > 8). Finally, all this functionality must integrate seamlessly on the > geoserver WFS and WMS services, which means that the GML production of a > complex dataset must be possible and no code modifications should be needed > to the GeoServer codebase other than for fixing those cases where it is > assumed a flat FeatureType structure. > > This are the changes implemented in accomplishing each of these specific > requirements: > > - For 1). the geometryless datastore has been reviewed and extended to > support 2). > > - For 2) and 3) a new subinterface of org.geotools.data.DataStore has been > defined, named SqlDataStore, which defines a method for registering feature > types from user defined SQL queries, and a series of utility classes was > developed to help any existing JDBCDataStore in the implementation of this > new interface. > > - For 4) the complexDataStore plugin implements a FeatureTypeMapping class, > which acts as a placeholder for the definition of attribute and id mappings > between an input schema and an output one. This mappings are based on a set > of XPath location paths that addresses a target property, each one paired > with an org.geotools.filter.Expression, that defines how to contruct the > target property value from an input Feature, giving a lot of power of > flexibility to the attribute mapping. Also, a XML schema was defined to > support the persistence of the mappings and the output schema, since it > needs to be parsed before use, and since its defined externally, it could > not be acquired from an existent data source. > > - For 5) and 6) the geotools Feature Model has been revised and updated, > generating a set of GeoAPI interfaces and an implementation with enough > functionality as needed by this project. In the hope of providing a workable > upgrade path from the old Feature model to the new one, old geotools > interfaces have beed deprecated and its implementations moved to the > implementation of the new interfaces, ensuring that all the pre-existing code > base still works as used (that is, no unit tests broken), and explicitly > using the new interfaces just where needed, like in the implementation of the > complexds plugin and the review of the Filter package to operate against > complex attributes. > > - For 7) each time a query is made against the output schema, it is > transformed to its equivalent in the input schema by the use of a > FilterVisitor that basically maps attribute expressions to the mapping > expression defined in a FeatureTypeMapping, and returns a new Filter to > operate against the surrogate feature type. > > -For 8), a new GeoServer branch was created from trunk, the geotools > FeatureTransformer has been updated to encode complex attributes, the new > jars are contained in this geoserver branch, and no code modifications were > needed to get it working other than some very trivial changes. Anyway, this > is what still needs a bit more of work and testing, since FeatureType > encoding to XML schema isn't still working, but WMS and GetFeature queries > seems to be working correctly. > > There are 2 code repository branches holding the implementation of this > functionality, one for geotools and another for geoserver: > > http://svn.geotools.org/geotools/branches/complex-features/ > https://svn.codehaus.org/geoserver/branches/ > > if you're going to use the geotools branch, note that the new feature model > interfaces and implementation are in gt/module/main/opengis/src and > gt/module/main/opengis/test. > > The above mentioned GeoServer branch has working examples of this stuff, you > should only need to edit file locations paths in the conf directory and set > up the test database. > > To set up the test dabase on PostgreSQL use the script in > conf/data/featureTypes/wq_ir_results/create_wqdp.sql, and import the data in > the wq_ir_results.dat file. > > To get them all running, adjust databse connection parameters and file > location paths in the geoserver catalog.xml found on conf/WEB-INF > > If you want to see/test/change the attribute mapping definitions from input > to output schemas, edit the file wq_plus_mappings.xml or roadsegments.xml in > their respective feature type directories under conf/data. > > > As a final note, all this stuff seems to be working, except a few issues I am > going to fix in the next couple days, but at least it is ready for testing > out. Once you have geoserver running and verified that the wq_plus feature > type is exposed in the GetCapabilities document, you can try the provided > sample WFS requests that you'll find on the geoserver "demo requests" web > page, or open the map preview page to see them being served by the WMS. > > > As always, comments/suggestions are appreciated, and thanks to all for your > involvement and support, it was a pleasure working on this project, and a bit > of a pain too due to the immense amount of work that have to be done to get > it as right as we could. Fortunatelly, I think we have overfilled the initial > project goals, and there is place for further enhancements and contributions. > > Gabriel. > > > ------------------------------------------------------- > This SF.Net email is sponsored by the JBoss Inc. Get Certified Today > Register for a JBoss Training Course. Free Certification Exam > for All Training Attendees Through End of 2005. For more info visit: > http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click > _______________________________________________ > Geotools-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/geotools-devel > -- ------------------------------------------------------- This SF.Net email is sponsored by the JBoss Inc. Get Certified Today Register for a JBoss Training Course. Free Certification Exam for All Training Attendees Through End of 2005. For more info visit: http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click _______________________________________________ Geotools-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geotools-devel
