Hi Nuno,
I’m not sure if you’ve seen it, but there has been some work on adding related
tables support to GeoPackage
This is public at http://www.geopackage.org/18-000.html (or
https://github.com/jyutzler/geopackage-related-tables which is basically the
source code for the spec).
If I got it right, this should provide for discoverable feature to feature(s)
linkage, along with feature to attribute sets. Note that the original scope was
feature-to-media (e.g. photos of a location), which is less applicable to
app-schema.
Brad
From: Nuno Oliveira <nuno.olive...@geo-solutions.it>
Sent: Monday, 30 April 2018 3:01 AM
To: Geoserver-devel <geoserver-de...@lists.sourceforge.net>; GeoTools
Developers <geotools-devel@lists.sourceforge.net>
Subject: [Geoserver-devel] Integrating App-Schema and Apache Solr
Dear all,
sorry for the cross posting on both mailing list ... but well App-Schema "as a
leg" on both projects
and this may be interesting for people on both mailing lists.
... and my apologies for the wall of text that follows ... what can I say
complex features are really
a complex subject :)
Lately I have been working in integrating App-Schema with Apache Solr [1] which
is a well know
solution for indexing and (faceting) searching document.
We already have an Apache Solr store in both GT and GS, configuring a layer for
it requires the user
to provide some extra information, e.g. which index fields should be
considered, should multivalued
fields be used, should empty fields be used and which is attribute should be
considered the
default geometry.
The first step for the integration between Apache Solr and App-Schema is to
allow App-Schema to
use Apache Solr has it one of its data store. When in the context of App-schema
we already have
most of the information we need, we already say indirectly through the mappings
which fields of
the index we are interested in, which ones are multivalued (<multivalue>), etc
...
The Apache-Solr data store configuration doesn't fit in the usual MAP
parameters configuration
model ... and we also need to automatically detect during the parsing of the
data stores
configuration that we are dealing with an Apache Solr data store and do all the
possible
automatic steps.
I have come up with the following syntax for the Apache Sorl data store
configuration, please find
the complete mappings file attached to this mail (mappings_solr.xml):
(...)
<sourceDataStores>
<SolrDataStore>
<id>stations</id>
<url>http://localhost:8983/solr/stations</url>
<index name="stations">
<geometry default="true">
<name>station_location</name>
<srid>4326</srid>
<type>POINT</type>
</geometry>
</index>
</SolrDataStore>
</sourceDataStores>
(...)
All the other necessary information will be automatically inferred from the
App-Schema mappings
file when possible, at the exception of multivalued fields. An Apache Solr
multivalued field
handles \ contains multiple values for the same attribute.
Apache Solr multivalued fields are related with App-Schema multivalued fields,
i.e. simple attributes
(String, numeric, dates, etc ...) that can be encoded multiple times. Currently
in App-Schema
this attributes can be mapped using a denormalized data store (database) or
using a specific
feature chaining syntax when client properties are involved.
Here I would like to extend the App-Schema notion of multivalued attributes and
allow each data
data store to contribute a specific handler with a custom syntax has needed:
* Apache Solr could use multivalued attributes
* MongoDB could use JSON arrays
* SQL could define explicit foreign key relation
Please find full examples of mappings using custom handlers attached to this
mail (multi_*). For SQL
data stores the syntax looks like this:
(...)
<AttributeMapping>
<targetAttribute>st:comment</targetAttribute>
<jdbcMultipleValue>
<sourceColumn>id</sourceColumn>
<targetTable>meteo_stations_comments</targetTable>
<targetColumn>station_id</targetColumn>
<targetValue>comment</targetValue>
</jdbcMultipleValue>
</AttributeMapping>
(...)
A station can have multiple comment associated to it, this comment are stored
in meteo_stations_comments
table which has a foreign key to stations table base on columns id and
station_id. The target value can be an
OCQL expression.
In the case of Apache Solr the syntax would look like this:
(...)
<AttributeMapping>
<targetAttribute>st:comment</targetAttribute>
<solrMultipleValue>station_comment</solrMultipleValue>
</AttributeMapping>
(...)
I have a prototype of all of this already working, all of the things above
required creating a few extensions
points in App-Schema:
* in the mappings configuration parsing module to handle the custom syntaxes of
each data store
* in the App-Schema feature iterator to give a chance to custom handlers to do
their job
This changes open the doors to the usage of non relational data stores with
App-Schema and IMHO this syntaxes
promote \ allow clearer mappings files, of curse this changes and new syntaxes
will need to be clearly documented
in the App-Schema documentation.
All comments about this are welcome.
Kind regards,
Nuno Oliveira
--
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo
è consentito esclusivamente al destinatario del messaggio, per le finalità
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro
sistema. Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the
attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act (Legislative
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in
accord with its purpose, any disclosure, reproduction, copying, distribution,
or either dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the intended
recipient, please contact immediately the sender by telephone, fax or e-mail
and delete the information in this message that has been received in error. The
sender does not give any warranty or accept liability as the content, accuracy
or completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
