Ah! my apologies Brad your reply got lost in the mail flow :(
That's an interesting reference indeed, thank you.
In this situation, I'm kind of tied to the App-Schema mappings files structure
and the code that's does the work behind the scenes. Maybe in the future it
could be used to provide an alternative syntax to the current App-Schema
features chaining syntax, although the App-Schema chaining mechanism handle a
few more complicated use cases.
On 05/03/2018 02:59 PM, br...@frogmouth.net wrote:
Hi Nuno,
I’m not sure if you’ve seen it, but there has been some work on adding related
tables support to GeoPackage
This is public at http://www.geopackage.org/18-000.html (or
https://github.com/jyutzler/geopackage-related-tables which is basically the
source code for the spec).
If I got it right, this should provide for discoverable feature to feature(s)
linkage, along with feature to attribute sets. Note that the original scope
was feature-to-media (e.g. photos of a location), which is less applicable to
app-schema.
Brad
*From:*Nuno Oliveira <nuno.olive...@geo-solutions.it>
*Sent:* Monday, 30 April 2018 3:01 AM
*To:* Geoserver-devel <geoserver-de...@lists.sourceforge.net>; GeoTools
Developers <geotools-devel@lists.sourceforge.net>
*Subject:* [Geoserver-devel] Integrating App-Schema and Apache Solr
Dear all,
sorry for the cross posting on both mailing list ... but well App-Schema "as a
leg" on both projects
and this may be interesting for people on both mailing lists.
... and my apologies for the wall of text that follows ... what can I say
complex features are really
a complex subject :)
Lately I have been working in integrating App-Schema with Apache Solr [1]
which is a well know
solution for indexing and (faceting) searching document.
We already have an Apache Solr store in both GT and GS, configuring a layer
for it requires the user
to provide some extra information, e.g. which index fields should be
considered, should multivalued
fields be used, should empty fields be used and which is attribute should be
considered the
default geometry.
The first step for the integration between Apache Solr and App-Schema is to
allow App-Schema to
use Apache Solr has it one of its data store. When in the context of
App-schema we already have
most of the information we need, we already say indirectly through the
mappings which fields of
the index we are interested in, which ones are multivalued (<multivalue>), etc
...
The Apache-Solr data store configuration doesn't fit in the usual MAP
parameters configuration
model ... and we also need to automatically detect during the parsing of the
data stores
configuration that we are dealing with an Apache Solr data store and do all
the possible
automatic steps.
I have come up with the following syntax for the Apache Sorl data store
configuration, please find
the complete mappings file attached to this mail (mappings_solr.xml):
(...)
<sourceDataStores>
<SolrDataStore>
<id>stations</id>
<url>http://localhost:8983/solr/stations</url>
<index name="stations">
<geometry default="true">
<name>station_location</name>
<srid>4326</srid>
<type>POINT</type>
</geometry>
</index>
</SolrDataStore>
</sourceDataStores>
(...)
All the other necessary information will be automatically inferred from the
App-Schema mappings
file when possible, at the exception of multivalued fields. An Apache Solr
multivalued field
handles \ contains multiple values for the same attribute.
Apache Solr multivalued fields are related with App-Schema multivalued fields,
i.e. simple attributes
(String, numeric, dates, etc ...) that can be encoded multiple times.
Currently in App-Schema
this attributes can be mapped using a denormalized data store (database) or
using a specific
feature chaining syntax when client properties are involved.
Here I would like to extend the App-Schema notion of multivalued attributes
and allow each data
data store to contribute a specific handler with a custom syntax has needed:
* Apache Solr could use multivalued attributes
* MongoDB could use JSON arrays
* SQL could define explicit foreign key relation
Please find full examples of mappings using custom handlers attached to this
mail (multi_*). For SQL
data stores the syntax looks like this:
(...)
<AttributeMapping>
<targetAttribute>st:comment</targetAttribute>
<jdbcMultipleValue>
<sourceColumn>id</sourceColumn>
<targetTable>meteo_stations_comments</targetTable>
<targetColumn>station_id</targetColumn>
<targetValue>comment</targetValue>
</jdbcMultipleValue>
</AttributeMapping>
(...)
A station can have multiple comment associated to it, this comment are stored
in /meteo_stations_comments
/table which has a foreign key to /stations /table base on columns /id /and
/station_id/. The target value can be an
OCQL expression.
In the case of Apache Solr the syntax would look like this:
(...)
<AttributeMapping>
<targetAttribute>st:comment</targetAttribute>
<solrMultipleValue>station_comment</solrMultipleValue>
</AttributeMapping>
(...)
I have a prototype of all of this already working, all of the things above
required creating a few extensions
points in App-Schema:
* in the mappings configuration parsing module to handle the custom
syntaxes of each data store
* in the App-Schema feature iterator to give a chance to custom handlers
to do their job
This changes open the doors to the usage of non relational data stores with
App-Schema and IMHO this syntaxes
promote \ allow clearer mappings files, of curse this changes and new syntaxes
will need to be clearly documented
in the App-Schema documentation.
All comments about this are welcome.
Kind regards,
Nuno Oliveira
--
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the experts! Visithttp://goo.gl/it488V
for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo
è consentito esclusivamente al destinatario del messaggio, per le finalità
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro
sistema. Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.
--
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the experts!
Visit http://goo.gl/it488V for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
Con riferimento alla normativa sul trattamento dei dati
personali (Reg. UE 2016/679 - Regolamento generale sulla
protezione dei dati “GDPR”), si precisa che ogni
circostanza inerente alla presente email (il suo contenuto,
gli eventuali allegati, etc.) è un dato la cui conoscenza
è riservata al/i solo/i destinatario/i indicati dallo
scrivente. Se il messaggio Le è giunto per errore, è
tenuta/o a cancellarlo, ogni altra operazione è illecita.
Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to
which it is addressed and may contain information that
is privileged, confidential or otherwise protected from
disclosure. We remind that - as provided by European
Regulation 2016/679 “GDPR” - copying, dissemination or
use of this e-mail or the information herein by anyone
other than the intended recipient is prohibited. If you
have received this email by mistake, please notify
us immediately by telephone or e-mail.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
