I was more thinking about this indeed: "your work could be extended to
automatically infer a mapping definition from the geopackage with related tables"
Implementing an output for this could be done, a few moths ago I implemented the
GeoJson complex output [1]. My main doubt (something I will need to think about)
is how to output the correct relations, right now App-Schema is mainly XML driven.
[1]
https://github.com/geoserver/geoserver/pull/2201/files#diff-4e0792e51b036935d54e2fb651e0ce7e
On 06/01/2018 01:09 AM, br...@frogmouth.net wrote:
I was more thinking about this as an output format / representation of the
complex feature. As a future consideration (if we ever had time😊) your work
could be extended to automatically infer a mapping definition from the
geopackage with related tables.
Brad
*From:*Nuno Oliveira <nuno.olive...@geo-solutions.it>
*Sent:* Friday, 1 June 2018 5:55 AM
*To:* br...@frogmouth.net; 'Geoserver-devel'
<geoserver-de...@lists.sourceforge.net>; 'GeoTools Developers'
<geotools-devel@lists.sourceforge.net>
*Subject:* Re: [Geoserver-devel] Integrating App-Schema and Apache Solr
Ah! my apologies Brad your reply got lost in the mail flow :(
That's an interesting reference indeed, thank you.
In this situation, I'm kind of tied to the App-Schema mappings files structure
and the code that's does the work behind the scenes.
Maybe in the future this could be used to provide an alternative syntax to the
current App-Schema features chaining syntax, although the App-Schema chaining
mechanism handle a few more complicated use cases ...
On 05/03/2018 02:59 PM, br...@frogmouth.net <mailto:br...@frogmouth.net> wrote:
Hi Nuno,
I’m not sure if you’ve seen it, but there has been some work on adding
related tables support to GeoPackage
This is public at http://www.geopackage.org/18-000.html (or
https://github.com/jyutzler/geopackage-related-tables which is basically
the source code for the spec).
If I got it right, this should provide for discoverable feature to
feature(s) linkage, along with feature to attribute sets. Note that the
original scope was feature-to-media (e.g. photos of a location), which is
less applicable to app-schema.
Brad
*From:*Nuno Oliveira <nuno.olive...@geo-solutions.it>
<mailto:nuno.olive...@geo-solutions.it>
*Sent:* Monday, 30 April 2018 3:01 AM
*To:* Geoserver-devel <geoserver-de...@lists.sourceforge.net>
<mailto:geoserver-de...@lists.sourceforge.net>; GeoTools Developers
<geotools-devel@lists.sourceforge.net>
<mailto:geotools-devel@lists.sourceforge.net>
*Subject:* [Geoserver-devel] Integrating App-Schema and Apache Solr
Dear all,
sorry for the cross posting on both mailing list ... but well App-Schema
"as a leg" on both projects
and this may be interesting for people on both mailing lists.
... and my apologies for the wall of text that follows ... what can I say
complex features are really
a complex subject :)
Lately I have been working in integrating App-Schema with Apache Solr [1]
which is a well know
solution for indexing and (faceting) searching document.
We already have an Apache Solr store in both GT and GS, configuring a
layer for it requires the user
to provide some extra information, e.g. which index fields should be
considered, should multivalued
fields be used, should empty fields be used and which is attribute should
be considered the
default geometry.
The first step for the integration between Apache Solr and App-Schema is
to allow App-Schema to
use Apache Solr has it one of its data store. When in the context of
App-schema we already have
most of the information we need, we already say indirectly through the
mappings which fields of
the index we are interested in, which ones are multivalued (<multivalue>),
etc ...
The Apache-Solr data store configuration doesn't fit in the usual MAP
parameters configuration
model ... and we also need to automatically detect during the parsing of
the data stores
configuration that we are dealing with an Apache Solr data store and do
all the possible
automatic steps.
I have come up with the following syntax for the Apache Sorl data store
configuration, please find
the complete mappings file attached to this mail (mappings_solr.xml):
(...)
<sourceDataStores>
<SolrDataStore>
<id>stations</id>
<url>http://localhost:8983/solr/stations</url>
<index name="stations">
<geometry default="true">
<name>station_location</name>
<srid>4326</srid>
<type>POINT</type>
</geometry>
</index>
</SolrDataStore>
</sourceDataStores>
(...)
All the other necessary information will be automatically inferred from
the App-Schema mappings
file when possible, at the exception of multivalued fields. An Apache Solr
multivalued field
handles \ contains multiple values for the same attribute.
Apache Solr multivalued fields are related with App-Schema multivalued
fields, i.e. simple attributes
(String, numeric, dates, etc ...) that can be encoded multiple times.
Currently in App-Schema
this attributes can be mapped using a denormalized data store (database)
or using a specific
feature chaining syntax when client properties are involved.
Here I would like to extend the App-Schema notion of multivalued
attributes and allow each data
data store to contribute a specific handler with a custom syntax has needed:
* Apache Solr could use multivalued attributes
* MongoDB could use JSON arrays
* SQL could define explicit foreign key relation
Please find full examples of mappings using custom handlers attached to
this mail (multi_*). For SQL
data stores the syntax looks like this:
(...)
<AttributeMapping>
<targetAttribute>st:comment</targetAttribute>
<jdbcMultipleValue>
<sourceColumn>id</sourceColumn>
<targetTable>meteo_stations_comments</targetTable>
<targetColumn>station_id</targetColumn>
<targetValue>comment</targetValue>
</jdbcMultipleValue>
</AttributeMapping>
(...)
A station can have multiple comment associated to it, this comment are
stored in /meteo_stations_comments
/table which has a foreign key to /stations /table base on columns /id
/and /station_id/. The target value can be an
OCQL expression.
In the case of Apache Solr the syntax would look like this:
(...)
<AttributeMapping>
<targetAttribute>st:comment</targetAttribute>
<solrMultipleValue>station_comment</solrMultipleValue>
</AttributeMapping>
(...)
I have a prototype of all of this already working, all of the things above
required creating a few extensions
points in App-Schema:
* in the mappings configuration parsing module to handle the custom
syntaxes of each data store
* in the App-Schema feature iterator to give a chance to custom
handlers to do their job
This changes open the doors to the usage of non relational data stores
with App-Schema and IMHO this syntaxes
promote \ allow clearer mappings files, of curse this changes and new
syntaxes will need to be clearly documented
in the App-Schema documentation.
All comments about this are welcome.
Kind regards,
Nuno Oliveira
--
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the experts! Visithttp://goo.gl/it488V
for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro
utilizzo è consentito esclusivamente al destinatario del messaggio, per le
finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio
senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via
e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal
Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act (Legislative
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in
accord with its purpose, any disclosure, reproduction, copying, distribution,
or either dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the intended
recipient, please contact immediately the sender by telephone, fax or e-mail
and delete the information in this message that has been received in error. The
sender does not give any warranty or accept liability as the content, accuracy
or completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.
--
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the experts!
Visithttp://goo.gl/it488V for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
Con riferimento alla normativa sul trattamento dei dati
personali (Reg. UE 2016/679 - Regolamento generale sulla
protezione dei dati “GDPR”), si precisa che ogni
circostanza inerente alla presente email (il suo contenuto,
gli eventuali allegati, etc.) è un dato la cui conoscenza
è riservata al/i solo/i destinatario/i indicati dallo
scrivente. Se il messaggio Le è giunto per errore, è
tenuta/o a cancellarlo, ogni altra operazione è illecita.
Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to
which it is addressed and may contain information that
is privileged, confidential or otherwise protected from
disclosure. We remind that - as provided by European
Regulation 2016/679 “GDPR” - copying, dissemination or
use of this e-mail or the information herein by anyone
other than the intended recipient is prohibited. If you
have received this email by mistake, please notify
us immediately by telephone or e-mail.
--
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the experts!
Visit http://goo.gl/it488V for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
Con riferimento alla normativa sul trattamento dei dati
personali (Reg. UE 2016/679 - Regolamento generale sulla
protezione dei dati “GDPR”), si precisa che ogni
circostanza inerente alla presente email (il suo contenuto,
gli eventuali allegati, etc.) è un dato la cui conoscenza
è riservata al/i solo/i destinatario/i indicati dallo
scrivente. Se il messaggio Le è giunto per errore, è
tenuta/o a cancellarlo, ogni altra operazione è illecita.
Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to
which it is addressed and may contain information that
is privileged, confidential or otherwise protected from
disclosure. We remind that - as provided by European
Regulation 2016/679 “GDPR” - copying, dissemination or
use of this e-mail or the information herein by anyone
other than the intended recipient is prohibited. If you
have received this email by mistake, please notify
us immediately by telephone or e-mail.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
