Stu B22,
A) AMQP / JMS publishing of graph change notices
B) OSGi component bundling
Both of those would be interesting.
Fuseki2 (or should that be Joseki5?) tries to abstract idea of an action
from the coupling to HTTP but it was as clean as it might be 0 things
leak back in HTTP like getting the status codes right. Those two items
don't depend on the SPARQL action side of things so should be cleaner to
have.
A general framework for changes, including driving RDF patch, is going
to arise sometime.
For OSGi packaging, adding the creation of OGSi components makes sense -
there will need to be several fuseki packaging maven modules. At the
moment distribution and war file building are all together but that is
creaking.
What it does need is expertise in OGSi - there is no point in someone
just learnign enough OGSi to get it working for it to then be inflexible
and likely rot over time. Apache Clerezza uses OGSi components for some
of Jena for core/arq/tdb - I wonder if adding build steps for OGSi
components to the main build would help everyone.
Andy
On 16/06/14 18:37, Stu Btwotwo wrote:
Hello Jena-Users,
In the jena-dev mailing list, on March 31, Andy announced a preliminary
build of Fuseki 2:
http://mail-archives.apache.org/mod_mbox/jena-dev/201403.mbox/%3C53394253.1070506%40apache.org%3E
He ended that post with the following invitation: “If you integrate
Fuseki with your own systems,
then it would be good to discuss how to make that easier because now is
a good time to make
changes.”
Here I will address that topic from my team’s point of view. I see this
topic as somewhere near the
boundary of “jena-users” and “jena-dev”, so perhaps we can start the
discussion here in the user
space, and then promote to the dev space as warranted.
We use RDF and Jena as the symbolic data backbone of our Glue-AI social
robotics platform,
http://glue.ai. It contains a sub-project called http://appdapter.org
which largely consists
of wrappers of Jena features.
There are several technology connection issues we face that I think may
be of general interest.
To boil down our platform as succinctly as I can:
1) Code is in Java and Scala
2) Build, deploy, and distribute using Maven
3) Semantic data access using Jena (ARQ, Fuseki), persistence using TDB
and AllegroGraph.
4) Packaging of components using OSGi bundles (run under Felix or other
container).
5) Webapp deployment using the Ops4j PAX-Web OSGi wrapper for Jetty.
(This component is quite spiffy, and demonstrates some of the power of
the OSGi platform).
https://ops4j1.jira.com/wiki/display/paxweb/Pax+Web
6) Semantic data change notification and other (non-semantic) data
plumbing using AMQP pub-sub Topics, implemented by Apache QPid, which is
compatible with JMS 1.1.
http://qpid.apache.org/
The interesting intersections occur between Jena/Fuseki #3, OSGi
PAX-Web webapp deployment #5,
and AMQP/QPid/JMS topics #6.
The component we really want to run is an HTTP-enabled SPARQL(+Update)
engine with TDB persistence
(i.e. Fuseki), deployed as a bundle to be discovered and exposed to the
web by PAX-Web,
and able to supply AMQP notifications when graphs change. Our initial
approach to the latter is to
publish to a single Topic called “GraphChangeNotice”, using a range of
QPid “subjects” corresponding
to the different graphs in our datasets. Thus any consumer agent in our
network can easily subscribe
to notices on the graphs it is interested in. An unfinished design area
is the exact representation
of graph change notices; many forms are possible but they are naturally
all similar to a bag of Quads
plus some metadata indicating when, where, and how the change occurred.
There is also a natural
relationship between these notifications and the transaction model used
by our ARQ datasets.
We have some prototypes of each aspect of this component I have
described, but we do not yet have a
complete implementation, in part because of challenges stemming from the
structure of older
versions of Fuseki and Joseki. Of course we can always hack the
features we want on top of
Fuseki source, but then we worry about the cost of maintaining
compatibility with future versions
of the original.
The new ability of Fuseki 2 to compile into a deployable .WAR webapp
will help us somewhat. From
there we will be looking for ways to layer on the kind of graph
monitoring feature that Andy kindly
sketched out for us in this thread from May 2014:
“Notification of Graph Changes by ARQ+TDB, during or after SPARQL Update”
http://mail-archives.apache.org/mod_mbox/jena-users/201405.mbox/%3C5378C885.3030306%40apache.org%3E
...and then connect those notifications to our AMQP topics as I
described above.
Given all this context, I could make suggestions about what I would most
like to see done with the
Fuseki codebase going forward, but I think first it would be good to
find some consensus about how
valuable these two feature categories are seen to be (or not to be) by
the Jena community:
A) AMQP / JMS publishing of graph change notices
B) OSGi component bundling
The purpose and utility of #A is presumably obvious, while the benefits
of #B are more complex and
subtle. Overall, I think the entire Jena stack would benefit from
some attention to the OSGi
universe (which does not require breaking compatibility with the
non-OSGi universe), but I don’t want
to launch into a manifest-o about that unless other folks are really
interested in the subject.
I look forward to hearing what other Jena users and developers think
about these topics!
peace,
Stu B22
