Hi guys,
El 01/07/14 10:20, Antonio David Perez Morales escribió:
Hi all
Continuing with the project, I have managed successfully the integration of
activemq camel component (and also jms) into the Stanbol Camel integration.
This has been a hard task due to the dependencies needed by the component
and also due to the fact that we had to provide an activemq component
configurable through Felix web console.
With this addition, we are in the position to integrate business logic into
Stanbol routes through a message service provided by activemq (jms).
Nice Antonio, let's see is someone has an interesting use case to
implement in this context.
As a first test, I have deployed a route which consumes messages (content)
from an activemq queue, enhance them using the default chain and then write
the result into a file. It's a simple test but it works quite well. In this
case, Stanbol is working in a standalone mode, that is to say, we don't
have to explicitly call Stanbol to enhance content but Stanbol is triggered
based on some external events (a new queue message)
As indicated in the previous mail, I still have some pending things to be
done (because I couldn't do them last week) but in order to go forward with
the project I ask you for some interesting use cases where to apply the new
workflow component in order to give added value to it and also in order to
develop and provide more workflow (camel) components useful for those and
other use cases.
Awaiting for the community feedback and also for Florent's opinion
regarding the rest of the project, as I have expressed in recent emails,
I'm eager to see the Content Hub back in Stanbol. And this is because
of, from the point of view of the use of Stanbol in the enterprise,
Semantic Search is one of the most common use cases. So, to have an
enterprise search backend as the last component of a processing route in
any architecture where stanbol could be plugged sounds key for me. In
recent discussions at the Stanbol IRC channel, we have been analysing
Siren (https://github.com/rdelbru/SIREn), a Lucene/Solr extension which
major advantage is the possibility to index tree structures, allowing
then to index structured data without losing full text search
capabilities. To refactor old ContentHub component to use Siren is out
of scope of this project but, in my opinion, an interesting use case
could be to develop a Siren Camel Component and a transformer from
ContentItem to Siren Object or whatever and integrate both in Stanbol.
What do you guys think?
Cheers,
Rafa
Regards
On Mon, Jun 23, 2014 at 6:16 PM, Antonio David Perez Morales <
ape...@zaizi.com> wrote:
Hi Stanbolers
The GSoC 2014 midterm is here and I want to give you a summary of the work
already done so far:
- Adapted previous Camel integration PoC done by Florent into Stanbol 1.0
version.
- Improved EngineComponent used by Camel to execute Enhancement Engines
(configured through Stanbol web console as usual) using the engine:// uri
scheme in routes.
- Created ChainComponent used by Camel to execute Enhancement Chains using
the chain:// uri scheme in routes (both Camel components are provided as
OSGI components, so the uri scheme can be changed through the Stanbol web
console)
- Created a custom artifact for Apache Felix Fileinstall in order to be
able to install routes defined in Camel Spring XML DSL placing a route file
(with 'route' extension) in the stanbol/fileinstall directory
- Created a custom archetype to ease the development of bundles containing
route definitions in Java DSL. The archetype generates a class extending
'RouteBuilder' which creates a default Camel direct endpoint used by other
Stanbol Workflow components to execute the route.
- Created a first version of Workflow API, which contains different OSGI
components which allow registering Camel components/routes,
start/stop/execute routes, add/remove components used in routes, etc.
- REST endpoint is provided to test the execution of routes using REST
requests (/flow/{routeId} )
- Modified the PoC full launcher to use all the new bundles to support the
workflow feature.
- Installed JBoss developer studio which comes with Camel support in order
to create routes in a visual way with the possibility to be exported as
Spring XML DSL format
Some pending things I will try to do during this week:
- Improve the web package to create the needed endpoints to query the
registered routes, registered camel components, etc
- Improve the web package to remove classes copied from Stanbol jersey
module used for testing
- Update README.md files in the repository with all the new information
- Document the installation and configuration of JBoss developer studio
for Camel routes creation
- Create all the JIRA issued related to the work already done
For the second part of the project, I would like to read some comments
about interesting use cases in order to develop the needed Stanbol and
Camel components to support them.
If you have any comment, please drop some lines in order to discuss the
new things to be done.
Regards
On Sat, Jun 14, 2014 at 3:39 PM, Antonio David Perez Morales <
ape...@zaizi.com> wrote:
Hi guys
Continuing with the project, and as part of the refactoring/new
architecture I have started to modify some workflow components in order to
create a better API and architecture based on OSGI components. As a first
step and in order to have the same behavior than the current one (regarding
enhancement process), a chain component has been created to simulate the
chain behaviour. This new component uses internally the ChainManager and
EnhancementJobManager component to perform the business logic. This way, a
new protocol 'chain' can be used in the routes deployed in Stanbol. The
chains are configured in the same way, using Stanbol admin console.
Now, we can combine single engine executions with chains executions in
routes deployed in Stanbol using the alternatives described in previous
mails and in the issue [1]. Both engines and chains are configured through
Stanbol admin console. You can see the refactoring advances in [2] (a
branch used for refactoring the current PoC of Workflow in Stanbol 1.0). Of
course, the Camel EIP and other Camel components can be used in the
deployed routes as well.
With the new Camel routes support, we can have a Stanbol running and
enhancing content without receiving any HTTP request to start the
enhancement process, because the routes can be triggered by external events
ocurred in a queue, database, etc. Moreover the semantic lifting process
can be splitted and merged with some application steps, so the issue [3]
requesting asynchronous call support for enhancement could be solved.
Anyway, if some of you have any suggestions for new components to be
deployed for the second part of the project, or another kind of suggestion,
please drop here some lines to continue with the discussion.
Regards
[1] https://issues.apache.org/jira/browse/STANBOL-1348
[2]
https://github.com/adperezmorales/stanbol-camel-workflow/tree/refactoring
[3] https://issues.apache.org/jira/browse/STANBOL-263
On Wed, Jun 11, 2014 at 10:01 AM, Antonio David Perez Morales <
ape...@zaizi.com> wrote:
Hi people
As part of the GSoC project for the midterm and according to the issue
[1], a custom Apache Felix Fileinstall artifact has been created in order
to deploy Camel routes defined in XML (Spring DSL) placing a file with
.route extension in a configured directory (like stanbol/fileinstall
directory). Moreover since this artifact depends on Fileinstall bundle, the
created launcher has been modified to have that bundle in the OSGI context
by default.
So, once the current Camel integration POC has been integrated in
Stanbol 1.0 and extended to support the deployment of routes defined by
Java DSL (through bundles) and XML (route files), the next step will be
thinking and redesigning the current architecture trying to avoid the
duplicated code and providing a more extendable and easy to use Workflow
API, because with the current integration only direct routes can be
triggered using REST API which means that the defined routes must be
configured properly using a direct endpoint consumer. Anyway, routes
starting in some other way like timers are triggered directly in the
deployment, so this has to be taken into account for the new API (and REST
API).
In parallel and for the second part, new Stanbol Camel components will
be developed in order to be used in new routes. So if any of you have use
cases for this involving Stanbol components, please drop some lines here in
order to prioritize the Stanbol Camel components to be developed.
Comments and suggestions are more than welcome
Regards
[1] https://issues.apache.org/jira/browse/STANBOL-1348
[2] https://github.com/adperezmorales/stanbol-camel-workflow/
On Mon, Jun 2, 2014 at 7:00 PM, Antonio David Perez Morales <
ape...@zaizi.com> wrote:
Hi stanbolers
As part of the issue [1] , I have created a maven archetype useful to
generate Camel routes in Java DSL.
The archetype generates a Java project with all the dependencies and
one Java class with a method which has to be filled. In this method, Camel
Java DSL syntax is used to create the route.
By default and as a first approach, the class will use the route name
given during the project creation to enable a Camel direct endpoint with
such name.
The code of the first archetype version can be found at [2].
The next task will be providing a Felix custom artifact to be able to
deploy XML-based routes in Stanbol, placing a custom file in the Stanbol
datafiles directory.
After that, it will be time to think and redesign the architecture to
integrate Camel workflows inside Stanbol in a better way, more configurable
and extendable.
Comments and suggestions are more than welcome
Regards
[1] https://issues.apache.org/jira/browse/STANBOL-1348
[2] https://github.com/adperezmorales/stanbol-camel-workflow/
On Fri, May 30, 2014 at 8:03 PM, Antonio David Perez Morales <
ape...@zaizi.com> wrote:
Hi all
After a hard fight this week, I managed to get it work the Florent's
proof of concept code in the Stanbol 1.0 branch [1]
The code is uploaded in my github account [3]. As I said in a previous
mail, I prefer to do it separately and after the project, uploading the
developed code into a Stanbol branch.
The 1.0.0 version has some changes in how the Jersey endpoints are
registered and also new classes and packages, so it was not a trivial task
to make work the current proof of concept. Moreover I don't like to simply
copy and paste code and make the needed changes. I always want to
understand how the things work and how they are developed in order to be
able to change/modify them or develop new code around them.
The steps done to achieve it have been the following:
- Updated pom files to the Stanbol 1.0.0-SNAPSHOT version
- Updated bundle levels in bundlelist package to fit the Stanbol 1.0
version levels
- Adapted cameljobmanager package code to Stanbol 1.0.0-SNAPSHOT
classes and using Java OSGI annotations instead of SCR annotations in
Javadoc
- Updated flow web package to Stanbol 1.0.0-SNAPSHOT classes and
modified needed resources
- Added Java OSGI annotations to the route (WeightedChain) instead of
SCR annotations in javadoc
- Updated launcher to use the 1.0.0-SNAPSHOT packages and needed
bundles
So now, the http://localhost:8080/flow endpoint will use the only
Camel route (defined by WeightedChain) to call all the registered
Enhancement Engines (ordered by EnhancementEngine order property).
For testing purposes, the /flow/{flowName} has been removed, because
all this code needs to be re-designed and re-implemented so I only wanted
to make it work to have a first (simple) integration in Stanbol 1.0. This
functionality will be added again to trigger custom routes once the next
step (defined below) is developed.
The next step [2] will be support to write and configure routes in XML
format, putting the file in datafiles in order to be loaded by a Felix
custom artifact (as Rupert pointed out in a previous mail) and create a
Maven archetype to create bundles defining routes which will be loaded
using the Felix bundle tab. If necessary, as we talked in previous
messages, a REST endpoint receiving routes in XML can be developed as an
alternative to the first approach. This is my objective for the midterm.
After the midterm, the new Stanbol components for Apache Camel will be
developed and also the new architecture for Camel in Stanbol.
Comments on this and for use cases for Stanbol Camel components are
more than welcome.
Regards
[1] https://issues.apache.org/jira/browse/STANBOL-1347
[2] https://issues.apache.org/jira/browse/STANBOL-1348
[3] https://github.com/adperezmorales/stanbol-camel-workflow/
On Tue, May 27, 2014 at 6:18 PM, Antonio David Perez Morales <
ape...@zaizi.com> wrote:
Hi people
I have already started to work on [1] to integrate current Florent's
code into Stanbol 1.0.
As a first approach, only changing the dependency versions to new
Stanbol 1.0, many issues have arisen:
- Deprecated use of classes
- Classes which have changed from package
- Some classes not necessary now
- Classes not used which were causing conflicts
- ...
So now I'm trying to resolve all these problems to replicate the same
behavior from 0.9 into 1.0. I will upload the code to a Github repository
in my account (which will be pushed later into a Stanbol branch after the
project) in order to track the advances.
Once I can resolve all these problems, I will take a look to the
Felix Custom Artifacts poiinted out by Rupert in a previous message to find
out the best way to deploy (and manage) route configurations (felix
artifacts, watchservice java, rest endpoint to receive xml routes, etc).
Comments on this and future tasks are more than welcome.
Regards
[1] https://issues.apache.org/jira/browse/STANBOL-1347
On Tue, May 27, 2014 at 9:53 AM, Rafa Haro <rh...@apache.org> wrote:
Hi Rupert, Florent and Antonio
El 27/05/14 08:51, Rupert Westenthaler escribió:
As the result of Enhancement Routes is content + metadata I can not
see what you want to "store" in the Entityhub that is about managing
Entities.
> - entityhub: To query/update the entityhub component
Maybe. If you can come up with a good use case ^^
> - contenthub: To develop a new content-hub using chain/engine
components
and solr/elasticsearch/whatever component (solr and elasticsearch
component
already exist in Camel)
IMO implementing a new Contenthub like component is outside the
scope
of this GSoC project. However If there is already Solr/Elasticsearch
component it would be a really useful thing
Regarding this, in my opinion, the use case of an eventual
integration with a Content hub is probably one of the most clear for this
project. I'm not sure if that is what Antonio was trying to explain but,
with a single route using as last endpoint Solr or any other backend
system, we would be almost cloning the same functionality than the previous
ContentHub implementation (Stanbol 0.12). Entities could be dereferenced
using the EntityHub before storing the content along with the metadata,
which is the point of integration of the EntityHub in such use case. And
even most interesting, now with the integration of Marmotta contributed by
Rupert, it would be possible to use a whole graph for dereferencing, so
"simply" routing components like Enhancer->Marmotta->Solr sounds to me like
an interesting use case.
wdyt?
Cheers,
Rafa
