Hi guys,
sorry, I'm very late on this due to some customers engagements.
Tonight (I'm still in Australia up to the end of this week), I'm working
on the next Karaf releases.
When back to France (this week end), I will move forward on the donation
of decanter (the git repo is ready, I will do the cleanup and legal
during the week end).
Sorry again for the delay.
Regards
JB
On 11/15/2014 10:08 AM, Andreas Pieber wrote:
Hey JB,
Thank you very much for the update! I'm already thrilled about that one :-)
Kind regards,
Andreas
On Fri Nov 14 2014 at 7:17:04 PM Achim Nierbeck <[email protected]>
wrote:
Hi JB,
awesome. Thanks for the feedback :-)
regards, Achim
2014-11-14 16:40 GMT+01:00 Jean-Baptiste Onofré <[email protected]>:
Hi guys,
just a quick update about Karaf Decanter.
The INFRA created the git repository. I will cleanup the legal files, and
add the latest features. I will push to karaf-decanter later today.
Regards
JB
On 10/19/2014 09:18 PM, Jean-Baptiste Onofré wrote:
Hi all,
this vote passed with only +1.
I will push my latest changes on the github, request the git repo to
INFRA (to push there and remove the github one), and create a component
in Jira.
Thanks all for your vote.
Regards
JB
On 10/14/2014 05:12 PM, Jean-Baptiste Onofré wrote:
Hi all,
First of all, sorry for this long e-mail ;)
Some weeks ago, I blogged about the usage of ELK
(Logstash/Elasticsearch/Kibana) with Karaf, Camel, ActiveMQ, etc to
provide a monitoring dashboard (know what's happen in Karaf and be able
to store it for a long period):
http://blog.nanthrax.net/2014/03/apache-karaf-cellar-camel-
activemq-monitoring-with-elk-elasticsearch-logstash-and-kibana/
If this solution works fine, there are some drawbacks:
- it requires additional middlewares on the machines. Additionally to
Karaf itself, we have to install logstash, elasticsearch nodes, and
kibana console
- it's not usable "out of the box": you need at least to configure
logstash (with the different input/output plugins), kibana (to create
the dashboard that you need)
- it doesn't cover all the monitoring needs, especially in term of SLA:
we want to be able to raise some alerts depending of some events (for
instance, when a regex is match in the log messages, when a feature is
uninstalled, when a JMX metric is greater than a given value, etc)
Actually, Karaf (and related projects) already provides most (all) data
required for the monitoring. However, it would be very helpful to have
a
"glue", ready to use and more user friendly, including a storage of the
metrics/monitoring data.
Regarding this, I started a prototype of a monitoring solution for
Karaf
and the applications running in Karaf.
The purpose is to be very extendible, flexible, easy to install and
use.
In term of architecture, we can find the following component:
1/ Collectors & SLA Policies
The collectors are services responsible of harvesting monitoring data.
We have two kinds of collectors:
- the polling collectors are invoked by a scheduler periodically.
- the event driven collectors react to some events.
Two collectors are already available:
- the JMX collector is a polling collector which harvest all MBeans
attributes
- the Log collector is a event driven collector, implementing a
PaxAppender which react when a log message occurs
We can planned the following collectors:
- a Camel Tracer collector would be an event driven collector, acting
as
a Camel Interceptor. It would allow to trace any Exchange in Camel.
It's very dynamic (thanks to OSGi services), so it's possible to add a
new custom collector (user/custom implementation).
The Collectors are also responsible of checking the SLA. As the SLA
policies are tight to the collected data, it makes sense that the
collector validates the SLA and call/delegate the alert to SLA
services.
2/ Scheduler
The scheduler service is responsible to call the Polling Collectors,
gather the harvested data, and delegate to the dispatcher.
We already have a simple scheduler (just a thread), but we can plan a
quartz scheduler (for advanced cron/trigger configuration), and another
one leveraging the Karaf scheduler.
3/ Dispatcher
The dispatcher is called by the scheduler or the event driven
collectors
to dispatch the collected data to the appenders.
4/ Appenders
The appender services are responsible to send/store the collected data
to target systems.
For now, we have two appenders:
- a log appender which just log the collected data
- a elasticsearch appender which send the collected data to a
elasticsearch instance. For now, it uses "external" elasticsearch, but
I'm working on an elasticsearch feature allowing to embed elasticsearch
in Karaf (it's mostly done).
We can plan the following other appenders:
- redis to send the collected data in Redis messaging system
- jdbc to store the collected data in a database
- jms to send the collected data to a JMS broker (like ActiveMQ)
- camel to send the collected data to a Camel direct-vm/vm endpoint of
a
route (it would create an internal route)
5/ Console/Kibana
The console is composed by two parts:
- a angularjs or bootstrap layer allowing to configure the SLA and
global settings
- embedded kibana instance with pre-configured dashboard (when the
elasticsearch appender is used). We will have a set of already created
lucene queries and a kind of "Karaf/Camel/ActiveMQ/CXF" dashboard
template. The kibana instance will be embedded in Karaf (not external).
Of course, we have ready to use features, allowing to very easily
install modules that we want.
I named the prototype Karaf Decanter. I don't have preference about the
name, and the location of the code (it could be as Karaf subproject
like
Cellar or Cave, or directly in the Karaf codebase).
Thoughts ?
Regards
JB
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>
Software Architect / Project Manager / Scrum Master
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
