Hi Isuru, using ES+Kibana is a very good idea. We also are on the way to provide some kibana dashboard to the devops of one of our client. Why would you need some embedded ElasticSearch ? is it a security issue ?
For the architecture list, the elastic suite is very nice and could provide a nice solution to provide rich dashboards. It would be very simple to target ES as a stream store to benefit from kibana time-series dashboard. We are thinking of using the complete ELK stack (adding logtash as well) to provide easy admin stats dashboard to multiple wso2 product, gaining the fact is that it can also aggregate data from other sources than wos2 stat (using logtash to gather other logs). The apache licence of the whole products is something that makes it possible. Cheers, Cyril > ---------- Forwarded message ---------- > From: Isuru Perera <[email protected]> > Date: Mon, Jan 12, 2015 at 2:18 PM > Subject: [Architecture] Metrics Visualization: Using Kibana & Elasticsearch > To: Srinath Perera <[email protected]>, WSO2 Architecture < > [email protected]> > > > Hi all, > > As part of Metrics [1] integration with WSO2 Platform [2], we are planning > to have a separate UI for Metrics visualization. > > One of the main requirements is that the UI application should be > standalone.There are Metrics reporters [3] for Ganglia & Graphite, which > require to setup separate servers. > > Since we need a UI application, I looked at current work on such > visualization applications. > > I mainly looked at Kibana [4] & Grafana [5]. Both applications are based > on HTML, JavaScript, etc. I could run both in WSO2 Application Server > without any issue. > > I chose Kibana, as it supports Elasticsearch and we can run Elasticsearch > as an embedded server [6]. Elasticsearch also provides a reporter for > Metrics [7]. > > Grafana also looks good and as I found out, we need to write a custom data > source to provide the Metrics data. > > Using mentioned software, I was able to run Kibana in a standalone > application and visualize some sample Metrics. > > This was just a prototype and I will create a proper web application. With > Kibana, we can easily create dashboards depending on our requirements. I'm > going down this path, as we can quickly create dashboards and identify > exact visualizations we need. Since we are in the initial stages of Metrics > integration, we have to see how we can use the data provided by Metrics. > > With Metrics Reporters, each metric will be reported with the configured > time period and there will be general statistical information, such as min, > max, mean, percentiles etc for Timers. (See JSON format for metrics in > README of [7]). We also need to figure out how we can visualize these data. > > I think we can use Kibana initially for our visualization purposes. All of > the mentioned software and dependencies are based on Apache License. > > Only concern here would be the use of an embedded Elasticsearch server. > IMO, Elasticsearch provides good features such as indexing data and smooth > integration with Kibana. > > As I mentioned earlier, I would like to think of this as a temporary > approach and we can create a separate dashboard once we have solid > requirements. There is another work going on to create custom dashboards > [8] and we can also make use of that later. > > We have not started the Metrics integration with products yet and we might > have more concerns when we start to integrate. > > Please share your suggestions. > > Thanks! > > Best Regards, > > [1] https://dropwizard.github.io/metrics/ > [2] http://markmail.org/message/akqyreopmxhcd6hx > [3] > https://dropwizard.github.io/metrics/3.1.0/manual/core/#other-reporters > [4] http://www.elasticsearch.org/overview/kibana/ > [5] http://grafana.org/ > [6] > http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html > [7] https://github.com/elasticsearch/elasticsearch-metrics-reporter-java > [8] http://markmail.org/message/74ogpnup5uqfqbps > > > -- > Isuru Perera > Senior Software Engineer | WSO2, Inc. | http://wso2.com/ > Lean . Enterprise . Middleware > > about.me/chrishantha > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
