I originally posted this question on StackOverflow, but I see that this group might be a more suitable place for it.
We are setting up logs from several related applications so the log events are imported into Elasticsearch (via Logstash). It was straightforward create Kibana dashboards to visualize log indexes for each application, but since the applications are related and its activities belong to the same pipeline, it would be great to build a dashboard that would show aggregated information, collected from different applications. Such dashboard would be especially useful to track failures and performance problems. Right now I can see three main ways to implement aggregated dashboard: 1. Keep separate application logs and configure Kibana dashboard that would consume information from different applications. I am afraid this can be a challenging task, I am not even sure Kibana fully supports it. 2. Revise application logging so they will all log to the same index. What I dislike about this is that log event structure must be then unified across applications, and they are built by different people in different languages. I've lost my faith to centralized control over such low level details like logging. 3. Keep applications log and corresponding Elastichsearch indexes as they are now, but set up a new index which will contain aggregate information. This article <http://www.elasticsearch.org/blog/logging-elasticsearch-events-with-logstash-and-elasticsearch/> describes how to configure Elasticsearch to dump it’s logs to Logstash which would then insert them back into Elasticsearch for searching. At first glance this approach may look surprising: why would you need to re-insert log data once again into the same database? It's another index, it adds overhead, uses more space etc. But it gives the opportunity to set up the index in a way that will be suitable for a aggregated Kibana dashboard. I wonder if someone has gone through a similar dilemma and can share their experience. Thanks in advance Vagif Abilov -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ac164ad2-ea7f-4b00-a9af-fc6e819949e8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
