ntoine.t...@thales-services.fr>
> *Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
> *Date: *Friday, March 31, 2017 at 9:58 AM
>
> *To: *"user@storm.apache.org" <user@storm.apache.org>
> *Subject: *Re: Centralized logging for storm
>
>
7 at 9:58 AM
To: "user@storm.apache.org" <user@storm.apache.org>
Subject: Re: Centralized logging for storm
How about modifying the worker.xml configuration, so that we add a appender to
logstash/elasticsearch ? No need to add a filebeat if this is handled by Storm
itself.
On 3
to have a richer search/filtering.
Cody
*From: *Harsh Choudhary <shry.ha...@gmail.com>
*Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
*Date: *Friday, March 31, 2017 at 4:38 AM
*To: *"user@storm.apache.org" <user@storm.apache.org>
*Subject:
"user@storm.apache.org" <user@storm.apache.org>
Subject: Re: Centralized logging for storm
Hi Shashank
What we do is, we have filebeats installed on our Storm clusters and they send
the log files data to our central log server, Graylog. This tool is great and
you can see your log
Hi Shashank
What we do is, we have filebeats installed on our Storm clusters and they
send the log files data to our central log server, Graylog. This tool is
great and you can see your logs like they are one stream of messages,
sorted by timestamp. One thing that really helps is that you can
Hello Shashank,
I understand this won't help with merging all the log outputs into one
single file, but you can configure your Storm workers to write logs to an
NFS mounted to each of the workers (that's what we do) and than access them
all at that one place with whichever tool you prefer.
Hi Shashank,
previously I used Fluentd with some plugins to tail log lines, parse,
transform, and store to ES. I think you can do it similar with Logstash.
There're some frameworks like Apache NiFi to help ETL and I guess it can
achieve what you'd like to do.
Thanks,
Jungtaek Lim (HeartSaVioR)
Hi folks,
Storm is a great tool but the logs are all over the place. As you increase
your workers, your log files will increase as well and there is no single
file it logs to.
This makes it very hard to troubleshoot since you have to tail multiple
logs.
Ideally, i would like to ship all the
