[
https://issues.apache.org/jira/browse/EAGLE-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Chen updated EAGLE-20:
--------------------------
Description:
Currently we provide two approaches for user to ship hadoop audit log from
local to kafka, while they have some problems for large log flow
1) log4j kafka appender: impact hadoop name node stability
2) logstash: consume too many machine resource like CPU/Memory
So we may need to evaluate some other possible solutions like Elastic Filebeat:
https://github.com/elastic/filebeat
was:
Currently we provide two approaches for user to ship hadoop audit log from
local to kafka, while they have some problems for large log flow
1) eagle log4j appender: impact hadoop name node stability
2) logstash: consume too many machine resource like CPU/Memory
So we may need to evaluate some other possible solutions like Elastic Filebeat:
https://github.com/elastic/filebeat
> Support Filebeat as Log Shipper for better performance
> ------------------------------------------------------
>
> Key: EAGLE-20
> URL: https://issues.apache.org/jira/browse/EAGLE-20
> Project: Eagle
> Issue Type: New Feature
> Reporter: Hao Chen
>
> Currently we provide two approaches for user to ship hadoop audit log from
> local to kafka, while they have some problems for large log flow
> 1) log4j kafka appender: impact hadoop name node stability
> 2) logstash: consume too many machine resource like CPU/Memory
> So we may need to evaluate some other possible solutions like Elastic
> Filebeat: https://github.com/elastic/filebeat
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
