[
https://issues.apache.org/jira/browse/CHUKWA-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14072541#comment-14072541
]
Eric Yang commented on CHUKWA-707:
----------------------------------
Kafka topic is designed to have one consumer per topic. Kafka topics should be
partitioned by host + data type. Otherwise, n consumers are running inside n
Chukwa agents, and causes the same message to be consumed n times and send to
writer n times.
> Replace Chukwa collector with Apache Kafka
> ------------------------------------------
>
> Key: CHUKWA-707
> URL: https://issues.apache.org/jira/browse/CHUKWA-707
> Project: Chukwa
> Issue Type: New Feature
> Reporter: Eric Yang
> Assignee: shreyas subramanya
> Attachments: CHUKWA-707.patch1
>
>
> Chukwa collector has stopped evolving since 2010. Newer framework has offer
> better features of message queues, and Apache Kafka looks like a good
> replacement for Chukwa collector.
> Chukwa agent can implement a connector to Apache Kafka to replace Chukwa
> collector, and HBase consumer to write data to HBase. HICC REST API change
> to new HBase storage format.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
