[
https://issues.apache.org/jira/browse/CHUKWA-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13527011#comment-13527011
]
shreyas subramanya commented on CHUKWA-674:
-------------------------------------------
I wonder if we should do away with the "Connector" interface and just have a
single connector class. Each class implementing this interface just duplicates
this code - loop on fetching from the EventQueue and call ChukwaSender.send().
Further, each addition of a new Sender (Ex. HBase sender) would require
addition of a new connector and duplicating code.
When we integrate collector into the agent, we would just need to add Senders
for sending through http, writing to HBase, HDFS or console and a pipeline
sender. The agent configuration would then have "agent.sender" property to
choose a sender and an "agent.pipeline" property if a pipeline sender is chosen.
> Integrate Chukwa collector feature to Chukwa agent
> --------------------------------------------------
>
> Key: CHUKWA-674
> URL: https://issues.apache.org/jira/browse/CHUKWA-674
> Project: Chukwa
> Issue Type: Improvement
> Components: Data Collection
> Environment: MacOSX, Java 6
> Reporter: Eric Yang
>
> Feature offered in Chukwa collector can be integrated into Chukwa agent, and
> use multi-tier Chukwa agent to collect data for large scale cluster. For
> small cluster, agents can talk directly to HDFS cluster to reduce the
> complexity of deployment. The required features to reduce the need of Chukwa
> collectors are:
> - Enhance agent rest api to receive chunk data.
> - Pipeline writer to channel data to storage destinations (HDFS, HBASE).
> - Improve connector interface and replace http connector with collector
> connector for bandwidth balance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
