[
https://issues.apache.org/jira/browse/CHUKWA-678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang resolved CHUKWA-678.
------------------------------
Resolution: Implemented
Fixed upstream.
> Make use of ChukwaWriter in agent
> ---------------------------------
>
> Key: CHUKWA-678
> URL: https://issues.apache.org/jira/browse/CHUKWA-678
> Project: Chukwa
> Issue Type: Sub-task
> Components: Data Collection
> Environment: MacOSX, Java 6
> Reporter: shreyas subramanya
> Assignee: shreyas subramanya
>
> The chukwa agent sends out data chunks to various destinations through the
> combination of Connector and ChukwaSender interfaces. For sending chunks to
> collector, we have http implementation of these interfaces. The collector
> writes out the received chunks to various destinations through classes
> implementing ChukwaWriter interface. Optionally, multiple destinations can be
> chosen by specifying PipelineStageWriter.
> The proposal is to:
> 1. Use ChukwaWriter to send out data chunks to multiple destinations from the
> agent. Further, PipelinestageWriter can be made default and pipeline
> configuration specified in the agent config file
> 2. Implement (or modify) Pipelineable writers for HBase, Http, Hdfs and
> WebHdfs
> 3. Do away with the Connector interface and have a single non configurable
> connector object as part of the agent. This class initiates the configured
> writer, waits for data chunks and passes the chunks to Writer.add()/send().
> Connection protocol for each destination is handled by the init() of the
> individual writers.
> Considerations:
> 1. In case of Pipelineable writers, we need a way to merge the results of
> each pipeline stage before the agent commits the chunk.
> 2. Handling pipeline failure
--
This message was sent by Atlassian JIRA
(v6.2#6252)
