[
https://issues.apache.org/jira/browse/HTRACE-18?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245094#comment-14245094
]
Long Zhou commented on HTRACE-18:
---------------------------------
Hi Stack,
htrace-flume can be useful in the following scenarios:
1) When we need to collect trace data from multiple remote data centers and
store the data to a local data center for analysis, it would be inefficient and
difficult to manage if every server opens connections and writes directly to
the storage sink.
A better solution is to deploy a Flume agent at each data center which
aggregate and forward data to the Flume agent at the storage data center. Since
every server only talks to the Flume agent, the solution reduces the number of
connections and writes. Flume also serves as a buffer to mitigate the impact of
latency and outage.
2) Currently htrace only support a few storage options (HBase, LevelDB), and
users may want to use a different storage. In my case, I would like to use our
existing MongoDB.
With htrace-flume, user can setup a flume gateway and forward the trace data to
any sink that flume supports. The htrace project does not need to support these
sinks directly.
3) Flume agent can filter the data and redirect/drop as configured, and the
filter configuration can be modified dynamically. For example, if a few servers
in a data center are having issues, we can change Flume filter to only collect
data from these servers and drop all other data. There are usually only a few
Flume servers in a large data center, and it is easier and safer to change
Flume servers than to change Hadoop and other servers.
Please let me know if you have any more questions.
Thanks,
Long
> Support flume receiver
> ----------------------
>
> Key: HTRACE-18
> URL: https://issues.apache.org/jira/browse/HTRACE-18
> Project: HTrace
> Issue Type: Improvement
> Reporter: Long Zhou
> Attachments: htrace-flume01.patch
>
>
> Hi htrace devs,
> I have been using htrace for a while and find it very useful.
> I needed a way to collect traces from remote servers via flume, so I
> implemented the flume receiver (patch attached). If this code is useful to
> other users, I would like to contribute it to the project.
> Please kindly review the patch, and let me know if anything I should
> fix/improve.
> Thanks,
> Long Zhou
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
