[
https://issues.apache.org/jira/browse/CHUKWA-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783514#action_12783514
]
Eric Yang commented on CHUKWA-102:
----------------------------------
Resurrecting this jira. The components to make this feature useful are:
Streaming adaptor - Listen to a port, and receive chunks, and send directly to
collector, spill to disk if it could not send the chunk.
Streaming appender - Chunk creator which can configure to send to an agent or
an collector. There is no fail safe buffering in the appender.
Streaming appender could send to collector directly for small clusters to avoid
agent setup. For large cluster, it should be configured to send to agent then
collector. These utilities support dag configuration and avoid tcp incast
problem. Streaming appender should communicate to streaming adaptor through
local loopback when they are deployed on the same system. This provides best
effort log delivery system with minimum effort for implementation.
> Provide streaming adapter
> -------------------------
>
> Key: CHUKWA-102
> URL: https://issues.apache.org/jira/browse/CHUKWA-102
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: input tools
> Reporter: Mac Yang
>
> Enable applications to send data directly to the agent without writing to
> disk first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
