Real time hadoop activity monitoring capabilities via Client Trace Logs
-----------------------------------------------------------------------
Key: CHUKWA-441
URL: https://issues.apache.org/jira/browse/CHUKWA-441
Project: Hadoop Chukwa
Issue Type: New Feature
Components: data collection, User Interface
Environment: Redhat EL 5.1, Java 6
Reporter: Eric Yang
Assignee: Eric Yang
Fix For: 0.4.0
Many moons ago, in one of the HUG talk for Chukwa, one user ask if it is
possible to create a splitter interface for Chukwa Collector for monitoring
data at near real time. The answer was possible, but there was no code to show
this capabilities. Over the Xmas holidays, I decided to take on this task and
write something simple to show how this is done.
The idea is to reuse existing Chukwa Collector with PipelineStageWriter coupled
with SocketTeeWriter for data stream split. By adding a SocketDataLoader
inside HICC to act as data stream subscriber and extract client trace
information to render on the browser. Hence, the additional pipeline looks
like this:
Adaptor -> Agent -> Collector -> SocketTeeWriter <- Socket Data Loader <- HICC
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
