[
https://issues.apache.org/jira/browse/CHUKWA-670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508109#comment-13508109
]
Eric Yang commented on CHUKWA-670:
----------------------------------
ZooKeeper has some nice properties that can be used for coordination. The
current design requires to use puppet or chef to make changes to collector
list, and issue refresh collector list command. It would be nice to have
ZooKeeper watcher node for agent to get notified and reshuffle the list,
without the 2 step process.
When a collector is started, it automatically register itself into the
ZooKeeper list as an ephemeral node. Agent who is watching the list, get
notified automatically, and will take advantage of the new list. The
centralized list helps to refresh the collector list more easily.
Second use case, ZooKeeper configuration will enable possibilities to manages
pipeline writer in collector to channel data to different destination.
> Use ZooKeeper for collector list and adaptor check list
> -------------------------------------------------------
>
> Key: CHUKWA-670
> URL: https://issues.apache.org/jira/browse/CHUKWA-670
> Project: Chukwa
> Issue Type: New Feature
> Components: Data Collection
> Environment: Java, ZooKeeper, Chukwa
> Reporter: Eric Yang
>
> Agent is currently using local disk to storage check point file, and
> collector list. This can be hard to manage for large scale system that needs
> to be updated more frequently. The proposal is to have a new configuration
> in chukwa agent configuration file to enable ZooKeeper as the location to
> store check point file and collector list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
