[
https://issues.apache.org/jira/browse/CHUKWA-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13554689#comment-13554689
]
shreyas subramanya commented on CHUKWA-679:
-------------------------------------------
The idea is to append to text files (named based on chunk metadata) through
webhdfs. The append requires 2 post operations - the latter being a url to
datanode. Since we expect small updates to the file (like data from log4j
appender), we want to cache the redirect url and keep posting to the same
datanode until the configured "rotate interval" (like 1 hour). Then we create a
new file, get a new datanode url to post to and continue.
> Create WebHDFSWriter
> --------------------
>
> Key: CHUKWA-679
> URL: https://issues.apache.org/jira/browse/CHUKWA-679
> Project: Chukwa
> Issue Type: Sub-task
> Components: Data Processors
> Affects Versions: 0.5.0, 0.6.0
> Environment: Java 6
> Reporter: shreyas subramanya
> Original Estimate: 10h
> Remaining Estimate: 10h
>
> Create a pipelineable writer which uses WebHDFS REST api.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
