[
https://issues.apache.org/jira/browse/CHUKWA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853475#action_12853475
]
Jerome Boulon commented on CHUKWA-4:
------------------------------------
the exception will be on this line "while (seqFileReader.next(key, evt)){
seqFileWriter.append(key, evt); } "
so you can add a 1st try/catch block around this line, then add another one
around rename/delete.
Also, don't delete the .chukwa file in the first place.
Steps:
1- Rename .recover to .recoverDone
2- delete .chukwa
3- rename .recoverDone to .done
At startup time, you should:
1- delete any .recover files
2.0- if you have a .recoverDone and .chukwa with the same name, then delete the
.chukwa file (Repeat until there's no more .chukwa file that matches this rule)
2.1- Rename all .recoverDone to .done
3- process .chukwa files
4- send .done files
> Collectors don't finish writing .done datasink from last .chukwa datasink
> when stopped using bin/stop-collectors
> ----------------------------------------------------------------------------------------------------------------
>
> Key: CHUKWA-4
> URL: https://issues.apache.org/jira/browse/CHUKWA-4
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection
> Environment: I am running on our local cluster. This is a linux
> machine that I also run Hadoop cluster from.
> Reporter: Andy Konwinski
> Priority: Minor
>
> When I use start-collectors, it creates the datasink as expected, writes to
> it as per normal, i.e. writes to the .chukwa file, and roll overs work fine
> when it renames the .chukwa file to .done. However, when I use
> bin/stop-collectors to shut down the running collector it leaves a .chukwa
> file in the HDFS file system. Not sure if this is a valid sink or not, but I
> think that the collector should gracefully clean up the datasink and rename
> it .done before exiting.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
