So, what would be an effective solution, reopen the FileSystem to rename the file from *.chukwa to *.done?
On Fri, Mar 26, 2010 at 6:38 PM, Jerome Boulon (JIRA) <[email protected]>wrote: > > [ > https://issues.apache.org/jira/browse/CHUKWA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12850206#action_12850206] > > Jerome Boulon commented on CHUKWA-4: > ------------------------------------ > > This is because HDFS is closing the fileSystem on kill -term > (shutdownHook). > The workaround is to remove the HDFS shurdown hook and call it after the > collector close but it's just a workaround. > > > Collectors don't finish writing .done datasink from last .chukwa datasink > when stopped using bin/stop-collectors > > > ---------------------------------------------------------------------------------------------------------------- > > > > Key: CHUKWA-4 > > URL: https://issues.apache.org/jira/browse/CHUKWA-4 > > Project: Hadoop Chukwa > > Issue Type: Bug > > Components: data collection > > Environment: I am running on our local cluster. This is a linux > machine that I also run Hadoop cluster from. > > Reporter: Andy Konwinski > > Priority: Minor > > > > When I use start-collectors, it creates the datasink as expected, writes > to it as per normal, i.e. writes to the .chukwa file, and roll overs work > fine when it renames the .chukwa file to .done. However, when I use > bin/stop-collectors to shut down the running collector it leaves a .chukwa > file in the HDFS file system. Not sure if this is a valid sink or not, but I > think that the collector should gracefully clean up the datasink and rename > it .done before exiting. > > -- > This message is automatically generated by JIRA. > - > You can reply to this email to add a comment to the issue online. > > -- Ahmed Fathalla
