[
https://issues.apache.org/jira/browse/FLUME-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ray updated FLUME-2614:
-----------------------
Description:
cannot create review, the website seems not work, https://reviews.apache.org
add new feature to create done file for HDFS Sink to denote a folder transfer
complete for integration with oozie.
I attached the patch.
it need following params, the reason using timer is to create doneFlag for
latest folder when long time no new file coming.
hdfs.doneFlag=done (default disabled)
hdfs.useLocalTimeStamp=true (must use local time to enable doneFlag)
hdfs.checkFolderDoneTimerInterval=1 (timer interval(s) to check if create
doneFlag)
was:
add new feature to create done file for HDFS Sink to denote a folder transfer
complete for integration with oozie.
I attached the patch.
it need following params, the reason using timer is to create doneFlag for
latest folder when long time no new file coming.
hdfs.doneFlag=done (default disabled)
hdfs.useLocalTimeStamp=true (must use local time to enable doneFlag)
hdfs.checkFolderDoneTimerInterval=1 (timer interval(s) to check if create
doneFlag)
> add done-flag to HDFS Sink for integration with oozie
> -----------------------------------------------------
>
> Key: FLUME-2614
> URL: https://issues.apache.org/jira/browse/FLUME-2614
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Affects Versions: v1.6.0
> Reporter: ray
> Attachments: FLUME-2614.patch
>
> Original Estimate: 10h
> Remaining Estimate: 10h
>
> cannot create review, the website seems not work, https://reviews.apache.org
> add new feature to create done file for HDFS Sink to denote a folder transfer
> complete for integration with oozie.
> I attached the patch.
> it need following params, the reason using timer is to create doneFlag for
> latest folder when long time no new file coming.
> hdfs.doneFlag=done (default disabled)
> hdfs.useLocalTimeStamp=true (must use local time to enable doneFlag)
> hdfs.checkFolderDoneTimerInterval=1 (timer interval(s) to check if create
> doneFlag)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
