Brock Noland created FLUME-1702:
-----------------------------------
Summary: HDFSEventSink should write to a hidden file as opposed to
a .tmp file
Key: FLUME-1702
URL: https://issues.apache.org/jira/browse/FLUME-1702
Project: Flume
Issue Type: Improvement
Reporter: Brock Noland
Currently we write to a .tmp file. The problem is that if MR jobs are being run
on the directory we are writing to, then it's common for an MR job to list the
directory, get a .tmp file and then in the mean time the .tmp file is renamed
causing the job to fail when run.
Using JavaMR you can use a PathFilter to avoid this, however a custom solution
is required for Pig, Hive, etc.
Perhaps we should write to a hidden file so that MR never tries to process data
in flight.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
