Hari Shreedharan created FLUME-2648:
---------------------------------------
Summary: File Channel restarts can fail if metadata file creation
fails during file roll
Key: FLUME-2648
URL: https://issues.apache.org/jira/browse/FLUME-2648
Project: Flume
Issue Type: Bug
Reporter: Hari Shreedharan
Assignee: Hari Shreedharan
If during the creation of a new File in the data dirs, the creation of the
metadata file fails - then the log-n file is simply left as an orphan. During
replay though, this causes issues - as we try to read the metadata file at this
time, causing the replay to fail and the file channel being unable to restart.
To fix this we must remove the data file if the creation of the metadata file
fails. Or we could simply ignore files whose metadata files don't exist (though
in this case, user error could cause data loss). If the delete fails, then
there would a problem though. So if the metafile is missing during replay, we
must still fail - though more gracefully, and ask the operator to remove the
corresponding data file. It might actually be possible to figure out if the
data file has any content at all - by checking if size == 0 (need to verify
this) - in that case we can probably remove/ignore that file.
I will submit a patch for this soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
