RAID blockfixer can use a heuristic to find unfixable files
------------------------------------------------------------
Key: MAPREDUCE-2352
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2352
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: contrib/raid
Reporter: Ramkumar Vadali
Assignee: Ramkumar Vadali
Priority: Minor
It is possible to have corrupt files that were never RAIDed. In such a case,
there is no use in trying to submit a block fixer job for that file. The RAID
code has the function filterUnfixableSourceFiles() that checks for the presence
of parity files for each source file. This is too expensive, since a lot of the
parity files can be HARed. Instead, we can use a heuristic where we just check
for the presence of the parent directory in the parity space. If the parent
directory is absent, the parity file cannot be present, and the source file
would be unfixable.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
