IFile.Reader should check whether data crc has checked before it stop reading.
------------------------------------------------------------------------------
Key: MAPREDUCE-1821
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1821
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: task
Reporter: ZhuGuanyin
Assignee: ZhuGuanyin
Currently IFile data has crc checked in IFileInputStream (doRead method),
Normally the IFile would end with 2 bytes of -1, which means EOF_MARKER for
keylength and valuelength, and then with 4 bytes crc checksum;
IFileInputStream checksumIn would check crc before IFile.Reader get
EOF_MARKER,
IFile.Reader would stop reading when positionToNextRecord() read keylength
EOF_MARKER(-1),and valuelength EOF_MARKER(-1);
But if something error happened(IFile corrupted), if the IFileReader read -1,
-1 not at end of the IFile, the data may not checked!
Then Reader thought it had got all data and close reader......the task may fake
success without any WARNing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
