[
https://issues.apache.org/jira/browse/MAPREDUCE-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872118#action_12872118
]
ZhuGuanyin commented on MAPREDUCE-1821:
---------------------------------------
positionToNextRecord() in IFile.Reader should check whether checksumIn had crc
checked when reader get EOF_MARKER for currentKeyLength and currentValueLength.
if IFileInputStream had crc checked in doRead() , it should set a class private
variable flag, and the Reader could check througth a IFileInputStream public
method to query this variable.
if IFile.Reader get 2 byrtes -1 for keylength and valuelength and crc not
checked in checksumIn, it would throw a exception to fail this task.
> IFile.Reader should check whether data crc has checked before it stop reading.
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1821
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1821
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
> Reporter: ZhuGuanyin
> Assignee: ZhuGuanyin
>
> Currently IFile data has crc checked in IFileInputStream (doRead method),
> Normally the IFile would end with 2 bytes of -1, which means EOF_MARKER for
> keylength and valuelength, and then with 4 bytes crc checksum;
> IFileInputStream checksumIn would check crc before IFile.Reader get
> EOF_MARKER,
> IFile.Reader would stop reading when positionToNextRecord() read keylength
> EOF_MARKER(-1),and valuelength EOF_MARKER(-1);
> But if something error happened(IFile corrupted), if the IFileReader read -1,
> -1 not at end of the IFile, the data may not checked!
> Then Reader thought it had got all data and close reader......the task may
> fake success without any WARNing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
