[
https://issues.apache.org/jira/browse/HADOOP-6834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881404#action_12881404
]
Hong Tang commented on HADOOP-6834:
-----------------------------------
You should always submit the patch for trunk. If you are using older releases,
you may also backport the patch (attach here) and request it to be committed to
older branches too.
> TFile.append compares initial key against null lastKey
> --------------------------------------------------------
>
> Key: HADOOP-6834
> URL: https://issues.apache.org/jira/browse/HADOOP-6834
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.20.1, 0.20.2
> Reporter: Ahad Rana
>
> The following code in TFile.KeyReigster.close:
> byte[] lastKey = lastKeyBufferOS.getBuffer();
> int lastLen = lastKeyBufferOS.size();
> if (tfileMeta.getComparator().compare(key, 0, len, lastKey, 0,
> lastLen) < 0) {
> throw new IOException("Keys are not added in sorted order");
> }
> compares the initial key (passed in via TFile.Writer.append) against a
> technically NULL lastKey. lastKey is not initialized until after the first
> call to TFile.Writer.append. The underlying RawComparator interface used for
> comparisons does not stipulate the proper behavior when either length 1 or
> length 2 is zero. In the case of LongWritable, its WritableComparator
> implementation does an unsafe read on the passed in byte arrays b1 and b2.
> Since TFile pre-allocates the buffer used for storing lastKey, this passes a
> valid buffer with zero count to LongWritable's comparator, which ignores
> length and thus produces incorrect results.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
