We have seen this behavior too On Wed, Mar 25, 2009 at 5:56 AM, Jingkei Ly (JIRA) <[email protected]> wrote:
> TupleWritable can return incorrect results if it contains more than 32 > values > > ----------------------------------------------------------------------------- > > Key: HADOOP-5571 > URL: https://issues.apache.org/jira/browse/HADOOP-5571 > Project: Hadoop Core > Issue Type: Bug > Components: mapred > Affects Versions: 0.19.1 > Reporter: Jingkei Ly > > > When attempting to do an outer join on 45 files with the > CompositeInputFormat, I've been encountering unexpected results in the > TupleWritable returned by the record reader. On closer inspection, it seems > to be because TupleWritable.setWritten(int) is incorrectly setting some > tuple positions as written, i.e when you set setWritten(42), it also sets > position 10. > > The following Junit test demonstrates the problem: > {code} > public void testWideTuple() throws Exception { > Text emptyText = new Text("Should be empty"); > Writable[] values = new Writable[64]; > Arrays.fill(values,emptyText); > values[42] = new Text("Number 42"); > > TupleWritable tuple = new TupleWritable(values); > tuple.setWritten(42); > > for (int pos=0; pos<tuple.size();pos++) { > boolean has = tuple.has(pos); > if (pos == 42) { > assertTrue(has); > } > else { > assertFalse("Tuple position is incorrectly labelled as set: " + pos, > has); > } > } > } > {code} > > Similarly, TupleWritable.setWritten(9) also causes TupleWritable.has(41) to > incorrectly return true. > > > -- > This message is automatically generated by JIRA. > - > You can reply to this email to add a comment to the issue online. > > -- Alpha Chapters of my book on Hadoop are available http://www.apress.com/book/view/9781430219422
