[
https://issues.apache.org/jira/browse/CASSANDRA-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacek Gerbszt updated CASSANDRA-2955:
-------------------------------------
Description:
Accessing CounterColumn from Hadoop ends up with an exception:
{noformat}
java.lang.NullPointerException
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown
Source)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
at
org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
{noformat}
If you look into ColumnFamilyRecordRerader, you will quickly realize that it
doesn't know about CounterColumn's existence:
{code:title=ColumnFamilyRecordRerader.java}
private IColumn unthriftify(ColumnOrSuperColumn cosc)
{
if (cosc.column == null)
return unthriftifySuper(cosc.super_column);
return unthriftifySimple(cosc.column);
}
{code}
It uses only column and super_column fields from ColumnOrSuperColumn class,
while this class keeps CounterColumn in one of two additional fields -
counter_column or counter_super_column:
{code:title=ColumnOrSuperColumn.java}
public class ColumnOrSuperColumn {
public org.apache.cassandra.thrift.Column column;
public org.apache.cassandra.thrift.SuperColumn super_column;
public org.apache.cassandra.thrift.CounterColumn counter_column;
public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
}
{code}
Please add CounterColumn support to ColumnFamilyRecordRerader class.
was:
Accessing CounterColumn from Hadoop ends up with an exception:
{noformat}
java.lang.NullPointerException
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown
Source)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
at
org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
{noformat}
If you look into ColumnFamilyRecordRerader, you will quickly realize that it
doesn't know about CounterColumn's existence:
{code:title=ColumnFamilyRecordRerader.java}
private IColumn unthriftify(ColumnOrSuperColumn cosc)
{
if (cosc.column == null)
return unthriftifySuper(cosc.super_column);
return unthriftifySimple(cosc.column);
}
{code}
It uses only column and super_column fields from ColumnOrSuperColumn class,
while it keeps CounterColumn in one of two additional fields - counter_column
or counter_super_column:
{code:title=ColumnOrSuperColumn.java}
public class ColumnOrSuperColumn {
public org.apache.cassandra.thrift.Column column;
public org.apache.cassandra.thrift.SuperColumn super_column;
public org.apache.cassandra.thrift.CounterColumn counter_column;
public org.apache.cassandra.thrift.CounterSuperColumn counter_super_column;
}
{code}
Please add CounterColumn support to ColumnFamilyRecordRerader class.
> Hadoop cannot read CounterColumn
> --------------------------------
>
> Key: CASSANDRA-2955
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2955
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 0.8.1
> Reporter: Jacek Gerbszt
>
> Accessing CounterColumn from Hadoop ends up with an exception:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftifySuper(ColumnFamilyRecordReader.java:303)
> at
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.unthriftify(ColumnFamilyRecordReader.java:297)
> at
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:288)
> at
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:177)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
> at
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:136)
> at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(Unknown
> Source)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
> at
> org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> {noformat}
> If you look into ColumnFamilyRecordRerader, you will quickly realize that it
> doesn't know about CounterColumn's existence:
> {code:title=ColumnFamilyRecordRerader.java}
> private IColumn unthriftify(ColumnOrSuperColumn cosc)
> {
> if (cosc.column == null)
> return unthriftifySuper(cosc.super_column);
> return unthriftifySimple(cosc.column);
> }
> {code}
> It uses only column and super_column fields from ColumnOrSuperColumn class,
> while this class keeps CounterColumn in one of two additional fields -
> counter_column or counter_super_column:
> {code:title=ColumnOrSuperColumn.java}
> public class ColumnOrSuperColumn {
> public org.apache.cassandra.thrift.Column column;
> public org.apache.cassandra.thrift.SuperColumn super_column;
> public org.apache.cassandra.thrift.CounterColumn counter_column;
> public org.apache.cassandra.thrift.CounterSuperColumn
> counter_super_column;
> }
> {code}
> Please add CounterColumn support to ColumnFamilyRecordRerader class.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
