[
https://issues.apache.org/jira/browse/HCATALOG-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148136#comment-13148136
]
[email protected] commented on HCATALOG-155:
--------------------------------------------------------
bq. On 2011-11-10 23:37:00, David Capwell wrote:
bq. >
storage-drivers/hbase/src/java/org/apache/hcatalog/hbase/HBaseBulkOutputFormat.java,
lines 42-43
bq. > <https://reviews.apache.org/r/2799/diff/1/?file=57429#file57429line42>
bq. >
bq. > Rather than Object it should be Writable since that is the interface
it is expecting
It doesn't really matter in this case. If we want to really be thorough it
should be ImmutableBytesWritable and Put since it's the only accepted classes
by the SequenceFile.
bq. On 2011-11-10 23:37:00, David Capwell wrote:
bq. >
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputStorageDriver.java,
line 315
bq. > <https://reviews.apache.org/r/2799/diff/1/?file=57430#file57430line315>
bq. >
bq. > Why BytesWritable, does it matter whats here?
I wanted to test wether we are successfully ignoring the key, since internally
we are passing an empty ImmutableBytesWritable. If BytesWritable was being
passed we'd get an exception.
bq. On 2011-11-10 23:37:00, David Capwell wrote:
bq. >
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputStorageDriver.java,
line 146
bq. > <https://reviews.apache.org/r/2799/diff/1/?file=57430#file57430line146>
bq. >
bq. > Can you change this to WritableComparable and still have the test
pass? Users should be able to work with <WritableComparable, HCatRecord>
Yes it will. As line #312 proves we ignore the keys any restrictions imposed on
the type of keys is enforced by HCatOutputFormat and containers in which should
be tested in the core module.
- Francis
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2799/#review3163
-----------------------------------------------------------
On 2011-11-10 19:50:20, Francis Liu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2799/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-10 19:50:20)
bq.
bq.
bq. Review request for hcatalog, Sushanth Sowmyan, Vandana Ayyalasomayajula,
and David Capwell.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Patch fixes errors thrown by explicit type checks made by SequenceFile
output format. I've updated both direct and bulk unit tests as well.
bq.
bq.
bq. This addresses bug HCATALOG-155.
bq. https://issues.apache.org/jira/browse/HCATALOG-155
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
storage-drivers/hbase/src/java/org/apache/hcatalog/hbase/HBaseBulkOutputFormat.java
5165f7f
bq.
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputStorageDriver.java
c25e56d
bq.
storage-drivers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputStorageDriver.java
d612584
bq.
bq. Diff: https://reviews.apache.org/r/2799/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. updated tests should verify things are working as should.
bq.
bq.
bq. Thanks,
bq.
bq. Francis
bq.
bq.
> HBase bulkOSD requires value to be Put rather than HCatRecord
> -------------------------------------------------------------
>
> Key: HCATALOG-155
> URL: https://issues.apache.org/jira/browse/HCATALOG-155
> Project: HCatalog
> Issue Type: Bug
> Components: hbase
> Affects Versions: 0.3
> Reporter: David Capwell
> Assignee: Francis Liu
> Labels: hbase
> Fix For: 0.3
>
> Attachments: HCATALOG-155.patch
>
>
> So I tried running a MR job writing to hcat+hbase and get the following error:
> [INFO] 02:42:47,857 [pool-1-thread-1] JobClient:1287 - Task Id :
> attempt_201111040030_0023_r_000000_0, Status : FAILED
> java.io.IOException: wrong value class: org.apache.hadoop.hbase.client.Put is
> not class org.apache.hcatalog.data.DefaultHCatRecord
> at
> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> at
> org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:74)
> at
> org.apache.hcatalog.mapreduce.DefaultRecordWriterContainer.write(DefaultRecordWriterContainer.java:64)
> at
> org.apache.hcatalog.mapreduce.DefaultRecordWriterContainer.write(DefaultRecordWriterContainer.java:33)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:587)
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> It looks like my key has to be ImmutableBytesWritable.class and my value must
> be Put.class? Shouldn't my key be WritableComparible and value
> DefaultHCatRecord (HCatRecord)?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
