[
https://issues.apache.org/jira/browse/SQOOP-931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13740468#comment-13740468
]
Sonya Ling commented on SQOOP-931:
----------------------------------
I tried sqoop-1.4.4 for this feature today. I have Hive 0.11.0 that has
hcatalog merged into it. I have Hadoop 2.0.0-cdh4.3.0. I run statement
similar to the followings:
sqoop import --connect jdbc:mysql://<host>/<database> --username <user>
--password <password> --table transaction --hcatalog-table transaction
--create-hcatalog-table --where "date >= 08-01-2012" --hive-partition-key date
It went through and started hadoop job but halt due the following error:
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
interface org.apache.hadoop.mapreduce.JobContext, but class was expected
at
org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:94)
at
org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getOutputFormat(HCatBaseOutputFormat.java:82)
at
org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.checkOutputSpecs(HCatBaseOutputFormat.java:72)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:417)
I know this is due to Hadoop version conflict issue. HCatBaseOutputFormat is
expecting Hadoop 1.0.x but I have Hadoop 2.0.x. I saw similar error when I
ran Ozzie. I could get around by add mapred.mapper.new-api and
mapred.reducer.new-api to true in workflow.xml of ozzie. I added the same
properties to proto-hive-site.xml in hcatalog/etc. It did not work. I
checked hcatalog 0.50 source codes. It does not use those properties.
How can I get around this issue? Please advise. Thanks.
> Integrate HCatalog with Sqoop
> -----------------------------
>
> Key: SQOOP-931
> URL: https://issues.apache.org/jira/browse/SQOOP-931
> Project: Sqoop
> Issue Type: New Feature
> Affects Versions: 1.4.2, 1.4.3
> Environment: All 1.x sqoop version
> Reporter: Venkat Ranganathan
> Assignee: Venkat Ranganathan
> Fix For: 1.4.4
>
> Attachments: SQOOP-931.patch, SQOOP-931.patch.14, SQOOP HCatalog
> Integration - 2.pdf, SQOOP HCatalog Integration - 3.pdf, SQOOP HCatalog
> Integration.pdf
>
>
> Apache HCatalog is a table and storage management service that provides a
> shared schema, data types and table abstraction freeing users from being
> concerned about where or how their data is stored. It provides
> interoperability across Pig, Map Reduce, and Hive.
> A sqoop hcatalog connector will help in supporting storage formats that are
> abstracted by HCatalog.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
