[
https://issues.apache.org/jira/browse/SQOOP-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14019615#comment-14019615
]
Souvik Bose commented on SQOOP-1334:
------------------------------------
How can I work on the standard file format. I haven't mentioned any input or
output format explicitly. Also the files are simple JSON files residing on hdfs
on which the query is build. Also I have included all the homes and classpath
explicitly for hcat before running the qeury.
Please note that my installation is pretty much standard out of the box cdh4.4
installed using cloudera manager on a 4 node AWS EC2 cluster. So I am wondering
what can go wrong.
Even if hcatalog doesn't work at all, Do you any ways for sqoop to handle
direct json files like :
{"hardware_status":"0000","software_version":"1","original_body":"663474007C3840E8","looqiId":"200","longitude":"51.44944444444444","latitude":"0.5205555555555557","battery_voltage":"29","time_stamp":"1400780444000","original_filename":"/home/ktspool/200_22052014194044.GPS"}
and export it into mysql with schema :
CREATE TABLE IF NOT EXISTS `bd_sqoop1` (
`time_stamp`
`hardware_status` varchar(250) DEFAULT NULL,
`original_body` varchar(250) DEFAULT NULL,
`software_version` varchar(250) DEFAULT NULL,
`looqiId` varchar(250) DEFAULT NULL,
`longitude` varchar(250) DEFAULT NULL,
`battery_voltage` varchar(250) DEFAULT NULL,
`latitude` varchar(250) DEFAULT NULL,
`original_filename` varchar(250) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
I have the hive table perfectly working, so through may be the sqoop hcatalog
export which comes with cdh 4.4 is the best option for me. But it seems like it
has got some problem by itself.
> Sqoop Hcatalog export failed with exception of Failed to load foster storage
> handler during map reduce
> ------------------------------------------------------------------------------------------------------
>
> Key: SQOOP-1334
> URL: https://issues.apache.org/jira/browse/SQOOP-1334
> Project: Sqoop
> Issue Type: Bug
> Components: hive-integration
> Affects Versions: 1.4.3
> Environment: CDH4.4.0 with cloudera manager.
> Reporter: Souvik Bose
> Priority: Blocker
> Labels: hcatalog, sqoop, storage-handler
> Attachments: Sqoop_Hcatalog_Export_Log_verbose.txt
>
>
> I tried to export a hive external table to mysql using sqoop hcatalog export
> command.
> sqoop export --connect jdbc:mysql://example.com:3306/demo1 --username
> demo-password demo1234 bd_sqoop1 --hcatalog-table test_data
> I have set the HCAT_HOME, HADOOP_CLASSPATH, HIVE_HOME and paths of Jars. Now
> the job is progressing upto the map reduce stage, but stops at map 0% reduce
> 0% with an exception as below:
> 14/06/04 13:27:21 INFO mapred.JobClient: Task Id :
> attempt_201406021302_0043_m_000002_1, Status : FAILED
> java.io.IOException: Error initializing HCat record reader
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatRecordReader.initNextRecordReader(SqoopHCatRecordReader.java:146)
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatRecordReader.<init>(SqoopHCatRecordReader.java:57)
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatExportFormat.createRecordReader(SqoopHCatExportFormat.java:128)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:644)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.mapred.Child.main(Child.java:262)
> Caused by: java.io.IOException: Failed to load foster storage handler
> at
> org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:416)
> at org.apache.hcatalog.common.HCatUtil.getStorageHandl
--
This message was sent by Atlassian JIRA
(v6.2#6252)
