Hi
I'm trying to import some tables into Hive. I'm using this kind of CLI
command:
sqoop import \
--driver oracle.jdbc.driver.OracleDriver \
--connect jdbc:oracle:thin:@localhost:1521:XE \
--username had_test --password abc1234 \
--query "select * from all_tables where \$CONDITIONS" \
--num-mappers 1 \
--hive-import \
--hive-table "sample_rm2" \
--target-dir /user/hive/warehouse/sample_rm2 \
--mysql-delimiters \
--verbose
The output table is created, everything looks good, but sqoop does not
exit properly. It just hangs indefinitelly on the last line (look in the
attached log file).
My environment is:
- Sqoop 1.4.1-incubating
git commit id
Compiled by jarcec on Tue Feb 7 20:54:25 CET 2012
- Hadoop 1.0.0
- hive 0.9.0-SNAPSHOT - built from git a few days ago
Is this a know issue?
Thanks for any help
Regards, Marcin
12/02/28 13:54:13 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
12/02/28 13:54:13 WARN tool.BaseSqoopTool: Hive does not support escape characters in fields;
12/02/28 13:54:13 WARN tool.BaseSqoopTool: parse errors in Hive may result from using --escaped-by.
12/02/28 13:54:13 WARN tool.BaseSqoopTool: Hive does not support quoted strings; parse errors
12/02/28 13:54:13 WARN tool.BaseSqoopTool: in Hive may result from using --enclosed-by.
12/02/28 13:54:13 INFO manager.SqlManager: Using default fetchSize of 1000
12/02/28 13:54:13 INFO tool.CodeGenTool: Beginning code generation
12/02/28 13:54:14 INFO manager.SqlManager: Executing SQL statement: select * from all_tables where (1 = 0)
12/02/28 13:54:14 INFO manager.SqlManager: Executing SQL statement: select * from all_tables where (1 = 0)
12/02/28 13:54:14 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop-1.0.0/libexec/..
Note: /tmp/sqoop-m/compile/e30db5359f37c6d21004e22c78085f74/QueryResult.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
12/02/28 13:54:16 ERROR orm.CompilationManager: Could not rename /tmp/sqoop-michler/compile/e30db5359f37c6d21004e22c78085f74/QueryResult.java to /home /michler/./QueryResult.java
java.io.IOException: Destination '/home/michler/./QueryResult.java' already exists
at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:1811)
at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:367)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:453)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
12/02/28 13:54:16 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-m/compile/e30db5359f37c6d21004e22c78085f74/QueryResult.jar
12/02/28 13:54:16 INFO mapreduce.ImportJobBase: Beginning query import.
12/02/28 13:54:18 INFO mapred.JobClient: Running job: job_201202281147_0003
12/02/28 13:54:19 INFO mapred.JobClient: map 0% reduce 0%
12/02/28 13:54:39 INFO mapred.JobClient: map 100% reduce 0%
12/02/28 13:54:44 INFO mapred.JobClient: Job complete: job_201202281147_0003
12/02/28 13:54:44 INFO mapred.JobClient: Counters: 18
12/02/28 13:54:44 INFO mapred.JobClient: Job Counters
12/02/28 13:54:44 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=18733
12/02/28 13:54:44 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
12/02/28 13:54:44 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
12/02/28 13:54:44 INFO mapred.JobClient: Launched map tasks=1
12/02/28 13:54:44 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
12/02/28 13:54:44 INFO mapred.JobClient: File Output Format Counters
12/02/28 13:54:44 INFO mapred.JobClient: Bytes Written=28941
12/02/28 13:54:44 INFO mapred.JobClient: FileSystemCounters
12/02/28 13:54:44 INFO mapred.JobClient: HDFS_BYTES_READ=87
12/02/28 13:54:44 INFO mapred.JobClient: FILE_BYTES_WRITTEN=29027
12/02/28 13:54:44 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=28941
12/02/28 13:54:44 INFO mapred.JobClient: File Input Format Counters
12/02/28 13:54:44 INFO mapred.JobClient: Bytes Read=0
12/02/28 13:54:44 INFO mapred.JobClient: Map-Reduce Framework
12/02/28 13:54:44 INFO mapred.JobClient: Map input records=95
12/02/28 13:54:44 INFO mapred.JobClient: Physical memory (bytes) snapshot=112734208
12/02/28 13:54:44 INFO mapred.JobClient: Spilled Records=0
12/02/28 13:54:44 INFO mapred.JobClient: CPU time spent (ms)=3640
12/02/28 13:54:44 INFO mapred.JobClient: Total committed heap usage (bytes)=99221504
12/02/28 13:54:44 INFO mapred.JobClient: Virtual memory (bytes) snapshot=623640576
12/02/28 13:54:44 INFO mapred.JobClient: Map output records=95
12/02/28 13:54:44 INFO mapred.JobClient: SPLIT_RAW_BYTES=87
12/02/28 13:54:44 INFO mapreduce.ImportJobBase: Transferred 28.2627 KB in 28.1098 seconds (1.0054 KB/sec)
12/02/28 13:54:44 INFO mapreduce.ImportJobBase: Retrieved 95 records.
12/02/28 13:54:44 INFO hive.HiveImport: Removing temporary files from import process: /user/hive/warehouse/sample_rm2/_logs
12/02/28 13:54:44 INFO hive.HiveImport: Loading uploaded data into Hive
12/02/28 13:54:44 INFO manager.SqlManager: Executing SQL statement: select * from all_tables where (1 = 0)
12/02/28 13:54:44 INFO manager.SqlManager: Executing SQL statement: select * from all_tables where (1 = 0)
12/02/28 13:54:44 WARN hive.TableDefWriter: Column PCT_FREE had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column PCT_USED had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column INI_TRANS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column MAX_TRANS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column INITIAL_EXTENT had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column NEXT_EXTENT had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column MIN_EXTENTS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column MAX_EXTENTS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column PCT_INCREASE had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column FREELISTS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column FREELIST_GROUPS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column NUM_ROWS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column BLOCKS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column EMPTY_BLOCKS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column AVG_SPACE had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column CHAIN_CNT had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column AVG_ROW_LEN had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column AVG_SPACE_FREELIST_BLOCKS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column NUM_FREELIST_BLOCKS had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column SAMPLE_SIZE had to be cast to a less precise type in Hive
12/02/28 13:54:44 WARN hive.TableDefWriter: Column LAST_ANALYZED had to be cast to a less precise type in Hive
12/02/28 13:54:45 INFO hive.HiveImport: WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.Ev entCounter in all the log4j.properties files.
12/02/28 13:54:46 INFO hive.HiveImport: Logging initialized using configuration in jar:file:/home/m/hive/lib/hive-common-0.9.0-SNAPSHOT.jar!/hiv e-log4j.properties
12/02/28 13:54:46 INFO hive.HiveImport: Hive history file=/tmp/m/hive_job_log_michler_201202281354_287717854.txt
12/02/28 13:54:51 INFO hive.HiveImport: OK
12/02/28 13:54:51 INFO hive.HiveImport: Time taken: 4.591 seconds
12/02/28 13:54:51 INFO hive.HiveImport: Loading data to table default.sample_rm2
