I'm not sure whether it's connected to your problem, but sqoop is designed to firstly import data into HDFS to a temporary directory and than using hive command "LOAD DATA" move them to target location in HIVE warehouse directory. You seems to be importing data directly into default HIVE warehouse directory (--targer-dir /user/hive/warehouse/sample_rm2). I would firstly try to change --target-dir parameter to simple "--target-dir sample_rm2" (directory with this name will be created in your HDFS home directory) or any other HDFS path outside /user/hive/warehouse.
Jarcec On Wed, Mar 07, 2012 at 03:10:57PM +0100, Marcin Cylke wrote: > Hi > > I'm trying to import some tables into Hive. I'm using this kind of > CLI command: > > sqoop import \ > --driver oracle.jdbc.driver.OracleDriver \ > --connect jdbc:oracle:thin:@localhost:1521:XE \ > --username had_test --password abc1234 \ > --query "select * from all_tables where \$CONDITIONS" \ > --num-mappers 1 \ > --hive-import \ > --hive-table "sample_rm2" \ > --target-dir /user/hive/warehouse/sample_rm2 \ > --mysql-delimiters \ > --verbose > > The output table is created, everything looks good, but sqoop does > not exit properly. It just hangs indefinitelly on the last line > (look in the attached log file). > > My environment is: > - Sqoop 1.4.1-incubating > git commit id > Compiled by jarcec on Tue Feb 7 20:54:25 CET 2012 > - Hadoop 1.0.0 > - hive 0.9.0-SNAPSHOT - built from git a few days ago > > Is this a know issue? > > Thanks for any help > Regards, Marcin > > 12/02/28 13:54:13 WARN tool.BaseSqoopTool: Setting your password on the > command-line is insecure. Consider using -P instead. > 12/02/28 13:54:13 WARN tool.BaseSqoopTool: Hive does not support escape > characters in fields; > 12/02/28 13:54:13 WARN tool.BaseSqoopTool: parse errors in Hive may result > from using --escaped-by. > 12/02/28 13:54:13 WARN tool.BaseSqoopTool: Hive does not support quoted > strings; parse errors > 12/02/28 13:54:13 WARN tool.BaseSqoopTool: in Hive may result from using > --enclosed-by. > 12/02/28 13:54:13 INFO manager.SqlManager: Using default fetchSize of 1000 > 12/02/28 13:54:13 INFO tool.CodeGenTool: Beginning code generation > 12/02/28 13:54:14 INFO manager.SqlManager: Executing SQL statement: select * > from all_tables where (1 = 0) > 12/02/28 13:54:14 INFO manager.SqlManager: Executing SQL statement: select * > from all_tables where (1 = 0) > 12/02/28 13:54:14 INFO orm.CompilationManager: HADOOP_HOME is > /usr/lib/hadoop-1.0.0/libexec/.. > Note: /tmp/sqoop-m/compile/e30db5359f37c6d21004e22c78085f74/QueryResult.java > uses or overrides a deprecated API. > Note: Recompile with -Xlint:deprecation for details. > 12/02/28 13:54:16 ERROR orm.CompilationManager: Could not rename > /tmp/sqoop-michler/compile/e30db5359f37c6d21004e22c78085f74/QueryResult.java > to /home /michler/./QueryResult.java > java.io.IOException: Destination '/home/michler/./QueryResult.java' already > exists > at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:1811) > at > org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:227) > at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83) > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:367) > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:453) > at org.apache.sqoop.Sqoop.run(Sqoop.java:145) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) > at org.apache.sqoop.Sqoop.main(Sqoop.java:238) > at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57) > 12/02/28 13:54:16 INFO orm.CompilationManager: Writing jar file: > /tmp/sqoop-m/compile/e30db5359f37c6d21004e22c78085f74/QueryResult.jar > 12/02/28 13:54:16 INFO mapreduce.ImportJobBase: Beginning query import. > 12/02/28 13:54:18 INFO mapred.JobClient: Running job: job_201202281147_0003 > 12/02/28 13:54:19 INFO mapred.JobClient: map 0% reduce 0% > 12/02/28 13:54:39 INFO mapred.JobClient: map 100% reduce 0% > 12/02/28 13:54:44 INFO mapred.JobClient: Job complete: job_201202281147_0003 > 12/02/28 13:54:44 INFO mapred.JobClient: Counters: 18 > 12/02/28 13:54:44 INFO mapred.JobClient: Job Counters > 12/02/28 13:54:44 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=18733 > 12/02/28 13:54:44 INFO mapred.JobClient: Total time spent by all reduces > waiting after reserving slots (ms)=0 > 12/02/28 13:54:44 INFO mapred.JobClient: Total time spent by all maps waiting > after reserving slots (ms)=0 > 12/02/28 13:54:44 INFO mapred.JobClient: Launched map tasks=1 > 12/02/28 13:54:44 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0 > 12/02/28 13:54:44 INFO mapred.JobClient: File Output Format Counters > 12/02/28 13:54:44 INFO mapred.JobClient: Bytes Written=28941 > 12/02/28 13:54:44 INFO mapred.JobClient: FileSystemCounters > 12/02/28 13:54:44 INFO mapred.JobClient: HDFS_BYTES_READ=87 > 12/02/28 13:54:44 INFO mapred.JobClient: FILE_BYTES_WRITTEN=29027 > 12/02/28 13:54:44 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=28941 > 12/02/28 13:54:44 INFO mapred.JobClient: File Input Format Counters > 12/02/28 13:54:44 INFO mapred.JobClient: Bytes Read=0 > 12/02/28 13:54:44 INFO mapred.JobClient: Map-Reduce Framework > 12/02/28 13:54:44 INFO mapred.JobClient: Map input records=95 > 12/02/28 13:54:44 INFO mapred.JobClient: Physical memory (bytes) > snapshot=112734208 > 12/02/28 13:54:44 INFO mapred.JobClient: Spilled Records=0 > 12/02/28 13:54:44 INFO mapred.JobClient: CPU time spent (ms)=3640 > 12/02/28 13:54:44 INFO mapred.JobClient: Total committed heap usage > (bytes)=99221504 > 12/02/28 13:54:44 INFO mapred.JobClient: Virtual memory (bytes) > snapshot=623640576 > 12/02/28 13:54:44 INFO mapred.JobClient: Map output records=95 > 12/02/28 13:54:44 INFO mapred.JobClient: SPLIT_RAW_BYTES=87 > 12/02/28 13:54:44 INFO mapreduce.ImportJobBase: Transferred 28.2627 KB in > 28.1098 seconds (1.0054 KB/sec) > 12/02/28 13:54:44 INFO mapreduce.ImportJobBase: Retrieved 95 records. > 12/02/28 13:54:44 INFO hive.HiveImport: Removing temporary files from import > process: /user/hive/warehouse/sample_rm2/_logs > 12/02/28 13:54:44 INFO hive.HiveImport: Loading uploaded data into Hive > 12/02/28 13:54:44 INFO manager.SqlManager: Executing SQL statement: select * > from all_tables where (1 = 0) > 12/02/28 13:54:44 INFO manager.SqlManager: Executing SQL statement: select * > from all_tables where (1 = 0) > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column PCT_FREE had to be cast to > a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column PCT_USED had to be cast to > a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column INI_TRANS had to be cast > to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column MAX_TRANS had to be cast > to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column INITIAL_EXTENT had to be > cast to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column NEXT_EXTENT had to be cast > to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column MIN_EXTENTS had to be cast > to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column MAX_EXTENTS had to be cast > to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column PCT_INCREASE had to be > cast to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column FREELISTS had to be cast > to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column FREELIST_GROUPS had to be > cast to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column NUM_ROWS had to be cast to > a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column BLOCKS had to be cast to a > less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column EMPTY_BLOCKS had to be > cast to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column AVG_SPACE had to be cast > to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column CHAIN_CNT had to be cast > to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column AVG_ROW_LEN had to be cast > to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column AVG_SPACE_FREELIST_BLOCKS > had to be cast to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column NUM_FREELIST_BLOCKS had to > be cast to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column SAMPLE_SIZE had to be cast > to a less precise type in Hive > 12/02/28 13:54:44 WARN hive.TableDefWriter: Column LAST_ANALYZED had to be > cast to a less precise type in Hive > 12/02/28 13:54:45 INFO hive.HiveImport: WARNING: > org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use > org.apache.hadoop.log.metrics.Ev entCounter in all the log4j.properties files. > 12/02/28 13:54:46 INFO hive.HiveImport: Logging initialized using > configuration in > jar:file:/home/m/hive/lib/hive-common-0.9.0-SNAPSHOT.jar!/hiv > e-log4j.properties > 12/02/28 13:54:46 INFO hive.HiveImport: Hive history > file=/tmp/m/hive_job_log_michler_201202281354_287717854.txt > 12/02/28 13:54:51 INFO hive.HiveImport: OK > 12/02/28 13:54:51 INFO hive.HiveImport: Time taken: 4.591 seconds > 12/02/28 13:54:51 INFO hive.HiveImport: Loading data to table > default.sample_rm2
signature.asc
Description: Digital signature
