[
https://issues.apache.org/jira/browse/SQOOP-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632193#comment-14632193
]
Kishore Veleti edited comment on SQOOP-2271 at 7/18/15 1:17 AM:
----------------------------------------------------------------
Hi Hari,
Did you tried by specifying --hcatalog-database "<database name>" option?
I tried similar query on Mysql and even I got the error. Later I added
--hcatalog-database and sqoop import executed successfully.
If you can execute in your environment " hcat
-f,/tmp/hcat-script-1427459532585 " you will get an error "FAILED:
SemanticException org.apache.hadoop.hive.ql.parse.SemanticException: Invalid
table name default."
(Note -f <file name> is shown in your initial message)
Below query worked without any issue - ( i added hcatalog-database option and
also removed the HCatalog database name from the --hcatalog-table option
-------------------------------------------------------
sqoop import --connect "jdbc:mysql://127.0.0.1/metastore" --username "etluser"
--password "etluser" --verbose -m 1 --query 'select * from metastore.TBLS WHERE
1=1 AND $CONDITIONS' --mapreduce-job-name "sqoop-import-myJob"
--create-hcatalog-table --hcatalog-database "sqoop146_db" --hcatalog-table
"metastore_tbls" --hcatalog-storage-stanza "STORED AS ORC"
Below query failed because I did not specified --hcatalog-database option but
specified the database name along with table name
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
sqoop import --connect "jdbc:mysql://127.0.0.1/metastore" --username "etluser"
--password "etluser" --verbose -m 1 --query 'select * from metastore.TBLS WHERE
1=1 AND $CONDITIONS' --mapreduce-job-name "sqoop-import-myJob"
--create-hcatalog-table --hcatalog-table "sqoop146_db.metastore_tbls"
--hcatalog-storage-stanza "STORED AS ORC"
15/07/17 18:14:08 INFO hcat.SqoopHCatUtilities: Executing external HCatalog CLI
process with args :-f,/tmp/hcat-script-1437182048694
15/07/17 18:14:22 INFO hcat.SqoopHCatUtilities: FAILED: SemanticException
org.apache.hadoop.hive.ql.parse.SemanticException: Invalid table name
default.sqoop146_db.metastore_tbls
15/07/17 18:14:22 ERROR tool.ImportTool: Encountered IOException running import
job: java.io.IOException: HCat exited with status 64
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.executeExternalHCatProgram(SqoopHCatUtilities.java:1142)
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.launchHCatCli(SqoopHCatUtilities.java:1091)
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.createHCatTable(SqoopHCatUtilities.java:624)
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:339)
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:782)
at
org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98)
at
org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259)
at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:724)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:499)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Thanks,
Kishore Veleti A.V.K.
was (Author: vavkkishore):
Hi Hari,
Did you tried by specifying --hcatalog-database "<database name>" option?
I tried similar query on Mysql and even I got the error. Later I added
--hcatalog-database and sqoop import executed successfully.
If you can execute in your environment " hcat
-f,/tmp/hcat-script-1427459532585 " you will get an error "FAILED:
SemanticException org.apache.hadoop.hive.ql.parse.SemanticException: Invalid
table name default."
(Note -f <file name> is shown in your initial message)
Below query worked without any issue - ( i added hcatalog-database option and
also removed the HCatalog database name from the --hcatalog-table option
-------------------------------------------------------
sqoop import --connect "jdbc:mysql://127.0.0.1/metastore" --username "etluser"
--password "etluser" --verbose -m 1 --query 'select * from metastore.TBLS WHERE
1=1 AND $CONDITIONS' --mapreduce-job-name "sqoop-import-myJob"
--create-hcatalog-table --hcatalog-database "sqoop146_db" --hcatalog-table
"metastore_tbls" --hcatalog-storage-stanza "STORED AS ORC"
Below query failed because I did not specified --hcatalog-database option but
specified the database name along with table name
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
sqoop import --connect "jdbc:mysql://127.0.0.1/metastore" --username "etluser"
--password "etluser" --verbose -m 1 --query 'select * from metastore.TBLS WHERE
1=1 AND $CONDITIONS' --mapreduce-job-name "sqoop-import-myJob"
--create-hcatalog-table --hcatalog-table "sqoop146_db.metastore_tbls"
--hcatalog-storage-stanza "STORED AS ORC"
15/07/17 18:14:08 INFO hcat.SqoopHCatUtilities: Executing external HCatalog CLI
process with args :-f,/tmp/hcat-script-1437182048694
15/07/17 18:14:22 INFO hcat.SqoopHCatUtilities: FAILED: SemanticException
org.apache.hadoop.hive.ql.parse.SemanticException: Invalid table name
default.sqoop146_db.metastore_tbls
15/07/17 18:14:22 ERROR tool.ImportTool: Encountered IOException running import
job: java.io.IOException: HCat exited with status 64
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.executeExternalHCatProgram(SqoopHCatUtilities.java:1142)
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.launchHCatCli(SqoopHCatUtilities.java:1091)
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.createHCatTable(SqoopHCatUtilities.java:624)
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:339)
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:782)
at
org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98)
at
org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259)
at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:724)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:499)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> HCatalog ORC Null Pointer Exception
> -----------------------------------
>
> Key: SQOOP-2271
> URL: https://issues.apache.org/jira/browse/SQOOP-2271
> Project: Sqoop
> Issue Type: Bug
> Components: codegen, hive-integration
> Affects Versions: 1.4.5
> Environment: HDP 2.2
> Reporter: Hari Sekhon
> Priority: Blocker
>
> Hitting a Null Pointer Exception when importing a table via HCatalog to ORC.
> We do a lot of SQL Server imports, the only additions are the hcatalog / orc
> options:{code}sqoop import --connect
> "jdbc:sqlserver://host:1533;database=myDB" --username "myUser" --password
> "myPassword" --verbose -m 1 --query "select * from [blah].[dbo].[blah2]"
> --mapreduce-job-name "sqoop-import-myJob" --create-hcatalog-table
> --hcatalog-table "mydb.mytable" --hcatalog-storage-stanza "STORED AS ORC"
> 15/03/27 12:32:12 INFO hcat.SqoopHCatUtilities: Executing external HCatalog
> CLI process with args :-f,/tmp/hcat-script-1427459532585
> 15/03/27 12:32:12 DEBUG util.ClassLoaderStack: Restoring classloader:
> java.net.URLClassLoader@7640f42a
> 15/03/27 12:32:12 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010)
> at java.lang.Runtime.exec(Runtime.java:617)
> at java.lang.Runtime.exec(Runtime.java:528)
> at org.apache.sqoop.util.Executor.exec(Executor.java:76)
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.executeExternalHCatProgram(SqoopHCatUtilities.java:1119)
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.launchHCatCli(SqoopHCatUtilities.java:1071)
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.createHCatTable(SqoopHCatUtilities.java:610)
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:325)
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:768)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:249)
> at
> org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:721)
> at
> org.apache.sqoop.manager.SQLServerManager.importQuery(SQLServerManager.java:346)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:499)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> {code}
> At first I thought this was SQOOP-1533 but the stack trace is different and
> we're on Oracle Java 7.
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
