[
https://issues.apache.org/jira/browse/SQOOP-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15486722#comment-15486722
]
Hui Cao commented on SQOOP-3011:
--------------------------------
took a further look which found out, that the sqoop import Hive command
translates into two separate steps,
1, import the data into HDFS,
2, call hive command as below
LOAD DATA INPATH 'hdfs://<hostname>:8020/user/hdfs/FOO' INTO TABLE `FOO`
with HIVE which either can load from local or same distributed file system,
Guessing this is the cause of the error, as object store and hdfs are two
different distributed systems.
Recommendation of fix:
1, When sqoop importing data into HDFS, add a logic for it to check if the
table is on other distributed system than the one defined in $defaultDS, if
not, put the import the temp data to the external system
2, Or, add an option for the command import, to specify the external system
location.
> sqoop import to HIVE external table based on file system other than HDFS
> ------------------------------------------------------------------------
>
> Key: SQOOP-3011
> URL: https://issues.apache.org/jira/browse/SQOOP-3011
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.6
> Reporter: Hui Cao
>
> 1, Create external Hive table using swift URI in Hive shell
> CREATE TABLE foo(id INT, msg STRING, INT) ROW FORMAT
> DELIMITED FIELDS TERMINATED BY ','
> LINES TERMINATED BY '\n'
> STORED AS TEXTFILE
> LOCATION ’swift://swift.location/’;
> This table is created on an external file system instead of hdfs, in this
> case, I’m using swift Object Store
> 2, Use sqoop to insert data to this table,
> $SQOOP_PATH/sqoop import --driver com.ibm.db2.jcc.DB2Driver --connect
> jdbc:db://<dbhostname>:<port>/<db> --username <username> --password
> <password> --table FOO --hive-import --hive-home hive
> the process shows following error:
> FAILED: SemanticException [Error 10028]: Line 2:17 Path is not legal
> ''hdfs://<hostname>:8020/user/hdfs/FOO'': Move from:
> hdfs://<hostname>:8020/user/hdfs/FOO to: swift://swift.location/ is not
> valid. Please check that values for params "default.fs.name" and
> "hive.metastore.warehouse.dir" do not conflict.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
