[
https://issues.apache.org/jira/browse/HIVE-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14952192#comment-14952192
]
Willem van Asperen commented on HIVE-12087:
-------------------------------------------
The method {{runDistCp(Path src, Path dst, Configuration conf)}} of
Hadoop23Shims, that is used for Hadoop 0.23, is not compatible with the
distributed copy tool {{org.apache.hadoop.tools.DistCp}}. That {{DistCp}} class
does not have a parameter-less constructor but an object is constructed by
calling {{Constructor c = clazzDistCp.getConstructor()}} on that class.
A work-around is to set property {{hive.exec.copyfile.maxsize}} to a large
enough size so that copying files is not done using distributed copy - and thus
bypassing this erroneous code.
> IMPORT TABLE fails
> ------------------
>
> Key: HIVE-12087
> URL: https://issues.apache.org/jira/browse/HIVE-12087
> Project: Hive
> Issue Type: Bug
> Components: Import/Export
> Affects Versions: 1.2.1
> Environment: Hortonworks HDP 2.3
> Reporter: Willem van Asperen
>
> IMPORT TABLE fails for larger tables with:
> {code}
> 0: jdbc:hive2://hdpprdhiv01.prd.rsg:10001/> import from
> '/tmp/export/repository/res_sales_navigator';
> INFO : Copying data from
> hdfs://hdpprdmas01.prd.rsg:8020/tmp/export/repository/res_sales_navigator/valid_from=201508250000
> to
> hdfs://hdpprdmas01.prd.rsg:8020/tmp/export/repository/res_sales_navigator/.hive-staging_hive_2015-10-07_20-55-37_456_5706704167497413401-2/-ext-10000
> INFO : Copying file:
> hdfs://hdpprdmas01.prd.rsg:8020/tmp/export/repository/res_sales_navigator/valid_from=201508250000/part-r-00000
> ERROR : Failed with exception Cannot get DistCp constructor:
> org.apache.hadoop.tools.DistCp.<init>()
> java.io.IOException: Cannot get DistCp constructor:
> org.apache.hadoop.tools.DistCp.<init>()
> at
> org.apache.hadoop.hive.shims.Hadoop23Shims.runDistCp(Hadoop23Shims.java:1160)
> at org.apache.hadoop.hive.common.FileUtils.copy(FileUtils.java:553)
> at org.apache.hadoop.hive.ql.exec.CopyTask.execute(CopyTask.java:82)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1054)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:154)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)
> at
> org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.CopyTask (state=08S01,code=1)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
