[
https://issues.apache.org/jira/browse/HIVE-5016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abin Shahab updated HIVE-5016:
------------------------------
Release Note: This change will allow hive to operate in local mode.
Status: Patch Available (was: Open)
This changes the classpath submitted by the jobsubmitter to the distributed
cache. The old path did not include the protocol. The new path includes the
protocol. This allows the classpath to be resolved for local mode.
> Local mode FileNotFoundException: File does not exist
> -----------------------------------------------------
>
> Key: HIVE-5016
> URL: https://issues.apache.org/jira/browse/HIVE-5016
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.10.0
> Environment: Centos 6.3 (final)
> Hadoop 2.0.2-alpha
> Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
> Hive libs:
> ls -1 lib/
> antlr-2.7.7.jar
> antlr-runtime-3.0.1.jar
> avro-1.7.1.jar
> avro-mapred-1.7.1.jar
> commons-cli-1.2.jar
> commons-codec-1.4.jar
> commons-collections-3.2.1.jar
> commons-compress-1.4.1.jar
> commons-configuration-1.6.jar
> commons-dbcp-1.4.jar
> commons-lang-2.4.jar
> commons-logging-1.0.4.jar
> commons-logging-api-1.0.4.jar
> commons-pool-1.5.4.jar
> datanucleus-connectionpool-2.0.3.jar
> datanucleus-core-2.0.3.jar
> datanucleus-enhancer-2.0.3.jar
> datanucleus-rdbms-2.0.3.jar
> derby-10.4.2.0.jar
> guava-r09.jar
> hbase-0.92.0.jar
> hbase-0.92.0-tests.jar
> hive-builtins-0.10.0.jar
> hive-cli-0.10.0.jar
> hive-common-0.10.0.jar
> hive-contrib-0.10.0.jar
> hive-exec-0.10.0.jar
> hive-hbase-handler-0.10.0.jar
> hive-hwi-0.10.0.jar
> hive-hwi-0.10.0.war
> hive-jdbc-0.10.0.jar
> hive-metastore-0.10.0.jar
> hive-pdk-0.10.0.jar
> hive-serde-0.10.0.jar
> hive-service-0.10.0.jar
> hive-shims-0.10.0.jar
> jackson-core-asl-1.8.8.jar
> jackson-jaxrs-1.8.8.jar
> jackson-mapper-asl-1.8.8.jar
> jackson-xc-1.8.8.jar
> JavaEWAH-0.3.2.jar
> javolution-5.5.1.jar
> jdo2-api-2.3-ec.jar
> jetty-6.1.26.jar
> jetty-util-6.1.26.jar
> jline-0.9.94.jar
> json-20090211.jar
> libfb303-0.9.0.jar
> libthrift-0.9.0.jar
> log4j-1.2.16.jar
> php
> py
> servlet-api-2.5-20081211.jar
> slf4j-api-1.6.1.jar
> slf4j-log4j12-1.6.1.jar
> sqlline-1_0_2.jar
> stringtemplate-3.1-b1.jar
> xz-1.0.jar
> zookeeper-3.4.3.jar
> Reporter: Abin Shahab
> Priority: Critical
> Attachments: HIVE-5016.patch
>
>
> Hive jobs in local mode fail with the error posted below. The jar file that's
> not being found exists and has the following access:
> > ls -l hive-0.10.0/lib/hive-builtins-0.10.0.jar
> rw-rw-r-- 1 ashahab ashahab 3914 Dec 18 2012
> hive-0.10.0/lib/hive-builtins-0.10.0.jar
> Steps to reproduce:
> hive> set hive.exec.mode.local.auto=true;
> hive> set hive.exec.mode.local.auto;
> hive.exec.mode.local.auto=true
> hive> select count(*) from abin_test_table;
> Automatically selecting local only mode for query
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
> set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
> set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
> set mapred.reduce.tasks=<number>
> 13/08/06 21:37:11 WARN conf.Configuration:
> file:/tmp/ashahab/hive_2013-08-06_21-37-09_046_3263640403676309186/-local-10002/jobconf.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 13/08/06 21:37:11 WARN conf.Configuration:
> file:/tmp/ashahab/hive_2013-08-06_21-37-09_046_3263640403676309186/-local-10002/jobconf.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
> org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
> Execution log at:
> /tmp/ashahab/ashahab_20130806213737_7d26b796-5f55-44ca-a755-8898153d963b.log
> java.io.FileNotFoundException: File does not exist:
> /home/ashahab/dev/hive-0.10.0/lib/hive-builtins-0.10.0.jar
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:782)
> at
> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:208)
> at
> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:71)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:252)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:290)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:361)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1215)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:617)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:612)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:612)
> at
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:689)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
