[
https://issues.apache.org/jira/browse/HIVE-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452578#comment-13452578
]
Zhenxiao Luo commented on HIVE-3435:
------------------------------------
1. pdk PluginTest is triggered in two places, either by builtin Test, or by pdk
Test.
2. On hadoop23, HIVE-3413 fixed pdk PluginTest when triggered by builtin Test,
by adding missing dependencies in builtin/ivy.xml, adding configuration files,
and creating configuration directory for testing in:
${pdk.script.dir}/../test-plugin/test/conf
3. On hadoop23, HIVE-3413 did fixed the pdk PluginTest when triggered by
builtin Test, while, when triggered by pdk Test directly, it is still failing,
due to missing dependency, incorrect configuration file, etc.
4. To fix pdk PluginTest when triggered by pdk Test. missing dependency is
added to pdk/ivy.xml, the following error would happen if this dependency is
not added:
java.io.IOException: Cannot initialize Cluster. Please check your configuration
for mapreduce.framework.name and the correspond server addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:123)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:85)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:78)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:487)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:466)
at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:424)
at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:688)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
Job Submission failed with exception 'java.io.IOException(Cannot initialize
Cluster. Please check your configuration for mapreduce.framework.name and the
correspond server addresses.)'
Execution failed with exit status: 1
Obtaining error information
Task failed!
Task ID:
Stage-1
Logs:
/tmp/cloudera/hive.log
HIVE-3413 already added the missing dependency in builtin/ivy.xml, while, if
run pdk PluginTest in pdk Test, this dependency is not resolved, and the
exception happened. We need to add this dependency so that when PluginTest is
running by pdk, also resolve the dependency.
5. HIVE-3413 created a directory for test configuration at:
pdk/test-plugin/test/conf
builtin Test could successfully get these configuration files by putting the
conf directory as HADOOP_CONF_DIR, while, pdk Test could not get these conf
files. When running pdk test, the value of HADOOP_CONF_DIR(which is set by
builtin Test) could not successfully get the configuration files. Actually,
builtin Test and pdk Test could not share the same {pdk.test.conf.dir}, so they
could not load the same configuration files to get passed.
builtin Test and pdk Test are sharing ${pdk.script.dir}, so I propose to delete
the pdk/test-plugin/test/conf, and put configuration files in the newly
created, pdk/scripts/conf, which could be read by both builtin Test and pdk
Test.
> Get pdk pluginTest passed when triggered from both builtin tests and pdk
> tests on hadoop23
> -------------------------------------------------------------------------------------------
>
> Key: HIVE-3435
> URL: https://issues.apache.org/jira/browse/HIVE-3435
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3435.1.patch.txt
>
>
> Hive pdk pluginTest is running twice in unit testing, one is triggered from
> running builtin tests, another is triggered from running pdk tests.
> HIVE-3413 fixed pdk pluginTest on hadoop23 when triggered from running
> builtin tests. While, when triggered from running pdk tests directly on
> hadoop23, it is failing:
> Testcase: SELECT tp_rot13('Mixed Up!') FROM onerow; took 6.426 sec
> FAILED
> expected:<[]Zvkrq Hc!> but was:<[2012-09-04 18:13:01,668 WARN [main]
> conf.HiveConf (HiveConf.java:<clinit>(73)) - hive-site.xml not found on
> CLASSPATH
> ]Zvkrq Hc!>
> junit.framework.ComparisonFailure: expected:<[]Zvkrq Hc!> but
> was:<[2012-09-04 18:13:01,668 WARN [main] conf.HiveConf
> (HiveConf.java:<clinit>(73)) - hive-site.xml not found on CLASSPATH
> ]Zvkrq Hc!>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
