[
https://issues.apache.org/jira/browse/TEZ-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15141534#comment-15141534
]
Greg Senia commented on TEZ-3105:
---------------------------------
[~bikassaha], [~sseth] [~gopalv] I made requested changes and tested in my
Sandbox cluster and definitely saw improvement about 10-15 seconds consistently
after setting the classes to final.
BEFORE CHANGE to FINAL:
hive> select count(symbol) from nyse;
Query ID = gsenia_20160210024638_405d17b3-6ecb-4ce5-b831-530d387e5e43
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1455042331631_0006)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 7 7 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 63.10 s
--------------------------------------------------------------------------------
OK
9211067
Time taken: 73.212 seconds, Fetched: 1 row(s)
AFTER CHANGE to FINAL:
hive> select count(symbol) from nyse;
Query ID = gsenia_20160210031453_dc08fb39-e3e6-4239-b09a-fb90648e608a
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1455042331631_0007)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 7 7 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 49.71 s
--------------------------------------------------------------------------------
OK
9211067
Time taken: 61.968 seconds, Fetched: 1 row(s)
Status: Running (Executing on YARN cluster with App id
application_1455042331631_0008)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 7 7 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 57.57 s
--------------------------------------------------------------------------------
OK
9211067
Time taken: 62.251 seconds, Fetched: 1 row(s)
Status: Running (Executing on YARN cluster with App id
application_1455042331631_0023)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 7 7 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 49.32 s
--------------------------------------------------------------------------------
OK
9211067
Time taken: 58.644 seconds, Fetched: 1 row(s)
> Tez does not run on IBM JDK 7 or 8
> ----------------------------------
>
> Key: TEZ-3105
> URL: https://issues.apache.org/jira/browse/TEZ-3105
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Greg Senia
> Assignee: Greg Senia
> Labels: ibm, ibm-jdk
> Attachments: TEZ-3105-2.patch, TEZ-3105.patch
>
>
> When testing Hive on Tez with IBM JDK 7 and 8. The following issue was
> discovered:
> 2016-02-08 22:25:22,869 [ERROR] [main] |app.DAGAppMaster|: Error starting
> DAGAppMaster
> java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
> at
> org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree.getResourceCalculatorProcessTree(ResourceCalculatorProcessTree.java:225)
> at
> org.apache.tez.dag.app.DAGAppMaster.initResourceCalculatorPlugins(DAGAppMaster.java:347)
> at
> org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:371)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at org.apache.tez.dag.app.DAGAppMaster$6.run(DAGAppMaster.java:2274)
> at
> java.security.AccessController.doPrivileged(AccessController.java:686)
> at javax.security.auth.Subject.doAs(Subject.java:569)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2271)
> at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2086)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:88)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:57)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:436)
> at
> org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree.getResourceCalculatorProcessTree(ResourceCalculatorProcessTree.java:221)
> ... 9 more
> Caused by: java.lang.ClassCastException:
> com.ibm.lang.management.ExtendedOperatingSystem incompatible with
> com.sun.management.OperatingSystemMXBean
> at
> org.apache.tez.util.TezMxBeanResourceCalculator.<init>(TezMxBeanResourceCalculator.java:44)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
