[
https://issues.apache.org/jira/browse/TEZ-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17030048#comment-17030048
]
Jonathan Turner Eagles commented on TEZ-4100:
---------------------------------------------
[~abstractdog], it's possible some guava dependencies probably leak into public
tez APIs which may cause problems. As another option, we could add a
compatibility layer that makes the correct calls at runtime. For example a
sample reflection based method below. (Probably needs to calculate the decision
once and cache the result and handle errors correctly)
The most concerning change is the need to call a different
registerApplicationMaster API to avoid NullPointerExceptions.
I imagine a solution where the two following build commands work correctly.
{code}
// Guava 27 and hadoop 3.1.3
mvn clean test -pl ./tez-dag -am
// Guava 11.0.2 and hadoop 3.0.3
mvn clean test -pl ./tez-dag -am -Dguava.version=11.0.2 -Dhadoop.version=3.0.3
{code}
{code}
package org.apache.tez.common;
import com.google.common.util.concurrent.MoreExecutors;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.concurrent.Executor;
/**
* A interoperability layer to work with multiple versions of guava.
*/
public class GuavaShim {
private GuavaShim() {
}
public static Executor directExecutor() {
Class klass = MoreExecutors.class;
Executor executor = null;
try {
Method m = klass.getDeclaredMethod("directExecutor");
executor = (Executor)m.invoke(null);
} catch (NoSuchMethodException nsme) {
try {
Method m = klass.getDeclaredMethod("sameThreadExecutor");
executor = (Executor)m.invoke(null);
} catch (NoSuchMethodException nsmeSame) {
} catch (IllegalAccessException iae) {
} catch (InvocationTargetException ite) {
}
} catch (IllegalAccessException iae) {
} catch (InvocationTargetException ite) {
}
return executor;
}
}
{code}
> Upgrade to hadoop 3.1.3 and Guava 27.0-jre accordingly
> ------------------------------------------------------
>
> Key: TEZ-4100
> URL: https://issues.apache.org/jira/browse/TEZ-4100
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Attachments: TEZ-4100.01.patch, TEZ-4100.02.patch, TEZ-4100.03.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
