[ https://issues.apache.org/jira/browse/TEZ-4376 ]
László Bodor deleted comment on TEZ-4376:
-----------------------------------
was (Author: abstractdog):
more interestingly, when I choose javax.activation.* to be excluded, it works:
{code}
mvn clean install -DskipTests -pl ./tez-plugins/tez-aux-services
jar tf
./tez-plugins/tez-aux-services/target/tez-aux-services-0.10.1-SNAPSHOT.jar |
grep javax
...
javax/
javax/activation/
javax/activation/DataHandlerDataSource.class
javax/activation/SecuritySupport$5.class
javax/activation/ActivationDataFlavor.class
{code}
but the same doesn't work for javax.xml.* for instance:
{code}
mvn clean install -DskipTests -pl ./tez-plugins/tez-aux-services
jar tf
./tez-plugins/tez-aux-services/target/tez-aux-services-0.10.1-SNAPSHOT.jar |
grep javax
...
org/apache/tez/shaded/javax/xml/stream/
org/apache/tez/shaded/javax/xml/stream/events/
org/apache/tez/shaded/javax/xml/stream/events/StartElement.class
org/apache/tez/shaded/javax/xml/stream/events/XMLEvent.class
org/apache/tez/shaded/javax/xml/stream/events/Attribute.class
{code}
> Remove javax from the tez-auxservices shaded jar
> ------------------------------------------------
>
> Key: TEZ-4376
> URL: https://issues.apache.org/jira/browse/TEZ-4376
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: László Bodor
> Priority: Major
>
> When I try to start yarn nodemanager with tez shufflehandler configured, I
> get this:
> {code:java}
> java.lang.NoClassDefFoundError:
> org/apache/tez/shaded/javax/security/cert/X509Certificate
> at
> org.apache.tez.shaded.io.netty.util.internal.EmptyArrays.<clinit>(EmptyArrays.java:38)
> at
> org.apache.tez.auxservices.ShuffleHandler.defaultProcessId(ShuffleHandler.java:601)
> at
> org.apache.tez.auxservices.ShuffleHandler.serviceStart(ShuffleHandler.java:567)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader.serviceStart(AuxiliaryServiceWithCustomClassLoader.java:76)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceStart(AuxServices.java:313)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceStart(ContainerManagerImpl.java:654)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:975)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:1054)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.tez.shaded.javax.security.cert.X509Certificate
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at
> org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:189)
> at
> org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:157)
> ... 15 more
> {code}
> this is because in netty 4, EmptyArrays class - which is referred to by
> DefaultChannelId - refers to X509Certificate
> I tried to add javax.security exclude as:
> {code:java}
> <relocation>
> <pattern>javax</pattern>
> <shadedPattern>org.apache.tez.shaded.$0</shadedPattern>
> <excludes>
> <exclude>javax.crypto.*</exclude>
> <exclude>javax.security.*</exclude> <!-- HERE -->
> </excludes>
> </relocation>
> {code}
> but it didn't work, not sure why
> *when I completely removed javax relocation, it started to work*
> I'm tempted to make javax.security exclusion work, but first I need to
> understand why javax relocation is needed at all
> is there a chance that we want to package a custom javax to be used instead
> of what is provided on nodemanager's classpath?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
