[
https://issues.apache.org/jira/browse/TEZ-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor resolved TEZ-4376.
-------------------------------
Resolution: Fixed
> Remove javax.security from the tez-auxservices shaded jar
> ---------------------------------------------------------
>
> Key: TEZ-4376
> URL: https://issues.apache.org/jira/browse/TEZ-4376
> Project: Apache Tez
> Issue Type: Sub-task
> Affects Versions: 0.10.1
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Labels: ShuffleHandler
> Fix For: 0.10.2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When I try to start yarn nodemanager with tez shufflehandler configured, I
> get this:
> {code:java}
> java.lang.NoClassDefFoundError:
> org/apache/tez/shaded/javax/security/cert/X509Certificate
> at
> org.apache.tez.shaded.io.netty.util.internal.EmptyArrays.<clinit>(EmptyArrays.java:38)
> at
> org.apache.tez.auxservices.ShuffleHandler.defaultProcessId(ShuffleHandler.java:601)
> at
> org.apache.tez.auxservices.ShuffleHandler.serviceStart(ShuffleHandler.java:567)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader.serviceStart(AuxiliaryServiceWithCustomClassLoader.java:76)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceStart(AuxServices.java:313)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceStart(ContainerManagerImpl.java:654)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:975)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:1054)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.tez.shaded.javax.security.cert.X509Certificate
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at
> org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:189)
> at
> org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:157)
> ... 15 more
> {code}
> this is because in netty 4, EmptyArrays class - which is originally referred
> to by DefaultChannelId, here the stacktrace shows that I simply copied it to
> ShuffleHandler code to insert some logging - refers to X509Certificate
> I tried to add javax.security exclude as:
> {code:java}
> <relocation>
> <pattern>javax</pattern>
> <shadedPattern>org.apache.tez.shaded.$0</shadedPattern>
> <excludes>
> <exclude>javax.crypto.*</exclude>
> <exclude>javax.security.*</exclude> <!-- HERE -->
> </excludes>
> </relocation>
> {code}
> but it didn't work, not sure why
> *when I completely removed javax relocation, it started to work*
> I'm tempted to make javax.security exclusion work, but first I need to
> understand why javax relocation is needed at all
> is there a chance that we want to package a custom javax to be used instead
> of what is provided on nodemanager's classpath?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
