[
https://issues.apache.org/jira/browse/YARN-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862500#comment-15862500
]
Weiwei Yang commented on YARN-6177:
-----------------------------------
With the work done in YARN-6137, it looks like now timeline client is only
initiated under secure environment and timeline service is enabled, it has
smaller chance to hit this issue but still a problem. An clear error message
can help user understand what might be the problem and how to work around.
When play with Spark2 on yarn, it fails to submit a job with
{quote}
./bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn
--deploy-mode cluster --driver-memory 4g --executor-memory 2g --executor-cores
1 --queue default lib/spark-examples*.jar 10
...
Exception in thread "main" java.lang.NoClassDefFoundError:
com/sun/jersey/api/client/config/ClientConfig
at
org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:55)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createTimelineClient(YarnClientImpl.java:181)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:168)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
...
{quote}
With the patch, it still fails but with some additional information added
{quote}
*Exception in thread "main" java.lang.NoClassDefFoundError:
com/sun/jersey/api/client/config/ClientConfig. It appears that the timeline
client failed to initiate because an incompatible dependency in classpath. If
timeline service is optional to this client, try to work around by setting
yarn.timeline-service.client.best-effort to true.*
at
org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:55)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createTimelineClient(YarnClientImpl.java:190)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getTimelineDelegationToken(YarnClientImpl.java:359)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:151)
at org.apache.spark.deploy.yarn.Client.run(Client.scala:1154)
at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1213)
at org.apache.spark.deploy.yarn.Client.main(Client.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
{quote}
I realized set {{yarn.timeline-service.client.best-effort}} to true (take
advantage of the work from YARN-3144) is better than asking user to disable
timeline service, make sense? Hope this will help users. Thanks.
> Yarn client should exit with an informative error message if an incompatible
> Jersey library is used at client
> -------------------------------------------------------------------------------------------------------------
>
> Key: YARN-6177
> URL: https://issues.apache.org/jira/browse/YARN-6177
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.8.0, 3.0.0-alpha2
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Attachments: YARN-6177.01.patch
>
>
> Per discussion in YARN-5271, lets provide an error message to suggest user to
> disable timeline service instead of disabling for them.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
