[
https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Or updated SPARK-1911:
-----------------------------
Description:
The root cause of the problem is detailed in:
https://issues.apache.org/jira/browse/SPARK-1520.
In short, an assembly jar built with Java 7+ is not always accessible by Python
or other versions of Java (especially Java 6). If the assembly jar is not built
on the cluster itself, this problem may manifest itself in strange exceptions
that are not trivial to debug.
Currently we warn users only in make-distribution.sh, but most users build the
jars directly. We should emphasize it in the docs especially for PySpark and
YARN because this issue is not trivial to debug.
was:
Currently we warn users only in make-distribution.sh, but most users build the
jars directly. We should emphasize it in the docs especially for PySpark and
YARN because this issue is not trivial to debug.
> Warn users if their assembly jars are not built with Java 6
> -----------------------------------------------------------
>
> Key: SPARK-1911
> URL: https://issues.apache.org/jira/browse/SPARK-1911
> Project: Spark
> Issue Type: Sub-task
> Components: Documentation
> Affects Versions: 1.1.0
> Reporter: Andrew Or
> Fix For: 1.1.0
>
>
> The root cause of the problem is detailed in:
> https://issues.apache.org/jira/browse/SPARK-1520.
> In short, an assembly jar built with Java 7+ is not always accessible by
> Python or other versions of Java (especially Java 6). If the assembly jar is
> not built on the cluster itself, this problem may manifest itself in strange
> exceptions that are not trivial to debug.
> Currently we warn users only in make-distribution.sh, but most users build
> the jars directly. We should emphasize it in the docs especially for PySpark
> and YARN because this issue is not trivial to debug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
