Thomas Graves created SPARK-1718:
------------------------------------
Summary: pyspark doesn't work with assembly jar containing over
65536 files/dirs built on redhat
Key: SPARK-1718
URL: https://issues.apache.org/jira/browse/SPARK-1718
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 1.0.0
Reporter: Thomas Graves
Recently pyspark was ported to yarn (pr 30), but when I went to try it I
couldn't get it work. I was building on a redhat 6 box. I figured out that if
the assembly jar file contained over 65536 files/directories then it wouldn't
work. If I unjarred the assembly and removed some stuff to get it under 65536
and jarred it back up, then it would work.
It appears to only be an issue when building on a redhat box as I can build on
my mac and it works just fine there.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
