[
https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029422#comment-14029422
]
Thomas Graves commented on SPARK-1718:
--------------------------------------
So this actually appears to be an issue because of using jdk7 on redhat. If I
switch back to use jdk6 then the build works and pyspark works. Note that in
both cases I'm using jdk7 to run with, so it doesn't appear to be the same as
SPARK-1520
> pyspark doesn't work with assembly jar containing over 65536 files/dirs built
> on redhat
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-1718
> URL: https://issues.apache.org/jira/browse/SPARK-1718
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 1.0.0
> Reporter: Thomas Graves
>
> Recently pyspark was ported to yarn (pr 30), but when I went to try it I
> couldn't get it work. I was building on a redhat 6 box. I figured out that
> if the assembly jar file contained over 65536 files/directories then it
> wouldn't work. If I unjarred the assembly and removed some stuff to get it
> under 65536 and jarred it back up, then it would work.
> It appears to only be an issue when building on a redhat box as I can build
> on my mac and it works just fine there.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
