[
https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989666#comment-13989666
]
Sean Owen commented on SPARK-1718:
----------------------------------
Yeah, but it seems like it could well be that the JDK binaries being used
during the build aren't quite what is expected, because some home or path
variable points to JDK6. That was the substance of Patrick's last comment, and
I wasn't sure whether Andrew was definitely confirming the build happened with
Java 7, just that it was installed. (?)
I suppose it could also be some old version of zip being used to re-zip jars
and such, though it strikes me as less likely, but hey.
> pyspark doesn't work with assembly jar containing over 65536 files/dirs built
> on redhat
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-1718
> URL: https://issues.apache.org/jira/browse/SPARK-1718
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 1.0.0
> Reporter: Thomas Graves
>
> Recently pyspark was ported to yarn (pr 30), but when I went to try it I
> couldn't get it work. I was building on a redhat 6 box. I figured out that
> if the assembly jar file contained over 65536 files/directories then it
> wouldn't work. If I unjarred the assembly and removed some stuff to get it
> under 65536 and jarred it back up, then it would work.
> It appears to only be an issue when building on a redhat box as I can build
> on my mac and it works just fine there.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
