[
https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14502695#comment-14502695
]
Sean Owen commented on SPARK-7009:
----------------------------------
Let's see if I remember this correctly: Java 7 supports zip64, so there's no
problem if building/running with Java 7+ only. Some (early) Java 6 won't read
zip64 correctly though. I think the implicit workaround there was to update to
a later Java 6, since it doesn't affect most releases. Java 6 has some
*different* hacky extension to zip that lets it read/write more than 65K files
though, which means that weirdly Java 6-built assemblies might work on old Java
6 after all.
I think we only officially support the zip64 version. Implicitly, actually,
early Java 6 doesn't necessarily work with Spark.
So... does this end up helping this weird situation if Ant is only making zip64
archives? (Nice that this doesn't actually involve adding an Ant script)
> Build assembly JAR via ant to avoid zip64 problems
> --------------------------------------------------
>
> Key: SPARK-7009
> URL: https://issues.apache.org/jira/browse/SPARK-7009
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Affects Versions: 1.3.0
> Environment: Java 7+
> Reporter: Steve Loughran
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> SPARK-1911 shows the problem that JDK7+ is using zip64 to build large JARs; a
> format incompatible with Java and pyspark.
> Provided the total number of .class files+resources is <64K, ant can be used
> to make the final JAR instead, perhaps by unzipping the maven-generated JAR
> then rezipping it with zip64=never, before publishing the artifact via maven.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
