[
https://issues.apache.org/jira/browse/SPARK-13599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299722#comment-15299722
]
Catalin Alexandru Zamfir commented on SPARK-13599:
--------------------------------------------------
Do you have any idea when 1.6.2 gets released?
We've just got in trouble by this bug while trying to develop a new
Groovy-based feature. We have a wrapper application (containing the Spark
context/driver) declaring the groovy-all-2.4.6 dependency. We are sending the
Groovy script to the driver application, it runs for the best part of it, but
when it's time to execute the FlatMapFunction (defined in Groovy as a clojure)
we get this:
Caused by: groovy.lang.GroovyRuntimeException: Conflicting module versions.
Module [groovy-all is loaded in version 2.4.6 and you are trying to load
version 2.1.6
We've tried everything one can imagine in our POM, from trying to exclude the
groovy-all-2.1.6 to matching the version. However, the spark-assembly.jar gets
picked-up first on the executor nodes so this Groovy check triggers and the
exception is thrown.
We've lost days on it, until finding this bug which explains where the groovy
2.1.6 dependency comes from.
Our sys-admins are building a custom package with this commit for now, but we'd
like to be on the official version after the release.
> Groovy-all ends up in spark-assembly if hive profile set
> --------------------------------------------------------
>
> Key: SPARK-13599
> URL: https://issues.apache.org/jira/browse/SPARK-13599
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Affects Versions: 1.5.0, 1.6.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Fix For: 1.6.2, 2.0.0
>
>
> If you do a build with {{-Phive,hive-thriftserver}} then the contents of
> {{org.codehaus.groovy:groovy-all}} gets into the spark-assembly.jar
> This bad because
> * it makes the JAR bigger
> * it makes the build longer
> * it's an uber-JAR itself, so can include things (maybe even conflicting
> things)
> * It's something else that needs to be kept up to date security-wise
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
