[
https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15158429#comment-15158429
]
Zhichao Zhang commented on SPARK-13431:
----------------------------------------
I meet the same problem with maven, how to resolve this issue?
*build command as follow:*
bq. mvn clean package -DskipTests -Pscala-2.11 -Phadoop-2.6 -Phive
-Phive-thriftserver -Pyarn -Dyarn.version=2.6.0-cdh5.4.7
-Dhadoop.version=2.6.0-cdh5.4.7
*build error message:*
{quote}
[INFO] Excluding org.scala-lang:scala-reflect:jar:2.11.7 from the shaded jar.
[INFO] Excluding org.scala-lang:scala-library:jar:2.11.7 from the shaded jar.
[INFO] Excluding org.apache.spark:spark-core_2.11:jar:2.0.0-SNAPSHOT from the
shaded jar.
[INFO] Excluding org.apache.avro:avro-mapred:jar:hadoop2:1.7.7 from the shaded
jar.
[INFO] Excluding org.apache.avro:avro-ipc:jar:1.7.7 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.7.7 from the shaded jar.
[INFO] Excluding org.apache.avro:avro-ipc:jar:tests:1.7.7 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the
shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the
shaded jar.
[INFO] Excluding com.twitter:chill_2.11:jar:0.5.0 from the shaded jar.
[INFO] Excluding com.esotericsoftware.kryo:kryo:jar:2.21 from the shaded jar.
[INFO] Excluding com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07
from the shaded jar.
[INFO] Excluding com.esotericsoftware.minlog:minlog:jar:1.2 from the shaded jar.
[INFO] Excluding org.objenesis:objenesis:jar:1.2 from the shaded jar.
[INFO] Excluding com.twitter:chill-java:jar:0.5.0 from the shaded jar.
[INFO] Excluding org.apache.xbean:xbean-asm5-shaded:jar:4.4 from the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-client:jar:2.2.0 from the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-common:jar:2.2.0 from the shaded jar.
[INFO] Excluding commons-cli:commons-cli:jar:1.2 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-math:jar:2.1 from the shaded jar.
[INFO] Excluding xmlenc:xmlenc:jar:0.52 from the shaded jar.
[INFO] Excluding commons-io:commons-io:jar:2.1 from the shaded jar.
[INFO] Excluding commons-lang:commons-lang:jar:2.6 from the shaded jar.
[INFO] Excluding commons-configuration:commons-configuration:jar:1.6 from the
shaded jar.
[INFO] Excluding commons-collections:commons-collections:jar:3.2.2 from the
shaded jar.
[INFO] Excluding commons-digester:commons-digester:jar:1.8 from the shaded jar.
[INFO] Excluding commons-beanutils:commons-beanutils:jar:1.7.0 from the shaded
jar.
[INFO] Excluding commons-beanutils:commons-beanutils-core:jar:1.8.0 from the
shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java:jar:2.5.0 from the shaded
jar.
[INFO] Excluding org.apache.hadoop:hadoop-auth:jar:2.2.0 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.4.1 from the shaded
jar.
[INFO] Excluding org.tukaani:xz:jar:1.0 from the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-hdfs:jar:2.2.0 from the shaded jar.
[INFO] Excluding org.mortbay.jetty:jetty-util:jar:6.1.26 from the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.2.0 from
the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.2.0
from the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-yarn-client:jar:2.2.0 from the shaded
jar.
[INFO] Excluding com.google.inject:guice:jar:3.0 from the shaded jar.
[INFO] Excluding javax.inject:javax.inject:jar:1 from the shaded jar.
[INFO] Excluding aopalliance:aopalliance:jar:1.0 from the shaded jar.
[INFO] Excluding
com.sun.jersey.jersey-test-framework:jersey-test-framework-grizzly2:jar:1.9
from the shaded jar.
[INFO] Excluding
com.sun.jersey.jersey-test-framework:jersey-test-framework-core:jar:1.9 from
the shaded jar.
[INFO] Excluding javax.servlet:javax.servlet-api:jar:3.0.1 from the shaded jar.
[INFO] Excluding com.sun.jersey:jersey-client:jar:1.9 from the shaded jar.
[INFO] Excluding com.sun.jersey:jersey-grizzly2:jar:1.9 from the shaded jar.
[INFO] Excluding org.glassfish.grizzly:grizzly-http:jar:2.1.2 from the shaded
jar.
[INFO] Excluding org.glassfish.grizzly:grizzly-framework:jar:2.1.2 from the
shaded jar.
[INFO] Excluding org.glassfish.gmbal:gmbal-api-only:jar:3.0.0-b023 from the
shaded jar.
[INFO] Excluding org.glassfish.external:management-api:jar:3.0.0-b012 from the
shaded jar.
[INFO] Excluding org.glassfish.grizzly:grizzly-http-server:jar:2.1.2 from the
shaded jar.
[INFO] Excluding org.glassfish.grizzly:grizzly-rcm:jar:2.1.2 from the shaded
jar.
[INFO] Excluding org.glassfish.grizzly:grizzly-http-servlet:jar:2.1.2 from the
shaded jar.
[INFO] Excluding org.glassfish:javax.servlet:jar:3.1 from the shaded jar.
[INFO] Excluding com.sun.jersey:jersey-json:jar:1.9 from the shaded jar.
[INFO] Excluding org.codehaus.jettison:jettison:jar:1.1 from the shaded jar.
[INFO] Excluding com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 from the shaded jar.
[INFO] Excluding javax.xml.bind:jaxb-api:jar:2.2.2 from the shaded jar.
[INFO] Excluding javax.activation:activation:jar:1.1 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-jaxrs:jar:1.9.13 from the shaded
jar.
[INFO] Excluding org.codehaus.jackson:jackson-xc:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.sun.jersey.contribs:jersey-guice:jar:1.9 from the shaded
jar.
[INFO] Excluding org.apache.hadoop:hadoop-yarn-server-common:jar:2.2.0 from the
shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.2.0
from the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-yarn-api:jar:2.2.0 from the shaded
jar.
[INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.2.0 from
the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-yarn-common:jar:2.2.0 from the shaded
jar.
[INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.2.0
from the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-annotations:jar:2.2.0 from the shaded
jar.
[INFO] Excluding org.apache.spark:spark-launcher_2.11:jar:2.0.0-SNAPSHOT from
the shaded jar.
[INFO] Excluding org.apache.spark:spark-network-common_2.11:jar:2.0.0-SNAPSHOT
from the shaded jar.
[INFO] Excluding org.apache.spark:spark-network-shuffle_2.11:jar:2.0.0-SNAPSHOT
from the shaded jar.
[INFO] Excluding org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 from the
shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.5.3 from
the shaded jar.
[INFO] Excluding net.java.dev.jets3t:jets3t:jar:0.7.1 from the shaded jar.
[INFO] Excluding commons-httpclient:commons-httpclient:jar:3.1 from the shaded
jar.
[INFO] Excluding org.apache.curator:curator-recipes:jar:2.4.0 from the shaded
jar.
[INFO] Excluding org.apache.curator:curator-framework:jar:2.4.0 from the shaded
jar.
[INFO] Excluding org.apache.curator:curator-client:jar:2.4.0 from the shaded
jar.
[INFO] Excluding org.apache.zookeeper:zookeeper:jar:3.4.5 from the shaded jar.
[INFO] Excluding jline:jline:jar:0.9.94 from the shaded jar.
[INFO] Excluding org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016
from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.3.2 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-math3:jar:3.4.1 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:1.3.9 from the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.16 from the shaded jar.
[INFO] Excluding org.slf4j:jul-to-slf4j:jar:1.7.16 from the shaded jar.
[INFO] Excluding org.slf4j:jcl-over-slf4j:jar:1.7.16 from the shaded jar.
[INFO] Excluding log4j:log4j:jar:1.2.17 from the shaded jar.
[INFO] Excluding org.slf4j:slf4j-log4j12:jar:1.7.16 from the shaded jar.
[INFO] Excluding com.ning:compress-lzf:jar:1.0.3 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.2 from the shaded jar.
[INFO] Excluding net.jpountz.lz4:lz4:jar:1.3.0 from the shaded jar.
[INFO] Excluding org.roaringbitmap:RoaringBitmap:jar:0.5.11 from the shaded jar.
[INFO] Excluding commons-net:commons-net:jar:2.2 from the shaded jar.
[INFO] Excluding org.json4s:json4s-jackson_2.11:jar:3.2.10 from the shaded jar.
[INFO] Excluding org.json4s:json4s-core_2.11:jar:3.2.10 from the shaded jar.
[INFO] Excluding org.json4s:json4s-ast_2.11:jar:3.2.10 from the shaded jar.
[INFO] Excluding org.scala-lang:scalap:jar:2.11.7 from the shaded jar.
[INFO] Excluding org.scala-lang:scala-compiler:jar:2.11.7 from the shaded jar.
[INFO] Excluding org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4
from the shaded jar.
[INFO] Excluding com.sun.jersey:jersey-server:jar:1.9 from the shaded jar.
[INFO] Excluding asm:asm:jar:3.1 from the shaded jar.
[INFO] Excluding com.sun.jersey:jersey-core:jar:1.9 from the shaded jar.
[INFO] Excluding org.apache.mesos:mesos:jar:shaded-protobuf:0.21.1 from the
shaded jar.
[INFO] Excluding io.netty:netty-all:jar:4.0.29.Final from the shaded jar.
[INFO] Excluding io.netty:netty:jar:3.8.0.Final from the shaded jar.
[INFO] Excluding com.clearspring.analytics:stream:jar:2.7.0 from the shaded jar.
[INFO] Excluding io.dropwizard.metrics:metrics-core:jar:3.1.2 from the shaded
jar.
[INFO] Excluding io.dropwizard.metrics:metrics-jvm:jar:3.1.2 from the shaded
jar.
[INFO] Excluding io.dropwizard.metrics:metrics-json:jar:3.1.2 from the shaded
jar.
[INFO] Excluding io.dropwizard.metrics:metrics-graphite:jar:3.1.2 from the
shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.5.3 from the
shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.5.3 from the
shaded jar.
[INFO] Excluding
com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.5.3 from the
shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.6 from the shaded
jar.
[INFO] Excluding org.apache.ivy:ivy:jar:2.4.0 from the shaded jar.
[INFO] Excluding oro:oro:jar:2.0.8 from the shaded jar.
[INFO] Excluding net.razorvine:pyrolite:jar:4.9 from the shaded jar.
[INFO] Excluding net.sf.py4j:py4j:jar:0.9.1 from the shaded jar.
[INFO] Excluding org.apache.spark:spark-unsafe_2.11:jar:2.0.0-SNAPSHOT from the
shaded jar.
[INFO] Excluding org.codehaus.janino:janino:jar:2.7.8 from the shaded jar.
[INFO] Excluding org.codehaus.janino:commons-compiler:jar:2.7.8 from the shaded
jar.
[INFO] Excluding org.antlr:antlr-runtime:jar:3.5.2 from the shaded jar.
[INFO] Excluding commons-codec:commons-codec:jar:1.10 from the shaded jar.
[INFO] Including org.spark-project.spark:unused:jar:1.0.0 in the shaded jar.
[INFO] Excluding org.scala-lang.modules:scala-xml_2.11:jar:1.0.2 from the
shaded jar.
[DEBUG] Processing JAR
/home/myubuntu/Works/workspace/spark/sql/catalyst/target/spark-catalyst_2.11-2.0.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 52.733 s (Wall Clock)
[INFO] Finished at: 2016-02-23T13:26:26+08:00
[INFO] Final Memory: 38M/744M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade (default) on project
spark-catalyst_2.11: Error creating shaded jar: Method code too large! -> [Help
1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade (default) on project
spark-catalyst_2.11: Error creating shaded jar: Method code too large!
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:185)
at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:181)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.maven.plugin.MojoExecutionException: Error creating
shaded jar: Method code too large!
at
org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:540)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 11 more
Caused by: java.lang.RuntimeException: Method code too large!
at org.objectweb.asm.MethodWriter.a(Unknown Source)
at org.objectweb.asm.ClassWriter.toByteArray(Unknown Source)
at
org.apache.maven.plugins.shade.DefaultShader.addRemappedClass(DefaultShader.java:453)
at
org.apache.maven.plugins.shade.DefaultShader.shadeSingleJar(DefaultShader.java:219)
at
org.apache.maven.plugins.shade.DefaultShader.shadeJars(DefaultShader.java:179)
at
org.apache.maven.plugins.shade.DefaultShader.shade(DefaultShader.java:104)
at
org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:454)
... 13 more
{quote}
> Maven build fails due to: Method code too large! in Catalyst
> ------------------------------------------------------------
>
> Key: SPARK-13431
> URL: https://issues.apache.org/jira/browse/SPARK-13431
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.0.0
> Reporter: Stavros Kontopoulos
> Priority: Blocker
>
> Cannot build the project when run the normal build commands:
> eg.
> {code}
> build/mvn -Phadoop-2.6 -Dhadoop.version=2.6.0 clean package
> ./make-distribution.sh --name test --tgz -Phadoop-2.6
> {code}
> Integration builds are also failing:
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.6/229/console
> https://ci.typesafe.com/job/mit-docker-test-zk-ref/12/console
> It looks like this is the commit that introduced the issue:
> https://github.com/apache/spark/commit/7925071280bfa1570435bde3e93492eaf2167d56
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
