[jira] [Commented] (SPARK-8768) SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in Akka Protobuf

2015-07-08 Thread Patrick Wendell (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619687#comment-14619687
 ] 

Patrick Wendell commented on SPARK-8768:


I created SPARK-8933 to track improvements to our maven script.

 SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in 
 Akka Protobuf
 -

 Key: SPARK-8768
 URL: https://issues.apache.org/jira/browse/SPARK-8768
 Project: Spark
  Issue Type: Bug
  Components: Spark Submit
Affects Versions: 1.5.0
Reporter: Josh Rosen
Priority: Blocker

 The end-to-end SparkSubmitSuite tests (launch simple application with 
 spark-submit, include jars passed in through --jars, and include jars 
 passed in through --packages) are currently failing for the pre-YARN Hadoop 
 builds.
 I managed to reproduce one of the Jenkins failures locally:
 {code}
 build/mvn -Phadoop-1 -Dhadoop.version=1.2.1 -Phive -Phive-thriftserver 
 -Pkinesis-asl test -DwildcardSuites=org.apache.spark.deploy.SparkSubmitSuite 
 -Dtest=none
 {code}
 Here's the output from unit-tests.log:
 {code}
 = TEST OUTPUT FOR o.a.s.deploy.SparkSubmitSuite: 'launch simple 
 application with spark-submit' =
 15/07/01 13:39:58.964 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Class path contains multiple SLF4J bindings.
 15/07/01 13:39:58.964 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Found binding in 
 [jar:file:/Users/joshrosen/Documents/spark-2/assembly/target/scala-2.10/spark-assembly-1.5.0-SNAPSHOT-hadoop1.2.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Found binding in 
 [jar:file:/Users/joshrosen/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
 explanation.
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
 15/07/01 13:39:58.966 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:58 INFO SparkContext: Running Spark version 
 1.5.0-SNAPSHOT
 15/07/01 13:39:59.334 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: Changing view acls to: 
 joshrosen
 15/07/01 13:39:59.335 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: Changing modify acls to: 
 joshrosen
 15/07/01 13:39:59.335 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: SecurityManager: 
 authentication disabled; ui acls disabled; users with view permissions: 
 Set(joshrosen); users with modify permissions: Set(joshrosen)
 15/07/01 13:39:59.898 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO Slf4jLogger: Slf4jLogger started
 15/07/01 13:39:59.934 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO Remoting: Starting remoting
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:40:00 ERROR ActorSystemImpl: Uncaught fatal error from 
 thread [sparkDriver-akka.remote.default-remote-dispatcher-5] shutting down 
 ActorSystem [sparkDriver]
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils: java.lang.VerifyError: class 
 akka.remote.WireFormats$AkkaControlMessage overrides final method 
 getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.lang.ClassLoader.defineClass1(Native Method)
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at 
 java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
 15/07/01 13:40:00.010 redirect

[jira] [Commented] (SPARK-8768) SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in Akka Protobuf

2015-07-08 Thread Patrick Wendell (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619681#comment-14619681
 ] 

Patrick Wendell commented on SPARK-8768:


So it turns out that build/mvn still uses the system maven even if it downloads 
the newer version (this was the original design). Is it possible that is why 
it's breaking?

 SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in 
 Akka Protobuf
 -

 Key: SPARK-8768
 URL: https://issues.apache.org/jira/browse/SPARK-8768
 Project: Spark
  Issue Type: Bug
  Components: Spark Submit
Affects Versions: 1.5.0
Reporter: Josh Rosen
Priority: Blocker

 The end-to-end SparkSubmitSuite tests (launch simple application with 
 spark-submit, include jars passed in through --jars, and include jars 
 passed in through --packages) are currently failing for the pre-YARN Hadoop 
 builds.
 I managed to reproduce one of the Jenkins failures locally:
 {code}
 build/mvn -Phadoop-1 -Dhadoop.version=1.2.1 -Phive -Phive-thriftserver 
 -Pkinesis-asl test -DwildcardSuites=org.apache.spark.deploy.SparkSubmitSuite 
 -Dtest=none
 {code}
 Here's the output from unit-tests.log:
 {code}
 = TEST OUTPUT FOR o.a.s.deploy.SparkSubmitSuite: 'launch simple 
 application with spark-submit' =
 15/07/01 13:39:58.964 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Class path contains multiple SLF4J bindings.
 15/07/01 13:39:58.964 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Found binding in 
 [jar:file:/Users/joshrosen/Documents/spark-2/assembly/target/scala-2.10/spark-assembly-1.5.0-SNAPSHOT-hadoop1.2.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Found binding in 
 [jar:file:/Users/joshrosen/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
 explanation.
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
 15/07/01 13:39:58.966 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:58 INFO SparkContext: Running Spark version 
 1.5.0-SNAPSHOT
 15/07/01 13:39:59.334 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: Changing view acls to: 
 joshrosen
 15/07/01 13:39:59.335 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: Changing modify acls to: 
 joshrosen
 15/07/01 13:39:59.335 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: SecurityManager: 
 authentication disabled; ui acls disabled; users with view permissions: 
 Set(joshrosen); users with modify permissions: Set(joshrosen)
 15/07/01 13:39:59.898 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO Slf4jLogger: Slf4jLogger started
 15/07/01 13:39:59.934 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO Remoting: Starting remoting
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:40:00 ERROR ActorSystemImpl: Uncaught fatal error from 
 thread [sparkDriver-akka.remote.default-remote-dispatcher-5] shutting down 
 ActorSystem [sparkDriver]
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils: java.lang.VerifyError: class 
 akka.remote.WireFormats$AkkaControlMessage overrides final method 
 getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.lang.ClassLoader.defineClass1(Native Method)
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at 
 java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 15/07/01 13:40:00.010 redirect stderr for command

[jira] [Commented] (SPARK-8768) SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in Akka Protobuf

2015-07-02 Thread Josh Rosen (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612724#comment-14612724
 ] 

Josh Rosen commented on SPARK-8768:
---

[~zsxwing], I don't think so: the master Maven build uses build/mvn, which, as 
far as I know, should now be downloading the newer Maven version that is 
supposed to work.

 SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in 
 Akka Protobuf
 -

 Key: SPARK-8768
 URL: https://issues.apache.org/jira/browse/SPARK-8768
 Project: Spark
  Issue Type: Bug
  Components: Spark Submit
Affects Versions: 1.5.0
Reporter: Josh Rosen
Priority: Blocker

 The end-to-end SparkSubmitSuite tests (launch simple application with 
 spark-submit, include jars passed in through --jars, and include jars 
 passed in through --packages) are currently failing for the pre-YARN Hadoop 
 builds.
 I managed to reproduce one of the Jenkins failures locally:
 {code}
 build/mvn -Phadoop-1 -Dhadoop.version=1.2.1 -Phive -Phive-thriftserver 
 -Pkinesis-asl test -DwildcardSuites=org.apache.spark.deploy.SparkSubmitSuite 
 -Dtest=none
 {code}
 Here's the output from unit-tests.log:
 {code}
 = TEST OUTPUT FOR o.a.s.deploy.SparkSubmitSuite: 'launch simple 
 application with spark-submit' =
 15/07/01 13:39:58.964 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Class path contains multiple SLF4J bindings.
 15/07/01 13:39:58.964 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Found binding in 
 [jar:file:/Users/joshrosen/Documents/spark-2/assembly/target/scala-2.10/spark-assembly-1.5.0-SNAPSHOT-hadoop1.2.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Found binding in 
 [jar:file:/Users/joshrosen/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
 explanation.
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
 15/07/01 13:39:58.966 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:58 INFO SparkContext: Running Spark version 
 1.5.0-SNAPSHOT
 15/07/01 13:39:59.334 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: Changing view acls to: 
 joshrosen
 15/07/01 13:39:59.335 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: Changing modify acls to: 
 joshrosen
 15/07/01 13:39:59.335 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: SecurityManager: 
 authentication disabled; ui acls disabled; users with view permissions: 
 Set(joshrosen); users with modify permissions: Set(joshrosen)
 15/07/01 13:39:59.898 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO Slf4jLogger: Slf4jLogger started
 15/07/01 13:39:59.934 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO Remoting: Starting remoting
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:40:00 ERROR ActorSystemImpl: Uncaught fatal error from 
 thread [sparkDriver-akka.remote.default-remote-dispatcher-5] shutting down 
 ActorSystem [sparkDriver]
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils: java.lang.VerifyError: class 
 akka.remote.WireFormats$AkkaControlMessage overrides final method 
 getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.lang.ClassLoader.defineClass1(Native Method)
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at 
 java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO

[jira] [Commented] (SPARK-8768) SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in Akka Protobuf

2015-07-02 Thread Shixiong Zhu (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612173#comment-14612173
 ] 

Shixiong Zhu commented on SPARK-8768:
-

[~joshrosen] is it the maven issue: 
https://github.com/apache/spark/pull/6492#issuecomment-110949299

 SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in 
 Akka Protobuf
 -

 Key: SPARK-8768
 URL: https://issues.apache.org/jira/browse/SPARK-8768
 Project: Spark
  Issue Type: Bug
  Components: Spark Submit
Affects Versions: 1.5.0
Reporter: Josh Rosen
Priority: Blocker

 The end-to-end SparkSubmitSuite tests (launch simple application with 
 spark-submit, include jars passed in through --jars, and include jars 
 passed in through --packages) are currently failing for the pre-YARN Hadoop 
 builds.
 I managed to reproduce one of the Jenkins failures locally:
 {code}
 build/mvn -Phadoop-1 -Dhadoop.version=1.2.1 -Phive -Phive-thriftserver 
 -Pkinesis-asl test -DwildcardSuites=org.apache.spark.deploy.SparkSubmitSuite 
 -Dtest=none
 {code}
 Here's the output from unit-tests.log:
 {code}
 = TEST OUTPUT FOR o.a.s.deploy.SparkSubmitSuite: 'launch simple 
 application with spark-submit' =
 15/07/01 13:39:58.964 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Class path contains multiple SLF4J bindings.
 15/07/01 13:39:58.964 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Found binding in 
 [jar:file:/Users/joshrosen/Documents/spark-2/assembly/target/scala-2.10/spark-assembly-1.5.0-SNAPSHOT-hadoop1.2.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Found binding in 
 [jar:file:/Users/joshrosen/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
 explanation.
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
 15/07/01 13:39:58.966 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:58 INFO SparkContext: Running Spark version 
 1.5.0-SNAPSHOT
 15/07/01 13:39:59.334 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: Changing view acls to: 
 joshrosen
 15/07/01 13:39:59.335 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: Changing modify acls to: 
 joshrosen
 15/07/01 13:39:59.335 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: SecurityManager: 
 authentication disabled; ui acls disabled; users with view permissions: 
 Set(joshrosen); users with modify permissions: Set(joshrosen)
 15/07/01 13:39:59.898 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO Slf4jLogger: Slf4jLogger started
 15/07/01 13:39:59.934 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO Remoting: Starting remoting
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:40:00 ERROR ActorSystemImpl: Uncaught fatal error from 
 thread [sparkDriver-akka.remote.default-remote-dispatcher-5] shutting down 
 ActorSystem [sparkDriver]
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils: java.lang.VerifyError: class 
 akka.remote.WireFormats$AkkaControlMessage overrides final method 
 getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.lang.ClassLoader.defineClass1(Native Method)
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at 
 java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

[jira] [Commented] (SPARK-8768) SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in Akka Protobuf

2015-07-01 Thread Josh Rosen (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610988#comment-14610988
 ] 

Josh Rosen commented on SPARK-8768:
---

We didn't notice this earlier because the Master Maven Pre-YARN build was 
misconfigured and was building against the wrong Hadoop versions.

 SparkSubmitSuite fails on Hadoop 1.x builds due to java.lang.VerifyError in 
 Akka Protobuf
 -

 Key: SPARK-8768
 URL: https://issues.apache.org/jira/browse/SPARK-8768
 Project: Spark
  Issue Type: Bug
  Components: Spark Submit
Affects Versions: 1.5.0
Reporter: Josh Rosen
Priority: Blocker

 The end-to-end SparkSubmitSuite tests (launch simple application with 
 spark-submit, include jars passed in through --jars, and include jars 
 passed in through --packages) are currently failing for the pre-YARN Hadoop 
 builds.
 I managed to reproduce one of the Jenkins failures locally:
 {code}
 build/mvn -Phadoop-1 -Dhadoop.version=1.2.1 -Phive -Phive-thriftserver 
 -Pkinesis-asl test -DwildcardSuites=org.apache.spark.deploy.SparkSubmitSuite 
 -Dtest=none
 {code}
 Here's the output from unit-tests.log:
 {code}
 = TEST OUTPUT FOR o.a.s.deploy.SparkSubmitSuite: 'launch simple 
 application with spark-submit' =
 15/07/01 13:39:58.964 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Class path contains multiple SLF4J bindings.
 15/07/01 13:39:58.964 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Found binding in 
 [jar:file:/Users/joshrosen/Documents/spark-2/assembly/target/scala-2.10/spark-assembly-1.5.0-SNAPSHOT-hadoop1.2.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Found binding in 
 [jar:file:/Users/joshrosen/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
 explanation.
 15/07/01 13:39:58.965 redirect stderr for command ./bin/spark-submit INFO 
 Utils: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
 15/07/01 13:39:58.966 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:58 INFO SparkContext: Running Spark version 
 1.5.0-SNAPSHOT
 15/07/01 13:39:59.334 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: Changing view acls to: 
 joshrosen
 15/07/01 13:39:59.335 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: Changing modify acls to: 
 joshrosen
 15/07/01 13:39:59.335 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO SecurityManager: SecurityManager: 
 authentication disabled; ui acls disabled; users with view permissions: 
 Set(joshrosen); users with modify permissions: Set(joshrosen)
 15/07/01 13:39:59.898 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO Slf4jLogger: Slf4jLogger started
 15/07/01 13:39:59.934 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:39:59 INFO Remoting: Starting remoting
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils: 15/07/01 13:40:00 ERROR ActorSystemImpl: Uncaught fatal error from 
 thread [sparkDriver-akka.remote.default-remote-dispatcher-5] shutting down 
 ActorSystem [sparkDriver]
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils: java.lang.VerifyError: class 
 akka.remote.WireFormats$AkkaControlMessage overrides final method 
 getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.lang.ClassLoader.defineClass1(Native Method)
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
 15/07/01 13:40:00.009 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at 
 java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 15/07/01 13:40:00.010 redirect stderr for command ./bin/spark-submit INFO 
 Utils:at