[
https://issues.apache.org/jira/browse/BEAM-1214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830122#comment-15830122
]
ASF GitHub Bot commented on BEAM-1214:
--------------------------------------
GitHub user kobisalant opened a pull request:
https://github.com/apache/beam/pull/1796
BEAM-1214 fail to run on SparkRunner with VerifyError
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ x] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ x] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [x ] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.txt).
---
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kobisalant/incubator-beam
BEAM-1214-fail-to-run-on-SparkRunner
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/1796.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1796
----
commit fbbb76038432dac1c36615770999186e48ecffcf
Author: ksalant <[email protected]>
Date: 2017-01-19T15:34:44Z
Add jackson specific version + shade only relocate classes
----
> fail to run on SparkRunner with VerifyError
> -------------------------------------------
>
> Key: BEAM-1214
> URL: https://issues.apache.org/jira/browse/BEAM-1214
> Project: Beam
> Issue Type: Bug
> Components: runner-spark
> Affects Versions: 0.3.0-incubating
> Reporter: Xu Mingmin
> Assignee: Kobi Salant
>
> I see this exception when running with SparkRunner locally. I fix it in my
> environment by change jackson-module-scala_2.10 to version 2.7.2.
> <dependency>
> <groupId>com.fasterxml.jackson.module</groupId>
> <artifactId>jackson-module-scala_2.10</artifactId>
> <version>2.7.2</version>
> </dependency>
> Here's the exception stack trace:
> Exception in thread "main" java.lang.VerifyError: class
> com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides
> final method
> withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase;
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(Unknown Source)
> at java.security.SecureClassLoader.defineClass(Unknown Source)
> at java.net.URLClassLoader.defineClass(Unknown Source)
> at java.net.URLClassLoader.access$100(Unknown Source)
> at java.net.URLClassLoader$1.run(Unknown Source)
> at java.net.URLClassLoader$1.run(Unknown Source)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(Unknown Source)
> at java.lang.ClassLoader.loadClass(Unknown Source)
> at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
> at java.lang.ClassLoader.loadClass(Unknown Source)
> at
> com.fasterxml.jackson.module.scala.ser.IteratorSerializerModule$class.$init$(IteratorSerializerModule.scala:70)
> at
> com.fasterxml.jackson.module.scala.DefaultScalaModule.<init>(DefaultScalaModule.scala:19)
> at
> com.fasterxml.jackson.module.scala.DefaultScalaModule$.<init>(DefaultScalaModule.scala:35)
> at
> com.fasterxml.jackson.module.scala.DefaultScalaModule$.<clinit>(DefaultScalaModule.scala)
> at
> org.apache.spark.rdd.RDDOperationScope$.<init>(RDDOperationScope.scala:81)
> at
> org.apache.spark.rdd.RDDOperationScope$.<clinit>(RDDOperationScope.scala)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
> at org.apache.spark.rdd.RDD.mapPartitions(RDD.scala:706)
> at
> org.apache.spark.api.java.JavaRDDLike$class.mapPartitions(JavaRDDLike.scala:161)
> at
> org.apache.spark.api.java.AbstractJavaRDDLike.mapPartitions(JavaRDDLike.scala:46)
> at
> org.apache.beam.runners.spark.translation.TransformTranslator$7.evaluate(TransformTranslator.java:262)
> at
> org.apache.beam.runners.spark.translation.TransformTranslator$7.evaluate(TransformTranslator.java:248)
> at
> org.apache.beam.runners.spark.SparkRunner$Evaluator.doVisitTransform(SparkRunner.java:267)
> at
> org.apache.beam.runners.spark.SparkRunner$Evaluator.visitPrimitiveTransform(SparkRunner.java:252)
> at
> org.apache.beam.sdk.runners.TransformTreeNode.visit(TransformTreeNode.java:224)
> at
> org.apache.beam.sdk.runners.TransformTreeNode.visit(TransformTreeNode.java:219)
> at
> org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:101)
> at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:292)
> at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:167)
> at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:75)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:182)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
