[ https://issues.apache.org/jira/browse/BEAM-592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439893#comment-15439893 ]
ASF GitHub Bot commented on BEAM-592: ------------------------------------- GitHub user markflyhigh opened a pull request: https://github.com/apache/incubator-beam/pull/892 [BEAM-592] Fix SparkRunner Dependency Problem in WordCount Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-<Jira issue #>] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `<Jira issue #>` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Fix SparkRunner logging dependency issue which causing stackOverflowError when running WordCount. Test done by running WordCount: ``` mvn clean compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="--inputFile=/tmp/kinglear.txt --runner=SparkRunner --sparkMaster=local --tempLocation=/tmp/out" ``` and WordCount e2e test: ``` mvn clean verify -am -DskipITs=false -DintegrationTestPipelineOptions='[ "--tempRoot=/tmp", "--runner=SparkRunner" ]' -Dit.test=WordCountIT You can merge this pull request into a Git repository by running: $ git pull https://github.com/markflyhigh/incubator-beam fix-sparkrunner-dependency-wordcount Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/892.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #892 ---- commit 0203423babf80f6cee09a370c2deb42b26bee684 Author: Mark Liu <mark...@markliu-macbookpro.roam.corp.google.com> Date: 2016-08-26T21:23:54Z [BEAM-592] Fix SparkRunner Dependency Problem in WordCount ---- > StackOverflowError Failed example/java/WordCount When Using SparkRunner > ----------------------------------------------------------------------- > > Key: BEAM-592 > URL: https://issues.apache.org/jira/browse/BEAM-592 > Project: Beam > Issue Type: Bug > Reporter: Mark Liu > > WordCount(example/java/WordCount) failed running with sparkRunner in > following command: > {code} > mvn clean compile exec:java > -Dexec.mainClass=org.apache.beam.examples.WordCount > -Dexec.args="--inputFile=/tmp/kinglear.txt --runner=SparkRunner > --sparkMaster=local --tempLocation=/tmp/out" > {code} > Following is part of stacktrace: > {code} > Caused by: java.lang.StackOverflowError > at > java.util.concurrent.CopyOnWriteArrayList.toArray(CopyOnWriteArrayList.java:374) > at java.util.logging.Logger.accessCheckedHandlers(Logger.java:1782) > at > java.util.logging.LogManager$RootLogger.accessCheckedHandlers(LogManager.java:1668) > at java.util.logging.Logger.getHandlers(Logger.java:1776) > at java.util.logging.Logger.log(Logger.java:735) > at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:580) > at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:650) > at > org.slf4j.bridge.SLF4JBridgeHandler.callLocationAwareLogger(SLF4JBridgeHandler.java:224) > at > org.slf4j.bridge.SLF4JBridgeHandler.publish(SLF4JBridgeHandler.java:301) > at java.util.logging.Logger.log(Logger.java:738) > at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:580) > at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:650) > at > org.slf4j.bridge.SLF4JBridgeHandler.callLocationAwareLogger(SLF4JBridgeHandler.java:224) > at > org.slf4j.bridge.SLF4JBridgeHandler.publish(SLF4JBridgeHandler.java:301) > ... > {code} > According to [http://slf4j.org/legacy.html#jul-to-slf4j] and in particular > section: "jul-to-slf4j.jar and slf4j-jdk14.jar cannot be present > simultaneously". > Change slf4j-jdk14 dependency scope to test solve the above problem, but > WordCountIT still failed in same reason. -- This message was sent by Atlassian JIRA (v6.3.4#6332)