damccorm commented on code in PR #39658:
URL: https://github.com/apache/beam/pull/39658#discussion_r3731872886
##########
release/src/main/groovy/quickstart-java-spark.groovy:
##########
@@ -30,10 +30,22 @@ t.describe 'Run Apache Beam Java SDK Quickstart - Spark'
t.intent 'Runs the WordCount Code with Spark runner'
// Run the wordcount example with the spark runner
- t.run """mvn compile exec:java -q \
- -Dexec.mainClass=org.apache.beam.examples.WordCount \
- -Dexec.args="--inputFile=pom.xml --output=counts \
- --runner=SparkRunner" -Pspark-runner"""
+
+ // Retrieve classpath
+ def deps = t.run """mvn compile dependency:build-classpath -q \
+ -Dmdep.outputFile=/dev/stdout \
+ -Dmaven.wagon.http.retryHandler.class=default \
+ -Dmaven.wagon.http.retryHandler.count=5 \
+ -Dmaven.wagon.http.pool=false \
+ -Dmaven.wagon.httpconnectionManager.ttlSeconds=120 \
+ -Dhttp.keepAlive=false \
+ -Pspark-runner"""
+
+ def cp = "target/classes:${deps}"
+ def jvmArgs = "--add-opens=java.base/sun.nio.ch=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED"
+ t.run """mvn exec:exec -q -Dexec.executable=java \
+ -Dexec.args="${jvmArgs} -cp ${cp} org.apache.beam.examples.WordCount \
+ --inputFile=pom.xml --output=counts --runner=SparkRunner" """
Review Comment:
These changes fix:
```
[ERROR] Failed to execute goal
org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project
word-count-beam:
An exception occured while executing the Java class. class
org.apache.spark.storage.StorageUtils$ (in unnamed module @0x1531e31f)
cannot access class sun.nio.ch.DirectBuffer (in module java.base) because
module java.base does not export sun.nio.ch to unnamed module
```
which can happen with Java 21 and was getting masked by timeouts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
