This is an automated email from the ASF dual-hosted git repository.
He-Pin pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pekko.git
The following commit(s) were added to refs/heads/main by this push:
new b3f259cc6c build: add nightly multi-jvm test workflow (#3602)
b3f259cc6c is described below
commit b3f259cc6cd88d9c57870727e534de6659ace0c5
Author: PJ Fanning <[email protected]>
AuthorDate: Sat Oct 10 06:27:31 2026 +0100
build: add nightly multi-jvm test workflow (#3602)
* build: add nightly multi-jvm test workflow
Motivation:
About 160 multi-jvm specs in cluster, cluster-sharding, cluster-tools,
distributed-data, cluster-typed, cluster-sharding-typed and remote-tests
never run in CI. Every test job passes -Dpekko.test.multi-in-test=false,
and only the cluster-metrics Sigar nightly runs multi-jvm tests.
Modification:
Add a nightly workflow that runs <module>/MultiJvm/test per module group
on JDK 17 / Scala 2.13 with the default Artery transport. It is
temporarily also triggered on pull_request so its runtime and flakiness
can be evaluated in this PR's CI; that trigger must be removed before
merge.
Result:
Multi-jvm specs are exercised nightly on main.
Tests:
- sbt "inspect cluster/MultiJvm/test" "inspect
cluster-sharding-typed/MultiJvm/test" "inspect remote-tests/MultiJvm/test":
task keys resolve
- Workflow itself is evaluated by this PR's CI run
References:
None - CI coverage gap found while reviewing multi-in-test usage
* build: add Apache license header to multi-jvm workflow
* test: fix multi-jvm specs that fail when run
Motivation:
Running the multi-jvm specs in CI showed two broken tests that had gone
unnoticed because CI never ran them.
Modification:
- FlightRecording looked up jdk.jfr.Recording.stop() as returning void.
It returns boolean on JDK 9+, so the lookup threw
NoSuchMethodException and ClusterShardingRememberEntitiesPerfSpec
failed on node 1.
- AeronStreamMultiNodeSpec.channel sent GetUdpPort with `!` from a base
class without ImplicitSender, so the reply went to dead letters and
every AeronStream*Spec timed out waiting for the UDP port. Send it
with testActor as the sender.
Result:
ClusterShardingRememberEntitiesPerfSpec and AeronStreamConsistencySpec
pass.
Tests:
- sbt "cluster-sharding-typed/MultiJvm/testOnly
*ClusterShardingRememberEntitiesPerf*": failed 1 / canceled 5 on node 1 before,
6/6 on all 3 nodes after
- sbt "remote-tests/MultiJvm/testOnly *AeronStreamConsistency*": 3/3 on
both nodes (CI before: 1 passed, 2 failed)
References:
None - found by the multi-jvm workflow in this PR
* build: report real test counts for multi-jvm tests
Motivation:
Multi-jvm test runs always ended with "No tests to run for MultiJvm"
and "Total 0" because the plugin returned Tests.Output with no events,
so the job summaries hid how many tests ran and failed.
Modification:
Wrap each test JVM's logger in TestCountingLogger, which reads the
ScalaTest "Tests: succeeded N, failed N, ..." summary line. Report one
SuiteResult per JVM in Tests.Output.events. A JVM that exits non-zero
without a failed test, or without printing a summary, is reported as an
error.
Result:
sbt prints real totals for MultiJvm/test and MultiJvm/testOnly (and for
Test/test when multi-in-test is enabled), e.g.
"Failed: Total 19, Failed 1, Errors 0, Passed 18, Canceled 5".
Tests:
- sbt "remote-tests/MultiJvm/testOnly *AeronStreamConsistency*
*RemoteReDeploymentFast*": Passed: Total 10 (previously Total 0)
- sbt "cluster-sharding-typed/MultiJvm/testOnly
*ClusterShardingRememberEntitiesPerf* *ClusterShardingPreparingForShutdown*"
without the JFR fix: Failed: Total 19, Failed 1, Errors 0, Passed 18, Canceled 5
References:
None - found by the multi-jvm workflow in this PR
* build: run multi-jvm workflow only on schedule and manual dispatch
---
.github/workflows/nightly-multi-jvm-tests.yml | 82 ++++++++++++++++++++++
.../pekko/cluster/sharding/FlightRecording.scala | 4 +-
project/SbtMultiJvmPlugin.scala | 71 +++++++++++++++----
.../artery/aeron/AeronStreamMultiNodeSpec.scala | 2 +-
4 files changed, 145 insertions(+), 14 deletions(-)
diff --git a/.github/workflows/nightly-multi-jvm-tests.yml
b/.github/workflows/nightly-multi-jvm-tests.yml
new file mode 100644
index 0000000000..ef3b389889
--- /dev/null
+++ b/.github/workflows/nightly-multi-jvm-tests.yml
@@ -0,0 +1,82 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Nightly Multi JVM Tests
+
+on:
+ schedule:
+ - cron: "0 2 * * *"
+ workflow_dispatch:
+
+permissions: {}
+
+jobs:
+ pekko-multi-jvm-tests:
+ name: Pekko Multi JVM Tests (${{ matrix.command }})
+ runs-on: ubuntu-22.04
+ if: github.repository == 'apache/pekko'
+ timeout-minutes: 240
+ strategy:
+ fail-fast: false
+ matrix:
+ command:
+ - cluster/MultiJvm/test
+ - cluster-sharding/MultiJvm/test
+ - cluster-typed/MultiJvm/test cluster-sharding-typed/MultiJvm/test
+ - distributed-data/MultiJvm/test cluster-tools/MultiJvm/test
cluster-metrics/MultiJvm/test
+ - remote-tests/MultiJvm/test
+ steps:
+ - name: Checkout
+ uses: actions/[email protected]
+ with:
+ # we don't know what commit the last tag was it's safer to get
entire repo so previousStableVersion resolves
+ fetch-depth: 0
+ fetch-tags: true
+ persist-credentials: false
+
+ - name: Setup Java 17
+ uses: actions/setup-java@v6
+ with:
+ distribution: temurin
+ java-version: 17
+
+ - name: Install sbt
+ uses: sbt/setup-sbt@c7d2d6258b4bd0d3ec5129e6b3453199d3c79729 # v1.5.8
+
+ - name: Cache Coursier cache
+ uses: coursier/cache-action@95e5b1029b6b86e7bac033ee44a0697d8a527d2d #
8.1.1
+
+ - name: Enable jvm-opts
+ run: cp .jvmopts-ci .jvmopts
+
+ - name: sbt ${{ matrix.command }}
+ env:
+ DEVELOCITY_ACCESS_KEY: ${{ secrets.DEVELOCITY_ACCESS_KEY }}
+ run: |-
+ sbt \
+ -Djava.security.egd=file:/dev/urandom \
+ -Dpekko.cluster.assert=on \
+ -Dpekko.test.timefactor=2 \
+ -Dpekko.actor.testkit.typed.timefactor=2 \
+ -Dpekko.test.tags.exclude=gh-exclude,timing \
+ -Dpekko.log.timestamps=true \
+ -Dio.netty.leakDetection.level=PARANOID \
+ -Dmultinode.XX:MetaspaceSize=128M \
+ -Dmultinode.Xms256M \
+ -Dmultinode.Xmx256M \
+ -Dmultinode.XX:+AlwaysActAsServerClassMachine \
+ clean ${{ matrix.command }}
diff --git
a/cluster-sharding-typed/src/test/scala/org/apache/pekko/cluster/sharding/FlightRecording.scala
b/cluster-sharding-typed/src/test/scala/org/apache/pekko/cluster/sharding/FlightRecording.scala
index c12ac3ce88..f3a7139d22 100644
---
a/cluster-sharding-typed/src/test/scala/org/apache/pekko/cluster/sharding/FlightRecording.scala
+++
b/cluster-sharding-typed/src/test/scala/org/apache/pekko/cluster/sharding/FlightRecording.scala
@@ -30,6 +30,8 @@ class FlightRecording(system: ActorSystem) {
private val lookup = MethodHandles.publicLookup()
private val noArgVoidMethodType = MethodType.methodType(Void.TYPE)
+ // Recording.stop() returns boolean (JDK 9+)
+ private val noArgBooleanMethodType =
MethodType.methodType(java.lang.Boolean.TYPE)
private val dumpMethodType = MethodType.methodType(Void.TYPE, classOf[Path])
private val dynamic = system.asInstanceOf[ExtendedActorSystem].dynamicAccess
@@ -39,7 +41,7 @@ class FlightRecording(system: ActorSystem) {
private val startMethod =
clazz.map(lookup.findVirtual(_, "start", noArgVoidMethodType))
private val stopMethod =
- clazz.map(lookup.findVirtual(_, "stop", noArgVoidMethodType))
+ clazz.map(lookup.findVirtual(_, "stop", noArgBooleanMethodType))
private val dumpMethod =
clazz.map(lookup.findVirtual(_, "dump", dumpMethodType))
diff --git a/project/SbtMultiJvmPlugin.scala b/project/SbtMultiJvmPlugin.scala
index 01c484f986..94189a4f6f 100644
--- a/project/SbtMultiJvmPlugin.scala
+++ b/project/SbtMultiJvmPlugin.scala
@@ -314,7 +314,7 @@ object MultiJvmPlugin extends AutoPlugin {
}
Tests.Output(
Tests.overall(results.map(_._2)),
- Map.empty,
+ results.flatMap(_._3).toMap,
results.map(result => Tests.Summary("multi-jvm", result._1)))
}
@@ -354,7 +354,7 @@ object MultiJvmPlugin extends AutoPlugin {
srcDir: File,
input: Boolean,
createLogger: String => Logger,
- log: Logger): (String, sbt.TestResult) = {
+ log: Logger): (String, sbt.TestResult, Map[String, SuiteResult]) = {
val logName = "* " + name
log.info(logName)
val classesHostsJavas = getClassesHostsJavas(classes, IndexedSeq.empty,
IndexedSeq.empty, "")
@@ -363,7 +363,7 @@ object MultiJvmPlugin extends AutoPlugin {
case (testClass, index) =>
val className = multiSimpleName(testClass)
val jvmName = "JVM-" + (index + 1) + "-" + className
- val jvmLogger = createLogger(jvmName)
+ val jvmLogger = new TestCountingLogger(createLogger(jvmName))
val optionsFile = (srcDir ** (className + ".opts")).get.headOption
val optionsFromFile =
optionsFile.map(IO.read(_)).map(_.trim.replace("\\n", "
").split("\\s+").toList).getOrElse(Seq.empty[String])
@@ -373,10 +373,9 @@ object MultiJvmPlugin extends AutoPlugin {
val connectInput = input && index == 0
log.debug("Starting %s for %s".format(jvmName, testClass))
log.debug(" with JVM options: %s".format(allJvmOptions.mkString(" ")))
- val testClass2Process = (testClass, Jvm.startJvm(javaBin,
allJvmOptions, runOptions, jvmLogger, connectInput))
- testClass2Process
+ (testClass, Jvm.startJvm(javaBin, allJvmOptions, runOptions,
jvmLogger, connectInput), jvmLogger)
}
- processExitCodes(name, processes, log)
+ testResults(name, processes, log)
}
def processExitCodes(name: String, processes: Seq[(String, Process)], log:
Logger): (String, sbt.TestResult) = {
@@ -391,6 +390,30 @@ object MultiJvmPlugin extends AutoPlugin {
(name, if (failures.nonEmpty) TestResult.Failed else TestResult.Passed)
}
+ /**
+ * Waits for the test JVMs and reports one suite result per JVM, with the
test counts taken from
+ * the ScalaTest summary line that each JVM prints, so sbt can show real
pass/fail totals.
+ */
+ def testResults(
+ name: String,
+ processes: Seq[(String, Process, TestCountingLogger)],
+ log: Logger): (String, sbt.TestResult, Map[String, SuiteResult]) = {
+ val (_, overall) =
+ processExitCodes(name, processes.map { case (testClass, process, _) =>
(testClass, process) }, log)
+ val events = processes.map {
+ case (testClass, process, counter) =>
+ val exitCode = process.exitValue()
+ val suiteResult = counter.counts match {
+ case Some(counts) if exitCode == 0 || counts.result !=
TestResult.Passed => counts
+ // the JVM failed without reporting a failed test, e.g. it crashed
or an afterAll failed
+ case Some(counts) => counts + SuiteResult.Error
+ case None => if (exitCode == 0) SuiteResult.Empty else
SuiteResult.Error
+ }
+ testClass -> suiteResult
+ }.toMap
+ (name, overall, events)
+ }
+
def multiNodeExecuteTestsTask: Def.Initialize[sbt.Task[Tests.Output]] =
Def.task {
val (_jarName, (hostsAndUsers, javas), targetDir) =
multiNodeWorkAround.value
runMultiNodeTests(
@@ -471,7 +494,7 @@ object MultiJvmPlugin extends AutoPlugin {
}
Tests.Output(
Tests.overall(results.map(_._2)),
- Map.empty,
+ results.flatMap(_._3).toMap,
results.map(result => Tests.Summary("multi-jvm", result._1)))
}
@@ -488,7 +511,7 @@ object MultiJvmPlugin extends AutoPlugin {
javas: IndexedSeq[String],
targetDir: String,
createLogger: String => Logger,
- log: Logger): (String, sbt.TestResult) = {
+ log: Logger): (String, sbt.TestResult, Map[String, SuiteResult]) = {
val logName = "* " + name
log.info(logName)
val classesHostsJavas = getClassesHostsJavas(classes, hostsAndUsers,
javas, defaultJava)
@@ -503,7 +526,7 @@ object MultiJvmPlugin extends AutoPlugin {
val processes = classesHostsJavas.zipWithIndex.map {
case ((testClass, hostAndUser, java), index) => {
val jvmName = "JVM-" + (index + 1)
- val jvmLogger = createLogger(jvmName)
+ val jvmLogger = new TestCountingLogger(createLogger(jvmName))
val className = multiSimpleName(testClass)
val optionsFile = (srcDir ** (className + ".opts")).get.headOption
val optionsFromFile = optionsFile
@@ -527,12 +550,13 @@ object MultiJvmPlugin extends AutoPlugin {
targetDir,
jvmLogger,
connectInput,
- log))
+ log),
+ jvmLogger)
}
}
- processExitCodes(name, processes, log)
+ testResults(name, processes, log)
} else {
- syncResult
+ (syncResult._1, syncResult._2, Map.empty[String, SuiteResult])
}
}
@@ -592,3 +616,26 @@ object MultiJvmPlugin extends AutoPlugin {
}.unzip
}
}
+
+/**
+ * Forwards to the underlying logger and records the counts from the ScalaTest
summary line
+ * (`Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0`) printed
by a test JVM.
+ */
+final class TestCountingLogger(underlying: Logger) extends Logger {
+ private val AnsiEscape = "\u001b\\[[0-9;]*m".r
+ private val Summary = "Tests: succeeded (\\d+), failed (\\d+), canceled
(\\d+), ignored (\\d+), pending (\\d+)".r
+
+ @volatile var counts: Option[SuiteResult] = None
+
+ def trace(t: => Throwable): Unit = underlying.trace(t)
+ def success(message: => String): Unit = underlying.success(message)
+ def log(level: Level.Value, message: => String): Unit = {
+ val msg = message
+ Summary.findFirstMatchIn(AnsiEscape.replaceAllIn(msg, "")).foreach { m =>
+ val Seq(succeeded, failed, canceled, ignored, pending) = (1 to
5).map(m.group(_).toInt)
+ val result = if (failed > 0) TestResult.Failed else TestResult.Passed
+ counts = Some(new SuiteResult(result, succeeded, failed, 0, 0, ignored,
canceled, pending))
+ }
+ underlying.log(level, msg)
+ }
+}
diff --git
a/remote-tests/src/multi-jvm/scala/org/apache/pekko/remote/artery/aeron/AeronStreamMultiNodeSpec.scala
b/remote-tests/src/multi-jvm/scala/org/apache/pekko/remote/artery/aeron/AeronStreamMultiNodeSpec.scala
index 138d2d1e99..693217d550 100644
---
a/remote-tests/src/multi-jvm/scala/org/apache/pekko/remote/artery/aeron/AeronStreamMultiNodeSpec.scala
+++
b/remote-tests/src/multi-jvm/scala/org/apache/pekko/remote/artery/aeron/AeronStreamMultiNodeSpec.scala
@@ -41,7 +41,7 @@ abstract class AeronStreamMultiNodeSpec(config:
MultiNodeConfig) extends MultiNo
val n = node(roleName)
val port = MultiNodeSpec.udpPort match {
case None =>
- system.actorSelection(n / "user" / "updPort") ! UdpPortActor.GetUdpPort
+ system.actorSelection(n / "user" /
"updPort").tell(UdpPortActor.GetUdpPort, testActor)
expectMsgType[Int]
case Some(p) => p
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
