I ran the smoketester. Two tests failed. I had Claude investigate both, sharing below. I have yet to check wether people have reported and/or fixed them:
----- Claude Opus: Both failures are timing flakes, not regressions in the release artifacts. Re-running with the smoke tester's seed (`-Ptests.seed=FE4CEBEDABF758B2`) on branch_10_1, **PlacementPluginIntegrationTest reproduces, but RecoveryAfterSoftCommitTest passed**. ### 1. `PlacementPluginIntegrationTest.testAttributeFetcherImpl` — "systemLoadAverage 2" - `NodeMetricImpl.SYSLOAD_AVG` reads `jvm_system_cpu_utilization`. That metric comes from OTel's JMX gauge (`SystemCpu`), which reads `OperatingSystemMXBean.getCpuLoad()` and **skips the data point unless the value is `>= 0`**. - On macOS, `getCpuLoad()` returns **NaN** if it's called again before the host CPU tick counters move. A small probe confirmed this: - In a tight loop, 199,999 of 200,000 calls returned NaN. - With 3 threads calling at the same moment (like the 3 nodes in this test's single JVM), 595 of 600 calls returned NaN. - So when nodes scrape metrics close together, some of them don't report the metric at all, and the assertion fails. I only confirmed this on macOS; I haven't checked whether the Linux JVM can return NaN the same way. The earlier fix, 319b70d24b3 (the metric rename), didn't address this. - **Fix options:** - Make the test tolerate an absent value or retry the fetch. - Have `OtelRuntimeJvmMetrics` prefer the JFR source (`OverallCpuLoadHandler`) for this metric, which always records a value. That handler updates once per second, and its value starts at 0. ### 2. `RecoveryAfterSoftCommitTest.test` — `ClosedChannelException` on an add during the partition - With this seed, `createCloudClient` randomly set `updatesToLeaders` to false. So the add was sent to `replica_n3` on port 50127, the non-leader replica whose `SocketProxy` the test had just closed to simulate a network partition. - The client reused a pooled HTTP/2 session through that proxy. Jetty had already marked the request "committed" (headers written) when the session shut down. Under SOLR-18401's logic, that counts as possibly sent, so `LBSolrClient` didn't retry on the leader and threw instead. - If the client had opened a fresh connection instead, it would have been refused, which is retried, and the test would pass. Whether that happens depends on timing, which is why it passed when I re-ran it. - This is a test-design weakness: the test shouldn't send updates to a node it has partitioned. Forcing updates to go to leaders only in this test would fix it. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
