nick-boss-tech opened a new pull request, #5106: URL: https://github.com/apache/solr/pull/5106
🤖 *AI text below* 🤖 *(posted on behalf of Nick Shanin)* https://issues.apache.org/jira/browse/SOLR-16630 ## What happens today **The test can stop the PULL replica while it is still adding documents through that replica. The add then fails with an exception the test does not catch.** This is a reopened ticket, not a new one. The failure is intermittent. In a failing CI run, `client.add` threw a `SolrServerException` whose cause was a `ClosedChannelException`. It happened after the test restarted the NRT replica. The add loop sends its documents through the PULL replica's client ([test](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/core/src/test/org/apache/solr/search/TestCoordinatorRole.java#L222-L222)). The test stops the PULL replica after a delay that is chosen at random when the test starts ([test](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/core/src/test/org/apache/solr/search/TestCoordinatorRole.java#L242-L242), [test](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/core/src/test/org/apache/solr/search/TestCoordinatorRole.java#L261-L266)). The stop does not wait for the adds to finish. The add loop catches only `SolrException` ([test](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/core/src/test/org/apache/solr/search/TestCoordinatorRole.java#L315-L315)). `SolrServerException` extends `Exception`, not `SolrException` ([class](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/solrj/src/java/org/apache/solr/client/solrj/SolrServerException.java#L24-L24)). So the exception is not caught, and the test fails. The likely cause is that the stop closes the client the loop is still using. This is not proven. `JettySolrRunner.stop()` closes its cached client first ([class](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/test-framework/src/java/org/apache/solr/embedded/JettySolrRunner.java#L616-L617)). The client the test holds comes from that cache ([class](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/test-framework/src/java/org/apache/solr/embedded/JettySolrRunner.java#L847-L852)). ## What this change does **The test now waits for its first successful add before it stops the PULL replica. This is a test-only change.** The fixed delay still runs first. After it, the test waits for the add loop to finish its first add and commit ([test](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/core/src/test/org/apache/solr/search/TestCoordinatorRole.java#L243-L243), [test](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/core/src/test/org/apache/solr/search/TestCoordinatorRole.java#L262-L264), [test](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/core/src/test/org/apache/solr/search/TestCoordinatorRole.java#L311-L313)). The wait is at most two minutes. If no add has succeeded by then, the test logs a warning and stops the PULL replica anyway. Behavior change: if no add ever succeeds, the test reaches its failure up to two minutes later than before. The diff touches only the test file and a changelog entry (see Changelog below). ## Proof **The test fails on the base code with the recorded failure, and passes six runs with this change.** The test is `TestCoordinatorRole.testNRTRestart`. On the base code (the commit this branch starts from), one run on 2026-10-11 (UTC) failed: the add threw a `SolrServerException` caused by a `ClosedChannelException`, after the test had stopped the PULL replica and before the add loop's first add had succeeded. That is the failure this change prevents, because the stop now waits for the first successful add. With this change, the test passed six runs on 2026-10-10 (UTC). Each run: 1 test, 0 failures, 0 errors. Those runs were at `9bea59741ac30ffd11ed11be5269ea025cf17f68`; the only commit since corrects the changelog title, and no code changed. ## Limits **Several ways this test can fail are not covered.** - A pooled connection from the test client to the PULL replica can go stale with no stop at all. This change does not address that. - The add loop still catches only `SolrException` ([test](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/solr/core/src/test/org/apache/solr/search/TestCoordinatorRole.java#L315-L315)). Widening that catch was not done. - The failure is intermittent. Six passing runs do not prove the fix holds in every run. - Only the test changes. A cause in product code would not be addressed here. Changelog: [`changelog/unreleased/SOLR-16630.yml`](https://github.com/nick-boss-tech/solr/blob/9c77a4db09d7a890f62dc9a9ffb2bf5613245da7/changelog/unreleased/SOLR-16630.yml#L1-L10) ### AI assistance AI agents assisted with research, implementation, review, and drafting. Nick Shanin directed the work and takes responsibility for this contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
