HyukjinKwon opened a new pull request, #260: URL: https://github.com/apache/spark-connect-rust/pull/260
### What changes were proposed in this pull request? The live test `test_explain_prints_with_python_print_as_pyspark` now waits for the first progress of its streaming query, rather than for `processAllAvailable()`, before it checks `StreamingQuery.explain`. ### Why are the changes needed? `processAllAvailable()` returns only when the source of the query has no new rows. A `rate` source makes a row every second. On a slow machine each batch took longer than that: on a local server under load, a batch took about 3.5 seconds for 4 rows, so there were always new rows. The call never returned, and the test hung the live suite. After its first progress, the query has the plan of a batch to explain. ### Does this PR introduce _any_ user-facing change? No. It changes a test only. ### How was this patch tested? The test passes with this client and with official PySpark 4.3.0, against a local Spark 4.3.0 server under the load where it hung before, in about 10 seconds. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
