Github user jose-torres commented on the issue:
https://github.com/apache/spark/pull/20225
The hang in test build #85939 was a test issue in ContinuousStressSuite,
which I could reproduce locally by bumping up the rows per second.
When the rate of incoming data is too high, the query execution *does* make
progress - but the answer checking operation in the test is expensive. And the
rate source was continuing to run as it happened. Since the executors are
local, an overload means the answer check will take unreasonably long to finish.
I've fixed this by stopping the rate source before checking the answer in
the stress tests.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
