GitHub user jose-torres opened a pull request:
https://github.com/apache/spark/pull/20896
[SPARK-23788][SS] Fix race in StreamingQuerySuite
## What changes were proposed in this pull request?
The serializability test uses the same MemoryStream instance for 3
different queries. If any of those queries ask it to commit before the others
have run, the rest will see empty dataframes. This can fail the test if q3 is
affected.
We should use one instance per query instead.
## How was this patch tested?
Existing unit test. If I move q2.processAllAvailable() before starting q3,
the test always fails without the fix.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jose-torres/spark fixrace
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20896.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20896
----
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
