siying opened a new pull request, #59665:
URL: https://github.com/apache/spark/pull/59665
### What changes were proposed in this pull request?
This PR extends `SQLQueryTestSuite` so SQL golden tests can exercise
Structured Streaming flows.
The test harness now:
- parses OSS `CREATE FLOW` statements and retains their streaming logical
plans;
- recognizes test-only, case-insensitive `REFRESH FLOW` statements because
OSS Spark does not
define that parser syntax;
- runs the saved flow with `Trigger.AvailableNow` into an existing table and
reuses its checkpoint
across refreshes; and
- normalizes event-time watermark UUIDs in golden output.
It adds golden coverage for writing to a table, stream-static joins,
streaming unions, and windowed
aggregations with watermarks.
### Why are the changes needed?
SQL golden tests currently cannot execute a query containing streaming
sources because ordinary
batch execution rejects its streaming logical plan. Running the saved
`CREATE FLOW` query through
`writeStream` lets SQL tests validate Structured Streaming behavior and
inspect the resulting table
with regular SQL.
### Does this PR introduce _any_ user-facing change?
No. This only changes the SQL test harness and its test coverage.
### How was this patch tested?
The following focused tests pass:
```
JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 build/sbt \
'sql/testOnly org.apache.spark.sql.SQLQueryTestSuite -- -z "test triggered
flow parsing"'
JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 build/sbt \
'sql/testOnly org.apache.spark.sql.SQLQueryTestSuite -- -z
"streaming/available-now"'
```
The second command runs four execution golden tests and their four analyzer
tests.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: OpenAI Codex (GPT-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
