yew1eb opened a new issue, #1769:
URL: https://github.com/apache/auron/issues/1769
## Background and Motivation
Our current CI uses auron-project/tpcds-validator, which runs TPC-DS queries
and passes them solely on the absence of exceptions. This is too permissive and
misses critical regressions:
- **Result correctness**: Query results diverging from vanilla Spark.
- **Plan stability**: Unintended loss of native operators, unstable plans,
or fallback.
Gating only on "no crash" in TPC-DS tests isn't enough as Auron evolves.
We need CI to enforce result equivalence and plan stability to protect core
optimizations.
## Proposal
Introduce a standalone integration test module **auron-it** that:
- Executes each TPC-DS query in **two modes**: vanilla Spark and Auron.
- Compares results (exact match or small floating-point tolerance).
Mismatches fail the CI and produce concise, readable diffs.
- Validates Auron’s physical plans against versioned “gold files” to enforce
plan stability and detect operator loss or unintended fallbacks.
## Maintainability and CI Integration
- Fully decouples TPC-DS tests from unit tests by placing them in the
dedicated **auron-it** module — easier to maintain and evolve.
- Tests run only when the **TPCDS_DATA_DIR** env var is set, keeping them
optional yet easy to enable in CI and convenient for local development and
debugging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
