andygrove opened a new issue, #5570:
URL: https://github.com/apache/datafusion-comet/issues/5570
`SQLQueryTestSuite.ignoreList` in the Spark test diffs carries two entries
with no linked tracking issue:
```scala
// Comet: ORDER BY column has ties; row order is non-deterministic when
// running with high parallelism. Tracked for restoration once Comet
// produces stable ordering for these queries.
"replacing-missing-expression-with-alias.sql",
"in-set-operations.sql",
```
Present in `dev/diffs/4.1.3.diff` and `dev/diffs/4.2.0.diff`.
### Why this is worse than it looks
`ignoreList` is consulted in `createScalaTestCase` **before** any
Comet-specific or test-type dispatch:
```scala
protected def createScalaTestCase(testCase: TestCase): Unit = {
if (ignoreList.exists(t =>
testCase.name.toLowerCase(Locale.ROOT).contains(t.toLowerCase(Locale.ROOT)))) {
ignore(testCase.name) { /* Do nothing */ }
} else testCase match {
```
so it registers a ScalaTest `ignore` unconditionally. The two files are
therefore skipped in the `ENABLE_COMET=false` baseline as well, and in both the
regular and analyzer variants: roughly 47 result queries plus three setup
statements never run at all, in any mode. Nothing is being verified for them,
not even Spark's own behaviour.
The same problem applied to `thetasketch.sql` / `tuplesketch.sql`; those
were moved to the input-file `--SET spark.comet.enabled = false` header in
#4950, which keeps every query running and compared against the golden file.
### What to do
1. Investigate the actual non-determinism. The stated cause is ORDER BY ties
under parallelism; confirm whether it still reproduces on `main`, and on which
queries.
2. If it does, replace the global `ignoreList` entries with the `--SET
spark.comet.enabled = false` header used by the other opted-out files, so the
Spark baseline and the golden comparison are preserved. Scope it to the
affected queries if that is possible without regenerating goldens.
3. If it does not reproduce, remove the entries.
Surfaced in review of #4950 by @sunchao. Related: #5569 (stale exclusions
whose tracking issues are already closed).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
