LuciferYang opened a new issue, #12856:
URL: https://github.com/apache/gluten/issues/12856
### What happens
`GlutenHiveSQLQuerySuite` creates a persistent Hive table, runs its
assertions, and only then drops it. The drop is a plain statement after the
`withSQLConf` block, not inside `withTable` or a `finally`, so any failure in
between skips it. Two cases are written this way, and they also mix two cleanup
paths: a `DROP TABLE IF EXISTS` at the top of the body and a
`spark.sessionState.catalog.dropTable(...)` at the bottom.
- `testGluten("hive orc scan")` (`test_orc`), e.g.
`gluten-ut/spark35/src/test/scala/org/apache/spark/sql/hive/execution/GlutenHiveSQLQuerySuite.scala:41`
with the drop at `:52`
- `testGluten("avoid unnecessary filter binding for subfield during scan")`
(`test_subfield`), same file at `:245` with the drop at `:259`
Both exist in all five `gluten-ut/spark3x` / `spark4x` modules.
### Why it matters
The Hive suites share one `TestHive` singleton, so a leftover table outlives
the case that created it. When `checkAnswer` or `checkOperatorMatch` fails, the
table stays in the metastore and the warehouse, and later cases in the same JVM
see it. The `DROP TABLE IF EXISTS` at the top of the body only rescues a rerun
inside the same session, which is not the case CI hits.
### History
Not a new problem. `test_orc` predates the migration in every module.
`test_subfield` came from `a662f3f75` (#3300, 2023) and was copied verbatim
into 3.4/3.5/4.0/4.1 by #12840, which is where GitHub Copilot flagged it five
review rounds in a row. It was left alone there on purpose: that PR was a
migration, and rewriting a migrated case would have made it differ from both
the 3.3 original and its own neighbour in the same file.
### Suggested fix
Wrap the create/insert/assert work in `withTable("test_orc")` /
`withTable("test_subfield")` and delete both manual drops. Do the two cases
together across all five modules, otherwise the same file ends up carrying two
different cleanup styles, which is worse than the current state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
