wombatu-kun opened a new pull request, #16416:
URL: https://github.com/apache/iceberg/pull/16416
Closes #15053
## What
`TestViews#createViewWithSubqueryExpressionInFilterThatIsRewritten` and
`...InQueryThatIsRewritten` were skipped for the Hive view catalog
(`SPARK_WITH_HIVE_VIEWS`) by an `assumeThat` that blamed a `FileInputFormat`
instantiation error. This re-enables both tests for the Hive view catalog on
Spark 3.5, 4.0 and 4.1.
## Why
Investigation (see #15053) shows the Hive-backed subquery view itself works
— creating the view, reading it, and the catalog/namespace SQL rewrite all
pass. The only failing step was the cross-catalog negative assertion: it
switches to `spark_catalog` and expects the unqualified table to be
unresolvable. That premise does not hold for the Hive view catalog, because it
shares its Hive Metastore with Spark's built-in `spark_catalog` — the Iceberg
table is resolvable there, and a native Hive read of it then fails at execution
time (Iceberg deliberately writes a placeholder InputFormat when
`engine.hive.enabled` is false, since a native read of an Iceberg table would
otherwise return silently-wrong results). So this is a test-correctness issue,
not an Iceberg view defect, and the abstract-InputFormat behavior is
intentional and out of scope.
## How
- Remove the blanket `assumeThat(...).isNotEqualTo("hive")` so both tests
run for the Hive view catalog.
- Branch the cross-catalog negative assertion: for the Hive view catalog
assert the raw SQL fails at execution with `SparkException` (table found in the
shared metastore but not natively readable); otherwise keep the existing
`AnalysisException` not-found assertion.
- Applied identically to `spark/v3.5`, `spark/v4.0` and `spark/v4.1`.
(`spark/v3.4` never had the Hive view catalog parameter, so it is unchanged.)
The Hive-specific assertion uses the top-level
`org.apache.spark.SparkException`, which is stable across Spark 3.5/4.0/4.1
(the underlying cause differs by version and is intentionally not asserted on).
## Testing
`createViewWithSubqueryExpressionInFilterThatIsRewritten` and
`createViewWithSubqueryExpressionInQueryThatIsRewritten` now pass for all
parameters, including `SPARK_WITH_HIVE_VIEWS`, on Spark 3.5, 4.0 and 4.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
