Admaing opened a new pull request, #59148:
URL: https://github.com/apache/spark/pull/59148
### What changes were proposed in this pull request?
Adds integration coverage for the JDBC external-engine syntax error
classification
introduced by SPARK-52184 and tightened for PostgreSQL by SPARK-59336, and
fixes the
dialects the new checks expose.
- `SharedJDBCIntegrationSuite` gets two shared checks: reading a
non-existent table and
reading a table as a user without the read privilege. Both require the
driver's own
`SQLException` to surface instead of `JDBC_EXTERNAL_ENGINE_SYNTAX_ERROR`.
Dialect suites
plug in the driver's SQLSTATE and, for the privilege check, a restricted
user.
- The two SPARK-59336 tests in `PostgresIntegrationSuite` are folded into
the shared
framework. PostgreSQL, SQL Server and Oracle set up a restricted user.
MySQL is left to
SPARK-58515 and DB2 cannot create a restricted user over JDBC (its users
are OS users);
both are documented in place. The StarRocks check is disabled because
StarRocks reports a
missing table as error 1064 with SQLSTATE 42000, the SQLSTATE it also uses
for syntax
errors, and it shares `MySQLDialect`.
- `DB2Dialect` only classifies SQLSTATE 42601 as a syntax error. Class 42
also covers 42704
(undefined object) and 42501 (insufficient privilege), which were
misclassified.
- `OracleDialect` keeps the ORA errors that share SQLSTATE 42000 but are not
syntax errors
(missing object, non-selectable object, missing privilege) out of the
classification.
- Unit tests for the two dialect classifiers in `JDBCSuite`.
### Why are the changes needed?
`JdbcDialect.isSyntaxErrorBestEffort` promises that `true` means a syntax
error. Class 42
also covers missing tables, missing privileges and missing columns, so those
failures were
reported as `JDBC_EXTERNAL_ENGINE_SYNTAX_ERROR`, hiding what actually went
wrong. SPARK-59336
did this for PostgreSQL; this PR covers the remaining dialects and moves the
checks into the
shared integration suite so every dialect is covered and regressions are
caught.
### Does this PR introduce _any_ user-facing change?
Yes. With DB2 and Oracle, errors that are not syntax errors, such as a
missing table or a
missing privilege, are no longer reported as
`JDBC_EXTERNAL_ENGINE_SYNTAX_ERROR`; the
original driver exception is thrown instead. Syntax errors are reported as
before.
### How was this patch tested?
- `build/sbt -Pdocker-integration-tests
docker-integration-tests/Test/compile`
- `build/sbt "sql/testOnly org.apache.spark.sql.jdbc.JDBCSuite"` (157 tests)
- `build/sbt "sql/testOnly org.apache.spark.sql.jdbc.PostgresDialectSuite"`
- `sql/scalastyle`, `sql/Test/scalastyle`,
`docker-integration-tests/Test/scalastyle`
- The new docker integration checks need `ENABLE_DOCKER_INTEGRATION_TESTS=1`
and the
database images, so they were not executed locally.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: DeepSeek Harness (deepseek-v4.1-flash)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
