j1wonpark opened a new pull request, #56447:
URL: https://github.com/apache/spark/pull/56447
### What changes were proposed in this pull request?
Implement the ARRAY / MAP / STRUCT cases in `JdbcTypeUtils`, which
previously threw
`SQLFeatureNotSupportedException`, covering `SparkConnectResultSetMetaData`
and
`SparkConnectDatabaseMetaData.getColumns`.
| | ARRAY | MAP | STRUCT |
|---|---|---|---|
| `getColumnType` | `Types.ARRAY` | `Types.JAVA_OBJECT` | `Types.STRUCT` |
| `getColumnClassName` | `scala.collection.Seq` | `scala.collection.Map` |
`org.apache.spark.sql.Row` |
| `isSigned` | `false` | `false` | `false` |
| `getPrecision` / `getScale` | 0 / 0 | 0 / 0 | 0 / 0 |
| `getColumnDisplaySize` | `Int.MaxValue` | `Int.MaxValue` | `Int.MaxValue` |
| `getColumns` COLUMN_SIZE | `null` | `null` | `null` |
Values follow the JDBC javadoc where it is explicit (precision/scale 0 and
COLUMN_SIZE
null for not-applicable types), and Hive JDBC where it is silent
(`JAVA_OBJECT` for MAP);
`getColumnClassName` reports what `getObject` actually returns. Metadata
only --
`getObject` / `getString` already work for complex values; `getArray()` is a
follow-up.
### Why are the changes needed?
JDBC tools read `ResultSetMetaData` before rendering rows, so any query
returning a
complex-typed column fails, and `getColumns` throws for an entire table
containing one
complex-typed column, breaking schema introspection (DataGrip, DBeaver).
### Does this PR introduce _any_ user-facing change?
Yes, in the unreleased master only: the methods above now return values
instead of
throwing. No existing behavior is removed.
### How was this patch tested?
New tests in `SparkConnectJdbcDataTypeSuite` (array/map/struct metadata) and
the extended
`getColumns` test in `SparkConnectDatabaseMetaDataSuite`.
build/sbt 'connect-client-jdbc/testOnly *SparkConnectJdbcDataTypeSuite
*SparkConnectDatabaseMetaDataSuite'
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Fable 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
