alekjarmov opened a new pull request, #59503: URL: https://github.com/apache/spark/pull/59503
### What changes were proposed in this pull request? Map SQL Server's native JSON JDBC type (`microsoft.sql.Types.JSON`, vendor type code `-159`) to Spark `StringType` in `MsSqlServerDialect`. SQL Server native `json` values are exposed to JDBC applications as text. Depending on the Microsoft JDBC driver, a native `json` column may be reported either through the standard NVARCHAR / string-compatible path or as this vendor-specific type. Spark's generic JDBC mapping does not know `-159`, so schema inference fails with `UNRECOGNIZED_SQL_TYPE`. This change recognizes `-159` in the SQL Server dialect and maps it to `StringType`. This does not parse JSON or introduce a Spark JSON data type. The payload remains a string, matching the driver's `getString` behavior. ### Why are the changes needed? Native `json` columns should have the same logical Spark type with both older and newer SQL Server JDBC drivers. Without this mapping, schema inference fails for `-159`, and affected tables can be unreadable (in some catalog paths they appear to have no columns). A common workaround is to expose the column through a view that casts it to `NVARCHAR(MAX)`, which restores standard string metadata. Handling `-159` in the dialect keeps the string representation without that workaround. ### Does this PR introduce _any_ user-facing change? Yes. SQL Server native `json` columns whose JDBC metadata reports type `-159` are inferred as `StringType` instead of failing with `UNRECOGNIZED_SQL_TYPE`. Columns already reported as NVARCHAR continue to map as they do today. ### How was this patch tested? Added a `JDBCSuite` unit test that maps `-159` / `"json"` to `StringType` and checks that `java.sql.Types.OTHER` with type name `"json"` is left unmapped. Local `sbt` could not be run in this environment because the sbt 1.13.0 launcher could not be downloaded. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor Grok 4.6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
