Johnny Walker created SPARK-49321:
-------------------------------------
Summary: Additional JDBC dialects not registered in executor
Key: SPARK-49321
URL: https://issues.apache.org/jira/browse/SPARK-49321
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.5.2, 3.5.1, 3.5.0, 4.0.0
Reporter: Johnny Walker
While working on a Spark application, I created a custom JDBC dialect and
registered before creating a {{SparkSession}}. Spark did not use the dialect's
{{JdbcSQLQueryBuilder}}, however, so I investigated further. I discovered that
{{JDBCRDD}} [repeats the dialect
resolution|https://github.com/apache/spark/blame/fe2174d8caaf6f9474319a8d729a2f537038b4c1/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala#L177],
and this runs on the executor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
