aleksandar-trajkovic-db commented on code in PR #59671:
URL: https://github.com/apache/spark/pull/59671#discussion_r4415201262
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCTable.scala:
##########
@@ -60,13 +60,13 @@ case class JDBCTable(
override def newScanBuilder(options: CaseInsensitiveStringMap):
JDBCScanBuilder = {
val mergedOptions = new JDBCOptions(
- jdbcOptions.parameters.originalMap ++
options.asCaseSensitiveMap().asScala)
Review Comment:
That is a pre-existing bug, that only now starts to bite. JdbcOptions should
be case insensitive, but originalMap made it case sensitive and the whole merge
case sensitive. That made is possible for two keys with different casing to get
into the map and and which one would get picked dependent on the hash of the
key, which makes it basically random. Now this becomes an issue, because we
need to pin our option, so Spark inserts a specific key and if the user set the
same option with eg all lowercase, it would not be certain that the user's
option would get picked. With this, user provided options get merged later and
always win.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
