Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/17527#discussion_r110392351
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala
---
@@ -26,11 +28,12 @@ package org.apache.spark.sql.catalyst.util
class CaseInsensitiveMap[T] private (val originalMap: Map[String, T])
extends Map[String, T]
with Serializable {
- val keyLowerCasedMap = originalMap.map(kv => kv.copy(_1 =
kv._1.toLowerCase))
+ val keyLowerCasedMap = originalMap.map(kv => kv.copy(_1 =
kv._1.toLowerCase(Locale.ROOT)))
--- End diff --
Yes, hardly possible really.. not user but a developer could use that
Turkish character for a external data source.. (external data source could use
this up to my knowledge, see
[interfaces.scala#L73-L74](https://github.com/apache/spark/blob/04ee8cf633e17b6bf95225a8dd77bf2e06980eb3/sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala#L73-L74)
and
[DataSource.scala#L306](https://github.com/apache/spark/blob/096df6d933c5326e5782aa8c5de842a0800eb369/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala#L306)).
For example, XML datasource is extending this -
[DefaultSource.scala#L30](https://github.com/databricks/spark-xml/blob/master/src/main/scala/com/databricks/spark/xml/DefaultSource.scala#L30).
Though.. one case I finally come up is use the map as a kind of arbitrary key
and value.
Someone opened a PR to Spark XML
`https://github.com/databricks/spark-xml/pull/247` and this supports to use
arbitrary key and value for writing out attributes to the root element.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
