cloud-fan commented on issue #24094: [SPARK-27162][SQL] Add new method 
getOriginalMap in CaseInsensitiveStringMap
URL: https://github.com/apache/spark/pull/24094#issuecomment-473142525
 
 
   AFAIK hadoop conf can be set in 3 ways:
   1. global level, via `SparkContext.hadoopConfiguration`
   2. session level, via `SparkSession.conf`
   3. operation level, via `DataFrameReader/Writer.option`
   
   1 and 2 are fine, as they are case sensitive. The problem is 3, as data 
source v2 treats options as case-insensitive.
   
   There are 2 solutions I can think of
   1. Do not support operation level hadoop conf for data source v2.
   2. Keep the original case sensitive map.
   
   I think 2 is more reasonable, which is this PR trying to do.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to