Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/13769#discussion_r67749047
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala
---
@@ -43,8 +43,128 @@ import org.apache.spark.unsafe.types.UTF8String
* Replaces generic operations with specific variants that are designed to
work with Spark
* SQL Data Sources.
*/
-private[sql] object DataSourceAnalysis extends Rule[LogicalPlan] {
+private[sql] case class DataSourceAnalysis(conf: CatalystConf) extends
Rule[LogicalPlan] {
+
+ def resolver: Resolver = {
+ if (conf.caseSensitiveAnalysis) {
+ caseSensitiveResolution
+ } else {
+ caseInsensitiveResolution
+ }
+ }
+
+ // The access modifier is used to expose this method to tests.
+ private[sql] def convertStaticPartitions(
--- End diff --
Why is this rule only applied to HadoopFsRelation? It would be fine to
change Hive to write using this rule as well and we would need fewer
relation-specific rules.
This isn't a huge issue, but I'm concerned about the proliferation of fixes
for either Hive or data sources that are never applied to the other. We should
be consolidating the implementation wherever possible.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
