Github user cammachusa commented on the issue:
https://github.com/apache/spark/pull/18423
Hi @cloud-fan , thanks for you quick response.
The failure test cases is very straight forward, and can be found in the
ticket's description: https://issues.apache.org/jira/browse/SPARK-21158.
To summary:
1. Set your spark sql to case insensitive: spark.sql("SET
spark.sql.caseSensitive=false");
2. Create and temp table/view e.g. CameraAndModels
3. Then, check if your CameraAndModes exists in spark.catalog.listTables().
It will return false, while you know for sure that your table does exist.
=> The reason is the formatTable() in SessionCatalog.scala convert your
table name to lowercase (btw, by default case sensitive is set to false)
To your second point:
The GlobalViewManager behaves just like the "LocalView", it also call the
formatTable() before creating a view. Below is the code in SessionCatalog.scala
for your references:
def createGlobalTempView(
name: String,
viewDefinition: LogicalPlan,
overrideIfExists: Boolean): Unit = {
globalTempViewManager.create(formatTableName(name), viewDefinition,
overrideIfExists)
}
I fixed this issue by simply return the table name itself from the
formatTable(), without convert to lowercase. By doing this way, I don't have to
clean up the callers which is erroneous (more than 20 references to this
function). And provide a convenience way to check if a table exists, and do
care about case sensitive at this point.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
