Github user erfangc commented on a diff in the pull request:
https://github.com/apache/spark/pull/13818#discussion_r73807925
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala
---
@@ -298,6 +298,7 @@ case class InsertIntoHiveTable(
// Invalidate the cache.
sqlContext.sharedState.cacheManager.invalidateCache(table)
+
sqlContext.sessionState.catalog.invalidateTable(table.catalogTable.identifier)
--- End diff --
Hi @mallman does this change cause new sessions (ex: external App to the
ThriftServer via JDBC) to not see the cached tables? I noticed this in the
released version 2.0.0 whereby `CACHE TABLE` in one session has no effect on
new sessions. Future SQL statements are still reading the underlying Parquet
files from Disk (as evidenced by tasks being `NODE_LOCAL` and `RACK_LOCAL`
instead of `PROCESS_LOCAL`). Sorry if this question is unrelated to your patch,
but this became a major issue in 2.0.0 for us, where as in 1.6.2 we do not have
an issue.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
