maropu commented on a change in pull request #32482:
URL: https://github.com/apache/spark/pull/32482#discussion_r630805446
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -1090,6 +1090,17 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val CAN_CHANGE_CACHED_PLAN_OUTPUT_PARTITIONING =
+ buildConf("spark.sql.optimizer.canChangeCachedPlanOutputPartitioning")
+ .internal()
+ .doc(s"When false, some configs are disabled during executing cache plan
that is to avoid " +
Review comment:
How about this?
```
Whether to forcibly enable some optimization rules that can change the
output partitioning
of a cached query when executing it for caching. If it is set to true,
queries may need an extra shuffle to read the cached data. This
configuration is disabled by default.
Currently, the optimization rules enabled by this configuration are
${ADAPTIVE_EXECUTION_ENABLED.key} and ${AUTO_BUCKETED_SCAN_ENABLED.key}.
```
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala
##########
@@ -1554,4 +1554,67 @@ class CachedTableSuite extends QueryTest with
SQLTestUtils
assert(!spark.catalog.isCached(viewName))
}
}
+
+ test("SPARK-35332: Make cache plan disable configs configurable - check
AQE") {
+ withSQLConf(SQLConf.SHUFFLE_PARTITIONS.key -> "2",
+ SQLConf.COALESCE_PARTITIONS_MIN_PARTITION_NUM.key -> "1",
+ SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "true") {
+
+ withTempView("t1", "t2", "t3") {
+ withCache("t1", "t2", "t3") {
Review comment:
`withTempView` drops caches, too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
