Manu Zhang created SPARK-39344:
----------------------------------
Summary: Only disable bucketing when autoBucketedScan is enabled
if bucket columns are not in scan output
Key: SPARK-39344
URL: https://issues.apache.org/jira/browse/SPARK-39344
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.3.0
Reporter: Manu Zhang
Currently, bucketing was disabled when bucket columns are not in scan output
after https://github.com/apache/spark/pull/27924. It break existing
applications whose input size is huge by creating too many FilePartitions and
causing driver hang. And it cannot be switched off. This is to propose merging
the rule into DisableUnnecessaryBucketedScan.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
