[
https://issues.apache.org/jira/browse/SPARK-26209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-26209:
----------------------------------
Affects Version/s: (was: 2.4.0)
3.0.0
> Allow for dataframe bucketization without Hive
> ----------------------------------------------
>
> Key: SPARK-26209
> URL: https://issues.apache.org/jira/browse/SPARK-26209
> Project: Spark
> Issue Type: Improvement
> Components: Input/Output, Java API, SQL
> Affects Versions: 3.0.0
> Reporter: Walt Elder
> Priority: Minor
>
> As a DataFrame author, I can elect to bucketize my output without involving
> Hive or HMS, so that my hive-less environment can benefit from this
> query-optimization technique.
>
> https://issues.apache.org/jira/browse/SPARK-19256?focusedCommentId=16345397&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16345397
> identifies this as a shortcoming with the umbrella feature in provided via
> SPARK-19256.
>
> In short, relying on Hive to store metadata *precludes* environments which
> don't have/use hive from making use of bucketization features.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
