[
https://issues.apache.org/jira/browse/HUDI-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-7906:
---------------------------------
Labels: pull-request-available (was: )
> improve the parallelism deduce in rdd write
> -------------------------------------------
>
> Key: HUDI-7906
> URL: https://issues.apache.org/jira/browse/HUDI-7906
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: KnightChess
> Assignee: KnightChess
> Priority: Major
> Labels: pull-request-available
>
> as [https://github.com/apache/hudi/issues/11274] and
> [https://github.com/apache/hudi/pull/11463] describe, there has two case
> question.
> # if the rdd is input rdd without shuffle, the partitiion number is too
> bigger or too small
> # user need can not control it easy
> ## in some case user can set `spark.default.parallelism` change it.
> ## in some case user can not change because hard-code
> ## and in spark, the better way is use `spark.default.parallelism` or
> `spark.sql.shuffle.partitions` can control it, other is advanced in hudi.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
