KnightChess created HUDI-7906:
---------------------------------
Summary: improve the parallelism deduce in rdd write
Key: HUDI-7906
URL: https://issues.apache.org/jira/browse/HUDI-7906
Project: Apache Hudi
Issue Type: Improvement
Reporter: KnightChess
Assignee: KnightChess
as [https://github.com/apache/hudi/issues/11274] and
[https://github.com/apache/hudi/pull/11463] describe, there has two case
question.
# if the rdd is input rdd without shuffle, the partitiion number is too bigger
or too small
# user need can not control it easy
## in some case user can set `spark.default.parallelism` change it.
## in some case user can not change because hard-code
## and in spark, the better way is use `spark.default.parallelism` or
`spark.sql.shuffle.partitions` can control it, other is advanced in hudi.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
