[
https://issues.apache.org/jira/browse/SPARK-19513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15857857#comment-15857857
]
leocook commented on SPARK-19513:
---------------------------------
Thank you,Sean Owen. coalesce means merge partitions，I want to repartition, to
get more partitios, means split partitions.
> Split partition without shuffle（means repartition locally）
> ----------------------------------------------------------
>
> Key: SPARK-19513
> URL: https://issues.apache.org/jira/browse/SPARK-19513
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Affects Versions: 3.0.0
> Reporter: leocook
>
> provide a method， that can split partition without shuffle， just repartition
> locally.
> Sample: there is a partition with too much data, it will be OOM in spark
> task,
> if we increase the num of partition, we usually call repartition，but it will
> shuffle.
> Now I want to split partitions locally, just make less data in each partition。
> !http://7xriy2.com1.z0.glb.clouddn.com/spark%20split%20partition%20no%20shuffer.png!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
