[
https://issues.apache.org/jira/browse/SPARK-31588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105530#comment-17105530
]
philipse commented on SPARK-31588:
--
Thanks Hyukjin for your advice , i will reconsider it.
[
https://issues.apache.org/jira/browse/SPARK-31588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103658#comment-17103658
]
Hyukjin Kwon commented on SPARK-31588:
--
I can't completely get the point of the physical size and
[
https://issues.apache.org/jira/browse/SPARK-31588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102414#comment-17102414
]
philipse commented on SPARK-31588:
--
yes, the block size can be controlled in HDFS.i mean we just take
[
https://issues.apache.org/jira/browse/SPARK-31588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102195#comment-17102195
]
Hyukjin Kwon commented on SPARK-31588:
--
the repartition won't set the hard limit on the size. You
[
https://issues.apache.org/jira/browse/SPARK-31588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17101775#comment-17101775
]
philipse commented on SPARK-31588:
--
For example:
if we have output 3 files,size as 10M,50M,200M,the
[
https://issues.apache.org/jira/browse/SPARK-31588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098210#comment-17098210
]
Hyukjin Kwon commented on SPARK-31588:
--
There are many other workarounds already. Can you show a
