[jira] [Commented] (SPARK-41386) There are some small files when using rebalance(column)

2022-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644224#comment-17644224 ] Apache Spark commented on SPARK-41386: -- User 'Juerin-Dong' has created a pull request for this

[jira] [Commented] (SPARK-41386) There are some small files when using rebalance(column)

2022-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644223#comment-17644223 ] Apache Spark commented on SPARK-41386: -- User 'Juerin-Dong' has created a pull request for this

[jira] [Commented] (SPARK-41386) There are some small files when using rebalance(column)

2022-12-06 Thread Zhe Dong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644175#comment-17644175 ] Zhe Dong commented on SPARK-41386: -- Hi. [~podongfeng]  That was my mistake. I removed it. sorry for

[jira] [Commented] (SPARK-41386) There are some small files when using rebalance(column)

2022-12-06 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644144#comment-17644144 ] Ruifeng Zheng commented on SPARK-41386: --- [~dongz] I think this ticket is irrelevant to

[jira] [Commented] (SPARK-41386) There are some small files when using rebalance(column)

2022-12-06 Thread Zhe Dong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644141#comment-17644141 ] Zhe Dong commented on SPARK-41386: --   {noformat}   if (mapStats.isEmpty ||      

[jira] [Commented] (SPARK-41386) There are some small files when using rebalance(column)

2022-12-06 Thread Zhe Dong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644131#comment-17644131 ] Zhe Dong commented on SPARK-41386: -- we may change this part to avoid files that are smaller than