[
https://issues.apache.org/jira/browse/SPARK-21682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120959#comment-16120959
]
DjvuLee commented on SPARK-21682:
-
Yes, our company also faced with this scalability problem, the driver
[
https://issues.apache.org/jira/browse/SPARK-21682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120537#comment-16120537
]
Ryan Williams commented on SPARK-21682:
---
bq. But do you really need to create so many partitions?
[
https://issues.apache.org/jira/browse/SPARK-21682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120415#comment-16120415
]
Shixiong Zhu commented on SPARK-21682:
--
I agree that driver is a bottleneck. I already saw several
[
https://issues.apache.org/jira/browse/SPARK-21682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120409#comment-16120409
]
Ryan Williams commented on SPARK-21682:
---
Interestingly, I thought the {{updatedBlockStatuses}}
