Ngone51 commented on PR #36716: URL: https://github.com/apache/spark/pull/36716#issuecomment-1151073534
> So last time I checked dynamic allocation in standalone mode had issues. Have this been addressed or do you not see those issues? one such issue was https://issues.apache.org/jira/browse/SPARK-30299?filter=-2 but I thought there was others. SPARK-30299 seems to be a long standing issue. For a standalone cluster with the default EXECUTOR_CORES, I think we have no idea to answer how many executors we needs but only how many cores we needs. So answering "how many cores we needs" may be a direction to fix the issue in this case but it'd of course require a lot of changes to support. That said, I think Standalone can still work well with dynamic allocation as along as the EXECUTOR_CORES is explicitly configured. So I think we may not have to block on that issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
