tgravescs commented on pull request #29002: URL: https://github.com/apache/spark/pull/29002#issuecomment-662473704
Yeah I think this is ok, especially since normally if your plugin is less then heartbeat time there shouldn't be any difference. I would like to test this with a couple scenarios though. When we first put this in, I brought up that the plugin could block other executor functionality by running a long time, one thought was we could put plugin into another thread so it didn't block the executor. I think the thought at the time was if you have a long running executor plugin, the user should handle that by doing it in another thread or something. Now you could make the argument you have something long running in the plugin and you want to block the executor for other initialization. The heartbeat is 10seconds by default and I thought the timeout was the network timeout by default (120s), which seems like a really long time. I guess if you change those timeouts, but even then seems like a long time. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
