otterc commented on pull request #30844: URL: https://github.com/apache/spark/pull/30844#issuecomment-748746063
> IMHO, `connectionCreationTimeout` seems to be not a common usage. But `sessionTimeout` and `connectionTimeout` are always used as a pair in RPC environments. Just stating the reason for why we went with creating `connectionCreationTimeout` instead of `idle/sessionTimeout`. We wanted to reduce the timeout for creating connections. Ideally this should be much lower and for our clusters we have set this to 30s. However, the default value for `connectionTimeout` is high which makes sense because it is also used as idleTimeout and since shuffle services can be overloaded, they may take longer to respond. The problem with creating a separate config for `idle/sessionTimeout` and **not** for `connectionCreation` is that some users (frameworks) may have also over-written the value of `connectionTimeout` to a much higher value. So, even if want to enforce all these jobs to use smaller timeouts for creating connections, they will still be using a larger timeouts for that. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
