pan3793 opened a new issue, #5373: URL: https://github.com/apache/kyuubi/issues/5373
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before creating - [X] I have searched in the [task list](https://github.com/orgs/apache/projects/296) and found no similar tasks. ### Mentor - [ ] I have sufficient knowledge and experience of this task, and I volunteer to be the mentor of this task to guide contributors to complete the task. ### Skill requirements - Basic knowledge of Scala programming language - Familiar with the architect of Apache Kyuubi ### Background and Goals In `kyuubi.engine.share.level` non-CONNECTION mode, how many engines the Kyuubi server starts is determined by `kyuubi.engine.pool.size`. The engine selection policy is determined by `kyuubi.engine.pool.selectPolicy`. The default is `RANDOM`, which means randomly selecting a number within the engine pool size range and checking whether the engine corresponding to this number has been created. If not, create one. When kyuubi.engine.pool.size is a relatively large value, it may randomly create many engines. In addition, `kyuubi.session.engine.idle.timeout` configures a relatively large idle time, so that once the engine is created, it will be slower to release the engine. This selection strategy is a bit of a waste of resources, especially since it takes a while to start the spark cluster mode. ### Implementation steps We can expose some information about the engine load in the Engine. When the server selects the engine, it will decide whether to use a new engine or reuse the old engine based on the load. ### Additional context Original reporter is @cxzl25 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
