patricker commented on a change in pull request #3511: NIFI-6175 Spark Livy -
Improving Livy
URL: https://github.com/apache/nifi/pull/3511#discussion_r342118230
##########
File path:
nifi-nar-bundles/nifi-spark-bundle/nifi-livy-controller-service/src/main/java/org/apache/nifi/controller/livy/LivySessionController.java
##########
@@ -389,9 +428,22 @@ private void manageSessions() throws
InterruptedException, IOException {
log.debug("manageSessions() Registered new session: " +
newSessionInfo);
}
} else {
- // Open one new session if there are no idle sessions
- if (idleSessions == 0) {
- log.debug("manageSessions() There are " + numSessions + "
sessions in the pool but none of them are idle sessions, creating...");
+ // If we exceeded our session pool size, look for `idle`
sessions we can shut down
+ // Two scenarios: we have no elastic pool sizing, in which
case definitely look for candidates
+ // Or, we do have elastic pool sizing, in which case we need
to make sure we are above our max
+ // pool size
+ if(idleSessions > 0 && numSessions > sessionPoolSize &&
(!elasticSessionPool || numSessions > maxSessionPoolSize)) {
Review comment:
It only stops 1 idle session at a time, that is true, but on the next run
through, which happens every few seconds, it will stop the next one, and the
next one, until it is back to where it is supposed to be.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
