JoonPark1 commented on code in PR #7227:
URL: https://github.com/apache/kyuubi/pull/7227#discussion_r2603654068
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/server/KyuubiRestFrontendService.scala:
##########
@@ -181,36 +181,51 @@ class KyuubiRestFrontendService(override val serverable:
Serverable)
@VisibleForTesting
private[kyuubi] def recoverBatchSessions(): Unit =
withBatchRecoveryLockRequired {
val recoveryNumThreads = conf.get(METADATA_RECOVERY_THREADS)
+ val recoveryBatchSize: Int = conf.get(BATCH_SESSIONS_RECOVERY_SIZE)
Review Comment:
Basically, the method`
sessionManager.getBatchSessionsToRecover(connectionUrl, offset,
recoveryNumThreads)` recovers `recoveryNumThreads` batches from metadata store
whose batches didn't reach terminal FINISHED state and reattempts it upon
kyuubi server restart and then each recovered session is opened via kyuubi
session manager: `sessionManager.openBatchSession(batchSession)` But, on second
thought, I think it's better to control behavior of how recovery thread for
each batch is done concurrently instead of constantly opening up new batch
sessions for batches that need recovery without confirming prior opened batch
sessions' associated batch job if it actually began execution (due to spinned
up compute engine by kyuubi server) @ruanwenjun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
