[
https://issues.apache.org/jira/browse/LIVY-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856646#comment-17856646
]
wangdengshan commented on LIVY-1003:
------------------------------------
As you know, it will mainly get stuck at the startup of RemoteDriver, which is
the pulling up of yarn. On the one hand, the waiting time can be reduced
through configuration, and at the same time, the use of the thread pool is also
adjusted in our case.
> Interactive session - Setting large value of rsc.server.connect.timeout
> blocks other tasks
> -------------------------------------------------------------------------------------------
>
> Key: LIVY-1003
> URL: https://issues.apache.org/jira/browse/LIVY-1003
> Project: Livy
> Issue Type: Bug
> Components: RSC
> Affects Versions: 0.8.0
> Reporter: Tom Chamnongvongse
> Priority: Minor
>
> Problem:
> Livy is configured to deploy interactive sessions on YARN with
> `livy.rsc.server.connect.timeout` configured to a high value. Timeout is
> increased to allow more time for Livy session to be in YARN `ACCEPTED` state
> to prevent Livy server from killing the YARN app within the default timeout
> of 90 seconds.
> Until the app is in YARN `RUNNING` state, it takes up a thread in Scala's
> global execution context - I think due to
> https://github.com/apache/incubator-livy/blob/v0.8.0-incubating/server/src/main/scala/org/apache/livy/server/interactive/InteractiveSession.scala#L474.
> Creating too many of these sessions that are stuck in `ACCEPTED` state
> causes other tasks that use that global execution context to be queued up.
> How to reproduce:
> 1. Set `livy.rsc.server.connect.timeout` to something high like 24h.
> 2. Create enough interactive livy sessions in YARN so that they are queued in
> ACCEPTED state. The number of sessions that are stuck in ACCEPTED state
> should be equal to global execution context [thread pool
> size|https://docs.scala-lang.org/overviews/core/futures.html#the-global-execution-context]
> (Runtime.availableProcessors)
> 3. Try to delete a session using DELETE /sessions/{sessionId} and it should
> hang until one of the sessions is no longer stuck in ACCEPTED state.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
