Repository: incubator-livy Updated Branches: refs/heads/master 4cfb6bcb8 -> 8504c641e
[LIVY-535] Fix non-atomic session creation ## What changes were proposed in this pull request? All steps, that include check for too many sessions and creating new session, should be atomic operation to avoid concurrent access of several threads to different part of this code in the same time. https://issues.apache.org/jira/browse/LIVY-535 ## How was this patch tested? If was tested manually on local virtual cluster with this different values of config option `livy.server.session.max-creation`: 1, 2 and 3 Then I run the following command to make sure that only allowed number of sessions were created in the same time and others were rejected: `for i in {1..5}; do curl -X POST --data '{"file": "/tmp/spark-examples-2.jar", "className": "org.apache.spark.examples.SparkPi"}' -H "Content-Type: application/json" http://localhost:8999/batches; done` Author: Alexey Romanenko <aromanenko....@gmail.com> Closes #129 from aromanenko-dev/LIVY-535-session-max-creation. Project: http://git-wip-us.apache.org/repos/asf/incubator-livy/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-livy/commit/8504c641 Tree: http://git-wip-us.apache.org/repos/asf/incubator-livy/tree/8504c641 Diff: http://git-wip-us.apache.org/repos/asf/incubator-livy/diff/8504c641 Branch: refs/heads/master Commit: 8504c641eae91473f0c112d7438def489484619d Parents: 4cfb6bc Author: Alexey Romanenko <aromanenko....@gmail.com> Authored: Mon Nov 26 09:21:09 2018 -0800 Committer: Marcelo Vanzin <van...@cloudera.com> Committed: Mon Nov 26 09:21:09 2018 -0800 ---------------------------------------------------------------------- .../org/apache/livy/server/SessionServlet.scala | 22 +++++++++++--------- 1 file changed, 12 insertions(+), 10 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-livy/blob/8504c641/server/src/main/scala/org/apache/livy/server/SessionServlet.scala ---------------------------------------------------------------------- diff --git a/server/src/main/scala/org/apache/livy/server/SessionServlet.scala b/server/src/main/scala/org/apache/livy/server/SessionServlet.scala index f7547c5..d62a96e 100644 --- a/server/src/main/scala/org/apache/livy/server/SessionServlet.scala +++ b/server/src/main/scala/org/apache/livy/server/SessionServlet.scala @@ -127,16 +127,18 @@ abstract class SessionServlet[S <: Session, R <: RecoveryMetadata]( } post("/") { - if (tooManySessions) { - BadRequest(ResponseMessage("Rejected, too many sessions are being created!")) - } else { - val session = sessionManager.register(createSession(request)) - // Because it may take some time to establish the session, update the last activity - // time before returning the session info to the client. - session.recordActivity() - Created(clientSessionView(session, request), - headers = Map("Location" -> - (getRequestPathInfo(request) + url(getSession, "id" -> session.id.toString)))) + synchronized { + if (tooManySessions) { + BadRequest(ResponseMessage("Rejected, too many sessions are being created!")) + } else { + val session = sessionManager.register(createSession(request)) + // Because it may take some time to establish the session, update the last activity + // time before returning the session info to the client. + session.recordActivity() + Created(clientSessionView(session, request), + headers = Map("Location" -> + (getRequestPathInfo(request) + url(getSession, "id" -> session.id.toString)))) + } } }
