Oleksandr Shevchenko commented on LIVY-664:

Thanks for your comment [~yihengw]!
Don't see that duplicated session stops now, we just throw an exception 
 Could you show where we stop this session?

I have tried to add `session.stop()` before throwing IllegalArgumentException 
and it works fine in the case when YARN/Spark standalone application isn't 
started yet but if application is already in ACCEPTED or RUNNING state stop 
session doesn't work.
By design, in this case, the application should be killed here 
 I guess. But we have bad error handling in RSCClient.java and error will not 
be propagated and also `app` in the following code is None since 
`session.start()` didn't execute:

app.foreach {
 warn(s"Failed to stop RSCDriver. Killing it...")

Better to check the session name before start application, but I don't see 
where we actually do it. So, I put off this bug fix since little busy now. I 
can continue my work on this issue if you help me to find where an application 
is submitted (hope this is not a Python code, but I didn't find related 
Scala/Java code for the first time).


> Spark application still running when Livy session creating was rejected 
> ------------------------------------------------------------------------
>                 Key: LIVY-664
>                 URL: https://issues.apache.org/jira/browse/LIVY-664
>             Project: Livy
>          Issue Type: Bug
>            Reporter: Oleksandr Shevchenko
>            Priority: Major
>         Attachments: image-2019-09-08-20-38-50-195.png, 
> image-2019-09-08-20-39-18-569.png
>          Time Spent: 10m
>  Remaining Estimate: 0h
> Steps for reproduce:
> 1. Create a session with some name
> 2. Create a second session with the same name
>   2.1 Second session creating will be rejected since duplicated session name 
> is not allowed.
>   2.2 Spark application will be submitted but Livy session won't be created
> Result: Spark application was submitted but Livy session is not created
> Expected result: Livy session creating rejected AND Spark application should 
> be finished with failed or killed state or even should not be submitted.
> !image-2019-09-08-20-38-50-195.png!
> !image-2019-09-08-20-39-18-569.png!

This message was sent by Atlassian Jira

Reply via email to