Saisai Shao created LIVY-471:
--------------------------------

             Summary: New session creation API set to support resource uploading
                 Key: LIVY-471
                 URL: https://issues.apache.org/jira/browse/LIVY-471
             Project: Livy
          Issue Type: Improvement
          Components: Server
    Affects Versions: 0.5.0
            Reporter: Saisai Shao


Already post in mail list.

In our current API design to create interactive / batch session, we assume end 
user should upload jars, pyFiles and related dependencies to HDFS before 
creating the session, and we use one POST request to create session. But 
usually end user may not have the permission to access the HDFS in their 
submission machine, so it makes them hard to create new sessions. So the 
requirement here is that if Livy could offer APIs to upload resources during 
session creation. One implementation is proposed in

[https://github.com/apache/incubator-livy/pull/91.]

This add a field in session creation request to delay the session creation, 
then adding a bunch of APIs to support resource upload, finally adding an API 
to start creating the session. This seems a feasible solution, but also a 
little hack to support such scenario. So I was thinking if we could a set of 
new APIs to support such scenarios, rather than hack the existing APIs.

To borrow the concept from yarn application submission, we can have 3 APIs to 
create session.
 * requesting a new session id from Livy Server.
 * uploading resources associate with this session id.
 * submitting request to create session. This is similar to YARN's process to 
submit application, and we can bump the supported API version for newly added 
APIs.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to