Saisai Shao created LIVY-471:
--------------------------------
Summary: New session creation API set to support resource uploading
Key: LIVY-471
URL: https://issues.apache.org/jira/browse/LIVY-471
Project: Livy
Issue Type: Improvement
Components: Server
Affects Versions: 0.5.0
Reporter: Saisai Shao
Already post in mail list.
In our current API design to create interactive / batch session, we assume end
user should upload jars, pyFiles and related dependencies to HDFS before
creating the session, and we use one POST request to create session. But
usually end user may not have the permission to access the HDFS in their
submission machine, so it makes them hard to create new sessions. So the
requirement here is that if Livy could offer APIs to upload resources during
session creation. One implementation is proposed in
[https://github.com/apache/incubator-livy/pull/91.]
This add a field in session creation request to delay the session creation,
then adding a bunch of APIs to support resource upload, finally adding an API
to start creating the session. This seems a feasible solution, but also a
little hack to support such scenario. So I was thinking if we could a set of
new APIs to support such scenarios, rather than hack the existing APIs.
To borrow the concept from yarn application submission, we can have 3 APIs to
create session.
* requesting a new session id from Livy Server.
* uploading resources associate with this session id.
* submitting request to create session. This is similar to YARN's process to
submit application, and we can bump the supported API version for newly added
APIs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
