That is ideal behavior for our purposes, thanks so much all. -Graham
On Wed, Sep 6, 2017 at 8:52 AM, Saisai Shao <[email protected]> wrote: > Session recovery should be enabled by configuration. If it is enabled, > session id will be recorded in the persistent storage, and the number will > continue to increment after restart. > > On Wed, Sep 6, 2017 at 8:27 PM, Graham Hukill <[email protected]> wrote: > >> Ah, also very good to know, thanks. That might be a great option. >> >> I'm going to look into this today, but I'm curious, are sessions >> recovered automatically? or, if not, and a session is not recovered, does >> the numbering continue to increment for sessionId's? >> >> Thanks, >> Graham >> >> On Tue, Sep 5, 2017 at 9:19 PM, Vivek <[email protected]> wrote: >> >>> One other option is to turn on session recovery. >>> This ensures that the session ids are unique >>> >>> Sent from my iPhone >>> >>> On 6 Sep 2017, at 12:10 AM, Graham Hukill <[email protected]> wrote: >>> >>> Prabhu, >>> >>> Thanks so much for the information, nice to know it's been formally >>> requested. >>> >>> thanks, >>> Graham >>> >>> On Tue, Sep 5, 2017 at 11:22 AM, Prabhu Kasinathan < >>> [email protected]> wrote: >>> >>>> Livy currently doesn't have this feature now. >>>> >>>> Here, is the Jira for that: https://issues.cloudera. >>>> org/plugins/servlet/mobile#issue/LIVY-41. PayPal is already having >>>> that feature in production and Meisam had submitted pull request for this >>>> feature to Livy. >>>> >>>> https://github.com/meisam/livy/wiki/Design-doc-for-Livy-41:- >>>> Accessing-sessions-by-name >>>> >>>> >>>> Thanks*!* >>>> Prabhu >>>> iPhone. iTypos. iApologize. >>>> >>>> On Sep 5, 2017, at 7:51 AM, Graham Hukill <[email protected]> wrote: >>>> >>>> I wondered if it was possible to set the session URI when creating a >>>> new one? I see from the Livy REST API documentation ( >>>> https://livy.incubator.apache.org/docs/latest/rest-api.html), under >>>> POST /sessions, some of the arguments that you can pass when initializing a >>>> new session, such as "kind", "name", etc. >>>> >>>> As I understand it now, sessionId's are automatically created when >>>> POSTing to /sessions, setting the ID as an incrementing digit from "0" when >>>> Livy starts. If Livy is restarted, the numbering drops back to "0". >>>> >>>> Normally this would not present a problem, but we are planning on >>>> starting/stopping Livy sessions from another application, and could >>>> envision a scenario where: >>>> >>>> - a session is started in Livy at /sessions/0 >>>> - that session ID is associated in our ancillary app with a >>>> particular user or job >>>> - Livy restarts, then when a new session is started, the pointer to >>>> /sessions/0 is actually a different Livy/SparkSession >>>> >>>> Ideally, it would be helpful if we could mint a session ID with a PUT >>>> request, something like PUT /sessions/foobar, where "foobar" is the newly >>>> created sessionId. >>>> >>>> Is this currently possible in any way? or are there undocumented >>>> configurations that can be passed with the POST request for creating new >>>> sessions that would define the sessionId? >>>> >>>> thanks, >>>> Graham >>>> >>>> >>> >> >
