Hi,
Not sure of your exact use case. I was also mentioning
different user code in the same job’s codebase. My use case is different input
while calling the same job multiple times.
Thanks and Regards
Partha Pratim Ghosh
From: Malveeka Bhandari [mailto:malveeka.bhand...@gmail.com]
Sent: Monday, May 21, 2018 3:43 AM
To: user@livy.incubator.apache.org
Subject: Re: Livy session
Hi,
Thanks for the idea. However, my use case is a little bit complicated. The job
code remains constant, but the user code goes in as a dependent class. For e.g.
the job might be to query data, filter for some constraints and run a
simulation (multiple transformations and actions) on the data. This job is
constant. One of the functions inside the simulation would need the user code.
So I would need to add the user code as a jar as opposed to a job class.
Is that possible in any way ?
Thanks !
On Mon, 21 May 2018 at 12:46 PM, Ghosh, Parthapratim
<parthapratim.gh...@citi.com<mailto:parthapratim.gh...@citi.com>> wrote:
Hi Malveeka,
I am another user in the forum. However, your requirement may
be met without creating a new jar. You may put the user code in a defined job
as an instance level variable and use it everytime you use the client to submit
a job. To call this, create a LivyClient and submit the job everytime with a
new code as required, instantiating the job with the client code in overloaded
constructor.
Thanks and Regards
Partha Pratim Ghosh
From: Malveeka Bhandari
[mailto:malveeka.bhand...@gmail.com<mailto:malveeka.bhand...@gmail.com>]
Sent: Monday, May 21, 2018 2:58 AM
To: user@livy.incubator.apache.org<mailto:user@livy.incubator.apache.org>
Subject: Fwd: Livy session
Hi everyone.
We are using Livy for our custom use case where a user can submit a batch
request. We send the user code in spark.jars configuration. But we are having
to make a new session everytime. Is there any way that there could be one
running session for a user and we could keep submitting new jobs but with a new
dependent jar each time.
Thanks
