Re: Why does jobContext.sparkSession doesn't return a SparkSession Object?

That is probably because Livy supports Spark 1.6 and "SparkSession" does not exist in that version, so the code wouldn't compile otherwise. If you try to cast it to some other type, though, you'll get an exception at run time. On Sat, Dec 2, 2017 at 10:11 AM, kant kodali

Re: After successfully computed job submitted programmatically, it is still running in Spark UI and marked as idle in Livy UI

You have to call "client.stop(true)" if you want to shut down the Spark application. On Sun, Oct 29, 2017 at 3:19 AM, Stefan Miklosovic wrote: > Title says it all, I upload a JAR, I run a job via client.run(Job > job).get(); and I do get a result - all is computed ok,

Re: How to cancel the running streaming job using livy?

A JobHandle (which you get by submitting a Job) is a Future, and Futures have a "cancel()" method. I don't remember the details about how "cancel()" is implemented in Livy, though. On Fri, Jan 19, 2018 at 3:52 PM, Alex Bozarth wrote: > Ok so I looked into this a bit more.

Re: How to cancel the running streaming job using livy?

eaming Query? Sorry if this > is too naive. > > On Wed, Jan 24, 2018 at 2:46 PM, Marcelo Vanzin <van...@cloudera.com> > wrote: > >> Then that has nothing to do with Livy. >> >> You need to store a reference to your StreamingQuery (returned by >> start

Re: How to cancel the running streaming job using livy?

States-26entry-3Dgmail-26source-3Dg=DwMFaQ=jf_iaSHvJObTbx-siA1ZOg=S1_S7Dymu4ZL6g7L21O78VQZ53vEnAyZ-cx37DPYDyo=Io6A_oOejKvX7wP9hqKWr0NXa729OGgy1e-qdIwelfI=GCO_bHHbb3d10NSMTDbyhfJqnEzkvlFZJoH4oND7x2w=> > *United States* > <https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-

Re: How to cancel the running streaming job using livy?

nt(*) from table"); > //Streaming query2 started > > sparkSession.streams().awaitAnyTermination(); > >return null; > > } > > > Thanks! > > > On Wed, Jan 24, 2018 at 1:47 PM, Marcelo Vanzin <van...@cloudera.com> > wrote: > >> I'm a little

Re: Can not deserialize instance of scala.collection.immutable.List out of VALUE_STRING (through reference chain: org.apache.livy.server.interactive.CreateInteractiveRequest)

"jars" is a list so you should probably try: "jars": [ "hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar" ] On Mon, Jan 29, 2018 at 3:35 PM, Kyunam Kim wrote: > I get the following when I try "jars" in POST /sessions. > > I tried: > - "jars":

Re: When I submit a livy job am I running in a client mode or cluster mode ?

That's defined in your server's configuration. On Sun, Feb 11, 2018 at 11:48 PM, kant kodali wrote: > Hi All, > > When I submit a livy job to livy server am I running in a client mode or > cluster mode ? How can I switch from one mode to another? > > Thanks! > -- Marcelo

Re: How to deploy generic Spark applications via Livy using the Java client

If your "orchestrator engine" is receiving pre-built apps from others and needs to execute them in the cluster, you could just use Livy's batch API. I don't think there are Java bindings for that, you'd need to talk to the REST endpoints directly. The code you're referring to is for "interactive"

Re: Enable HTTP Basic Authenticaton for Livy Rest APIs

nterprise > requirements. > > Is there any developments guidelined available to get me started to change > the code? > > On Fri, Oct 5, 2018, 2:50 PM Marcelo Vanzin wrote: > >> I don't think Livy currently has support for basic auth. >> >> Although this PR would allow you

[ANNOUNCE] Apache Livy 0.6.0-incubating released

The Apache Livy team is proud to announce the release of Apache Livy 0.6.0-incubating. Livy is web service that exposes a REST interface for managing long running Apache Spark contexts in your cluster. Livy enables programmatic, fault-tolerant, multi-tenant submission of Spark jobs from

Re: [ANNOUNCE] Apache Livy 0.6.0-incubating released

07:05, Jeff Zhang wrote: > > Thanks Marcelo > > > 于2019年4月3日周三 下午12:15写道： >> >> Nice!! >> >> Sent from my iPhone >> >> On Apr 2, 2019, at 8:21 PM, Ruslan Dautkhanov wrote: >> >> Thanks a lot Marcelo ! >> >> Ruslan >>

Re: [ANNOUNCE] Apache Livy 0.6.0-incubating released

com/artifact/org.apache.livy/livy-api > > On Wed, Apr 3, 2019 at 9:54 AM Marcelo Vanzin wrote: >> >> I closed the bug and will fix the website link. Thanks for pointing it out! >> >> On Wed, Apr 3, 2019 at 4:55 AM Alexey Romanenko >> wrote: >> > >> >

Re: Livy-0.6 release?

shall we enable it by default or leave it disabled? > > Thanks > Saisai > > Jeff Zhang 于2019年3月12日周二 上午10:54写道： >> >> Thanks Marcelo, I can help to test it in zeppelin side which use livy as one >> interpreter. >> >> Marcelo Vanzin 于2019年3月12日周二 上

Re: Livy-0.6 release?

celo. >> >> I see LIVY-551 is now part of the master. Hope to see Livy 0.6 perhaps soon. >> >> >> Thank you! >> Ruslan Dautkhanov >> >> >> On Tue, Feb 5, 2019 at 12:38 PM Marcelo Vanzin wrote: >>> >>> I think LIVY-551 is th

Re: Livy-0.6 release?

arcelo, what particular patches are blocking Livy 0.6 release? > > I see 3 jiras with 0.6 as Fix Version - not sure if that's correct way to > find blockers. > https://goo.gl/9axfsw > > > Thank you! > Ruslan Dautkhanov > > > On Mon, Jan 28, 2019 at 2:24 PM Marcelo Vanzi

Re: Livy-0.6 release?

There are a couple of patches under review that are currently blocking the release. Once those are done, we can work on releasing 0.6. On Mon, Jan 28, 2019 at 11:18 AM Roger Liu wrote: > > Hey there, > > > > I’m wondering if we have a timeline for releasing Livy-0.6? Its been a year > since