Re: is it ok to have multiple sparksession's in one spark structured streaming app?

2017-09-08 Thread Paul
nce and reliability perspective it is not good idea. >> >>> On Thu, Sep 7, 2017 at 2:40 AM, kant kodali <kanth...@gmail.com> wrote: >>> Hi All, >>> >>> I am wondering if it is ok to have multiple sparksession's in one spark >>> structured

Re: is it ok to have multiple sparksession's in one spark structured streaming app?

2017-09-08 Thread kant kodali
perspective it is not good > idea. > > On Thu, Sep 7, 2017 at 2:40 AM, kant kodali <kanth...@gmail.com> wrote: > >> Hi All, >> >> I am wondering if it is ok to have multiple sparksession's in one spark >> structured streaming app? Basically, I want to create 1)

Re: is it ok to have multiple sparksession's in one spark structured streaming app?

2017-09-08 Thread Arkadiusz Bicz
sparksession's in one spark > structured streaming app? Basically, I want to create 1) Spark session for > reading from Kafka and 2) Another Spark session for storing the mutations > of a dataframe/dataset to a persistent table as I get the mutations from > #1? > > Finally, is t

is it ok to have multiple sparksession's in one spark structured streaming app?

2017-09-06 Thread kant kodali
Hi All, I am wondering if it is ok to have multiple sparksession's in one spark structured streaming app? Basically, I want to create 1) Spark session for reading from Kafka and 2) Another Spark session for storing the mutations of a dataframe/dataset to a persistent table as I get the mutations