You don't need multiple spark sessions to have more than one stream working, but from maintenance and reliability perspective it is not good idea.
On Thu, Sep 7, 2017 at 2:40 AM, kant kodali <kanth...@gmail.com> wrote: > Hi All, > > I am wondering if it is ok to have multiple sparksession's in one spark > structured streaming app? Basically, I want to create 1) Spark session for > reading from Kafka and 2) Another Spark session for storing the mutations > of a dataframe/dataset to a persistent table as I get the mutations from > #1? > > Finally, is this a common practice? > > Thanks, > kant >
