Running a stream job using different users with spark-0.8.0. The output
publishing into a queue as json format using spray json. After some
hours, noticed that only getting the output for the first user and getting
the keys for other users in the json string. Actually the value in the json
is the output of spark streaming streaming job.
My additional Spark streaming settings are.
System.setProperty("spark.driver.port", "0")
System.setProperty("spark.cores.max", "1")
System.setProperty("spark.mesos.coarse", "true")
System.setProperty("spark.executor.memory", "2g")
System.setProperty("spark.scheduler.mode", "FAIR")
Are these settings correct for long running multi-user Spark streaming job.
Am I doing anything wrong please suggest.
Thanks,
prabeesh
On Sat, Nov 23, 2013 at 5:26 AM, Patrick Wendell <[email protected]> wrote:
> Spark has good support for multi-tenancy. Prabeesh could you describe
> in more detail what you are actually running, what the expected
> behavior is and what you are observing?
>
> On Fri, Nov 22, 2013 at 3:09 AM, Sam Bessalah <[email protected]> wrote:
> > Spark doesn't support multi users, as far as I can tell.
> >
> > Sam Bessalah
> >
> >> On Nov 22, 2013, at 7:54 AM, prabeesh k <[email protected]> wrote:
> >>
> >> Hi all,
> >> While running a Spark job using different users, getting only
> output of first user. After some hours no output for other users . Please
> help me , please suggest the settings for multi-user.
> >>
> >> Thanks in Advance
> >> Regars,
> >> Prabeesh
>
