RE: Spark streaming on spark-standalone/ yarn inside Spring XD

2015-09-17 Thread Vignesh Radhakrishnan
spark-standalone/ yarn inside Spring XD I am not at all familiar with how SpringXD works so hard to say. On Wed, Sep 16, 2015 at 12:12 PM, Vignesh Radhakrishnan <vignes...@altiux.com<mailto:vignes...@altiux.com>> wrote: Yes, it is TD. I'm able to run word count etc on spark standalon

Spark streaming on spark-standalone/ yarn inside Spring XD

2015-09-16 Thread Vignesh Radhakrishnan
Hi, I am trying to run a Spark processor on Spring XD for streaming operation. The spark processor module on Spring XD works when spark is pointing to local. The processor fails to run when we point spark to spark standalone (running on the same machine) or yarn-client. Is it possible to run

Re: Spark streaming on spark-standalone/ yarn inside Spring XD

2015-09-16 Thread Tathagata Das
I would check the following. See if your setup (spark master, etc.) is correct for running simple applications in Yarn/Standalone, like the SparkPi example. If that does not work then the problem is elsewhere. If that works, then the problem could be in the Spring XD. On Wed, Sep 16, 2015 at

Re: Spark streaming on spark-standalone/ yarn inside Spring XD

2015-09-16 Thread Vignesh Radhakrishnan
Yes, it is TD. I'm able to run word count etc on spark standalone/ yarn when it's not integrated with spring xd. But the same breaks when used as processor on spring. Was trying to get an opinion on whether it's doable or it's something that's not supported at the moment On 16 Sep 2015 23:50,

Re: Spark streaming on spark-standalone/ yarn inside Spring XD

2015-09-16 Thread Tathagata Das
I am not at all familiar with how SpringXD works so hard to say. On Wed, Sep 16, 2015 at 12:12 PM, Vignesh Radhakrishnan < vignes...@altiux.com> wrote: > Yes, it is TD. I'm able to run word count etc on spark standalone/ yarn > when it's not integrated with spring xd. > But the same breaks when