cc+ Anjana Hi Supun,
As per the M1 release, we have only the standalone 'local' mode spark instance. as you correctly said, we have set the master to 'local' in the BAM . please refer init() method here [1]. In the init() method, we are starting a spark context. we are still working on spark clustering. the plan is to release it in the M2. [1] https://github.com/wso2/carbon-analytics/blob/master/components/analytics/analytics-processors/org.wso2.carbon.analytics.spark.core/src/main/java/org/wso2/carbon/analytics/spark/core/AnalyticsExecutionService.java On Mon, Mar 9, 2015 at 5:48 PM, Supun Sethunga <[email protected]> wrote: > Hi, > > I tried starting up a BAM server, and submitting a spark-job to the Spark > which is built in BAM, from an external java client. In doing this, I was > not able to create a Spark Context using BAM's spark instance as master. > (AFAIK, the way to do this is to create a Spark context inside the client, > and point BAM's Spark instance as the master, and then submit any jobs to > the created context.) > > Possible reason might be, in BAM, Spark instance (i.e. SparkContext) is > started locally with one worker thread.(setting master as 'local'). Hence > the spark instance cannot be accessed by any external client. (Refer > attached image "BAM-spark.png") > AFAIK, to be accessible, BAM's Spark instance needs be started in the > clustering mode. Then any external client also can use the BAM's spark, > through the "spark.master" URL, which is "spark://localhost:7077" by > default. (Refer attached image " > external-spark-cluster.png"). > > I'm not sure whether this feature (clustering Spark) is currently > supported by BAM (in M1 release)? > > If so, whats the way forward? > > Thanks, > Supun > > -- > *Supun Sethunga* > Software Engineer > WSO2, Inc. > lean | enterprise | middleware > Mobile : +94 716546324 > -- *Niranda Perera* Software Engineer, WSO2 Inc. Mobile: +94-71-554-8430 Twitter: @n1r44 <https://twitter.com/N1R44>
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
