[looping Nirmal] Hi Niranda,
Thanks for the clarification. @Nirmal : Seems we have to wait till the BAM M2 release for ML-BAM Integration :) . But we still can enable to use an external spark cluster for ML. Regards, Supun On Tue, Mar 10, 2015 at 9:49 AM, Niranda Perera <[email protected]> wrote: > cc+ Anjana > > Hi Supun, > > As per the M1 release, we have only the standalone 'local' mode spark > instance. as you correctly said, we have set the master to 'local' in the > BAM . please refer init() method here [1]. In the init() method, we are > starting a spark context. > > we are still working on spark clustering. the plan is to release it in the > M2. > > [1] > https://github.com/wso2/carbon-analytics/blob/master/components/analytics/analytics-processors/org.wso2.carbon.analytics.spark.core/src/main/java/org/wso2/carbon/analytics/spark/core/AnalyticsExecutionService.java > > On Mon, Mar 9, 2015 at 5:48 PM, Supun Sethunga <[email protected]> wrote: > >> Hi, >> >> I tried starting up a BAM server, and submitting a spark-job to the Spark >> which is built in BAM, from an external java client. In doing this, I was >> not able to create a Spark Context using BAM's spark instance as master. >> (AFAIK, the way to do this is to create a Spark context inside the client, >> and point BAM's Spark instance as the master, and then submit any jobs to >> the created context.) >> >> Possible reason might be, in BAM, Spark instance (i.e. SparkContext) is >> started locally with one worker thread.(setting master as 'local'). Hence >> the spark instance cannot be accessed by any external client. (Refer >> attached image "BAM-spark.png") >> AFAIK, to be accessible, BAM's Spark instance needs be started in the >> clustering mode. Then any external client also can use the BAM's spark, >> through the "spark.master" URL, which is "spark://localhost:7077" by >> default. (Refer attached image " >> external-spark-cluster.png"). >> >> I'm not sure whether this feature (clustering Spark) is currently >> supported by BAM (in M1 release)? >> >> If so, whats the way forward? >> >> Thanks, >> Supun >> >> -- >> *Supun Sethunga* >> Software Engineer >> WSO2, Inc. >> lean | enterprise | middleware >> Mobile : +94 716546324 >> > > > > -- > *Niranda Perera* > Software Engineer, WSO2 Inc. > Mobile: +94-71-554-8430 > Twitter: @n1r44 <https://twitter.com/N1R44> > -- *Supun Sethunga* Software Engineer WSO2, Inc. lean | enterprise | middleware Mobile : +94 716546324
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
