Re: Setting number of CORES from inside the Topology (JAVA code )

2015-08-29 Thread Akhil Das
When you set .setMaster to local[4], it means that you are allocating 4 threads on your local machine. You can change it to local[1] to run it on a single thread. If you are submitting the job to a standalone spark cluster and you wanted to limit the # cores for your job, then you can do it like

Setting number of CORES from inside the Topology (JAVA code )

2015-08-26 Thread anshu shukla
Hey , I need to set the number of cores from inside the topology . Its working fine by setting in spark-env.sh but unable to do via setting key/value for conf . SparkConf sparkConf = new SparkConf().setAppName(JavaCustomReceiver).setMaster(local[4]); if(toponame.equals(IdentityTopology))