I see. A pull request can be submitted for spark-standalone.md
On Mon, Feb 1, 2016 at 2:51 PM, Lin, Hao <hao....@finra.org> wrote: > If you look at the Spark Doc, variable SPARK_WORKER_INSTANCES can still > be specified but yet the SPARK_EXECUTOR_INSTANCES > > > > http://spark.apache.org/docs/1.5.2/spark-standalone.html > > > > > > *From:* Ted Yu [mailto:yuzhih...@gmail.com] > *Sent:* Monday, February 01, 2016 5:45 PM > *To:* Lin, Hao > *Cc:* user > *Subject:* Re: SPARK_WORKER_INSTANCES deprecated > > > > As the message (from SparkConf.scala) showed, you shouldn't > use SPARK_WORKER_INSTANCES any more. > > > > FYI > > > > On Mon, Feb 1, 2016 at 2:19 PM, Lin, Hao <hao....@finra.org> wrote: > > Can I still use SPARK_WORKER_INSTANCES in conf/spark-env.sh? the > following is what I’ve got after trying to set this parameter and run > spark-shell > > > > SPARK_WORKER_INSTANCES was detected (set to '32'). > > This is deprecated in Spark 1.0+. > > > > Please instead use: > > - ./spark-submit with --num-executors to specify the number of executors > > - Or set SPARK_EXECUTOR_INSTANCES > > - spark.executor.instances to configure the number of instances in the > spark config. > > Confidentiality Notice:: This email, including attachments, may include > non-public, proprietary, confidential or legally privileged information. If > you are not an intended recipient or an authorized agent of an intended > recipient, you are hereby notified that any dissemination, distribution or > copying of the information contained in or transmitted with this e-mail is > unauthorized and strictly prohibited. If you have received this email in > error, please notify the sender by replying to this message and permanently > delete this e-mail, its attachments, and any copies of it immediately. You > should not retain, copy or use this e-mail or any attachment for any > purpose, nor disclose all or any part of the contents to any other person. > Thank you. > > > Confidentiality Notice:: This email, including attachments, may include > non-public, proprietary, confidential or legally privileged information. If > you are not an intended recipient or an authorized agent of an intended > recipient, you are hereby notified that any dissemination, distribution or > copying of the information contained in or transmitted with this e-mail is > unauthorized and strictly prohibited. If you have received this email in > error, please notify the sender by replying to this message and permanently > delete this e-mail, its attachments, and any copies of it immediately. You > should not retain, copy or use this e-mail or any attachment for any > purpose, nor disclose all or any part of the contents to any other person. > Thank you. >
