Hi, thank you for your interest in Zeppelin!
You just have to set the 'Spark' interpreter properties in 'Interpreters' menu: CPU spark.cores.max: 24 Mem spark.executor.memory 22g You actually can use any of the http://spark.apache.org/docs/latest/configuration.html#application-properties there. Hope this helps! On Thu, Jul 23, 2015 at 10:32 PM, PHELIPOT, REMY <remy.pheli...@atos.net> wrote: > Hello ! > > I am trying to launch some very greedy processes on a Spark 1.4 Cluster > using Zeppelin, and I don't understand how to configure Spark memory > properly. I’ve tried to set SPARK_MASTER_MEMORY, SPARK_WORKER_MEMORY and > SPARK_EXECUTOR_MEMORY environment variables on the spark cluster nodes, with > no effect. I’ve also tried to add the following line in zeppelin-env.sh: > export ZEPPELIN_JAVA_OPTS="-Dspark.home=/usr/spark > -Dspark.spark.executor.memory=15g -Dspark.spark.worker.memory=15g" > But it didn’t work.. > > How to configure properly Spark? It seems the configuration files are not > used and the memory available is limited to 512m. > > Kind regards, > > Rémy > > Ce message et toutes les pièces jointes (ci-après le "message") sont établis > à l’intention exclusive des destinataires désignés. Il contient des > informations confidentielles et pouvant être protégé par le secret > professionnel. Si vous recevez ce message par erreur, merci d'en avertir > immédiatement l'expéditeur et de détruire le message. Toute utilisation de > ce message non conforme à sa destination, toute diffusion ou toute > publication, totale ou partielle, est interdite, sauf autorisation expresse > de l’émetteur. L'internet ne garantissant pas l'intégrité de ce message lors > de son acheminement, Atos (et ses filiales) décline(nt) toute responsabilité > au titre de son contenu. Bien que ce message ait fait l’objet d’un > traitement anti-virus lors de son envoi, l’émetteur ne peut garantir > l’absence totale de logiciels malveillants dans son contenu et ne pourrait > être tenu pour responsable des dommages engendrés par la transmission de > l’un d’eux. > > This message and any attachments (the "message") are intended solely for the > addressee(s). It contains confidential information, that may be privileged. > If you receive this message in error, please notify the sender immediately > and delete the message. Any use of the message in violation of its purpose, > any dissemination or disclosure, either wholly or partially is strictly > prohibited, unless it has been explicitly authorized by the sender. As its > integrity cannot be secured on the internet, Atos and its subsidiaries > decline any liability for the content of this message. Although the sender > endeavors to maintain a computer virus-free network, the sender does not > warrant that this transmission is virus-free and will not be liable for any > damages resulting from any virus transmitted. -- -- Kind regards, Alexander.
