[ https://issues.apache.org/jira/browse/SPARK-17907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-17907. ------------------------------- Resolution: Invalid The error tells you the problem: {code} ValueError: Cannot run multiple SparkContexts at once; existing SparkContext(app=PYSPARK, master=spark://172.31.28.208:7077) created by _init_ at <ipython-input-2-c7c8de510121>:6 {code} You already have a context, and can't create a new one. The console mentions this. Have a look at https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark please. > Not allowing more spark console > ------------------------------- > > Key: SPARK-17907 > URL: https://issues.apache.org/jira/browse/SPARK-17907 > Project: Spark > Issue Type: Bug > Components: PySpark > Affects Versions: 2.0.0 > Reporter: Sankar Mittapally > > We are exploring pyspark and spark cluster, We are able to initiated single > spark console connection, while trying to establish new connection. We are > getting error. > --------------------------------------------------------------------------- > ValueError Traceback (most recent call last) > <ipython-input-15-05f9533b85b9> in <module>() > 4 .set("spark.executor.memory", "1g") > 5 > .set("spark.cores.max","1").set("spark.driver.allowMultipleContexts", "true") > ) > ----> 6 sc = SparkContext(conf = conf) > /opt/spark-2.0.0-bin-hadoop2.7/python/pyspark/context.py in __init__(self, > master, appName, sparkHome, pyFiles, environment, batchSize, serializer, > conf, gateway, jsc, profiler_cls) > 110 """ > 111 self._callsite = first_spark_call() or CallSite(None, None, > None) > --> 112 SparkContext._ensure_initialized(self, gateway=gateway) > 113 try: > 114 self._do_init(master, appName, sparkHome, pyFiles, > environment, batchSize, serializer, > /opt/spark-2.0.0-bin-hadoop2.7/python/pyspark/context.py in > _ensure_initialized(cls, instance, gateway) > 257 " created by %s at %s:%s " > 258 % (currentAppName, currentMaster, > --> 259 callsite.function, callsite.file, > callsite.linenum)) > 260 else: > 261 SparkContext._active_spark_context = instance > ValueError: Cannot run multiple SparkContexts at once; existing > SparkContext(app=PYSPARK, master=spark://172.31.28.208:7077) created by > __init__ at <ipython-input-2-c7c8de510121>:6 > Command We are using > {code} > conf = (SparkConf() > .setMaster("spark://172.31.28.208:7077") > .setAppName("sankar") > .set("spark.executor.memory", "1g") > .set("spark.cores.max","1").set("spark.driver.allowMultipleContexts", > "true") ) > sc = SparkContext(conf = conf) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org