[ https://issues.apache.org/jira/browse/SPARK-11463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997231#comment-14997231 ]
Apache Spark commented on SPARK-11463: -------------------------------------- User 'davies' has created a pull request for this issue: https://github.com/apache/spark/pull/9574 > SparkContext fail to create in non-main thread in Python > -------------------------------------------------------- > > Key: SPARK-11463 > URL: https://issues.apache.org/jira/browse/SPARK-11463 > Project: Spark > Issue Type: Bug > Components: PySpark > Reporter: Davies Liu > Assignee: Davies Liu > Priority: Blocker > > https://github.com/apache/spark/commit/2e572c4135c3f5ad3061c1f58cdb8a70bed0a9d3#commitcomment-14137386 > {code} > Exception in thread Thread-3: > Traceback (most recent call last): > File > "/home/asoni/.conda/envs/py35_spark_django/lib/python3.5/threading.py", line > 923, in _bootstrap_inner > self.run() > File > "/home/asoni/.conda/envs/py35_spark_django/lib/python3.5/threading.py", line > 871, in run > self._target(*self._args, **self._kwargs) > File "/home/asoni/spark-server/spark_server/spark_jobs/views.py", line 67, > in process_job > from spark_jobs.job_runner import JobRunner > File "/home/asoni/spark-server/spark_server/spark_jobs/job_runner.py", line > 2, in <module> > from spark_jobs.extraction_plan_group_runner import * > File > "/home/asoni/spark-server/spark_server/spark_jobs/extraction_plan_group_runner.py", > line 1, in <module> > from spark_jobs.spark_connection import SparkConnection > File > "/home/asoni/spark-server/spark_server/spark_jobs/spark_connection.py", line > 5, in <module> > class SparkConnection: > File > "/home/asoni/spark-server/spark_server/spark_jobs/spark_connection.py", line > 7, in SparkConnection > sc = SparkContext(conf=conf) > File > "/home/asoni/spark-1.6.0-SNAPSHOT-bin-custom-spark/python/lib/pyspark.zip/pyspark/context.py", > line 114, in __init__ > conf, jsc, profiler_cls) > File > "/home/asoni/spark-1.6.0-SNAPSHOT-bin-custom-spark/python/lib/pyspark.zip/pyspark/context.py", > line 225, in _do_init > signal.signal(signal.SIGINT, signal_handler) > File "/home/asoni/.conda/envs/py35_spark_django/lib/python3.5/signal.py", > line 47, in signal > handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler)) > ValueError: signal only works in main thread > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org