[
https://issues.apache.org/jira/browse/SPARK-11463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-11463:
------------------------------------
Assignee: Apache Spark (was: Davies Liu)
> SparkContext fail to create in non-main thread in Python
> --------------------------------------------------------
>
> Key: SPARK-11463
> URL: https://issues.apache.org/jira/browse/SPARK-11463
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Reporter: Davies Liu
> Assignee: Apache Spark
> Priority: Blocker
>
> https://github.com/apache/spark/commit/2e572c4135c3f5ad3061c1f58cdb8a70bed0a9d3#commitcomment-14137386
> {code}
> Exception in thread Thread-3:
> Traceback (most recent call last):
> File
> "/home/asoni/.conda/envs/py35_spark_django/lib/python3.5/threading.py", line
> 923, in _bootstrap_inner
> self.run()
> File
> "/home/asoni/.conda/envs/py35_spark_django/lib/python3.5/threading.py", line
> 871, in run
> self._target(*self._args, **self._kwargs)
> File "/home/asoni/spark-server/spark_server/spark_jobs/views.py", line 67,
> in process_job
> from spark_jobs.job_runner import JobRunner
> File "/home/asoni/spark-server/spark_server/spark_jobs/job_runner.py", line
> 2, in <module>
> from spark_jobs.extraction_plan_group_runner import *
> File
> "/home/asoni/spark-server/spark_server/spark_jobs/extraction_plan_group_runner.py",
> line 1, in <module>
> from spark_jobs.spark_connection import SparkConnection
> File
> "/home/asoni/spark-server/spark_server/spark_jobs/spark_connection.py", line
> 5, in <module>
> class SparkConnection:
> File
> "/home/asoni/spark-server/spark_server/spark_jobs/spark_connection.py", line
> 7, in SparkConnection
> sc = SparkContext(conf=conf)
> File
> "/home/asoni/spark-1.6.0-SNAPSHOT-bin-custom-spark/python/lib/pyspark.zip/pyspark/context.py",
> line 114, in __init__
> conf, jsc, profiler_cls)
> File
> "/home/asoni/spark-1.6.0-SNAPSHOT-bin-custom-spark/python/lib/pyspark.zip/pyspark/context.py",
> line 225, in _do_init
> signal.signal(signal.SIGINT, signal_handler)
> File "/home/asoni/.conda/envs/py35_spark_django/lib/python3.5/signal.py",
> line 47, in signal
> handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
> ValueError: signal only works in main thread
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
