Rohith Bhattaram created SPARK-31152:
----------------------------------------
Summary: Issue with Spark Context Initialization
Key: SPARK-31152
URL: https://issues.apache.org/jira/browse/SPARK-31152
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 2.4.5, 3.0.0
Environment: OS : Windows 10
Spark : spark-3.0.0-preview2-bin-hadoop2.7
Env_variables :
SPARK_HOME :
C:\Users\rohit\spark\spark-3.0.0-preview2-bin-hadoop2.7\spark-3.0.0-preview2-bin-hadoop2.7
HADOOP_HOME :
C:\Users\rohit\spark\spark-3.0.0-preview2-bin-hadoop2.7\spark-3.0.0-preview2-bin-hadoop2.7
JAVA_HOME : C:\Program Files\Java\jdk1.8.0_191 ; C:\Program
Files\Java\jre1.8.0_241\bin
PYTHON VERSION : Python 3.7.1
ANACONDA VERSION : conda 4.8.2
I am not running this pyspark code locally (no hadoop setup)- to develop on NLP
script.
Reporter: Rohith Bhattaram
Issue :
I am trying to initialize spark context using below code :
from pyspark.sql import SparkSession
spark = SparkSession.builder.getOrCreate()
import pandas as pd
sc = spark.sparkContext
At getOrCreate() step , code is going to infinite loop and never respond back
either with exception or timeout.
After execuring if i check on VS Terminal or Anaconda cmd prompot - below
statement is getting displayed :
[I 19:13:47.973 NotebookApp] Saving file at /LearnPython/Assignment3/NLP.ipynb
*The filename, directory name, or volume label syntax is incorrect.*
Note : This used to work fine till a month back - not sure what chnaged, it
stopped working from yesterday.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
