Dear Spark devs,
I am debugging a weird "Java gateway process exited before sending the
driver its port number" when creating SparkSession with pyspark. I am
running the following simple code with pytest:
"
from pyspark.sql import SparkSession
def test_spark():
spark = SparkSession.builder.getOrCreate()
spark.range(10).show()
"
(I've setup an conda environment and conda installed pyspark version 2.4.4)
I am wondering if there is a way for me to find the stdout/stderr/log for
the Java driver process to see what's going on?
Help much appreciated!
Thanks,
Li
