angelosnm commented on issue #1723: URL: https://github.com/apache/sedona/issues/1723#issuecomment-2545323136
@jiayuasu I was monitoring the Spark Web UI console and nothing specific was logged... My guess is that it was a network related issue. I am using a Jupyter based container image inside a k8s cluster where I send PySpark jobs to a spark cluster that is in the same local network with the k8s cluster but referencing it externally by exposing the master endpoint to the global internet. This setup introduces lots of network configurations that need to be done on both clusters (such as having to route k8s pods CIDR from the Spark nodes, set up networking policies on k8s, etc). I just deployed Spark inside the k8s cluster (using the official `apache/spark` image) and the code ran successfully! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
