Jagadeesh Marada created SPARK-47489:
----------------------------------------
Summary: Action performed on Dataset is triggering an error as
"The Spark SQL phase planning failed with an internal error."
Key: SPARK-47489
URL: https://issues.apache.org/jira/browse/SPARK-47489
Project: Spark
Issue Type: Bug
Components: k8s, Spark Core, Spark Submit
Affects Versions: 3.4.2
Reporter: Jagadeesh Marada
Hi team,
We are upgrading to spark 3.4.2 from 3.3.2 and facing issue in the spark driver
as below.
"org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase planning
failed with an internal error. You hit a bug in Spark or the Spark plugins you
use. Please, report this bug to the corresponding communities or vendors, and
provide the full stack trace."
Complete stack trace is as below.
After analysing further , found that when we try to perform any actions on the
dataset, spark is unable to plan its actions, results in exceptions.
attaching a code snippet where exactly we are trigging an action to convert it
as java RDD, is failing.
Attaching the spark submit command as well for your reference.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
