[
https://issues.apache.org/jira/browse/SPARK-47489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jagadeesh Marada updated SPARK-47489:
-------------------------------------
Description:
Hi team,
We are upgrading to spark 3.4.2 from 3.3.2 and facing issue in the spark driver
as below.
"org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase planning
failed with an internal error. You hit a bug in Spark or the Spark plugins you
use. Please, report this bug to the corresponding communities or vendors, and
provide the full stack trace."
Complete stack trace is as attached.
After analysing further , found that when we try to perform any actions on the
dataset, spark is unable to plan its actions, results in this exception.
Attaching a Exception stack trace & code snippet where exactly we are trigging
an action to convert it as java RDD.
Attaching the spark submit command as well for your reference.
Can you please check this issue let us know if any fix available for this issue.
was:
Hi team,
We are upgrading to spark 3.4.2 from 3.3.2 and facing issue in the spark driver
as below.
"org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase planning
failed with an internal error. You hit a bug in Spark or the Spark plugins you
use. Please, report this bug to the corresponding communities or vendors, and
provide the full stack trace."
Complete stack trace is as below.
After analysing further , found that when we try to perform any actions on the
dataset, spark is unable to plan its actions, results in exceptions.
attaching a code snippet where exactly we are trigging an action to convert it
as java RDD, is failing.
Attaching the spark submit command as well for your reference.
> Action performed on Dataset is triggering an error as "The Spark SQL phase
> planning failed with an internal error."
> -------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-47489
> URL: https://issues.apache.org/jira/browse/SPARK-47489
> Project: Spark
> Issue Type: Bug
> Components: k8s, Spark Core, Spark Submit
> Affects Versions: 3.4.2
> Reporter: Jagadeesh Marada
> Priority: Major
> Attachments: code snippet.jpeg, exception_stack_trace.txt, jars added
> to spark_submit.jpeg, spark-submit cmd.txt
>
>
> Hi team,
> We are upgrading to spark 3.4.2 from 3.3.2 and facing issue in the spark
> driver as below.
> "org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase
> planning failed with an internal error. You hit a bug in Spark or the Spark
> plugins you use. Please, report this bug to the corresponding communities or
> vendors, and provide the full stack trace."
>
> Complete stack trace is as attached.
> After analysing further , found that when we try to perform any actions on
> the dataset, spark is unable to plan its actions, results in this exception.
> Attaching a Exception stack trace & code snippet where exactly we are
> trigging an action to convert it as java RDD.
>
> Attaching the spark submit command as well for your reference.
> Can you please check this issue let us know if any fix available for this
> issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
