[
https://issues.apache.org/jira/browse/SPARK-5926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329269#comment-14329269
]
Yanbo Liang edited comment on SPARK-5926 at 2/20/15 6:14 PM:
-------------------------------------------------------------
This is because that for DDL like commands with side effects, DataFrame forces
it to execute right away. However if we just want to know the execution plan,
we do not need it to execute.
{code:title=DataFrameImpl.scala|borderStyle=solid}
@transient protected[sql] override lazy val logicalPlan: LogicalPlan =
queryExecution.logical match {
// For various commands (like DDL) and queries with side effects, we force
query optimization to
// happen right away to let these side effects take place eagerly.
case _: Command |
_: InsertIntoTable |
_: CreateTableAsSelect[_] |
_: CreateTableUsingAsSelect |
_: WriteToFile =>
LogicalRDD(queryExecution.analyzed.output,
queryExecution.toRdd)(sqlContext)
case _ =>
queryExecution.logical
}
{code}
was (Author: yanboliang):
This is because that for DDL like queries with side effects, and DataFrame
force it happen right away. We should use the former queryExecution.logical to
explain.
{code:title=DataFrameImpl.scala|borderStyle=solid}
@transient protected[sql] override lazy val logicalPlan: LogicalPlan =
queryExecution.logical match {
// For various commands (like DDL) and queries with side effects, we force
query optimization to
// happen right away to let these side effects take place eagerly.
case _: Command |
_: InsertIntoTable |
_: CreateTableAsSelect[_] |
_: CreateTableUsingAsSelect |
_: WriteToFile =>
LogicalRDD(queryExecution.analyzed.output,
queryExecution.toRdd)(sqlContext)
case _ =>
queryExecution.logical
}
{code}
> [SQL] DataFrame.explain() return false result for DDL command
> -------------------------------------------------------------
>
> Key: SPARK-5926
> URL: https://issues.apache.org/jira/browse/SPARK-5926
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: Yanbo Liang
>
> This bug is easy to reproduce, the following two queries should print out the
> same explain result, but it's not.
> sql("create table tb as select * from src where key > 490").explain(true)
> sql("explain extended create table tb as select * from src where key > 490")
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
