GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/11599
[SPARK-13763] [SQL] Remove Project when its projectList is Empty
#### What changes were proposed in this pull request?
As shown in another PR: https://github.com/apache/spark/pull/11596, we are
using `SELECT 1` as a dummy table, when the table is used for SQL statements in
which a table reference is required, but the contents of the table are not
important. For example,
```SQL
SELECT value FROM (select 1) dummyTable Lateral View explode(array(1,2,3))
adTable as value
```
Before the PR, the optimized plan contains a useless `Project` after
Optimizer executing `ColumnPruning` rule, as shown below:
```
== Analyzed Logical Plan ==
value: int
Project [value#22]
+- Generate explode(array(1, 2, 3)), true, false, Some(adtable), [value#22]
+- SubqueryAlias dummyTable
+- Project [1 AS 1#21]
+- OneRowRelation$
== Optimized Logical Plan ==
Generate explode([1,2,3]), false, false, Some(adtable), [value#22]
+- Project
+- OneRowRelation$
```
After the fix, the optimized plan removed the useless `Project`, as shown
below:
```
== Optimized Logical Plan ==
Generate explode([1,2,3]), false, false, Some(adtable), [value#22]
+- OneRowRelation$
```
#### How was this patch tested?
Added a new unit test case into the suite `ColumnPruningSuite.scala`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark projectOneRowRelation
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11599.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11599
----
commit 0fa21acd2614fbdf128561b34c72088008d62a05
Author: gatorsmile <[email protected]>
Date: 2016-03-09T04:06:39Z
remove Project with an empty projectList
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
