Paul Rogers created DRILL-7451:
----------------------------------
Summary: Planner inserts project node even if scan handles project
push-down
Key: DRILL-7451
URL: https://issues.apache.org/jira/browse/DRILL-7451
Project: Apache Drill
Issue Type: Bug
Reporter: Paul Rogers
I created a "dummy" storage plugin for testing. The test does a simple query:
{code:sql}
SELECT a, b, c from dummy.myTable
{code}
The first test is to mark the plugin's group scan as supporting projection push
down. However, Drill still creates a projection node in the logical plan:
{code:json}
"graph" : [ {
"pop" : "DummyGroupScan",
"@id" : 2,
"columns" : [ "`**`" ],
"userName" : "progers",
"cost" : {
"memoryCost" : 1.6777216E7,
"outputRowCount" : 10000.0
}
}, {
"pop" : "project",
"@id" : 1,
"exprs" : [ {
"ref" : "`a`",
"expr" : "`a`"
}, {
"ref" : "`b`",
"expr" : "`b`"
}, {
"ref" : "`c`",
"expr" : "`c`"
} ],
"child" : 2,
"outputProj" : true,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : {
"memoryCost" : 1.6777216E7,
"outputRowCount" : 10000.0
}
}, {
"pop" : "screen",
"@id" : 0,
"child" : 1,
"initialAllocation" : 1000000,
"maxAllocation" : 10000000000,
"cost" : {
"memoryCost" : 1.6777216E7,
"outputRowCount" : 10000.0
}
} ]
{code}
There is [a comment in the
code|https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushProjectIntoScanRule.java#L109]
that suggests the project should be removed:
{code:java}
// project above scan may be removed in ProjectRemoveRule for
// the case when it is trivial
{code}
As shown in the example, the project is trivial. There is a subtlety: it may be
that the scan, unknown to the planner, produce additional columns, say {{d}}
and {{e}} which the project operator is needed to remove.
If this is the reason the project remains, perhaps we can add a flag of some
kind where the group scan can insist that not only does it handle projection,
it will not insert additional columns. At that point, the project is completely
unnecessary in this case.
This is not a functional bug; just a performance issue: we exercise the
machinery of the project operator to do exactly nothing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
