[
https://issues.apache.org/jira/browse/DRILL-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16526948#comment-16526948
]
ASF GitHub Bot commented on DRILL-6545:
---------------------------------------
amansinha100 commented on a change in pull request #1347: DRILL-6545:
Projection Push down into Lateral Join operator.
URL: https://github.com/apache/drill/pull/1347#discussion_r199016141
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillLateralJoinRelBase.java
##########
@@ -49,7 +60,51 @@ public DrillLateralJoinRelBase(RelOptCluster cluster,
RelTraitSet traits, RelNod
double rowSize = (this.getLeft().getRowType().getFieldList().size()) *
fieldWidth;
double cpuCost = rowCount * rowSize * DrillCostBase.BASE_CPU_COST;
- double memCost = 0;
+ double memCost = !excludeCorrelateColumn ? CORRELATE_MEM_COPY_COST : 0.0;
return costFactory.makeCost(rowCount, cpuCost, 0, 0, memCost);
}
+
+ @Override
+ protected RelDataType deriveRowType() {
+ switch (joinType) {
+ case LEFT:
+ case INNER:
+ return
constructRowType(SqlValidatorUtil.deriveJoinRowType(left.getRowType(),
+ right.getRowType(), joinType.toJoinType(),
+ getCluster().getTypeFactory(), null,
+ ImmutableList.<RelDataTypeField>of()));
+ case ANTI:
+ case SEMI:
+ return constructRowType(left.getRowType());
+ default:
+ throw new IllegalStateException("Unknown join type " + joinType);
+ }
+ }
+
+ public int getInputSize(int offset, RelNode input) {
+ if (this.excludeCorrelateColumn &&
+ offset == 0) {
+ return input.getRowType().getFieldList().size() - 1;
+ }
+ return input.getRowType().getFieldList().size();
+ }
+
+ public RelDataType constructRowType(RelDataType inputRowType) {
+ List<RelDataType> fields = new ArrayList<>();
+ List<String> fieldNames = new ArrayList<>();
+ if (excludeCorrelateColumn) {
+ int corrVariable = this.requiredColumns.nextSetBit(0);
Review comment:
An assumption here is that there is only 1 correlated column which is true
for Drill's Lateral-Unnest implementation but I believe Calcite allows
multiple. You could just put a Preconditions check here to be safe.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Projection Push down into Lateral Join operator.
> ------------------------------------------------
>
> Key: DRILL-6545
> URL: https://issues.apache.org/jira/browse/DRILL-6545
> Project: Apache Drill
> Issue Type: Improvement
> Components: Query Planning & Optimization
> Reporter: Hanumath Rao Maduri
> Assignee: Hanumath Rao Maduri
> Priority: Major
> Fix For: 1.14.0
>
>
> For the Lateral’s logical and physical plan node, we would need to add an
> output RowType such that a Projection can be pushed down to Lateral.
> Currently, Lateral will produce all columns from left and right and it
> depends on a subsequent Project to eliminate unneeded columns. However, this
> will blow up the memory use of Lateral since each column from the left will
> be replicated N times based on N rows coming from UNNEST. We can have a
> ProjectLateralPushdownRule that pushes only the plain columns onto LATERAL
> but keeps the expression evalulations as part of the Project above the
> Lateral.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
