sohami commented on a change in pull request #1504: DRILL-6792: Find the right
probe side fragment wrapper & fix DrillBuf…
URL: https://github.com/apache/drill/pull/1504#discussion_r232529229
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/RuntimeFilterVisitor.java
##########
@@ -100,8 +102,18 @@ public Prel visitJoin(JoinPrel prel, Void value) throws
RuntimeException {
@Override
public Prel visitScan(ScanPrel prel, Void value) throws RuntimeException {
if (toAddRuntimeFilter.contains(prel)) {
- //Spawn a fresh RuntimeFilterPrel over the previous identified probe
side scan node.
- RuntimeFilterPrel runtimeFilterPrel = new RuntimeFilterPrel(prel);
+ //Spawn a fresh RuntimeFilterPrel over the previous identified probe
side scan node or a runtime filter node.
+ Collection<HashJoinPrel> hashJoinPrels = probeSideScan2hj.get(prel);
+ RuntimeFilterPrel runtimeFilterPrel = null;
+ for (HashJoinPrel hashJoinPrel : hashJoinPrels) {
+ long identifier = atomicLong.incrementAndGet();
+
hashJoinPrel.getRuntimeFilterDef().setRuntimeFilterIdentifier(identifier);
+ if (runtimeFilterPrel == null) {
+ runtimeFilterPrel = new RuntimeFilterPrel(prel, identifier);
+ } else {
+ runtimeFilterPrel = new RuntimeFilterPrel(runtimeFilterPrel,
identifier);
Review comment:
in this case it will generate 2 RuntimeFilter's stacked on top of each
other. Shouldn't we instead combine the 2 RTF into single RTF with bloomFilter
aggregation using some AND operation ? Example of case where this may happen:
HJ
/ \
HJ Scan
/ \
Scan Scan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
