sohami commented on a change in pull request #1504: DRILL-6792: Find the right
probe side fragment wrapper & fix DrillBuf…
URL: https://github.com/apache/drill/pull/1504#discussion_r236396758
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java
##########
@@ -1013,9 +1041,9 @@ public IterOutcome executeBuildPhase() throws
SchemaChangeException {
}
if (spilledState.isFirstCycle() && enableRuntimeFilter) {
- if (bloomFilters.size() > 0) {
+ if (bloomFilter2buildId.size() > 0) {
int hashJoinOpId = this.popConfig.getOperatorId();
- runtimeFilterReporter.sendOut(bloomFilters, probeFields,
this.popConfig.getRuntimeFilterDef(), hashJoinOpId);
+ runtimeFilterReporter.sendOut(new
ArrayList<>(bloomFilter2buildId.keySet()), probeFields,
this.popConfig.getRuntimeFilterDef(), hashJoinOpId);
Review comment:
We create `BloomFilter` for each build side field. But here when sending out
the filters for all fields the order of `bloomFilter` and `probeFields` in
their respective list can be different. On the receiving end first message can
have bloomFilter <bf11, bf12> with probe fields <f1, f2> whereas second message
can have <bf22, bf21> and probe fields still <f1, f2>. Hence `aggregator` will
aggregate <bf11 with bf22> and <bf12 with bf21> which can result in wrong
results. Same is happening in `applyRuntimeFilter` in RTF operator where I
guess it's assumed order of probe fields and bloomfilter list is consistent.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
