sohami commented on a change in pull request #1459: DRILL-6731: Move the BFs
aggregating work from the Foreman to the RuntimeFi…
URL: https://github.com/apache/drill/pull/1459#discussion_r220710086
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/work/filter/RuntimeFilterSink.java
##########
@@ -89,16 +142,27 @@ public void close() throws Exception {
@Override
public void run() {
- while (running.get()) {
- RuntimeFilterWritable toAggregate = rfQueue.poll();
- if (toAggregate != null) {
- if (aggregated != null) {
- aggregated.aggregate(toAggregate);
- currentBookId.incrementAndGet();
+ try {
Review comment:
I still see there are race conditions. 1) containOne() is checked outside
the lock 2) Same running flag is used by all 3 threads (netty thread,
AsyncAggregateWorker and MinorFragment). Consider a case when Netty thread
checks the `running` state and it is true and was preempted, then minor
fragment thread came along and reset the running state and cleared up the
queue. After this the netty thread was scheduled and it add's the
`RumtimeFilterWritable` inside the queue. Now no-one will cleanup this newly
added filter.
Please see the top commit in this branch:
https://github.com/sohami/drill/commits/DRILL-6731, if you think it looks good
to you then cherry-pick this commit in your PR and only squash your original
commit not the cherry-picked commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
