ASF GitHub Bot commented on DRILL-6295:

Github user ilooner commented on a diff in the pull request:

    --- Diff: 
    @@ -118,105 +127,114 @@ public PartitionOutgoingBatch 
getOutgoingBatches(int index) {
         return null;
    -  @VisibleForTesting
    -  protected List<Partitioner> getPartitioners() {
    +  List<Partitioner> getPartitioners() {
         return partitioners;
        * Helper to execute the different methods wrapped into same logic
        * @param iface
    -   * @throws IOException
    +   * @throws ExecutionException
    -  protected void executeMethodLogic(final GeneralExecuteIface iface) 
throws IOException {
    -    if (partitioners.size() == 1 ) {
    -      // no need for threads
    -      final OperatorStats localStatsSingle = 
    -      localStatsSingle.clear();
    -      localStatsSingle.startProcessing();
    +  @VisibleForTesting
    +  void executeMethodLogic(final GeneralExecuteIface iface) throws 
ExecutionException {
    +    // To simulate interruption of main fragment thread and interrupting 
the partition threads, create a
    +    // CountDownInject latch. Partitioner threads await on the latch and 
main fragment thread counts down or
    +    // interrupts waiting threads. This makes sure that we are actually 
interrupting the blocked partitioner threads.
    +    try (CountDownLatchInjection testCountDownLatch = 
injector.getLatch(context.getExecutionControls(), "partitioner-sender-latch")) {
    --- End diff --
    I'm not sure that we should be using the injector to create a count down 
latch here. My understanding is that we have to define a 
`partitioner-sender-latch` injection site on the 
`"drill.exec.testing.controls"` property and it is intended only for testing. 
See ControlsInjectionUtil.createLatch(). The default value for 
`drill.exec.testing.controls` is empty so the getLatch method would return a 
Noop latch since `partitioner-sender-latch` is undefined. Since we always want 
to create a count down latch here (not just for testing) shouldn't we directly 
create one?

> PartitionerDecorator may close partitioners while CustomRunnable are active 
> during query cancellation
> -----------------------------------------------------------------------------------------------------
>                 Key: DRILL-6295
>                 URL: https://issues.apache.org/jira/browse/DRILL-6295
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Vlad Rozov
>            Assignee: Vlad Rozov
>            Priority: Critical
>             Fix For: 1.14.0
> During query cancellation, in case 
> {{PartitionerDecorator.executeMethodLogic()}} is active (waiting on the 
> {{latch}}), the wait will be interrupted and {{Futures}} cancelled, but there 
> is no guarantee that all {{CustomRunnable}} terminate before returning from 
> {{PartitionerDecorator.executeMethodLogic()}}. On exit, both income and 
> outgoing batches are cleared, leading to clearing of underlying {{Vectors}} 
> and {{DrillBufs}}. This eventually causes unallocated memory access and JVM 
> crash as {{CustomRunnable}} may execute after income/outgoing batches are 
> cleared.

This message was sent by Atlassian JIRA

Reply via email to