ASF GitHub Bot commented on FLINK-8694:

GitHub user pnowojski opened a pull request:


    [FLINK-8694][runtime] Fix notifyDataAvailable race condition

    This fixes two bugs in network stack:
    ## Brief change log
    Please check individual commit messages.
    ## Verifying this change
    This PR adds new tests covering the previously bugged cases.
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): (yes / **no**)
      - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (yes / **no**)
      - The serializers: (yes / **no** / don't know)
      - The runtime per-record code paths (performance sensitive): (yes / 
**no** / don't know)
      - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
      - The S3 file system connector: (yes / **no** / don't know)
    ## Documentation
      - Does this pull request introduce a new feature? (yes / **no**)
      - If yes, how is the feature documented? (**not applicable** / docs / 
JavaDocs / not documented)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/pnowojski/flink f8694-proper-fix

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5572
commit 388d16118763dddff7d4c3593572169ad3e65c0d
Author: Piotr Nowojski <piotr.nowojski@...>
Date:   2018-02-23T10:37:37Z

    [hotfix][tests] Deduplicate code in SingleInputGateTest

commit e22a44b24ab1e9f02c236440f899a1f4dfdfc873
Author: Piotr Nowojski <piotr.nowojski@...>
Date:   2018-02-23T11:11:14Z

    [hotfix][runtime] Remove duplicated check

commit 5c16e565c4a7f0ffdaec888696d98e3c2c221d99
Author: Piotr Nowojski <piotr.nowojski@...>
Date:   2018-02-23T10:20:21Z

    [FLINK-8760][runtime] Correctly propagate moreAvailable flag through 
    Previously if we SingleInputGate was re-eqnqueuing an input channel, 
    might incorrectly return false. This might caused some dead locks.

commit a451006fd2e38e478ef745fd9de0ebc5fb2fd5c2
Author: Piotr Nowojski <piotr.nowojski@...>
Date:   2018-02-23T10:27:54Z

    [hotfixu][tests] Do not hide original exception in 

commit e70cd04424f0f92b9d5127e7c4a351d3823d20bd
Author: Piotr Nowojski <piotr.nowojski@...>
Date:   2018-02-23T10:28:20Z

    [FLINK-8694][runtime] Fix notifyDataAvailable race condition
    Before there was a race condition that might resulted in igonoring some 
notifyDataAvailable calls.
    This fixes the problem by moving buffersAvailable handling to Supartitions 
and adds stress test
    for flushAlways (without this fix this test is dead locking).


> Make notifyDataAvailable call reliable
> --------------------------------------
>                 Key: FLINK-8694
>                 URL: https://issues.apache.org/jira/browse/FLINK-8694
>             Project: Flink
>          Issue Type: Sub-task
>            Reporter: Piotr Nowojski
>            Assignee: Piotr Nowojski
>            Priority: Major
> After FLINK-8591 
> org.apache.flink.runtime.io.network.netty.SequenceNumberingViewReader#notifyDataAvailable
>  (and the same for Credit base flow control) due to race condition can be 
> sometimes ignored. 

This message was sent by Atlassian JIRA

Reply via email to