[
https://issues.apache.org/jira/browse/BEAM-690?focusedWorklogId=139594&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139594
]
ASF GitHub Bot logged work on BEAM-690:
---------------------------------------
Author: ASF GitHub Bot
Created on: 30/Aug/18 08:24
Start Date: 30/Aug/18 08:24
Worklog Time Spent: 10m
Work Description: janotav opened a new pull request #6303: [BEAM-690]
Backoff in the DirectRunner if no work is available
URL: https://github.com/apache/beam/pull/6303
Implementing backoff as described in the JIRA ticker for [BEAM-690].
Basically this PR:
1. adds new DriverState (CONTINUE_THROTTLE) that signals that no work was
available
2. performs capped exponential backoff in the
ExecutorServiceParallelExecutor when CONTINUE_THROTTLE is encountered
@tgroh , you are probably the most suitable reviewer
------------------------
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone
(e.g. `@username`) to look at it.
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
| --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
</br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| --- | --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 139594)
Time Spent: 10m
Remaining Estimate: 0h
> Backoff in the DirectRunner Monitor if no work is Available
> -----------------------------------------------------------
>
> Key: BEAM-690
> URL: https://issues.apache.org/jira/browse/BEAM-690
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
> Reporter: Thomas Groh
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When a Pipeline has no elements available to process, the Monitor Runnable
> will be repeatedly scheduled. Given that there is no work to be done, this
> will loop over the steps in the transform looking for timers, and prompt the
> sources to perform additional work, even though there is no work to be done.
> This consumes the entirety of a single core.
> Add a bounded backoff to rescheduling the monitor runnable if no work has
> been done since it last ran. This will reduce resource consumption on
> low-throughput Pipelines.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
