[
https://issues.apache.org/jira/browse/BEAM-9049?focusedWorklogId=367186&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367186
]
ASF GitHub Bot logged work on BEAM-9049:
----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Jan/20 02:43
Start Date: 07/Jan/20 02:43
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on issue #10499: [BEAM-9049] Add
opt-in --shutDownOnThrashing flag
URL: https://github.com/apache/beam/pull/10499#issuecomment-571408052
Closing this as I think we're better off making changes just for the batch
worker.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 367186)
Time Spent: 1.5h (was: 1h 20m)
> MemoryMonitor thrashing detection is too aggressive for batch workers
> ---------------------------------------------------------------------
>
> Key: BEAM-9049
> URL: https://issues.apache.org/jira/browse/BEAM-9049
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Brian Hulette
> Assignee: Brian Hulette
> Priority: Major
> Fix For: 2.19.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> In the streaming dataflow worker we've implemented push-back so that we will
> reduce parallelism when there is memory pressure. Since we cannot do this on
> the batch worker, it doesn't make sense for our thrashing detection to be so
> aggressive. We should increase the thresholds used for thrashing detection
> when running on Batch workloads.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
