[
https://issues.apache.org/jira/browse/BEAM-14157?focusedWorklogId=748791&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748791
]
ASF GitHub Bot logged work on BEAM-14157:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Mar/22 17:22
Start Date: 28/Mar/22 17:22
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on a change in pull request
#17191:
URL: https://github.com/apache/beam/pull/17191#discussion_r836670151
##########
File path:
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/GrpcWindmillServer.java
##########
@@ -632,6 +632,8 @@ private static long uniqueId() {
// The following should be protected by synchronizing on this, except for
// the atomics which may be read atomically for status pages.
private StreamObserver<RequestT> requestObserver;
+ // Indicates if the current stream in requestObserver is client closed
Review comment:
Could you clarify the language here? I'm not sure what "is client
closed" means
##########
File path:
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/GrpcWindmillServer.java
##########
@@ -742,10 +745,11 @@ public final void appendSummaryHtml(PrintWriter writer) {
writer.format(", %dms backoff remaining", sleepLeft);
}
writer.format(
- ", current stream is %dms old, last send %dms, last response %dms",
+ ", current stream is %dms old, last send %dms, last response %dms,
closed: %s",
debugDuration(nowMs, startTimeMs.get()),
debugDuration(nowMs, lastSendTimeMs.get()),
- debugDuration(nowMs, lastResponseTimeMs.get()));
+ debugDuration(nowMs, lastResponseTimeMs.get()),
+ streamClosed.get() ? "true" : "false");
Review comment:
nit: this is the same as using `Boolean.toString`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 748791)
Time Spent: 2h 40m (was: 2.5h)
> Don't send requests on a closed windmill Grpc streams
> -----------------------------------------------------
>
> Key: BEAM-14157
> URL: https://issues.apache.org/jira/browse/BEAM-14157
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Reporter: Arun Pandian
> Assignee: Arun Pandian
> Priority: P2
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> GrpcWindmillServer could send requests requests on client closed streams.
> This leads to windmill streams getting stalling occasionally for few seconds
> to few minutes. grpc-java doc says not to call onNext to send after a stream
> is client closed.
> When the streams get stalled it is logged as "Output channel stalled for {}s,
> outbound thread {}." from
> [here|https://github.com/apache/beam/blob/7727dc99ed5dc1fb46166ef496ab3607ee2779f8/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/DirectStreamObserver.java#L100]
> Ref:
> [https://github.com/grpc/grpc-java/blob/master/stub/src/main/java/io/grpc/stub/StreamObserver.java#L62]
> [https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/GrpcWindmillServer.java#L939]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
