[
https://issues.apache.org/jira/browse/BEAM-14157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Pandian updated BEAM-14157:
--------------------------------
Description:
GrpcWindmillServer could send requests requests on client closed streams. This
leads to windmill streams getting stalling occasionally for few seconds to few
minutes. grpc-java doc says not to call onNext to send after a stream is client
closed.
When the streams get stalled it is logged as "Output channel stalled for {}s,
outbound thread {}." from
[here](https://github.com/apache/beam/blob/7727dc99ed5dc1fb46166ef496ab3607ee2779f8/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/DirectStreamObserver.java#L100)
Ref:
[https://github.com/grpc/grpc-java/blob/master/stub/src/main/java/io/grpc/stub/StreamObserver.java#L62]
[https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/GrpcWindmillServer.java#L939]
was:
GetWorkStream could send new getwork requests on a closed stream
https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/GrpcWindmillServer.java#L939
> Don't send requests on a closed windmill Grpc streams
> -----------------------------------------------------
>
> Key: BEAM-14157
> URL: https://issues.apache.org/jira/browse/BEAM-14157
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Reporter: Arun Pandian
> Priority: P2
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> GrpcWindmillServer could send requests requests on client closed streams.
> This leads to windmill streams getting stalling occasionally for few seconds
> to few minutes. grpc-java doc says not to call onNext to send after a stream
> is client closed.
> When the streams get stalled it is logged as "Output channel stalled for {}s,
> outbound thread {}." from
> [here](https://github.com/apache/beam/blob/7727dc99ed5dc1fb46166ef496ab3607ee2779f8/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/DirectStreamObserver.java#L100)
>
> Ref:
> [https://github.com/grpc/grpc-java/blob/master/stub/src/main/java/io/grpc/stub/StreamObserver.java#L62]
> [https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/GrpcWindmillServer.java#L939]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
