[
https://issues.apache.org/jira/browse/BEAM-6708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17547499#comment-17547499
]
Kenneth Knowles commented on BEAM-6708:
---------------------------------------
This issue has been migrated to https://github.com/apache/beam/issues/19260
> Too many open files due grpc channel not being closed (direct runnner Pubsub
> source)
> ------------------------------------------------------------------------------------
>
> Key: BEAM-6708
> URL: https://issues.apache.org/jira/browse/BEAM-6708
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
> Affects Versions: 2.10.0
> Reporter: Stefan Boenders
> Priority: P3
>
> When '_read_from_pubsub' (runnners/direct/transform_evaluator.py:448) is
> being invoked, it will create a Pubsub SubscriberClient. This will create a
> grpc channel which is not being closed. When you run a pipeline with the
> direct runner which has a pubsub source in it, it will run eventually into an
> error: "too many open files". This is because these grpc channels are never
> being closed.
> The suggestion is to wrap the code after the creation of the client in a
> try/finally block and close the channel (exposed on
> sub_client.api.transport._channel) in the finally block.
>
> Some extra info about the issue:
> https://github.com/googleapis/google-cloud-python/issues/5523
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
