[
https://issues.apache.org/jira/browse/BEAM-10334?focusedWorklogId=451640&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-451640
]
ASF GitHub Bot logged work on BEAM-10334:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Jun/20 16:26
Start Date: 26/Jun/20 16:26
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #12104:
URL: https://github.com/apache/beam/pull/12104#issuecomment-650270936
R: @lukecwik
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 451640)
Time Spent: 40m (was: 0.5h)
> Message limit for grpc message is set to default 4 MB
> -----------------------------------------------------
>
> Key: BEAM-10334
> URL: https://issues.apache.org/jira/browse/BEAM-10334
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Affects Versions: 2.18.0, 2.22.0
> Reporter: Keshav
> Priority: P2
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The default limit of a grpc message is set low to 4 MB, and is not overridden
> in the sdk_worker.py for the _control_channel. This was first discovered when
> submitting a beam job via a flink operator orchestrated by kubernetes
> ([https://github.com/GoogleCloudPlatform/flink-on-k8s-operator)|https://github.com/GoogleCloudPlatform/flink-on-k8s-operator].
> The below error (for apache-beam 2.18.0) was seen.
>
> {code:java}
> 2020/06/24 18:37:28 Executing: python -m
> apache_beam.runners.worker.sdk_worker_main Traceback (most recent call last):
> File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
> "__main__", mod_spec) File "/usr/local/lib/python3.7/runpy.py", line 85, in
> _run_code exec(code, run_globals) File
> "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py",
> line 221, in <module> main(sys.argv) File
> "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py",
> line 155, in main sdk_pipeline_options.view_as(ProfilingOptions)) File
> "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 117, in run for work_request in control_stub.Control(get_responses()):
> File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 416, in
> __next__ return self._next() File
> "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 706, in _next
> raise self grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous
> of RPC that terminated with: status = StatusCode.RESOURCE_EXHAUSTED details =
> "Received message larger than max (4847139 vs. 4194304)" debug_error_string =
> "{"created":"@1593023849.325990420","description":"Received message larger
> than max (4847139 vs.
> 4194304)","file":"src/core/ext/filters/message_size/message_size_filter.cc","file_line":188,"grpc_status":8}"
> {code}
>
> This can be fixed by modifying the options parameter to allow any size grpc
> message to be sent or received.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
