[
https://issues.apache.org/jira/browse/BEAM-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091822#comment-16091822
]
ASF GitHub Bot commented on BEAM-1234:
--------------------------------------
GitHub user vikkyrk opened a pull request:
https://github.com/apache/beam/pull/3586
Increase the gRPC message size to max value
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [ ] Make sure there is a [JIRA
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the
change (usually before you start working on it). Trivial changes like typos do
not require a JIRA issue. Your pull request should address just this issue,
without pulling in other changes.
- [ ] Each commit in the pull request should have a meaningful subject
line and body.
- [ ] Format the pull request title like `[BEAM-1234] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-1234` with the appropriate JIRA
issue.
- [ ] Write a pull request description that is detailed enough to
understand what the pull request does, how, and why.
- [ ] Run `mvn clean verify` to make sure basic checks pass. A more
thorough check will be performed on your pull request automatically.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
---
The default message size that can be sent or received over gRPC is 4MB. For
the fn API case we do buffering at a layer above, so setting the message size
at the gRPC level to a max value.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vikkyrk/incubator-beam msg_size
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3586.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3586
----
commit 2814ed32b79c53451635b8cbc5b264b2f6c0837c
Author: Vikas Kedigehalli <[email protected]>
Date: 2017-07-18T17:06:46Z
Increase the gRPC message size to max value
----
> Consider a hint ParDo.withHighFanout()
> --------------------------------------
>
> Key: BEAM-1234
> URL: https://issues.apache.org/jira/browse/BEAM-1234
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Eugene Kirpichov
> Priority: Minor
>
> I'm finding myself again and again suggesting users on StackOverflow to
> insert fusion breaks after high-fanout ParDo's.
> I think we should just implement this as a hint on ParDo and MapElements
> transforms, like we have on GroupByKey.fewKeys() or
> Combine.withHotKeyFanout().
> E.g.: c.apply(ParDo.of(some high-fanout DoFn).withHighFanout()), and a runner
> that implements fusion could decide to insert a runner-specific fusion break.
> This somewhat sidesteps the issues in
> https://issues.apache.org/jira/browse/BEAM-730 and
> https://lists.apache.org/thread.html/ac34c9ac665a8d9f67b0254015e44c59ea65ecc1360d4014b95d3b2e@%3Cdev.beam.apache.org%3E
> because every runner can decide how to do the right thing, or is free to
> ignore the hint.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
