[
https://issues.apache.org/jira/browse/BEAM-7647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17122765#comment-17122765
]
Beam JIRA Bot commented on BEAM-7647:
-------------------------------------
This issue is P2 but has been unassigned without any comment for 60 days so it
has been labeled "stale-P2". If this issue is still affecting you, we care!
Please comment and remove the label. Otherwise, in 14 days the issue will be
moved to P3.
Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed
explanation of what these priorities mean.
> CombineGlobally translation is risky and not very performant.
> -------------------------------------------------------------
>
> Key: BEAM-7647
> URL: https://issues.apache.org/jira/browse/BEAM-7647
> Project: Beam
> Issue Type: Improvement
> Components: runner-spark
> Reporter: Etienne Chauchot
> Priority: P2
> Labels: stale-P2
>
> In combine globally:
> {code:java}
> Iterable<WindowedValue<OutputT>> output =
> sparkCombineFn.extractOutput(maybeAccumulated.get());
> outRdd =
> context
> .getSparkContext()
> .parallelize(CoderHelpers.toByteArrays(output, wvoCoder))
> .map(CoderHelpers.fromByteFunction(wvoCoder));
> {code}
> => risk of OOM in the list, get data to a single worker (the driver)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
