[
https://issues.apache.org/jira/browse/BEAM-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh resolved BEAM-3045.
-------------------------------
Resolution: Resolved
Fix Version/s: Not applicable
The implementation of Dataflow autoscaling algorithms is not part of the Beam
SDK. It's implemented within the managed Dataflow service, and is not open
sourced. The flags are used only to signal the desired behavior to the service.
> Can't find autoscaling algorithm for streaming workflows
> --------------------------------------------------------
>
> Key: BEAM-3045
> URL: https://issues.apache.org/jira/browse/BEAM-3045
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Zhuo Liu
> Assignee: Thomas Groh
> Priority: Minor
> Fix For: Not applicable
>
>
> Hi I see there is an auto-scaling algorithm for google dataflow.
> https://cloud.google.com/dataflow/service/dataflow-service-desc
> Also enum AutoscalingAlgorithmType{NONE, BASIC, THROUGHPUT_BASED} could be
> found in
> org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions,
> however, I was not able to find the actual algorithm implementation for the
> THROUGHPUT_BASED scaling algorithm.
> Anyone could help?
> I was especially interested in the autoscaling algorithm for streaming
> workflows.
> Thanks a lot!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
