Kip Kohn created GOBBLIN-2185: --------------------------------- Summary: Implement heuristic-based GoT Dynamic Auto-Scaling Key: GOBBLIN-2185 URL: https://issues.apache.org/jira/browse/GOBBLIN-2185 Project: Apache Gobblin Issue Type: New Feature Components: gobblin-core Reporter: Kip Kohn Assignee: Abhishek Tiwari
Using a configured (constant) Data Transfer Rate (in bytes per time), presume a linear relationship holds between "Work" (WU) throughput and scaling the number of worker-containers. Provide a heuristic-based recommendation for how many worker-containers to allocate in order to complete processing of a job within a given time budget, with volume of Work conveyed via `WorkUnitsSizeSummary` -- This message was sent by Atlassian Jira (v8.20.10#820010)