Daniel Dai created PIG-4434:
-------------------------------
Summary: Improve auto-parallelism for tez
Key: PIG-4434
URL: https://issues.apache.org/jira/browse/PIG-4434
Project: Pig
Issue Type: Improvement
Components: tez
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.15.0
Tez auto-parallelism currently has some limitation:
1. ShuffledVertexManager only decrease parallelism not increase
2. Pig currently exaggerate parallelism at frontend, ShuffledVertexManager
might get initial parallelism way large than actual, that would be costly
Instead of that, we can gradually adjust initial vertex parallelism at runtime
once upstream vertexes finishes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
