[
https://issues.apache.org/jira/browse/TEZ-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005251#comment-16005251
]
Jonathan Eagles commented on TEZ-3719:
--------------------------------------
I noticed this exact slow down when I was profiling tez recently and
calculating progress was showing up as a AM scalability and performance issue.
{code}
+ if (v.getState() == VertexState.SUCCEEDED) {
+ progress += 1f;
+ }
progress += v.getProgress();
{code}
Couple of things. Progress should only be calculated if the vertex is running.
Since it should be 0 before running and 1 after finished. Also the calculation
should be in the "else" block. Both VertexImpl#getState and
VertexImpl#getProgress lock and unlock the readlock. If you are adding logic to
getProgress to handle the vertex state. You can avoid the double locking. The
same could be said for the TaskImpl#getState TaskImpl#getProgress calls.
> DAGImpl.computeProgress slows down dispatcher and ipc threads
> -------------------------------------------------------------
>
> Key: TEZ-3719
> URL: https://issues.apache.org/jira/browse/TEZ-3719
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Gopal V
> Attachments: dispatcher_holding_lock.png, TEZ-3719.wip.patch,
> tez_computeProgress_waiting_for_lock.png
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
