[
https://issues.apache.org/jira/browse/TEZ-714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367542#comment-14367542
]
Bikas Saha commented on TEZ-714:
--------------------------------
I understand that. What I expected was code changes around the code that
invokes the commit to change from sync to async. And new transitions from
committing state. But there were changes to other parts of the code too.
Perhaps I am missing something. I will take a closer look in the next patch
where async operation are on a per commit basis. Also, not sure why
group-commit and non-group commit need to be differentiated in different
transitions. If the next patch continues to differentiate them (instead of just
being able to count pending operations) then perhaps you can add a comment on
why its necessary so that its easy to understand the cause.
> OutputCommitters should not run in the main AM dispatcher thread
> ----------------------------------------------------------------
>
> Key: TEZ-714
> URL: https://issues.apache.org/jira/browse/TEZ-714
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Jeff Zhang
> Priority: Critical
> Attachments: DAG_2.pdf, TEZ-714-1.patch, Vertex_2.pdf
>
>
> Follow up jira from TEZ-41.
> 1) If there's multiple OutputCommitters on a Vertex, they can be run in
> parallel.
> 2) Running an OutputCommitter in the main thread blocks all other event
> handling, w.r.t the DAG, and causes the event queue to back up.
> 3) This should also cover shared commits that happen in the DAG.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
