[
https://issues.apache.org/jira/browse/TEZ-714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384421#comment-14384421
]
Bikas Saha commented on TEZ-714:
--------------------------------
bq. Maybe an issue: when vertex's output is being committing as an vertex group
output, then what state should this vertex in ? Currently vertex will go to
SUCCEEDED state, but may be better to move to COMMITTING state
It depends. The state issue is probably minor. There is complexity when there
are multiple outputs which may be independent or grouped.
bq. Should Task wait for all the task attempts completed before move to
SUCCEEDED, otherwise it is possible that vertex is in COMMITTING while there
still task attempt is still running in the case of speculation.
This is a known issue. There is a comment somewhere in the code but I dont
think its tracked by a jira.
> OutputCommitters should not run in the main AM dispatcher thread
> ----------------------------------------------------------------
>
> Key: TEZ-714
> URL: https://issues.apache.org/jira/browse/TEZ-714
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Jeff Zhang
> Priority: Critical
> Attachments: DAG_2.pdf, TEZ-714-1.patch, TEZ-714-2.patch,
> TEZ-714-3.patch, TEZ-714-4.patch, TEZ-714-5.patch, Vertex_2.pdf
>
>
> Follow up jira from TEZ-41.
> 1) If there's multiple OutputCommitters on a Vertex, they can be run in
> parallel.
> 2) Running an OutputCommitter in the main thread blocks all other event
> handling, w.r.t the DAG, and causes the event queue to back up.
> 3) This should also cover shared commits that happen in the DAG.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
