[
https://issues.apache.org/jira/browse/TEZ-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14363512#comment-14363512
]
Bikas Saha commented on TEZ-2193:
---------------------------------
You are right about numConsumerTasks.
The message for ScatterGather could be more informative that the current one.
Sorry I should have made this comment earlier.
{code}PhysicalOutput should not be negative for ScatterGatherEdge{code}
Perhaps, something like - "scatter gather edge manager must have destination
vertex task parallelism specified"
The Precondition will be true almost always, but the in order to call the
Precondition we will be assembling 4-5 strings for the message argument, which
is unnecessary. This can be a hot code path during job startup for 10K or more
mappers when all their outputs are being setup on the single dispatcher thread.
So optimizing a little bit would help. Maybe pre-assemble part of the string or
we could do the check in an if stmt and throw in its body.
Looks good. +1. No need to another review after making changes for the above
comments. This could potentially go all the way to 0.5 branch.
Thanks!
> Check returned value from EdgeManagerPlugin before using it
> -----------------------------------------------------------
>
> Key: TEZ-2193
> URL: https://issues.apache.org/jira/browse/TEZ-2193
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
> Attachments: TEZ-2193-1.patch, TEZ-2193-2.patch, TEZ-2193-3.patch,
> TEZ-2193-4.patch
>
>
> e.g. dag has vertices v1, v2 and shuffle edge between them, and v2 has custom
> vertex manager and -1 parallelism. In this case v1's output spec may be has
> -1 physical edge which will cause task hangs in TezChild.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
