[
https://issues.apache.org/jira/browse/TEZ-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362836#comment-14362836
]
Jeff Zhang commented on TEZ-2193:
---------------------------------
Upload new patch (add test for ScatterGatherEdge in TestEdge) [~bikassaha]
Please help review.
bq. Perhaps we can use a pre-assembled string here if we don't print the
actual invalid value.
I think the context is important for diagnostic. So still assemble the message
with +
bq. Consumer task num can be 0 because a task in the source may not have any
consumers in this edge but may have consumers on a different edge.
EdgeManager#getNumDestinationConsumerTasks is only called when
INPUT_READ_ERROR_EVENT happens, and this event must come from the consumer
vertex with non-zero consumer tasks. Besides, if the consumer task num is 0,
there would be divided by 0 exception in TaskAttemptImpl.
{code}
float failureFraction = ((float) attempt.uniquefailedOutputReports.size())
/ outputFailedEvent.getConsumerTaskNumber();
{code}
> Check returned value from EdgeManagerPlugin before using it
> -----------------------------------------------------------
>
> Key: TEZ-2193
> URL: https://issues.apache.org/jira/browse/TEZ-2193
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
> Attachments: TEZ-2193-1.patch, TEZ-2193-2.patch, TEZ-2193-3.patch,
> TEZ-2193-4.patch
>
>
> e.g. dag has vertices v1, v2 and shuffle edge between them, and v2 has custom
> vertex manager and -1 parallelism. In this case v1's output spec may be has
> -1 physical edge which will cause task hangs in TezChild.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
