[
https://issues.apache.org/jira/browse/TEZ-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4582:
------------------------------
Description:
ShuffleErrors counters are implemented only in ShuffleScheduler (ordered,
grouped) as far as I can see. Need to investigate whether any of them is
applicable to ShuffleManager as well.
https://github.com/apache/tez/blob/607b2bc225a37b99931b860ad9e38474d4224178/tez-runtime-library/src/main/java/org/apache/tez/runtime/library/common/shuffle/orderedgrouped/ShuffleScheduler.java#L136
{code}
INFO : Shuffle Errors:
INFO : BAD_ID: 0
INFO : CONNECTION: 0
INFO : IO_ERROR: 0
INFO : WRONG_LENGTH: 0
INFO : WRONG_MAP: 0
INFO : WRONG_REDUCE: 0
{code}
was:
ShuffleErrors counters are implemented only in ShuffleScheduler (ordered,
grouped) as far as I can see. Need to investigate whether they are applicable
for ShuffleManager as well.
https://github.com/apache/tez/blob/607b2bc225a37b99931b860ad9e38474d4224178/tez-runtime-library/src/main/java/org/apache/tez/runtime/library/common/shuffle/orderedgrouped/ShuffleScheduler.java#L136
{code}
INFO : Shuffle Errors:
INFO : BAD_ID: 0
INFO : CONNECTION: 0
INFO : IO_ERROR: 0
INFO : WRONG_LENGTH: 0
INFO : WRONG_MAP: 0
INFO : WRONG_REDUCE: 0
{code}
> Implement ShuffleErrors for ShuffleManager where applicable
> -----------------------------------------------------------
>
> Key: TEZ-4582
> URL: https://issues.apache.org/jira/browse/TEZ-4582
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: László Bodor
> Priority: Major
>
> ShuffleErrors counters are implemented only in ShuffleScheduler (ordered,
> grouped) as far as I can see. Need to investigate whether any of them is
> applicable to ShuffleManager as well.
> https://github.com/apache/tez/blob/607b2bc225a37b99931b860ad9e38474d4224178/tez-runtime-library/src/main/java/org/apache/tez/runtime/library/common/shuffle/orderedgrouped/ShuffleScheduler.java#L136
> {code}
> INFO : Shuffle Errors:
> INFO : BAD_ID: 0
> INFO : CONNECTION: 0
> INFO : IO_ERROR: 0
> INFO : WRONG_LENGTH: 0
> INFO : WRONG_MAP: 0
> INFO : WRONG_REDUCE: 0
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
