[
https://issues.apache.org/jira/browse/TEZ-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17428360#comment-17428360
]
László Bodor edited comment on TEZ-4180 at 10/13/21, 6:01 PM:
--------------------------------------------------------------
another example is:
{code}
2021-10-13 04:12:26,553 [INFO] [TezChild] |output.UnorderedKVOutput|: Map 3
started. MemoryAssigned=0
{code}
actually, this is a Map 1 task, which could become obvious in a form of:
{code}
Map 1 -> Map 3 started...
{code}
was (Author: abstractdog):
another example is:
{code}
2021-10-13 04:12:26,553 [INFO] [TezChild] |output.UnorderedKVOutput|: Map 3
started. MemoryAssigned=0
{code}
actually this is a Map 1 task, which could obvious in a form of:
{code}
Map 1 -> Map 3 started...
{code}
> Show convenient input -> output vertex names in output/sort messages
> --------------------------------------------------------------------
>
> Key: TEZ-4180
> URL: https://issues.apache.org/jira/browse/TEZ-4180
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Trivial
>
> While looking at aggregated yarn app logs, this message could be confusing
> (for those who're not yet familiar enough with sort/merge/etc., or tired of
> looking at huge, aggregated logs, or application logs in LLAP), as it makes
> the user think that something happens in a reducer task, but map output
> spilling happens in the Map task.
> {code}
> 2020-05-14 09:23:55,471 [INFO] [TezChild] |impl.PipelinedSorter|: Reducer 5:
> Spilling to
> /grid/2/yarn/nm/usercache/hive/appcache/application_1576231194218_0094/output/attempt_1576231194218_0094_1_12_000497_0_10147_0/file.out
> {code}
> I would prefer something like "Map 3 -> Reducer 5", and it's possible by:
> {code}
> outputContext.getTaskVertexName() -> outputContext.getDestinationVertexName()
> {code}
> This can also be useful while looking at only an excerpt from app logs (e.g.
> grepped for "Spilling").
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
