[
https://issues.apache.org/jira/browse/TEZ-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4180:
------------------------------
Description:
While looking at aggregated yarn app logs, this message could be confusing (for
those who're not yet familiar enough with sort/merge/etc., or tired of looking
at huge, aggregated logs, or application logs in LLAP), as it makes the user
think that something happens in a reducer task, but map output spilling happens
in the Map task.
{code}
2020-05-14 09:23:55,471 [INFO] [TezChild] |impl.PipelinedSorter|: Reducer 5:
Spilling to
/grid/2/yarn/nm/usercache/hive/appcache/application_1576231194218_0094/output/attempt_1576231194218_0094_1_12_000497_0_10147_0/file.out
{code}
I would prefer something like "Map 3 -> Reducer 5", and it's possible by:
{code}
outputContext.getTaskVertexName() -> outputContext.getDestinationVertexName()
{code}
> Show source vertex name in spilling messages
> --------------------------------------------
>
> Key: TEZ-4180
> URL: https://issues.apache.org/jira/browse/TEZ-4180
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: László Bodor
> Priority: Trivial
>
> While looking at aggregated yarn app logs, this message could be confusing
> (for those who're not yet familiar enough with sort/merge/etc., or tired of
> looking at huge, aggregated logs, or application logs in LLAP), as it makes
> the user think that something happens in a reducer task, but map output
> spilling happens in the Map task.
> {code}
> 2020-05-14 09:23:55,471 [INFO] [TezChild] |impl.PipelinedSorter|: Reducer 5:
> Spilling to
> /grid/2/yarn/nm/usercache/hive/appcache/application_1576231194218_0094/output/attempt_1576231194218_0094_1_12_000497_0_10147_0/file.out
> {code}
> I would prefer something like "Map 3 -> Reducer 5", and it's possible by:
> {code}
> outputContext.getTaskVertexName() -> outputContext.getDestinationVertexName()
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
