[
https://issues.apache.org/jira/browse/TEZ-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4477:
------------------------------
Description:
And optionally make it logged, e.g.
1. AM can log the full dag once
2. Tasks can log only the actual vertex and its inputs/outputs
{code:java}
-------- ----------
| Map1 | -SIMPLE_EDGE->| Reducer2 |
-------- ----------{code}
currently, we have a log line like this in tasks (which can be translated to
ascii art maybe):
{code}
2022-11-16 18:33:01,627 [INFO] [main] |runtime.LogicalIOProcessorRuntimeTask|:
Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : delete from
concurrent_delete_dynamic ...0.0 (Stage-1), VertexName: Reducer 2,
VertexParallelism: 10, TaskAttemptID:attempt_1668609417637_0043_5_01_000006_0,
processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor,
inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{
sourceVertexName=Map 1, physicalEdgeCount=4,
inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }},
], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0,
outputClassName=org.apache.hadoop.hive.ql.exec.tez.NullMROutput }}, ],
taskConfEntryCount=1
{code}
was:
And optionally make it logged, e.g.
1. AM can log the full dag once
2. Tasks can log only the actual vertex and its inputs/outputs
{code:java}
-------- ----------
| Map1 | -SIMPLE_EDGE->| Reducer2 |
-------- ----------{code}
> Create ASCII art DAG representation for fun
> -------------------------------------------
>
> Key: TEZ-4477
> URL: https://issues.apache.org/jira/browse/TEZ-4477
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: László Bodor
> Priority: Minor
>
> And optionally make it logged, e.g.
> 1. AM can log the full dag once
> 2. Tasks can log only the actual vertex and its inputs/outputs
> {code:java}
> -------- ----------
> | Map1 | -SIMPLE_EDGE->| Reducer2 |
> -------- ----------{code}
>
>
> currently, we have a log line like this in tasks (which can be translated to
> ascii art maybe):
> {code}
> 2022-11-16 18:33:01,627 [INFO] [main]
> |runtime.LogicalIOProcessorRuntimeTask|: Initializing
> LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : delete from
> concurrent_delete_dynamic ...0.0 (Stage-1), VertexName: Reducer 2,
> VertexParallelism: 10,
> TaskAttemptID:attempt_1668609417637_0043_5_01_000006_0,
> processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor,
> inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{
> sourceVertexName=Map 1, physicalEdgeCount=4,
> inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }},
> ], outputSpecList=[{{ destinationVertexName=out_Reducer 2,
> physicalEdgeCount=0,
> outputClassName=org.apache.hadoop.hive.ql.exec.tez.NullMROutput }}, ],
> taskConfEntryCount=1
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
