[
https://issues.apache.org/jira/browse/HIVE-28111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-28111:
--------------------------------
Description:
FSCountersSummary is only printed when LLAP IO is enabled, however, it's not
LLAP IO specific as far as I know:
https://github.com/apache/hive/blob/03a76ac70370fb94a78b00496ec511e671c652f2/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java#L446-L449
it gives useful summary like:
{code}
INFO : Scheme: S3A
INFO :
----------------------------------------------------------------------------------------------
INFO : VERTICES BYTES_READ READ_OPS LARGE_READ_OPS
BYTES_WRITTEN WRITE_OPS
INFO :
----------------------------------------------------------------------------------------------
INFO : Map 1 68.53KB 4 0
0B 0
INFO : Map 11 5.54MB 68 0
0B 0
INFO : Map 2 238.48MB 480 0
0B 0
INFO : Map 5 10.71MB 80 0
0B 0
INFO : Map 6 60.68MB 360 0
0B 0
INFO : Map 8 2.69MB 24 0
0B 0
INFO : Map 9 120.03MB 480 0
0B 0
INFO : Reducer 10 0B 0 0
0B 0
INFO : Reducer 3 0B 0 0
0B 0
INFO : Reducer 4 0B 0 0
0B 0
INFO : Reducer 7 0B 0 0
0B 0
INFO :
----------------------------------------------------------------------------------------------
{code}
need to check if it prints similar data correctly in case of Tez container mode
as this is specific to the tez runner callable and the TaskCounterUpdater is
called from tez codepath
was:
FSCountersSummary is only printed when LLAP IO is enabled, however, it's not
LLAP IO specific as far as I know:
https://github.com/apache/hive/blob/03a76ac70370fb94a78b00496ec511e671c652f2/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java#L446-L449
it gives useful summary like:
{code}
INFO : Scheme: S3A
INFO :
----------------------------------------------------------------------------------------------
INFO : VERTICES BYTES_READ READ_OPS LARGE_READ_OPS
BYTES_WRITTEN WRITE_OPS
INFO :
----------------------------------------------------------------------------------------------
INFO : Map 1 68.53KB 4 0
0B 0
INFO : Map 11 5.54MB 68 0
0B 0
INFO : Map 2 238.48MB 480 0
0B 0
INFO : Map 5 10.71MB 80 0
0B 0
INFO : Map 6 60.68MB 360 0
0B 0
INFO : Map 8 2.69MB 24 0
0B 0
INFO : Map 9 120.03MB 480 0
0B 0
INFO : Reducer 10 0B 0 0
0B 0
INFO : Reducer 3 0B 0 0
0B 0
INFO : Reducer 4 0B 0 0
0B 0
INFO : Reducer 7 0B 0 0
0B 0
INFO :
----------------------------------------------------------------------------------------------
{code}
need to check if it prints similar data correctly in case of Tez container mode
> FSCountersSummary is only printed when LLAP io is enabled
> ---------------------------------------------------------
>
> Key: HIVE-28111
> URL: https://issues.apache.org/jira/browse/HIVE-28111
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Priority: Major
>
> FSCountersSummary is only printed when LLAP IO is enabled, however, it's not
> LLAP IO specific as far as I know:
> https://github.com/apache/hive/blob/03a76ac70370fb94a78b00496ec511e671c652f2/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java#L446-L449
> it gives useful summary like:
> {code}
> INFO : Scheme: S3A
> INFO :
> ----------------------------------------------------------------------------------------------
> INFO : VERTICES BYTES_READ READ_OPS LARGE_READ_OPS
> BYTES_WRITTEN WRITE_OPS
> INFO :
> ----------------------------------------------------------------------------------------------
> INFO : Map 1 68.53KB 4 0
> 0B 0
> INFO : Map 11 5.54MB 68 0
> 0B 0
> INFO : Map 2 238.48MB 480 0
> 0B 0
> INFO : Map 5 10.71MB 80 0
> 0B 0
> INFO : Map 6 60.68MB 360 0
> 0B 0
> INFO : Map 8 2.69MB 24 0
> 0B 0
> INFO : Map 9 120.03MB 480 0
> 0B 0
> INFO : Reducer 10 0B 0 0
> 0B 0
> INFO : Reducer 3 0B 0 0
> 0B 0
> INFO : Reducer 4 0B 0 0
> 0B 0
> INFO : Reducer 7 0B 0 0
> 0B 0
> INFO :
> ----------------------------------------------------------------------------------------------
> {code}
> need to check if it prints similar data correctly in case of Tez container
> mode as this is specific to the tez runner callable and the
> TaskCounterUpdater is called from tez codepath
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
