[
https://issues.apache.org/jira/browse/HIVE-6314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Steingold updated HIVE-6314:
--------------------------------
Description:
The progress report is issued every second even when no progress have been made:
{code}
2014-01-27 10:35:55,209 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68
sec
2014-01-27 10:35:56,678 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68
sec
2014-01-27 10:35:59,344 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68
sec
2014-01-27 10:36:01,268 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 8.67
sec
2014-01-27 10:36:03,149 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 8.67
sec
{code}
This pollutes the logs and the screen, and people do not appreciate it as much
as the designers might have thought ([How do I limit log verbosity of
hive?|http://stackoverflow.com/q/20849289/850781], [controlling the level of
verbosity in Hive|http://stackoverflow.com/q/14121543/850781]).
It would be nice to be able to control the level of verbosity (but *not* by the
{{-v}} switch!):
# Make sure that the progress report is only issued where there is something
new to report; or
# Remove all the progress messages; or
# Make sure that progress is reported only every X sec (instead of every 1
second)
was:
The progress report is issued every second even when no progress have been made:
{code}
2014-01-27 10:35:55,209 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68
sec
2014-01-27 10:35:56,678 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68
sec
2014-01-27 10:35:59,344 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68
sec
2014-01-27 10:36:01,268 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 8.67
sec
2014-01-27 10:36:03,149 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 8.67
sec
{code}
This pollutes the logs and the screen, and people do not appreciate it as much
as the designers might have thought
([http://stackoverflow.com/questions/20849289/how-do-i-limit-log-verbosity-of-hive],
[http://stackoverflow.com/questions/14121543/controlling-the-level-of-verbosity-in-hive]).
It would be nice to be able to control the level of verbosity (but *not* by the
{{-v}} switch!):
# Make sure that the progress report is only issued where there is something
new to report; or
# Remove all the progress messages; or
# Make sure that progress is reported only every X sec (instead of every 1
second)
> The logging (progress reporting) is too verbose
> -----------------------------------------------
>
> Key: HIVE-6314
> URL: https://issues.apache.org/jira/browse/HIVE-6314
> Project: Hive
> Issue Type: Bug
> Reporter: Sam Steingold
> Assignee: Navis
> Labels: logger
> Attachments: HIVE-6314.1.patch.txt, HIVE-6314.2.patch
>
>
> The progress report is issued every second even when no progress have been
> made:
> {code}
> 2014-01-27 10:35:55,209 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68
> sec
> 2014-01-27 10:35:56,678 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68
> sec
> 2014-01-27 10:35:59,344 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68
> sec
> 2014-01-27 10:36:01,268 Stage-1 map = 100%, reduce = 100%, Cumulative CPU
> 8.67 sec
> 2014-01-27 10:36:03,149 Stage-1 map = 100%, reduce = 100%, Cumulative CPU
> 8.67 sec
> {code}
> This pollutes the logs and the screen, and people do not appreciate it as
> much as the designers might have thought ([How do I limit log verbosity of
> hive?|http://stackoverflow.com/q/20849289/850781], [controlling the level of
> verbosity in Hive|http://stackoverflow.com/q/14121543/850781]).
> It would be nice to be able to control the level of verbosity (but *not* by
> the {{-v}} switch!):
> # Make sure that the progress report is only issued where there is something
> new to report; or
> # Remove all the progress messages; or
> # Make sure that progress is reported only every X sec (instead of every 1
> second)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
