Marcelo Vanzin created SPARK-21568:
--------------------------------------
Summary: ConsoleProgressBar should only be enabled in shells
Key: SPARK-21568
URL: https://issues.apache.org/jira/browse/SPARK-21568
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.2.0
Reporter: Marcelo Vanzin
Priority: Minor
This is the current logic that enables the progress bar:
{code}
_progressBar =
if (_conf.getBoolean("spark.ui.showConsoleProgress", true) &&
!log.isInfoEnabled) {
Some(new ConsoleProgressBar(this))
} else {
None
}
{code}
That is based on the logging level; it just happens to align with the default
configuration for shells (WARN) and normal apps (INFO).
But if someone changes the default logging config for their app, this may
break; they may silence logs by setting the default level to WARN or ERROR, and
a normal application will see a lot of log spam from the progress bar (which is
especially bad when output is redirected to a file, as is usually done when
running in cluster mode).
While it's possible to disable the progress bar separately, this behavior is
not really expected.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
