vanzin commented on a change in pull request #23618: [SPARK-26694][CORE]
Progress bar should be enabled by default for spark-shell
URL: https://github.com/apache/spark/pull/23618#discussion_r250345359
##########
File path: core/src/main/scala/org/apache/spark/SparkContext.scala
##########
@@ -434,7 +434,7 @@ class SparkContext(config: SparkConf) extends Logging {
_statusTracker = new SparkStatusTracker(this, _statusStore)
_progressBar =
- if (_conf.get(UI_SHOW_CONSOLE_PROGRESS) && !log.isInfoEnabled) {
+ if (_conf.get(UI_SHOW_CONSOLE_PROGRESS)) {
Review comment:
The issue with removing the log check is that if you actually have info logs
enabled, it looks a bit weird. e.g.:
```
[Stage 0:=================================================> (88 + 8) /
100]19/01/23 11:40:31 INFO Executor: Finished task 88.0 in stage 0.0 (TID 88).
658 bytes result sent to driver
19/01/23 11:40:31 INFO Executor: Finished task 89.0 in stage 0.0 (TID 89).
615 bytes result sent to driver
19/01/23 11:40:31 INFO Executor: Finished task 90.0 in stage 0.0 (TID 90).
658 bytes result sent to driver
...
[Stage 0:=====================================================> (96 + 4) /
100]19/01/23 11:40:32 INFO Executor: Finished task 96.0 in stage 0.0 (TID 96).
658 bytes result sent to driver
19/01/23 11:40:32 INFO Executor: Finished task 97.0 in stage 0.0 (TID 97).
658 bytes result sent to driver
19/01/23 11:40:32 INFO Executor: Finished task 98.0 in stage 0.0 (TID 98).
658 bytes result sent to driver
...
19/01/23 11:40:32 INFO DAGScheduler: Job 0 finished: collect at
<console>:25, took 13.862774 s
```
You could keep this info when you override the configuration in
`Logging.scala` (keep a boolean that says whether the progress bar should be
enabled, e.g.), but not sure that's worth it. The output looks weird but then,
in this mode, it's already pretty noisy.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
