Satha created SPARK-61830:
-----------------------------

             Summary: ConsoleProgressBar fails when COLUMNS is empty
                 Key: SPARK-61830
                 URL: https://issues.apache.org/jira/browse/SPARK-61830
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 4.3.0
            Reporter: Satha


ConsoleProgressBar reads the terminal width with 
\{{sys.env.getOrElse("COLUMNS", "80").toInt}}. The default only covers the 
unset case.

If COLUMNS is set but empty (we see this on headless hosts), SparkContext fails 
to start:
{noformat}
$ COLUMNS= ./bin/spark-shell
ERROR SparkContext: Error initializing SparkContext.
java.lang.NumberFormatException: For input string: ""
{noformat}

A negative value parses, but then every job that draws the bar fails when the 
bar is cleared:
{noformat}
$ COLUMNS=-1 ./bin/spark-shell
scala> sc.parallelize(1 to 4, 4).map \{ x => Thread.sleep(1500); x }.count()
java.lang.IllegalArgumentException: count is negative: -1
  at java.lang.String.repeat(String.java:4651)
  at org.apache.spark.ui.ConsoleProgressBar.clear(ConsoleProgressBar.scala:108)
  at 
org.apache.spark.ui.ConsoleProgressBar.finishAll(ConsoleProgressBar.scala:118)
  at org.apache.spark.SparkContext.runJob(SparkContext.scala:2807)
{noformat}

The progress bar is cosmetic, so it should just fall back to 80 when COLUMNS is 
not a positive integer.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to