[
https://issues.apache.org/jira/browse/SPARK-19466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kay Ousterhout resolved SPARK-19466.
------------------------------------
Resolution: Fixed
Assignee: Eren Avsarogullari
Fix Version/s: 2.2.0
> Improve Fair Scheduler Logging
> ------------------------------
>
> Key: SPARK-19466
> URL: https://issues.apache.org/jira/browse/SPARK-19466
> Project: Spark
> Issue Type: Improvement
> Components: Scheduler
> Affects Versions: 2.1.0
> Reporter: Eren Avsarogullari
> Assignee: Eren Avsarogullari
> Priority: Minor
> Fix For: 2.2.0
>
>
> Fair Scheduler Logging for the following cases can be useful for the user.
> 1- If *valid* spark.scheduler.allocation.file property is set, user can be
> informed so user can aware which scheduler file is processed when
> SparkContext initializes.
> 2- If *invalid* spark.scheduler.allocation.file property is set, currently,
> the following stacktrace is shown to user. In addition to this, more
> meaningful message can be shown to user by emphasizing the problem at
> building level of fair scheduler and covering other potential issues at this
> level.
> {code:xml}
> Exception in thread "main" java.io.FileNotFoundException: INVALID_FILE (No
> such file or directory)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.<init>(FileInputStream.java:138)
> at java.io.FileInputStream.<init>(FileInputStream.java:93)
> at
> org.apache.spark.scheduler.FairSchedulableBuilder$$anonfun$buildPools$1.apply(SchedulableBuilder.scala:76)
> at
> org.apache.spark.scheduler.FairSchedulableBuilder$$anonfun$buildPools$1.apply(SchedulableBuilder.scala:75)
> {code}
> 3- If spark.scheduler.allocation.file property is not set and *default* fair
> scheduler file(fairscheduler.xml) is found in classpath, it will be loaded
> but currently, user is not informed so logging can be useful.
> 4- If spark.scheduler.allocation.file property is not set and default fair
> scheduler file does not exist, currently, user is not informed so logging can
> be useful.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
