The JavaDocs <https://spark.apache.org/docs/2.0.2/api/java/org/apache/spark/launcher/SparkAppHandle.Listener.html> say that `infoChanged` is a "callback for changes in any information that is not the handle's state."
*Can I get more information on what counts as a change in information? How often does this callback occur?* I am running into an issue on YARN where the Mapper container is failing due to a timeout. I want to pass a Hadoop Context <https://github.com/apache/hadoop/blob/f67237cbe7bc48a1b9088e990800b37529f1db2a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Mapper.java#L106> and periodically update the status to let the Mapper know that the Spark job is making progress. I've included some pseudocode below for what I would like to do. This code would only work if `infoChanged` is called frequently (i.e. if my Mapper timeout is 10 minutes, `infoChanged` should be called more frequently than once every 10 minutes). public static class SparkAppHandleListener implements SparkAppHandle.Listener { private final Context context; public SparkAppHandleListener(Context context) { this.context = context; } @Override public void stateChanged(SparkAppHandle appHandle) { this.context.setStatus("some update here"); } @Override public void infoChanged(SparkAppHandle appHandle) { this.context.setStatus("some update here"); } } SparkLauncher sparkLauncher = ... Context context = ... sparkLauncher.startApplication(new SparkAppHandleListener(context)); Thanks, Benson
