Sean Owen commented on SPARK-650:

BTW I am not suggesting an "empty RDD" for your case. That was specific to the 
streaming scenario.

For this, again, why not just access some initialization method during class 
init of some class that is referenced wherever you want, including a custom 
InputFormat? This can be made to happen once per JVM (class loader), from any 
code, at class init time before anything else can happen. It's just a standard 
Java mechanism.

If you mean it requires some configuration not available at class-loading time 
you can still make such an init take place wherever, as soon as, such 
configuration is available. Even in an InputFormat.

Although I can imagine corner cases where this becomes hard, I think it's 
over-thinking this to imagine a whole new lifecycle method to accomplish what 
basic JVM mechanisms allow.

> Add a "setup hook" API for running initialization code on each executor
> -----------------------------------------------------------------------
>                 Key: SPARK-650
>                 URL: https://issues.apache.org/jira/browse/SPARK-650
>             Project: Spark
>          Issue Type: New Feature
>          Components: Spark Core
>            Reporter: Matei Zaharia
>            Priority: Minor
> Would be useful to configure things like reporting libraries

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to