I am an apache spark newbie and have a question regarding
spark.executor.extrajavaoptions configuration property present in spark
I have a requirement to start a javaagent on spark executor in standalone
mode of spark interactive shell & Spark-submit. In order to do the same, I
thought of using "spark.executor.extrajavaoptions" configuration property
and passing it through "properties-file" option.
Interestingly when I pass javaagent in "spark.driver.extrajavaoptions", it
works just fine, but I need to do it at executor level and somehow
spark.executor.extrajavaoptions=javaagent:<JAR_PATH> does not work.
I am not sure why this is happening. Is it part of spark design itself? Can
you give pointers on why javaagent does not work with executor but works
just fine with driver? Any idea what I may be doing wrong here?
I really appreciate your help and support in this regard.
Thanks and Regards