Re: how to specify external jars in program with SparkConf

2018-07-12 Thread Prem Sure
I think JVM is initiated with available classpath by the time your conf execution comes... I faced this earlier during Spark1.6 and ended up moving to Spark Submit using --jars found it was not part of runtime config changes.. May I know the advantage you are trying to get programmatically On Thu,

how to specify external jars in program with SparkConf

2018-07-12 Thread mytramesh
Context :- In EMR class path has old version of jar, want to refer new version of jar in my code. through bootstrap while spinning new nodes , copied necessary jars to local folder from S3. In spark-submit command by using extra class path parameter my code able refer new version jar which is i