Hi all, I wrote an application that needs an environment variable. I can set this variable with
--conf 'spark.executor.extraJavaOptions=-Dbasicauth=myuser:mypwd' in spark-submit and it works well in standalone cluster mode. But, I want set it inside the application code, because the variable contains a password. How can I do ? I tried with: SparkSession spark = SparkSession .builder() .appName("Java Spark Solr ETL") .getOrCreate(); spark.sparkContext().conf().setExecutorEnv("spark.executor.extraJavaOptions", "-Dbasicauth=myuser:mypassword"); but it doesn't work. Thanks.