Hey, I defined a bunch of spark configurations <https://spark.apache.org/docs/latest/configuration.html> in spark-defaults.conf file. I made sure that the file is present under $SPARK_HOME/conf/ directory on the Livy host. All of the configs are getting picked up except spark.jars and I'm really stumped what could be going wrong.
It works fine when I pass the jars directly with the request object but not when I add them to the config file. All the jars are coming from S3 file paths if that's relevant. Any help would be appreciated! - Sanchay