shardulm94 commented on pull request #31741: URL: https://github.com/apache/spark/pull/31741#issuecomment-791471536
@xkrogen In my experience, I haven't seen an intentional case of non-jar dependencies that are required for runtime/compilation. Most such dependencies are often bundled as part of resource files within jars. So my first instinct was to ignore those. Secondly existing code is built around these being jars, e.g. https://github.com/apache/spark/blob/17601e014c6ccb48958d35ffb04bedeac8cfc66a/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L318 uses `addJarToClasspath`, although technically we can pass any file to `addJarToClasspath`. I am open to implementing this either way. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
