srowen commented on issue #21339: [SPARK-24287][Core] Spark -packages option should support classifier, no-transitive, and custom conf URL: https://github.com/apache/spark/pull/21339#issuecomment-476406980 This was not meant to replicate the type of dependency handling you get in a build tool. It's a convenience mostly for `spark-shell`, where you don't necessarily want to package your dependencies into a JAR to run the shell. It use, like the shell's, is more for ad hoc experimentation. If you're dealing with complicated enough dependencies that you need to manage exclusions, you need to have a build for an app. `classifier` is a pretty niche use case, but it has come up. Example: `corenlp` publishes language models as artifacts with classifiers. I can see wanting to actually use them in a simple shell, and you can't right now. Same for (as I recall) some deeplearning4j artifacts that have per-architecture variants. Anything more though I'm really not sure about.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
