yhuai commented on issue #27177: [SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier URL: https://github.com/apache/spark/pull/27177#issuecomment-573863580 @dongjoon-hyun Sure. Say we have a Spark application that depends on a third-party dependency foo, which pulls in jackson as a transient dependency. Unfortunately, foo depends on a different version of jackson than Spark. So, in the pom of this Spark application, we use dependency management section to pin the version of jackson. By doing this, we are lifting jackson to the top level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. For my case, I am fine to directly access git to get these audit files. @srowen This new format helps us get the classifier in an indirect way. Taking `avro-mapred/1.8.2/avro-mapred-1.8.2-hadoop2.jar` as an example, the first field is artifact id and the secon field is the version number. Then, when we look at the jar name, we will know hadoop2 is the classifier instead treating 1.8.2-hadoop2 as the version number.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
