yhuai commented on issue #27177: [SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier URL: https://github.com/apache/spark/pull/27177#issuecomment-573504504 @dongjoon-hyun @srowen What we are trying to get is a flattened list of dependencies that still allow us to tell the values of artifact id, classifier, and version. So, these files can be consumed by automated tests or downstream tools. Given we can have jar names like `avro-mapred-1.8.2-hadoop2.jar` and `htrace-core-3.1.0-incubating.jar`, we will not be able to have a simple rule on how to parse the jar name to get values of different fields. Also, for someone who is looking at jar names, he/she will need to search around in order to figure out that hadoop2 is a classifier and incubating is part of the version number. I think it is helpful to save people's time on figuring out which part is the classifier. Regarding having some redundant information, these information can help us figure out the exact values of each field used to identify a maven dependency. So, I think it is fine. These files are automatically generated. So, having redundant information does not introduce a higher cost of updating these files. Finally, regarding backport, I suggest that we push this change to branch-2.4 as well. So, 2.4 and master will use the same format. Checking it into branch-2.4 is fine since these audit files and the shell script are not something affecting how spark works.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
