andygrove opened a new pull request, #6657: URL: https://github.com/apache/datafusion-comet/pull/6657
## Which issue does this PR close? No issue filed. This was raised during the 1.1.0-rc2 vote: in the staged Spark 4.1 binary and sources jars, `META-INF/NOTICE` omits the Apache Gluten attribution that the source `NOTICE.txt` carries. ## Rationale for this change The jars never contained `NOTICE.txt`. The ASF parent POM runs `maven-remote-resources-plugin`, which generates a `META-INF/NOTICE` holding only the ASF boilerplate (`comet-spark`, `Copyright 2020 The Apache Software Foundation`), and the shade plugin kept that one. So an attribution added to `NOTICE.txt`, such as the Apache Gluten one for `RewriteJoin`, could not reach the released jars. The 1.0.0 jars are affected the same way. ## What changes are included in this PR? In `spark/pom.xml`, the shade plugin now writes the repository's `NOTICE.txt` as `META-INF/NOTICE` (an `IncludeResourceTransformer`), and a filter stops any input jar supplying its own, since shade would otherwise write the entry twice. The shaded jar and the shaded sources jar carry `NOTICE.txt` verbatim, and a stanza added to `NOTICE.txt` later reaches them with no second copy to keep in step. Not changed: `comet-common` and the `-test-sources` jar, which are not shaded and keep the generated ASF boilerplate (neither contains Gluten-derived code), and `META-INF/LICENSE` and `META-INF/DEPENDENCIES`. One cosmetic quirk: shade 3.2.4 leaves the time on an `IncludeResourceTransformer` entry unset, so `unzip -l` shows a meaningless date (`11-27-2017`) for `META-INF/NOTICE`. It is the same on every build, so the jar stays reproducible. ## How are these changes tested? Checked on the built artifacts, since the behavior is in the packaging: - `./mvnw -P spark-4.1 -DskipTests clean verify`: the build and Apache RAT pass (`Unapproved: 0, unknown: 0`). `META-INF/NOTICE` in the jar and in the sources jar is byte-identical to `NOTICE.txt` (`cmp`), appears once, and the archives have no duplicate entries. - `./mvnw -P spark-3.5 -P scala-2.12 -DskipTests clean package`: the same result. - Against a build of `main` without this change, `META-INF/NOTICE` is the only entry that differs. The other 5,951 entries of the jar and 1,830 of the sources jar have identical names, sizes and CRCs. - Appending a line to `NOTICE.txt` and rebuilding put the line into both jars. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
