ppkarwasz opened a new pull request, #4690: URL: https://github.com/apache/solr/pull/4690
https://issues.apache.org/jira/browse/SOLR-17328 Generates precise SBOM for the Solr binary distributions (`solr-<version>.tgz` and `solr-<version>-slim.tgz`) that include **all** the information available today to the build tool: 1. Maven artifacts included in the distribution by `:solr:packaging`, 2. Vendored JavaScript libraries in `solr/webapp/web/libs`, 3. NPM libraries bundled by `:solr:webapp:js-client`, inserted as *nested* assemblies of `solr-js-client`. 4. Maven artifacts compiled into `:solr:ui`, inserted as *nested* assemblies of `solr-ui`. The components for 1 and 4 are generated by the CycloneDX Gradle Plugin, while components from 3 are generated by `@cyclonedx/cyclonedx-npm`. Vendored libraries, of course, are handled as a static list that maintainers need to keep up to date. This is the second attempt after #3929. ## Known unknowns This PR does **not** attempt to manually detect the components **embedded** in JARs through shading or binary library inclusion. An incomplete list of artifacts with such hidden dependencies is: ``` com.fasterxml.woodstox:woodstox-core:7.2.1 com.ibm.icu:icu4j:78.3 io.netty:netty-common:4.2.15.Final io.netty:netty-tcnative-boringssl-static:2.0.79.Final io.netty:netty-transport-native-epoll:4.2.15.Final io.opentelemetry:opentelemetry-sdk-trace:1.56.0 org.apache.curator:curator-client:5.9.0 ``` (The Hadoop artifacts and `org.tallison.xmp:xmpcore-shaded` from earlier iterations of this list no longer ship in the distribution.) ## SBOM location Each archive contains its SBOM as `bom.json` in the root of the distribution, next to `LICENSE.txt` and `NOTICE.txt` (see [this comment on the previous PR](https://github.com/apache/solr/pull/3929#issuecomment-5150686346) for the full discussion): * The SBOM travels with the artifact, so it cannot become detached or lost. This is loosely inspired by [PEP 770](https://peps.python.org/pep-0770/), which standardizes the same idea for Python wheels. * `bom.json` is one of the [recognized CycloneDX filename patterns](https://cyclonedx.org/specification/overview/#recognized-file-patterns), so tools can discover it without configuration. * This is a **Build SBOM** in [CISA's classification](https://www.cisa.gov/resources-tools/resources/types-software-bill-materials-sbom): it records the dependency information available to Gradle at build time. An Analyzed SBOM produced from the finished archive by tools such as `syft` or `cdxgen` remains a useful, complementary verification step. There are other publication models: a `solr-<version>.tgz.cdx.json` sidecar next to the `.asc` and `.sha512` files on `downloads.apache.org`, or embedding the SBOM in an in-toto attestation bundle together with SLSA provenance. ## SBOM postprocessing The raw output of the CycloneDX Gradle Plugin and `@cyclonedx/cyclonedx-npm` is postprocessed this way: 1. **Metadata**: the `build` lifecycle phase is recorded, and the post-processing and cyclonedx-npm are listed in the tools next to the plugin. 2. **Main component**: the binary release is identified by a draft `sid` purl ([purl-spec issue #516](https://github.com/package-url/purl-spec/issues/516)), `pkg:sid/apache.org/solr/solr@<version>?edition=full|slim`, and by the Solr CPE used by the NVD. 3. **Removed components**: Maven BOM/platform dependencies and the internal `:platform` project are stripped, they are not part of the distribution. 4. **Solr components**: valid Maven purls (the CycloneDX Gradle Plugin generates incorrect Maven coordinates), description and `Apache-2.0` license. 5. **Vendored JavaScript libraries**: the third-party files checked into `solr/webapp/web/libs` are added. 6. **JavaScript client bundle**: the npm packages bundled by browserify into `server/solr-webapp/webapp/libs/solr/index.js` are nested as subassemblies of a first-party `solr-js-client` component. 7. **New UI bundle**: the Maven artifacts compiled into `server/solr-webapp/webapp/ui` are nested as subassemblies of a first-party `solr-ui` component, from a child SBOM generated in `:solr:ui` over `wasmJsRuntimeClasspath`, together with the `@js-joda/core` npm package bundled by the Kotlin toolchain. 8. **Archive locations**: the assembled distribution directories are scanned, and files are matched to components by SHA-256 hash, recorded as `evidence.occurrences`. This also proves that each vendored JavaScript library ships unmodified. 9. **Hashes**: only the SHA-256 hash of each component is kept; the plugin emits eight algorithms per artifact, which only adds bulk. The npm integrity hashes on `externalReferences` (SHA-512 of the registry tarballs) are preserved. This change was developed with significant use of an AI coding assistant (Claude), with human review of every step; commits carry an `Assisted-By` trailer. ## Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [x] I have developed this patch against the `main` branch. - [x] I have run `./gradlew precommit`; the full `./gradlew check` test suite was not run for this build-only change. - [x] I have verified the change by generating both SBOMs and validating them (graph integrity at every nesting level, occurrence coverage, SHA-256 hashes, opt-out builds, up-to-date re-runs); there is no unit-testable production code. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) (the shipped `bom.json` could be documented in a follow-up) - [x] I have added a [changelog entry](https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc) for my change -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
