| Issue |
204093
|
| Summary |
[llvm-cov][coverage] Add a public coverage-mapping bundle/SPI workflow for large multi-binary source coverage
|
| Labels |
|
| Assignees |
|
| Reporter |
ankurrj7
|
## Problem
LLVM source-based coverage works well when the reporting side can name the
instrumented executable or shared library:
```console
llvm-cov show ./my_binary --instr-profile=""
```
For large product builds this does not scale cleanly. A single test run can
exercise hundreds of executables and shared libraries, and the test/reporting
machine may not have the full build tree or even know the exact set of loaded
images. The profile data is already collected into `.profraw` / `.profdata`,
but the coverage mapping still has to be read from every final binary/DSO.
The practical request is an LLVM-supported way to carry the coverage mapping as
a small report-time artifact, similar in usability to an ICC-style `.spi` file:
```console
llvm-cov show coverage.spi --instr-profile=""
```
or:
```console
llvm-cov show --coverage-bundle=coverage.bundle
```
This is not a request to copy ICC's format exactly. The useful part is the
workflow: one compact mapping bundle plus the merged profile, instead of a long
list of original executables and DSOs.
## Current behavior
Today, the normal production workflow requires passing the original coverage
mapping containers to `llvm-cov`:
```console
llvm-cov show ./main --object=./liba.so --object=./libb.so \
--instr-profile=""
```
This is awkward for large builds because the reporting side must either:
1. preserve every instrumented executable and shared library,
2. know which of those images are relevant to the report, and
3. pass them all to `llvm-cov`.
LLVM does have pieces that show this is feasible:
* `llvm-cov convert-for-testing` can extract coverage mapping from an executable
into a small mapping file.
* `llvm-cov show <mapping-file> --instr-profile="" can report from that
extracted mapping file.
* For an executable plus DSO, separate extracted mapping files can be passed as
object inputs and produce the expected combined report.
* Build IDs plus `--debug-file-directory` can help find objects for modules
that appear in a profile.
However, `convert-for-testing` is not a public production interface, and there
is no first-class way to create or consume a coverage mapping bundle for a whole
product build.
## Why existing workarounds are incomplete
Build-ID lookup is useful, but it mostly solves the "which executed modules are
referenced by this profile?" problem. It does not fully solve build-wide
coverage reporting, where an instrumented binary or DSO that was never loaded
should still be present in the report as zero-covered.
For build-wide coverage, the report needs a manifest or bundle that represents
the final instrumented product images, not only the images that happened to
write profile data.
Manually enumerating hundreds of `--object` arguments works in principle, but it
is brittle in large test farms and across build/test/report machine boundaries.
## Requested feature
Please consider adding a public coverage mapping bundle workflow to `llvm-cov`.
One possible shape:
```console
llvm-cov extract-mapping ./main -o main.covmap
llvm-cov extract-mapping ./liba.so -o liba.covmap
llvm-cov create-bundle \
--objects-from final-instrumented-images.txt \
--output coverage.spi
llvm-cov show coverage.spi --instr-profile=""
llvm-cov export coverage.spi --instr-profile=""
llvm-cov report coverage.spi --instr-profile=""
```
An equivalent design using a bundle directory or manifest would also work, for
example:
```text
coverage.bundle/
manifest.json
profiles/merged.profdata
objects/
.build-id/...
mappings/
main.covmap
liba.covmap
```
The important properties are:
1. a stable, documented producer for coverage mapping sidecars or bundles,
2. report commands that can consume the bundle directly,
3. support for many final executables and DSOs,
4. diagnostics for stale or missing mappings, preferably using build IDs when
available, and
5. a way to represent build-wide scope, including images that did not execute.
## Possible implementation direction
LLVM already has most of the underlying machinery:
* coverage mapping sections are read from objects today,
* `convert-for-testing` demonstrates extracting mapping into a compact file,
* `llvm-cov` can already consume extracted mapping files as object inputs, and
* binary IDs can be used to validate that profile data and mapping inputs match.
The missing part appears to be a supported packaging/reporting interface rather
than a fundamentally new coverage mapping model.
One conservative design would keep the current mapping format and add tooling
around final linked images:
1. scan or consume a manifest of final instrumented executables/DSOs,
2. extract their coverage mapping into compact per-image files,
3. write an aggregate manifest or bundle,
4. let `llvm-cov show/export/report` accept that bundle as the coverage mapping
input.
This avoids relying on compile-time sidecar emission, which can accidentally
include objects that were compiled but never linked into the final product.
## Local evidence
A local probe confirmed the basic model:
```console
llvm-cov convert-for-testing spi_probe -o coverage.testing
llvm-cov show coverage.testing --instr-profile=""
```
produced the same source report as:
```console
llvm-cov show spi_probe --instr-profile=""
```
For the small probe, the extracted mapping file was much smaller than the
binary:
```text
spi_probe executable: 101944 bytes
coverage.testing: 231 bytes
default.profdata: 672 bytes
```
A second probe using an executable plus an instrumented shared library also
worked after extracting both mappings and passing them to `llvm-cov` as object
inputs. This suggests a public bundle format would mainly improve packaging,
scale, and ergonomics rather than require a new coverage data model.
## Open questions
* Should the public artifact be a single aggregate file, a directory bundle, or
a manifest that references per-image mapping files?
* Should bundle creation be a pure post-link `llvm-cov` operation, or should
Clang also support optional compile-time mapping sidecar emission?
* How should `llvm-cov` distinguish executed-only reporting from build-wide
reporting where never-loaded images should appear as zero-covered?
* Should build IDs be required in the bundle, or only used when available for
diagnostics?
![]()
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs
