On Wed, 4 Sep 2024 16:08:28 +0100
Anatoly Burakov <[email protected]> wrote:
> As part of the meson build, we can record the dependencies for each
> component as we process it, logging them to a file. This file can be
> used as input to a number of other scripts and tools, for example, to
> graph the dependencies, or to allow higher-level build-config tools to
> automatically enable component requirements, etc.
>
> The first patch of this set separates dependencies inside meson into
> optional or mandatory. The second patch of this set generates the basic
> dependency tree. The third patch does some processing of that dependency
> tree to identify cases where dependencies are being unnecessarily
> specified. Reducing these makes it easier to have readable dependency
> graphs in future, without affecting the build.
>
> The following 4 patches are based on the output of the second patch, and
> greatly cut down the number of direct dependency links between
> components. Even with the cut-down dependencies, the full dependency
> graph is nigh-unreadable, so the final patch adds a new script to
> generate dependency tree subgraphs, creating dot files for e.g. the
> dependencies of a particular component, or a component class such as
> mempool drivers.
>
> v2 -> v3:
> - Split dependencies into optional and mandatory
> - Fixup graph scripts to read and generate graphs that encode optional
> dependencies into the graph
> - Python version fixes to avoid using features not available in minimum
> supported Python version
> - Formatting with Ruff, and PEP-484 compliance
>
> Anatoly Burakov (1):
> build: split dependencies into mandatory and optional
>
> Bruce Richardson (7):
> build: output a dependency log in build directory
> devtools: add script to flag unneeded dependencies
> build: remove kvargs from driver class dependencies
> build: reduce library dependencies
> build: reduce driver dependencies
> build: reduce app dependencies
> devtools: add script to generate DPDK dependency graphs
>
> app/dumpcap/meson.build | 2 +-
> app/graph/meson.build | 2 +-
> app/meson.build | 11 +-
> app/pdump/meson.build | 2 +-
> app/proc-info/meson.build | 4 +-
> app/test-bbdev/meson.build | 8 +-
> app/test-crypto-perf/meson.build | 4 +-
> app/test-fib/meson.build | 2 +-
> app/test-pmd/meson.build | 26 +--
> app/test-sad/meson.build | 2 +-
> app/test/meson.build | 14 +-
> buildtools/log-deps.py | 81 ++++++++
> buildtools/meson.build | 2 +
> devtools/draw-dependency-graphs.py | 223 +++++++++++++++++++++
> devtools/find-duplicate-deps.py | 53 +++++
> drivers/baseband/fpga_5gnr_fec/meson.build | 2 +-
> drivers/baseband/fpga_lte_fec/meson.build | 2 +-
> drivers/baseband/la12xx/meson.build | 2 +-
> drivers/baseband/null/meson.build | 2 +-
> drivers/baseband/turbo_sw/meson.build | 2 +-
> drivers/bus/auxiliary/meson.build | 2 -
> drivers/bus/dpaa/meson.build | 2 +-
> drivers/bus/fslmc/meson.build | 2 +-
> drivers/bus/ifpga/meson.build | 2 +-
> drivers/bus/pci/meson.build | 4 +-
> drivers/bus/platform/meson.build | 1 -
> drivers/bus/uacce/meson.build | 2 -
> drivers/bus/vdev/meson.build | 2 -
> drivers/common/cnxk/meson.build | 4 +-
> drivers/common/cpt/meson.build | 2 +-
> drivers/common/idpf/meson.build | 2 +-
> drivers/common/mlx5/meson.build | 2 +-
> drivers/compress/mlx5/meson.build | 2 +-
> drivers/compress/nitrox/meson.build | 2 +-
> drivers/compress/octeontx/meson.build | 2 +-
> drivers/crypto/bcmfs/meson.build | 2 +-
> drivers/crypto/cnxk/meson.build | 2 +-
> drivers/crypto/dpaa_sec/meson.build | 2 +-
> drivers/crypto/ipsec_mb/meson.build | 2 +-
> drivers/crypto/mlx5/meson.build | 2 +-
> drivers/crypto/nitrox/meson.build | 2 +-
> drivers/dma/cnxk/meson.build | 2 +-
> drivers/dma/dpaa/meson.build | 2 +-
> drivers/dma/dpaa2/meson.build | 2 +-
> drivers/dma/odm/meson.build | 2 +-
> drivers/dma/skeleton/meson.build | 2 +-
> drivers/event/cnxk/meson.build | 2 +-
> drivers/event/dlb2/meson.build | 2 +-
> drivers/event/dpaa2/meson.build | 2 +-
> drivers/event/meson.build | 2 +-
> drivers/event/octeontx/meson.build | 3 +-
> drivers/event/sw/meson.build | 2 +-
> drivers/mempool/cnxk/meson.build | 2 +-
> drivers/mempool/dpaa/meson.build | 2 +-
> drivers/mempool/dpaa2/meson.build | 2 +-
> drivers/mempool/octeontx/meson.build | 2 +-
> drivers/meson.build | 9 +-
> drivers/net/cnxk/meson.build | 3 +-
> drivers/net/iavf/meson.build | 2 +-
> drivers/net/ice/meson.build | 2 +-
> drivers/net/mana/meson.build | 2 +-
> drivers/net/meson.build | 2 +-
> drivers/net/mlx5/meson.build | 2 +-
> drivers/net/sfc/meson.build | 2 +-
> drivers/net/softnic/meson.build | 2 +-
> drivers/raw/cnxk_bphy/meson.build | 2 +-
> drivers/raw/cnxk_gpio/meson.build | 2 +-
> drivers/raw/ntb/meson.build | 2 +-
> drivers/raw/skeleton/meson.build | 2 +-
> drivers/regex/meson.build | 2 +-
> drivers/regex/mlx5/meson.build | 2 +-
> drivers/vdpa/ifc/meson.build | 2 +-
> drivers/vdpa/meson.build | 3 +-
> drivers/vdpa/mlx5/meson.build | 2 +-
> drivers/vdpa/sfc/meson.build | 2 +-
> examples/ethtool/meson.build | 2 +-
> examples/l2fwd-crypto/meson.build | 2 +-
> examples/l3fwd/meson.build | 2 +-
> examples/meson.build | 11 +-
> examples/vm_power_manager/meson.build | 6 +-
> lib/argparse/meson.build | 2 +-
> lib/bbdev/meson.build | 2 +-
> lib/bitratestats/meson.build | 2 +-
> lib/bpf/meson.build | 2 +-
> lib/cmdline/meson.build | 2 +-
> lib/compressdev/meson.build | 2 +-
> lib/cryptodev/meson.build | 2 +-
> lib/dispatcher/meson.build | 2 +-
> lib/distributor/meson.build | 2 +-
> lib/dmadev/meson.build | 2 -
> lib/eal/meson.build | 5 +-
> lib/efd/meson.build | 2 +-
> lib/ethdev/meson.build | 2 +-
> lib/eventdev/meson.build | 3 +-
> lib/fib/meson.build | 2 +-
> lib/gpudev/meson.build | 2 +-
> lib/graph/meson.build | 2 +-
> lib/gro/meson.build | 2 +-
> lib/gso/meson.build | 2 +-
> lib/hash/meson.build | 4 +-
> lib/ip_frag/meson.build | 2 +-
> lib/ipsec/meson.build | 2 +-
> lib/kvargs/meson.build | 2 +-
> lib/latencystats/meson.build | 2 +-
> lib/lpm/meson.build | 3 +-
> lib/mbuf/meson.build | 2 +-
> lib/member/meson.build | 2 +-
> lib/mempool/meson.build | 2 +-
> lib/meson.build | 8 +-
> lib/metrics/meson.build | 2 +-
> lib/mldev/meson.build | 2 +-
> lib/net/meson.build | 2 +-
> lib/node/meson.build | 2 +-
> lib/pcapng/meson.build | 2 +-
> lib/pdcp/meson.build | 2 +-
> lib/pdump/meson.build | 2 +-
> lib/pipeline/meson.build | 2 +-
> lib/port/meson.build | 2 +-
> lib/power/meson.build | 2 +-
> lib/rawdev/meson.build | 2 -
> lib/rcu/meson.build | 2 +-
> lib/regexdev/meson.build | 2 +-
> lib/reorder/meson.build | 2 +-
> lib/rib/meson.build | 2 +-
> lib/ring/meson.build | 1 -
> lib/sched/meson.build | 2 +-
> lib/security/meson.build | 2 +-
> lib/table/meson.build | 2 +-
> lib/telemetry/meson.build | 2 +-
> lib/vhost/meson.build | 2 +-
> 130 files changed, 533 insertions(+), 171 deletions(-)
> create mode 100644 buildtools/log-deps.py
> create mode 100755 devtools/draw-dependency-graphs.py
> create mode 100755 devtools/find-duplicate-deps.py
This patchset needs to be rebased. Please resubmit.
AI code review only found one minor thing.
## DPDK Patchset Review: build: introduce optional internal dependencies (v7)
**Series:** [PATCH v7 1-8/8] by Anatoly Burakov / Bruce Richardson
**Patchwork IDs:** 151640-151647
---
### Summary
This 8-patch series introduces a framework for optional internal dependencies
in DPDK's Meson build system and reduces unnecessary dependencies across
libraries, drivers, and applications. It also adds tooling to log, analyze, and
visualize the dependency graph.
---
### Patch-by-Patch Review
#### Patch 1/8: `build: introduce optional internal dependencies`
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 42 characters |
| Lowercase after colon | ✅ PASS | |
| Imperative mood | ✅ PASS | |
| No trailing period | ✅ PASS | |
| Signed-off-by present | ✅ PASS | Two signatures (Bruce Richardson, Anatoly
Burakov) |
| Body ≤75 chars | ✅ PASS | |
| Body not starting with "It" | ✅ PASS | |
**⚠️ Warning - Typo in commit message body (line 79):**
```
said dependency is not being built. At build
time, the build system will resolve any optional dependencies and add
them to the list of dependencies to be built. Any source files requiring
said optional depepdencies will still have to be added explicitly...
^^^^^^^^^^^^
```
Should be: `dependencies`
**Meson Style:** ✅ Follows 4-space indentation, proper list formatting.
---
#### Patch 2/8: `build: output a dependency log in build directory`
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 45 characters |
| Signed-off-by present | ✅ PASS | Bruce Richardson + Acked-by + Anatoly |
| From: line | ✅ PASS | Correct attribution to Bruce Richardson |
| SPDX in new file | ✅ PASS | `buildtools/log-deps.py` line 2 |
| Copyright follows SPDX | ✅ PASS | |
| Shebang correct | ✅ PASS | `#! /usr/bin/env python3` |
**Code Quality:** `buildtools/log-deps.py` is well-structured with proper
docstrings.
---
#### Patch 3/8: `devtools: add script to find duplicate dependencies`
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 46 characters |
| Signed-off-by present | ✅ PASS | Bruce Richardson |
| SPDX in new file | ✅ PASS | `devtools/find-duplicate-deps.py` |
| Shebang correct | ✅ PASS | |
**ℹ️ Info - Minor style suggestion:**
The class name `dep` at line 934 uses lowercase, which is unconventional for
Python classes. Consider `Dep` for PEP8 compliance (though this is not a DPDK
requirement).
---
#### Patch 4/8: `drivers: remove kvargs from class dependencies`
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 44 characters |
| Signed-off-by present | ✅ PASS | Bruce Richardson |
| Correct component prefix | ✅ PASS | `drivers:` |
**Changes are minimal and correct** - removes redundant `kvargs` dependency
from 4 driver class meson files since `kvargs` is already pulled in via EAL.
---
#### Patch 5/8: `lib: reduce library dependencies`
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 26 characters |
| Signed-off-by present | ✅ PASS | Bruce Richardson |
| Correct component prefix | ✅ PASS | `lib:` |
**Code Style:** All changes correctly convert `deps +=` to `deps =` pattern for
direct assignment, leveraging Meson's recursive dependency support.
---
#### Patch 6/8: `drivers: reduce driver dependencies`
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 30 characters |
| Signed-off-by present | ✅ PASS | Anatoly Burakov |
| Correct component prefix | ✅ PASS | `drivers:` |
**Large patch (83 files changed)** but changes are systematic - removing
explicit dependencies that are transitively satisfied.
---
#### Patch 7/8: `app: reduce app dependencies`
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 26 characters |
| Signed-off-by present | ✅ PASS | Bruce Richardson |
| Notes present | ✅ PASS | v6→v7 changelog |
| Correct component prefix | ✅ PASS | `app:` |
---
#### Patch 8/8: `devtools: add script to generate DPDK dependency graphs`
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 47 characters |
| Signed-off-by present | ✅ PASS | Bruce Richardson, Anatoly Burakov |
| SPDX in new file | ✅ PASS | |
| Copyright follows SPDX | ✅ PASS | |
| Shebang correct | ✅ PASS | |
**Code Quality:** Well-structured Python with type hints, docstrings, and
proper error handling.
---
### Overall Assessment
| Category | Count |
|----------|-------|
| **Errors** | 0 |
| **Warnings** | 1 |
| **Info** | 1 |
#### ⚠️ Warnings (should fix):
1. **Patch 1/8:** Typo in commit message body - "depepdencies" should be
"dependencies"
#### ℹ️ Info (consider):
1. **Patch 3/8:** Class `dep` could be `Dep` for Python convention (minor)
---
### Recommendation
**PASS with minor fix required**
The patchset is well-structured and follows DPDK coding guidelines. The only
actionable issue is the typo in patch 1's commit message body. Once that's
corrected, this series should be ready for merge.
The changes are functionally sound - they leverage Meson's recursive dependency
resolution to eliminate redundant explicit dependencies, which will improve
build configuration time and simplify the dependency graph. The new tooling
(`log-deps.py`, `find-duplicate-deps.py`, `draw-dependency-graphs.py`) provides
useful visibility into the dependency structure.
