Hi all, I'd like to propose changing how Dependabot updates our Maven dependencies: run weekly instead of daily, group minor and patch updates into one PR, and keep major updates as separate PRs. I'd also like to discuss how we handle licenses.yaml for these PRs, since that is the most common reason they fail and need some manual intervention.
This should hopefully help with both the CI load (and mental PR load), and also streamline the dependency updates more. My apologies if this has been discussed before. I also know that Frank's Codex automation helped tremendously with getting through many of the trivial and not-so-trivial dependency updates, which is great. Why Apart from the obvious answer of "it's a lot!", I had Claude pull some stats (2026-07-22 to 2026-10-22): - 381 of the 696 commits on master were Dependabot PRs. - Dependabot triggered 422 of the 2,389 PR runs of the "Unit & Integration tests CI" workflow (about 18%), plus a run of Static Checks and CodeQL each time. - One CI pass for a Maven bump PR costs about 450 runner-minutes (median of a dozen PRs from September and October), or roughly 14,600 runner-minutes a week. That is well within the ASF caps [1], but the runner queue is shared across the foundation, and each PR also needs a committer to review, often fix, and merge it (so additional commits and additional runs). - A single upstream release often arrives as several PRs. gRPC 1.84.1 opened four today (#20676 grpc-api, #20679 grpc-bom, #20681 grpc-context, #20677 shade.grpc.version), because those artifacts are pinned in four places. Also, over a third of Dependabot's "Static Checks" runs fail (58 of 159 since 2026-09-08, against 6% on master): because licenses.yaml doesn't match, because maven-enforcer throws RequireUpperBoundDeps when an artifact moves without its siblings, or because of compile errors (kinda normal, at least for major bumps). Separately, about 30% of their unit-test runs fail, the same rate as on master, so that's general flaky tests and another problem. Proposal At various projects I work on, we update dependencies weekly, so I was thinking, why don't we do that for apache/druid as well? Idea: 1. Schedule: weekly (Monday) instead of daily. The existing 14-day cooldown stays. (Neither applies to security updates: Dependabot opens those when GitHub raises an alert for a vulnerable dependency, and bumps to the lowest fixed version [2]. Security updates are already enabled for our repo.) 2. One group for all minor and patch updates, to keep the configuration simple. Not every library follows semantic versioning, so the reviewer of the weekly PR can drop an update that looks risky or breaking (maybe best by adding an ignore rule to the yml file so we can easily track it) and handle it in its own PR. This works well in other codebases I work on. Major updates stay ungrouped, one PR each, since they are the ones most likely to need code changes. The existing maven-resolver group and all ignore rules stay as they are. 3. One group per ecosystem for security updates (Maven, website, web-console, GitHub Actions), so a burst of advisories opens one PR instead of one per advisory. Again, I asked Claude to replay this proposal against the last 13 weeks of merged bumps: we get about 68 PRs instead of 381, 43 of them majors from the August catch-up. The last seven weeks go from 117 PRs to 17 (rough estimate). ASF Infra's Dependabot guidance [3] suggests weekly updates, grouping, and grouped security updates. licenses.yaml licenses.yaml records an exact version for every bundled dependency, and check-licenses matches on group, artifact and version. Every bump of a shipped dependency therefore fails Static Checks until someone edits licenses.yaml by hand: 166 of the 381 Dependabot PRs merged in the last 13 weeks needed that edit. Grouping cuts it to one edit per week, but here's another idea. Options, in the order I'd prefer them: a) Check versions at release time, not on every PR. In PR CI, check-licenses would keep failing a PR that adds a dependency or changes a license, but only warn when a version differs. The release build keeps the strict check, so a release still cannot ship a stale LICENSE. Before cutting a release branch, the release manager runs a new script that refreshes the versions in licenses.yaml from the Maven license reports, and commits the result to master. Dependabot PRs then pass as they are. Pinot runs Dependabot the same way and refreshes its LICENSE-binary with a script at release time [4]. b) Keep the strict check, and have a workflow push the licenses.yaml fix to the Dependabot branch. This is the most automated option, but I'm not sure how easy it is to do. I think we'd have to mint a GH App token instead of the GITHUB_TOKEN so CI would run, and store the app's key as a Dependabot secret, since workflows triggered by Dependabot only see those. We do that in private repos, but I'm not sure whether that works on Apache Infra. c) Keep the strict check, and have a workflow post the needed licenses.yaml change on each Dependabot PR as a comment. But that would still need manual work on each PR, and we might as well not do the additional workflow stuff and just run the script locally on that branch quickly. Thoughts? Cheers, Andy [1] https://infra.apache.org/github-actions-policy.html [2] https://docs.github.com/en/code-security/concepts/supply-chain-security/dependabot-security-updates [3] https://infra.apache.org/dependabot.html [4] https://github.com/apache/pinot/pull/17818
