Hello Axel, Bastien, Lucas, and other members of the lintian and QA teams,
it's been a long time that I have been interested in building some
infrastructure to manage scheduling and distribution of Debian-related
build, QA and data collection tasks to a network of worker machines. I
called this project debusine:
https://salsa.debian.org/freexian-team/debusine/
It has been started a while ago but thanks to external funding, its
development pace is about to increase significantly. It is being developed
by Freexian with the intention of giving people access to a range of
pre-configured tools and workflows running on remote hardware.
We want to make it easy for Debian contributors to leverage all the great
QA tools that Debian provides. We want to improve and modernize Debian's
build infrastructure, while also enabling distribution-wide experiments,
custom package repositories and custom workflows with advanced package
reviews.
Analyzing lintian results is an important step in any serious package
review workflow and as such, our current milestone plans to make it
possible to run lintian on debusine workers. As lintian and QA experts, we
would love to have your feedback on our plans, in particular on the public
interface that we have designed.
To give a bit more context, as a debusine user with an API key, you can
submit "work requests". Each work request has a "task" assigned to it
(e.g. sbuild, autopkgtest, lintian) and some JSON data that gives more
details about the specific task. The structure of the JSON data varies
from task to task but it basically defines the public API that lets users
schedule work requests to debusine. The result of the work request
(including any artifact generated) will then be stored in the database and
made available for consumption by the user through the API.
As an example, here's how the data could look like for a simple lintian
work request:
{
"input": {
"source_artifact_id": 1234, # References the source package to analyze
"binary_artifact_id": 1235, # References the set of binary packages to
analyze
},
"target_distribution": "debian:unstable",
"lintian_version": "2.116.3", # Or minimal_lintian_version ?
"fail_on_severity": "error",
}
We are currently drafting the expected structure of that JSON data for
lintian (and autopkgtest) work requests in this merge request and we would
appreciate if you could review it:
https://salsa.debian.org/freexian-team/debusine/-/merge_requests/300
I have left some open questions in the document and there are some
unresolved review threads that have interesting questions too.
We are aware that this duplicates work made by Lucas in the context of
UDD, but we are trying to achieve a high level of integration and sharing
worker setups across different QA tools, and duplication seems unavoidable
in that context. We are certainly eager to cooperate and make it easy for
UDD and debusine to work hand-in-hand. A future milestone will make it
easier to export distribution-wide summary of data collected through such
work requests.
We plan to provide a debusine.debian.net/org instance accessible to all
Debian developers in the near future, so you will be able to experiment
and make use of the API that you will have helped to shape.
If you have questions about debusine, don't hesitate to ask. If you are
interested to follow along and/or help, you are more than welcome to:
https://freexian-team.pages.debian.net/debusine/devel/contributing.html
Have a nice day,
Raphaël.
PS: You might find it useful to browse the documentation to learn more about the
goals and the high level concepts:
https://freexian-team.pages.debian.net/debusine/devel/why.html
https://freexian-team.pages.debian.net/debusine/design/index.html
--
⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hert...@debian.org>
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/
⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS
