Nicolas Boulenguez:
Package: dh-debputy Version: 0.1.78 Severity: wishlist Tags: patchHello. The attachment lets debputy generate the (Static-)Built-Using fields from manifest rules, replacing the dh-builtusing debhelper plugin. It seems ready for a first review, if you are interested.
Hi Nicolas, Thanks for proposing this patch and thanks for using `debputy`.I am interested, but I am also finding myself without a lot of Debian time these days. You have to prod a bit for updates on my end.
'bug1120283.py' should only exist until the patch with the same contents is applied to the python3-debian package.
Noted. Personally, I think `python3-debian` could do itself a favor and turn `PkgRelation.ParsedRelation` into a proper entity rather than a typed dict, so those features can become instance methods rather than functions.
But either way, I appreciate `debputy` will have to carry some kind of shim/backwards compat code unless `python3-debian` is backported and I am happy that this part of it has been thought into the patch. I have some overdue merging to do myself on that front.
I have made the source_package and dpkg_arch_query_table HighLevelManifest attributes public. Their value is accessible via various tricks like manifest.condition_context(pkg).source_package anyway.
Ok. I think this is fine. The `HighLevelManifest` is a bit of a god-object, but that is my mess to sort out later. I cherry-picked the `source_package` attribute in my attached patch.
'test_built_using.py' demonstrates various possible use cases. Advices about the way to merge it into the test suite would be welcome.
My dream for this was to have this kind of feature be a "metadata detector". The key "problem" was that currently "metadata detectors" do not have access to configuration from the manifest. The main thing keeping me back is that I want to keep the two very loosely coupled.
I have tried to make a solution for this in attached prototype (also as the branch accessible-manifest-configuration if you are ok with git/salsa; note, might be rebased). With your patch then being rebased on top of that patch, then it could be re-implemented something like this:
```
class BuiltUsingBase:
...
class StaticBuiltUsing(StaticBuiltUsingBase)
pass
class BuiltUsing(StaticBuiltUsingBase)
pass
def _unpack_built_using(
_name: str,
parsed_data: list[BuiltUsingParsedFormat],
_attribute_path: AttributePath,
_parser_context: ParserContextData,
) -> BuiltUsing:
return StaticBuiltUsing(...)
# This part is still a bit more WET than I want it to be
def _unpack_static_built_using(
_name: str,
parsed_data: list[BuiltUsingParsedFormat],
_attribute_path: AttributePath,
_parser_context: ParserContextData,
) -> BuiltUsing:
return StaticBuiltUsing(...)
# This would be split into different files most likely
# - one part for binary_package_rules.py
# - the other part for metadata_detectors.py
def initialize(api: DebputyPluginInitializer) -> None:
api.pluggable_manifest_rule(
rule_type=OPARSER_PACKAGES,
rule_name="built-using",
parsed_format=list[BuiltUsingParsedFormat],
_unpack_built_using,
)
api.pluggable_manifest_rule(
rule_type=OPARSER_PACKAGES,
rule_name="static-built-using",
parsed_format=list[BuiltUsingParsedFormat],
_unpack_static_built_using,
)
api.metadata_or_maintscript_detector(
"detect-built-using",
detect_built_using,
)
def detect_built_using(
fs_root: "VirtualPath",
ctrl: "BinaryCtrlAccessor",
context: "PackageProcessingContext",
) -> None:
built_using_conf = context.manifestEntity(
context.binary_package,
BuiltUsing,
)
static_built_using_conf = context.manifestEntity(
context.binary_package,
StaticBuiltUsing,
)
# built_using_conf is an instance of BuiltUsing | None
# static_built_using_conf is an instance of StaticBuiltUsing | None
built_using = ...
static_built_using = ...
if built_using:
ctrl.substvars.add_dependency("debputy:Built-Using", ...)
if static_built_using:
ctrl.substvars.add_dependency("debputy:Static-Built-Using", ...)
```
If this is accomplished, the feature could be installed via the
register_package_metadata_detectors in
`src/debputy/plugins/debputy/debputy_plugin.py` with the implementation
going into in `.../metadata_detectors.py`. Related tests are in
`tests/test_debputy_plugin.py` (search for `run_metadata_detector`). You
can also find some examples in `tests/plugin_tests/*` (which are tests
for "non-core" plugins, so they load their plugin differently, but is
otherwise the same).
I might need to make some testing infrastructure to make it easier to create real manifest with arbitrary d/control contents.
The main downside with this approach is that your work on avoiding calls to `dpkg` will be difficult. I think you can at best get it to be once per BinaryPackage with this approach. The slowness of `dpkg` is one of the things I need to look at, but I think that can come after.
I hope this was interesting to you as well. Best regards, Niels
From 84515366607854034d6aa7b24944adc6d29fa887 Mon Sep 17 00:00:00 2001 From: Niels Thykier <[email protected]> Date: Wed, 26 Nov 2025 20:24:47 +0000 Subject: [PATCH] Plugin API: Make some pluggable manifest rules accessible via context With this change, plugins are able to register a pluggable manifest rule (PMR) and then access the value from it in a metadata-detector or a package-processor. The PMR must be registered directly on an OPARSER (accordingly, rule type must be of type `str`). The declared return type of the handler is as part of the key for resolving the value later. Every registered type must be unique as a consequence. Example being: ``` @dataclasses.dataclass class Foo: ... def initialize(api: DebputyPluginInitializer) -> None: api.pluggable_manifest_rule( OPARSER_PACKAGES, "my-foo", str, _parse_my_foo, ) api.metadata_or_maintscript_detector("foobinator", foobinate) def _parse_my_foo( ... ) -> Foo: return Foo(...) def foobinate( fs_root: VirtualPath, ctrl: BinaryCtrlAccessor, context: PackageProcessingContext, ) -> None: foo = context.manifest_configuration( context.binary_package, Foo, ) if foo is None: return ... # Foobinate with Foo ``` --- src/debputy/highlevel_manifest.py | 18 ++++- src/debputy/highlevel_manifest_parser.py | 48 +++++++++---- src/debputy/plugin/api/impl.py | 67 ++++++++++++++----- src/debputy/plugin/api/impl_types.py | 22 +++--- src/debputy/plugin/api/spec.py | 26 ++++++- src/debputy/plugin/api/test_api/test_impl.py | 35 ++++++---- src/debputy/plugin/plugin_state.py | 65 +++++++++++++++++- .../plugins/debputy/binary_package_rules.py | 6 ++ .../plugins/debputy/build_system_rules.py | 3 + .../plugins/debputy/manifest_root_rules.py | 4 ++ .../plugins/debputy/metadata_detectors.py | 2 +- 11 files changed, 238 insertions(+), 58 deletions(-) diff --git a/src/debputy/highlevel_manifest.py b/src/debputy/highlevel_manifest.py index c5255626..fc657a06 100644 --- a/src/debputy/highlevel_manifest.py +++ b/src/debputy/highlevel_manifest.py @@ -2,6 +2,7 @@ import dataclasses import functools import os import textwrap +import typing from contextlib import suppress from dataclasses import dataclass, field from typing import ( @@ -1197,6 +1198,10 @@ class HighLevelManifest: build_env: DebBuildOptionsAndProfiles, build_environments: BuildEnvironments, build_rules: list[BuildRule] | None, + value_table: Mapping[ + tuple[SourcePackage | BinaryPackage, type[Any]], + Any, + ], plugin_provided_feature_set: PluginProvidedFeatureSet, debian_dir: VirtualPath, ) -> None: @@ -1206,7 +1211,7 @@ class HighLevelManifest: remove_during_clean_rules ) self._install_rules = install_rules - self._source_package = source_package + self.source_package = source_package self._binary_packages = binary_packages self.substitution = substitution self.package_transformations = package_transformations @@ -1216,6 +1221,7 @@ class HighLevelManifest: self._used_for: set[str] = set() self.build_environments = build_environments self.build_rules = build_rules + self._value_table = value_table self._plugin_provided_feature_set = plugin_provided_feature_set self._debian_dir = debian_dir self._source_condition_context = ConditionContext( @@ -1270,6 +1276,14 @@ class HighLevelManifest: def all_packages(self) -> Iterable[BinaryPackage]: yield from self._binary_packages.values() + def manifest_configuration[T]( + self, + context_package: SourcePackage | BinaryPackage, + value_type: type[T], + ) -> T | None: + res = self._value_table.get((context_package, value_type)) + return typing.cast("T | None", res) + def package_state_for(self, package: str) -> PackageTransformationDefinition: return self.package_transformations[package] @@ -1574,7 +1588,7 @@ class HighLevelManifest: ) package_data_dict[package] = BinaryPackageData( - self._source_package, + self.source_package, dctrl_bin, build_system_pkg_staging_dir, fs_root, diff --git a/src/debputy/highlevel_manifest_parser.py b/src/debputy/highlevel_manifest_parser.py index 73280956..fcbb7ef8 100644 --- a/src/debputy/highlevel_manifest_parser.py +++ b/src/debputy/highlevel_manifest_parser.py @@ -1,16 +1,12 @@ import collections import contextlib +from collections.abc import Callable, Mapping, Iterator from typing import ( - Optional, - Dict, - List, Any, - Union, IO, cast, - Tuple, + TYPE_CHECKING, ) -from collections.abc import Callable, Mapping, Iterator from debian.debian_support import DpkgArchTable @@ -30,6 +26,8 @@ from debputy.path_matcher import ( ExactFileSystemPath, MatchRule, ) +from debputy.plugin.api.parser_tables import OPARSER_MANIFEST_ROOT +from debputy.plugins.debputy.build_system_rules import BuildRule from debputy.substitution import Substitution from debputy.util import ( _normalize_path, @@ -68,11 +66,15 @@ from .plugin.api.impl_types import ( DispatchingTableParser, PackageContextData, ) -from debputy.plugin.api.parser_tables import OPARSER_MANIFEST_ROOT from .plugin.api.spec import DebputyIntegrationMode -from debputy.plugins.debputy.build_system_rules import BuildRule +from .plugin.plugin_state import with_binary_pkg_parsing_context, begin_parsing_context from .yaml import YAMLError, MANIFEST_YAML + +if TYPE_CHECKING: + from .plugins.debputy.binary_package_rules import BinaryVersion + + try: from Levenshtein import distance except ImportError: @@ -158,6 +160,10 @@ class HighLevelManifestParser(ParserContextData): self._has_set_default_build_environment = False self._read_build_environment = False self._build_rules: list[BuildRule] | None = None + self._value_table: dict[ + tuple[SourcePackage | BinaryPackage, type[Any]], + Any, + ] = {} if isinstance(debian_dir, str): debian_dir = OSFSROOverlay.create_root_dir("debian", debian_dir) @@ -243,6 +249,13 @@ class HighLevelManifestParser(ParserContextData): if self._used: raise TypeError("build_manifest can only be called once!") self._used = True + return begin_parsing_context( + self._value_table, + self._source_package, + self._build_manifest, + ) + + def _build_manifest(self) -> HighLevelManifest: self._ensure_package_states_is_initialized() for var, attribute_path in self._declared_variables.items(): if not self.substitution.is_used(var): @@ -291,13 +304,15 @@ class HighLevelManifestParser(ParserContextData): self._deb_options_and_profiles, build_environments, self._build_rules, + self._value_table, self._plugin_provided_feature_set, self._debian_dir, ) @contextlib.contextmanager def binary_package_context( - self, package_name: str + self, + package_name: str, ) -> Iterator[PackageTransformationDefinition]: if package_name not in self._package_states: self._error( @@ -308,7 +323,8 @@ class HighLevelManifestParser(ParserContextData): package_state = self._package_states[package_name] self._package_state_stack.append(package_state) ps_len = len(self._package_state_stack) - yield package_state + with with_binary_pkg_parsing_context(package_state.binary_package): + yield package_state if ps_len != len(self._package_state_stack): raise RuntimeError("Internal error: Unbalanced stack manipulation detected") self._package_state_stack.pop() @@ -552,7 +568,7 @@ class YAMLManifestParser(HighLevelManifestParser): ) return v - def from_yaml_dict(self, yaml_data: object) -> "HighLevelManifest": + def _from_yaml_dict(self, yaml_data: object) -> "HighLevelManifest": attribute_path = AttributePath.root_path(yaml_data) parser_generator = self._plugin_provided_feature_set.manifest_parser_generator dispatchable_object_parsers = parser_generator.dispatchable_object_parsers @@ -587,7 +603,7 @@ class YAMLManifestParser(HighLevelManifestParser): f'Cannot define rules for package "{package_name}" (at {definition_source.path}). It is an' " auto-generated package." ) - binary_version = parsed.get(MK_BINARY_VERSION) + binary_version: str | None = parsed.get(MK_BINARY_VERSION) if binary_version is not None: package_state.binary_version = ( package_state.substitution.substitute( @@ -638,7 +654,13 @@ class YAMLManifestParser(HighLevelManifestParser): f"Could not parse {self.manifest_path} as a YAML document: {msg}" ) from e self._mutable_yaml_manifest = MutableYAMLManifest(data) - return self.from_yaml_dict(data) + + return begin_parsing_context( + self._value_table, + self._source_package, + self._from_yaml_dict, + data, + ) def parse_manifest( self, diff --git a/src/debputy/plugin/api/impl.py b/src/debputy/plugin/api/impl.py index f88a3d5a..aa407665 100644 --- a/src/debputy/plugin/api/impl.py +++ b/src/debputy/plugin/api/impl.py @@ -4,6 +4,7 @@ import functools import importlib import importlib.resources import importlib.util +import inspect import itertools import json import os @@ -11,22 +12,16 @@ import re import subprocess import sys from abc import ABC +from collections.abc import Callable, Iterable, Sequence, Iterator, Mapping, Container from importlib.resources.abc import Traversable -from io import IOBase, BytesIO +from io import IOBase from json import JSONDecodeError from pathlib import Path +from types import NoneType from typing import ( - Optional, - Dict, - Tuple, - Type, - List, - Union, - Set, IO, AbstractSet, cast, - FrozenSet, Any, Literal, TYPE_CHECKING, @@ -34,7 +29,6 @@ from typing import ( AnyStr, overload, ) -from collections.abc import Callable, Iterable, Sequence, Iterator, Mapping, Container import debputy from debputy import DEBPUTY_DOC_ROOT_DIR @@ -86,6 +80,7 @@ from debputy.plugin.api.impl_types import ( PluginProvidedTypeMapping, PluginProvidedBuildSystemAutoDetection, BSR, + TP, ) from debputy.plugin.api.plugin_parser import ( PLUGIN_METADATA_PARSER, @@ -126,15 +121,16 @@ from debputy.plugin.api.spec import ( DebputyPluginDefinition, ) from debputy.plugin.api.std_docs import _STD_ATTR_DOCS -from debputy.plugins.debputy.to_be_api_types import ( - BuildRuleParsedFormat, - BSPF, - debputy_build_system, -) from debputy.plugin.plugin_state import ( run_in_context_of_plugin, run_in_context_of_plugin_wrap_errors, wrap_plugin_code, + register_manifest_type_value_in_context, +) +from debputy.plugins.debputy.to_be_api_types import ( + BuildRuleParsedFormat, + BSPF, + debputy_build_system, ) from debputy.substitution import ( Substitution, @@ -218,6 +214,7 @@ class DebputyPluginInitializerProvider(DebputyPluginInitializer): "_unloaders", "_is_doc_cache_resolved", "_doc_cache", + "_registered_manifest_types", "_load_started", ) @@ -234,6 +231,7 @@ class DebputyPluginInitializerProvider(DebputyPluginInitializer): self._unloaders: list[Callable[[], None]] = [] self._is_doc_cache_resolved: bool = False self._doc_cache: DebputyParsedDoc | None = None + self._registered_manifest_types: dict[type[Any], DebputyPluginMetadata] = {} self._load_started = False @property @@ -961,6 +959,7 @@ class DebputyPluginInitializerProvider(DebputyPluginInitializer): Container[DebputyIntegrationMode] ) = None, apply_standard_attribute_documentation: bool = False, + register_value: bool = True, ) -> None: # When changing this, consider which types will be unrestricted self._restricted_api() @@ -978,7 +977,18 @@ class DebputyPluginInitializerProvider(DebputyPluginInitializer): f"The rule_type was not a supported type. It must be one of {types}" ) dispatching_parser = parser_generator.dispatchable_object_parsers[rule_type] + signature = inspect.signature(handler) + if ( + signature.return_annotation is signature.empty + or signature.return_annotation == NoneType + ): + raise ValueError( + "The handler must have a return type (that is not None)" + ) + register_as_type = signature.return_annotation else: + # Dispatchable types cannot be resolved + register_as_type = None if rule_type not in parser_generator.dispatchable_table_parsers: types = ", ".join( sorted( @@ -990,6 +1000,19 @@ class DebputyPluginInitializerProvider(DebputyPluginInitializer): ) dispatching_parser = parser_generator.dispatchable_table_parsers[rule_type] + if register_as_type is not None and not register_value: + register_as_type = None + + if register_as_type is not None: + existing_registration = self._registered_manifest_types.get( + register_as_type + ) + if existing_registration is not None: + raise ValueError( + f"Cannot register rule {rule_name!r} for plugin {self._plugin_name}. The plugin {existing_registration.plugin_name} already registered a manifest rule with type {register_as_type!r}" + ) + self._registered_manifest_types[register_as_type] = self._plugin_metadata + inline_reference_documentation = self._pluggable_manifest_docs_for( rule_type, rule_name, @@ -1008,10 +1031,22 @@ class DebputyPluginInitializerProvider(DebputyPluginInitializer): expected_debputy_integration_mode=expected_debputy_integration_mode, automatic_docs=docs, ) + + def _registering_handler( + name: str, + parsed_data: PF, + attribute_path: AttributePath, + parser_context: ParserContextData, + ) -> TP: + value = handler(name, parsed_data, attribute_path, parser_context) + if register_as_type is not None: + register_manifest_type_value_in_context(register_as_type, value) + return value + dispatching_parser.register_parser( rule_name, parser, - wrap_plugin_code(self._plugin_name, handler), + wrap_plugin_code(self._plugin_name, _registering_handler), self._plugin_metadata, ) diff --git a/src/debputy/plugin/api/impl_types.py b/src/debputy/plugin/api/impl_types.py index 0ef3646e..88190ff0 100644 --- a/src/debputy/plugin/api/impl_types.py +++ b/src/debputy/plugin/api/impl_types.py @@ -1,27 +1,20 @@ import dataclasses import os.path +from collections.abc import Callable, Sequence, Iterable, Mapping, Iterator, Container from importlib.resources.abc import Traversable from pathlib import Path from typing import ( Optional, - FrozenSet, - Dict, - List, - Tuple, Generic, TYPE_CHECKING, TypeVar, cast, Any, - Union, - Type, TypedDict, NotRequired, Literal, - Set, Protocol, ) -from collections.abc import Callable, Sequence, Iterable, Mapping, Iterator, Container from weakref import ref from debputy.exceptions import ( @@ -36,7 +29,7 @@ from debputy.filesystem_scan import as_path_def from debputy.manifest_parser.exceptions import ManifestParseException from debputy.manifest_parser.tagging_types import DebputyParsedContent, TypeMapping from debputy.manifest_parser.util import AttributePath, check_integration_mode -from debputy.packages import BinaryPackage +from debputy.packages import BinaryPackage, SourcePackage from debputy.plugin.api import ( VirtualPath, BinaryCtrlAccessor, @@ -1253,6 +1246,10 @@ class PackageProcessingContextProvider(PackageProcessingContext): include_binnmu_version=not package.is_arch_all ) + @property + def source_package(self) -> SourcePackage: + return self._manifest.source_package + @property def binary_package(self) -> BinaryPackage: return self._binary_package @@ -1297,6 +1294,13 @@ class PackageProcessingContextProvider(PackageProcessingContext): self._cross_check_cache = cache return cache + def manifest_configuration[T]( + self, + context_package: SourcePackage | BinaryPackage, + value_type: type[T], + ) -> T | None: + return self._manifest.manifest_configuration(context_package, value_type) + @dataclasses.dataclass(slots=True, frozen=True) class PluginProvidedTrigger: diff --git a/src/debputy/plugin/api/spec.py b/src/debputy/plugin/api/spec.py index 928a9140..05aaaab4 100644 --- a/src/debputy/plugin/api/spec.py +++ b/src/debputy/plugin/api/spec.py @@ -36,7 +36,7 @@ from debputy.exceptions import ( from debputy.interpreter import Interpreter, extract_shebang_interpreter_from_file from debputy.manifest_parser.tagging_types import DebputyDispatchableType from debputy.manifest_parser.util import parse_symbolic_mode -from debputy.packages import BinaryPackage +from debputy.packages import BinaryPackage, SourcePackage from debputy.types import S if TYPE_CHECKING: @@ -332,6 +332,11 @@ class PackageProcessingContext: __slots__ = () + @property + def source_package(self) -> SourcePackage: + """The source package stanza from `debian/control`""" + raise NotImplementedError + @property def binary_package(self) -> BinaryPackage: """The binary package stanza from `debian/control`""" @@ -361,8 +366,23 @@ class PackageProcessingContext: def accessible_package_roots(self) -> Iterable[tuple[BinaryPackage, "VirtualPath"]]: raise NotImplementedError - # """The source package stanza from `debian/control`""" - # source_package: SourcePackage + def manifest_configuration[T]( + self, + context_package: SourcePackage | BinaryPackage, + value_type: type[T], + ) -> T | None: + """Request access to configuration from the manifest + + This method will return the value associated with a pluggable manifest rule assuming + said configuration was provided. + + + :param context_package: The context in which the configuration will be. Generally, it will be + the binary package for anything under `packages:` and the source package otherwise. + :param value_type: The + :return: + """ + raise NotImplementedError class DebputyPluginDefinition: diff --git a/src/debputy/plugin/api/test_api/test_impl.py b/src/debputy/plugin/api/test_api/test_impl.py index e29bba35..a3209d4f 100644 --- a/src/debputy/plugin/api/test_api/test_impl.py +++ b/src/debputy/plugin/api/test_api/test_impl.py @@ -2,21 +2,13 @@ import contextlib import dataclasses import inspect import os.path +from collections.abc import Mapping, Sequence, Iterator, KeysView, Callable from importlib.resources.abc import Traversable from io import BytesIO from pathlib import Path from typing import ( - Dict, - Optional, - Tuple, - List, - cast, - FrozenSet, - Union, - Type, - Set, + cast, TYPE_CHECKING, ) -from collections.abc import Mapping, Sequence, Iterator, KeysView, Callable from debian.deb822 import Deb822 from debian.debian_support import DpkgArchTable @@ -25,7 +17,7 @@ from debian.substvars import Substvars from debputy import DEBPUTY_PLUGIN_ROOT_DIR from debputy.architecture_support import faked_arch_table from debputy.filesystem_scan import OSFSROOverlay, FSRootDir -from debputy.packages import BinaryPackage +from debputy.packages import BinaryPackage, SourcePackage from debputy.plugin.api import ( PluginInitializationEntryPoint, VirtualPath, @@ -34,6 +26,7 @@ from debputy.plugin.api import ( Maintscript, ) from debputy.plugin.api.example_processing import process_discard_rule_example +from debputy.plugin.api.feature_set import PluginProvidedFeatureSet from debputy.plugin.api.impl import ( plugin_metadata_for_debputys_own_plugin, DebputyPluginInitializerProvider, @@ -50,7 +43,6 @@ from debputy.plugin.api.impl_types import ( PluginProvidedTrigger, ServiceManagerDetails, ) -from debputy.plugin.api.feature_set import PluginProvidedFeatureSet from debputy.plugin.api.spec import ( MaintscriptAccessor, FlushableSubstvars, @@ -72,16 +64,25 @@ from debputy.plugins.debputy.debputy_plugin import initialize_debputy_features from debputy.substitution import SubstitutionImpl, VariableContext, Substitution from debputy.util import package_cross_check_precheck +if TYPE_CHECKING: + from debputy.highlevel_manifest import HighLevelManifest + + RegisteredPackagerProvidedFile.register(PackagerProvidedFileClassSpec) +type ManifestConfigurationImplementation[T] = Callable[[SourcePackage | BinaryPackage, type[T]], T] + + @dataclasses.dataclass(frozen=True, slots=True) class PackageProcessingContextTestProvider(PackageProcessingContext): + source_package: SourcePackage binary_package: BinaryPackage binary_package_version: str related_udeb_package: BinaryPackage | None related_udeb_package_version: str | None accessible_package_roots: Callable[[], Sequence[tuple[BinaryPackage, VirtualPath]]] + manifest_configuration: ManifestConfigurationImplementation def _initialize_plugin_under_test( @@ -280,6 +281,8 @@ def package_metadata_context( should_be_acted_on: bool = True, related_udeb_fs_root: VirtualPath | None = None, accessible_package_roots: Sequence[tuple[Mapping[str, str], VirtualPath]] = tuple(), + source_package_fields: dict[str, str] | None = None, + manifest_configuration: ManifestConfigurationImplementation = lambda x, y: None, ) -> PackageProcessingContext: process_table = faked_arch_table(host_arch) f = { @@ -297,6 +300,12 @@ def package_metadata_context( should_be_acted_on=should_be_acted_on, ) udeb_package = None + s = { + "Source": bin_package.name, + } + if source_package_fields is not None: + s.update(source_package_fields) + source_package = SourcePackage(Deb822(s)) if related_udeb_package_fields is not None: uf = dict(related_udeb_package_fields) uf.setdefault("Package", f'{f["Package"]}-udeb') @@ -354,11 +363,13 @@ def package_metadata_context( final_apr = tuple() return PackageProcessingContextTestProvider( + source_package=source_package, binary_package=bin_package, related_udeb_package=udeb_package, binary_package_version=binary_package_version, related_udeb_package_version=related_udeb_package_version, accessible_package_roots=lambda: final_apr, + manifest_configuration=manifest_configuration, ) diff --git a/src/debputy/plugin/plugin_state.py b/src/debputy/plugin/plugin_state.py index 91f87c94..c074eb9e 100644 --- a/src/debputy/plugin/plugin_state.py +++ b/src/debputy/plugin/plugin_state.py @@ -1,25 +1,86 @@ +import collections.abc +import contextlib import contextvars import functools import inspect -from contextvars import ContextVar -from typing import Optional, ParamSpec, TypeVar, NoReturn, Union from collections.abc import Callable +from contextvars import ContextVar +from typing import ParamSpec, TypeVar, NoReturn, Any from debputy.exceptions import ( UnhandledOrUnexpectedErrorFromPluginError, DebputyRuntimeError, ) +from debputy.packages import SourcePackage, BinaryPackage from debputy.util import _trace_log, _is_trace_log_enabled _current_debputy_plugin_cxt_var: ContextVar[str | None] = ContextVar( "current_debputy_plugin", default=None, ) +_current_debputy_parsing_context: ContextVar[ + tuple[ + dict[tuple[SourcePackage | BinaryPackage, type[Any]], Any], + SourcePackage | BinaryPackage, + ] + | None +] = ContextVar( + "current_debputy_parsing_context", + default=None, +) P = ParamSpec("P") R = TypeVar("R") +def register_manifest_type_value_in_context( + value_type: type[Any], + value: Any, +) -> None: + context_vars = _current_debputy_parsing_context.get() + if context_vars is None: + raise AssertionError( + "register_manifest_type_value_in_context() was called, but no context was set." + ) + value_table, context_pkg = context_vars + if (context_pkg, value_type) in value_table: + raise AssertionError( + f"The type {value_type!r} was already registered for {context_pkg}, which the plugin API should have prevented" + ) + value_table[(context_pkg, value_type)] = value + + +def begin_parsing_context( + value_table: dict[tuple[SourcePackage | BinaryPackage, type[Any]], Any], + context_pkg: SourcePackage, + func: Callable[P, R], + *args: P.args, + **kwargs: P.kwargs, +) -> R: + context = contextvars.copy_context() + # Wish we could just do a regular set without wrapping it in `context.run` + context.run(_current_debputy_parsing_context.set, (value_table, context_pkg)) + assert context.get(_current_debputy_parsing_context) == (value_table, context_pkg) + return context.run(func, *args, **kwargs) + + [email protected] +def with_binary_pkg_parsing_context( + context_pkg: BinaryPackage, +) -> collections.abc.Iterator[None]: + context_vars = _current_debputy_parsing_context.get() + if context_vars is None: + raise AssertionError( + "with_binary_pkg_parsing_context() was called, but no context was set." + ) + value_table, _ = context_vars + token = _current_debputy_parsing_context.set((value_table, context_pkg)) + try: + yield + finally: + _current_debputy_parsing_context.reset(token) + + def current_debputy_plugin_if_present() -> str | None: return _current_debputy_plugin_cxt_var.get() diff --git a/src/debputy/plugins/debputy/binary_package_rules.py b/src/debputy/plugins/debputy/binary_package_rules.py index e6cd1169..fddc67e7 100644 --- a/src/debputy/plugins/debputy/binary_package_rules.py +++ b/src/debputy/plugins/debputy/binary_package_rules.py @@ -78,6 +78,7 @@ def register_binary_package_rules(api: DebputyPluginInitializerProvider) -> None BinaryVersionParsedFormat, _parse_binary_version, source_format=str, + register_value=False, ) api.pluggable_manifest_rule( @@ -85,6 +86,7 @@ def register_binary_package_rules(api: DebputyPluginInitializerProvider) -> None "transformations", list[TransformationRule], _unpack_list, + register_value=False, ) api.pluggable_manifest_rule( @@ -95,6 +97,7 @@ def register_binary_package_rules(api: DebputyPluginInitializerProvider) -> None expected_debputy_integration_mode=not_integrations( INTEGRATION_MODE_DH_DEBPUTY_RRR ), + register_value=False, ) api.pluggable_manifest_rule( @@ -106,6 +109,7 @@ def register_binary_package_rules(api: DebputyPluginInitializerProvider) -> None expected_debputy_integration_mode=not_integrations( INTEGRATION_MODE_DH_DEBPUTY_RRR ), + register_value=False, ) api.pluggable_manifest_rule( @@ -119,6 +123,7 @@ def register_binary_package_rules(api: DebputyPluginInitializerProvider) -> None expected_debputy_integration_mode=not_integrations( INTEGRATION_MODE_DH_DEBPUTY_RRR ), + register_value=False, ) api.pluggable_manifest_rule( @@ -130,6 +135,7 @@ def register_binary_package_rules(api: DebputyPluginInitializerProvider) -> None expected_debputy_integration_mode=not_integrations( INTEGRATION_MODE_DH_DEBPUTY_RRR ), + register_value=False, ) diff --git a/src/debputy/plugins/debputy/build_system_rules.py b/src/debputy/plugins/debputy/build_system_rules.py index bc2f580a..2813e089 100644 --- a/src/debputy/plugins/debputy/build_system_rules.py +++ b/src/debputy/plugins/debputy/build_system_rules.py @@ -98,6 +98,7 @@ def register_build_keywords(api: DebputyPluginInitializerProvider) -> None: "build-environments", list[NamedEnvironmentSourceFormat], _parse_build_environments, + register_value=False, expected_debputy_integration_mode=only_integrations(INTEGRATION_MODE_FULL), inline_reference_documentation=reference_documentation( title="Build Environments (`build-environments`)", @@ -192,6 +193,7 @@ def register_build_keywords(api: DebputyPluginInitializerProvider) -> None: "default-build-environment", EnvironmentSourceFormat, _parse_default_environment, + register_value=False, expected_debputy_integration_mode=only_integrations(INTEGRATION_MODE_FULL), inline_reference_documentation=reference_documentation( title="Default Build Environment (`default-build-environment`)", @@ -272,6 +274,7 @@ def register_build_keywords(api: DebputyPluginInitializerProvider) -> None: MK_BUILDS, list[BuildRule], _handle_build_rules, + register_value=False, expected_debputy_integration_mode=only_integrations(INTEGRATION_MODE_FULL), inline_reference_documentation=reference_documentation( title=f"Build rules (`{MK_BUILDS}`)", diff --git a/src/debputy/plugins/debputy/manifest_root_rules.py b/src/debputy/plugins/debputy/manifest_root_rules.py index 8e54edf9..513b4f34 100644 --- a/src/debputy/plugins/debputy/manifest_root_rules.py +++ b/src/debputy/plugins/debputy/manifest_root_rules.py @@ -45,6 +45,7 @@ def register_manifest_root_rules(api: DebputyPluginInitializerProvider) -> None: ManifestVersionFormat, _handle_version, source_format=ManifestVersion, + register_value=False, ) api.pluggable_object_parser( OPARSER_MANIFEST_ROOT, @@ -58,12 +59,14 @@ def register_manifest_root_rules(api: DebputyPluginInitializerProvider) -> None: ManifestVariablesParsedFormat, _handle_manifest_variables, source_format=dict[str, str], + register_value=False, ) api.pluggable_manifest_rule( OPARSER_MANIFEST_ROOT, MK_INSTALLATIONS, list[InstallRule], _handle_installation_rules, + register_value=False, expected_debputy_integration_mode=not_integrations( INTEGRATION_MODE_DH_DEBPUTY_RRR ), @@ -73,6 +76,7 @@ def register_manifest_root_rules(api: DebputyPluginInitializerProvider) -> None: MK_MANIFEST_REMOVE_DURING_CLEAN, list[RemoveDuringCleanParsedFormat], _handle_remove_during_clean, + register_value=False, expected_debputy_integration_mode=only_integrations( INTEGRATION_MODE_FULL, ), diff --git a/src/debputy/plugins/debputy/metadata_detectors.py b/src/debputy/plugins/debputy/metadata_detectors.py index 3a08a18a..3f3a1726 100644 --- a/src/debputy/plugins/debputy/metadata_detectors.py +++ b/src/debputy/plugins/debputy/metadata_detectors.py @@ -17,7 +17,7 @@ from debputy.plugins.debputy.paths import ( SYSTEMD_SYSUSERS_DIR, ) from debputy.plugins.debputy.types import DebputyCapability -from debputy.util import assume_not_none, _warn +from debputy.util import assume_not_none, _warn, _error DPKG_ROOT = '"${DPKG_ROOT}"' DPKG_ROOT_UNQUOTED = "${DPKG_ROOT}" -- 2.51.0
OpenPGP_signature.asc
Description: OpenPGP digital signature
