Re: %pyproject_buildrequires -r/-x: Attempt to read runtime dependencies from pyproject.toml?
On 27-03-2024 15:48, Karolina Surma wrote: One way to mitigate would be to make the proposed behavior opt-in only, with the possibility to either build wheel with -w option (already existing) or e.g. -p (now-proposed: reading from pyproject.toml) in case backend doesn't have prepare_metadata_for_build_wheel. However, this adds a layer of complexity for packagers and macros maintainers. The questions we'd love your input for: - Should %pyproject_buildrequires try to read dependencies from pyproject.toml first and fall back to calling hooks only if that fails? - Should %pyproject_buildrequires call the hook and try to fall back to reading dependencies from pyproject.toml when the hook is not availbale? - Should this behavior exist but not be the default (explicit flag would be required to opt-in)? - Can you think of a better alternative than the ones described here? I'd be in favor of option two in combination with option three. That is, set a flag explicitly two enable the behavior described in option two, fallback to pyproject.toml if hook is not available. Doing it that way also means the current behavior remains unchanged for package maintainers being unaware similar to the -l flag of %pyproject_save_files. I'm approaching this purely from a packager's point of view. I dislike having the wheel built twice, once for the metadata and again for the package itself. The documentation could point to the build backends, where it makes sense enabling -p, e.g. meson-python. -- Sandro -- ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: %pyproject_buildrequires -r/-x: Attempt to read runtime dependencies from pyproject.toml?
On 27. 03. 24 23:15, Maxwell G wrote: One way to mitigate would be to make the proposed behavior opt-in only, with the possibility to either build wheel with -w option (already existing) or e.g. -p (now-proposed: reading from pyproject.toml) in case backend doesn't have prepare_metadata_for_build_wheel. Yeah, I think -p (for *p*yproject) is good flag name choice. Or even for [*p*roject] table. It is double good. I guess I will throw something out there, but I am not convinced it is a great idea: what about making the `-p` flag fail if `prepare_metadata_for_build_wheel` is available? In my opinion, this should only be a last resort for backends that do not implement the hook. I am not particularly keen on this. This means that once the backend starts supporting it, all the spec files using -p need to drop it. And if the backend only supports it in rawhide, the spec files need to diverge and/or %if-guard the flag. If the backend followed PEP 621 before adding the hook and now it added the hook, it is unlikely the PEP 621 support was dropped. OTHO if the backed was changed (e.g. meson -> poetry), this *could* happen. So I am not entirely opposed for this guard. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok -- ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: %pyproject_buildrequires -r/-x: Attempt to read runtime dependencies from pyproject.toml?
On Wed Mar 27, 2024 at 15:48 +0100, Karolina Surma wrote: > Hello, Hi Karolina, Thank you for bring this to the mailing list. > recently, we were suggested an improvement for %pyproject_buildrequires > -r/-x. > We could read the project's runtime dependencies (if they're not marked > as `dynamic`) from pyproject.toml [project] table directly, without > calling prepare_metadata_for_build_wheel or building the wheel to read > the dependency metadata from it. > Reading the metadata via prepare_metadata_for_build_wheel is already > quite fast, however implementing that hook is optional for the build > backends. I'll note that all build backends packaged in Fedora (setuptools, hatchling, flit-core, poetry-core, the pdm one, and scikit) other than meson-python support this hook. Only 9 packages BuildRequire python3-meson-python. > Maxwell has raised some valid concerns there: > - Other tools (build frontends, e.g. pip/build) call > prepare_metadata_for_build_wheel, our macros would diverge in > functionality compared to the rest of the landscape. > - pyproject.toml's [project] may not be the source of metadata that the > build backend uses. A project could switch to a build backend without > PEP 621 support (e.g. poetry-core) and forget to remove the [project] table. > - There can be potential differences between BuildRequires and Requires > generators when one generates dependencies based on the pyproject.toml > [project] table while the other on the packaged dist-info metadata. > - Using macros to build on older systems: e.g. RHEL 9's old setuptools > version silently ignores the pyproject.toml [project] table - > %pyproject_buildrequires could still pull the dependency information > from it, but the resulting package would be broken because it did not > include those in the packaged dist-info metadata. I could not have said it better myself :D. > One way to mitigate would be to make the proposed behavior opt-in only, > with the possibility to either build wheel with -w option (already > existing) or e.g. -p (now-proposed: reading from pyproject.toml) in case > backend doesn't have prepare_metadata_for_build_wheel. Yeah, I think -p (for *p*yproject) is good flag name choice. > However, this adds a layer of complexity for packagers and macros > maintainers. > The questions we'd love your input for: > - Should this behavior exist but not be the default (explicit flag > would be required to opt-in)? I consider this the only reasonable solution other than not implementing the feature at all. > - Can you think of a better alternative than the ones described here? I guess I will throw something out there, but I am not convinced it is a great idea: what about making the `-p` flag fail if `prepare_metadata_for_build_wheel` is available? In my opinion, this should only be a last resort for backends that do not implement the hook. —Maxwell -- ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
%pyproject_buildrequires -r/-x: Attempt to read runtime dependencies from pyproject.toml?
Hello, recently, we were suggested an improvement for %pyproject_buildrequires -r/-x. We could read the project's runtime dependencies (if they're not marked as `dynamic`) from pyproject.toml [project] table directly, without calling prepare_metadata_for_build_wheel or building the wheel to read the dependency metadata from it. Reading the metadata via prepare_metadata_for_build_wheel is already quite fast, however implementing that hook is optional for the build backends. When the hook is not available, we need to build the wheel, which can be potentially very slow and resource-hungry (imagine building the entire scipy project). Metadata in pyproject.toml is standardized (PEP 621), however there is no way of telling ahead of time if the build backend already supports that standard. See the details and discussion in bugzilla: https://bugzilla.redhat.com/2261939 Maxwell has raised some valid concerns there: - Other tools (build frontends, e.g. pip/build) call prepare_metadata_for_build_wheel, our macros would diverge in functionality compared to the rest of the landscape. - pyproject.toml's [project] may not be the source of metadata that the build backend uses. A project could switch to a build backend without PEP 621 support (e.g. poetry-core) and forget to remove the [project] table. - There can be potential differences between BuildRequires and Requires generators when one generates dependencies based on the pyproject.toml [project] table while the other on the packaged dist-info metadata. - Using macros to build on older systems: e.g. RHEL 9's old setuptools version silently ignores the pyproject.toml [project] table - %pyproject_buildrequires could still pull the dependency information from it, but the resulting package would be broken because it did not include those in the packaged dist-info metadata. One way to mitigate would be to make the proposed behavior opt-in only, with the possibility to either build wheel with -w option (already existing) or e.g. -p (now-proposed: reading from pyproject.toml) in case backend doesn't have prepare_metadata_for_build_wheel. However, this adds a layer of complexity for packagers and macros maintainers. The questions we'd love your input for: - Should %pyproject_buildrequires try to read dependencies from pyproject.toml first and fall back to calling hooks only if that fails? - Should %pyproject_buildrequires call the hook and try to fall back to reading dependencies from pyproject.toml when the hook is not availbale? - Should this behavior exist but not be the default (explicit flag would be required to opt-in)? - Can you think of a better alternative than the ones described here? Cheers, Karolina -- ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
