I have filed https://bugzilla.yoctoproject.org/show_bug.cgi?id=14638 to capture and track this necessary change.
Konrad, you don't seem to have a BZ account? so I couldn't CC you. On Wed, Nov 24, 2021 at 10:52 AM Tim Orling via lists.openembedded.org <[email protected]> wrote: > > > On Wed, Nov 24, 2021 at 10:43 AM Konrad Weihmann <[email protected]> > wrote: > >> >> >> On 24.11.21 19:30, Konrad Weihmann wrote: >> > On 24.11.21 18:44, Tim Orling wrote: >> >> >> >> >> >> On Tue, Nov 23, 2021 at 9:40 AM Konrad Weihmann <[email protected] >> >> <mailto:[email protected]>> wrote: >> >> >> >> Hi all, >> >> >> >> recently more and more python module (preferably downloaded from >> >> pypi) >> >> do not contain a setup.py anymore. >> >> >> >> >> >> Can you list some examples, so we can start a todo list? I have come >> >> across one or two, but it is difficult to search for them. It would >> >> really help me to have some test cases. (I have a similar issue with >> >> pyypi packages with Rust extensions... not easy to find them). >> > >> > In the things that I maintain I so far came across >> > >> > javaproperties -> using setup.cfg >> > jsonschema -> using setup.cfg >> > tomli -> same >> > typing_extensions -> using just pyproject.toml >> > >> > for the first ones I started working on a small conversion util that >> > will create a "fake" setup.py using the input from setup.cfg (there is >> a >> > well documented tranlation table available directly from pypa). >> > Results have been quite good so far - so we could at least cover most >> of >> > the projects that made the transition away from setup.py. >> > >> > For the last one, I'm currently out of ideas, but to patch in a crafted >> > setup.py. >> > >> > From what I understood in the pypa discussions pyproject.toml will >> > likely be the preferred choice. >> > support for setup.cfg probably will be declared outdated, but likely >> > support will remain indefinitely. >> > >> >> >> >> >> >> I remember there were attempts to use >> https://github.com/pypa/build >> >> <https://github.com/pypa/build> >> >> instead, but to me it always boiled down to a huge chicken and egg >> >> issue, as the tool itself has quite a large dependency list [1]. >> >> >> >> I have been working with python3-pep517, python3-build, python3-flit >> >> recipes, but they are not ready yet. More below. >> > >> > Thanks for the update - so I haven't missed out on any so far >> > >> >> >> >> Lately I've seen rather hackish approaches to the issue [2]. >> >> Is this the way we need to go? >> >> >> >> >> >> As Denys said, this was ONLY because of the circular dependency issue. >> >> Until upstream Python has a TOML parser in CPython, we need this >> >> setup.cfg workaround for python3-tomli specifically. >> > >> > I think my new util experiment will solve that >> > >> >> >> >> If so, wouldn't it make s-ense to create some bb-helper-class for >> it >> >> (I'm >> >> willing to give it a try). >> >> >> >> What we really need is a bbclass that knows how to build and install >> >> wheels. We will need to pass the --no-dependencies argument to >> >> pip-native. We need to make absolutely certain that pip is not >> >> downloading and installing anything for us. The concept is that the >> >> contents from bdist_wheel (the "wheel") would be installed into the >> >> sysroot and then the resulting files would be what we actually package >> >> in ipk/deb/rpm. I haven't come up with a brilliant name for the new >> >> bbclass, but perhaps pypa-wheels? >> > >> > Wheels are definitely not the right choice IMO - and looking at the >> > different directions the discussions at pypa go, I think we are still >> > far from having an accepted and stable packaging solution for python >> > code (that comes without the very same dependency loop such as >> toml/tomli) >> > >> >> >> >> NOTE: I have zero intention of supporting installation of third-party >> >> wheels. Only wheels we have built. >> >> >> >> I have begun some local sandbox on this, but again, not ready yet. >> >> Between the Phosh and Kernel-Lab presentations for the Yocto Summit I >> >> have been very backlogged. One promising package is flit, it has a >> >> bootstrapping module in flit_core that we can use to build the >> >> bdist_wheel with very little else. That can then be used to build >> >> python3-flit-native, which could then be used for other packages that >> >> use flit in their pyproject.toml. >> >> >> >> Did anyone had a in-depth talk with pypa upstream about that and >> what >> >> was the outcome of that discussion? >> >> >> >> You can find various conversations in GitHub issues or PEP >> >> discussions, but ultimately it comes down to: >> >> (1) setup.py might be provided on an interim basis, but is deprecated >> >> (2) egg-info is deprecated in favor of wheels >> >> (3) setup.py is deprecated in favor of pyproject.toml (and also >> >> sometimes setup.cfg, but this is inconsistent) >> >> (3) packages are installed with pip >> > >> > Thanks, will look into it in the next days >> > >> >> >> >> We cannot fight this change, which has been a long time coming >> >> already. We need to adapt to it. >> > >> > So far unfortunately (also judging from the pypa discussions) I don't >> > see many options, but to remain on the outdated approach in core and >> > apply some crafted or patched fallback setup.pys. >> > >> >> >> >> We have a "languages" topic for the OEDVM, and python is amongst the >> >> things affected. >> >> As of now I think the most viable solution would be if setuptools and >> all that toml parsing stuff would be moved into python-core, but I have >> no idea if that is somehow on the python upstream's roadmap. >> >> I agree. The argument against is that having things outside python-core > (CPython) allows for a more nimble, agile development process. But this > feels like it assumes that there are infinite developers with infinite time > to fix infinite packages. On the other hand, we ourselves are making > breaking changes to oe-core so that we can move forward. I can understand > both sides of the argument. My plan is to adapt to upstream pypa. > > >> >> >> Cheers, >> >> >> >> --Tim >> >> >> >> Regards >> >> Konrad >> >> >> >> [1] https://github.com/pypa/build/blob/main/setup.cfg >> >> <https://github.com/pypa/build/blob/main/setup.cfg> >> >> [2] >> >> >> >> >> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta/recipes-devtools/python?id=6cc9954c6bac9e3c19b2b0f26a69b629a8ee7273 >> >> >> >> >> >> < >> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta/recipes-devtools/python?id=6cc9954c6bac9e3c19b2b0f26a69b629a8ee7273> >> >> >> >> >> >> >> >> > >> > >> > >> > >> > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1368): https://lists.openembedded.org/g/openembedded-architecture/message/1368 Mute This Topic: https://lists.openembedded.org/mt/87264004/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
