On Mon, 2022-03-07 at 08:36 +0100, Konrad Weihmann wrote: > > On 04.03.22 00:46, Richard Purdie wrote: > > On Thu, 2022-03-03 at 15:28 -0800, Tim Orling wrote: > > > > > > > > > On Thu, Mar 3, 2022 at 9:16 AM Richard Purdie > > > <[email protected]> wrote: > > > > On Thu, 2022-03-03 at 17:14 +0000, Ross Burton wrote: > > > > > On Thu, 3 Mar 2022 at 16:34, Richard Purdie > > > > > <[email protected]> wrote: > > > > > > This removes a further 1600 files from sstate handling and lets > > > > > > python > > > > > > create the ones it wants at runtime which is likely much better > > > > > > overall > > > > > > for performance. > > > > > > > > > > Playing devil's advocate: doesn't having them in sstate mean they're > > > > > generated once and hardlinked, instead of needing to be generated for > > > > > every recipe which runs pythonnative? > > > > > > > > > > Whilst I can't disagree that 1600 files being dropped from sstate is > > > > > good, we're just punting the recompile step to every recipe when it > > > > > runs python code. > > > > > > > > > > I guess the question here is how long does the Python library take to > > > > recompile. > > > > > > > > > > > > > At runtime it should only generate pyc for modules actually used? > > > > Yes. > > > > > > Another consideration is that there are many sysroots pulling in > > > > python3- > > > > native > > > > which don't run python and they're only there as there are python > > > > scripts > > > > being > > > > added which means python has to come too. > > > > > > > > I suspect for that reason it could be a net win but it is a tough call. > > > > > > > > > > I tend to be in favor of the reduced number of files moving around. I > > > suppose > > > we could do some timing comparisons if we really care. It will be a > > > trade-off > > > of network/io vs compute/io, correct? > > > > Yes. I suspect the compile cost is tiny compared to the io/network cost of > > chucking so many small files around when I suspect they're not used that > > often. > > > > Cheers, > > > > Richard > > Now as this is in master and works reasonably well, could we extent the > removal of pyc for native recipes to *all* python packages. > I'm still seeing occasional staging issues on random native python > recipes as described in > https://bugzilla.yoctoproject.org/show_bug.cgi?id=13868. > And as the common understanding here seems to be that the additional > compile costs are negligible compared to the staging costs of additional > files, I would propose to add > > do_install:append:class-native() { > find ${D} -name *.pyc -delete > } > > to setuptools3.bbclass (as this looks like the one piece of code that > almost all python packages have in common) > > Any thoughts?
I don't really like working around errors like the above bug where we haven't understood the underlying issue :/ I suspect those pyc files are nowhere near as large an issue as the ones in the python recipe itself either, there are likely only small numbers of them. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#162803): https://lists.openembedded.org/g/openembedded-core/message/162803 Mute This Topic: https://lists.openembedded.org/mt/89528765/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
