Do you remember kergoth's take on this? From "[RFC PATCH] devtool: remove _PYTHON_SYSCONFIGDATA_NAME to fix do_unpack" thread and https://lists.yoctoproject.org/g/yocto/topic/74637733#49571
https://github.com/openembedded/openembedded-core/compare/master...kergoth:python-sysconfig I think wrapper is a bit better, because it will also catch cases where host python3 is being called from e.g. do_compile (e.g. by full path - after debugging ROS catkin in last few days, some components are really creative how to find their preferred python) even when python3native is inherited. And also doesn't need another bbclass to be inherited in cases where target _sysconfigdata is really needed. On Thu, Nov 12, 2020 at 8:26 PM Alexander Kanavin <[email protected]> wrote: > Thanks - I have a preliminary fix here (needs thorough testing): > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=akanavin/package-version-updates&id=d6b6a6f67cc967c0ead6dbab3c95435b7ca85246 > > Alex > > On Thu, 12 Nov 2020 at 19:46, Glenn Stroz <[email protected]> wrote: > >> Hi Alex, >> >> We have a custom fetcher implemented as a bbclass that calls out to a >> python script. That is why we see these issues with native. Here's a >> sample bbclass and recipe that reproduces the issue: >> >> impfail.bbclass >> ``` >> inherit python3native >> >> do_unpack[noexec] = "1" >> do_patch[noexec] = "1" >> do_install[noexec] = "1" >> >> do_fetch() { >> python3 -c "import pydoc" >> } >> ``` >> >> classrunner_0.0.1.bb >> ``` >> SUMMARY = "Python runner" >> LICENSE = "CLOSED" >> inherit impfail >> ``` >> >> -Glenn >> >> >> On Thu, Nov 12, 2020 at 4:52 AM Alexander Kanavin <[email protected]> >> wrote: >> >>> A quick fix may be to add a target python3 dependency to your recipe, >>> yes ugly, but I can't figure out how to provide target python configuration >>> to native python used in the context of building target recipes otherwise. >>> >>> Alex >>> >>> >>> On Thu, 12 Nov 2020 at 10:49, Alexander Kanavin <[email protected]> >>> wrote: >>> >>>> On Thu, 12 Nov 2020 at 02:23, Glenn Stroz via lists.openembedded.org >>>> <[email protected]> wrote: >>>> >>>>> We've been upgrading our work to Dunfell and have encountered an issue >>>>> with python3-native. >>>>> >>>>> One of our python3-native scripts uses pydoc and that in turn >>>>> eventually pulls in python's sysconfig. However, this is failing to import >>>>> with the following stack trace: >>>>> >>>>> ``` >>>>> import pydoc >>>>> File >>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/pydoc.py", >>>>> line 366, in <module> >>>>> class Doc: >>>>> File >>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/pydoc.py", >>>>> line 396, in Doc >>>>> def getdocloc(self, object, basedir=sysconfig.get_path('stdlib')): >>>>> File >>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/sysconfig.py", >>>>> line 519, in get_path >>>>> return get_paths(scheme, vars, expand)[name] >>>>> File >>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/sysconfig.py", >>>>> line 509, in get_paths >>>>> return _expand_vars(scheme, vars) >>>>> File >>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/sysconfig.py", >>>>> line 172, in _expand_vars >>>>> _extend_dict(vars, get_config_vars()) >>>>> File >>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/sysconfig.py", >>>>> line 558, in get_config_vars >>>>> _init_posix(_CONFIG_VARS) >>>>> File >>>>> "/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python3.8/sysconfig.py", >>>>> line 428, in _init_posix >>>>> _temp = __import__(name, globals(), locals(), ['build_time_vars'], >>>>> 0) >>>>> ModuleNotFoundError: No module named '_sysconfigdata' >>>>> ``` >>>>> >>>>> I did a little digging and the following commit seems to reference >>>>> related work: >>>>> >>>>> >>>>> https://github.com/openembedded/openembedded-core/commit/02714c105426b0d687620913c1a7401b386428b6 >>>>> >>>>> I see that there is a patch added which adds the following to >>>>> `sysconfig.py`: >>>>> >>>>> ``` >>>>> if 'STAGING_LIBDIR' in os.environ: >>>>> >>>>> sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata') >>>>> ``` >>>>> >>>>> However, STAGING_LIBDIR does not contain this path: >>>>> >>>>> ``` >>>>> ls >>>>> /home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot/usr/lib/python-sysconfigdata >>>>> >>>>> ls: cannot access >>>>> '/home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot/usr/lib/python-sysconfigdata': >>>>> No such file or directory >>>>> ``` >>>>> >>>>> It appears as if this should really be pointing >>>>> to STAGING_LIBDIR_NATIVE which does exist: >>>>> >>>>> ``` >>>>> ls >>>>> /home/gstroz/.cache/avos/_avos_gstroz/avos/yocto-dunfell/build/tmp/work/corei7-64-poky-linux/ebpf-daemon/2020.282.17-r0/recipe-sysroot-native/usr/lib/python-sysconfigdata >>>>> _sysconfigdata.py >>>>> ``` >>>>> >>>>> The referenced commit made efforts to combine the native and cross >>>>> python3 bbclass files into one. However, I think this may be something >>>>> that >>>>> needs to be treated differently. >>>>> >>>>> Anyone have any thoughts on how to proceed? >>>>> >>>> >>>> Can you share the exact context where this is happening? Is your script >>>> coming from a recipe and can you show that recipe? >>>> >>>> Alex >>>> >>>> >>> > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#144533): https://lists.openembedded.org/g/openembedded-core/message/144533 Mute This Topic: https://lists.openembedded.org/mt/78197661/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
