Re: [OE-core] [PATCH v6 03/12] devtool: new ide plugin
On Thu, Oct 12, 2023 at 02:53 PM, Ross Burton wrote:
>
> I’d not noticed image-combined-dbg existed and do wonder if that shoud be
> the behaviour of the debug rootfs. Is there actually a use-case for a tarball
> which is _just_ the symbols?
>
The use for a rootfs containing just the debug tarballs is explicited in the
documentation here:
https://docs.yoctoproject.org/4.0.4/singleindex.html#using-the-gdbserver-method
$ tar xvfj build-dir/tmp-glibc/deploy/images/machine/image.rootfs.tar.bz2
$ tar xvfj build-dir/tmp-glibc/deploy/images/machine/image-dbg.rootfs.tar.bz2
This debug-rootfs is "combined" with the rootfs image which already contains
the binaries. It could also be done at runtime on a live target (through tar,
overlayfs, ...).
The later patch-set makes image-combined-dbg the default which makes the
debug-rootfs already containing the image rootfs. This would not produce an
error with those tar commands because they silently overwrite existing files.
However, it would break some users setups with these conditions:
a. The user has another way for combining the rootfs and dbg rootfs which
produces an error if files exist in both archives
b. The user has storage limitations for the dbg rootfs which would be exceeded
when adding the binaries to it
c. The user customizes the original rootfs output. These customizations would
be lost when combining.
d. The user is extracting the debug rootfs on a runtime rootfs with runtime
modifications
An example for C is wic fstab customization, or any modifications which happen
at the do_rootfs step:
do_rootfs_postprocess() {
echo "Customized issue" >> ${IMAGE_ROOTFS}/etc/issue
}
addtask do_rootfs_postprocess after do_rootfs before do_image
When extracting the debug-rootfs with image-combined-dbg, this customization
gets lost because the debug-rootfs contains the original files from the package.
That's why I suggest keeping the debug-rootfs as it is and keep
image-combined-debug as a separate option.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190490):
https://lists.openembedded.org/g/openembedded-core/message/190490
Mute This Topic: https://lists.openembedded.org/mt/101275530/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][PATCH v6 03/12] devtool: new ide plugin
Hi Ross
Thank you for the re-view.
Summary: I hope everything is fixed with v7:
https://lists.openembedded.org/g/openembedded-core/message/189812
On Thu, 2023-10-12 at 12:53 +, Ross Burton wrote:
> Finally looking at the code…
>
> On 10 Sep 2023, at 16:52, Adrian Freihofer via lists.openembedded.org
> wrote:
> >
> > The new devtool ide plugin configures an IDE to work with the eSDK.
> >
> > With this initial implementation VSCode is the default IDE.
> > The plugin works for recipes inheriting the cmake or the meson
> > bbclass.
> > Support for more programming languages and build tools may be added
> > in
> > the future.
>
> Can the vscode pieces be split out to a separate file, to separate
> out the high level logic and the vscode specifics. I’m pretty sure
> there would be interest in adding qtcreator, for example. By doing
> this sooner rather than later we avoid adding cheeky “lets just
> handle vscode specially” blocks. Also, splitting out the
> cmake/meson/none logic where it isn’t IDE-specific.
>
> Also one thing I’d like to encourage more is decent code
> documentation, to make the code easier to both review and work on in
> the future. Could you add at least basic documentation to the
> classes and major functions, with a summary of how the plugin works?
>
> > + def __gdbserver_start_cmd(self, binary, port):
> > + if self.gdbserver_multi:
> > + gdbserver_cmd = "/usr/bin/gdbserver --multi :%s" % (
> > + port)
> > + else:
> > + gdbserver_cmd = "/usr/bin/gdbserver --once :%s %s" % (
> > + port, binary)
>
> If I’m being pedantic, that should be ${bindir}.
>
> > + if 'debuginfod' in
> > image_d.getVar('DISTRO_FEATURES').split():
> > + # image_config.debuginfod = True
> > + logger.warning("Support for debuginfod is not
> > implemented yet.")
>
> What doesn’t work, and why is this warning needed?
I removed the warnings related to debuginfod. Currently the rootfs-dbg
is used, but debuginfod should not harm and no warning is needed.
>
> > + def solib_search_path_rootfs(self):
> > + """Search for folders with shared libraries in the rootfs
> > and rootfs-dbg
> > +
> > + This is based on the assumption that the
> > PACKAGE_DEBUG_SPLIT_STYLE variable from the image
> > + is the global setting which is used by most packages. Even
> > if this variable does not seem
> > + to make sense in the image context.
> > + """
> > + rootfs_solib_search_path = []
> > + rootfs_dbg_solib_search_path = []
> > + if self.package_debug_split_style in ['debug-with-srcpkg',
> > '.debug']:
> > + if self.combine_dbg_image:
> > + rootfs_dbg_solib_search_path = [
> > + "/lib", "/lib/.debug", "/usr/lib",
> > "/usr/lib/.debug"]
> > + else:
> > + logger.warn(
> > + 'Adding IMAGE_CLASSES += "image-combined-dbg"
> > offers better remote debugging experience.')
> > + rootfs_solib_search_path = [
> > + "/lib", "/usr/lib"]
> > + rootfs_dbg_solib_search_path = [
> > + "/lib/.debug", "/usr/lib/.debug"]
> > + elif self.package_debug_split_style == 'debug-file-
> > directory':
> > + rootfs_dbg_solib_search_path = ["/usr/lib/debug"]
> > + else:
> > + logger.warning(
> > + "Cannot find solib search path for a rootfs built
> > with PACKAGE_DEBUG_SPLIT_STYLE=%s." %
> > self.package_debug_split_style)
> > +
> > + sym_dirs = []
> > + for dbgdir in rootfs_solib_search_path:
> > + sym_dirs.append(os.path.join(
> > + self.rootfs, dbgdir.lstrip('/')))
> > + for dbgdir in rootfs_dbg_solib_search_path:
> > + sym_dirs.append(os.path.join(
> > + self.rootfs_dbg, dbgdir.lstrip('/')))
> > +
> > + return sym_dirs
>
> Feels like there should be a much better way to do this. Does the
> choice of debug split style actually matter? Just search all of the
> candidates and avoid trying to special-case everything?
It's probably still not as simple as you would expect. But it's much
simpler because I dropped the combined-debug-dbg cases. I prefer to
handle the different cases separately and provide a reasonable error
message to the user. There are so many reasons why the debugger cannot
find the symbols and sources.
>
> I’d not noticed image-combined-dbg existed and do wonder if that
> shoud be the behaviour of the debug rootfs. Is there actually a use-
> case for a tarball which is _just_ the symbols?
>
Yes, it seems best to remove the image-combined-dbg.bbclass and make it
the default. There is now a patch that does this. This solves many
problems when tools need debug symbols and sources.
> > +class RecipeMetaIdeSupport:
> > + """Handle some meta-ide-support recipe related properties"""
>
>
Re: [OE-core][PATCH v6 03/12] devtool: new ide plugin
Finally looking at the code…
On 10 Sep 2023, at 16:52, Adrian Freihofer via lists.openembedded.org
wrote:
>
> The new devtool ide plugin configures an IDE to work with the eSDK.
>
> With this initial implementation VSCode is the default IDE.
> The plugin works for recipes inheriting the cmake or the meson bbclass.
> Support for more programming languages and build tools may be added in
> the future.
Can the vscode pieces be split out to a separate file, to separate out the high
level logic and the vscode specifics. I’m pretty sure there would be interest
in adding qtcreator, for example. By doing this sooner rather than later we
avoid adding cheeky “lets just handle vscode specially” blocks. Also,
splitting out the cmake/meson/none logic where it isn’t IDE-specific.
Also one thing I’d like to encourage more is decent code documentation, to make
the code easier to both review and work on in the future. Could you add at
least basic documentation to the classes and major functions, with a summary of
how the plugin works?
> +def __gdbserver_start_cmd(self, binary, port):
> +if self.gdbserver_multi:
> +gdbserver_cmd = "/usr/bin/gdbserver --multi :%s" % (
> +port)
> +else:
> +gdbserver_cmd = "/usr/bin/gdbserver --once :%s %s" % (
> +port, binary)
If I’m being pedantic, that should be ${bindir}.
> +if 'debuginfod' in image_d.getVar('DISTRO_FEATURES').split():
> +# image_config.debuginfod = True
> +logger.warning("Support for debuginfod is not implemented yet.")
What doesn’t work, and why is this warning needed?
> +def solib_search_path_rootfs(self):
> +"""Search for folders with shared libraries in the rootfs and
> rootfs-dbg
> +
> +This is based on the assumption that the PACKAGE_DEBUG_SPLIT_STYLE
> variable from the image
> +is the global setting which is used by most packages. Even if this
> variable does not seem
> +to make sense in the image context.
> +"""
> +rootfs_solib_search_path = []
> +rootfs_dbg_solib_search_path = []
> +if self.package_debug_split_style in ['debug-with-srcpkg', '.debug']:
> +if self.combine_dbg_image:
> +rootfs_dbg_solib_search_path = [
> +"/lib", "/lib/.debug", "/usr/lib", "/usr/lib/.debug"]
> +else:
> +logger.warn(
> +'Adding IMAGE_CLASSES += "image-combined-dbg" offers
> better remote debugging experience.')
> +rootfs_solib_search_path = [
> +"/lib", "/usr/lib"]
> +rootfs_dbg_solib_search_path = [
> +"/lib/.debug", "/usr/lib/.debug"]
> +elif self.package_debug_split_style == 'debug-file-directory':
> +rootfs_dbg_solib_search_path = ["/usr/lib/debug"]
> +else:
> +logger.warning(
> +"Cannot find solib search path for a rootfs built with
> PACKAGE_DEBUG_SPLIT_STYLE=%s." % self.package_debug_split_style)
> +
> +sym_dirs = []
> +for dbgdir in rootfs_solib_search_path:
> +sym_dirs.append(os.path.join(
> +self.rootfs, dbgdir.lstrip('/')))
> +for dbgdir in rootfs_dbg_solib_search_path:
> +sym_dirs.append(os.path.join(
> +self.rootfs_dbg, dbgdir.lstrip('/')))
> +
> +return sym_dirs
Feels like there should be a much better way to do this. Does the choice of
debug split style actually matter? Just search all of the candidates and avoid
trying to special-case everything?
I’d not noticed image-combined-dbg existed and do wonder if that shoud be the
behaviour of the debug rootfs. Is there actually a use-case for a tarball
which is _just_ the symbols?
> +class RecipeMetaIdeSupport:
> +"""Handle some meta-ide-support recipe related properties"""
I’m obviously missing something: what’s the use-case for using meta-ide-support
instead of the recipe’s depends?
> +def solib_search_path_sysroot(self):
> +return [os.path.join(self.recipe_sysroot, p) for p in ['lib',
> 'usr/lib’]]
${base_libdir}, ${libdir}. Some systems use /usr/lib64 or /usr/lib32 and this
assumption will fail.
Ross
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188995):
https://lists.openembedded.org/g/openembedded-core/message/188995
Mute This Topic: https://lists.openembedded.org/mt/101275530/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core][PATCH v6 03/12] devtool: new ide plugin
The new devtool ide plugin configures an IDE to work with the eSDK.
With this initial implementation VSCode is the default IDE.
The plugin works for recipes inheriting the cmake or the meson bbclass.
Support for more programming languages and build tools may be added in
the future.
Using the plugin in recipe modes:
$ devtool modify a-recipe
$ devtool ide a-recipe a-image
$ code "$BUILDDIR/workspace/sources/a-recipe"
Work in VSCode, after installing the proposed plugins
Using the plugin without a recipe
$ devtool ide none a-image
vscode where/the/sources/are
Use the cross tool-chain which is provided as a cmake-kit.
The goal of this implementation is to create a configuration for VSCode
(or other IDEs) that allows to work on the code of a recipe completely
independent from bitbake. bitbake is only called if the configuration or
the whole SDK has to be regenerated. But bitbake should not need to be
called while working in the IDE. This has two major advantages over
calling devtool build from the IDE:
- The IDE provides plugins for integration with cmake, for example.
These features are usable, which would not be the case if bitbake or
devtool are called from within the IDE.
- It is much faster.
Signed-off-by: Adrian Freihofer
---
scripts/lib/devtool/ide.py | 1190
1 file changed, 1190 insertions(+)
create mode 100755 scripts/lib/devtool/ide.py
diff --git a/scripts/lib/devtool/ide.py b/scripts/lib/devtool/ide.py
new file mode 100755
index 000..7c7040232c4
--- /dev/null
+++ b/scripts/lib/devtool/ide.py
@@ -0,0 +1,1190 @@
+#! /usr/bin/env python3
+#
+# Copyright (C) 2023 Siemens AG
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+"""Devtool ide plugin"""
+
+import os
+import stat
+import logging
+import json
+import re
+import shutil
+from argparse import RawTextHelpFormatter
+from enum import IntEnum, auto
+
+import bb
+from devtool import exec_build_env_command, setup_tinfoil,
check_workspace_recipe, DevtoolError, parse_recipe
+from devtool.standard import get_real_srctree
+
+SHARED_SYSROOT_RECIPES = ['none', 'meta-ide-support', 'build-sysroots']
+SUPPORTED_IDES = ['code', 'none']
+
+logger = logging.getLogger('devtool')
+
+
+class TargetDevice:
+"""SSH remote login parameters"""
+
+def __init__(self, args):
+self.extraoptions = ''
+if args.no_host_check:
+self.extraoptions += '-o UserKnownHostsFile=/dev/null -o
StrictHostKeyChecking=no'
+self.ssh_sshexec = 'ssh'
+if args.ssh_exec:
+self.ssh_sshexec = args.ssh_exec
+self.ssh_port = ''
+if args.port:
+self.ssh_port = "-p %s" % args.port
+if args.key:
+self.extraoptions += ' -i %s' % args.key
+
+self.target = args.target
+target_sp = args.target.split('@')
+if len(target_sp) == 1:
+self.login = ""
+self.host = target_sp[0]
+elif len(target_sp) == 2:
+self.login = target_sp[0]
+self.host = target_sp[1]
+else:
+logger.error("Invalid target argument: %s" % args.target)
+
+
+class RecipeNative:
+def __init__(self, name, target_arch=None):
+self.name = name
+self.target_arch = target_arch
+self.bootstrap_tasks = [self.name + ':do_addto_recipe_sysroot']
+self.staging_bindir_native = None
+self.target_sys = None
+self.__native_bin = None
+
+def initialize(self, config, workspace, tinfoil):
+recipe_d = parse_recipe(
+config, tinfoil, self.name, appends=True, filter_workspace=False)
+if not recipe_d:
+raise DevtoolError("Parsing %s recipe failed" % self.name)
+self.staging_bindir_native = os.path.realpath(
+recipe_d.getVar('STAGING_BINDIR_NATIVE'))
+self.target_sys = recipe_d.getVar('TARGET_SYS')
+
+@property
+def native_bin(self):
+if not self.__native_bin:
+raise DevtoolError("native binary name is not defined.")
+return self.__native_bin
+
+
+class RecipeGdbCross(RecipeNative):
+def __init__(self, args, target_arch, target_device, gdbserver_multi=True):
+super().__init__('gdb-cross-' + target_arch, target_arch)
+self.target_device = target_device
+self.gdb = None
+self.gdbserver_port_next = int(args.gdbserver_port_start)
+self.gdbserver_multi = gdbserver_multi
+self.config_db = {}
+
+def initialize(self, config, workspace, tinfoil):
+super().initialize(config, workspace, tinfoil)
+gdb_bin = self.target_sys + '-gdb'
+gdb_path = os.path.join(
+self.staging_bindir_native, self.target_sys, gdb_bin)
+self.gdb = gdb_path
+
+@property
+def host(self):
+return self.target_device.host
+
+def __gdbserver_start_cmd(self, binary, port):
+if self.gdbserver_multi:
+gdbserver_cmd = "/usr/bin/gdbserver --multi :%s" % (
