On Wed, 2023-11-01 at 12:01 +0100, Adrian Freihofer wrote: > The new devtool ide plugin configures an IDE to work with the eSDK. > > With this initial implementation VSCode is the default IDE. > The plugin works for recipes inheriting the cmake or the meson bbclass. > Support for more programming languages and build tools may be added in > the future. > > Using the plugin in recipe modes: > $ devtool modify a-recipe > $ devtool ide a-recipe a-image > $ code "$BUILDDIR/workspace/sources/a-recipe" > Work in VSCode, after installing the proposed plugins > > Using the plugin without a recipe > $ devtool ide none a-image > vscode where/the/sources/are > Use the cross tool-chain which is provided as a cmake-kit.
Firstly, we should think carefully about the namespace. "ide" is in this patch and I've heard talk about "esdk" elsewhere. The plugin makes some assumptions since it is effectively behaving more like an SDK than a full build environment. I did wonder if "ide-sdk" might be the most appropriate? Somehow we need to ensure users to be aware this uses a single shared sysroot which is quite different to a specific single recipe environment. > The goal of this implementation is to create a configuration for VSCode > (or other IDEs) that allows to work on the code of a recipe completely > independent from bitbake. bitbake is only called if the configuration or > the whole SDK has to be regenerated. But bitbake should not need to be > called while working in the IDE. This has two major advantages over > calling devtool build from the IDE: > - The IDE provides plugins for integration with cmake, for example. > These features are usable, which would not be the case if bitbake or > devtool are called from within the IDE. > - It is much faster. > > Many thanks to Enguerrand de Ribaucourt for testing and bug fixing. > > Signed-off-by: Adrian Freihofer <[email protected]> The main thing worrying me about this plugin/code is the code effectively copying and emulating bitbake: > + def gen_fakeroot_install_script(self): > + """Generate a helper script to execute make install with pseudo > + > + For the deployment to the target device the do_install task must be > + executed out of the IDE as well. This function generates a script > which > + runs the run.do_install script from bitbake under pseudo so that it > picks > + up the appropriate file permissions. If the intent is to keep > forking off bits of code which are too slow. Generating a self-contained > script > + is much quicker than calling bitbake or devtool build from an IDE. > + """ > + cmd_lines = ['#!/bin/sh'] > + # Ensure the do compile step gets always executed without pseuso > before do install > + # Running do_compile always without pseudo is probably better than > trying to have > + # all the paths referred by compiling added to PSEUDO_IGNORE_PATHS. > + if self.cmd_compile: > + cmd_compile = "( cd %s && %s)" % ( > + self.real_srctree, self.cmd_compile) > + cmd_lines.append(cmd_compile) > + if not os.access(self.fakerootcmd, os.X_OK): > + raise DevtoolError( > + "pseudo executable %s could not be found" % self.fakerootcmd) > + run_do_install = os.path.join(self.workdir, 'temp', 'run.do_install') > + > + if not os.access(run_do_install, os.X_OK): > + raise DevtoolError( > + "run script does not exists: %s" % run_do_install) > + > + # Set up the appropriate environment > + newenv = dict(os.environ) > + for varvalue in self.fakerootenv.split(): > + if '=' in varvalue: > + splitval = varvalue.split('=', 1) > + newenv[splitval[0]] = splitval[1] > + > + # Replicate the environment variables from bitbake > + for var, val in newenv.items(): > + cmd_lines.append('export %s="%s"' % (var, val)) > + > + # Setup the task environment as bitbake would do it based on the > varFlags > + for d in self.f_do_install_cleandirs: > + cmd_lines.append('%s rm -rf %s' % (self.fakerootcmd, d)) > + for d in self.f_do_install_dirs: > + cmd_lines.append('%s mkdir -p %s' % (self.fakerootcmd, d)) > + if len(self.f_do_install_dirs) > 0: > + cmd = "cd %s" % self.f_do_install_dirs[-1] > + cmd_lines.append('%s || { "%s failed"; exit 1; }' % (cmd, cmd)) > + > + # Remove the package* folders from TMPDIR. These folders might > contain the sources for the -src packages. > + # This likely breaks pseudo like: > + # path mismatch [3 links]: ino 79147802 db > + # > .../build/tmp/.../cmake-example/1.0/package/usr/src/debug/cmake-example/1.0-r0/oe-local-files/cpp-example-lib.cpp > + # > .../build/workspace/sources/cmake-example/oe-local-files/cpp-example-lib.cpp > + # Since the files are anyway outdated lets deleted them (also from > pseudo's db) to workaround this issue. > + pkg_dirs = ' '.join([os.path.join(self.workdir, d) for d in [ > + "package", "packages-split", "pkgdata", > "sstate-install-package", "debugsources.list", "*.spec"]]) > + cmd = "%s rm -rf %s" % (self.fakerootcmd, pkg_dirs) > + cmd_lines.append('%s || { "%s failed"; exit 1; }' % (cmd, cmd)) > + > + # Finally call run.do_install on pseudo > + cmd = "%s %s" % (self.fakerootcmd, run_do_install) > + cmd_lines.append('%s || { "%s failed"; exit 1; }' % (cmd, cmd)) > + > + return self.write_script(cmd_lines, 'bb_run_do_install') > + > + def gen_deploy_target_script(self, args): > + """Generate a script which does what devtool deploy-target does > + > + This script is much quicker than devtool target-deploy. Because it > + does not need to start a bitbake server. All information from tinfoil > + is hard-coded in the generated script. > + """ > + cmd_lines = ['#!/usr/bin/env python3'] > + cmd_lines.append('import sys') > + cmd_lines.append('devtool_sys_path = %s' % str(sys.path)) > + cmd_lines.append('devtool_sys_path.reverse()') > + cmd_lines.append('for p in devtool_sys_path:') > + cmd_lines.append(' if p not in sys.path:') > + cmd_lines.append(' sys.path.insert(0, p)') > + cmd_lines.append('from devtool.deploy import deploy_cached') > + args_filter = ['debug', 'dry_run', 'key', 'no_check_space', > 'no_host_check', > + 'no_preserve', 'port', 'show_status', 'ssh_exec', > 'strip', 'target'] > + filtered_args_dict = {key: value for key, value in vars( > + args).items() if key in args_filter} > + cmd_lines.append('filtered_args_dict = %s' % str(filtered_args_dict)) > + cmd_lines.append('class Dict2Class(object):') > + cmd_lines.append(' def __init__(self, my_dict):') > + cmd_lines.append(' for key in my_dict:') > + cmd_lines.append(' setattr(self, key, my_dict[key])') > + cmd_lines.append('filtered_args = Dict2Class(filtered_args_dict)') > + cmd_lines.append( > + 'setattr(filtered_args, "recipename", "%s")' % self.bpn) > + cmd_lines.append('deploy_cached("%s", "%s", "%s", "%s", "%s", "%s", > %d, "%s", "%s", filtered_args)' % > + (self.d, self.work If the intent is to keep forking > off bits of code which are too slow.dir, self.path, self.strip_cmd, > + self.libdir, self.base_libdir, self.max_process, > + self.fakerootcmd, self.fakerootenv)) > + return self.write_script(cmd_lines, 'deploy_target') > + > + def gen_install_deploy_script(self, args): > + """Generate a script which does install and deploy""" > + cmd_lines = ['#!/bin/sh -e'] > + cmd_lines.append(self.gen_fakeroot_install_script()) > + cmd_lines.append(self.gen_deploy_target_script(args)) > + return self.write_script(cmd_lines, 'install_and_deploy') > + > + def write_script(self, cmd_lines, script_name): > + bb.utils.mkdirhier(self.temp_dir) > + script_name_arch = script_name + '_' + self.recipe_id > + script_file = os.path.join(self.temp_dir, script_name_arch) > + with open(script_file, 'w') as script_f: > + script_f.write(os.linesep.join(cmd_lines)) > + st = os.stat(script_file) > + os.chmod(script_file, st.st_mode | stat.S_IEXEC) > + return script_file You say devtool is too slow as it has to start a bitbake server. Are there ways we could get bitbake to behave in a fast enough way to make this work well enough for the IDE so we don't have to duplicate this? There are a few things which can be done: a) memory resident bitbake means the start/stop of the server is no longer needed b) bitbake -b specifying a specific recipe is extremely fast as it bypasses any need to parse multiple recipes c) could a tinfoil client/daemon persist as a helper for the plugin in the background to avoid having to fork these bits of code? I suspect a lot comes down to intent with this code. If the intent is that this is "done" and once merged, will stay like this or even duplicate more bits as needed in future, I don't think I'm prepared to take it. If we document clearly that this approach is a WIP and ultimately we want to use something directly with bitbake, there are probably paths to merging it. I do want to have a clear understanding on what the ultimate goal and code would look like. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#190220): https://lists.openembedded.org/g/openembedded-core/message/190220 Mute This Topic: https://lists.openembedded.org/mt/102316030/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
