On Fri, Mar 27, 2020 at 3:29 AM Nandor Han <[email protected]> wrote:
>
> FIT format is very versatile allowing various combination of booting
> sequences. In the same time different U-Boot boot stages can use FIT
> blobs to pack various binaries (e.g. SPL supports reading U-Boot from a
> FIT blob). Because of the allowed level of customization, the generation
> of a FIT blob using a fixed image tree source, becomes challenging and
> increase the level of complexity where different configurations and
> combinations are needed.
>
> This bbclass will know how to generate a FIT blob, leaving the mechanics
> of the process (dependencies, task order...) to be handled by the users
> of the bbclass. In the same time will allow to separate the knowledge of
> the FIT format leaving the user code cleaner and more readable.
>
> Signed-off-by: Nandor Han <[email protected]>
This class looks very useful, but I did have a question. How would you
account for creating nodes dynamically? One use case of this would be
adding a DT node for each reference in KERNEL_DEVICETREE. Would that
functionality be expected to go in the recipe including this class or
the class itself?
Thanks,
Zach
> ---
>
> Notes:
> Testing
> -------
>
> 1. linux-yocto_5.4.bbappend was modified to have the following
> configuration:
>
> ```
> inherit fit-image
>
> KERNEL_IMAGE_NODE[name] = "kernel"
> KERNEL_IMAGE_NODE[description] = "${PF}"
> KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")'
> KERNEL_IMAGE_NODE[type] = "kernel"
> KERNEL_IMAGE_NODE[arch] = "${ARCH}"
> KERNEL_IMAGE_NODE[os] = "linux"
> KERNEL_IMAGE_NODE[compression] = "none"
> KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}"
> KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}"
> KERNEL_IMAGE_NODE[hash] = "sha256"
>
> FDT_IMAGE_NODE[name] = "fdt"
> FDT_IMAGE_NODE[description] = "FDT blob"
> FDT_IMAGE_NODE[data] =
> '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")'
> FDT_IMAGE_NODE[type] = "flat_dt"
> FDT_IMAGE_NODE[arch] = "${ARCH}"
> FDT_IMAGE_NODE[compression] = "none"
> FDT_IMAGE_NODE[hash] = "sha256"
>
> CONF1_CONF_NODE[name] = "conf"
> CONF1_CONF_NODE[description] = "Linux kernel and FDT blob"
> CONF1_CONF_NODE[kernel] = "kernel"
> CONF1_CONF_NODE[fdt] = "fdt"
>
> FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE"
> FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE"
> FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE',
> 'name') or ""}"
> ```
> 2. Build the kernel: `bitbake virtual/kernel`
> 3. Verify that `image-fit.itb` is present in the build directory: PASS
> 4. Disassemble the image using the command: `dtc -I dtb -O dts
> image-fit.itb`
> 5. Verify that the FIT source contains the expected configuration: PASS
>
> Changes since v1:
> ----------------
> - Change the format of short-log to "<target>: <summary>"
>
> Changes since v2:
> ----------------
> - rename the file from `fit-image` to `fit_image` to
> successfully export the class functions.
> - adding new sanity checks.
> - add missing dependency.
> - fix a variable reference in a debug log.
>
> meta/classes/fit_image.bbclass | 382 +++++++++++++++++++++++++++++++++
> 1 file changed, 382 insertions(+)
> create mode 100644 meta/classes/fit_image.bbclass
>
> diff --git a/meta/classes/fit_image.bbclass b/meta/classes/fit_image.bbclass
> new file mode 100644
> index 0000000000..87d92db122
> --- /dev/null
> +++ b/meta/classes/fit_image.bbclass
> @@ -0,0 +1,382 @@
> +#
> +# The class will facilitate the generation of FIT blobs.
> +#
> +# Glossary
> +# FIT - Flattened uImage Tree
> +#
> +# Requirements:
> +#
> +# * The user need to specify the image content using the format specified
> in the "FIT Image API" section.
> +#
> +# FIT Image API
> +#
> +# The bbclass is using variable and variable flags to declare the FIT image
> content.
> +#
> +# * Sub-Images and Configuration Nodes
> +#
> +# ** A sub-image node content is declared using the format
> `VAR_NODE[<property-name>] = <value>`.
> +# * VAR_NODE - freely selected name of the variable representing a
> node.
> +# * <property-name> - a sub-image property (e.g. description,
> type...).
> +# * <value> - the property value.
> +# Depending of the property the value can support different
> formats.
> +# ** Property Values Formats
> +#
> +# string property
> +# ---------------
> +# format: "<text>" - in case the property expects a text.
> +# (e.g. IMAGE_NODE_KERNEL[type] = "kernel")
> +#
> +# address property
> +# ----------------
> +# format: "<address>" - in case the property expects an address.
> +# (e.g. IMAGE_NODE_KERNEL[entry] = "0xABCDABCD")
> +#
> +# hash property
> +# -------------
> +# format: "<hash type>" - for hash property the hash type needs
> to be specified.
> +# (e.g. IMAGE_NODE_KERNEL[hash] = "sha256")
> +#
> +# sub-image signature property
> +# ----------------------------
> +# format: "<algo>;<key-name-hint>;" - for image signature node.
> +# Both algorithm and key name needs to be provided.
> +# (e.g. IMAGE_NODE_KERNEL[signature] =
> "sha256,rsa2048;kernel;"
> +#
> +# configuration signature property
> +# --------------------------------
> +# format: "<algo>;<key-name-hint>;<sub-image list>" - for
> configuration signature properties algorithm,
> +# key name and sub-image nodes needs to be provided.
> +# (e.g. CONF_NODE_CONF1[signature] =
> "sha256,rsa2048;kernel;"kernel","fdt";")
> +#
> +# ** Sub-Image and Configuration Nodes Flags
> +# See the code for supported flags.
> +#
> +# * FIT_IMAGES_NODE - contains a list of variables used to declare the
> sub-images nodes, separated by space.
> +# (e.g. FIT_IMAGES_NODE = "IMAGE_NODE_KERNEL
> IMAGE_NODE_FDT")
> +#
> +# * FIT_CONFIGURATIONS_NODE - contains a list of variables used to
> declare the configuration nodes,
> +# separated by space. (e.g. FIT_CONFIGURATIONS_NODE =
> "CONF_NODE_CONF1")
> +# ** Flags
> +# - "default": used to configure the default configuration node.
> +# (e.g. FIT_CONFIGURATIONS_NODE[default] = "conf@0")
> +#
> +# Example:
> +# This is part of a linux_%.bbappend recipe.
> +#
> +# KERNEL_IMAGE_NODE[name] = "kernel"
> +# KERNEL_IMAGE_NODE[description] = "${PF}"
> +# KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")'
> +# KERNEL_IMAGE_NODE[type] = "kernel"
> +# KERNEL_IMAGE_NODE[arch] = "${ARCH}"
> +# KERNEL_IMAGE_NODE[os] = "linux"
> +# KERNEL_IMAGE_NODE[compression] = "none"
> +# KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}"
> +# KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}"
> +# KERNEL_IMAGE_NODE[hash] = "sha256"
> +#
> +# FDT_IMAGE_NODE[name] = "fdt"
> +# FDT_IMAGE_NODE[description] = "FDT blob"
> +# FDT_IMAGE_NODE[data] =
> '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")'
> +# FDT_IMAGE_NODE[type] = "flat_dt"
> +# FDT_IMAGE_NODE[arch] = "${ARCH}"
> +# FDT_IMAGE_NODE[compression] = "none"
> +# FDT_IMAGE_NODE[hash] = "sha256"
> +#
> +# CONF1_CONF_NODE[name] = "conf"
> +# CONF1_CONF_NODE[description] = "Linux kernel and FDT blob"
> +# CONF1_CONF_NODE[kernel] = "kernel"
> +# CONF1_CONF_NODE[fdt] = "fdt"
> +#
> +# FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE"
> +# FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE"
> +# FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE',
> 'name') or ""}"
> +#
> +
> +DEPENDS += "\
> + dtc-native \
> + u-boot-mkimage-native \
> +"
> +
> +FIT_IMAGE_DESCRIPTION ??= "Generic FIT image"
> +FIT_IMAGE_FILENAME ??= "image-fit"
> +FIT_IMAGE_UBOOT_MKIMAGE_OPTS ??= ""
> +
> +def get_subimage_node_rules():
> + """
> + Defines the properties format and validation for sub-image nodes.
> +
> + :return: Return a dictionary with the format rules.
> + """
> +
> + #
> + # Rules Format: [Mandatory, Template String, Dictionary keys]
> + # Mandatory: True, False - used to verify if this property is mandatory
> for generating the image.
> + # Note: Doesn't take in consideration the conditionally
> mandatory property.
> + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt)
> + # Template String: Property content format. Used to generate the
> parameter content.
> + # Dictionary Keys: Keys used to be replaced in the Template String.
> + #
> + from collections import OrderedDict
> +
> + rules = OrderedDict()
> +
> + rules['description'] = [True, '= "$value"', ["value"]]
> + rules['data'] = [True, '= $value', ["value"]]
> + rules['type'] = [True, '= "$value"', ["value"]]
> + rules['arch'] = [False, '= "$value"', ["value"]]
> + rules['os'] = [False, '= "$value"', ["value"]]
> + rules['compression'] = [True, '= "$value"', ["value"]]
> + rules['load'] = [False, '= <$value>', ["value"]]
> + rules['entry'] = [False, '= <$value>', ["value"]]
> + rules['hash'] = [False, '{ algo = "$algo"; }', ['algo']]
> + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key";
> }', ['algo', 'key']]
> +
> + return rules
> +
> +
> +def get_conf_node_rules():
> + """
> + Defines the properties format and validation for configuration nodes.
> +
> + :return: Return a dictionary with the format rules.
> + """
> + #
> + # Rules Format: [Mandatory, Template String, Dictionary keys]
> + # Mandatory: True, False - used to verify if this property is mandatory
> for generating the image.
> + # Note: Doesn't take in consideration the conditionally
> mandatory property.
> + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt)
> + # Template String: Property content format. Used to generate the
> parameter content.
> + # Dictionary Keys: Keys used to be replaced in the Template String.
> + #
> + from collections import OrderedDict
> +
> + rules = OrderedDict()
> +
> + rules['description'] = [True, '= "$value"', ["value"]]
> + rules['kernel'] = [True, '= "$value"', ["value"]]
> + rules['ramdisk'] = [False, '= "$value"', ["value"]]
> + rules['fdt'] = [False, '= "$value"', ["value"]]
> + rules['loadables'] = [False, '= "$value"', ["value"]]
> + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key";
> sign-images = $images; }',
> + ['algo', 'key', 'images']]
> +
> + return rules
> +
> +
> +def generate_node(name, params, rules):
> + """
> + Generates a node.
> +
> + :param name: Node name.
> + :param params: A dictionary containing the properties values.
> + :param rules: A dictionary containing the properties values validation
> and format.
> +
> + :return: A string containing the node, including the new line characters.
> + """
> + from string import Template
> +
> + content = []
> +
> + for rule in rules.keys():
> + if rule in params.keys():
> + content.append('{param} {value}; '.format(
> + param=rule,
> + value=Template(rules[rule][1]).substitute(
> + dict(zip(rules[rule][2], params[rule].split(';'))))))
> + elif rules[rule][0]:
> + bb.fatal('Missing mandatory parameter "{param}" from "{section}"
> section'.format(param=rule, section=name))
> +
> + content = """
> + """.join(content)
> + node = """ {name} {{
> + {content}
> + }};
> + """.format(name=name, content=content)
> +
> + return node
> +
> +
> +def get_section_configuration(var, d):
> + """
> + Generates a string build from variable's flags.
> +
> + :param var: variable to extract the flags.
> + :param d: bitbake environment.
> +
> + :return: A string with the format '<flag1> = <value1>; <flag2> =
> <value2>;...'.
> + """
> + flags = d.getVarFlags(var)
> + if flags is not None:
> + flags = dict((flag, d.expand(value))
> + for flag, value in list(flags.items()))
> + else:
> + flags = {}
> +
> + configuration = ''.join((
> + """{name} = "{value}";
> + """.format(name=name, value=value) for name, value in
> flags.items()))
> +
> + return configuration
> +
> +
> +def get_section_properties(var, d):
> + """
> + Extract the nodes and parameters for a section.
> +
> + :param var: variable containing the variable names of the nodes that are
> part of this section.
> + :param d: bitbake environment.
> +
> + :return: a list containing dictionaries with section nodes parameters.
> + """
> + nodes = []
> + parameters = {}
> +
> + for node in d.getVar(var).split():
> + parameters = d.getVarFlags(node)
> + if parameters is not None:
> + parameters = dict((parameter, d.expand(value))
> + for parameter, value in
> list(parameters.items()))
> + nodes.append(parameters)
> +
> + return nodes
> +
> +
> +def generate_section(var, rules, d):
> + """
> + Generates a section node (configuration or sub-image).
> +
> + :param var: Variable to extract the node names.
> + :param rules: Rules to use for generating this section.
> + :param d: bitbake environment.
> +
> + :return: A string containing the section, including the new line
> characters.
> + """
> +
> + section = get_section_configuration(var, d)
> +
> + nodes_parameters = get_section_properties(var, d)
> + for parameters in nodes_parameters:
> + name = parameters.pop('name')
> + node = generate_node(name, parameters, rules)
> + section += node
> +
> + return section
> +
> +
> +def get_fit_image_template():
> + """
> + Get the FIT format.
> +
> + :return: A Template string containing the FIT image format.
> + """
> + from string import Template
> +
> + template = Template("""/dts-v1/;
> + /{
> + description = "$description";
> + #address-cells = <1>;
> + images {
> + $images_section
> + };
> + configurations {
> + $configurations_section
> + };
> + };""")
> + return template
> +
> +
> +def generate_image_tree_source(d):
> + """
> + Generates a string containing the image tree source.
> +
> + :return: A string representing the image tree.
> + """
> + from string import Template
> +
> + values = {}
> + values['description'] = d.getVar('FIT_IMAGE_DESCRIPTION')
> +
> + image_rules = get_subimage_node_rules()
> + if d.getVar('FIT_IMAGES_NODE', False) is None:
> + bb.fatal("Please add the FIT image nodes to FIT_IMAGES_NODE
> variable.")
> + values['images_section'] = generate_section('FIT_IMAGES_NODE',
> image_rules, d)
> +
> + conf_rules = get_conf_node_rules()
> + if d.getVar('FIT_CONFIGURATIONS_NODE', False) is None:
> + bb.fatal("Please add the FIT configuration nodes to
> FIT_CONFIGURATIONS_NODE variable.")
> + values['configurations_section'] =
> generate_section('FIT_CONFIGURATIONS_NODE', conf_rules, d)
> +
> + image_tree_source = get_fit_image_template().substitute(values)
> +
> + return image_tree_source
> +
> +
> +def generate_image_blob(file_name, image, d):
> + """
> + Generates a FIT blob.
> +
> + :param file_name: FIT blob file name.
> + :param image: String containing the image tree source.
> + :param d: Bitbake environment.
> + """
> + import tempfile
> + import subprocess
> +
> + bb.debug(1, "Generated FIT source is:\n {image}".format(image=image))
> +
> + builddir = d.getVar('B')
> + blob_file_name = file_name + '.itb'
> +
> + try:
> + fd, faux = tempfile.mkstemp(dir=builddir, prefix=file_name,
> suffix=".its")
> + with os.fdopen(fd, "w") as f:
> + f.write(image)
> +
> + mkimage_opts = d.getVar('FIT_IMAGE_UBOOT_MKIMAGE_OPTS').split() or ""
> + cmd = ['mkimage', '-f', faux]
> + cmd.extend(mkimage_opts)
> + cmd.append('{output_name}'.format(output_name=blob_file_name))
> +
> + ret = subprocess.run(
> + cmd,
> + check=True,
> + universal_newlines=True,
> + cwd=builddir,
> + stderr=subprocess.STDOUT,
> + stdout=subprocess.PIPE)
> +
> + bb.debug(1, "Command for generating the FIT blob is:\n
> {cmd}".format(cmd=" ".join(ret.args)))
> +
> + except subprocess.CalledProcessError as e:
> + bb.fatal('Failed to generate the FIT blob: {message}:
> {output}'.format(
> + message=str(e), output=e.stdout))
> + finally:
> + os.remove(faux)
> +
> +
> +def generate_fit_image(file_name, d):
> + """
> + Create and generate FIT blob.
> +
> + :param file_name: FIT blob file name.
> + :param d: Bitbake environment.
> + """
> + image = generate_image_tree_source(d)
> + generate_image_blob(file_name, image, d)
> +
> +
> +python fit_image_do_generate_fit_image() {
> + generate_fit_image(d.getVar('FIT_IMAGE_FILENAME'), d)
> +}
> +
> +do_generate_fit_image[vardeps] += " \
> + ${FIT_CONFIGURATIONS_NODE} \
> + ${FIT_IMAGES_NODE} \
> + FIT_CONFIGURATIONS_NODE \
> + FIT_IMAGES_NODE \
> + FIT_IMAGE_FILENAME \
> +"
> +
> +addtask do_generate_fit_image after do_compile before do_deploy
> +
> +EXPORT_FUNCTIONS do_generate_fit_image
> --
> 2.24.1
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#136798):
https://lists.openembedded.org/g/openembedded-core/message/136798
Mute This Topic: https://lists.openembedded.org/mt/72583032/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
