This has been asked in the past and I did have a "mini" layer that
could be used in addition to meta-virt to allow you to get what you
need in an image fairly easily. We were going to do some work to make
this easier but I haven't looked in a while so I can't say where
things are at off the top of my head. At any rate I am just back from
some travel but I will try to take a look at this tomorrow, after
which I should be able to provide some better guidance.
Mark
On Fri, Jan 11, 2019 at 7:39 PM Muhlenkamp, Lewis
<lewis.muhlenk...@stryker.com> wrote:
>
> Hello,
>
> TLDR: How do I get docker fully functional in my openembedded linux image?
>
> I've been trying to get docker included into my image. All of my attempts
> lead to the same error messages appearing in the log file, and docker not
> starting.
>
> The error messages are
>
> === Start docker messages ===
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.414778299Z" level=error msg="Failed to built-in
> GetDriver graph btrfs /var/lib/docker"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.460695720Z" level=warning msg="Your kernel does not
> support cgroup cfs period"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.460795185Z" level=warning msg="Your kernel does not
> support cgroup cfs quotas"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.460896539Z" level=warning msg="Your kernel does not
> support cgroup cfs blkio weight"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.461255643Z" level=warning msg="Your kernel does not
> support cgroup cfs blkio throttle.read_bps_device"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.461381616Z" level=warning msg="Your kernel does not
> support cgroup cfs blkio throttle.write_bps_device"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.461503746Z" level=warning msg="Your kernel does not
> support cgroup cfs blkio throttle.read_iops_device"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.461601879Z" level=warning msg="Your kernel does not
> support cgroup cfs blkio throttle.write_iops_device"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.475747665Z" level=warning msg="Running modprobe
> bridge br_netfilter failed with message: modprobe: WARNING: Module
> br_netfilter not found in directory
> /lib/modules/4.14.78-intel-pk-standard\ninsmod
> /lib/modules/4.14.78-intel-pk-standard/kernel/net/llc.ko \ninsmod
> /lib/modules/4.14.78-intel-pk-standard/kernel/net/802/stp.ko \ninsmod
> /lib/modules/4.14.78-intel-pk-standard/kernel/net/bridge/bridge.ko \n, error:
> exit status 1"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.659844723Z" level=warning msg="Could not load
> necessary modules for IPSEC rules: Running modprobe xfrm_user failed with
> message: `modprobe: WARNING: Module xfrm_user not found in directory
> /lib/modules/4.14.78-intel-pk-standard`, error: exit status 1"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]:
> time="2019-01-10T15:56:25.662494167Z" level=warning msg="Could not load
> necessary modules for Conntrack: Running modprobe nf_conntrack_netlink failed
> with message: `modprobe: WARNING: Module nf_conntrack_netlink not found in
> directory /lib/modules/4.14.78-intel-pk-standard`, error: exit status 1"
> Jan 10 15:56:25 intel-corei7-64 dockerd[210]: failed to start daemon: Error
> initializing network controller: Error creating default "bridge" network:
> Failed to program NAT chain: Failed to inject DOCKER in PREROUTING chain:
> iptables failed: iptables -wait -t nat -A PREROUTING -m addrtype -dst-type
> LOCAL -j DOCKER: iptables: No chain/target/match by that name.
> === End docker messages ===
>
> I was using my own custom image type, but I got the same results when trying
> to build and use core-image-minimal.
>
> I tried including the
> meta-virtualization/recipes/kernel/linux/linux-yocto/docker.scc stuff in, but
> since I set MACHINE to intel-corei7-64, I copied the docker.scc and
> docker.cfg into my custom layer:
>
> meta-stryker/common/recipes-kernel/linux/linux-intel/docker.cfg
> meta-stryker/common/recipes-kernel/linux/linux-intel/kernel_baseline.scc
> meta-stryker/common/recipes-kernel/linux/linux-intel_%.bbappend
>
> That didn't seem to work either. The modules always got built. For example,
> br_netfilter.ko is built:
>
> lmuhlenkamp@c71703b3ba7d:~/build-20181213a/tmp-glibc$ find . -name
> br_netfilter.ko
> ./work/corei7-64-intel-common-oe-linux/linux-intel/4.14.78+gitAUTOINC+6a3254e7b3_56f15146cf-r0/image/lib/modules/4.14.78-intel-pk-standard/kernel/net/bridge/br_netfilter.ko
> ./work/corei7-64-intel-common-oe-linux/linux-intel/4.14.78+gitAUTOINC+6a3254e7b3_56f15146cf-r0/packages-split/kernel-module-br-netfilter-4.14.78-intel-pk-standard/lib/modules/4.14.78-intel-pk-standard/kernel/net/bridge/br_netfilter.ko
> ./work/corei7-64-intel-common-oe-linux/linux-intel/4.14.78+gitAUTOINC+6a3254e7b3_56f15146cf-r0/linux-corei7-64-intel-common-standard-build/net/bridge/br_netfilter.ko
> ./work/corei7-64-intel-common-oe-linux/linux-intel/4.14.78+gitAUTOINC+6a3254e7b3_56f15146cf-r0/package/lib/modules/4.14.78-intel-pk-standard/kernel/net/bridge/br_netfilter.ko
>
> But these modules are not included in my image. For example, if I do "find /
> -name br_netfilter.ko" on my target install, nothing is returned.
>
> My bblayers.conf is as follows:
>
> === Start conf/bblayers.conf ===
> # LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
> # changes incompatibly
> LCONF_VERSION = "7"
>
> BBPATH = "${TOPDIR}"
> BBFILES ?= ""
>
> BBLAYERS ?= " \
> /home/lmuhlenkamp/oe-core/meta \
> /home/lmuhlenkamp/meta-openembedded/meta-python \
> /home/lmuhlenkamp/meta-openembedded/meta-gnome \
> /home/lmuhlenkamp/meta-openembedded/meta-filesystems \
> /home/lmuhlenkamp/meta-openembedded/meta-oe \
> /home/lmuhlenkamp/meta-openembedded/meta-networking \
> /home/lmuhlenkamp/meta-openembedded/meta-initramfs \
> /home/lmuhlenkamp/meta-openembedded/meta-webserver \
> /home/lmuhlenkamp/meta-intel \
> /home/lmuhlenkamp/meta-virtualization \
> /home/lmuhlenkamp/meta-cloud-services \
> /home/lmuhlenkamp/meta-cloud-services/meta-openstack \
> /home/lmuhlenkamp/meta-iot-cloud \
> /home/lmuhlenkamp/meta-secure-core/meta-tpm \
> /home/lmuhlenkamp/meta-stryker/common \
> /home/lmuhlenkamp/meta-stryker/testing \
> "
> === End conf/bblayers.conf ===
>
> The customizations to my local.conf file are as follows:
>
> === Start local.conf excerpt ===
> MACHINE ?= "intel-corei7-64"
> IMAGE_FSTYPES += "live"
> NOISO = "0"
> IMAGE_INSTALL_append = " glibc-utils localedef"
> GLIBC_GENERATE_LOCALES = "el_GR.UTF-8 en_GB.UTF-8 en_US.UTF-8 es_ES.UTF-8
> de_DE.UTF-8 fa_IR fr_FR.UTF-8 hr_HR.UTF-8 ja_JP.UTF-8 ja_JP.EUC-JP
> lt_LT.UTF-8 ru_RU.UTF-8 tr_TR.UTF-8"
> IMAGE_LINGUAS = "el-gr en-gb en-us es-es de-de fa-ir fr-fr hr-hr ja-jp
> ja-jp.euc-jp lt-lt ru-ru tr-tr"
> DISTRO_FEATURES_append = " systemd virtualization"
> DISTRO_FEATURES_BACKFILL_CONSIDERED += "sysvinit"
> VIRTUAL-RUNTIME_init_manager = "systemd"
> VIRTUAL-RUNTIME_initscripts = "systemd-compat-units"
> DISTRO_FEATURES_append = " opengl"
> CORE_IMAGE_EXTRA_INSTALL += "rpm python3 python3-pip python3-flask
> python3-requests python3-coverage python3-pylint"
> CORE_IMAGE_EXTRA_INSTALL += "python-sphinx"
> CORE_IMAGE_EXTRA_INSTALL += "python-flake8"
> CORE_IMAGE_EXTRA_INSTALL += "python3-doxypypy"
> CORE_IMAGE_EXTRA_INSTALL += "trousers tpm-tools openssl-tpm-engine"
> KERNEL_FEATURES_append = " features/netfilter/netfilter.scc
> features/overlayfs/overlayfs.scc"
> KERNEL_ENABLE_CGROUPS = "1"
> CORE_IMAGE_EXTRA_INSTALL += "docker"
> SERIAL_CONSOLES = "38400 tty1"
> CORE_IMAGE_EXTRA_INSTALL += "flaskhello"
> === End local.conf excerpt ===
>
> I did not have the KERNEL_FEATURES_append line in when using the
> recipes-kernel/linux/linux-intel stuff. I didn't have the
> recipes-kernel/linux/linux-intel directory available when using the
> KERNEL_FEATURES_append line in local.conf. Based on what I read, they were
> mutually exclusive.
>
> What am I missing? Why are the kernel modules not being included in my image?
>
> I did try tweaking my docker.cfg file in my custom kernel recipe to include
> cgroups, but that did not seem to change anything. Here are the contents of
> my custom docker.cfg file
>
> === Start
> meta-stryker/common/recipes-kernel/linux/linux-intel/docker/docker.cfg ===
> CONFIG_CGROUP_DEVICE=y
>
> CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m
> CONFIG_IP_NF_FILTER=m
> CONFIG_NF_NAT=m
> CONFIG_NF_CONNTRACK_IPV4=y
> CONFIG_NF_CT_NETLINK=y
>
> CONFIG_BRIDGE_NETFILTER=m
> CONFIG_XFRM_USER=m
>
> CONFIG_DM_THIN_PROVISIONING=m
>
>
> CONFIG_IP_NF_NAT=m
> CONFIG_IP_NF_TARGET_MASQUERADE=m
>
> CONFIG_OVERLAY_FS=y
> === End
> meta-stryker/common/recipes-kernel/linux/linux-intel/docker/docker.cfg ===
>
> And for completeness, here are the contents of the other 2 files in that
> recipe
>
> === Start meta-stryker/common/recipes-kernel/linux/linux-intel_%.bbappend ===
> FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
> SRC_URI += "file://kernel_baseline.scc"
> === End meta-stryker/common/recipes-kernel/linux/linux-intel_%.bbappend ===
>
> === Start
> meta-stryker/common/recipes-kernel/linux/linux-intel/kernel_baseline.scc ===
> define KFEATURE_DESCRIPTION "Enable Features needed by docker in addition to
> LXC features"
> define KFEATURE_COMPATIBILITY board
>
> kconf non-hardware docker.cfg
> === End
> meta-stryker/common/recipes-kernel/linux/linux-intel/kernel_baseline.scc ===
>
> Any help that would allow me to get docker functionality working in my
> openembedded linux image would be greatly appreciated.
>
> Thank you
>
> Lewis Muhlenkamp
>
> --
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-devel
--
_______________________________________________
Openembedded-devel mailing list
Openembedded-devel@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-devel
