Re: [OE-core] [PATCH 0/1] scripts: Add cleanup-downloads-dir tool
Hi Ross, On Mon, Jun 15, 2015 at 11:38:01PM +0100, Burton, Ross wrote: On 12 June 2015 at 13:15, Laurentiu Palcu laurentiu.pa...@intel.com wrote: A few days ago I noticed I had no space on disk and decided to make some space. Since my downloads directory was never deleted since denzil, it consumed 25GB of space which I needed. However, I wanted to delete only old and unneeded tarbals/git repos and keep the ones that my current Yocto version needed, so I wrote this tool. Something is not quite right, after running this and causing a rebuild I got this: ERROR: No checksum specified for /home/ross/Yocto/downloads/git2_git.lttng.org.lttng-modules.git.tar.gz, please add at least one to the recipe: SRC_URI[md5sum] = c32ceb7d4b208b2b1d8132c3bbc7d638 SRC_URI[sha256sum] = d593cca435003770e981c8ae5def3db0e95d72bc3f83f3d43a18287ea1f6d165 ERROR: Function failed: Fetcher failure for URL: 'http://downloads.yoctoproject.org/mirror/sources/ git2_git.lttng.org.lttng-modules.git.tar.gz'. Missing SRC_URI checksum A few too many files pruned? Thanks for giving this a test. However, I have no idea why this happens for you... I ran it on my side, after compiling lttng-modules (so it fetches the latest, etc), and the tarballs were not removed. Actually, nothing was removed since I ran it on master, after doing a fresh core-image-minimal build... See below. Can you please try a: 'bitbake lttng-modules' followed by a 'cleanup-downloads-dir'? Do you get the same behavior on the first rebuid? Could there's something else going on at your side? test@test-machine:/ssd/work/yp1/test$ bitbake lttng-modules ... build output skipped ... test@test-machine:/ssd/work/yp1/test$ cleanup-downloads-dir Parsing recipes..done. Generating unused files list... done No obsolete files found in /ssd/work/yp1/test/downloads. test@test-machine:/ssd/work/yp1/test$ ls -la downloads/ |grep lttng-modules -rw-rw-r-- 1 test test 1861595 May 13 11:26 git2_git.lttng.org.lttng-modules.git.tar.gz -rw-rw-r-- 1 test test 0 Jun 16 09:45 git2_git.lttng.org.lttng-modules.git.tar.gz.done -rw-rw-r-- 1 test test 125 Jun 16 09:45 lttng-modules-replace-KERNELDIR-with-KERNEL_SRC.patch.done test@test-machine:/ssd/work/yp1/test$ ll downloads/git2 |grep lttng-modules drwxr-xr-x 7 test test 4096 May 13 11:26 git.lttng.org.lttng-modules.git/ -rw-rw-r-- 1 test test 6 Jun 16 09:45 git.lttng.org.lttng-modules.git.done laurentiu -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] scripts: Add cleanup-downloads-dir tool
Hi all, A few days ago I noticed I had no space on disk and decided to make some space. Since my downloads directory was never deleted since denzil, it consumed 25GB of space which I needed. However, I wanted to delete only old and unneeded tarbals/git repos and keep the ones that my current Yocto version needed, so I wrote this tool. After running it, my downloads size became 7GB. I also tested on a brand new downloads directory, after doing a core-image-minimal build, and it didn't find any obsolete files (which is kind of normal). :) I hope someone finds it helpful. Also, I encourage you to give it a test. Don't worry, it will not delete anything unless you allow it and you can review the list of files to be removed. If you see anything suspicious, let me know. laurentiu Laurentiu Palcu (1): scripts: Add cleanup-downloads-dir tool scripts/cleanup-downloads-dir | 158 ++ 1 file changed, 158 insertions(+) create mode 100755 scripts/cleanup-downloads-dir -- 1.9.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/1] scripts: Add cleanup-downloads-dir tool
The tool will remove obsolete files/directories from the downloads directory(DL_DIR): tarballs belonging to old versions of a certain package, .done files that are no longer needed, git repos. This way, you don't have to delete the entire downloads directory to make some space on disk! Warning: The tool works fine if DL_DIR is not shared. If you share DL_DIR among various Yocto versions, only the files belonging to the Yocto version you run this script on will be kept. Users that have the same downloads directory since the early versions of Yocto, will benefit the most. :) Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- scripts/cleanup-downloads-dir | 158 ++ 1 file changed, 158 insertions(+) create mode 100755 scripts/cleanup-downloads-dir diff --git a/scripts/cleanup-downloads-dir b/scripts/cleanup-downloads-dir new file mode 100755 index 000..b48b24d --- /dev/null +++ b/scripts/cleanup-downloads-dir @@ -0,0 +1,158 @@ +#!/usr/bin/env python + +# This script will compute a list of obsolete/unused files in downloads directory +# (DL_DIR) and will wait for the user to acknowledge their removal. Also, the +# user can manually alter the contents of the list, before removal, if some of +# the directories/files are not to be removed. +# +# Author: Laurentiu Palcu laurentiu.pa...@intel.com +# +# Copyright (c) 2015 Intel Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. +# See the GNU General Public License for more details. + +import os +import sys +import tempfile +import shutil + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), lib)) +import scriptpath + +sys.path.insert(0, scriptpath.add_bitbake_lib_path()) +import bb.tinfoil +import bb.fetch2 as fetcher + + +# This function looks at the URLs and computes the list of files in DL_DIR, +# adding also the donestamp files to the list. +def compute_paths(bb_hdlr, fetcher_hdlr): +dl_dir = bb_hdlr.config_data.getVar('DL_DIR', True) +paths = [] + +for url in fetcher_hdlr.urls: +ud = fetcher_hdlr.ud[url] +ud.setup_localpath(fetcher_hdlr.d) + +if ud.localpath.startswith(dl_dir): +paths.append(ud.localpath) + +if ud.mirrortarball is not None: +if not ud.mirrortarball.startswith(dl_dir): +mirrortarball_path = os.path.join(dl_dir, ud.mirrortarball) +else: +mirrortarball_path = ud.mirrortarball + +paths.append(mirrortarball_path) +paths.append(mirrortarball_path + .done) + +paths.append(ud.donestamp) + +return paths + + +# Go through all recipe providers and return a list of files that should belong +# to DL_DIR. +def get_used_dldir_paths(bb_hdlr): +pkg_providers = bb.providers.allProviders(bb_hdlr.cooker.recipecache) +pnlist = sorted(pkg_providers) +used_paths = [] +pkgs_no = len(pnlist) +pkgs_scanned = 0 +cache = bb.cache.Cache + +for pn in pnlist: +sys.stderr.write(Computing used paths: %d%%\r % (pkgs_scanned * 100 / pkgs_no)) + +for provider in pkg_providers[pn]: +fn = provider[1] + +data = cache.loadDataFull(fn, + bb_hdlr.cooker.collection.get_file_appends(fn), + bb_hdlr.config_data) + +used_paths += compute_paths(bb_hdlr, fetcher.Fetch([], data)) + +pkgs_scanned += 1 + +# return the list with no duplicates +return list(set(used_paths)) + + +# Go through all files in DL_DIR and return those files that are not in the +# used_paths list. These files/directories are the ones that can be removed. +def get_unused_dldir_paths(bb_hdlr, used_paths): +dl_dir = bb_hdlr.config_data.getVar('DL_DIR', True) +used_dirs = set() +unused_paths = [] + +sys.stderr.write(Generating unused files list... ) +# Do a quick scan of all used files and extract the directories in which +# they are located. We'll use these when we scan the downloads directory, so +# we don't descend, to subdirectories,unnecessarily. +for up in used_paths: +used_dirs.add(os.path.dirname(up)) + +for root, dirs, files in os.walk(dl_dir): +dirs_copy = list(dirs) +for d in dirs_copy: +path = os.path.join(root, d) + +if path in used_dirs: +continue + +dirs[:] = [dr for dr in dirs if dr != d] + +if path not in used_paths: +unused_paths.append(path) + +for f in files: +path = os.path.join(root, f) + +if path
Re: [OE-core] [RFC] kernel: Enable externalsrc on kernels which instantiate kernel.bbclass
Hi Paul, On Wed, Dec 03, 2014 at 12:00:31PM +, Paul Eggleton wrote: On Monday 22 September 2014 13:04:47 Bruce Ashfield wrote: On 14-09-22 01:03 PM, Khem Raj wrote: On Mon, Sep 22, 2014 at 8:27 AM, Bruce Ashfield bruce.ashfi...@windriver.com wrote: But the reports we've been getting have been that externalsrc builds are working for kernels, and linux-yocto without the change in place, so I'm looking to reduce the patch footprint and re-submit. no, it cant work if folks were trying the usecase I have mentioned. The fix is infact for non linux-yocto kernels. I'll send out my updated patch shortly, it should fix both cases. Bruce, did you get around to doing this? At the moment I am trying to make my workflow tool (devtool modify) work with linux-yocto. I've at least got to the point where I can extract the appropriate source from the recipe, but when I use externalsrc to point to it and then try to build it, it doesn't even get past do_configure, so something must be missing (make complains about missing targets, presumably because it's running in ${B} and there's nothing in that directory). Do you expect linux-yocto + externalsrc to work at the moment in master? If so, are there any special steps I need to do with respect to preparing the external source tree that my tool might not be performing? FWIW, I'm using externalsrc to build non linux-yocto kernels but, to make it work, I had to overwrite the KERNEL_CONFIG_COMMAND: KERNEL_CONFIG_COMMAND = oe_runmake_call -C ${S} O=${B} oldnoconfig || yes '' | oe_runmake -C ${S} O=${B} oldconfig This might be the reason of your do_configure failure... I hope it helps, laurentiu -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] struct.error: unpack requires a string argument of length 32 in Yocto
Hi Wilder, On Tue, Nov 04, 2014 at 07:02:04PM +, Wilder wrote: Hi Laurentiu I am a student from Mexico, and we are using Yocto to build our images, however there is a unique error that is bothering me while trying to set up the environment to build an image with sdk, I am not so familiar with structs using python and this is the error I get: Traceback (most recent call last): File /media/wilder/Seagate_Backup_Plus_Drive/ITESM/MSE/te5009/opt/poky/1.6.1/relocate_sdk.py, line 231, in module change_interpreter(e) File /media/wilder/Seagate_Backup_Plus_Drive/ITESM/MSE/te5009/opt/poky/1.6.1/relocate_sdk.py, line 89, in change_interpreter p_memsz, p_flags, p_align = struct.unpack(ph_fmt, ph_hdr) struct.error: unpack requires a string argument of length 32 I am using a Chromebook with Ubuntu 14 installed on it and it seems that this error is not showing up in other computers from my classmates, At the beginning I thought the issue was because of the python version, since by default it uses Python 3, but now I have python 2.7.6 and the same error message is shown. It's hard to tell what the real issue is. But, you'd get this kind of error if the binary being relocated is corrupt. The script reads the ELF header first and then goes through the program header table to find the INTERP section. Apparently, in your case, the size of e_phentsize (which is taken from the ELF header) is different from the actual size of one program header entry for a 32bit binary. But, you could easily find out which is the problematic binary: 1. Run the SDK installer with -S -R 2. go to the install directory and change the relocate_sdk.py to print the name of the binary being relocated 3. execute ./relocate_sdk.sh After you find the binary, use readelf to inspect it. Could you please point me to a forum where I can ask for help_ You can always find help on the OE-core mailing list: openembedded-core@lists.openembedded.org (CCed) You can subscribe here: http://lists.openembedded.org/mailman/listinfo/openembedded-core laurentiu Thanks in advance, Wilder -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/1] package_manager.py: fix rpm based do_rootfs failed while IMAGE_INSTALL_append = python3
On Fri, Oct 31, 2014 at 03:05:58PM +0800, Hongxu Jia wrote: While a pkg name (such as python3) not existed in variable PACKAGES, but provided by another pkg (such as python-core), in this situation, rpm based do_rootfs could not search it in the feeds. The fix is to invoke 'smart query --provides' to do the search if the feeds search failed. We don't drop the feeds search, because the smart query search is too slow, and the feeds search is fast and could work well with most pkgs. [YOCTO #6918] Signed-off-by: Hongxu Jia hongxu@windriver.com --- meta/lib/oe/package_manager.py | 21 + 1 file changed, 21 insertions(+) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index ffb83b2..d5704e6 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -671,6 +671,23 @@ class RpmPM(PackageManager): return +def _search_pkg_provider_in_smart(self, pkg, feed_archs): +output = self._invoke_smart('query --provides %s' % pkg) or +for provider in output.split('\n'): +# Filter out the line which is empty or start with space +if provider.strip() == '' or provider[0] == ' ': +continue + +for arch in feed_archs: +arch = '@' + arch.replace('-', '_') +if arch in provider: +# First found is best match +# bb.note(%s - %s % (pkg, provider)) Why not remove the bb.note above since it's commented out? +return provider + +return '' + + ''' Translate the OE multilib format names to the RPM/Smart format names It searched the RPM/Smart format names in probable multilib feeds first, @@ -693,6 +710,8 @@ class RpmPM(PackageManager): feed_archs = self.ml_prefix_list[mlib] new_pkg = self._search_pkg_name_in_feeds(subst, feed_archs) if not new_pkg: +new_pkg = self._search_pkg_provider_in_smart(subst, feed_archs) +if not new_pkg: This entire construct: ... new_pkg = self._search_pkg_name_in_feeds(subst, feed_archs) if not new_pkg: new_pkg = self._search_pkg_provider_in_smart(subst, feed_archs) if not new_pkg: ... could be replaced with: ... new_pkg = self._search_pkg_name_in_feeds(subst, feed_archs) or \ self._search_pkg_provider_in_smart(subst, feed_archs) if not new_pkg: ... I'd say it looks cleaner but it's a matter of personal taste. :) # Failed to translate, package not found! err_msg = '%s not found in the %s feeds (%s).\n' % \ (pkg, mlib, .join(feed_archs)) @@ -711,6 +730,8 @@ class RpmPM(PackageManager): default_archs = self.ml_prefix_list['default'] new_pkg = self._search_pkg_name_in_feeds(pkg, default_archs) if not new_pkg: +new_pkg = self._search_pkg_provider_in_smart(pkg, default_archs) +if not new_pkg: ditto laurentiu -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] image.py: Fix error in graph sorting
On Fri, Oct 24, 2014 at 09:41:10AM +0200, Pascal Bach wrote: The graph sorting algorithm for image dependencies does a look for an occurrence of a searched string instead of comparing the chunk to the searched string. This leads to the problem that ubifs is recognized as ubi aswell. This fixes this by splitting up the string into chunks. Signed-off-by: Pascal Bach pascal.b...@siemens.com Reviewed-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/image.py |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py index c9b9033..5e07187 100644 --- a/meta/lib/oe/image.py +++ b/meta/lib/oe/image.py @@ -109,7 +109,7 @@ class ImageDepGraph(object): # remove added nodes from deps_array for item in group: for node in self.graph: -if item in self.graph[node]: +if item in self.graph[node].split(): self.deps_array[node][0] -= 1 self.deps_array.pop(item, None) -- 1.7.10.4 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/8] lttng-modules: update to 2.5.0
Hi Martin, On Tue, Aug 12, 2014 at 01:33:05PM +0200, Martin Jansa wrote: Sorry I thought you're maintainer of lttng-*, it seems like Laurentiu is No worries. I just get caught in the blast when lttng blows up during kernel uprevs, and I fix the wreckage so I can send out my updates :) If we don't hear from Laurentiu, I can still do the update, so I'll hold it as a separate patch here. Ping! Someone just sent patch to upgrade lttng-ust, so I've asked him for LICENSE fix as well. It seems I missed your previous email... :/ My apologies. Regarding lttng* recipes, as of a couple of months ago, RP is the maintainer: RECIPE_MAINTAINER_pn-lttng-modules = Richard Purdie richard.pur...@linuxfoundation.org RECIPE_MAINTAINER_pn-lttng-tools = Richard Purdie richard.pur...@linuxfoundation.org RECIPE_MAINTAINER_pn-lttng-ust = Richard Purdie richard.pur...@linuxfoundation.org laurentiu -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] populate_sdk_base: add auto-completion in setup
On Tue, Jun 17, 2014 at 03:16:53PM +0200, Dennis Meier wrote: Signed-off-by: Dennis Meier meier.den...@siemens.com --- meta/classes/populate_sdk_base.bbclass |3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass index 35d837d..1182425 100644 --- a/meta/classes/populate_sdk_base.bbclass +++ b/meta/classes/populate_sdk_base.bbclass @@ -173,9 +173,8 @@ if [ $verbose = 1 ] ; then set -x fi -printf Enter target directory for SDK (default: $DEFAULT_INSTALL_DIR): if [ $target_sdk_dir = ]; then - read target_sdk_dir + read -e -p Enter target directory for SDK (default: $DEFAULT_INSTALL_DIR): target_sdk_dir [ $target_sdk_dir = ] target_sdk_dir=$DEFAULT_INSTALL_DIR else echo $target_sdk_dir Moving the above printf in the 'if' branch will make this 'echo' look strange when the target directory is provided with the -d option... I believe you can remove this 'echo' completely though. There is another message, later, informing the user where the SDK gets installed. laurentiu -- 1.7.10.4 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] relocate_sdk.py: fix lib path error
On Thu, Jun 12, 2014 at 06:14:46PM +0800, yzhu1 wrote: On 06/06/2014 08:22 PM, Khem Raj wrote: On Friday, June 6, 2014, rongqing...@windriver.com wrote: From: yzhu1 yanjun@windriver.com In centos 5.9 32bit, ld lib does not contain some flags, so ld lib is not parsed. So correct lib path is not got from ld lib. Can you explain with examples what's going on here ? Hi, Before relocate_sdk.sh is executed, it needs the parameters: path/ ld-linux-x86-64.so.2 path/dmesg.util-linux path/kill.util-linux path/ reset.util-linux . The file list (path/dmesg.util-linux path/kill.util-linux path/reset.util-linux .) is collected by grep '\(executable\|dynamically linked\)'. So dynamically linked is very import to the file ld-linux-x86-64.so.2. But in redhat 5.9(32 bit) This file is as below: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), stripped In Ubuntu 12.04 (64bit) This file is as below: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=0xbb8c5184d8d41f31093193a2ec8d3f6f10964cd2, stripped In this case, we can find in Ubuntu 12.04(64 bit), the flag dynamically linked is present, so ld-linux-x86-64.so.2 can be included in the file list. relocate_sdk.py can work according to the information from ld-linux-x86-64.so.2. But in redhat 5.9(32bit), the flag does not exist. relocate_sdk.py can not get the information from ld-linux-x86-64.so.2 since this file is not included in the file list. The same error occurs on redhat6.0(32 bit). So the direct solution to this defect is to force this file ld-linux-x86-64.so.2 exist in file list. If any problem, please feel free to let me know. This looks like a 'file' display issue. It has nothing to do with the dynamic loader itself. However, the dynamic loader is the reason you spotted the problem because it wasn't relocated and all the other dynamically linked binaries depend on it... Originally, all executable files were selected no matter if they were scripts or other non-elf files. The relocation script checked for the ELF magic number anyway and it moved on if the file was not ELF format. But, the following commit added this 'file' dependency for detecting executables: commit 65535bff09afd21b6d9f129651df6c70570c3aa0 Author: yzhu1 yanjun@windriver.com Date: Tue Nov 26 08:38:28 2013 + populate_sdk: verify executable or dynamically linked library My guess is that this commit was not the right fix for the problem it describes. In fact, I just had a better look, and I spotted the problem you were experiencing with empty files: in scripts/relocate_sdk.py, in get_arch(), we check for the ELF magic number and we do a read(16). Hence, if the file was zero sized, this read would fail... I suggest you add a check in relocate_sdk.py and make sure the file size is bigger than the e_ident size (which is 16). Then, you can safely revert 65535bff09afd21b6d9f129651df6c70570c3aa0. Also, the current patch would no longer be needed. laurentiu -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] adt_installer: run autoreconf before configuring opkg
The following changes since commit 50067e069c8cbc0d339819494f22ae7623d47aa6: bitbake: fetch2/perforce: Ensure command has a default (2014-05-11 15:26:25 +0100) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b6293_adt_installer_issue http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b6293_adt_installer_issue Laurentiu Palcu (1): adt_installer: run autoreconf before configuring opkg .../installer/adt-installer/adt_installer |1 + 1 file changed, 1 insertion(+) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/1] adt_installer: run autoreconf before configuring opkg
opkg fails to build on hosts with older autotools versions. [YOCTO #6293] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../installer/adt-installer/adt_installer |1 + 1 file changed, 1 insertion(+) diff --git a/meta/recipes-devtools/installer/adt-installer/adt_installer b/meta/recipes-devtools/installer/adt-installer/adt_installer index 1f2d039..7fc37f8 100755 --- a/meta/recipes-devtools/installer/adt-installer/adt_installer +++ b/meta/recipes-devtools/installer/adt-installer/adt_installer @@ -182,6 +182,7 @@ if [ ! -x $LOCAL_OPKG_LOC/bin/opkg-cl ]; then check_result echo_info Configure opkg ...\n + autoreconf ./configure --prefix=$parent_folder/$LOCAL_OPKG_LOC --with-opkglibdir=$OPKG_LIBDIR --enable-shared=no --disable-curl --disable-ssl-curl --disable-gpg --disable-shave $parent_folder/$YOCTOADT_INSTALL_LOG_FILE check_result -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 00/12] package upgrades
The packages were individually compiled for x86, x86-64, arm, mips, ppc, then fired the AB nightlies for the mentioned archs. Basic sanity tests, apart from the AB ones, were performed on a core-image-sato image for qemux86. Builds finished successfully and no issues were spotted during tests. laurentiu The following changes since commit 183dac4036cb73ad6cc620002b0044d3484a398d: bitbake: Revert fetch2: Cleanup file checksum generation (2014-04-28 12:54:06 +0100) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/upgrades http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/upgrades Laurentiu Palcu (12): xf86-input-synaptics: upgrade to 1.7.4 libdrm: upgrade to 2.4.53 xserver-xorg: upgrade to 1.15.1 python-pygobject: upgrade to 2.28.3 gdb: upgrade to 7.7 xproto: upgrade to 7.0.26 xrandr: upgrade to 1.4.2 xtrans: upgrade to 1.3.4 kexec-tools: upgrade to 2.0.6 nasm: upgrade to 2.11.02 xauth: upgrade to 1.0.9 fontconfig: upgrade to 2.11.1 .../gdb/{gdb-7.6.2.inc = gdb-7.7.inc} |6 ++-- ...canadian_7.6.2.bb = gdb-cross-canadian_7.7.bb} |0 .../gdb/{gdb-cross_7.6.2.bb = gdb-cross_7.7.bb} |0 .../gdb-fix-cygwin-check-in-configure-script.patch | 38 .../gdb/{gdb_7.6.2.bb = gdb_7.7.bb} |0 .../nasm/{nasm_2.11.bb = nasm_2.11.02.bb} |4 +-- ...bject_2.27.91.bb = python-pygobject_2.28.3.bb} |5 ++- .../drm/{libdrm_2.4.52.bb = libdrm_2.4.53.bb} |4 +-- .../fontconfig/fontconfig/sysroot-arg.patch| 32 - .../{fontconfig_2.11.0.bb = fontconfig_2.11.1.bb} |8 ++--- .../xorg-app/{xauth_1.0.8.bb = xauth_1.0.9.bb}|4 +-- .../xorg-app/{xrandr_1.4.1.bb = xrandr_1.4.2.bb} |4 +-- ...tics_1.7.3.bb = xf86-input-synaptics_1.7.4.bb} |4 +-- .../xorg-lib/{xtrans_1.3.3.bb = xtrans_1.3.4.bb} |4 +-- .../{xproto_7.0.25.bb = xproto_7.0.26.bb} |4 +-- ...erver-xorg_1.15.0.bb = xserver-xorg_1.15.1.bb} |4 +-- .../{kexec-tools_2.0.4.bb = kexec-tools_2.0.6.bb} |4 +-- 17 files changed, 26 insertions(+), 99 deletions(-) rename meta/recipes-devtools/gdb/{gdb-7.6.2.inc = gdb-7.7.inc} (61%) rename meta/recipes-devtools/gdb/{gdb-cross-canadian_7.6.2.bb = gdb-cross-canadian_7.7.bb} (100%) rename meta/recipes-devtools/gdb/{gdb-cross_7.6.2.bb = gdb-cross_7.7.bb} (100%) delete mode 100644 meta/recipes-devtools/gdb/gdb/gdb-fix-cygwin-check-in-configure-script.patch rename meta/recipes-devtools/gdb/{gdb_7.6.2.bb = gdb_7.7.bb} (100%) rename meta/recipes-devtools/nasm/{nasm_2.11.bb = nasm_2.11.02.bb} (81%) rename meta/recipes-devtools/python/{python-pygobject_2.27.91.bb = python-pygobject_2.28.3.bb} (87%) rename meta/recipes-graphics/drm/{libdrm_2.4.52.bb = libdrm_2.4.53.bb} (46%) delete mode 100644 meta/recipes-graphics/fontconfig/fontconfig/sysroot-arg.patch rename meta/recipes-graphics/fontconfig/{fontconfig_2.11.0.bb = fontconfig_2.11.1.bb} (83%) rename meta/recipes-graphics/xorg-app/{xauth_1.0.8.bb = xauth_1.0.9.bb} (68%) rename meta/recipes-graphics/xorg-app/{xrandr_1.4.1.bb = xrandr_1.4.2.bb} (72%) rename meta/recipes-graphics/xorg-driver/{xf86-input-synaptics_1.7.3.bb = xf86-input-synaptics_1.7.4.bb} (81%) rename meta/recipes-graphics/xorg-lib/{xtrans_1.3.3.bb = xtrans_1.3.4.bb} (82%) rename meta/recipes-graphics/xorg-proto/{xproto_7.0.25.bb = xproto_7.0.26.bb} (74%) rename meta/recipes-graphics/xorg-xserver/{xserver-xorg_1.15.0.bb = xserver-xorg_1.15.1.bb} (88%) rename meta/recipes-kernel/kexec/{kexec-tools_2.0.4.bb = kexec-tools_2.0.6.bb} (57%) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 02/12] libdrm: upgrade to 2.4.53
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../drm/{libdrm_2.4.52.bb = libdrm_2.4.53.bb} |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename meta/recipes-graphics/drm/{libdrm_2.4.52.bb = libdrm_2.4.53.bb} (46%) diff --git a/meta/recipes-graphics/drm/libdrm_2.4.52.bb b/meta/recipes-graphics/drm/libdrm_2.4.53.bb similarity index 46% rename from meta/recipes-graphics/drm/libdrm_2.4.52.bb rename to meta/recipes-graphics/drm/libdrm_2.4.53.bb index 75b1846..323bef5 100644 --- a/meta/recipes-graphics/drm/libdrm_2.4.52.bb +++ b/meta/recipes-graphics/drm/libdrm_2.4.53.bb @@ -3,6 +3,6 @@ require libdrm.inc SRC_URI += file://installtests.patch \ file://GNU_SOURCE_definition.patch \ -SRC_URI[md5sum] = 01b75624a5da3a7543923e54c3547a24 -SRC_URI[sha256sum] = fa693c2f1f61befcefbdcc396673e38481110bac9db610afa4b8afb2be0218c1 +SRC_URI[md5sum] = 342886a137ddd9ed4341675d132388ad +SRC_URI[sha256sum] = 1b0c28fd2f2b92d2df0a73d1aed88f43cb0dee1267aea6bc52ccb5fca5757a08 -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 01/12] xf86-input-synaptics: upgrade to 1.7.4
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- ...tics_1.7.3.bb = xf86-input-synaptics_1.7.4.bb} |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename meta/recipes-graphics/xorg-driver/{xf86-input-synaptics_1.7.3.bb = xf86-input-synaptics_1.7.4.bb} (81%) diff --git a/meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.3.bb b/meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.4.bb similarity index 81% rename from meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.3.bb rename to meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.4.bb index 89ae45c..8045790 100644 --- a/meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.3.bb +++ b/meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.4.bb @@ -12,8 +12,8 @@ advanced features of the touchpad to become available. LIC_FILES_CHKSUM = file://COPYING;md5=55aacd3535a741824955c5eb8f061398 -SRC_URI[md5sum] = 74c83e6cb53a0e15bcbe7cc73d63d2a1 -SRC_URI[sha256sum] = 8b2a972043961195d056b84346317ec42bfa029095c9ee7aaf6deceba12e32d5 +SRC_URI[md5sum] = deaa740072c19fef8e2fb1d7787392b7 +SRC_URI[sha256sum] = 56a2d2df7bd39e29f56102c62f153e023f3e9b2f5e255309d33fab8e81945af7 DEPENDS += libxi mtdev libxtst -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 04/12] python-pygobject: upgrade to 2.28.3
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- ...bject_2.27.91.bb = python-pygobject_2.28.3.bb} |5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) rename meta/recipes-devtools/python/{python-pygobject_2.27.91.bb = python-pygobject_2.28.3.bb} (87%) diff --git a/meta/recipes-devtools/python/python-pygobject_2.27.91.bb b/meta/recipes-devtools/python/python-pygobject_2.28.3.bb similarity index 87% rename from meta/recipes-devtools/python/python-pygobject_2.27.91.bb rename to meta/recipes-devtools/python/python-pygobject_2.28.3.bb index 49ee108..9dbe47c 100644 --- a/meta/recipes-devtools/python/python-pygobject_2.27.91.bb +++ b/meta/recipes-devtools/python/python-pygobject_2.28.3.bb @@ -5,7 +5,6 @@ LIC_FILES_CHKSUM = file://COPYING;md5=a916467b91076e631dd8edb7424769c7 DEPENDS = python python-pygobject-native glib-2.0 DEPENDS_class-native = python-native glib-2.0-native RDEPENDS_class-native = -PR = r6 MAJ_VER = ${@d.getVar('PV',1).split('.')[0]}.${@d.getVar('PV',1).split('.')[1]} @@ -13,8 +12,8 @@ SRC_URI = ${GNOME_MIRROR}/pygobject/${MAJ_VER}/pygobject-${PV}.tar.bz2 \ file://obsolete_automake_macros.patch \ -SRC_URI[md5sum] = 2b11a3050264721aac83188224b093a8 -SRC_URI[sha256sum] = a1dffbe2a8e0d490594554ed8d06f0ee4a371acb6c210e7f35158e9ae77e0df4 +SRC_URI[md5sum] = aa64900b274c4661a5c32e52922977f9 +SRC_URI[sha256sum] = 7da88c169a56efccc516cebd9237da3fe518a343095a664607b368fe21df95b6 S = ${WORKDIR}/pygobject-${PV} FILESPATH = ${FILE_DIRNAME}/python-pygobject:${FILE_DIRNAME}/files -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 05/12] gdb: upgrade to 7.7
Removed the following patch(es): * gdb-fix-cygwin-check-in-configure-script.patch (changes included in release) Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../gdb/{gdb-7.6.2.inc = gdb-7.7.inc} |6 ++-- ...canadian_7.6.2.bb = gdb-cross-canadian_7.7.bb} |0 .../gdb/{gdb-cross_7.6.2.bb = gdb-cross_7.7.bb} |0 .../gdb-fix-cygwin-check-in-configure-script.patch | 38 .../gdb/{gdb_7.6.2.bb = gdb_7.7.bb} |0 5 files changed, 2 insertions(+), 42 deletions(-) rename meta/recipes-devtools/gdb/{gdb-7.6.2.inc = gdb-7.7.inc} (61%) rename meta/recipes-devtools/gdb/{gdb-cross-canadian_7.6.2.bb = gdb-cross-canadian_7.7.bb} (100%) rename meta/recipes-devtools/gdb/{gdb-cross_7.6.2.bb = gdb-cross_7.7.bb} (100%) delete mode 100644 meta/recipes-devtools/gdb/gdb/gdb-fix-cygwin-check-in-configure-script.patch rename meta/recipes-devtools/gdb/{gdb_7.6.2.bb = gdb_7.7.bb} (100%) diff --git a/meta/recipes-devtools/gdb/gdb-7.6.2.inc b/meta/recipes-devtools/gdb/gdb-7.7.inc similarity index 61% rename from meta/recipes-devtools/gdb/gdb-7.6.2.inc rename to meta/recipes-devtools/gdb/gdb-7.7.inc index ea69b39..62adcaa 100644 --- a/meta/recipes-devtools/gdb/gdb-7.6.2.inc +++ b/meta/recipes-devtools/gdb/gdb-7.7.inc @@ -4,9 +4,7 @@ LIC_FILES_CHKSUM = file://COPYING;md5=59530bdf33659b29e73d4adb9f9f6552 \ file://COPYING3.LIB;md5=6a6a8e020838b23406c81b19c1d46df6 \ file://COPYING.LIB;md5=9f604d8a4f8e74f4f5140845a21b6674 -SRC_URI += file://gdb-fix-cygwin-check-in-configure-script.patch - S = ${WORKDIR}/${BPN}-${PV} -SRC_URI[md5sum] = 9ebf09fa76e4ca6034157086e9b20348 -SRC_URI[sha256sum] = 17f7bcda1b24336aadcb1c5e703c31e59467e221c6f4353b720dffad73a7 +SRC_URI[md5sum] = 40051ff95b39bd57b14b1809e2c16152 +SRC_URI[sha256sum] = 8814d98c2733639cb602b6ecd8d69e02498017e02b5724c9451c285b0e9ee173 diff --git a/meta/recipes-devtools/gdb/gdb-cross-canadian_7.6.2.bb b/meta/recipes-devtools/gdb/gdb-cross-canadian_7.7.bb similarity index 100% rename from meta/recipes-devtools/gdb/gdb-cross-canadian_7.6.2.bb rename to meta/recipes-devtools/gdb/gdb-cross-canadian_7.7.bb diff --git a/meta/recipes-devtools/gdb/gdb-cross_7.6.2.bb b/meta/recipes-devtools/gdb/gdb-cross_7.7.bb similarity index 100% rename from meta/recipes-devtools/gdb/gdb-cross_7.6.2.bb rename to meta/recipes-devtools/gdb/gdb-cross_7.7.bb diff --git a/meta/recipes-devtools/gdb/gdb/gdb-fix-cygwin-check-in-configure-script.patch b/meta/recipes-devtools/gdb/gdb/gdb-fix-cygwin-check-in-configure-script.patch deleted file mode 100644 index 4e4647b..000 --- a/meta/recipes-devtools/gdb/gdb/gdb-fix-cygwin-check-in-configure-script.patch +++ /dev/null @@ -1,38 +0,0 @@ -Avoid false positives if the search pattern lose is found in path -descriptions in comments generated by the preprocessor. - -See https://sourceware.org/bugzilla/show_bug.cgi?id=16152. - gdb/configure| 2 +- - gdb/configure.ac | 2 +- - 3 files changed, 7 insertions(+), 2 deletions(-) - -diff --git a/gdb/configure b/gdb/configure -index 5514b2f..b38e183 100755 a/gdb/configure -+++ b/gdb/configure -@@ -12446,7 +12446,7 @@ lose - #endif - _ACEOF - if (eval $ac_cpp conftest.$ac_ext) 25 | -- $EGREP lose /dev/null 21; then : -+ $EGREP ^lose$ /dev/null 21; then : - gdb_cv_os_cygwin=yes - else - gdb_cv_os_cygwin=no -diff --git a/gdb/configure.ac b/gdb/configure.ac -index 9b73887..2947293 100644 a/gdb/configure.ac -+++ b/gdb/configure.ac -@@ -1877,7 +1877,7 @@ AC_SUBST(WERROR_CFLAGS) - - # In the Cygwin environment, we need some additional flags. - AC_CACHE_CHECK([for cygwin], gdb_cv_os_cygwin, --[AC_EGREP_CPP(lose, [ -+[AC_EGREP_CPP(^lose$, [ - #if defined (__CYGWIN__) || defined (__CYGWIN32__) - lose - #endif],[gdb_cv_os_cygwin=yes],[gdb_cv_os_cygwin=no])]) --- -1.8.4 - diff --git a/meta/recipes-devtools/gdb/gdb_7.6.2.bb b/meta/recipes-devtools/gdb/gdb_7.7.bb similarity index 100% rename from meta/recipes-devtools/gdb/gdb_7.6.2.bb rename to meta/recipes-devtools/gdb/gdb_7.7.bb -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 07/12] xrandr: upgrade to 1.4.2
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../xorg-app/{xrandr_1.4.1.bb = xrandr_1.4.2.bb} |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename meta/recipes-graphics/xorg-app/{xrandr_1.4.1.bb = xrandr_1.4.2.bb} (72%) diff --git a/meta/recipes-graphics/xorg-app/xrandr_1.4.1.bb b/meta/recipes-graphics/xorg-app/xrandr_1.4.2.bb similarity index 72% rename from meta/recipes-graphics/xorg-app/xrandr_1.4.1.bb rename to meta/recipes-graphics/xorg-app/xrandr_1.4.2.bb index 6ca6863..84f01b0 100644 --- a/meta/recipes-graphics/xorg-app/xrandr_1.4.1.bb +++ b/meta/recipes-graphics/xorg-app/xrandr_1.4.2.bb @@ -11,5 +11,5 @@ LIC_FILES_CHKSUM = file://COPYING;md5=fe1608bdb33cf8c62a4438f7d34679b3 DEPENDS += libxrandr libxrender PE = 1 -SRC_URI[md5sum] = 52c3de0297bf45be6a189dc2e0515638 -SRC_URI[sha256sum] = 67b554ab975652778bef587f86dab7fec8cb95dfd21c11d98a203dac5c241e50 +SRC_URI[md5sum] = 78fd973d9b532106f8777a3449176148 +SRC_URI[sha256sum] = b2e76ee92ff827f1c52ded7c666fe6f2704ca81cdeef882397da4e3e8ab490bc -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 03/12] xserver-xorg: upgrade to 1.15.1
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- ...erver-xorg_1.15.0.bb = xserver-xorg_1.15.1.bb} |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename meta/recipes-graphics/xorg-xserver/{xserver-xorg_1.15.0.bb = xserver-xorg_1.15.1.bb} (88%) diff --git a/meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.0.bb b/meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.1.bb similarity index 88% rename from meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.0.bb rename to meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.1.bb index 1f9fa04..6a4168e 100644 --- a/meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.0.bb +++ b/meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.1.bb @@ -8,8 +8,8 @@ SRC_URI += file://crosscompile.patch \ file://xorg-CVE-2013-6424.patch \ -SRC_URI[md5sum] = c2ace3697b32414094cf8c597c39d7d9 -SRC_URI[sha256sum] = 613b2f2e7ee2a06d2f8e862c836dc70d319c52f1537749e027398f40086aabb8 +SRC_URI[md5sum] = e4c70262ed89764be8f8f5d699ed9227 +SRC_URI[sha256sum] = 626db6882602ebe1ff81f7a4231c7ccc6ceb5032f2b5b3954bf749e1567221e2 # These extensions are now integrated into the server, so declare the migration # path for in-place upgrades. -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 06/12] xproto: upgrade to 7.0.26
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../{xproto_7.0.25.bb = xproto_7.0.26.bb} |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename meta/recipes-graphics/xorg-proto/{xproto_7.0.25.bb = xproto_7.0.26.bb} (74%) diff --git a/meta/recipes-graphics/xorg-proto/xproto_7.0.25.bb b/meta/recipes-graphics/xorg-proto/xproto_7.0.26.bb similarity index 74% rename from meta/recipes-graphics/xorg-proto/xproto_7.0.25.bb rename to meta/recipes-graphics/xorg-proto/xproto_7.0.26.bb index cf24125..0d591dc 100644 --- a/meta/recipes-graphics/xorg-proto/xproto_7.0.25.bb +++ b/meta/recipes-graphics/xorg-proto/xproto_7.0.26.bb @@ -15,6 +15,6 @@ SRC_URI += file://xproto_fix_for_x32.patch EXTRA_OECONF_append = --enable-specs=no BBCLASSEXTEND = native nativesdk -SRC_URI[md5sum] = 28311ef4edbbbf89f617a7f8a2e5648f -SRC_URI[sha256sum] = 92247485dc4ffc3611384ba84136591923da857212a7dc29f4ad7797e13909fe +SRC_URI[md5sum] = 4dc2464bfeade23dab5de38da0f6b1b5 +SRC_URI[sha256sum] = 636162c1759805a5a0114a369dffdeccb8af8c859ef6e1445f26a4e6e046514f -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 09/12] kexec-tools: upgrade to 2.0.6
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../{kexec-tools_2.0.4.bb = kexec-tools_2.0.6.bb} |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename meta/recipes-kernel/kexec/{kexec-tools_2.0.4.bb = kexec-tools_2.0.6.bb} (57%) diff --git a/meta/recipes-kernel/kexec/kexec-tools_2.0.4.bb b/meta/recipes-kernel/kexec/kexec-tools_2.0.6.bb similarity index 57% rename from meta/recipes-kernel/kexec/kexec-tools_2.0.4.bb rename to meta/recipes-kernel/kexec/kexec-tools_2.0.6.bb index de44cf0..38a8947 100644 --- a/meta/recipes-kernel/kexec/kexec-tools_2.0.4.bb +++ b/meta/recipes-kernel/kexec/kexec-tools_2.0.6.bb @@ -2,8 +2,8 @@ require kexec-tools.inc export LDFLAGS = -L${STAGING_LIBDIR} EXTRA_OECONF = --with-zlib=yes -SRC_URI[md5sum] = b9f2a3ba0ba9c78625ee7a50532500d8 -SRC_URI[sha256sum] = 6ba1872c58434b8e92506ff515c7ef64555671af54097bae51b833bda3f5126c +SRC_URI[md5sum] = c39ea40a7598e49b6dc961ee7de38f57 +SRC_URI[sha256sum] = a9c6cd9adc8c1c37b3272782f581cb8b4b4070d0e3e921a558a9083f68dcf29a PACKAGES =+ kexec kdump -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 10/12] nasm: upgrade to 2.11.02
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../nasm/{nasm_2.11.bb = nasm_2.11.02.bb} |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename meta/recipes-devtools/nasm/{nasm_2.11.bb = nasm_2.11.02.bb} (81%) diff --git a/meta/recipes-devtools/nasm/nasm_2.11.bb b/meta/recipes-devtools/nasm/nasm_2.11.02.bb similarity index 81% rename from meta/recipes-devtools/nasm/nasm_2.11.bb rename to meta/recipes-devtools/nasm/nasm_2.11.02.bb index 5f4b953..db84ec5 100644 --- a/meta/recipes-devtools/nasm/nasm_2.11.bb +++ b/meta/recipes-devtools/nasm/nasm_2.11.02.bb @@ -6,8 +6,8 @@ COMPATIBLE_HOST = '(x86_64|i.86).*-(linux|freebsd.*)' SRC_URI = http://www.nasm.us/pub/nasm/releasebuilds/${PV}/nasm-${PV}.tar.bz2 -SRC_URI[md5sum] = 4cd558047ea5ed51fc2c7c94e249c7b8 -SRC_URI[sha256sum] = 1ce7e897c67255a195367a60c739a90a0b33a4a73f058f7cda3253bcf975642b +SRC_URI[md5sum] = 3bbc8ed83115b8caf7931f35ec3bc5e0 +SRC_URI[sha256sum] = ece26b5ef565f94d19a72756d05965e424d2e5ca55f88b949852da70dd62f0e0 inherit autotools-brokensep -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 11/12] xauth: upgrade to 1.0.9
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../xorg-app/{xauth_1.0.8.bb = xauth_1.0.9.bb}|4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename meta/recipes-graphics/xorg-app/{xauth_1.0.8.bb = xauth_1.0.9.bb} (68%) diff --git a/meta/recipes-graphics/xorg-app/xauth_1.0.8.bb b/meta/recipes-graphics/xorg-app/xauth_1.0.9.bb similarity index 68% rename from meta/recipes-graphics/xorg-app/xauth_1.0.8.bb rename to meta/recipes-graphics/xorg-app/xauth_1.0.9.bb index a2da49e..b173f4f 100644 --- a/meta/recipes-graphics/xorg-app/xauth_1.0.8.bb +++ b/meta/recipes-graphics/xorg-app/xauth_1.0.9.bb @@ -8,5 +8,5 @@ LIC_FILES_CHKSUM = file://COPYING;md5=5ec74dd7ea4d10c4715a7c44f159a40b DEPENDS += libxau libxext libxmu PE = 1 -SRC_URI[md5sum] = 50ee2ec0836c0186b05ec8fdcfd566d0 -SRC_URI[sha256sum] = a8696ae7a50c699d5fb3a41408b60d98843d19ea46e9f09e391cb98c8f7fd4f7 +SRC_URI[md5sum] = 7d6003f32838d5b688e2c8a131083271 +SRC_URI[sha256sum] = 56ce1523eb48b1f8a4f4244fe1c3d8e6af1a3b7d4b0e6063582421b0b68dc28f -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 12/12] fontconfig: upgrade to 2.11.1
Removed the following patch(es): * sysroot-arg.patch (changes included in release) Correct the fccache.c license checksum and the line numbers because the license snippet has moved and, also, at the previous upgrade the lines were wrong (shifted a couple of lines). Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../fontconfig/fontconfig/sysroot-arg.patch| 32 .../{fontconfig_2.11.0.bb = fontconfig_2.11.1.bb} |8 ++--- 2 files changed, 4 insertions(+), 36 deletions(-) delete mode 100644 meta/recipes-graphics/fontconfig/fontconfig/sysroot-arg.patch rename meta/recipes-graphics/fontconfig/{fontconfig_2.11.0.bb = fontconfig_2.11.1.bb} (83%) diff --git a/meta/recipes-graphics/fontconfig/fontconfig/sysroot-arg.patch b/meta/recipes-graphics/fontconfig/fontconfig/sysroot-arg.patch deleted file mode 100644 index abfdbc7..000 --- a/meta/recipes-graphics/fontconfig/fontconfig/sysroot-arg.patch +++ /dev/null @@ -1,32 +0,0 @@ -Upstream-Status: Submitted [https://bugs.freedesktop.org/show_bug.cgi?id=72044] -Signed-off-by: Ross Burton ross.bur...@intel.com - -From f2ade764cc9f009e1fe25b856b24b7695f66a952 Mon Sep 17 00:00:00 2001 -From: Ross Burton ross.bur...@intel.com -Date: Tue, 26 Nov 2013 17:18:25 + -Subject: [PATCH] fc-cache: --sysroot option takes an argument - -The getopt_long option definitions say that sysroot doesn't take an argument, -when it in fact does. - -Signed-off-by: Ross Burton ross.bur...@intel.com - fc-cache/fc-cache.c | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/fc-cache/fc-cache.c b/fc-cache/fc-cache.c -index 27c7513..bf3b6b4 100644 a/fc-cache/fc-cache.c -+++ b/fc-cache/fc-cache.c -@@ -67,7 +67,7 @@ - const struct option longopts[] = { - {force, 0, 0, 'f'}, - {really-force, 0, 0, 'r'}, --{sysroot, 0, 0, 'y'}, -+{sysroot, required_argument, 0, 'y'}, - {system-only, 0, 0, 's'}, - {version, 0, 0, 'V'}, - {verbose, 0, 0, 'v'}, --- -1.8.4.4 - diff --git a/meta/recipes-graphics/fontconfig/fontconfig_2.11.0.bb b/meta/recipes-graphics/fontconfig/fontconfig_2.11.1.bb similarity index 83% rename from meta/recipes-graphics/fontconfig/fontconfig_2.11.0.bb rename to meta/recipes-graphics/fontconfig/fontconfig_2.11.1.bb index 48385a3..797b321 100644 --- a/meta/recipes-graphics/fontconfig/fontconfig_2.11.0.bb +++ b/meta/recipes-graphics/fontconfig/fontconfig_2.11.1.bb @@ -14,16 +14,16 @@ BUGTRACKER = https://bugs.freedesktop.org/enter_bug.cgi?product=fontconfig; LICENSE = MIT-style MIT PD LIC_FILES_CHKSUM = file://COPYING;md5=7a0449e9bc5370402a94c00204beca3d \ file://src/fcfreetype.c;endline=45;md5=5d9513e3196a1fbfdfa94051c09dfc84 \ - file://src/fccache.c;beginline=1131;endline=1146;md5=754c7b855210ee746e5f0b840fad9a9f + file://src/fccache.c;beginline=1143;endline=1158;md5=0326cfeb4a7333dd4dd25fbbc4b9f27f SECTION = libs DEPENDS = expat freetype zlib SRC_URI = http://fontconfig.org/release/fontconfig-${PV}.tar.gz \ - file://sysroot-arg.patch -SRC_URI[md5sum] = 84278204cd7f36adbea7ad8094e039ac -SRC_URI[sha256sum] = 274c047487b90dacbaa55f4d70b8cdcd556944e7251ce9cf1de442c00a16343b + +SRC_URI[md5sum] = e75e303b4f7756c2b16203a57ac87eba +SRC_URI[sha256sum] = b6b066c7dce3f436fdc0dfbae9d36122b38094f4f53bd8dffd45e195b0540d8d PACKAGES =+ fontconfig-utils FILES_${PN} =+ ${datadir}/xml/* -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 08/12] xtrans: upgrade to 1.3.4
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../xorg-lib/{xtrans_1.3.3.bb = xtrans_1.3.4.bb} |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename meta/recipes-graphics/xorg-lib/{xtrans_1.3.3.bb = xtrans_1.3.4.bb} (82%) diff --git a/meta/recipes-graphics/xorg-lib/xtrans_1.3.3.bb b/meta/recipes-graphics/xorg-lib/xtrans_1.3.4.bb similarity index 82% rename from meta/recipes-graphics/xorg-lib/xtrans_1.3.3.bb rename to meta/recipes-graphics/xorg-lib/xtrans_1.3.4.bb index 9895ea3..f2b8921 100644 --- a/meta/recipes-graphics/xorg-lib/xtrans_1.3.3.bb +++ b/meta/recipes-graphics/xorg-lib/xtrans_1.3.4.bb @@ -20,5 +20,5 @@ inherit gettext BBCLASSEXTEND = native nativesdk -SRC_URI[md5sum] = 2f14c31ab556fc91039f51a113b38aa2 -SRC_URI[sha256sum] = 622db4adce224581a44fbe41321bbb0bdc0c78aec586ba83548f1f1c6e8a09bf +SRC_URI[md5sum] = a615e17d9fee6f097fc3b716eacb3dca +SRC_URI[sha256sum] = 054d4ee3efd52508c753e9f7bc655ef185a29bd2850dd9e2fc2ccc33544f583a -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] relocate_sdk.py: Possible bug, /lib64/ld-linux-x86-64.so.2 not relocated
Hi Stefan, Can you please have a look at the binaries before relocation? Just to make sure... So, for that, run the installer with -R option: ./your_toolchain_installer.sh -R Here, I'm interested which is the default suggested path (see below): Enter target directory for SDK (default: /opt/poky/1.5+snapshot): Then, run 'readelf -p .interp' on those binaries. They should all start with the default prefix. laurentiu On Thu, Apr 03, 2014 at 09:59:34AM +0200, Stefan Agner wrote: Hi Laurentiu, Am 2014-03-03 09:38, schrieb Laurentiu Palcu: This is the correct behavior. We shouldn't relocate binaries that use host's dynamic loader. When I install the SDK with -S (copy the relocate scripts), and remove the condition around line 95, the binaries work as expected. Can you please run the installer script with -R so it doesn't perform any relocation on binaries, and then: readelf -p .interp /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc It looks like the default dynamic loader path of the toolchain binaries start with /lib* or /usr/lib* which is not quite right... It should be: ${SDKPATH}/sysroots/${SDK_SYS}. $ readelf -p .interp /usr/local/oecore-x86_64-non-reloc/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc String dump of section '.interp': [ 0] /lib64/ld-linux-x86-64.so.2 So this looks wrong then, right? I also get the same for python and qmake: $ readelf -p .interp /usr/local/oecore-x86_64-new/sysroots/x86_64-angstromsdk-linux/usr/bin/python2 String dump of section '.interp': [ 0] /lib64/ld-linux-x86-64.so.2 $ readelf -p .interp /usr/local/oecore-x86_64-new/sysroots/x86_64-angstromsdk-linux/usr/bin/qmake2 String dump of section '.interp': [ 0] /lib64/ld-linux-x86-64.so.2 How can I make sure all those binaries get linked against the SDK link loader? -- Stefan -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] image.bbclass: USE_DEVFS is now useless
Hi Matthieu, On Wed, Feb 26, 2014 at 11:11:28AM +, Matthieu CRAPET wrote: Hi, Since : http://cgit.openembedded.org/openembedded-core/commit/meta/classes/ image.bbclass?id=a83144bac8d67704ff66f5dc0fc56f5b63979694 USE_DEVFS is not considered anymore. Setting IMAGE_DEVICE_TABLE or IMAGE_DEVICE_TABLES to empty string in an image recipe works (makedevs is not called) Should be we drop USE_DEVFS variable (meta/classes/image.bbclass) or (re)add check in meta/lib/oe/rootfs.py? I can propose a patch. Please do. It looks like I totally missed the USE_DEVFS check... laurentiu Regards, Matthieu ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 0/1] package_manager.py: delete RPM db locks after calling rpmresolve
Put this on hold. I'll do some more investigations. I want to see what's the deal with those rpm lock DBs and why we don't delete them only once, after the rootfs generation has completely finished (after postprocess commands are called, etc). laurentiu On Thu, Mar 27, 2014 at 06:18:52PM +0200, Laurentiu Palcu wrote: The following changes since commit c4eeaa8e35e926b6d1f633549f76d1ba9ed9278b: bitbake: knotty: Show a link to the logfile for failed setscene tasks (2014-03-27 10:42:08 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b6049_do_rootfs_crash http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b6049_do_rootfs_crash Laurentiu Palcu (1): package_manager.py: delete RPM db locks after calling rpmresolve meta/lib/oe/package_manager.py | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] package_manager.py: leave the __db.00* files in place
Do not delete the __db.00* files in the PackageManager class. Leave this operation up to the client classes. One side effect of this deletion was the following message appearing in the output of the next rpm command executed: rpmdb: BDB1540 configured environment flags incompatible with existing environment We might also gain some time here by not deleting/creating those files very often. [YOCTO #6049] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py |8 +--- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 764ab72..a8360fe 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -299,9 +299,6 @@ class RpmPkgsList(PkgsList): # bb.note(cmd) tmp_output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).strip() -rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.rootfs_dir) -for f in rpm_db_locks: -bb.utils.remove(f, True) except subprocess.CalledProcessError as e: bb.fatal(Cannot get the installed packages list. Command '%s' returned %d:\n%s % (cmd, e.returncode, e.output)) @@ -1101,7 +1098,6 @@ class RpmPM(PackageManager): output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).strip() bb.note(output) os.chmod(saved_dir, 0755) -self._unlock_rpm_db() except subprocess.CalledProcessError as e: bb.fatal(Invoke save_rpmpostinst failed. Command '%s' returned %d:\n%s % (cmd, e.returncode, e.output)) @@ -1117,14 +1113,12 @@ class RpmPM(PackageManager): self._invoke_smart('flag --set ignore-recommends %s' % i) self._invoke_smart('channel --add rpmsys type=rpm-sys -y') -self._unlock_rpm_db() - ''' The rpm db lock files were produced after invoking rpm to query on build system, and they caused the rpm on target didn't work, so we need to unlock the rpm db by removing the lock files. ''' -def _unlock_rpm_db(self): +def unlock_rpm_db(self): # Remove rpm db lock files rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.target_rootfs) for f in rpm_db_locks: -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/2] Don't fool around with rpm __db.00* files
We were deleting/creating those files too many times during the rootfs creation process. That led to various rpmdb related issues. Instead, remove those files at the end of the rootfs creation process which seems the most sane thing to do. laurentiu The following changes since commit 8c3eb5ee4582b6f6d489549290937657f37fc19e: packagegroup-toolset-native: Update after ocf-linux - cryptodev-linux change (2014-03-27 19:53:30 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b6049_do_rootfs_crash http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b6049_do_rootfs_crash Laurentiu Palcu (2): package_manager.py: leave the __db.00* files in place rootfs.py: add new cleanup method meta/lib/oe/package_manager.py |8 +--- meta/lib/oe/rootfs.py | 25 + 2 files changed, 26 insertions(+), 7 deletions(-) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] rootfs.py: add new cleanup method
This commit adds a new _cleanup() internal method that will be called at the end of rootfs creation, so that each backend can delete various files that were probably generated during rootfs postprocess execution, etc. [YOCTO #6049] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/rootfs.py | 25 + 1 file changed, 25 insertions(+) diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py index 0e6c8bc..3eac3c9 100644 --- a/meta/lib/oe/rootfs.py +++ b/meta/lib/oe/rootfs.py @@ -50,6 +50,15 @@ class Rootfs(object): def _handle_intercept_failure(self, failed_script): pass + +The _cleanup() method should be used to clean-up stuff that we don't really +want to end up on target. For example, in the case of RPM, the DB locks. +The method is called, once, at the end of create() method. + +@abstractmethod +def _cleanup(self): +pass + def _exec_shell_cmd(self, cmd): fakerootcmd = self.d.getVar('FAKEROOT', True) if fakerootcmd is not None: @@ -117,6 +126,8 @@ class Rootfs(object): self._generate_kernel_module_deps() +self._cleanup() + def _uninstall_uneeded(self): if base_contains(IMAGE_FEATURES, package-management, True, False, self.d): @@ -358,6 +369,13 @@ class RpmRootfs(Rootfs): for pkg in registered_pkgs.split(): self.pm.save_rpmpostinst(pkg) +def _cleanup(self): +# during the execution of postprocess commands, rpm is called several +# times to get the files installed, dependencies, etc. This creates the +# __db.00* (Berkeley DB files that hold locks, rpm specific environment +# settings, etc.), that should not get into the final rootfs +self.pm.unlock_rpm_db() + class DpkgRootfs(Rootfs): def __init__(self, d, manifest_dir): @@ -431,6 +449,9 @@ class DpkgRootfs(Rootfs): def _log_check(self): pass +def _cleanup(self): +pass + class OpkgRootfs(Rootfs): def __init__(self, d, manifest_dir): @@ -694,6 +715,10 @@ class OpkgRootfs(Rootfs): def _log_check(self): pass +def _cleanup(self): +pass + + def create_rootfs(d, manifest_dir=None): env_bkp = os.environ.copy() -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] populate_sdk_base: add dependency of do_package_write_* tasks
The following changes since commit 5eceedf0326abaee632b32c7cb7ec74e6ba2d6d7: ocf-linux: remove recipe (2014-03-27 15:46:52 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b5900_fix_adt_installer_issue http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5900_fix_adt_installer_issue Laurentiu Palcu (1): populate_sdk_base: add dependency of do_package_write_* tasks meta/classes/populate_sdk_base.bbclass |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/1] populate_sdk_base: add dependency of do_package_write_* tasks
nativesdk packages were created only for the first backend listed in PACKAGE_CLASSES. Hence, if one had it set to package_rpm package_ipk and did a 'bitbake -c populate_sdk core-image-something', the nativesdk packages were created only for rpm. This is particularily bad for adt-installer which is based on opkg repos. Credits go to richard.pur...@linuxfoundation.org who suggested me this fix. [YOCTO #5900] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/populate_sdk_base.bbclass |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass index 235d672..81da206 100644 --- a/meta/classes/populate_sdk_base.bbclass +++ b/meta/classes/populate_sdk_base.bbclass @@ -333,5 +333,5 @@ populate_sdk_log_check() { do_populate_sdk[dirs] = ${TOPDIR} do_populate_sdk[depends] += ${@' '.join([x + ':do_populate_sysroot' for x in d.getVar('SDK_DEPENDS', True).split()])} do_populate_sdk[rdepends] = ${@' '.join([x + ':do_populate_sysroot' for x in d.getVar('SDK_RDEPENDS', True).split()])} -do_populate_sdk[recrdeptask] += do_packagedata +do_populate_sdk[recrdeptask] += do_packagedata do_package_write_rpm do_package_write_ipk do_package_write_deb addtask populate_sdk -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] image.py: check file exists before deleting
The following changes since commit 8c3eb5ee4582b6f6d489549290937657f37fc19e: packagegroup-toolset-native: Update after ocf-linux - cryptodev-linux change (2014-03-27 19:53:30 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b6029_ignore_broken_symlinks http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b6029_ignore_broken_symlinks Laurentiu Palcu (1): image.py: check file exists before deleting meta/lib/oe/image.py |3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/1] image.py: check file exists before deleting
When RM_OLD_IMAGE = 1, we delete old images but we didn't check they actually exist... [YOCTO #6029] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/image.py |3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py index a03b73e..c9b9033 100644 --- a/meta/lib/oe/image.py +++ b/meta/lib/oe/image.py @@ -192,7 +192,8 @@ class Image(ImageDepGraph): if img.find(self.d.getVar('IMAGE_LINK_NAME', True)) == 0: img = os.path.join(deploy_dir, img) if os.path.islink(img): -if self.d.getVar('RM_OLD_IMAGE', True) == 1: +if self.d.getVar('RM_OLD_IMAGE', True) == 1 and \ +os.path.exists(os.path.realpath(img)): os.remove(os.path.realpath(img)) os.remove(img) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/1] run-postinsts: fix issue with checking IMAGE_FEATURES
The old implementation was wrong. It was not very generic and it checked IMAGE_FEATURES while building the recipe, which led to various issues with the generation of the final script. That is, the run-postinsts script was generated once, while building the package for the first time. Hence, any other changes to IMAGE_FEATURES, like removing/adding 'package-management' did not reflect in the final script. This commit makes run-postinsts script autodetect the backend used for creating the image, making it generic. [YOCTO #5666] [YOCTO #5972] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../run-postinsts/run-postinsts/run-postinsts | 30 +++- .../run-postinsts/run-postinsts_1.0.bb |2 -- 2 files changed, 23 insertions(+), 9 deletions(-) diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts index 08cfa9e..f547a7b 100755 --- a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts +++ b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts @@ -8,9 +8,29 @@ # The following script will run all the scriptlets found in #SYSCONFDIR#/deb-postinsts, # #SYSCONFDIR#/ipk-postinsts or #SYSCONFDIR#/rpm-posinsts. -pm=#IMAGE_PKGTYPE# -pm_installed=#PM_INSTALLED# -pi_dir=#SYSCONFDIR#/${pm}-postinsts +# the order of this list is important, do not change! +backend_list=rpm deb ipk + +pm_installed=false + +for pm in $backend_list; do + pi_dir=#SYSCONFDIR#/$pm-postinsts + + [ -d $pi_dir ] break + + case $pm in + deb) + if [ -s /var/lib/dpkg/status ]; then + pm_installed=true + break + fi + ;; + + ipk) + pm_installed=true + ;; + esac +done remove_rcsd_link () { if [ -n `which update-rc.d` ]; then @@ -56,10 +76,6 @@ if $pm_installed; then deb) eval dpkg --configure -a $append_log ;; - - rpm) - exec_postinst_scriptlets - ;; esac else exec_postinst_scriptlets diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb b/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb index e990c67..64f85c2 100644 --- a/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb +++ b/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb @@ -37,8 +37,6 @@ do_install() { sed -i -e 's:#SYSCONFDIR#:${sysconfdir}:g' \ -e 's:#SBINDIR#:${sbindir}:g' \ -e 's:#BASE_BINDIR#:${base_bindir}:g' \ - -e 's:#IMAGE_PKGTYPE#:${IMAGE_PKGTYPE}:g' \ - -e 's:#PM_INSTALLED#:${@base_contains(IMAGE_FEATURES, package-management, true, false, d)}:g' \ ${D}${sbindir}/run-postinsts \ ${D}${systemd_unitdir}/system/run-postinsts.service } -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] Fix run-postinsts
The following changes since commit 0150bc30d3674301631c2e9b6c64e01058fd1070: bitbake: runqueue: Really fix sigchld handling (2014-03-18 23:05:53 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b5666_unique_run_postinsts http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5666_unique_run_postinsts Laurentiu Palcu (1): run-postinsts: fix issue with checking IMAGE_FEATURES .../run-postinsts/run-postinsts/run-postinsts | 30 +++- .../run-postinsts/run-postinsts_1.0.bb |2 -- 2 files changed, 23 insertions(+), 9 deletions(-) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] package_manager.py: delete RPM db locks after calling rpmresolve
The following changes since commit c4eeaa8e35e926b6d1f633549f76d1ba9ed9278b: bitbake: knotty: Show a link to the logfile for failed setscene tasks (2014-03-27 10:42:08 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b6049_do_rootfs_crash http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b6049_do_rootfs_crash Laurentiu Palcu (1): package_manager.py: delete RPM db locks after calling rpmresolve meta/lib/oe/package_manager.py | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/1] package_manager.py: delete RPM db locks after calling rpmresolve
If the locks are not removed, the output of the next rpm command executed will contain the following string: rpmdb: BDB1540 configured environment flags incompatible with existing environment And this will create various parsing issues. [YOCTO #6049] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 764ab72..0f22e46 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -242,6 +242,12 @@ class RpmPkgsList(PkgsList): self.ml_prefix_list, self.ml_os_list = \ RpmIndexer(d, rootfs_dir).get_ml_prefix_and_os_list(arch_var, os_var) +def _unlock_rpm_db(self): +# Remove rpm db lock files +rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.rootfs_dir) +for f in rpm_db_locks: +bb.utils.remove(f, True) + ''' Translate the RPM/Smart format names to the OE multilib format names ''' @@ -281,6 +287,8 @@ class RpmPkgsList(PkgsList): try: output = subprocess.check_output(cmd, stderr=subprocess.STDOUT).strip() + +self._unlock_rpm_db() except subprocess.CalledProcessError as e: bb.fatal(Cannot get the package dependencies. Command '%s' returned %d:\n%s % (' '.join(cmd), e.returncode, e.output)) @@ -299,9 +307,7 @@ class RpmPkgsList(PkgsList): # bb.note(cmd) tmp_output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).strip() -rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.rootfs_dir) -for f in rpm_db_locks: -bb.utils.remove(f, True) +self._unlock_rpm_db() except subprocess.CalledProcessError as e: bb.fatal(Cannot get the installed packages list. Command '%s' returned %d:\n%s % (cmd, e.returncode, e.output)) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] SDK build refers to existing installed SDK
On Wed, Mar 26, 2014 at 07:40:26AM +, Mats Kärrman wrote: On Wednesday, March 26, 2014 4:23 AM, Khem Raj wrote: Hi, I just ran into an unexpected error while doing some multitasking. Project is based on OE-core Dora distroless. While my build machine was busy building a new SDK using -c populate_sdk I simultaneously deleted an older SDK that was installed under /usr/local and installed a different one. This resulted in the build stopping with the following error: | x86_64-oesdk-linux-gcc --sysroot=/home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/sysroots/x86_64-nativesdk-oesdk-linux -DHAVE_CONFIG_H -I../ncurses -I/home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/work/x86_64-nativesdk-oesdk-linux/nativesdk-ncurses/5.9-r15.1/ncurses-5.9/ncurses -isystem/home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/sysroots/x86_64-nativesdk-oesdk-linux/usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/include -D_GNU_SOURCE -DNDEBUG -I. -I../include -I/home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/work/x86_64-nativesdk-oesdk-linux/nativesdk-ncurses/5.9-r15.1/ncurses-5.9/ncurses/../include -I/usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/include -isystem/home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/sysroots/x86_64-nativesdk-oesdk-linux/usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/include -O2 -pipe --param max-inline-insns-single=1200 -fPIC -c /home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/work/x86 _64- nativesdk-oesdk-linux/nativesdk-ncurses/5.9-r15.1/ncurses-5.9/ncurses/base/lib_beep.c -o ../obj_s/lib_beep.o | cc1: error: /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/include: Permission denied The offending directory is given with an include directive which seems strange, building a new SDK should not poke around in an old one... Config log files available on request. Was the installed SDK in your path that fed into the OE build env which was generating new SDK ? No, it was not. I have saved the output of bitbake nativesdk-ncurses -e if you're interested. I had a quick look at the generated Makefile for nativesdk-ncurses and, apparently, it seems to have -I${includedir} added to the compilation flags. This doesn't look like the right thing... laurentiu BR // Mats -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
On Wed, Mar 26, 2014 at 09:15:43AM +, Sarbu, Florin-Ionut (Florin) wrote: I get the same error on do_rootfs for a rpm based image: Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment Does this happen on latest master? Do you have the 4 patches, starting with this one http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d91e35640d19471213122d36288315f071c37432 in your tree? Did you activate buildhistory? Can you provide more info regarding the steps you followed to reproduce this? laurentiu Florin From: openembedded-core-boun...@lists.openembedded.org [openembedded-core-boun...@lists.openembedded.org] on behalf of Laurentiu Palcu [laurentiu.pa...@intel.com] Sent: Friday, March 07, 2014 8:35 AM To: Richard Purdie Cc: openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation On Thu, Mar 06, 2014 at 09:55:38PM +, Richard Purdie wrote: On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote: Call the new python routines. [YOCTO #5904] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/buildhistory.bbclass |9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass index ef4135b..01b0082 100644 --- a/meta/classes/buildhistory.bbclass +++ b/meta/classes/buildhistory.bbclass @@ -319,6 +319,12 @@ python buildhistory_list_installed() { with open(pkgs_list_file, 'w') as pkgs_list: pkgs_list.write(list_installed_packages(d, 'file')) + +pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True), + bh_installed_pkgs_deps.txt) + +with open(pkgs_deps_file, 'w') as pkgs_deps: +pkgs_deps.write(list_installed_packages(d, 'deps')) } @@ -338,7 +344,8 @@ buildhistory_get_installed() { # Produce dependency graph # First, quote each name to handle characters that cause issues for dot - rootfs_list_installed_depends | sed 's:\([^| ]*\):\1:g' $1/depends.tmp + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):\1:g' $1/depends.tmp \ + rm ${WORKDIR}/bh_installed_pkgs_deps.txt # Change delimiter from pipe to - and set style for recommend lines sed -i -e 's:|: - :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' $1/depends.tmp # Add header, sorted and de-duped contents and footer and then delete the temp file With this patch, a bitbake core-image-minimal -c populate_sdk resulted No, not quite. The bug appears to be in RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched. CCing Hongxu. laurentiu in: ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb: The stack trace of python calls that resulted in this exception/failure was: File: 'buildhistory_list_installed', lineno: 18, function: module 0014:with open(pkgs_deps_file, 'w') as pkgs_deps: 0015:pkgs_deps.write(list_installed_packages(d, 'deps')) 0016: 0017: *** 0018:buildhistory_list_installed(d) 0019: File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed 0005:pkgs_list_file = os.path.join(d.getVar('WORKDIR', True), 0006: bh_installed_pkgs.txt) 0007: 0008:with open(pkgs_list_file, 'w') as pkgs_list: *** 0009:pkgs_list.write(list_installed_packages(d, 'file')) 0010: 0011:pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True), 0012: bh_installed_pkgs_deps.txt) 0013: File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages 0717:if img_type == rpm: 0718:return RpmPM(d, 0719: rootfs_dir, 0720: d.getVar('TARGET_VENDOR', True) *** 0721: ).list_installed(format) 0722:elif img_type == ipk: 0723:return OpkgPM(d, 0724: rootfs_dir, 0725: d.getVar(IPKGCONF_TARGET, True), File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed 0850:pkg = line.split()[0] 0851:arch = line.split()[1] 0852:ver = line.split()[2] 0853:pkgorigin = line.split()[3] *** 0854:new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch) 0855: 0856:if format == arch: 0857:output.append('%s %s' % (new_pkg, new_arch)) 0858:elif format == file
Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
Please file a bug on this. thanks, laurentiu On Wed, Mar 26, 2014 at 01:45:40PM +0200, Florin Sarbu wrote: On 03/26/2014 12:09 PM, Laurentiu Palcu wrote: On Wed, Mar 26, 2014 at 09:15:43AM +, Sarbu, Florin-Ionut (Florin) wrote: I get the same error on do_rootfs for a rpm based image: Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment Does this happen on latest master? Yes. Do you have the 4 patches, starting with this one http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d91e35640d19471213122d36288315f071c37432 in your tree? Yes, I have that commit in poky. Did you activate buildhistory? Can you provide more info regarding the steps you followed to reproduce this? I am trying to build the gemini-image from the meta-ivi layer g...@git.yoctoproject.org:meta-ivi.git The build steps are those from the README.md file Florin laurentiu Florin From: openembedded-core-boun...@lists.openembedded.org [openembedded-core-boun...@lists.openembedded.org] on behalf of Laurentiu Palcu [laurentiu.pa...@intel.com] Sent: Friday, March 07, 2014 8:35 AM To: Richard Purdie Cc: openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation On Thu, Mar 06, 2014 at 09:55:38PM +, Richard Purdie wrote: On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote: Call the new python routines. [YOCTO #5904] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/buildhistory.bbclass |9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass index ef4135b..01b0082 100644 --- a/meta/classes/buildhistory.bbclass +++ b/meta/classes/buildhistory.bbclass @@ -319,6 +319,12 @@ python buildhistory_list_installed() { with open(pkgs_list_file, 'w') as pkgs_list: pkgs_list.write(list_installed_packages(d, 'file')) + +pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True), + bh_installed_pkgs_deps.txt) + +with open(pkgs_deps_file, 'w') as pkgs_deps: +pkgs_deps.write(list_installed_packages(d, 'deps')) } @@ -338,7 +344,8 @@ buildhistory_get_installed() { # Produce dependency graph # First, quote each name to handle characters that cause issues for dot - rootfs_list_installed_depends | sed 's:\([^| ]*\):\1:g' $1/depends.tmp + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):\1:g' $1/depends.tmp \ + rm ${WORKDIR}/bh_installed_pkgs_deps.txt # Change delimiter from pipe to - and set style for recommend lines sed -i -e 's:|: - :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' $1/depends.tmp # Add header, sorted and de-duped contents and footer and then delete the temp file With this patch, a bitbake core-image-minimal -c populate_sdk resulted No, not quite. The bug appears to be in RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched. CCing Hongxu. laurentiu in: ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb: The stack trace of python calls that resulted in this exception/failure was: File: 'buildhistory_list_installed', lineno: 18, function: module 0014:with open(pkgs_deps_file, 'w') as pkgs_deps: 0015:pkgs_deps.write(list_installed_packages(d, 'deps')) 0016: 0017: *** 0018:buildhistory_list_installed(d) 0019: File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed 0005:pkgs_list_file = os.path.join(d.getVar('WORKDIR', True), 0006: bh_installed_pkgs.txt) 0007: 0008:with open(pkgs_list_file, 'w') as pkgs_list: *** 0009:pkgs_list.write(list_installed_packages(d, 'file')) 0010: 0011:pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True), 0012: bh_installed_pkgs_deps.txt) 0013: File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages 0717:if img_type == rpm: 0718:return RpmPM(d, 0719: rootfs_dir, 0720: d.getVar('TARGET_VENDOR', True) *** 0721: ).list_installed(format) 0722:elif img_type == ipk: 0723:return OpkgPM(d, 0724: rootfs_dir, 0725: d.getVar(IPKGCONF_TARGET, True), File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed 0850:pkg = line.split()[0] 0851:arch = line.split()[1] 0852:ver = line.split()[2] 0853
[OE-core] [PATCH v2 0/4] SDK buildhistory fixes (cover letter only)
v2: * I passed deploy_dir instead of target_rootfs to RpmPkgsList constructor... Strangely enough, this could be caught only when building core-image-minimal (which I didn't use in my tests). The buildhistory dependency files for target/host SDK packages were not properly created because the wrapper function called, list_installed_packages(), was always looking in the image rootfs. This patchset will rename the old wrapper function to image_list_installed_packages() and create a new one, for SDK stuff, sdk_list_installed_packages(). The changes in package_manager.py, even though they appear to be lots, its the same code moved around from one class to a newly created PkgsList class. So, the logic remains the same. Tested for all backends (buildhistory activated) with the following: bitbake core-image-sato bitbake -c populate_sdk core-image-sato laurentiu The following changes since commit 0150bc30d3674301631c2e9b6c64e01058fd1070: bitbake: runqueue: Really fix sigchld handling (2014-03-18 23:05:53 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/buildhistory_sdk_dep_files_fix http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/buildhistory_sdk_dep_files_fix Laurentiu Palcu (4): package_manager.py: create separate class for installed packages listing rootfs.py, sdk.py: adjust/create the wrappers for creating installed packages list image.bbclass, license.bbclass: adjust the name of list_installed_packages() buildhistory.bbclass: create proper dependency files for SDK meta/classes/buildhistory.bbclass | 40 ++-- meta/classes/image.bbclass|4 +- meta/classes/license.bbclass |4 +- meta/lib/oe/package_manager.py| 376 + meta/lib/oe/rootfs.py | 19 +- meta/lib/oe/sdk.py| 18 ++ 6 files changed, 264 insertions(+), 197 deletions(-) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/4] rootfs.py, sdk.py: adjust/create the wrappers for creating installed packages list
Since we created a new PkgsList object that will deal with listing the installed packages in a rootfs, use the new class both for images and SDKs in the wrapper functions. The old list_installed_packages() wrapper listed only the packages inside an image rootfs. It didn't deal with target/host SDK rootfs's. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/rootfs.py | 19 --- meta/lib/oe/sdk.py| 18 ++ 2 files changed, 22 insertions(+), 15 deletions(-) diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py index 30a1321..0e6c8bc 100644 --- a/meta/lib/oe/rootfs.py +++ b/meta/lib/oe/rootfs.py @@ -709,28 +709,17 @@ def create_rootfs(d, manifest_dir=None): os.environ.update(env_bkp) -def list_installed_packages(d, format=None, rootfs_dir=None): +def image_list_installed_packages(d, format=None, rootfs_dir=None): if not rootfs_dir: rootfs_dir = d.getVar('IMAGE_ROOTFS', True) img_type = d.getVar('IMAGE_PKGTYPE', True) if img_type == rpm: -return RpmPM(d, - rootfs_dir, - d.getVar('TARGET_VENDOR', True) - ).list_installed(format) +return RpmPkgsList(d, rootfs_dir).list(format) elif img_type == ipk: -return OpkgPM(d, - rootfs_dir, - d.getVar(IPKGCONF_TARGET, True), - d.getVar(ALL_MULTILIB_PACKAGE_ARCHS, True) - ).list_installed(format) +return OpkgPkgsList(d, rootfs_dir, d.getVar(IPKGCONF_TARGET, True)).list(format) elif img_type == deb: -return DpkgPM(d, - rootfs_dir, - d.getVar('PACKAGE_ARCHS', True), - d.getVar('DPKG_ARCH', True) - ).list_installed(format) +return DpkgPkgsList(d, rootfs_dir).list(format) if __name__ == __main__: diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py index 01a1807..5643199 100644 --- a/meta/lib/oe/sdk.py +++ b/meta/lib/oe/sdk.py @@ -289,6 +289,24 @@ class DpkgSdk(Sdk): bb.utils.remove(os.path.join(self.sdk_output, var), True) +def sdk_list_installed_packages(d, target, format=None, rootfs_dir=None): +if rootfs_dir is None: +sdk_output = d.getVar('SDK_OUTPUT', True) +target_path = d.getVar('SDKTARGETSYSROOT', True).strip('/') + +rootfs_dir = [sdk_output, os.path.join(sdk_output, target_path)][target is True] + +img_type = d.getVar('IMAGE_PKGTYPE', True) +if img_type == rpm: +arch_var = [SDK_PACKAGE_ARCHS, None][target is True] +os_var = [SDK_OS, None][target is True] +return RpmPkgsList(d, rootfs_dir, arch_var, os_var).list(format) +elif img_type == ipk: +conf_file_var = [IPKGCONF_SDK, IPKGCONF_Target][target is True] +return OpkgPkgsList(d, rootfs_dir, d.getVar(conf_file_var, True)).list(format) +elif img_type == deb: +return DpkgPkgsList(d, rootfs_dir).list(format) + def populate_sdk(d, manifest_dir=None): env_bkp = os.environ.copy() -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/4] package_manager.py: create separate class for installed packages listing
This commit creates a new class that has the only purpose to generate various listings of installed packages in the rootfs. Basically, the methods involved in listing the installed packages, that were part of each backend PM class implementation, were moved to this new class. This change avoids instantiating a new PM object just to get the list of installed packages in a certain rootfs. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py | 376 ++-- 1 file changed, 211 insertions(+), 165 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 1279b50..0426b23 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -213,6 +213,213 @@ class DpkgIndexer(Indexer): return(result) +class PkgsList(object): +__metaclass__ = ABCMeta + +def __init__(self, d, rootfs_dir): +self.d = d +self.rootfs_dir = rootfs_dir + +@abstractmethod +def list(self, format=None): +pass + + +class RpmPkgsList(PkgsList): +def __init__(self, d, rootfs_dir, arch_var=None, os_var=None): +super(RpmPkgsList, self).__init__(d, rootfs_dir) + +self.rpm_cmd = bb.utils.which(os.getenv('PATH'), rpm) +self.image_rpmlib = os.path.join(self.rootfs_dir, 'var/lib/rpm') + +self.ml_prefix_list, self.ml_os_list = \ +RpmIndexer(d, rootfs_dir).get_ml_prefix_and_os_list(arch_var, os_var) + +''' +Translate the RPM/Smart format names to the OE multilib format names +''' +def _pkg_translate_smart_to_oe(self, pkg, arch): +new_pkg = pkg +fixed_arch = arch.replace('_', '-') +found = 0 +for mlib in self.ml_prefix_list: +for cmp_arch in self.ml_prefix_list[mlib]: +fixed_cmp_arch = cmp_arch.replace('_', '-') +if fixed_arch == fixed_cmp_arch: +if mlib == 'default': +new_pkg = pkg +new_arch = cmp_arch +else: +new_pkg = mlib + '-' + pkg +# We need to strip off the ${mlib}_ prefix on the arch +new_arch = cmp_arch.replace(mlib + '_', '') + +# Workaround for bug 3565. Simply look to see if we +# know of a package with that name, if not try again! +filename = os.path.join(self.d.getVar('PKGDATA_DIR', True), +'runtime-reverse', +new_pkg) +if os.path.exists(filename): +found = 1 +break + +if found == 1 and fixed_arch == fixed_cmp_arch: +break +#bb.note('%s, %s - %s, %s' % (pkg, arch, new_pkg, new_arch)) +return new_pkg, new_arch + +def _list_pkg_deps(self): +cmd = [bb.utils.which(os.getenv('PATH'), rpmresolve), + -t, self.image_rpmlib] + +try: +output = subprocess.check_output(cmd, stderr=subprocess.STDOUT).strip() +except subprocess.CalledProcessError as e: +bb.fatal(Cannot get the package dependencies. Command '%s' + returned %d:\n%s % (' '.join(cmd), e.returncode, e.output)) + +return output + +def list(self, format=None): +if format == deps: +return self._list_pkg_deps() + +cmd = self.rpm_cmd + ' --root ' + self.rootfs_dir +cmd += ' -D _dbpath /var/lib/rpm -qa' +cmd += --qf '[%{NAME} %{ARCH} %{VERSION} %{PACKAGEORIGIN}\n]' + +try: +# bb.note(cmd) +tmp_output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).strip() + +rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.rootfs_dir) +for f in rpm_db_locks: +bb.utils.remove(f, True) +except subprocess.CalledProcessError as e: +bb.fatal(Cannot get the installed packages list. Command '%s' + returned %d:\n%s % (cmd, e.returncode, e.output)) + +output = list() +for line in tmp_output.split('\n'): +if len(line.strip()) == 0: +continue +pkg = line.split()[0] +arch = line.split()[1] +ver = line.split()[2] +pkgorigin = line.split()[3] +new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch) + +if format == arch: +output.append('%s %s' % (new_pkg, new_arch)) +elif format == file: +output.append('%s %s %s' % (new_pkg, pkgorigin, new_arch)) +elif format == ver: +output.append('%s %s %s' % (new_pkg, new_arch, ver)) +else: +output.append('%s' % (new_pkg)) + +output.sort
[OE-core] [PATCH 3/4] image.bbclass, license.bbclass: adjust the name of list_installed_packages()
The old wrapper got renamed to image_list_installed_packages(). Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/image.bbclass |4 ++-- meta/classes/license.bbclass |4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 51d16d7..9a04288 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -330,9 +330,9 @@ make_zimage_symlink_relative () { } python write_image_manifest () { -from oe.rootfs import list_installed_packages +from oe.rootfs import image_list_installed_packages with open(d.getVar('IMAGE_MANIFEST', True), 'w+') as image_manifest: -image_manifest.write(list_installed_packages(d, 'ver')) +image_manifest.write(image_list_installed_packages(d, 'ver')) } # Make login manager(s) enable automatic login. diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass index a0b877d..08f0665 100644 --- a/meta/classes/license.bbclass +++ b/meta/classes/license.bbclass @@ -20,9 +20,9 @@ python write_package_manifest() { # Get list of installed packages license_image_dir = d.expand('${LICENSE_DIRECTORY}/${IMAGE_NAME}') bb.utils.mkdirhier(license_image_dir) - from oe.rootfs import list_installed_packages + from oe.rootfs import image_list_installed_packages open(os.path.join(license_image_dir, 'package.manifest'), - 'w+').write(list_installed_packages(d)) + 'w+').write(image_list_installed_packages(d)) } license_create_manifest() { -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/4] SDK buildhistory fixes
The buildhistory dependency files for target/host SDK packages were not properly created because the wrapper function called, list_installed_packages(), was always looking in the image rootfs. This patchset will rename the old wrapper function to image_list_installed_packages() and create a new one, for SDK stuff, sdk_list_installed_packages(). The changes in package_manager.py, even though they appear to be lots, its the same code moved around from one class to a newly created PkgsList class. So, the logic remains the same. Tested for all backends (buildhistory activated) with the following: bitbake core-image-sato bitbake -c populate_sdk core-image-sato laurentiu The following changes since commit 6bbb179cc526c86631dfcb140e3dd51a8c07a52d: bitbake: runqueue: More carefully handle the sigchld handler (2014-03-18 10:23:13 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/buildhistory_sdk_dep_files_fix http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/buildhistory_sdk_dep_files_fix Laurentiu Palcu (4): package_manager.py: create separate class for installed packages listing rootfs.py, sdk.py: adjust/create the wrappers for creating installed packages list image.bbclass, license.bbclass: adjust the name of list_installed_packages() buildhistory.bbclass: create proper dependency files for SDK meta/classes/buildhistory.bbclass | 40 ++-- meta/classes/image.bbclass|4 +- meta/classes/license.bbclass |4 +- meta/lib/oe/package_manager.py| 376 + meta/lib/oe/rootfs.py | 19 +- meta/lib/oe/sdk.py| 18 ++ 6 files changed, 264 insertions(+), 197 deletions(-) -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 4/4] buildhistory.bbclass: create proper dependency files for SDK
The old functions were calling the list_installed_packages() wrapper function that only listed the packages in an image rootfs. Even for target/host SDK. Also, a python crash was possible if 'bitbake -c populate_sdk core-image-*' was called without calling 'bitbake core-image-*' first. That's because the wrapper was always looking into the image rootfs... This commit fixes the problem and calls the right wrapper for image/sdk. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/buildhistory.bbclass | 40 + 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass index 5d0a229..262095f 100644 --- a/meta/classes/buildhistory.bbclass +++ b/meta/classes/buildhistory.bbclass @@ -313,22 +313,36 @@ def write_pkghistory(pkginfo, d): if os.path.exists(filevarpath): os.unlink(filevarpath) -python buildhistory_list_installed() { -from oe.rootfs import list_installed_packages +# +# rootfs_type can be: image, sdk_target, sdk_host +# +def buildhistory_list_installed(d, rootfs_type=image): +from oe.rootfs import image_list_installed_packages +from oe.sdk import sdk_list_installed_packages + +process_list = [('file', 'bh_installed_pkgs.txt'),\ +('deps', 'bh_installed_pkgs_deps.txt')] -pkgs_list_file = os.path.join(d.getVar('WORKDIR', True), - bh_installed_pkgs.txt) +for output_type, output_file in process_list: +output_file_full = os.path.join(d.getVar('WORKDIR', True), output_file) -with open(pkgs_list_file, 'w') as pkgs_list: -pkgs_list.write(list_installed_packages(d, 'file')) +with open(output_file_full, 'w') as output: +if rootfs_type == image: +output.write(image_list_installed_packages(d, output_type)) +else: +output.write(sdk_list_installed_packages(d, rootfs_type == sdk_target, output_type)) -pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True), - bh_installed_pkgs_deps.txt) +python buildhistory_list_installed_image() { +buildhistory_list_installed(d) +} -with open(pkgs_deps_file, 'w') as pkgs_deps: -pkgs_deps.write(list_installed_packages(d, 'deps')) +python buildhistory_list_installed_sdk_target() { +buildhistory_list_installed(d, sdk_target) } +python buildhistory_list_installed_sdk_host() { +buildhistory_list_installed(d, sdk_host) +} buildhistory_get_installed() { mkdir -p $1 @@ -471,15 +485,15 @@ END } # By prepending we get in before the removal of packaging files -ROOTFS_POSTPROCESS_COMMAND =+ buildhistory_list_installed ;\ +ROOTFS_POSTPROCESS_COMMAND =+ buildhistory_list_installed_image ;\ buildhistory_get_image_installed ; IMAGE_POSTPROCESS_COMMAND += buildhistory_get_imageinfo ; # We want these to be the last run so that we get called after complementary package installation -POPULATE_SDK_POST_TARGET_COMMAND_append = buildhistory_list_installed ;\ +POPULATE_SDK_POST_TARGET_COMMAND_append = buildhistory_list_installed_sdk_target ;\ buildhistory_get_sdk_installed_target ; -POPULATE_SDK_POST_HOST_COMMAND_append = buildhistory_list_installed ;\ +POPULATE_SDK_POST_HOST_COMMAND_append = buildhistory_list_installed_sdk_host ;\ buildhistory_get_sdk_installed_host ; SDK_POSTPROCESS_COMMAND += buildhistory_get_sdkinfo ; -- 1.7.9.5 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
I succeeded to replicate this. I'll take a look at it. laurentiu On Sat, Mar 08, 2014 at 04:23:22PM +0800, Hongxu Jia wrote: On 03/07/2014 02:35 PM, Laurentiu Palcu wrote: On Thu, Mar 06, 2014 at 09:55:38PM +, Richard Purdie wrote: On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote: Call the new python routines. [YOCTO #5904] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/buildhistory.bbclass |9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass index ef4135b..01b0082 100644 --- a/meta/classes/buildhistory.bbclass +++ b/meta/classes/buildhistory.bbclass @@ -319,6 +319,12 @@ python buildhistory_list_installed() { with open(pkgs_list_file, 'w') as pkgs_list: pkgs_list.write(list_installed_packages(d, 'file')) + +pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True), + bh_installed_pkgs_deps.txt) + +with open(pkgs_deps_file, 'w') as pkgs_deps: +pkgs_deps.write(list_installed_packages(d, 'deps')) } @@ -338,7 +344,8 @@ buildhistory_get_installed() { # Produce dependency graph # First, quote each name to handle characters that cause issues for dot - rootfs_list_installed_depends | sed 's:\([^| ]*\):\1:g' $1/depends.tmp + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):\1:g' $1/depends.tmp \ + rm ${WORKDIR}/bh_installed_pkgs_deps.txt # Change delimiter from pipe to - and set style for recommend lines sed -i -e 's:|: - :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' $1/depends.tmp # Add header, sorted and de-duped contents and footer and then delete the temp file With this patch, a bitbake core-image-minimal -c populate_sdk resulted No, not quite. The bug appears to be in RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched. CCing Hongxu. Hi Laurentiu and Richard, The reason is pkg's arch was not found in the multilib arch list. I could not reproduce the defect on my local build with these patches applyed. The attach a patch that add some trace code to see which pkg's arch is not found in this situation. //Hongxu laurentiu in: ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb: The stack trace of python calls that resulted in this exception/failure was: File: 'buildhistory_list_installed', lineno: 18, function: module 0014:with open(pkgs_deps_file, 'w') as pkgs_deps: 0015:pkgs_deps.write(list_installed_packages(d, 'deps')) 0016: 0017: *** 0018:buildhistory_list_installed(d) 0019: File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed 0005:pkgs_list_file = os.path.join(d.getVar('WORKDIR', True), 0006: bh_installed_pkgs.txt) 0007: 0008:with open(pkgs_list_file, 'w') as pkgs_list: *** 0009:pkgs_list.write(list_installed_packages(d, 'file')) 0010: 0011:pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True), 0012: bh_installed_pkgs_deps.txt) 0013: File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages 0717:if img_type == rpm: 0718:return RpmPM(d, 0719: rootfs_dir, 0720: d.getVar('TARGET_VENDOR', True) *** 0721: ).list_installed(format) 0722:elif img_type == ipk: 0723:return OpkgPM(d, 0724: rootfs_dir, 0725: d.getVar(IPKGCONF_TARGET, True), File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed 0850:pkg = line.split()[0] 0851:arch = line.split()[1] 0852:ver = line.split()[2] 0853:pkgorigin = line.split()[3] *** 0854:new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch) 0855: 0856:if format == arch: 0857:output.append('%s %s' % (new_pkg, new_arch)) 0858:elif format == file: File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, function: _pkg_translate_smart_to_oe 0472: 0473:if found == 1 and fixed_arch == fixed_cmp_arch: 0474:break 0475:#bb.note('%s, %s - %s, %s' % (pkg, arch, new_pkg, new_arch)) *** 0476:return new_pkg, new_arch 0477: 0478:def _search_pkg_name_in_feeds(self, pkg, feed_archs): 0479:for arch in feed_archs: 0480:arch = arch.replace('-', '_') Exception: UnboundLocalError: local variable
[OE-core] [PATCH 1/1] xf86-video-intel: add recipe for 2.99.910, remove the git one
Intel graphics stack releases = 2013Q3 need xf86-video-intel = 2.99.902. However, keep the stable release around too, in case people need it. The git recipe is not really used. Remove. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/conf/distro/include/default-versions.inc |3 ++ .../xorg-driver/xf86-video-intel_2.99.910.bb | 31 .../xorg-driver/xf86-video-intel_git.bb| 23 --- 3 files changed, 34 insertions(+), 23 deletions(-) create mode 100644 meta/recipes-graphics/xorg-driver/xf86-video-intel_2.99.910.bb delete mode 100644 meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb diff --git a/meta/conf/distro/include/default-versions.inc b/meta/conf/distro/include/default-versions.inc index 53ec2e7..a9a931d 100644 --- a/meta/conf/distro/include/default-versions.inc +++ b/meta/conf/distro/include/default-versions.inc @@ -9,3 +9,6 @@ PREFERRED_VERSION_python-native ?= 2.7.3 # Force the older version of liberation-fonts until we fix the fontforge issue PREFERRED_VERSION_liberation-fonts ?= 1.04 + +# Intel video stack 2013Q3 and newer need a version =2.99.902 +PREFERRED_VERSION_xf86-video-intel ?= 2.99.910 diff --git a/meta/recipes-graphics/xorg-driver/xf86-video-intel_2.99.910.bb b/meta/recipes-graphics/xorg-driver/xf86-video-intel_2.99.910.bb new file mode 100644 index 000..b74d174 --- /dev/null +++ b/meta/recipes-graphics/xorg-driver/xf86-video-intel_2.99.910.bb @@ -0,0 +1,31 @@ +require xorg-driver-video.inc + +SUMMARY = X.Org X server -- Intel integrated graphics chipsets driver + +DESCRIPTION = intel is an Xorg driver for Intel integrated graphics \ +chipsets. The driver supports depths 8, 15, 16 and 24. On some chipsets, \ +the driver supports hardware accelerated 3D via the Direct Rendering \ +Infrastructure (DRI). + +LIC_FILES_CHKSUM = file://COPYING;md5=8730ad58d11c7bbad9a7066d69f7808e + +DEPENDS += virtual/libx11 drm libpciaccess pixman + +SRC_URI += file://compat-api-Map-changes-of-DamageUnregister-API-in-1..patch \ + + +PACKAGECONFIG ??= sna udev ${@base_contains('DISTRO_FEATURES', 'opengl', 'dri', '', d)} + +PACKAGECONFIG[dri] = --enable-dri,--disable-dri,xf86driproto dri2proto +PACKAGECONFIG[sna] = --enable-sna,--disable-sna +PACKAGECONFIG[uxa] = --enable-uxa,--disable-uxa +PACKAGECONFIG[udev] = --enable-udev,--disable-udev,udev +PACKAGECONFIG[xvmc] = --enable-xvmc,--disable-xvmc,libxvmc + +# --enable-kms-only option is required by ROOTLESS_X +EXTRA_OECONF += '${@base_conditional( ROOTLESS_X, 1, --enable-kms-only, , d )}' + +COMPATIBLE_HOST = '(i.86|x86_64).*-linux' + +SRC_URI[md5sum] = a9a5c2c15766c06a024381efe0d724bb +SRC_URI[sha256sum] = 203d46064449da0e23a111418dfb189422ba96ea08707167c8dee463e2d745b1 diff --git a/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb b/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb deleted file mode 100644 index d4c7654..000 --- a/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb +++ /dev/null @@ -1,23 +0,0 @@ -require xorg-driver-video.inc - -SUMMARY = X.Org X server -- Intel integrated graphics chipsets driver - -DESCRIPTION = intel is an Xorg driver for Intel integrated graphics \ -chipsets. The driver supports depths 8, 15, 16 and 24. On some chipsets, \ -the driver supports hardware accelerated 3D via the Direct Rendering \ -Infrastructure (DRI). - -DEPENDS += virtual/libx11 drm dri2proto glproto \ - virtual/libgl xineramaproto libpciaccess udev - -SRCREV = 87ea531c5dc5b39809395b277c330854f019 -PV = 2.10.0+git${SRCPV} -PR = ${INC_PR}.1 - -EXTRA_OECONF += --disable-xvmc --enable-dri --disable-static - -SRC_URI = git://anongit.freedesktop.org/git/xorg/driver/xf86-video-intel - -S = ${WORKDIR}/git - -COMPATIBLE_HOST = '(i.86|x86_64).*-linux' -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] xf86-video-intel: add recipe for 2.99.910, remove the git one
The following changes since commit 2f9bf7bc97cb832f4db13df62ce09d96ce75e810: PR bumps to remove PRINC use in meta-fsl-arm (2014-03-07 13:39:11 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b5705_q4_intel_graphics_stack http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5705_q4_intel_graphics_stack Laurentiu Palcu (1): xf86-video-intel: add recipe for 2.99.910, remove the git one meta/conf/distro/include/default-versions.inc |3 ++ .../xorg-driver/xf86-video-intel_2.99.910.bb | 31 .../xorg-driver/xf86-video-intel_git.bb| 23 --- 3 files changed, 34 insertions(+), 23 deletions(-) create mode 100644 meta/recipes-graphics/xorg-driver/xf86-video-intel_2.99.910.bb delete mode 100644 meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] run-postinsts: use it for opkg/dpkg too
Currently, opkg/dpkg have their own postinstalls that create a run-postinsts script which is run at first boot. This commit prepares the run-postinsts recipe/script to be used by opkg/dpkg when DISTRO_FEATURES includes package-management. [YOCTO #5666] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../run-postinsts/run-postinsts/run-postinsts | 63 +--- .../run-postinsts/run-postinsts_1.0.bb |2 + 2 files changed, 42 insertions(+), 23 deletions(-) diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts index 11141ec..08cfa9e 100755 --- a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts +++ b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts @@ -8,13 +8,9 @@ # The following script will run all the scriptlets found in #SYSCONFDIR#/deb-postinsts, # #SYSCONFDIR#/ipk-postinsts or #SYSCONFDIR#/rpm-posinsts. -pi_dir= -for pm in rpm deb ipk; do - if [ -d #SYSCONFDIR#/${pm}-postinsts ]; then - pi_dir=#SYSCONFDIR#/${pm}-postinsts - break - fi -done +pm=#IMAGE_PKGTYPE# +pm_installed=#PM_INSTALLED# +pi_dir=#SYSCONFDIR#/${pm}-postinsts remove_rcsd_link () { if [ -n `which update-rc.d` ]; then @@ -29,24 +25,45 @@ fi [ -e #SYSCONFDIR#/default/postinst ] . #SYSCONFDIR#/default/postinst -remove_pi_dir=1 -for i in `ls $pi_dir`; do - i=$pi_dir/$i - echo Running postinst $i... - [ $POSTINST_LOGGING = 1 ] echo Running postinst $i... $LOGFILE - if [ -x $i ]; then - if [ $POSTINST_LOGGING = 1 ]; then - sh -c $i $LOGFILE 21 +if [ $POSTINST_LOGGING = 1 ]; then + rm -f $LOGFILE + append_log=$LOGFILE 21 +fi + +exec_postinst_scriptlets() { + for i in `ls $pi_dir`; do + i=$pi_dir/$i + echo Running postinst $i... + [ $POSTINST_LOGGING = 1 ] eval echo Running postinst $i... $append_log + if [ -x $i ]; then + eval sh -c $i $append_log + rm $i else - sh -c $i + echo ERROR: postinst $i failed. + [ $POSTINST_LOGGING = 1 ] eval echo ERROR: postinst $i failed. $append_log + remove_pi_dir=0 fi - rm $i - else - echo ERROR: postinst $i failed. - [ $POSTINST_LOGGING = 1 ] echo ERROR: postinst $i failed. $LOGFILE - remove_pi_dir=0 - fi -done + done +} + +remove_pi_dir=1 +if $pm_installed; then + case $pm in + ipk) + eval opkg-cl configure $append_log + ;; + + deb) + eval dpkg --configure -a $append_log + ;; + + rpm) + exec_postinst_scriptlets + ;; + esac +else + exec_postinst_scriptlets +fi # since all postinstalls executed successfully, remove the postinstalls directory # and the rcS.d link diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb b/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb index 64f85c2..e990c67 100644 --- a/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb +++ b/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb @@ -37,6 +37,8 @@ do_install() { sed -i -e 's:#SYSCONFDIR#:${sysconfdir}:g' \ -e 's:#SBINDIR#:${sbindir}:g' \ -e 's:#BASE_BINDIR#:${base_bindir}:g' \ + -e 's:#IMAGE_PKGTYPE#:${IMAGE_PKGTYPE}:g' \ + -e 's:#PM_INSTALLED#:${@base_contains(IMAGE_FEATURES, package-management, true, false, d)}:g' \ ${D}${sbindir}/run-postinsts \ ${D}${systemd_unitdir}/system/run-postinsts.service } -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] opkg/dpkg: remove the postinstalls
Just use the run-postinsts recipe for running first boot postinstalls. [YOCTO #5666] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/recipes-devtools/dpkg/dpkg.inc | 24 +--- meta/recipes-devtools/opkg/opkg.inc | 28 +--- 2 files changed, 2 insertions(+), 50 deletions(-) diff --git a/meta/recipes-devtools/dpkg/dpkg.inc b/meta/recipes-devtools/dpkg/dpkg.inc index 471151e..eef7ce9 100644 --- a/meta/recipes-devtools/dpkg/dpkg.inc +++ b/meta/recipes-devtools/dpkg/dpkg.inc @@ -7,7 +7,7 @@ SRC_URI = ${DEBIAN_MIRROR}/main/d/dpkg/dpkg_${PV}.tar.xz \ DEPENDS = zlib bzip2 perl ncurses DEPENDS_class-native = bzip2-replacement-native zlib-native virtual/update-alternatives-native gettext-native perl-native -RDEPENDS_${PN} = ${VIRTUAL-RUNTIME_update-alternatives} xz +RDEPENDS_${PN} = ${VIRTUAL-RUNTIME_update-alternatives} xz run-postinsts RDEPENDS_${PN}_class-native = xz-native S = ${WORKDIR}/${BPN}-${PV} @@ -43,8 +43,6 @@ do_configure () { autotools_do_configure } -DPKG_INIT_POSITION ?= 98 - do_install_append () { if [ ${PN} = dpkg-native ]; then # update-alternatives doesn't have an offline mode @@ -66,26 +64,6 @@ do_install_append () { fi } -pkg_postinst_${PN} () { -#!/bin/sh -if ${@base_contains('DISTRO_FEATURES','sysvinit','true','false',d)} \ - [ x$D != x ] [ -f $D/var/lib/dpkg/status ]; then - install -d $D${sysconfdir}/rcS.d - - # this happens at S98 where our good 'ole packages script used to run - echo #!/bin/sh -[ -e ${sysconfdir}/default/postinst ] . ${sysconfdir}/default/postinst -if [ \\$POSTINST_LOGGING\ = \1\ ]; then -dpkg --configure -a \$LOGFILE 21 -else -dpkg --configure -a -fi -rm -f ${sysconfdir}/rcS.d/S${DPKG_INIT_POSITION}run-postinsts - $D${sysconfdir}/rcS.d/S${DPKG_INIT_POSITION}run-postinsts - chmod 0755 $D${sysconfdir}/rcS.d/S${DPKG_INIT_POSITION}run-postinsts -fi -} - PROV = virtual/update-alternatives PROV_class-native = diff --git a/meta/recipes-devtools/opkg/opkg.inc b/meta/recipes-devtools/opkg/opkg.inc index 7bea26d..9f87df5 100644 --- a/meta/recipes-devtools/opkg/opkg.inc +++ b/meta/recipes-devtools/opkg/opkg.inc @@ -38,7 +38,7 @@ EXTRA_OECONF = \ --with-opkglibdir=${OPKGLIBDIR} \ -RDEPENDS_${PN} = ${VIRTUAL-RUNTIME_update-alternatives} opkg-config-base +RDEPENDS_${PN} = ${VIRTUAL-RUNTIME_update-alternatives} opkg-config-base run-postinsts RDEPENDS_${PN}_class-native = RDEPENDS_${PN}_class-nativesdk = RREPLACES_${PN} = opkg-nogpg @@ -73,30 +73,4 @@ do_install_append() { rm ${D}${bindir}/update-alternatives } -pkg_postinst_${PN} () { -#!/bin/sh -if ${@base_contains('DISTRO_FEATURES','sysvinit','true','false',d)} \ - [ x$D != x ] [ -f $D${OPKGLIBDIR}/opkg/status ]; then - install -d $D${sysconfdir}/rcS.d - - # this happens at S98 where our good 'ole packages script used to run - echo #!/bin/sh -[ -e ${sysconfdir}/default/postinst ] . ${sysconfdir}/default/postinst -if [ \\$POSTINST_LOGGING\ = \1\ ]; then -opkg-cl configure \$LOGFILE 21 -else -opkg-cl configure -fi -rm -f ${sysconfdir}/rcS.d/S${POSTINSTALL_INITPOSITION}run-postinsts - $D${sysconfdir}/rcS.d/S${POSTINSTALL_INITPOSITION}run-postinsts - chmod 0755 $D${sysconfdir}/rcS.d/S${POSTINSTALL_INITPOSITION}run-postinsts -fi -} - BBCLASSEXTEND = native nativesdk - -# Define a variable to allow distros to run configure earlier. -# (for example, to enable loading of ethernet kernel modules before networking starts) -# note: modifying name or default value for POSTINSTALL_INITPOSITION requires -# changes in systemd-compat-units.bb -POSTINSTALL_INITPOSITION ?= 98 -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/2] Unique run-postinsts
This patchset removes the opkg/dpkg postinstalls that were creating their own run-postinsts script which, sometimes, confused people... Instead, use the already available run-postinsts recipe in order to create the run-postinsts init script. It will be used for both 'package-management'/no 'package-management' in DISTRO_FEATURES. In my tests I tried to cover as much usecases as possible, in various combinations: * rpm/ipk/deb; * package-management/no package-management; * delayed/no delayed postinstalls; * sysvinit/systemd; I hope I covered most of them. If there are any other 'dirty' usecases I should be aware of, don't be shy. laurentiu The following changes since commit a01af0202558e6ed9d16590b3a8d1dd1b95c0374: recipes: bump PRs (2014-03-05 17:36:37 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b5666_unique_run_postinsts http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5666_unique_run_postinsts Laurentiu Palcu (2): run-postinsts: use it for opkg/dpkg too opkg/dpkg: remove the postinstalls meta/recipes-devtools/dpkg/dpkg.inc| 24 +--- meta/recipes-devtools/opkg/opkg.inc| 28 + .../run-postinsts/run-postinsts/run-postinsts | 63 +--- .../run-postinsts/run-postinsts_1.0.bb |2 + 4 files changed, 44 insertions(+), 73 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
On Thu, Mar 06, 2014 at 09:55:38PM +, Richard Purdie wrote: On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote: Call the new python routines. [YOCTO #5904] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/buildhistory.bbclass |9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass index ef4135b..01b0082 100644 --- a/meta/classes/buildhistory.bbclass +++ b/meta/classes/buildhistory.bbclass @@ -319,6 +319,12 @@ python buildhistory_list_installed() { with open(pkgs_list_file, 'w') as pkgs_list: pkgs_list.write(list_installed_packages(d, 'file')) + +pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True), + bh_installed_pkgs_deps.txt) + +with open(pkgs_deps_file, 'w') as pkgs_deps: +pkgs_deps.write(list_installed_packages(d, 'deps')) } @@ -338,7 +344,8 @@ buildhistory_get_installed() { # Produce dependency graph # First, quote each name to handle characters that cause issues for dot - rootfs_list_installed_depends | sed 's:\([^| ]*\):\1:g' $1/depends.tmp + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):\1:g' $1/depends.tmp \ + rm ${WORKDIR}/bh_installed_pkgs_deps.txt # Change delimiter from pipe to - and set style for recommend lines sed -i -e 's:|: - :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' $1/depends.tmp # Add header, sorted and de-duped contents and footer and then delete the temp file With this patch, a bitbake core-image-minimal -c populate_sdk resulted No, not quite. The bug appears to be in RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched. CCing Hongxu. laurentiu in: ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb: The stack trace of python calls that resulted in this exception/failure was: File: 'buildhistory_list_installed', lineno: 18, function: module 0014:with open(pkgs_deps_file, 'w') as pkgs_deps: 0015:pkgs_deps.write(list_installed_packages(d, 'deps')) 0016: 0017: *** 0018:buildhistory_list_installed(d) 0019: File: 'buildhistory_list_installed', lineno: 9, function: buildhistory_list_installed 0005:pkgs_list_file = os.path.join(d.getVar('WORKDIR', True), 0006: bh_installed_pkgs.txt) 0007: 0008:with open(pkgs_list_file, 'w') as pkgs_list: *** 0009:pkgs_list.write(list_installed_packages(d, 'file')) 0010: 0011:pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True), 0012: bh_installed_pkgs_deps.txt) 0013: File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: list_installed_packages 0717:if img_type == rpm: 0718:return RpmPM(d, 0719: rootfs_dir, 0720: d.getVar('TARGET_VENDOR', True) *** 0721: ).list_installed(format) 0722:elif img_type == ipk: 0723:return OpkgPM(d, 0724: rootfs_dir, 0725: d.getVar(IPKGCONF_TARGET, True), File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, function: list_installed 0850:pkg = line.split()[0] 0851:arch = line.split()[1] 0852:ver = line.split()[2] 0853:pkgorigin = line.split()[3] *** 0854:new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch) 0855: 0856:if format == arch: 0857:output.append('%s %s' % (new_pkg, new_arch)) 0858:elif format == file: File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, function: _pkg_translate_smart_to_oe 0472: 0473:if found == 1 and fixed_arch == fixed_cmp_arch: 0474:break 0475:#bb.note('%s, %s - %s, %s' % (pkg, arch, new_pkg, new_arch)) *** 0476:return new_pkg, new_arch 0477: 0478:def _search_pkg_name_in_feeds(self, pkg, feed_archs): 0479:for arch in feed_archs: 0480:arch = arch.replace('-', '_') Exception: UnboundLocalError: local variable 'new_arch' referenced before assignment ERROR: Function failed: buildhistory_list_installed ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.d ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/3] Fix issue with unpopulated buildhistory depends files
The following changes since commit f03955041d0e44d377ca1c4def630982f24f1e8b: Revert ncurses: use ln -r to generate relative symlink (2014-03-03 15:55:27 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b5904_buildhistory_depends_empty http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5904_buildhistory_depends_empty Laurentiu Palcu (3): package_manager.py: make list_installed() list pkg dependencies too buildhistory.bbclass: Fix dependency files creation populate_sdk_*.bbclass: remove old rootfs_list_installed_depends() meta/classes/buildhistory.bbclass |9 - meta/classes/populate_sdk_deb.bbclass |5 - meta/classes/populate_sdk_ipk.bbclass |4 meta/classes/populate_sdk_rpm.bbclass |4 meta/lib/oe/package_manager.py| 31 +++ 5 files changed, 39 insertions(+), 14 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/3] package_manager.py: make list_installed() list pkg dependencies too
list_installed(deps) will now return the package dependencies. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py | 31 +++ 1 file changed, 31 insertions(+) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index ff4f1de..f2c88bb 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -765,7 +765,22 @@ class RpmPM(PackageManager): self.image_rpmlib, symlinks=True) +def _list_pkg_deps(self): +cmd = [bb.utils.which(os.getenv('PATH'), rpmresolve), + -t, self.image_rpmlib] + +try: +output = subprocess.check_output(cmd, stderr=subprocess.STDOUT).strip() +except subprocess.CalledProcessError as e: +bb.fatal(Cannot get the package dependencies. Command '%s' + returned %d:\n%s % (' '.join(cmd), e.returncode, e.output)) + +return output + def list_installed(self, format=None): +if format == deps: +return self._list_pkg_deps() + cmd = self.rpm_cmd + ' --root ' + self.target_rootfs cmd += ' -D _dbpath /var/lib/rpm -qa' cmd += --qf '[%{NAME} %{ARCH} %{VERSION} %{PACKAGEORIGIN}\n]' @@ -1131,6 +1146,9 @@ class OpkgPM(PackageManager): elif format == ver: cmd = %s %s status | %s -v % \ (self.opkg_cmd, self.opkg_args, opkg_query_cmd) +elif format == deps: +cmd = %s %s status | %s % \ +(self.opkg_cmd, self.opkg_args, opkg_query_cmd) else: cmd = %s %s list_installed | cut -d' ' -f1 % \ (self.opkg_cmd, self.opkg_args) @@ -1494,6 +1512,8 @@ class DpkgPM(PackageManager): cmd.append(-f=${Package} ${Package}_${Version}_${Architecture}.deb ${PackageArch}\n) elif format == ver: cmd.append(-f=${Package} ${PackageArch} ${Version}\n) +elif format == deps: +cmd.append(-f=Package: ${Package}\nDepends: ${Depends}\nRecommends: ${Recommends}\n\n) else: cmd.append(-f=${Package}\n) @@ -1514,6 +1534,17 @@ class DpkgPM(PackageManager): tmp_output += %s %s %s\n % (pkg, pkg_file, pkg_arch) output = tmp_output +elif format == deps: +opkg_query_cmd = bb.utils.which(os.getenv('PATH'), opkg-query-helper.py) + +try: +output = subprocess.check_output(echo -e '%s' | %s % + (output, opkg_query_cmd), + stderr=subprocess.STDOUT, + shell=True) +except subprocess.CalledProcessError as e: +bb.fatal(Cannot compute packages dependencies. Command '%s' + returned %d:\n%s % (e.cmd, e.returncode, e.output)) return output -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation
Call the new python routines. [YOCTO #5904] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/buildhistory.bbclass |9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass index ef4135b..01b0082 100644 --- a/meta/classes/buildhistory.bbclass +++ b/meta/classes/buildhistory.bbclass @@ -319,6 +319,12 @@ python buildhistory_list_installed() { with open(pkgs_list_file, 'w') as pkgs_list: pkgs_list.write(list_installed_packages(d, 'file')) + +pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True), + bh_installed_pkgs_deps.txt) + +with open(pkgs_deps_file, 'w') as pkgs_deps: +pkgs_deps.write(list_installed_packages(d, 'deps')) } @@ -338,7 +344,8 @@ buildhistory_get_installed() { # Produce dependency graph # First, quote each name to handle characters that cause issues for dot - rootfs_list_installed_depends | sed 's:\([^| ]*\):\1:g' $1/depends.tmp + cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):\1:g' $1/depends.tmp \ + rm ${WORKDIR}/bh_installed_pkgs_deps.txt # Change delimiter from pipe to - and set style for recommend lines sed -i -e 's:|: - :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' $1/depends.tmp # Add header, sorted and de-duped contents and footer and then delete the temp file -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 3/3] populate_sdk_*.bbclass: remove old rootfs_list_installed_depends()
We're using the python routines now. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/populate_sdk_deb.bbclass |5 - meta/classes/populate_sdk_ipk.bbclass |4 meta/classes/populate_sdk_rpm.bbclass |4 3 files changed, 13 deletions(-) diff --git a/meta/classes/populate_sdk_deb.bbclass b/meta/classes/populate_sdk_deb.bbclass index 462525f..833feca 100644 --- a/meta/classes/populate_sdk_deb.bbclass +++ b/meta/classes/populate_sdk_deb.bbclass @@ -10,8 +10,3 @@ do_populate_sdk[lockfiles] += ${DEPLOY_DIR_DEB}/deb.lock # This will of course only work after rootfs_deb_do_rootfs or populate_sdk_deb has been called DPKG_QUERY_COMMAND = ${STAGING_BINDIR_NATIVE}/dpkg-query --admindir=$INSTALL_ROOTFS_DEB/var/lib/dpkg - -rootfs_list_installed_depends() { - # Cheat here a little bit by using the opkg query helper util - ${DPKG_QUERY_COMMAND} -W -f='Package: ${Package}\nDepends: ${Depends}\nRecommends: ${Recommends}\n\n' | opkg-query-helper.py -} diff --git a/meta/classes/populate_sdk_ipk.bbclass b/meta/classes/populate_sdk_ipk.bbclass index 4c23f05..e2e3523 100644 --- a/meta/classes/populate_sdk_ipk.bbclass +++ b/meta/classes/populate_sdk_ipk.bbclass @@ -2,7 +2,3 @@ do_populate_sdk[depends] += opkg-native:do_populate_sysroot opkg-utils-native:d do_populate_sdk[recrdeptask] += do_package_write_ipk do_populate_sdk[lockfiles] += ${WORKDIR}/ipk.lock - -rootfs_list_installed_depends() { - opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py -} diff --git a/meta/classes/populate_sdk_rpm.bbclass b/meta/classes/populate_sdk_rpm.bbclass index 06605d3..0fdef3b 100644 --- a/meta/classes/populate_sdk_rpm.bbclass +++ b/meta/classes/populate_sdk_rpm.bbclass @@ -16,7 +16,3 @@ do_populate_sdk[recrdeptask] += do_package_write_rpm rpmlibdir = /var/lib/rpm do_populate_sdk[lockfiles] += ${DEPLOY_DIR_RPM}/rpm.lock - -rootfs_list_installed_depends() { - rpmresolve -t $INSTALL_ROOTFS_RPM/${rpmlibdir} -} -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] relocate_sdk.py: Possible bug, /lib64/ld-linux-x86-64.so.2 not relocated
Hi Stefan, On Sat, Mar 01, 2014 at 03:28:09PM +0100, Stefan Agner wrote: Hi, Using top of dylan branch, I generated a SDK using bitbake meta-toolchain. I'm running Arch Linux, but I also see similar issues on Ubuntu 12.04.4 LTS: Some binaries segfault when running them. I discovered, that the dynamic linker of the host is used: $ ldd /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc linux-vdso.so.1 (0x7fffc8f11000) libc.so.6 = /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/../../../lib/libc.so.6 (0x7f4d85eb1000) /lib64/ld-linux-x86-64.so.2 (0x7f4d8625f000) When forcing the dynamic linker of the SDK, the binary works: $ /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/lib/ld-linux-x86-64.so.2 /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc arm-angstrom-linux-gnueabi-gcc: fatal error: no input files compilation terminated. Digging around I found this commit by Jason: 3752a9c6d772b39bbe04d62ef4d3527b4c7198c1 relocate_sdk.py: Fix corruption of sdk binaries It changes the behavior and don't relocates stuff which start with /lib64. This is the correct behavior. We shouldn't relocate binaries that use host's dynamic loader. When I install the SDK with -S (copy the relocate scripts), and remove the condition around line 95, the binaries work as expected. Can you please run the installer script with -R so it doesn't perform any relocation on binaries, and then: readelf -p .interp /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc It looks like the default dynamic loader path of the toolchain binaries start with /lib* or /usr/lib* which is not quite right... It should be: ${SDKPATH}/sysroots/${SDK_SYS}. laurentiu ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/2] fix package archs ordering for RPM
This got messed up by me trying to fix a multilib problem using python sets. Unfortunately sets do not keep the initial ordering and an image might end up containing packages meant for other architectures... So, revert the offending commit and fix the original issue by just checking if a channel was already added or not. laurentiu The following changes since commit 8f36e1ef1afd10594391596391564e8904a840f2: poky.conf: add CentOS 6.5 to SANITY_TESTED_DISTROS (2014-02-28 18:05:47 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b5890_rpm_pkg_arch_selection_logic_fix http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5890_rpm_pkg_arch_selection_logic_fix Laurentiu Palcu (2): Revert lib/oe/package_manager.py: RpmPM: fix issue with multilib builds package_manager.py: RpmPM: don't add smart channel if already added meta/lib/oe/package_manager.py | 19 +++ 1 file changed, 11 insertions(+), 8 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] Revert lib/oe/package_manager.py: RpmPM: fix issue with multilib builds
Converting a list to a set does not keep the order of the items, as they were added, which might result in the wrong packages being installed in the final image... This reverts commit 12f47c23df8c109676f66d580d666a3147f3b046. [YOCTO #5890] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index ff4f1de..6c133c3 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -368,18 +368,19 @@ class RpmPM(PackageManager): self.ml_os_list['default']) # List must be prefered to least preferred order -default_platform_extra = set() -platform_extra = set() +default_platform_extra = list() +platform_extra = list() bbextendvariant = self.d.getVar('BBEXTENDVARIANT', True) or for mlib in self.ml_os_list: for arch in self.ml_prefix_list[mlib]: plt = arch.replace('-', '_') + '-.*-' + self.ml_os_list[mlib] if mlib == bbextendvariant: -default_platform_extra.add(plt) +if plt not in default_platform_extra: +default_platform_extra.append(plt) else: -platform_extra.add(plt) - -platform_extra = platform_extra.union(default_platform_extra) +if plt not in platform_extra: +platform_extra.append(plt) +platform_extra = default_platform_extra + platform_extra self._create_configs(platform, platform_extra) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] package_manager.py: RpmPM: don't add smart channel if already added
Make sure the channel was not already added, before attempting to add. [YOCTO #5890] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py |6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 6c133c3..ee42952 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -604,11 +604,11 @@ class RpmPM(PackageManager): # self._invoke_smart('config --set rpm-log-level=debug') # cmd = 'config --set rpm-log-file=/tmp/smart-debug-logfile' # self._invoke_smart(cmd) - +ch_already_added = [] for canonical_arch in platform_extra: arch = canonical_arch.split('-')[0] arch_channel = os.path.join(self.deploy_dir, arch) -if os.path.exists(arch_channel): +if os.path.exists(arch_channel) and not arch in ch_already_added: bb.note('Note: adding Smart channel %s (%s)' % (arch, channel_priority)) self._invoke_smart('channel --add %s type=rpm-md baseurl=%s -y' @@ -617,6 +617,8 @@ class RpmPM(PackageManager): (arch, channel_priority)) channel_priority -= 5 +ch_already_added.append(arch) + bb.note('adding Smart RPM DB channel') self._invoke_smart('channel --add rpmsys type=rpm-sys -y') -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/2] package_manager.py: RpmPM: don't add smart channel if already added
Hi Mark, On Mon, Mar 03, 2014 at 09:02:35AM -0600, Mark Hatle wrote: On 3/3/14, 3:43 AM, Laurentiu Palcu wrote: Make sure the channel was not already added, before attempting to add. [YOCTO #5890] When generating the rootfs/etc/rpm/platform file, the order should be in the order of highest to lowest priority as well. Smart uses this order (via an RPM API) to help determine scores when two packages from repositories at the same priority (with different architectures) are selected. Thanks for the info. This patchset should restore the behavior to the one you describe. Python's sets messed up the order, so I reverted that patch. laurentiu --Mark Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py |6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 6c133c3..ee42952 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -604,11 +604,11 @@ class RpmPM(PackageManager): # self._invoke_smart('config --set rpm-log-level=debug') # cmd = 'config --set rpm-log-file=/tmp/smart-debug-logfile' # self._invoke_smart(cmd) - +ch_already_added = [] for canonical_arch in platform_extra: arch = canonical_arch.split('-')[0] arch_channel = os.path.join(self.deploy_dir, arch) -if os.path.exists(arch_channel): +if os.path.exists(arch_channel) and not arch in ch_already_added: bb.note('Note: adding Smart channel %s (%s)' % (arch, channel_priority)) self._invoke_smart('channel --add %s type=rpm-md baseurl=%s -y' @@ -617,6 +617,8 @@ class RpmPM(PackageManager): (arch, channel_priority)) channel_priority -= 5 +ch_already_added.append(arch) + bb.note('adding Smart RPM DB channel') self._invoke_smart('channel --add rpmsys type=rpm-sys -y') ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] image.py, rootfs.py, package_manager.py: redirect stderr to stdout when calling check_output()
The following changes since commit 54c2e993ec129563c7ae9f3fdee74ab60da28698: bsps: update H/W reference boards to v3.10.32 (2014-03-02 17:25:30 +) are available in the git repository at: git://git.yoctoproject.org/poky-contrib lpalcu/b5902_errors_not_displayed http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5902_errors_not_displayed Laurentiu Palcu (1): image.py, rootfs.py, package_manager.py: redirect stderr to stdout when calling check_output() meta/lib/oe/image.py |2 +- meta/lib/oe/package_manager.py | 48 +--- meta/lib/oe/rootfs.py |2 +- 3 files changed, 27 insertions(+), 25 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/1] image.py, rootfs.py, package_manager.py: redirect stderr to stdout when calling check_output()
If a command executed with subprocess.check_output() fails, the subprocess.CalledProcessError.output contains only STDOUT and the user needs to check the log.do_rootfs to see any other details. This commit forwards stderr to stdout so that, in case of failure, the entire error output will be displayed in terminal. [YOCTO #5902] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/image.py |2 +- meta/lib/oe/package_manager.py | 48 +--- meta/lib/oe/rootfs.py |2 +- 3 files changed, 27 insertions(+), 25 deletions(-) diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py index 488683e..a03b73e 100644 --- a/meta/lib/oe/image.py +++ b/meta/lib/oe/image.py @@ -11,7 +11,7 @@ def generate_image(arg): (type, create_img_cmd)) try: -subprocess.check_output(create_img_cmd) +subprocess.check_output(create_img_cmd, stderr=subprocess.STDOUT) except subprocess.CalledProcessError as e: return(Error: The image creation script '%s' returned %d:\n%s % (e.cmd, e.returncode, e.output)) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index ff4f1de..90884cd 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -14,7 +14,7 @@ def create_index(arg): try: bb.note(Executing '%s' ... % index_cmd) -subprocess.check_output(index_cmd, shell=True) +subprocess.check_output(index_cmd, stderr=subprocess.STDOUT, shell=True) except subprocess.CalledProcessError as e: return(Index creation command '%s' failed with return code %d:\n%s % (e.cmd, e.returncode, e.output)) @@ -298,7 +298,7 @@ class PackageManager(object): globs] try: bb.note(Installing complementary packages ...) -complementary_pkgs = subprocess.check_output(cmd) +complementary_pkgs = subprocess.check_output(cmd, stderr=subprocess.STDOUT) except subprocess.CalledProcessError as e: bb.fatal(Could not compute complementary packages list. Command '%s' returned %d:\n%s % @@ -387,7 +387,9 @@ class RpmPM(PackageManager): cmd = %s %s %s % (self.smart_cmd, self.smart_opt, args) # bb.note(cmd) try: -complementary_pkgs = subprocess.check_output(cmd, shell=True) +complementary_pkgs = subprocess.check_output(cmd, + stderr=subprocess.STDOUT, + shell=True) # bb.note(complementary_pkgs) return complementary_pkgs except subprocess.CalledProcessError as e: @@ -569,7 +571,7 @@ class RpmPM(PackageManager): self.rpm_cmd, self.target_rootfs) try: -subprocess.check_output(cmd, shell=True) +subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True) except subprocess.CalledProcessError as e: bb.fatal(Create rpm database failed. Command '%s' returned %d:\n%s % (cmd, e.returncode, e.output)) @@ -691,7 +693,7 @@ class RpmPM(PackageManager): cmd = %s %s install --attempt -y %s % \ (self.smart_cmd, self.smart_opt, ' '.join(pkgs)) try: -output = subprocess.check_output(cmd.split()) +output = subprocess.check_output(cmd.split(), stderr=subprocess.STDOUT) bb.note(output) except subprocess.CalledProcessError as e: bb.fatal(Unable to install packages. Command '%s' @@ -721,7 +723,7 @@ class RpmPM(PackageManager): try: bb.note(cmd) -output = subprocess.check_output(cmd, shell=True) +output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True) bb.note(output) except subprocess.CalledProcessError as e: bb.note(Unable to remove packages. Command '%s' @@ -772,7 +774,7 @@ class RpmPM(PackageManager): try: # bb.note(cmd) -tmp_output = subprocess.check_output(cmd, shell=True).strip() +tmp_output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).strip() self._unlock_rpm_db() except subprocess.CalledProcessError as e: bb.fatal(Cannot get the installed packages list. Command '%s' @@ -824,7 +826,7 @@ class RpmPM(PackageManager): # Disable rpmsys channel for the fake install self._invoke_smart('channel --disable rpmsys') -subprocess.check_output(cmd, shell=True) +subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True) with open(self.solution_manifest, 'r') as manifest: for pkg in manifest.read().split('\n'): if '@' in pkg
Re: [OE-core] [RFC][PATCH] Attempt to unify predefined package feed handling
Hi David, My comments/questions below. laurentiu On Thu, Feb 27, 2014 at 02:28:21PM +0100, David Nyström wrote: local.conf defines PACKAGE_FEED_URIS = http://www.feed_repo.org/repo/; should we do it this way or some other way ? [Bug 5407] Signed-off-by: David Nyström david.nyst...@enea.com --- meta/lib/oe/package_manager.py | 76 +- meta/lib/oe/rootfs.py | 15 +++-- 2 files changed, 87 insertions(+), 4 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index d29adac..d4050c6 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -223,6 +223,7 @@ class PackageManager(object): self.d = d self.deploy_dir = None self.deploy_lock = None +self.feed_uris = None I guess you can do: self.feed_uris = self.d.getVar('PACKAGE_FEED_URIS', True) or since we're using the same variable for all backends. Update the package manager package database. @@ -262,6 +263,10 @@ class PackageManager(object): def list_installed(self, format=None): pass +@abstractmethod +def insert_feeds_uris(self): +pass + Install complementary packages based upon the list of currently installed packages e.g. locales, *-dev, *-dbg, etc. This will only attempt to install @@ -339,6 +344,7 @@ class RpmPM(PackageManager): self.providename = providename self.fullpkglist = list() self.deploy_dir = self.d.getVar('DEPLOY_DIR_RPM', True) +self.feed_uris = self.d.getVar('PACKAGE_FEED_URIS', True) can be removed, see above. self.etcrpm_dir = os.path.join(self.target_rootfs, etc/rpm) self.install_dir = os.path.join(self.target_rootfs, install) self.rpm_cmd = bb.utils.which(os.getenv('PATH'), rpm) @@ -358,6 +364,41 @@ class RpmPM(PackageManager): self.ml_prefix_list, self.ml_os_list = self.indexer.get_ml_prefix_and_os_list(arch_var, os_var) + +def insert_feeds_uris(self): shouldn't we check that self.feed_uris != before continuing? +# List must be prefered to least preferred order +channel_priority = 5 +default_platform_extra = set() +platform_extra = set() +bbextendvariant = self.d.getVar('BBEXTENDVARIANT', True) or +for mlib in self.ml_os_list: +for arch in self.ml_prefix_list[mlib]: +plt = arch.replace('-', '_') + '-.*-' + self.ml_os_list[mlib] +if mlib == bbextendvariant: +default_platform_extra.add(plt) +else: +platform_extra.add(plt) +channel_priority += 5 + +platform_extra = platform_extra.union(default_platform_extra) + +for uri in self.feed_uris.split(): +channel_priority *= 2 +uri_iterator = 0 +for canonical_arch in platform_extra: +arch = canonical_arch.split('-')[0] +if arch == noarch or arch == any: +continue + +bb.note('Note: adding Smart channel url%d%s (%s)' % +(uri_iterator, arch, channel_priority)) +self._invoke_smart('channel --add url%d-%s type=rpm-md baseurl=%s/rpm/%s -y' + % (uri_iterator, arch, uri, arch)) +self._invoke_smart('channel --set url%d-%s priority=%d' % + (uri_iterator, arch, channel_priority)) +channel_priority -= 5 +uri_iterator += 1 + ''' Create configs for rpm and smart, and multilib is supported ''' @@ -944,7 +985,7 @@ class OpkgPM(PackageManager): self.deploy_dir = self.d.getVar(DEPLOY_DIR_IPK, True) self.deploy_lock_file = os.path.join(self.deploy_dir, deploy.lock) - +self.feed_uris = self.d.getVar('PACKAGE_FEED_URIS', True) can be removed, see above. self.opkg_cmd = bb.utils.which(os.getenv('PATH'), opkg-cl) self.opkg_args = -f %s -o %s % (self.config_file, target_rootfs) self.opkg_args += self.d.getVar(OPKG_ARGS, True) @@ -1050,6 +1091,24 @@ class OpkgPM(PackageManager): config_file.write(src oe-%s file:%s\n % (arch, pkgs_dir)) +def insert_feeds_uris(self): check self.feed_uris != here? +rootfs_config = os.path.join('%s/etc/opkg/base-feeds.conf' + % self.target_rootfs) + +with open(rootfs_config, w+) as config_file: +priority = 5 apparently this is not used at all in the code below, can be removed. +uri_iterator = 0 +for uri in self.feed_uris.split(): +config_file.write(src/gz url-%d %s/ipk\n % +
Re: [OE-core] [RFC][PATCH] Attempt to unify predefined package feed handling
On Thu, Feb 27, 2014 at 03:35:50PM +0100, David Nyström wrote: On 2014-02-27 15:24, Laurentiu Palcu wrote: Hi David, My comments/questions below. Thank you for your detailed comments on the RFC. I will return with V2, where your comments are addressed when the general approach is Acked. What do you think about the general approach ? I think the approach is ok. I don't know a simpler way than this to add package feeds into the image. I guess it was you who added the insert_feeds_uris stubs in rootfs.py ? Yes, I added the stubs in preparation for this. But I didn't know when/if this will be done. Thanks for taking care of this. laurentiu ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] do_rootfs: Added PACKAGE_FEED_URIS functionality
Reviewed-by: Laurentiu Palcu laurentiu.pa...@intel.com On Thu, Feb 27, 2014 at 09:20:37PM +0100, David Nyström wrote: Adding a common interface to add predefined package manager channels to prebuilt rootfs:es. Adding PACKAGE_FEED_URIS = http://myre.po/repo/, will assume repo directories named (rpm,ipk,deb) as subdirectories and statically add them to the rootfs, using the same PKG_ARCHs as the build which produced the images. Tested with RPM, IPK and DEB. deb feed functionality seem broken, is anyone using this ? Signed-off-by: David Nyström david.c.nyst...@gmail.com Signed-off-by: David Nyström david.nyst...@enea.com --- meta/lib/oe/package_manager.py | 89 +- meta/lib/oe/rootfs.py | 16 ++-- 2 files changed, 92 insertions(+), 13 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index ff4f1de..c930572 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -223,6 +223,7 @@ class PackageManager(object): self.d = d self.deploy_dir = None self.deploy_lock = None +self.feed_uris = self.d.getVar('PACKAGE_FEED_URIS', True) or Update the package manager package database. @@ -262,6 +263,10 @@ class PackageManager(object): def list_installed(self, format=None): pass +@abstractmethod +def insert_feeds_uris(self): +pass + Install complementary packages based upon the list of currently installed packages e.g. locales, *-dev, *-dbg, etc. This will only attempt to install @@ -358,6 +363,46 @@ class RpmPM(PackageManager): self.ml_prefix_list, self.ml_os_list = self.indexer.get_ml_prefix_and_os_list(arch_var, os_var) + +def insert_feeds_uris(self): +if self.feed_uris == : +return + +# List must be prefered to least preferred order +default_platform_extra = set() +platform_extra = set() +bbextendvariant = self.d.getVar('BBEXTENDVARIANT', True) or +for mlib in self.ml_os_list: +for arch in self.ml_prefix_list[mlib]: +plt = arch.replace('-', '_') + '-.*-' + self.ml_os_list[mlib] +if mlib == bbextendvariant: +default_platform_extra.add(plt) +else: +platform_extra.add(plt) + +platform_extra = platform_extra.union(default_platform_extra) + +arch_list = [] +for canonical_arch in platform_extra: +arch = canonical_arch.split('-')[0] +if not os.path.exists(os.path.join(self.deploy_dir, arch)): +continue +arch_list.append(arch) + +uri_iterator = 0 +channel_priority = 10 + 5 * len(self.feed_uris.split()) * len(arch_list) + +for uri in self.feed_uris.split(): +for arch in arch_list: +bb.note('Note: adding Smart channel url%d%s (%s)' % +(uri_iterator, arch, channel_priority)) +self._invoke_smart('channel --add url%d-%s type=rpm-md baseurl=%s/rpm/%s -y' + % (uri_iterator, arch, uri, arch)) +self._invoke_smart('channel --set url%d-%s priority=%d' % + (uri_iterator, arch, channel_priority)) +channel_priority -= 5 +uri_iterator += 1 + ''' Create configs for rpm and smart, and multilib is supported ''' @@ -944,7 +989,6 @@ class OpkgPM(PackageManager): self.deploy_dir = self.d.getVar(DEPLOY_DIR_IPK, True) self.deploy_lock_file = os.path.join(self.deploy_dir, deploy.lock) - self.opkg_cmd = bb.utils.which(os.getenv('PATH'), opkg-cl) self.opkg_args = -f %s -o %s % (self.config_file, target_rootfs) self.opkg_args += self.d.getVar(OPKG_ARGS, True) @@ -1050,6 +1094,29 @@ class OpkgPM(PackageManager): config_file.write(src oe-%s file:%s\n % (arch, pkgs_dir)) +def insert_feeds_uris(self): +if self.feed_uris == : +return + +rootfs_config = os.path.join('%s/etc/opkg/base-feeds.conf' + % self.target_rootfs) + +with open(rootfs_config, w+) as config_file: +uri_iterator = 0 +for uri in self.feed_uris.split(): +config_file.write(src/gz url-%d %s/ipk\n % + (uri_iterator, uri)) + +for arch in self.pkg_archs.split(): +if not os.path.exists(os.path.join(self.deploy_dir, arch)): +continue +bb.note('Note: adding opkg channel url-%s-%d (%s)' % +(arch, uri_iterator, uri
[OE-core] [PATCH 1/1] adt_installer: fix issue with x86_64
When the default tune for x86_64 was changed to core2-64, the environment setup script name did not contain x86 anymore. Hence, the adt_installer failed for x86_64. This commit contains a generic fix and is supposed to work with any kind of machine/tune setting. It's actually extracting the environment script name using 'opkg files meta-environment-MACHINE'. So, no need to do any other sort of searches. We know exactly which is the environment setup script for the specified machine. [YOCTO #5806] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- .../adt-installer/scripts/adt_installer_internal | 61 +++- 1 file changed, 35 insertions(+), 26 deletions(-) diff --git a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal index e3422f5..432ba41 100755 --- a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal +++ b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal @@ -64,6 +64,23 @@ get_sudo_app() fi } +# this function accepts arch_type (x86, x86_64, arm, ppc, mips) as the first +# argument, returning the location of the target rootfs +get_target_rootfs_location() { + [ -z $1 ] return + + arch_type=$1 + # rootfs extraction directory + target_sysroot_var=\$YOCTOADT_TARGET_SYSROOT_LOC_$arch_type + target_sysroot=`eval echo $target_sysroot_var` + + if [ $target_sysroot == ]; then + return + else + echo `readlink -m $target_sysroot` + fi +} + #let us install a qemu-native firstly #installation step 2 @@ -134,6 +151,22 @@ for target_type in $YOCTOADT_TARGETS; do echo_info Installing cross canadian packages for $machine ... $OPKG_INSTALL_NATIVE_CMD packagegroup-cross-canadian-$machine $YOCTOADT_INSTALL_LOG_FILE check_result + + target_sysroot=`get_target_rootfs_location $target_type` + [ -z $target_sysroot ] continue + + # get the environment setup script paths: original (the one before relocation) + # and relocated + env_script_original=`$OPKG_CMD -f $OPKG_CONFIG_FILE -o $NATIVE_INSTALL_DIR files meta-environment-$machine|\ + grep environment-setup` + env_script_relocated=$INSTALL_FOLDER/${env_script_original##*/} + + # opkg will not install packagegroup-cross-canadian package if it was already + # installed. So, the environment script is in one place or the other. + [ -e $env_script_original ] env_script=$env_script_original + [ -e $env_script_relocated ] env_script=$env_script_relocated + + $SUDO sed -i -e s%##SDKTARGETSYSROOT##%$target_sysroot%g $env_script done if [ $YOCTOADT_QEMU == Y ] || [ $YOCTOADT_QEMU = y ]; then @@ -232,15 +265,8 @@ echo_info \nSuccessfully installed selected native ADT! install_target() { -# rootfs extraction directory -target_sysroot_var=\$YOCTOADT_TARGET_SYSROOT_LOC_$1 -target_sysroot=`eval echo $target_sysroot_var` - -if [ $target_sysroot == ]; then - return 0 -else - target_sysroot=`readlink -m $target_sysroot` -fi +target_sysroot=`get_target_rootfs_location $1` +[ -z $target_sysroot ] return 0 target_sysroot_image_var=\$YOCTOADT_TARGET_SYSROOT_IMAGE_$1 target_sysroot_image=`eval echo $target_sysroot_image_var` @@ -261,23 +287,6 @@ SUDO=$(get_sudo_app $target_sysroot) $SUDO scripts/extract_rootfs $sysroot_image_name $target_sysroot $OECORE_NATIVE_SYSROOT $user_inst_type check_result - -echo_info Updating environment script with target sysroot location. -if [ $1 == x86 ]; then - env_filename=`ls $INSTALL_FOLDER/environment-setup-i586*` -else - env_filename=`ls $INSTALL_FOLDER/environment-setup-$1*` -fi - -if [ ! -z $env_filename ]; then - SUDO=$(get_sudo_app $INSTALL_FOLDER) - $SUDO sed -i -e s%##SDKTARGETSYSROOT##%$target_sysroot%g $env_filename -else - echo_info [ADT_INST] Error: Failed to find environment script for arch: $1 - return 1 -fi - - } -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] adt_installer: fix issue with x86_64
The following changes since commit e47afff7d812c74f8091d85facfdf34d6a1b698e: runqemu: Use readlink instead of realpath (2014-02-25 21:29:53 +) are available in the git repository at: git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib lpalcu/b5806_adt_installer_fails_for_xf86_64 for you to fetch changes up to 72cf90ecf51d88becfaf037b6233c78e8c6e4a9c: adt_installer: fix issue with x86_64 (2014-02-26 15:36:44 +0200) Laurentiu Palcu (1): adt_installer: fix issue with x86_64 .../adt-installer/scripts/adt_installer_internal | 61 +++- 1 file changed, 35 insertions(+), 26 deletions(-) Laurentiu Palcu (1): adt_installer: fix issue with x86_64 .../adt-installer/scripts/adt_installer_internal | 61 +++- 1 file changed, 35 insertions(+), 26 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/1] package_manager.py: create index files for all backends in PACKAGE_CLASSES
The following changes since commit 72576e2d04dc8b43ae0d7f31cddd74318f52103e: bitbake: hob: replace the use of hob-toolchain with populate_sdk task (2014-02-25 08:06:11 +) are available in the git repository at: git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib lpalcu/b5827_package_index_fix for you to fetch changes up to ae8627a6eb1d76b1535a222d9011093bdee79cdb: package_manager.py: create index files for all backends in PACKAGE_CLASSES (2014-02-25 14:25:35 +0200) Laurentiu Palcu (1): package_manager.py: create index files for all backends in PACKAGE_CLASSES meta/lib/oe/package_manager.py | 24 +++- 1 file changed, 15 insertions(+), 9 deletions(-) Laurentiu Palcu (1): package_manager.py: create index files for all backends in PACKAGE_CLASSES meta/lib/oe/package_manager.py | 24 +++- 1 file changed, 15 insertions(+), 9 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/1] package_manager.py: create index files for all backends in PACKAGE_CLASSES
The previous implementation was checking IMAGE_PKGTYPE and created the index files just for the backend used to create the image. Apparently, 'bitbake package-index' should attempt to create the index files for all backends specified in PACKAGE_CLASSES. [YOCTO #5827] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py | 24 +++- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index d29adac..ff4f1de 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -1519,19 +1519,25 @@ class DpkgPM(PackageManager): def generate_index_files(d): -img_type = d.getVar('IMAGE_PKGTYPE', True) +classes = d.getVar('PACKAGE_CLASSES', True).replace(package_, ).split() + +indexer_map = { +rpm: (RpmIndexer, d.getVar('DEPLOY_DIR_RPM', True)), +ipk: (OpkgIndexer, d.getVar('DEPLOY_DIR_IPK', True)), +deb: (DpkgIndexer, d.getVar('DEPLOY_DIR_DEB', True)) +} result = None -if img_type == rpm: -result = RpmIndexer(d, d.getVar('DEPLOY_DIR_RPM', True)).write_index() -elif img_type == ipk: -result = OpkgIndexer(d, d.getVar('DEPLOY_DIR_IPK', True)).write_index() -elif img_type == deb: -result = DpkgIndexer(d, d.getVar('DEPLOY_DIR_DEB', True)).write_index() +for pkg_class in classes: +if not pkg_class in indexer_map: +continue + +if os.path.exists(indexer_map[pkg_class][1]): +result = indexer_map[pkg_class][0](d, indexer_map[pkg_class][1]).write_index() -if result is not None: -bb.fatal(result) +if result is not None: +bb.fatal(result) if __name__ == __main__: -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH V3 0/5] manifest.py/package_manager.py/rootfs.py: support ipk incremental image generation
Reviewed-by: Laurentiu Palcu laurentiu.palcu.intel.com laurentiu On Fri, Feb 21, 2014 at 02:38:16PM +0800, Hongxu Jia wrote: Change in V2: - Move dump_install_solution to Manifest class as create_full; - Move load_old_install_solution to Manifest class as parse_full_manifest; - Handle the output of dummy install in Python rather than shell; - Fix typos - Rename _file_duplicate with _file_equal to avoid confusion; Change in V3: - While create_full calls dummy_install, pass an instance to PM as an argument rather than instantiate another PM object; - Rename _create_incremental with _remove_extra_packages; - Move the call of _remove_extra_packages below the call of self.pm.handle_bad_recommendations(); - Remove redundant self.pm.update in _remove_extra_packages; - Rename _remove_existing_image with _remove_old_rootfs; Test Cases - Case 1: Rebuild in place 1. Edit local.conf, enable multilib and ipk incremental image generation ... 46 MACHINE ?= qemux86-64 254 require conf/multilib.conf 255 MULTILIBS = multilib:lib32 256 DEFAULTTUNE_virtclass-multilib-lib32 = x86 257 258 IMAGE_INSTALL_append = lib32-gzip 260 IMAGE_FEATURES_append = package-management ssh-server-dropbear 262 263 INC_IPK_IMAGE_GEN = 1 ... bb core-image-minimal 2. Edit local.conf, add some spaces in IMAGE_INSTALL_append vim local.conf ... 258 IMAGE_INSTALL_append = lib32-gzip ... 3. bb core-image-minimal, the image should not be rebuilt vim log.do_rootfs: ... 16 NOTE: Package packagegroup-core-ssh-dropbear (1.0-r1) installed in root is up to date. 17 Package opkg-collateral (1.0-r2) installed in root is up to date. 18 Package run-postinsts (1.0-r9) installed in root is up to date. 19 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. 20 Package opkg (1:0.2.0-r0) installed in root is up to date. 21 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date. ... 53 NOTE: Package lib32-gzip (1.6-r7) installed in root is up to date. ... vim installed_pkgs.txt ... lib32-gzip x86 ... 4. runqemu qemux86-64, ssh could work and lib32-gzip is installed On host: runqemu qemux86-64 ssh root@192.168.7.2 and run 'opkg-cl list-installed' ... lib32-gzip - 1.6-r7 ... - Case 2: Decremental Rebuild 1. Edit local.conf, enable multilib and ipk incremental image generation ... 46 MACHINE ?= qemux86-64 254 require conf/multilib.conf 255 MULTILIBS = multilib:lib32 256 DEFAULTTUNE_virtclass-multilib-lib32 = x86 257 258 IMAGE_INSTALL_append = lib32-gzip openssl 260 IMAGE_FEATURES_append = package-management ssh-server-dropbear 262 263 INC_IPK_IMAGE_GEN = 1 ... bb core-image-minimal 2. Edit local.conf, remove 'lib32-gzip openssl' vim local.conf ... 258 # IMAGE_INSTALL_append = lib32-gzip openssl ... 3. bb core-image-minimal, lib32-gzip and openssl should be removed. vim log.do_rootfs: ... 20 NOTE: Removing package lib32-eglibc from root... 21 Removing package lib32-gzip from root... 22 Removing package lib32-update-alternatives-opkg from root... 23 Removing package libcrypto1.0.0 from root... 24 Removing package libssl1.0.0 from root... 25 Removing package openssl from root... 26 Removing package openssl-conf from root... ... 31 Package opkg-collateral (1.0-r2) installed in root is up to date. 32 Package run-postinsts (1.0-r9) installed in root is up to date. 33 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. 34 Package opkg (1:0.2.0-r0) installed in root is up to date. 35 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date. ... - Case 3: Incremental Rebuild 1. Edit local.conf, enable multilib and ipk incremental image generation ... 46 MACHINE ?= qemux86-64 254 require conf/multilib.conf 255 MULTILIBS = multilib:lib32 256 DEFAULTTUNE_virtclass-multilib-lib32 = x86 257 258 # IMAGE_INSTALL_append = lib32-gzip openssl 260 IMAGE_FEATURES_append = package-management ssh-server-dropbear 262 263 INC_IPK_IMAGE_GEN = 1 ... bb core-image-minimal 2. Edit local.conf, append 'lib32-gzip openssl' to IMAGE_INSTALL vim local.conf ... 258 IMAGE_INSTALL_append = lib32-gzip openssl ... 3. bb core-image-minimal, lib32-gzip and openssl should be added. vim log.do_rootfs: ... 20 NOTE: Package packagegroup-core-ssh-dropbear (1.0-r1) installed in root is up to date. 21 Package opkg-collateral (1.0-r2) installed in root is up to date. 22 Package run-postinsts (1.0-r9) installed in root is up to date. 23 Installing openssl (1.0.1e-r15.0) to root... ... 31 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. 32 Package opkg (1:0.2.0-r0) installed in root is up to date. 33 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date. ... 54 NOTE
Re: [OE-core] [PATCH 1/5] manifest.py: add create_full for OpkgManifest class
On Thu, Feb 20, 2014 at 03:06:51PM +0800, Hongxu Jia wrote: The function create_full creates the manifest after the package in initial manifest has been dummy installed. It lists all *to be installed* packages. There is no real installation, just a test. [YOCTO #1894] Signed-off-by: Hongxu Jia hongxu@windriver.com --- meta/lib/oe/manifest.py | 65 - 1 file changed, 64 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/manifest.py b/meta/lib/oe/manifest.py index a4bc04b..6e40a9f 100644 --- a/meta/lib/oe/manifest.py +++ b/meta/lib/oe/manifest.py @@ -1,6 +1,8 @@ from abc import ABCMeta, abstractmethod +from oe.package_manager import * import os import re +import bb class Manifest(object): @@ -69,6 +71,7 @@ class Manifest(object): self.initial_manifest = os.path.join(self.manifest_dir, %s_initial_manifest % manifest_type) self.final_manifest = os.path.join(self.manifest_dir, %s_final_manifest % manifest_type) +self.full_manifest = os.path.join(self.manifest_dir, %s_full_manifest % manifest_type) # packages in the following vars will be split in 'must install' and # 'multilib' @@ -128,6 +131,15 @@ class Manifest(object): pass +This creates the manifest after the package in initial manifest has been +dummy installed. It lists all *to be installed* packages. There is no real +installation, just a test. + +@abstractmethod +def create_full(self): +pass + + The following function parses an initial manifest and returns a dictionary object with the must install, attempt only, multilib and language packages. @@ -158,6 +170,22 @@ class Manifest(object): return pkgs +''' +This following function parses a full manifest and return a list +object with packages. +''' +def parse_full_manifest(self): +installed_pkgs = list() +if not os.path.exists(self.full_manifest): +bb.note('full manifest not exist') +return installed_pkgs + +with open(self.full_manifest, 'r') as manifest: +for pkg in manifest.read().split('\n'): +installed_pkgs.append(pkg.strip()) + +return installed_pkgs + class RpmManifest(Manifest): @@ -202,10 +230,12 @@ class RpmManifest(Manifest): for pkg in pkgs[pkg_type].split(): manifest.write(%s,%s\n % (pkg_type, pkg)) - def create_final(self): pass +def create_full(self): +pass + class OpkgManifest(Manifest): @@ -253,6 +283,36 @@ class OpkgManifest(Manifest): def create_final(self): pass +def create_full(self): +if not os.path.exists(self.initial_manifest): +self.create_initial() + +initial_manifest = self.parse_initial_manifest() +pkgs_to_install = list() +for pkg_type in initial_manifest: +pkgs_to_install += initial_manifest[pkg_type] +if len(pkgs_to_install) == 0: +return + +image_rootfs = self.d.expand(${T}/opkg) +opkg_conf = self.d.getVar(IPKGCONF_TARGET, True) +pkg_archs = self.d.getVar(ALL_MULTILIB_PACKAGE_ARCHS, True) +pm = OpkgPM(self.d, image_rootfs, opkg_conf, pkg_archs) Do we really need to instantiate another PM object here? Can't the caller pass an instance to it? After all, dummy_install() is not supposed to alter the metadata, am I wrong? So, in this case, I believe the caller can safely pass an instance to PM as an argument to the function. + +pm.update() +pm.handle_bad_recommendations() +output = pm.dummy_install(pkgs_to_install) + +with open(self.full_manifest, 'w+') as manifest: +pkg_re = re.compile('^Installing ([^ ]+) [^ ].*') +for line in set(output.split('\n')): +m = pkg_re.match(line) +if m: +manifest.write(m.group(1) + '\n') + +bb.utils.remove(image_rootfs, True) +return + class DpkgManifest(Manifest): def create_initial(self): @@ -272,6 +332,9 @@ class DpkgManifest(Manifest): def create_final(self): pass +def create_full(self): +pass + def create_manifest(d, final_manifest=False, manifest_dir=None, manifest_type=Manifest.MANIFEST_TYPE_IMAGE): -- 1.8.1.2 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] package_manager.py: support ipk incremental image generation
On Thu, Feb 20, 2014 at 03:06:52PM +0800, Hongxu Jia wrote: Add the following three functions to OpkgPM class: - The 'dummy_install' is used to dummy install pkgs, and returns the log of output; - The 'backup_packaging_data' is used to back up the current opkg database; - The 'recover_packaging_data' is used to recover the opkg database which backed up by the previous image creation; Tweak 'remove' function in OpkgPM class, which the options for remove with dependencies was incorrect. Tweak 'handle_bad_recommendations' function in OpkgPM class: - Fix none value check; - Add the existance check of opkg status file; - Fix the log format typo; [YOCTO #1894] Signed-off-by: Hongxu Jia hongxu@windriver.com --- meta/lib/oe/package_manager.py | 66 +- 1 file changed, 59 insertions(+), 7 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 6dc8fbd..eb96727 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -934,12 +934,13 @@ class RpmPM(PackageManager): class OpkgPM(PackageManager): -def __init__(self, d, target_rootfs, config_file, archs): +def __init__(self, d, target_rootfs, config_file, archs, task_name='target'): super(OpkgPM, self).__init__(d) self.target_rootfs = target_rootfs self.config_file = config_file self.pkg_archs = archs +self.task_name = task_name self.deploy_dir = self.d.getVar(DEPLOY_DIR_IPK, True) self.deploy_lock_file = os.path.join(self.deploy_dir, deploy.lock) @@ -956,6 +957,10 @@ class OpkgPM(PackageManager): bb.utils.mkdirhier(self.opkg_dir) +self.saved_opkg_dir = self.d.expand('${T}/saved/%s' % self.task_name) +if not os.path.exists(self.d.expand('${T}/saved')): +bb.utils.mkdirhier(self.d.expand('${T}/saved')) + if (self.d.getVar('BUILD_IMAGES_FROM_FEEDS', True) or ) != 1: self._create_config() else: @@ -1075,7 +1080,9 @@ class OpkgPM(PackageManager): try: bb.note(Installing the following packages: %s % ' '.join(pkgs)) -subprocess.check_output(cmd.split()) +bb.note(cmd) +output = subprocess.check_output(cmd.split()) +bb.note(output) except subprocess.CalledProcessError as e: (bb.fatal, bb.note)[attempt_only](Unable to install packages. Command '%s' returned %d:\n%s % @@ -1083,14 +1090,16 @@ class OpkgPM(PackageManager): def remove(self, pkgs, with_dependencies=True): if with_dependencies: -cmd = %s %s remove %s % \ +cmd = %s %s --force-depends --force-remove --force-removal-of-dependent-packages remove %s % \ (self.opkg_cmd, self.opkg_args, ' '.join(pkgs)) else: cmd = %s %s --force-depends remove %s % \ (self.opkg_cmd, self.opkg_args, ' '.join(pkgs)) try: -subprocess.check_output(cmd.split()) +bb.note(cmd) +output = subprocess.check_output(cmd.split()) +bb.note(output) except subprocess.CalledProcessError as e: bb.fatal(Unable to remove packages. Command '%s' returned %d:\n%s % (e.cmd, e.returncode, e.output)) @@ -1146,12 +1155,17 @@ class OpkgPM(PackageManager): return output def handle_bad_recommendations(self): -bad_recommendations = self.d.getVar(BAD_RECOMMENDATIONS, True) -if bad_recommendations is None: +bad_recommendations = self.d.getVar(BAD_RECOMMENDATIONS, True) or +if bad_recommendations.strip() == : return status_file = os.path.join(self.opkg_dir, status) +# If status file existed, it means the bad recommendations has already +# been handled +if os.path.exists(status_file): +return + cmd = %s %s info % (self.opkg_cmd, self.opkg_args) with open(status_file, w+) as status: @@ -1165,7 +1179,7 @@ class OpkgPM(PackageManager): returned %d:\n%s % (pkg_info, e.returncode, e.output)) if output == : -bb.note(Requested ignored recommendation $i is +bb.note(Requested ignored recommendation %s is I already saw a patch fixing this on the mailing list. You might need to rebase. not a package % pkg) continue @@ -1175,6 +1189,44 @@ class OpkgPM(PackageManager): else: status.write(line + \n) +''' +The following function dummy installs pkgs and returns the log of output. +''' +def dummy_install(self, pkgs): +
Re: [OE-core] [PATCH 3/5] rootfs.py: support ipk incremental image generation
On Thu, Feb 20, 2014 at 03:06:53PM +0800, Hongxu Jia wrote: The incremental image generation is based on the previous existing image, adds new packages, upgrades existing packages, and removes unused packages. [YOCTO #1894] Signed-off-by: Hongxu Jia hongxu@windriver.com --- meta/lib/oe/rootfs.py | 52 --- 1 file changed, 49 insertions(+), 3 deletions(-) diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py index 3bcb812..93106c2 100644 --- a/meta/lib/oe/rootfs.py +++ b/meta/lib/oe/rootfs.py @@ -438,13 +438,25 @@ class OpkgRootfs(Rootfs): def __init__(self, d, manifest_dir): super(OpkgRootfs, self).__init__(d) -bb.utils.remove(self.image_rootfs, True) -bb.utils.remove(self.d.getVar('MULTILIB_TEMP_ROOTFS', True), True) self.manifest = OpkgManifest(d, manifest_dir) self.opkg_conf = self.d.getVar(IPKGCONF_TARGET, True) self.pkg_archs = self.d.getVar(ALL_MULTILIB_PACKAGE_ARCHS, True) -self.pm = OpkgPM(d, self.image_rootfs, self.opkg_conf, self.pkg_archs) +self.inc_opkg_image_gen = self.d.getVar('INC_IPK_IMAGE_GEN', True) or +if self.inc_opkg_image_gen != '1': +bb.utils.remove(self.image_rootfs, True) +self.pm = OpkgPM(d, + self.image_rootfs, + self.opkg_conf, + self.pkg_archs) +else: +self.pm = OpkgPM(d, + self.image_rootfs, + self.opkg_conf, + self.pkg_archs) +self.pm.recover_packaging_data() + +bb.utils.remove(self.d.getVar('MULTILIB_TEMP_ROOTFS', True), True) This function was reused from the old implementation. @@ -508,6 +520,34 @@ class OpkgRootfs(Rootfs): self._multilib_sanity_test(dirs) +''' +While ipk incremental image generation is enabled, it will remove the +unneeded pkgs by comparing the old full manifest in previous existing +image and the new full manifest in the current image. +''' +def _create_incremental(self, pkgs_initial_install): Can we name this function _remove_extra_packages()? Because it doesn't actually install anything... it just removes packages, correct? +if self.inc_opkg_image_gen == 1: +# Parse full manifest in previous existing image creation session +old_full_manifest = self.manifest.parse_full_manifest() + +# Create full manifest for the current image session, the old one +# will be replaced by the new one. +self.manifest.create_full() + +# Parse full manifest in current image creation session +new_full_manifest = self.manifest.parse_full_manifest() + +pkg_to_remove = list() +for pkg in old_full_manifest: +if pkg not in new_full_manifest: +pkg_to_remove.append(pkg) + +self.pm.update() this update could be removed, if we move the function call below self.pm.handle_bad_recommendations(). See below. + +if pkg_to_remove != []: +bb.note('decremental removed: %s' % ' '.join(pkg_to_remove)) +self.pm.remove(pkg_to_remove) + def _create(self): pkgs_to_install = self.manifest.parse_initial_manifest() opkg_pre_process_cmds = self.d.getVar('OPKG_PREPROCESS_COMMANDS', True) @@ -518,6 +558,9 @@ class OpkgRootfs(Rootfs): if (self.d.getVar('BUILD_IMAGES_FROM_FEEDS', True) or ) != 1: self.pm.write_index() +if self.inc_opkg_image_gen == 1: +self._create_incremental(pkgs_to_install) Why not move this piece lower, below self.pm.handle_bad_recommendations()? Correct me if I'm wrong, but I believe the rootfs pre process commands should be executed before we do anything on the rootfs generation. + execute_pre_post_process(self.d, opkg_pre_process_cmds) self.pm.update() @@ -540,6 +583,9 @@ class OpkgRootfs(Rootfs): execute_pre_post_process(self.d, opkg_post_process_cmds) execute_pre_post_process(self.d, rootfs_post_install_cmds) +if self.inc_opkg_image_gen == 1: +self.pm.backup_packaging_data() + def _get_delayed_postinsts(self): pkg_list = [] status_file = os.path.join(self.image_rootfs, -- 1.8.1.2 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 5/5] rootfs.py: support BAD_RECOMMENDATIONS for ipk incremental image generation
On Thu, Feb 20, 2014 at 03:06:55PM +0800, Hongxu Jia wrote: While incremental image generation enabled and the previous image is existed, if BAD_RECOMMENDATIONS is changed, the operation on the existing image is complicated, so remove the existing image in this situation. The same with PACKAGE_EXCLUDE and NO_RECOMMENDATIONS. [YOCTO #1894] Signed-off-by: Hongxu Jia hongxu@windriver.com --- meta/lib/oe/rootfs.py | 29 - 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py index 6120bce..80be490 100644 --- a/meta/lib/oe/rootfs.py +++ b/meta/lib/oe/rootfs.py @@ -445,7 +445,7 @@ class OpkgRootfs(Rootfs): self.pkg_archs = self.d.getVar(ALL_MULTILIB_PACKAGE_ARCHS, True) self.inc_opkg_image_gen = self.d.getVar('INC_IPK_IMAGE_GEN', True) or -if self.inc_opkg_image_gen != '1': +if self._remove_existing_image(): bb.utils.remove(self.image_rootfs, True) self.pm = OpkgPM(d, self.image_rootfs, @@ -595,6 +595,33 @@ class OpkgRootfs(Rootfs): bb.note('decremental removed: %s' % ' '.join(pkg_to_remove)) self.pm.remove(pkg_to_remove) +''' +Compare with previous existing image creation, if some conditions +triggered, the previous existing image should be removed. +The conditions include any of 'PACKAGE_EXCLUDE, NO_RECOMMENDATIONS +and BAD_RECOMMENDATIONS' has been changed. +''' +def _remove_existing_image(self): Since we're not removing the image, but the old rootfs, a better name for this would be: _remove_old_rootfs(). +if self.inc_opkg_image_gen != 1: +return True + +vars_list_file = self.d.expand('${T}/vars_list') + +old_vars_list = +if os.path.exists(vars_list_file): +old_vars_list = open(vars_list_file, 'r+').read() + +new_vars_list = '%s:%s:%s\n' % \ +((self.d.getVar('BAD_RECOMMENDATIONS', True) or '').strip(), + (self.d.getVar('NO_RECOMMENDATIONS', True) or '').strip(), + (self.d.getVar('PACKAGE_EXCLUDE', True) or '').strip()) +open(vars_list_file, 'w+').write(new_vars_list) + +if old_vars_list != new_vars_list: +return True + +return False + def _create(self): pkgs_to_install = self.manifest.parse_initial_manifest() opkg_pre_process_cmds = self.d.getVar('OPKG_PREPROCESS_COMMANDS', True) -- 1.8.1.2 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/3] package_manager.py: fix installed package list creation for Opkg/Dpkg
Small error in the package list creation routine. Buildhistory was supposed to use this but was never called. Hence, it escaped tests... Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py |6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 6dc8fbd..a3c0a8e 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -1134,7 +1134,8 @@ class OpkgPM(PackageManager): if format == file: tmp_output = -for pkg, pkg_file, pkg_arch in tuple(output.split('\n')): +for line in output.split('\n'): +pkg, pkg_file, pkg_arch = line.split() full_path = os.path.join(self.deploy_dir, pkg_arch, pkg_file) if os.path.exists(full_path): tmp_output += %s %s %s\n % (pkg, full_path, pkg_arch) @@ -1435,7 +1436,8 @@ class DpkgPM(PackageManager): if format == file: tmp_output = -for pkg, pkg_file, pkg_arch in tuple(output.split('\n')): +for line in tuple(output.split('\n')): +pkg, pkg_file, pkg_arch = line.split() full_path = os.path.join(self.deploy_dir, pkg_arch, pkg_file) if os.path.exists(full_path): tmp_output += %s %s %s\n % (pkg, full_path, pkg_arch) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/3] Fix buildhistory problem
Tested with all backends. Fixed some other issues in the process... Buildhistory should be fine now. laurentiu The following changes since commit 54562006c1327c5b99daa4cc05a3ba7e38412da1: image_types.bbclass: Fix tar IMAGE_CMD to not change directories (2014-02-18 08:38:52 +) are available in the git repository at: git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib lpalcu/buildhistory_fix for you to fetch changes up to 15ce4b01408e1ec91655103191c4c9436d894597: populate_sdk_*.bbclass: remove left over bash routines (2014-02-19 12:45:51 +0200) Laurentiu Palcu (3): package_manager.py: fix installed package list creation for Opkg/Dpkg buildhistory.bbclass: fix creation of installed packages list populate_sdk_*.bbclass: remove left over bash routines meta/classes/buildhistory.bbclass | 21 + meta/classes/populate_sdk_deb.bbclass | 21 - meta/classes/populate_sdk_ipk.bbclass | 20 meta/lib/oe/package_manager.py|6 -- 4 files changed, 21 insertions(+), 47 deletions(-) Laurentiu Palcu (3): package_manager.py: fix installed package list creation for Opkg/Dpkg buildhistory.bbclass: fix creation of installed packages list populate_sdk_*.bbclass: remove left over bash routines meta/classes/buildhistory.bbclass | 21 + meta/classes/populate_sdk_deb.bbclass | 21 - meta/classes/populate_sdk_ipk.bbclass | 20 meta/lib/oe/package_manager.py|6 -- 4 files changed, 21 insertions(+), 47 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/3] buildhistory.bbclass: fix creation of installed packages list
Call the new python routines to create the packages list. [YOCTO #5831] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/buildhistory.bbclass | 21 + 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass index 0033b5a..ef4135b 100644 --- a/meta/classes/buildhistory.bbclass +++ b/meta/classes/buildhistory.bbclass @@ -311,13 +311,23 @@ def write_pkghistory(pkginfo, d): if os.path.exists(filevarpath): os.unlink(filevarpath) +python buildhistory_list_installed() { +from oe.rootfs import list_installed_packages + +pkgs_list_file = os.path.join(d.getVar('WORKDIR', True), + bh_installed_pkgs.txt) + +with open(pkgs_list_file, 'w') as pkgs_list: +pkgs_list.write(list_installed_packages(d, 'file')) +} + buildhistory_get_installed() { mkdir -p $1 # Get list of installed packages pkgcache=$1/installed-packages.tmp - list_installed_packages file | sort $pkgcache + cat ${WORKDIR}/bh_installed_pkgs.txt | sort $pkgcache rm ${WORKDIR}/bh_installed_pkgs.txt cat $pkgcache | awk '{ print $1 }' $1/installed-package-names.txt if [ -s $pkgcache ] ; then @@ -452,13 +462,16 @@ END } # By prepending we get in before the removal of packaging files -ROOTFS_POSTPROCESS_COMMAND =+ buildhistory_get_image_installed ; +ROOTFS_POSTPROCESS_COMMAND =+ buildhistory_list_installed ;\ +buildhistory_get_image_installed ; IMAGE_POSTPROCESS_COMMAND += buildhistory_get_imageinfo ; # We want these to be the last run so that we get called after complementary package installation -POPULATE_SDK_POST_TARGET_COMMAND_append = buildhistory_get_sdk_installed_target ; -POPULATE_SDK_POST_HOST_COMMAND_append = buildhistory_get_sdk_installed_host ; +POPULATE_SDK_POST_TARGET_COMMAND_append = buildhistory_list_installed ;\ + buildhistory_get_sdk_installed_target ; +POPULATE_SDK_POST_HOST_COMMAND_append = buildhistory_list_installed ;\ + buildhistory_get_sdk_installed_host ; SDK_POSTPROCESS_COMMAND += buildhistory_get_sdkinfo ; -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 3/3] populate_sdk_*.bbclass: remove left over bash routines
The list_installed_packages bash routine is no longer needed here. We've got a python replacement. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/populate_sdk_deb.bbclass | 21 - meta/classes/populate_sdk_ipk.bbclass | 20 2 files changed, 41 deletions(-) diff --git a/meta/classes/populate_sdk_deb.bbclass b/meta/classes/populate_sdk_deb.bbclass index d03b055..462525f 100644 --- a/meta/classes/populate_sdk_deb.bbclass +++ b/meta/classes/populate_sdk_deb.bbclass @@ -11,27 +11,6 @@ do_populate_sdk[lockfiles] += ${DEPLOY_DIR_DEB}/deb.lock # This will of course only work after rootfs_deb_do_rootfs or populate_sdk_deb has been called DPKG_QUERY_COMMAND = ${STAGING_BINDIR_NATIVE}/dpkg-query --admindir=$INSTALL_ROOTFS_DEB/var/lib/dpkg -list_installed_packages() { - if [ $1 = arch ] ; then - # Here we want the PACKAGE_ARCH not the deb architecture - ${DPKG_QUERY_COMMAND} -W -f='${Package} ${PackageArch}\n' - elif [ $1 = file ] ; then - ${DPKG_QUERY_COMMAND} -W -f='${Package} ${Package}_${Version}_${Architecture}.deb ${PackageArch}\n' | while read pkg pkgfile pkgarch - do - fullpath=`find ${DEPLOY_DIR_DEB} -name $pkgfile || true` - if [ $fullpath = ] ; then - echo $pkg $pkgfile $pkgarch - else - echo $pkg $fullpath $pkgarch - fi - done - elif [ $1 = ver ] ; then - ${DPKG_QUERY_COMMAND} -W -f='${Package} ${PackageArch} ${Version}\n' - else - ${DPKG_QUERY_COMMAND} -W -f='${Package}\n' - fi -} - rootfs_list_installed_depends() { # Cheat here a little bit by using the opkg query helper util ${DPKG_QUERY_COMMAND} -W -f='Package: ${Package}\nDepends: ${Depends}\nRecommends: ${Recommends}\n\n' | opkg-query-helper.py diff --git a/meta/classes/populate_sdk_ipk.bbclass b/meta/classes/populate_sdk_ipk.bbclass index f51a22c..4c23f05 100644 --- a/meta/classes/populate_sdk_ipk.bbclass +++ b/meta/classes/populate_sdk_ipk.bbclass @@ -3,26 +3,6 @@ do_populate_sdk[recrdeptask] += do_package_write_ipk do_populate_sdk[lockfiles] += ${WORKDIR}/ipk.lock -list_installed_packages() { - if [ $1 = arch ] ; then - opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py -a - elif [ $1 = file ] ; then - opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py -f | while read pkg pkgfile pkgarch - do - fullpath=`find ${DEPLOY_DIR_IPK} -name $pkgfile || true` - if [ $fullpath = ] ; then - echo $pkg $pkgfile $pkgarch - else - echo $pkg $fullpath $pkgarch - fi - done - elif [ $1 = ver ] ; then - opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py -v - else - opkg-cl ${OPKG_ARGS} list_installed | awk '{ print $1 }' - fi -} - rootfs_list_installed_depends() { opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py } -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] update-rc.d.bbclass: fix inhibit check
On Wed, Feb 19, 2014 at 06:59:26PM +0800, Kai Kang wrote: In update-rc.d.bbclass it checks variable INITSCRIPT_PACKAGES to avoid inherit this class. But it is wrong logic to check INITSCRIPT_PACKAGES. When 'sysvinit' is in 'DISTRO_FEATURES', INITSCRIPT_PACKAGES will not be checked. s/INITSCRIPT_PACKAGES/INHIBIT_UPDATERCD_BBCLASS/ above will make more sense, I guess. Because, as I see it, that 'if' will evaluate to True if 'sysvinit' is in DISTRO_FEATURES, irrespective of how INHIBIT_UPDATERCD_BBCLASS is set... which is a little bit wrong. So, I think you just need to adjust the commit message. laurentiu Replace 'or' with 'and' to fix it. Signed-off-by: Kai Kang kai.k...@windriver.com --- meta/classes/update-rc.d.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/update-rc.d.bbclass b/meta/classes/update-rc.d.bbclass index 4b92d8d..0ac2af7 100644 --- a/meta/classes/update-rc.d.bbclass +++ b/meta/classes/update-rc.d.bbclass @@ -117,7 +117,7 @@ python populate_packages_updatercd () { # Check that this class isn't being inhibited (generally, by # systemd.bbclass) before doing any work. -if oe.utils.contains('DISTRO_FEATURES', 'sysvinit', True, False, d) or \ +if oe.utils.contains('DISTRO_FEATURES', 'sysvinit', True, False, d) and \ not d.getVar(INHIBIT_UPDATERCD_BBCLASS, True): pkgs = d.getVar('INITSCRIPT_PACKAGES', True) if pkgs == None: -- 1.8.1.2 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 3/6] rootfs.py: Check for LDCONFIGDEPEND being empty string
Hi Khem, On Mon, Feb 17, 2014 at 11:34:15AM -0800, Khem Raj wrote: We override LDCONFIGDEPEND to be empty string for uclibc however the current check is for it being None as a result the function is still executed but ldconfig-native is not built as dependency for rootfs when building with uclibc Fixes errors like below File: '/home/kraj/work/angstrom-repo/sources/openembedded-core/meta/lib/oe/rootfs.py', lineno: 191, function: _run_ldconfig 0187:def _run_ldconfig(self): 0188:if self.d.getVar('LDCONFIGDEPEND', True) is not None: 0189:bb.note(Executing: ldconfig -r + self.image_rootfs + -c new -v) 0190:self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c', *** 0191: 'new', '-v']) Signed-off-by: Khem Raj raj.k...@gmail.com --- meta/lib/oe/rootfs.py |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py index d149ca3..3bcb812 100644 --- a/meta/lib/oe/rootfs.py +++ b/meta/lib/oe/rootfs.py @@ -185,7 +185,7 @@ class Rootfs(object): self._handle_intercept_failure(registered_pkgs) def _run_ldconfig(self): -if self.d.getVar('LDCONFIGDEPEND', True) is not None: +if self.d.getVar('LDCONFIGDEPEND', True) != : May I suggest: if (self.d.getVar('LDCONFIGDEPEND', True) or ) != : here? Otherwise, the condition will be true even if LDCONFIGDEPEND is None. laurentiu bb.note(Executing: ldconfig -r + self.image_rootfs + -c new -v) self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c', 'new', '-v']) -- 1.7.10.4 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] lib/oe/image.py: add image dependency mechanism
This commit adds a dependency mechanism to image creation, so that we can split the images creation execution in groups, that can be executed in parallel, having the dependencies satisfied in the same time. The old code didn't need this since everything was serialized. Technically, it adds a dependency graph topological sort class that the main Image class can use to sort out the dependencies. Images that have dependencies have to declare them using the NEW IMAGE_TYPEDEP variable, like in the example below: For: IMAGE_FSTYPES = i1 i2 i3 i4 i5 IMAGE_TYPEDEP_i4 = i2 IMAGE_TYPEDEP_i5 = i6 i4 IMAGE_TYPEDEP_i6 = i7 IMAGE_TYPEDEP_i7 = i2 We'll get the following image groups, sorted out by their dependencies: [['i1', 'i3', 'i2'], ['i4', 'i7'], ['i6'], ['i5']] The algorithm can probably be optimized but, given the small size of the graphs, it'll do. [YOCTO #5830] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/image.py | 282 ++ 1 file changed, 191 insertions(+), 91 deletions(-) diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py index c15296f..488683e 100644 --- a/meta/lib/oe/image.py +++ b/meta/lib/oe/image.py @@ -19,9 +19,124 @@ def generate_image(arg): return None -class Image(object): + +This class will help compute IMAGE_FSTYPE dependencies and group them in batches +that can be executed in parallel. + +The next example is for illustration purposes, highly unlikely to happen in real life. +It's just one of the test cases I used to test the algorithm: + +For: +IMAGE_FSTYPES = i1 i2 i3 i4 i5 +IMAGE_TYPEDEP_i4 = i2 +IMAGE_TYPEDEP_i5 = i6 i4 +IMAGE_TYPEDEP_i6 = i7 +IMAGE_TYPEDEP_i7 = i2 + +We get the following list of batches that can be executed in parallel, having the +dependencies satisfied: + +[['i1', 'i3', 'i2'], ['i4', 'i7'], ['i6'], ['i5']] + +class ImageDepGraph(object): def __init__(self, d): self.d = d +self.graph = dict() +self.deps_array = dict() + +def _construct_dep_graph(self, image_fstypes): +graph = dict() + +def add_node(node): +deps = (self.d.getVar('IMAGE_TYPEDEP_' + node, True) or ) +if deps != : +graph[node] = deps + +for dep in deps.split(): +if not dep in graph: +add_node(dep) +else: +graph[node] = + +for fstype in image_fstypes: +add_node(fstype) + +return graph + +def _clean_graph(self): +# Live and VMDK images will be processed via inheriting +# bbclass and does not get processed here. Remove them from the fstypes +# graph. Their dependencies are already added, so no worries here. +remove_list = (self.d.getVar('IMAGE_TYPES_MASKED', True) or ).split() + +for item in remove_list: +self.graph.pop(item, None) + +def _compute_dependencies(self): + +returns dict object of nodes with [no_of_depends_on, no_of_depended_by] +for each node + +deps_array = dict() +for node in self.graph: +deps_array[node] = [0, 0] + +for node in self.graph: +deps = self.graph[node].split() +deps_array[node][0] += len(deps) +for dep in deps: +deps_array[dep][1] += 1 + +return deps_array + +def _sort_graph(self): +sorted_list = [] +group = [] +for node in self.graph: +if node not in self.deps_array: +continue + +depends_on = self.deps_array[node][0] + +if depends_on == 0: +group.append(node) + +if len(group) == 0 and len(self.deps_array) != 0: +bb.fatal(possible fstype circular dependency...) + +sorted_list.append(group) + +# remove added nodes from deps_array +for item in group: +for node in self.graph: +if item in self.graph[node]: +self.deps_array[node][0] -= 1 + +self.deps_array.pop(item, None) + +if len(self.deps_array): +# recursive call, to find the next group +sorted_list += self._sort_graph() + +return sorted_list + +def group_fstypes(self, image_fstypes): +self.graph = self._construct_dep_graph(image_fstypes) + +self._clean_graph() + +self.deps_array = self._compute_dependencies() + +alltypes = [node for node in self.graph] + +return (alltypes, self._sort_graph()) + + +class Image(ImageDepGraph): +def __init__(self, d): +self.d = d + +super(Image, self).__init__(d) def _get_rootfs_size(self): compute the rootfs size @@ -82,66 +197,44 @@ class Image(object): os.remove(img) + +This function will just filter out the compressed image types from the +fstype groups
[OE-core] [PATCH 0/2] Add image creation dependency mechanism
The image creation is done now in parallel. However, we didn't have any dependency handling in place since the old code did that serial. This patchset adds such dependency mechanism and all images types are split in groups, according to dependency, which can be done in parallel. The users, however, have to declare the image dependency using the IMAGE_TYPEDEP variable: Example: IMAGE_TYPEDEP_rpi-sdimg = ext3 laurentiu The following changes since commit 54562006c1327c5b99daa4cc05a3ba7e38412da1: image_types.bbclass: Fix tar IMAGE_CMD to not change directories (2014-02-18 08:38:52 +) are available in the git repository at: git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib lpalcu/image_deps_fix for you to fetch changes up to e7feddd1dafc5eb02b80b8d43aca504b17151e5c: lib/oe/image.py: add image dependency mechanism (2014-02-18 15:16:21 +0200) Laurentiu Palcu (2): image*.bbclass, bootimg.bbclass: add image type dependencies lib/oe/image.py: add image dependency mechanism meta/classes/bootimg.bbclass |4 + meta/classes/image-live.bbclass |3 + meta/classes/image-vmdk.bbclass |3 + meta/classes/image_types.bbclass |5 + meta/lib/oe/image.py | 282 ++ 5 files changed, 206 insertions(+), 91 deletions(-) Laurentiu Palcu (2): image*.bbclass, bootimg.bbclass: add image type dependencies lib/oe/image.py: add image dependency mechanism meta/classes/bootimg.bbclass |4 + meta/classes/image-live.bbclass |3 + meta/classes/image-vmdk.bbclass |3 + meta/classes/image_types.bbclass |5 + meta/lib/oe/image.py | 282 ++ 5 files changed, 206 insertions(+), 91 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] image*.bbclass, bootimg.bbclass: add image type dependencies
The following dependencies were manually added in the image creation code. However, in order to have an image dependency mechanism in place, use a new variable, IMAGE_TYPEDEP, to declare that an image type depends on another being already created. The following dependencies are added by this commit: elf - cpio.gz live - ext3 vmdk - ext3 iso - ext3 hddimg - ext3 This commit adds also another new variable: IMAGE_TYPES_MASKED. Currently, masking out certain types from IMAGE_FSTYPES was hardcoded in the image creation code. [YOCTO #5830] Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/bootimg.bbclass |4 meta/classes/image-live.bbclass |3 +++ meta/classes/image-vmdk.bbclass |3 +++ meta/classes/image_types.bbclass |5 + 4 files changed, 15 insertions(+) diff --git a/meta/classes/bootimg.bbclass b/meta/classes/bootimg.bbclass index 395085d..c370bd6 100644 --- a/meta/classes/bootimg.bbclass +++ b/meta/classes/bootimg.bbclass @@ -232,4 +232,8 @@ python do_bootimg() { bb.build.exec_func('build_iso', d) } +IMAGE_TYPEDEP_iso = ext3 +IMAGE_TYPEDEP_hddimg = ext3 +IMAGE_TYPES_MASKED += iso hddimg + addtask bootimg before do_build diff --git a/meta/classes/image-live.bbclass b/meta/classes/image-live.bbclass index bfb59f8..c7e6937 100644 --- a/meta/classes/image-live.bbclass +++ b/meta/classes/image-live.bbclass @@ -13,3 +13,6 @@ do_bootimg[depends] += ${INITRD_IMAGE}:do_rootfs do_bootimg[depends] += ${PN}:do_rootfs inherit bootimg + +IMAGE_TYPEDEP_live = ext3 +IMAGE_TYPES_MASKED += live diff --git a/meta/classes/image-vmdk.bbclass b/meta/classes/image-vmdk.bbclass index 6983e5c..6a98f17 100644 --- a/meta/classes/image-vmdk.bbclass +++ b/meta/classes/image-vmdk.bbclass @@ -16,6 +16,9 @@ ROOTFS ?= ${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext3 #inherit image-live inherit boot-directdisk +IMAGE_TYPEDEP_vmdk = ext3 +IMAGE_TYPES_MASKED += vmdk + create_vmdk_image () { qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hdddirect ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk ln -sf ${IMAGE_NAME}.vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.vmdk diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass index 944e6db..200e2e7 100644 --- a/meta/classes/image_types.bbclass +++ b/meta/classes/image_types.bbclass @@ -84,6 +84,7 @@ IMAGE_CMD_elf () { test -f ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.elf rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.elf mkelfImage --kernel=${ELF_KERNEL} --initrd=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio.gz --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.elf --append='${ELF_APPEND}' ${EXTRA_IMAGECMD} } +IMAGE_TYPEDEP_elf = cpio.gz UBI_VOLNAME ?= ${MACHINE}-rootfs @@ -150,3 +151,7 @@ DEPLOYABLE_IMAGE_TYPES ?= hddimg iso # Use IMAGE_EXTENSION_xxx to map image type 'xxx' with real image file extension name(s) for Hob IMAGE_EXTENSION_live = hddimg iso + +# The IMAGE_TYPES_MASKED variable will be used to mask out from the IMAGE_FSTYPES, +# images that will not be built at do_rootfs time: vmdk, hddimg, iso, etc. +IMAGE_TYPES_MASKED ?= -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/5] package_manager.py: support ipk incremental image generation
On Tue, Feb 18, 2014 at 05:42:24PM +0800, Hongxu Jia wrote: While incremental image generation enabled, 'load_old_install_solution' is used to determine what we've got in the previous (existed) image and 'dump_install_solution' to determine what we need to install in the current image. The 'backup_packaging_data' is used to back up the current opkg database. The 'recovery_packaging_data' is used to recover the opkg database which backed up by the previous image creation. Tweak 'remove' function in OpkgPM class, which the options for remove with dependencies was incorrect. [YOCTO #1894] Signed-off-by: Hongxu Jia hongxu@windriver.com --- meta/lib/oe/package_manager.py | 106 +++-- 1 file changed, 102 insertions(+), 4 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 6dc8fbd..4ea9677 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -934,12 +934,13 @@ class RpmPM(PackageManager): class OpkgPM(PackageManager): -def __init__(self, d, target_rootfs, config_file, archs): +def __init__(self, d, target_rootfs, config_file, archs, task_name='target'): super(OpkgPM, self).__init__(d) self.target_rootfs = target_rootfs self.config_file = config_file self.pkg_archs = archs +self.task_name = task_name self.deploy_dir = self.d.getVar(DEPLOY_DIR_IPK, True) self.deploy_lock_file = os.path.join(self.deploy_dir, deploy.lock) @@ -956,6 +957,13 @@ class OpkgPM(PackageManager): bb.utils.mkdirhier(self.opkg_dir) +self.solution_manifest = self.d.expand('${T}/saved/%s_solution' % + self.task_name) +self.saved_opkg_dir = self.d.expand('${T}/saved/%s' % self.task_name) +if not os.path.exists(self.d.expand('${T}/saved')): +bb.utils.mkdirhier(self.d.expand('${T}/saved')) + + if (self.d.getVar('BUILD_IMAGES_FROM_FEEDS', True) or ) != 1: self._create_config() else: @@ -1075,7 +1083,9 @@ class OpkgPM(PackageManager): try: bb.note(Installing the following packages: %s % ' '.join(pkgs)) -subprocess.check_output(cmd.split()) +bb.note(cmd) +output = subprocess.check_output(cmd.split()) +bb.note(output) except subprocess.CalledProcessError as e: (bb.fatal, bb.note)[attempt_only](Unable to install packages. Command '%s' returned %d:\n%s % @@ -1083,14 +1093,16 @@ class OpkgPM(PackageManager): def remove(self, pkgs, with_dependencies=True): if with_dependencies: -cmd = %s %s remove %s % \ +cmd = %s %s --force-depends --force-remove --force-removal-of-dependent-packages remove %s % \ (self.opkg_cmd, self.opkg_args, ' '.join(pkgs)) else: cmd = %s %s --force-depends remove %s % \ (self.opkg_cmd, self.opkg_args, ' '.join(pkgs)) try: -subprocess.check_output(cmd.split()) +bb.note(cmd) +output = subprocess.check_output(cmd.split()) +bb.note(output) except subprocess.CalledProcessError as e: bb.fatal(Unable to remove packages. Command '%s' returned %d:\n%s % (e.cmd, e.returncode, e.output)) @@ -1175,6 +1187,92 @@ class OpkgPM(PackageManager): else: status.write(line + \n) +''' +If incremental install, we need to determine what we've got, +what we need to add, and what to remove... +The dump_install_solution will dump and save the new install +solution. +''' +def dump_install_solution(self, pkgs): Why not have this function in the Manifest class? We have an API in place: Manifest.create_final() that should probably take care of anything related to manifest(s) creation after all packages are installed. +bb.note('creating new install solution for incremental install') +if len(pkgs) == 0: +return + +install_pkgs = list() + +# Create an temp dir as opkg root for simulating the installation +temp_rootfs = self.d.expand('${T}/opkg') +temp_opkg_dir = os.path.join(temp_rootfs, 'var/lib/opkg') +bb.utils.mkdirhier(temp_opkg_dir) + +opkg_args = -f %s -o %s % (self.config_file, temp_rootfs) +opkg_args += self.d.getVar(OPKG_ARGS, True) + +cmd = %s %s update % (self.opkg_cmd, +opkg_args) +try: +subprocess.check_output(cmd, shell=True) +except subprocess.CalledProcessError as e: +bb.note(Unable to dump install packages. Command '%s' +
Re: [OE-core] [PATCH 4/5] rootfs.py: fix BAD_RECOMMENDATIONS for ipk incremental image generation
On Tue, Feb 18, 2014 at 05:42:27PM +0800, Hongxu Jia wrote: While incremental image generation enabled and the previous image existed, if BAD_RECOMMENDATIONS is changed, the operation on the existed image is complicated, so remove the existed image in this situation. The same with PACKAGE_EXCLUDE and NO_RECOMMENDATIONS. [YOCTO #1894] Signed-off-by: Hongxu Jia hongxu@windriver.com --- meta/lib/oe/rootfs.py | 30 +- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py index 5561bb9..3d7adf9 100644 --- a/meta/lib/oe/rootfs.py +++ b/meta/lib/oe/rootfs.py @@ -443,7 +443,7 @@ class OpkgRootfs(Rootfs): self.pkg_archs = self.d.getVar(ALL_MULTILIB_PACKAGE_ARCHS, True) self.inc_opkg_image_gen = self.d.getVar('INC_IPK_IMAGE_GEN', True) -if self.inc_opkg_image_gen != 1: +if self._remove_existed_image(): bb.utils.remove(self.image_rootfs, True) self.pm = OpkgPM(d, self.image_rootfs, @@ -546,6 +546,34 @@ class OpkgRootfs(Rootfs): bb.note('incremental removed: %s' % ' '.join(pkg_to_remove)) self.pm.remove(pkg_to_remove) +''' +Compare with previous (existed) image creation, if some conditions +triggered, the previous (existed) image should be removed. +The conditions included any of 'PACKAGE_EXCLUDE, NO_RECOMMENDATIONS +and BAD_RECOMMENDATIONS' has been changed. +''' +def _remove_existed_image(self): s/existed/existing/ Also in the comment above. +# Incremental image creation is not enable +if self.inc_opkg_image_gen != 1: +return True + +vars_list_dir = self.d.expand('${T}/vars_list') s/vars_list_dir/vars_list_file/ in the line above and the code below. Leaving this variable with a 'dir' suffix while we're opening a file for reading/writing, is a little misleading. + +old_vars_list = +if os.path.exists(vars_list_dir): +old_vars_list = open(vars_list_dir, 'r+').read() + +new_vars_list = '%s:%s:%s\n' % \ +((self.d.getVar('BAD_RECOMMENDATIONS', True) or '').strip(), + (self.d.getVar('NO_RECOMMENDATIONS', True) or '').strip(), + (self.d.getVar('PACKAGE_EXCLUDE', True) or '').strip()) +open(vars_list_dir, 'w+').write(new_vars_list) + +if old_vars_list != new_vars_list: +return True + +return False + def _create(self): pkgs_to_install = self.manifest.parse_initial_manifest() opkg_pre_process_cmds = self.d.getVar('OPKG_PREPROCESS_COMMANDS', True) -- 1.8.1.2 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 5/5] rootfs.py: tweak _multilib_sanity_test for ipk incremental image generation
On Tue, Feb 18, 2014 at 05:42:28PM +0800, Hongxu Jia wrote: The _multilib_sanity_test installs multilib packages in a temporary root fs, and compare with the current image to figure out duplicated file that comes from different packages. While incremental image generation enabled and the previous image existed, there was a Multilib check error: ... ERROR: Multilib check error: duplicate files tmp/work/qemux86_64-poky- linux/core-image-minimal/1.0-r0/multilib/lib32/lib/libc.so.6 tmp/work/ qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/lib/libc.so.6 is not the same ... The reason is the file in current image has been prelinked in previous image generation and the file in a temporary root fs is not prelinked, even though the files come from the same package, the Multilib check considers they are different. [YOCTO #1894] Signed-off-by: Hongxu Jia hongxu@windriver.com --- meta/lib/oe/rootfs.py | 56 +++ 1 file changed, 52 insertions(+), 4 deletions(-) diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py index 3d7adf9..5054d1e 100644 --- a/meta/lib/oe/rootfs.py +++ b/meta/lib/oe/rootfs.py @@ -3,6 +3,8 @@ from oe.utils import execute_pre_post_process from oe.utils import contains as base_contains from oe.package_manager import * from oe.manifest import * +import oe.path +import filecmp import shutil import os import subprocess @@ -458,13 +460,61 @@ class OpkgRootfs(Rootfs): bb.utils.remove(self.d.getVar('MULTILIB_TEMP_ROOTFS', True), True) +def _prelink_file(self, root_dir, filename): +bb.note('prelink %s in %s' % (filename, root_dir)) +prelink_cfg = oe.path.join(root_dir, + self.d.expand('${sysconfdir}/prelink.conf')) +if not os.path.exists(prelink_cfg): + shutil.copy(self.d.expand('${STAGING_DIR_NATIVE}${sysconfdir_native}/prelink.conf'), +prelink_cfg) + +cmd_prelink = self.d.expand('${STAGING_DIR_NATIVE}${sbindir_native}/prelink') +self._exec_shell_cmd([cmd_prelink, + '--root', + root_dir, + '-amR', + '-N', + '-c', + self.d.expand('${sysconfdir}/prelink.conf')]) + +''' +Compare two files with the same key twice to see if they came +from the same package. If they are not same, they are duplicated +and come from different packages. I'm kind of confused by this comment. Doesn't same = duplicate? There might be a small confusion of terms here because the function's behavior is not as the name implies. +1st: Comapre them directly; +2nd: While incremental image creation is enabled, one of the + files could be probaly prelinked in the previous image + creation and the file has been changed, so we need to + prelink the other one and compare them. +''' +def _file_duplicate(self, key, f1, f2): Shouldn't be better to rename this function to something else in order to avoid confusion? Let's say: _files_are_equal() ? + +if not os.path.exists(f1) or not os.path.exists(f2): +return False + +# f1 is the same with f2, both of them were not prelinked +if filecmp.cmp(f1, f2): +return False filecmp.cmp() returns True if files are equal. Hence the confusion: _file_duplicate() returns False here, if files are equal... I think the logic is a little bit the other way around! :) + +if self.image_rootfs not in f1: +self._prelink_file(f1.replace(key, ''), f1) + +if self.image_rootfs not in f2: +self._prelink_file(f2.replace(key, ''), f2) + +# f1 is the same with f2, both of them were prelinked +if filecmp.cmp(f1, f2): +return False + +# Duplicated +return True here the return value matches the comment and the function name! :) + This function was reused from the old implementation. See commit: image.bbclass: Added variables for multilib support. by Lianhao Lu. def _multilib_sanity_test(self, dirs): -import filecmp allow_replace = self.d.getVar(MULTILIBRE_ALLOW_REP, True) if allow_replace is None: @@ -486,9 +536,7 @@ class OpkgRootfs(Rootfs): if allow_rep.match(key): valid = True else: -if os.path.exists(files[key]) and \ - os.path.exists(item) and \ - not filecmp.cmp(files[key], item): +if self._file_duplicate(key, files[key], item): valid = False
Re: [OE-core] [PATCH 3/7] lsbinitscripts: Update to 9.52
Hi Saul, On Mon, Feb 17, 2014 at 12:22:58AM -0800, Saul Wold wrote: Signed-off-by: Saul Wold s...@linux.intel.com --- meta/recipes-extended/lsb/lsbinitscripts_9.52.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-extended/lsb/lsbinitscripts_9.52.bb b/meta/recipes-extended/lsb/lsbinitscripts_9.52.bb index 11deae2..bc3e620 100644 --- a/meta/recipes-extended/lsb/lsbinitscripts_9.52.bb +++ b/meta/recipes-extended/lsb/lsbinitscripts_9.52.bb @@ -6,7 +6,7 @@ DEPENDS = popt glib-2.0 LIC_FILES_CHKSUM = file://COPYING;md5=ebf4e8b49780ab187d51bd26aaa022c6 S=${WORKDIR}/initscripts-${PV} -SRC_URI = http://pkgs.fedoraproject.org/repo/pkgs/initscripts/initscripts-${PV}.tar.bz2/2453811ec27a781a77f309f356663d9e/initscripts-${PV}.tar.bz2 \ +SRC_URI = http://pkgs.fedoraproject.org/repo/pkgs/initscripts/initscripts-9.52.tar.bz2/2453811ec27a781a77f309f356663d9e/initscripts-9.52.tar.bz2 \ This doesn't look like an upgrade... Besides, I believe you already upgraded this recipe to 9.52: Author: Saul Wold s...@linux.intel.com Date: Mon Jan 20 18:54:30 2014 -0800 lsbinitscripts: Update to 9.52 Am I missing something? laurentiu file://functions.patch \ -- 1.8.3.1 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/3] package-index.bb: use the new python indexing routines
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/recipes-core/meta/package-index.bb |7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/meta/recipes-core/meta/package-index.bb b/meta/recipes-core/meta/package-index.bb index 3cc1239..27b6d8e 100644 --- a/meta/recipes-core/meta/package-index.bb +++ b/meta/recipes-core/meta/package-index.bb @@ -21,10 +21,9 @@ do_populate_sysroot[noexec] = 1 do_package_index[nostamp] = 1 do_package_index[depends] += ${PACKAGEINDEXDEPS} -do_package_index() { - set -ex - ${PACKAGEINDEXES} - set +ex +python do_package_index() { +from oe.rootfs import generate_index_files +generate_index_files(d) } addtask do_package_index before do_build EXCLUDE_FROM_WORLD = 1 -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 3/3] package_*.bbclass: remove references to the old bash indexing routines
Package indexing is done in python and package-index.bb uses the new routines. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/classes/package_deb.bbclass |1 - meta/classes/package_ipk.bbclass |1 - meta/classes/package_rpm.bbclass |1 - 3 files changed, 3 deletions(-) diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass index a0a31bd..b9d7978 100644 --- a/meta/classes/package_deb.bbclass +++ b/meta/classes/package_deb.bbclass @@ -309,6 +309,5 @@ do_package_write_deb[umask] = 022 addtask package_write_deb before do_package_write after do_packagedata do_package -PACKAGEINDEXES += [ ! -e ${DEPLOY_DIR_DEB} ] || package_update_index_deb; PACKAGEINDEXDEPS += dpkg-native:do_populate_sysroot PACKAGEINDEXDEPS += apt-native:do_populate_sysroot diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass index 516ec4b..7cf2c8a 100644 --- a/meta/classes/package_ipk.bbclass +++ b/meta/classes/package_ipk.bbclass @@ -255,6 +255,5 @@ do_package_write_ipk[cleandirs] = ${PKGWRITEDIRIPK} do_package_write_ipk[umask] = 022 addtask package_write_ipk before do_package_write after do_packagedata do_package -PACKAGEINDEXES += [ ! -e ${DEPLOY_DIR_IPK} ] || package_update_index_ipk; PACKAGEINDEXDEPS += opkg-utils-native:do_populate_sysroot PACKAGEINDEXDEPS += opkg-native:do_populate_sysroot diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass index 82ea187..f267b8a 100644 --- a/meta/classes/package_rpm.bbclass +++ b/meta/classes/package_rpm.bbclass @@ -734,6 +734,5 @@ do_package_write_rpm[cleandirs] = ${PKGWRITEDIRRPM} do_package_write_rpm[umask] = 022 addtask package_write_rpm before do_package_write after do_packagedata do_package -PACKAGEINDEXES += [ ! -e ${DEPLOY_DIR_RPM} ] || package_update_index_rpm; PACKAGEINDEXDEPS += rpm-native:do_populate_sysroot PACKAGEINDEXDEPS += createrepo-native:do_populate_sysroot -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/3] fix package-index build issue
After rootfs code refactoring, 'bitbake package-index' fails because it's calling the old bash indexing routines. This patchset should fix that. laurentiu The following changes since commit b60ed2d0fd86a3ae5ce7facbb044677aa7ec2889: bitbake: runqueue: Ensure we do run 'masked' setscene tasks if specified as targets (2014-02-13 17:57:20 +) are available in the git repository at: git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib lpalcu/package_index_recipe_fix for you to fetch changes up to db1e6488ef483ab76f7eca961b214bd8f8758b2e: package_*.bbclass: remove references to the old bash indexing routines (2014-02-14 12:47:53 +0200) Laurentiu Palcu (3): package_manager.py, rootfs.py, sdk.py: add Indexer class package-index.bb: use the new python indexing routines package_*.bbclass: remove references to the old bash indexing routines meta/classes/package_deb.bbclass|1 - meta/classes/package_ipk.bbclass|1 - meta/classes/package_rpm.bbclass|1 - meta/lib/oe/package_manager.py | 339 --- meta/lib/oe/rootfs.py | 58 -- meta/lib/oe/sdk.py | 48 + meta/recipes-core/meta/package-index.bb |7 +- 7 files changed, 224 insertions(+), 231 deletions(-) Laurentiu Palcu (3): package_manager.py, rootfs.py, sdk.py: add Indexer class package-index.bb: use the new python indexing routines package_*.bbclass: remove references to the old bash indexing routines meta/classes/package_deb.bbclass|1 - meta/classes/package_ipk.bbclass|1 - meta/classes/package_rpm.bbclass|1 - meta/lib/oe/package_manager.py | 339 --- meta/lib/oe/rootfs.py | 58 -- meta/lib/oe/sdk.py | 48 + meta/recipes-core/meta/package-index.bb |7 +- 7 files changed, 224 insertions(+), 231 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/3] package_manager.py, rootfs.py, sdk.py: add Indexer class
Because the package-index.bb needs to create package indexes outside do_rootfs environment, move the indexing capability out of PackageManager class to a smaller Indexer class. This commit: * simply moves the indexing functions for ipk/deb with no changes; * rewrites the RPM indexing function so that it can be easily moved out of the PackageManager class; * removes some RPM duplicate code, moves it into a method inside RpmPM class and changes the RpmPM constructor so that the new method is effective; Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py | 339 +--- meta/lib/oe/rootfs.py | 58 --- meta/lib/oe/sdk.py | 48 +- 3 files changed, 221 insertions(+), 224 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index 9884c3a..af14d5a 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -22,6 +22,144 @@ def create_index(arg): return None +class Indexer(object): +__metaclass__ = ABCMeta + +def __init__(self, d, deploy_dir): +self.d = d +self.deploy_dir = deploy_dir + +@abstractmethod +def write_index(self): +pass + + +class RpmIndexer(Indexer): +def write_index(self): +sdk_pkg_archs = (self.d.getVar('SDK_PACKAGE_ARCHS', True) or ).replace('-', '_').split() +mlb_prefix_list = (self.d.getVar('MULTILIB_PREFIX_LIST', True) or ).replace('-', '_').split() +all_mlb_pkg_archs = (self.d.getVar('ALL_MULTILIB_PACKAGE_ARCHS', True) or ).replace('-', '_').split() + +archs = set() +for item in mlb_prefix_list: +archs = archs.union(set(item.split(':')[1:])) + +if len(archs) == 0: +archs = archs.union(set(all_mlb_pkg_archs)) + +archs = archs.union(set(sdk_pkg_archs)) + +rpm_createrepo = bb.utils.which(os.getenv('PATH'), createrepo) +index_cmds = [] +rpm_dirs_found = False +for arch in archs: +arch_dir = os.path.join(self.deploy_dir, arch) +if not os.path.isdir(arch_dir): +continue + +index_cmds.append(%s --update -q %s % (rpm_createrepo, arch_dir)) + +rpm_dirs_found = True + +if not rpm_dirs_found: +return(There are no packages in %s % self.deploy_dir) + +nproc = multiprocessing.cpu_count() +pool = bb.utils.multiprocessingpool(nproc) +results = list(pool.imap(create_index, index_cmds)) +pool.close() +pool.join() + +for result in results: +if result is not None: +return(result) + + +class OpkgIndexer(Indexer): +def write_index(self): +arch_vars = [ALL_MULTILIB_PACKAGE_ARCHS, + SDK_PACKAGE_ARCHS, + MULTILIB_ARCHS] + +opkg_index_cmd = bb.utils.which(os.getenv('PATH'), opkg-make-index) + +if not os.path.exists(os.path.join(self.deploy_dir, Packages)): +open(os.path.join(self.deploy_dir, Packages), w).close() + +index_cmds = [] +for arch_var in arch_vars: +archs = self.d.getVar(arch_var, True) +if archs is None: +continue + +for arch in archs.split(): +pkgs_dir = os.path.join(self.deploy_dir, arch) +pkgs_file = os.path.join(pkgs_dir, Packages) + +if not os.path.isdir(pkgs_dir): +continue + +if not os.path.exists(pkgs_file): +open(pkgs_file, w).close() + +index_cmds.append('%s -r %s -p %s -m %s' % + (opkg_index_cmd, pkgs_file, pkgs_file, pkgs_dir)) + +if len(index_cmds) == 0: +return(There are no packages in %s! % self.deploy_dir) + +nproc = multiprocessing.cpu_count() +pool = bb.utils.multiprocessingpool(nproc) +results = list(pool.imap(create_index, index_cmds)) +pool.close() +pool.join() + +for result in results: +if result is not None: +return(result) + + +class DpkgIndexer(Indexer): +def write_index(self): +pkg_archs = self.d.getVar('PACKAGE_ARCHS', True) +if pkg_archs is not None: +arch_list = pkg_archs.split() +sdk_pkg_archs = self.d.getVar('SDK_PACKAGE_ARCHS', True) +if sdk_pkg_archs is not None: +arch_list += sdk_pkg_archs.split() + +dpkg_scanpackages = bb.utils.which(os.getenv('PATH'), dpkg-scanpackages) +gzip = bb.utils.which(os.getenv('PATH'), gzip) + +index_cmds = [] +deb_dirs_found = False +for arch in arch_list: +arch_dir = os.path.join(self.deploy_dir, arch) +if not os.path.isdir(arch_dir): +continue + +with open(os.path.join(arch_dir, Release), w
[OE-core] [PATCH 0/1] package_manager.py: move multilib prefix list computation function to RpmIndexer
I tested this with the following: rm -rf tmp/ bb core-image-sato bb -c populate_sdk core-image-sato bb lib32-core-image-minimal rm -rf tmp/ bb lib32-core-image-minimal laurentiu The following changes since commit b38c7cb7a6ab787d7be2b2c295ba58a98a954cd5: socat: upgrade to 1.7.2.3 (2014-02-14 12:31:10 +) are available in the git repository at: git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib lpalcu/rpm_ml_build_fix for you to fetch changes up to 4e62efc1659a4be94b6437049a771ec444515600: package_manager.py: move multilib prefix list computation function to RpmIndexer (2014-02-14 20:59:17 +0200) Laurentiu Palcu (1): package_manager.py: move multilib prefix list computation function to RpmIndexer meta/lib/oe/package_manager.py | 105 1 file changed, 53 insertions(+), 52 deletions(-) Laurentiu Palcu (1): package_manager.py: move multilib prefix list computation function to RpmIndexer meta/lib/oe/package_manager.py | 105 1 file changed, 53 insertions(+), 52 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/1] package_manager.py: move multilib prefix list computation function to RpmIndexer
Since the code from anonymous function in rootfs_rpm.bbclass has been removed, MULTILIB_PREFIX_LIST variable was never set. Hence not all directories got indexed. This commit will move the multilib prefix list computation function from RpmPM class to RpmIndexer, since the indexer needs it too. I was hoping to avoid this but, unfortunately, I couldn't. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/package_manager.py | 105 1 file changed, 53 insertions(+), 52 deletions(-) diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py index af14d5a..2faf422 100644 --- a/meta/lib/oe/package_manager.py +++ b/meta/lib/oe/package_manager.py @@ -35,14 +35,64 @@ class Indexer(object): class RpmIndexer(Indexer): +def get_ml_prefix_and_os_list(self, arch_var=None, os_var=None): +package_archs = { +'default': [], +} + +target_os = { +'default': , +} + +if arch_var is not None and os_var is not None: +package_archs['default'] = self.d.getVar(arch_var, True).split() +package_archs['default'].reverse() +target_os['default'] = self.d.getVar(os_var, True).strip() +else: +package_archs['default'] = self.d.getVar(PACKAGE_ARCHS, True).split() +# arch order is reversed. This ensures the -best- match is +# listed first! +package_archs['default'].reverse() +target_os['default'] = self.d.getVar(TARGET_OS, True).strip() +multilibs = self.d.getVar('MULTILIBS', True) or +for ext in multilibs.split(): +eext = ext.split(':') +if len(eext) 1 and eext[0] == 'multilib': +localdata = bb.data.createCopy(self.d) +default_tune_key = DEFAULTTUNE_virtclass-multilib- + eext[1] +default_tune = localdata.getVar(default_tune_key, False) +if default_tune: +localdata.setVar(DEFAULTTUNE, default_tune) +bb.data.update_data(localdata) +package_archs[eext[1]] = localdata.getVar('PACKAGE_ARCHS', + True).split() +package_archs[eext[1]].reverse() +target_os[eext[1]] = localdata.getVar(TARGET_OS, + True).strip() + +ml_prefix_list = dict() +for mlib in package_archs: +if mlib == 'default': +ml_prefix_list[mlib] = package_archs[mlib] +else: +ml_prefix_list[mlib] = list() +for arch in package_archs[mlib]: +if arch in ['all', 'noarch', 'any']: +ml_prefix_list[mlib].append(arch) +else: +ml_prefix_list[mlib].append(mlib + _ + arch) + +return (ml_prefix_list, target_os) + def write_index(self): sdk_pkg_archs = (self.d.getVar('SDK_PACKAGE_ARCHS', True) or ).replace('-', '_').split() -mlb_prefix_list = (self.d.getVar('MULTILIB_PREFIX_LIST', True) or ).replace('-', '_').split() all_mlb_pkg_archs = (self.d.getVar('ALL_MULTILIB_PACKAGE_ARCHS', True) or ).replace('-', '_').split() +mlb_prefix_list = self.get_ml_prefix_and_os_list()[0] + archs = set() for item in mlb_prefix_list: -archs = archs.union(set(item.split(':')[1:])) +archs = archs.union(set(i.replace('-', '_') for i in mlb_prefix_list[item])) if len(archs) == 0: archs = archs.union(set(all_mlb_pkg_archs)) @@ -303,56 +353,7 @@ class RpmPM(PackageManager): self.indexer = RpmIndexer(self.d, self.deploy_dir) -self.ml_prefix_list, self.ml_os_list = self._get_prefix_and_os_list(arch_var, os_var) - -def _get_prefix_and_os_list(self, arch_var, os_var): -package_archs = { -'default': [], -} - -target_os = { -'default': , -} - -if arch_var is not None and os_var is not None: -package_archs['default'] = self.d.getVar(arch_var, True).split() -package_archs['default'].reverse() -target_os['default'] = self.d.getVar(os_var, True).strip() -else: -package_archs['default'] = self.d.getVar(PACKAGE_ARCHS, True).split() -# arch order is reversed. This ensures the -best- match is -# listed first! -package_archs['default'].reverse() -target_os['default'] = self.d.getVar(TARGET_OS, True).strip() -multilibs = self.d.getVar('MULTILIBS', True) or -for ext in multilibs.split(): -eext = ext.split(':') -if len(eext) 1 and eext[0] == 'multilib
Re: [OE-core] [PATCH 1/1] lib/oe/rootfs.py: fix RPM multilib issue
Hi Mark, On Wed, Feb 12, 2014 at 01:46:31PM -0600, Mark Hatle wrote: On 2/12/14, 12:54 PM, Laurentiu Palcu wrote: For some odd reason (at least I couldn't find an explanation to this, yet), if a multilib version of a package is installed after the main one (that is: in a different smart session), the main package binaries are not overwritten. For two packages with the same name, but different architectures -- the non-ELF files must be identical (or different file names in each package.) The ELF files are identified by type and there is a resolution mechanism within RPM to determine which is the 'preferred' version. We do NOT have the resolution mechanism set, so the first package installed 'wins'. So if you do two packages in two separate transactions, the first version installed will be kept. If you do it in one transaction, then I believe it ends up being the last version [or maybe it's random due to sort order?] There is a Yocto Project defect to configure the RPM multilib settings, but so far it's just sat there, as nobody has either needed it -- or cared enough to implement it. There are several rpm related multilib issues in bugzilla left hanging, unfortunately. :| The reality is installed two packages with the same /bin, /sbin binaries is rare these days.. it's much more useful to install the libraries. Apparently, the multilib test we currently have on AB is to install lib32-connman-gnome and test the ELF class of the /usr/bin/connman-applet binary... The code you have below may turn out to be a performance improvement.. (each smart/rpm transaction takes setup and cleanup time, so multiple transactions are slower then one larger transaction.) But it's likely not fixing the actual problem you have. True, this problem should be properly fixed. This patch, however, just restores the behavior we already used in the old bash code which people seemed to have used for a long time now. # The default transaction color. This value is a set of bits to # determine file and dependency affinity for this arch. # 0 uncolored (i.e. use only arch as install hint) # 1 Elf32 permitted # 2 Elf64 permitted # 4 MIPS reserved %_transaction_color 3 (4 BTW is MIPS64 - n32) _transaction_color of 3 indicates we allow Elf32 and Elf64 to be installed.. There is a second item: _prefer_color that is used to define which item should be installed when there is that ELF conflict. The numbers above follow there as well. By default I believe it sets itself to '2' when doing a single transaction installed, preferring Elf64. If these parameters (_transaction_color and _prefer_color) can be easily set before each transaction, I suppose we could alter them accordingly, depending on the type of packages we're going to install. Doesn't seem very complicated, in theory... laurentiu --Mark This commit restores the functionality to the original one, before migrating to python: feed all the packages to smart, apart from attempt only ones which are installed separately. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/rootfs.py | 16 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py index b6baf77..9162e52 100644 --- a/meta/lib/oe/rootfs.py +++ b/meta/lib/oe/rootfs.py @@ -317,10 +317,18 @@ class RpmRootfs(Rootfs): self.pm.update() -for pkg_type in self.install_order: -if pkg_type in pkgs_to_install: -self.pm.install(pkgs_to_install[pkg_type], -[False, True][pkg_type == aop]) +pkgs = [] +pkgs_attempt = [] +for pkg_type in pkgs_to_install: +if pkg_type == Manifest.PKG_TYPE_ATTEMPT_ONLY: +pkgs_attempt = pkgs_to_install[pkg_type] +continue + +pkgs += pkgs_to_install[pkg_type] + +self.pm.install(pkgs) + +self.pm.install(pkgs_attempt, True) self.pm.install_complementary() ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/1] lib/oe/rootfs.py: fix RPM multilib issue
On Thu, Feb 13, 2014 at 11:32:36AM +, Richard Purdie wrote: On Wed, 2014-02-12 at 20:54 +0200, Laurentiu Palcu wrote: For some odd reason (at least I couldn't find an explanation to this, yet), if a multilib version of a package is installed after the main one (that is: in a different smart session), the main package binaries are not overwritten. This commit restores the functionality to the original one, before migrating to python: feed all the packages to smart, apart from attempt only ones which are installed separately. Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com --- meta/lib/oe/rootfs.py | 16 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py index b6baf77..9162e52 100644 --- a/meta/lib/oe/rootfs.py +++ b/meta/lib/oe/rootfs.py @@ -317,10 +317,18 @@ class RpmRootfs(Rootfs): self.pm.update() -for pkg_type in self.install_order: -if pkg_type in pkgs_to_install: -self.pm.install(pkgs_to_install[pkg_type], -[False, True][pkg_type == aop]) +pkgs = [] +pkgs_attempt = [] +for pkg_type in pkgs_to_install: +if pkg_type == Manifest.PKG_TYPE_ATTEMPT_ONLY: +pkgs_attempt = pkgs_to_install[pkg_type] Should this be a += ? Indeed, I guess I was thinking that there can be only one attempt only list. Nevertheless, should be += there. Thanks! This might be clearer as an if: else: Agreed. Incoming v2... laurentiu +continue + +pkgs += pkgs_to_install[pkg_type] + +self.pm.install(pkgs) + +self.pm.install(pkgs_attempt, True) self.pm.install_complementary() ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 0/1] lib/oe/rootfs.py: fix RPM multilib issue
v2: * address RP's comments; laurentiu The following changes since commit b70a10ca3ee4a0add90d387e2c8288e531aaa47c: systemd-serialgetty: Update to match version in recent systemd (2014-02-11 22:59:38 +) are available in the git repository at: git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib lpalcu/rootfs_refactoring_rpm_multilib_fix for you to fetch changes up to 453395e793d0d5c30d4e8092ee6faf80484ad447: lib/oe/rootfs.py: fix RPM multilib issue (2014-02-13 13:52:38 +0200) Laurentiu Palcu (1): lib/oe/rootfs.py: fix RPM multilib issue meta/lib/oe/rootfs.py | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) Laurentiu Palcu (1): lib/oe/rootfs.py: fix RPM multilib issue meta/lib/oe/rootfs.py | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) -- 1.7.9.5 ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core