Re: [OE-core] [PATCH 0/1] scripts: Add cleanup-downloads-dir tool

2015-06-16 Thread Laurentiu Palcu
Hi Ross,

On Mon, Jun 15, 2015 at 11:38:01PM +0100, Burton, Ross wrote:
 
 On 12 June 2015 at 13:15, Laurentiu Palcu laurentiu.pa...@intel.com wrote:
 
 A few days ago I noticed I had no space on disk and decided to make some
 space. Since my downloads directory was never deleted since denzil, it
 consumed 25GB of space which I needed. However, I wanted to delete only 
 old
 and unneeded tarbals/git repos and keep the ones that my current Yocto
 version needed, so I wrote this tool.
 
 
 Something is not quite right, after running this and causing a rebuild I got 
 this:
 
 ERROR: No checksum specified for 
 /home/ross/Yocto/downloads/git2_git.lttng.org.lttng-modules.git.tar.gz, 
 please add at least one to
 the recipe:
 SRC_URI[md5sum] = c32ceb7d4b208b2b1d8132c3bbc7d638
 SRC_URI[sha256sum] = 
 d593cca435003770e981c8ae5def3db0e95d72bc3f83f3d43a18287ea1f6d165
 ERROR: Function failed: Fetcher failure for URL: 
 'http://downloads.yoctoproject.org/mirror/sources/
 git2_git.lttng.org.lttng-modules.git.tar.gz'. Missing SRC_URI checksum
 
 A few too many files pruned?
Thanks for giving this a test. However, I have no idea why this happens
for you... I ran it on my side, after compiling lttng-modules (so it
fetches the latest, etc), and the tarballs were not removed. Actually,
nothing was removed since I ran it on master, after doing a fresh
core-image-minimal build... See below. Can you please try a: 'bitbake
lttng-modules' followed by a 'cleanup-downloads-dir'? Do you get the
same behavior on the first rebuid? Could there's something else going on
at your side?

test@test-machine:/ssd/work/yp1/test$ bitbake lttng-modules
...
build output skipped
...
test@test-machine:/ssd/work/yp1/test$ cleanup-downloads-dir
Parsing recipes..done.
Generating unused files list... done

No obsolete files found in /ssd/work/yp1/test/downloads.
test@test-machine:/ssd/work/yp1/test$ ls -la downloads/ |grep lttng-modules
-rw-rw-r--  1 test test   1861595 May 13 11:26 
git2_git.lttng.org.lttng-modules.git.tar.gz
-rw-rw-r--  1 test test 0 Jun 16 09:45 
git2_git.lttng.org.lttng-modules.git.tar.gz.done
-rw-rw-r--  1 test test   125 Jun 16 09:45 
lttng-modules-replace-KERNELDIR-with-KERNEL_SRC.patch.done
test@test-machine:/ssd/work/yp1/test$ ll downloads/git2 |grep lttng-modules
drwxr-xr-x  7 test test  4096 May 13 11:26 git.lttng.org.lttng-modules.git/
-rw-rw-r--  1 test test 6 Jun 16 09:45 git.lttng.org.lttng-modules.git.done

laurentiu
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/1] scripts: Add cleanup-downloads-dir tool

2015-06-12 Thread Laurentiu Palcu
Hi all,

A few days ago I noticed I had no space on disk and decided to make some
space. Since my downloads directory was never deleted since denzil, it
consumed 25GB of space which I needed. However, I wanted to delete only old
and unneeded tarbals/git repos and keep the ones that my current Yocto
version needed, so I wrote this tool.

After running it, my downloads size became 7GB.

I also tested on a brand new downloads directory, after doing a
core-image-minimal build, and it didn't find any obsolete files (which
is kind of normal). :)

I hope someone finds it helpful. Also, I encourage you to give it a
test. Don't worry, it will not delete anything unless you allow it and
you can review the list of files to be removed. If you see anything
suspicious, let me know.

laurentiu

Laurentiu Palcu (1):
  scripts: Add cleanup-downloads-dir tool

 scripts/cleanup-downloads-dir | 158 ++
 1 file changed, 158 insertions(+)
 create mode 100755 scripts/cleanup-downloads-dir

-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/1] scripts: Add cleanup-downloads-dir tool

2015-06-12 Thread Laurentiu Palcu
The tool will remove obsolete files/directories from the downloads
directory(DL_DIR): tarballs belonging to old versions of a certain
package, .done files that are no longer needed, git repos. This way, you
don't have to delete the entire downloads directory to make some space
on disk!

Warning: The tool works fine if DL_DIR is not shared. If you share
 DL_DIR among various Yocto versions, only the files belonging to
 the Yocto version you run this script on will be kept.

Users that have the same downloads directory since the early versions of
Yocto, will benefit the most. :)

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 scripts/cleanup-downloads-dir | 158 ++
 1 file changed, 158 insertions(+)
 create mode 100755 scripts/cleanup-downloads-dir

diff --git a/scripts/cleanup-downloads-dir b/scripts/cleanup-downloads-dir
new file mode 100755
index 000..b48b24d
--- /dev/null
+++ b/scripts/cleanup-downloads-dir
@@ -0,0 +1,158 @@
+#!/usr/bin/env python
+
+# This script will compute a list of obsolete/unused files in downloads 
directory
+# (DL_DIR) and will wait for the user to acknowledge their removal. Also, the
+# user can manually alter the contents of the list, before removal, if some of
+# the directories/files are not to be removed.
+#
+# Author: Laurentiu Palcu laurentiu.pa...@intel.com
+#
+# Copyright (c) 2015 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License for more details.
+
+import os
+import sys
+import tempfile
+import shutil
+
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), lib))
+import scriptpath
+
+sys.path.insert(0, scriptpath.add_bitbake_lib_path())
+import bb.tinfoil
+import bb.fetch2 as fetcher
+
+
+# This function looks at the URLs and computes the list of files in DL_DIR,
+# adding also the donestamp files to the list.
+def compute_paths(bb_hdlr, fetcher_hdlr):
+dl_dir = bb_hdlr.config_data.getVar('DL_DIR', True)
+paths = []
+
+for url in fetcher_hdlr.urls:
+ud = fetcher_hdlr.ud[url]
+ud.setup_localpath(fetcher_hdlr.d)
+
+if ud.localpath.startswith(dl_dir):
+paths.append(ud.localpath)
+
+if ud.mirrortarball is not None:
+if not ud.mirrortarball.startswith(dl_dir):
+mirrortarball_path = os.path.join(dl_dir, ud.mirrortarball)
+else:
+mirrortarball_path = ud.mirrortarball
+
+paths.append(mirrortarball_path)
+paths.append(mirrortarball_path + .done)
+
+paths.append(ud.donestamp)
+
+return paths
+
+
+# Go through all recipe providers and return a list of files that should belong
+# to DL_DIR.
+def get_used_dldir_paths(bb_hdlr):
+pkg_providers = bb.providers.allProviders(bb_hdlr.cooker.recipecache)
+pnlist = sorted(pkg_providers)
+used_paths = []
+pkgs_no = len(pnlist)
+pkgs_scanned = 0
+cache = bb.cache.Cache
+
+for pn in pnlist:
+sys.stderr.write(Computing used paths: %d%%\r % (pkgs_scanned * 100 
/ pkgs_no))
+
+for provider in pkg_providers[pn]:
+fn = provider[1]
+
+data = cache.loadDataFull(fn,
+  
bb_hdlr.cooker.collection.get_file_appends(fn),
+  bb_hdlr.config_data)
+
+used_paths += compute_paths(bb_hdlr, fetcher.Fetch([], data))
+
+pkgs_scanned += 1
+
+# return the list with no duplicates
+return list(set(used_paths))
+
+
+# Go through all files in DL_DIR and return those files that are not in the
+# used_paths list. These files/directories are the ones that can be removed.
+def get_unused_dldir_paths(bb_hdlr, used_paths):
+dl_dir = bb_hdlr.config_data.getVar('DL_DIR', True)
+used_dirs = set()
+unused_paths = []
+
+sys.stderr.write(Generating unused files list... )
+# Do a quick scan of all used files and extract the directories in which
+# they are located. We'll use these when we scan the downloads directory, 
so
+# we don't descend, to subdirectories,unnecessarily.
+for up in used_paths:
+used_dirs.add(os.path.dirname(up))
+
+for root, dirs, files in os.walk(dl_dir):
+dirs_copy = list(dirs)
+for d in dirs_copy:
+path = os.path.join(root, d)
+
+if path in used_dirs:
+continue
+
+dirs[:] = [dr for dr in dirs if dr != d]
+
+if path not in used_paths:
+unused_paths.append(path)
+
+for f in files:
+path = os.path.join(root, f)
+
+if path

Re: [OE-core] [RFC] kernel: Enable externalsrc on kernels which instantiate kernel.bbclass

2014-12-03 Thread Laurentiu Palcu
Hi Paul,

On Wed, Dec 03, 2014 at 12:00:31PM +, Paul Eggleton wrote:
 On Monday 22 September 2014 13:04:47 Bruce Ashfield wrote:
  On 14-09-22 01:03 PM, Khem Raj wrote:
   On Mon, Sep 22, 2014 at 8:27 AM, Bruce Ashfield
   
   bruce.ashfi...@windriver.com wrote:
   But the reports we've been getting have been that externalsrc
   builds are working for kernels, and linux-yocto without the change
   in place, so I'm looking to reduce the patch footprint and re-submit.
   
   no, it cant work if folks were trying the usecase I have mentioned.
   The fix is infact for
   non linux-yocto kernels.
  
  I'll send out my updated patch shortly, it should fix both cases.
 
 Bruce, did you get around to doing this?
 
 At the moment I am trying to make my workflow tool (devtool modify) work 
 with 
 linux-yocto. I've at least got to the point where I can extract the 
 appropriate source from the recipe, but when I use externalsrc to point to it 
 and then try to build it, it doesn't even get past do_configure, so something 
 must be missing (make complains about missing targets, presumably because 
 it's 
 running in ${B} and there's nothing in that directory).
 
 Do you expect linux-yocto + externalsrc to work at the moment in master? If 
 so, are there any special steps I need to do with respect to preparing the 
 external source tree that my tool might not be performing?
FWIW, I'm using externalsrc to build non linux-yocto kernels but, to make it
work, I had to overwrite the KERNEL_CONFIG_COMMAND:

KERNEL_CONFIG_COMMAND = oe_runmake_call -C ${S} O=${B} oldnoconfig || yes '' | 
oe_runmake -C ${S} O=${B} oldconfig

This might be the reason of your do_configure failure...

I hope it helps,
laurentiu

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] struct.error: unpack requires a string argument of length 32 in Yocto

2014-11-05 Thread Laurentiu Palcu
Hi Wilder,

On Tue, Nov 04, 2014 at 07:02:04PM +, Wilder wrote:
 Hi Laurentiu
 
 
 I am a student from Mexico, and we are using Yocto to build our images, 
 however there is a unique error that is bothering me while
 trying to set up the environment to build an image with sdk, I am not so 
 familiar with structs using python and this is the error I
 get:
 
 
 Traceback (most recent call last):
 
   File 
 /media/wilder/Seagate_Backup_Plus_Drive/ITESM/MSE/te5009/opt/poky/1.6.1/relocate_sdk.py,
  line 231, in module
 
 change_interpreter(e)
 
   File 
 /media/wilder/Seagate_Backup_Plus_Drive/ITESM/MSE/te5009/opt/poky/1.6.1/relocate_sdk.py,
  line 89, in change_interpreter
 
 p_memsz, p_flags, p_align = struct.unpack(ph_fmt, ph_hdr)
 
 struct.error: unpack requires a string argument of length 32
 
 
 I am using a Chromebook with Ubuntu 14 installed on it and it seems that this 
 error is not showing up in other computers from my
 classmates, 
 
 
 At the beginning I thought the issue was because of the python version, since 
 by default it uses Python 3, but now I have python
 2.7.6 and the same error message is shown.
It's hard to tell what the real issue is. But, you'd get this kind of
error if the binary being relocated is corrupt. The script reads the ELF
header first and then goes through the program header table to find the
INTERP section. Apparently, in your case, the size of e_phentsize (which
is taken from the ELF header) is different from the actual size of one
program header entry for a 32bit binary.

But, you could easily find out which is the problematic binary:
1. Run the SDK installer with -S -R
2. go to the install directory and change the relocate_sdk.py to print
the name of the binary being relocated
3. execute ./relocate_sdk.sh

After you find the binary, use readelf to inspect it.

 
 
 Could you please point me to a forum where I can ask for help_
You can always find help on the OE-core mailing list:
openembedded-core@lists.openembedded.org (CCed)

You can subscribe here:
http://lists.openembedded.org/mailman/listinfo/openembedded-core

laurentiu
 
 
 Thanks in advance,
 
 Wilder
 
 
 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/1] package_manager.py: fix rpm based do_rootfs failed while IMAGE_INSTALL_append = python3

2014-10-31 Thread Laurentiu Palcu
On Fri, Oct 31, 2014 at 03:05:58PM +0800, Hongxu Jia wrote:
 While a pkg name (such as python3) not existed in variable
 PACKAGES, but provided by another pkg (such as python-core),
 in this situation, rpm based do_rootfs could not search it in
 the feeds.
 
 The fix is to invoke 'smart query --provides' to do the search
 if the feeds search failed. We don't drop the feeds search,
 because the smart query search is too slow, and the feeds
 search is fast and could work well with most pkgs.
 
 [YOCTO #6918]
 
 Signed-off-by: Hongxu Jia hongxu@windriver.com
 ---
  meta/lib/oe/package_manager.py | 21 +
  1 file changed, 21 insertions(+)
 
 diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
 index ffb83b2..d5704e6 100644
 --- a/meta/lib/oe/package_manager.py
 +++ b/meta/lib/oe/package_manager.py
 @@ -671,6 +671,23 @@ class RpmPM(PackageManager):
  
  return 
  
 +def _search_pkg_provider_in_smart(self, pkg, feed_archs):
 +output = self._invoke_smart('query  --provides %s' % pkg) or 
 +for provider in output.split('\n'):
 +# Filter out the line which is empty or start with space
 +if provider.strip() == '' or provider[0] == ' ':
 +continue
 +
 +for arch in feed_archs:
 +arch = '@' + arch.replace('-', '_')
 +if arch in provider:
 +# First found is best match
 +# bb.note(%s - %s % (pkg, provider))
Why not remove the bb.note above since it's commented out?

 +return provider
 +
 +return ''
 +
 +
  '''
  Translate the OE multilib format names to the RPM/Smart format names
  It searched the RPM/Smart format names in probable multilib feeds first,
 @@ -693,6 +710,8 @@ class RpmPM(PackageManager):
  feed_archs = self.ml_prefix_list[mlib]
  new_pkg = self._search_pkg_name_in_feeds(subst, 
 feed_archs)
  if not new_pkg:
 +new_pkg = self._search_pkg_provider_in_smart(subst, 
 feed_archs)
 +if not new_pkg:
This entire construct:
...
new_pkg = self._search_pkg_name_in_feeds(subst, feed_archs)
if not new_pkg:
new_pkg = self._search_pkg_provider_in_smart(subst, feed_archs)
if not new_pkg:
...

could be replaced with:
...
new_pkg = self._search_pkg_name_in_feeds(subst, feed_archs) or \
self._search_pkg_provider_in_smart(subst, feed_archs)
if not new_pkg:
...

I'd say it looks cleaner but it's a matter of personal taste. :)

  # Failed to translate, package not found!
  err_msg = '%s not found in the %s feeds (%s).\n' % \
(pkg, mlib,  .join(feed_archs))
 @@ -711,6 +730,8 @@ class RpmPM(PackageManager):
  default_archs = self.ml_prefix_list['default']
  new_pkg = self._search_pkg_name_in_feeds(pkg, default_archs)
  if not new_pkg:
 +new_pkg = self._search_pkg_provider_in_smart(pkg, 
 default_archs)
 +if not new_pkg:
ditto

laurentiu
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] image.py: Fix error in graph sorting

2014-10-24 Thread Laurentiu Palcu
On Fri, Oct 24, 2014 at 09:41:10AM +0200, Pascal Bach wrote:
 The graph sorting algorithm for image dependencies does a look for an
 occurrence of a searched string instead of comparing the chunk to the
 searched string. This leads to the problem that ubifs is recognized as ubi 
 aswell.
 
 This fixes this by splitting up the string into chunks.
 
 Signed-off-by: Pascal Bach pascal.b...@siemens.com
Reviewed-by: Laurentiu Palcu laurentiu.pa...@intel.com

 ---
  meta/lib/oe/image.py |2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
 index c9b9033..5e07187 100644
 --- a/meta/lib/oe/image.py
 +++ b/meta/lib/oe/image.py
 @@ -109,7 +109,7 @@ class ImageDepGraph(object):
  # remove added nodes from deps_array
  for item in group:
  for node in self.graph:
 -if item in self.graph[node]:
 +if item in self.graph[node].split():
  self.deps_array[node][0] -= 1
  
  self.deps_array.pop(item, None)
 -- 
 1.7.10.4
 
 -- 
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 2/8] lttng-modules: update to 2.5.0

2014-08-12 Thread Laurentiu Palcu
Hi Martin,

On Tue, Aug 12, 2014 at 01:33:05PM +0200, Martin Jansa wrote:
   Sorry I thought you're maintainer of lttng-*, it seems like Laurentiu is
  
  No worries. I just get caught in the blast when lttng blows up during
  kernel uprevs, and I fix the wreckage so I can send out my updates :)
  
  If we don't hear from Laurentiu, I can still do the update, so I'll hold it
  as a separate patch here.
 
 Ping!
 
 Someone just sent patch to upgrade lttng-ust, so I've asked him for
 LICENSE fix as well.
It seems I missed your previous email... :/ My apologies.

Regarding lttng* recipes, as of a couple of months ago, RP is the
maintainer:

RECIPE_MAINTAINER_pn-lttng-modules = Richard Purdie 
richard.pur...@linuxfoundation.org
RECIPE_MAINTAINER_pn-lttng-tools = Richard Purdie 
richard.pur...@linuxfoundation.org
RECIPE_MAINTAINER_pn-lttng-ust = Richard Purdie 
richard.pur...@linuxfoundation.org

laurentiu
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] populate_sdk_base: add auto-completion in setup

2014-06-17 Thread Laurentiu Palcu
On Tue, Jun 17, 2014 at 03:16:53PM +0200, Dennis Meier wrote:
 Signed-off-by: Dennis Meier meier.den...@siemens.com
 ---
  meta/classes/populate_sdk_base.bbclass |3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)
 
 diff --git a/meta/classes/populate_sdk_base.bbclass 
 b/meta/classes/populate_sdk_base.bbclass
 index 35d837d..1182425 100644
 --- a/meta/classes/populate_sdk_base.bbclass
 +++ b/meta/classes/populate_sdk_base.bbclass
 @@ -173,9 +173,8 @@ if [ $verbose = 1 ] ; then
   set -x
  fi
  
 -printf Enter target directory for SDK (default: $DEFAULT_INSTALL_DIR): 
  if [ $target_sdk_dir =  ]; then
 - read target_sdk_dir
 + read -e -p Enter target directory for SDK (default: 
 $DEFAULT_INSTALL_DIR):  target_sdk_dir
   [ $target_sdk_dir =  ]  target_sdk_dir=$DEFAULT_INSTALL_DIR
  else
   echo $target_sdk_dir
Moving the above printf in the 'if' branch will make this 'echo' look
strange when the target directory is provided with the -d option... I
believe you can remove this 'echo' completely though. There is another
message, later, informing the user where the SDK gets installed.

laurentiu

 -- 
 1.7.10.4
 
 -- 
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] relocate_sdk.py: fix lib path error

2014-06-12 Thread Laurentiu Palcu
On Thu, Jun 12, 2014 at 06:14:46PM +0800, yzhu1 wrote:
 On 06/06/2014 08:22 PM, Khem Raj wrote:
 
 
 
 On Friday, June 6, 2014, rongqing...@windriver.com wrote:
 
 From: yzhu1 yanjun@windriver.com
 
 In centos 5.9 32bit, ld lib does not contain some flags, so ld
 lib is not parsed. So correct lib path is not got from ld lib.
 
 
 Can you explain with examples what's going on here ?
 
 Hi,
 
 Before relocate_sdk.sh is executed, it needs the parameters: path/
 ld-linux-x86-64.so.2 path/dmesg.util-linux path/kill.util-linux path/
 reset.util-linux .
 The file list (path/dmesg.util-linux path/kill.util-linux 
 path/reset.util-linux
 .) is collected by grep '\(executable\|dynamically linked\)'.
 
 So dynamically linked is very import to the file ld-linux-x86-64.so.2. But 
 in
 redhat 5.9(32 bit)
 
 This file is as below:
 
 ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), stripped
 
 In Ubuntu 12.04 (64bit)
 
 This file is as below:
 
 ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically
 linked, BuildID[sha1]=0xbb8c5184d8d41f31093193a2ec8d3f6f10964cd2, stripped
 
 In this case, we can find in Ubuntu 12.04(64 bit), the flag dynamically
 linked is present, so ld-linux-x86-64.so.2 can be included in the file list.
 relocate_sdk.py can work according to the information from
 ld-linux-x86-64.so.2. But in redhat 5.9(32bit), the flag does not exist.
 relocate_sdk.py can not get the information from ld-linux-x86-64.so.2 since
 this file is not included in the file list.
 
 The same error occurs on redhat6.0(32 bit).
 
 So the direct solution to this defect is to force this file
 ld-linux-x86-64.so.2 exist in file list.
 
 If any problem, please feel free to let me know.

This looks like a 'file' display issue. It has nothing to do with the
dynamic loader itself. However, the dynamic loader is the reason you
spotted the problem because it wasn't relocated and all the other
dynamically linked binaries depend on it...

Originally, all executable files were selected no matter if they were
scripts or other non-elf files. The relocation script checked for the
ELF magic number anyway and it moved on if the file was not ELF
format.

But, the following commit added this 'file' dependency for detecting
executables:

commit 65535bff09afd21b6d9f129651df6c70570c3aa0
Author: yzhu1 yanjun@windriver.com
Date:   Tue Nov 26 08:38:28 2013 +

populate_sdk: verify executable or dynamically linked library

My guess is that this commit was not the right fix for the problem it
describes. In fact, I just had a better look, and I spotted the problem
you were experiencing with empty files:

in scripts/relocate_sdk.py, in get_arch(), we check for the ELF magic
number and we do a read(16). Hence, if the file was zero sized, this
read would fail...

I suggest you add a check in relocate_sdk.py and make sure the file size is
bigger than the e_ident size (which is 16). Then, you can safely revert
65535bff09afd21b6d9f129651df6c70570c3aa0. Also, the current patch would no
longer be needed.

laurentiu
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/1] adt_installer: run autoreconf before configuring opkg

2014-05-12 Thread Laurentiu Palcu
The following changes since commit 50067e069c8cbc0d339819494f22ae7623d47aa6:

  bitbake: fetch2/perforce: Ensure command has a default (2014-05-11 15:26:25 
+0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/b6293_adt_installer_issue
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b6293_adt_installer_issue

Laurentiu Palcu (1):
  adt_installer: run autoreconf before configuring opkg

 .../installer/adt-installer/adt_installer  |1 +
 1 file changed, 1 insertion(+)

-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/1] adt_installer: run autoreconf before configuring opkg

2014-05-12 Thread Laurentiu Palcu
opkg fails to build on hosts with older autotools versions.

[YOCTO #6293]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../installer/adt-installer/adt_installer  |1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-devtools/installer/adt-installer/adt_installer 
b/meta/recipes-devtools/installer/adt-installer/adt_installer
index 1f2d039..7fc37f8 100755
--- a/meta/recipes-devtools/installer/adt-installer/adt_installer
+++ b/meta/recipes-devtools/installer/adt-installer/adt_installer
@@ -182,6 +182,7 @@ if [ ! -x $LOCAL_OPKG_LOC/bin/opkg-cl ]; then
   check_result
 
   echo_info Configure opkg ...\n
+  autoreconf
   ./configure --prefix=$parent_folder/$LOCAL_OPKG_LOC 
--with-opkglibdir=$OPKG_LIBDIR --enable-shared=no --disable-curl 
--disable-ssl-curl --disable-gpg --disable-shave  
$parent_folder/$YOCTOADT_INSTALL_LOG_FILE
   check_result
 
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 00/12] package upgrades

2014-04-29 Thread Laurentiu Palcu
The packages were individually compiled for x86, x86-64, arm, mips, ppc, then
fired the AB nightlies for the mentioned archs. Basic sanity tests, apart from
the AB ones, were performed on a core-image-sato image for qemux86.  Builds
finished successfully and no issues were spotted during tests.

laurentiu

The following changes since commit 183dac4036cb73ad6cc620002b0044d3484a398d:

  bitbake: Revert fetch2: Cleanup file checksum generation (2014-04-28 
12:54:06 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/upgrades
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/upgrades

Laurentiu Palcu (12):
  xf86-input-synaptics: upgrade to 1.7.4
  libdrm: upgrade to 2.4.53
  xserver-xorg: upgrade to 1.15.1
  python-pygobject: upgrade to 2.28.3
  gdb: upgrade to 7.7
  xproto: upgrade to 7.0.26
  xrandr: upgrade to 1.4.2
  xtrans: upgrade to 1.3.4
  kexec-tools: upgrade to 2.0.6
  nasm: upgrade to 2.11.02
  xauth: upgrade to 1.0.9
  fontconfig: upgrade to 2.11.1

 .../gdb/{gdb-7.6.2.inc = gdb-7.7.inc} |6 ++--
 ...canadian_7.6.2.bb = gdb-cross-canadian_7.7.bb} |0
 .../gdb/{gdb-cross_7.6.2.bb = gdb-cross_7.7.bb}   |0
 .../gdb-fix-cygwin-check-in-configure-script.patch |   38 
 .../gdb/{gdb_7.6.2.bb = gdb_7.7.bb}   |0
 .../nasm/{nasm_2.11.bb = nasm_2.11.02.bb} |4 +--
 ...bject_2.27.91.bb = python-pygobject_2.28.3.bb} |5 ++-
 .../drm/{libdrm_2.4.52.bb = libdrm_2.4.53.bb} |4 +--
 .../fontconfig/fontconfig/sysroot-arg.patch|   32 -
 .../{fontconfig_2.11.0.bb = fontconfig_2.11.1.bb} |8 ++---
 .../xorg-app/{xauth_1.0.8.bb = xauth_1.0.9.bb}|4 +--
 .../xorg-app/{xrandr_1.4.1.bb = xrandr_1.4.2.bb}  |4 +--
 ...tics_1.7.3.bb = xf86-input-synaptics_1.7.4.bb} |4 +--
 .../xorg-lib/{xtrans_1.3.3.bb = xtrans_1.3.4.bb}  |4 +--
 .../{xproto_7.0.25.bb = xproto_7.0.26.bb} |4 +--
 ...erver-xorg_1.15.0.bb = xserver-xorg_1.15.1.bb} |4 +--
 .../{kexec-tools_2.0.4.bb = kexec-tools_2.0.6.bb} |4 +--
 17 files changed, 26 insertions(+), 99 deletions(-)
 rename meta/recipes-devtools/gdb/{gdb-7.6.2.inc = gdb-7.7.inc} (61%)
 rename meta/recipes-devtools/gdb/{gdb-cross-canadian_7.6.2.bb = 
gdb-cross-canadian_7.7.bb} (100%)
 rename meta/recipes-devtools/gdb/{gdb-cross_7.6.2.bb = gdb-cross_7.7.bb} 
(100%)
 delete mode 100644 
meta/recipes-devtools/gdb/gdb/gdb-fix-cygwin-check-in-configure-script.patch
 rename meta/recipes-devtools/gdb/{gdb_7.6.2.bb = gdb_7.7.bb} (100%)
 rename meta/recipes-devtools/nasm/{nasm_2.11.bb = nasm_2.11.02.bb} (81%)
 rename meta/recipes-devtools/python/{python-pygobject_2.27.91.bb = 
python-pygobject_2.28.3.bb} (87%)
 rename meta/recipes-graphics/drm/{libdrm_2.4.52.bb = libdrm_2.4.53.bb} (46%)
 delete mode 100644 
meta/recipes-graphics/fontconfig/fontconfig/sysroot-arg.patch
 rename meta/recipes-graphics/fontconfig/{fontconfig_2.11.0.bb = 
fontconfig_2.11.1.bb} (83%)
 rename meta/recipes-graphics/xorg-app/{xauth_1.0.8.bb = xauth_1.0.9.bb} (68%)
 rename meta/recipes-graphics/xorg-app/{xrandr_1.4.1.bb = xrandr_1.4.2.bb} 
(72%)
 rename meta/recipes-graphics/xorg-driver/{xf86-input-synaptics_1.7.3.bb = 
xf86-input-synaptics_1.7.4.bb} (81%)
 rename meta/recipes-graphics/xorg-lib/{xtrans_1.3.3.bb = xtrans_1.3.4.bb} 
(82%)
 rename meta/recipes-graphics/xorg-proto/{xproto_7.0.25.bb = xproto_7.0.26.bb} 
(74%)
 rename meta/recipes-graphics/xorg-xserver/{xserver-xorg_1.15.0.bb = 
xserver-xorg_1.15.1.bb} (88%)
 rename meta/recipes-kernel/kexec/{kexec-tools_2.0.4.bb = 
kexec-tools_2.0.6.bb} (57%)

-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 02/12] libdrm: upgrade to 2.4.53

2014-04-29 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../drm/{libdrm_2.4.52.bb = libdrm_2.4.53.bb} |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-graphics/drm/{libdrm_2.4.52.bb = libdrm_2.4.53.bb} (46%)

diff --git a/meta/recipes-graphics/drm/libdrm_2.4.52.bb 
b/meta/recipes-graphics/drm/libdrm_2.4.53.bb
similarity index 46%
rename from meta/recipes-graphics/drm/libdrm_2.4.52.bb
rename to meta/recipes-graphics/drm/libdrm_2.4.53.bb
index 75b1846..323bef5 100644
--- a/meta/recipes-graphics/drm/libdrm_2.4.52.bb
+++ b/meta/recipes-graphics/drm/libdrm_2.4.53.bb
@@ -3,6 +3,6 @@ require libdrm.inc
 SRC_URI += file://installtests.patch \
 file://GNU_SOURCE_definition.patch \

-SRC_URI[md5sum] = 01b75624a5da3a7543923e54c3547a24
-SRC_URI[sha256sum] = 
fa693c2f1f61befcefbdcc396673e38481110bac9db610afa4b8afb2be0218c1
+SRC_URI[md5sum] = 342886a137ddd9ed4341675d132388ad
+SRC_URI[sha256sum] = 
1b0c28fd2f2b92d2df0a73d1aed88f43cb0dee1267aea6bc52ccb5fca5757a08
 
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 01/12] xf86-input-synaptics: upgrade to 1.7.4

2014-04-29 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 ...tics_1.7.3.bb = xf86-input-synaptics_1.7.4.bb} |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-graphics/xorg-driver/{xf86-input-synaptics_1.7.3.bb = 
xf86-input-synaptics_1.7.4.bb} (81%)

diff --git a/meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.3.bb 
b/meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.4.bb
similarity index 81%
rename from meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.3.bb
rename to meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.4.bb
index 89ae45c..8045790 100644
--- a/meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.3.bb
+++ b/meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.7.4.bb
@@ -12,8 +12,8 @@ advanced features of the touchpad to become available.
 
 LIC_FILES_CHKSUM = file://COPYING;md5=55aacd3535a741824955c5eb8f061398
 
-SRC_URI[md5sum] = 74c83e6cb53a0e15bcbe7cc73d63d2a1
-SRC_URI[sha256sum] = 
8b2a972043961195d056b84346317ec42bfa029095c9ee7aaf6deceba12e32d5
+SRC_URI[md5sum] = deaa740072c19fef8e2fb1d7787392b7
+SRC_URI[sha256sum] = 
56a2d2df7bd39e29f56102c62f153e023f3e9b2f5e255309d33fab8e81945af7
 
 DEPENDS += libxi mtdev libxtst
 
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 04/12] python-pygobject: upgrade to 2.28.3

2014-04-29 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 ...bject_2.27.91.bb = python-pygobject_2.28.3.bb} |5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
 rename meta/recipes-devtools/python/{python-pygobject_2.27.91.bb = 
python-pygobject_2.28.3.bb} (87%)

diff --git a/meta/recipes-devtools/python/python-pygobject_2.27.91.bb 
b/meta/recipes-devtools/python/python-pygobject_2.28.3.bb
similarity index 87%
rename from meta/recipes-devtools/python/python-pygobject_2.27.91.bb
rename to meta/recipes-devtools/python/python-pygobject_2.28.3.bb
index 49ee108..9dbe47c 100644
--- a/meta/recipes-devtools/python/python-pygobject_2.27.91.bb
+++ b/meta/recipes-devtools/python/python-pygobject_2.28.3.bb
@@ -5,7 +5,6 @@ LIC_FILES_CHKSUM = 
file://COPYING;md5=a916467b91076e631dd8edb7424769c7
 DEPENDS = python python-pygobject-native glib-2.0
 DEPENDS_class-native = python-native glib-2.0-native
 RDEPENDS_class-native = 
-PR = r6
 
 MAJ_VER = 
${@d.getVar('PV',1).split('.')[0]}.${@d.getVar('PV',1).split('.')[1]}
 
@@ -13,8 +12,8 @@ SRC_URI = 
${GNOME_MIRROR}/pygobject/${MAJ_VER}/pygobject-${PV}.tar.bz2 \
file://obsolete_automake_macros.patch \
 
 
-SRC_URI[md5sum] = 2b11a3050264721aac83188224b093a8
-SRC_URI[sha256sum] = 
a1dffbe2a8e0d490594554ed8d06f0ee4a371acb6c210e7f35158e9ae77e0df4
+SRC_URI[md5sum] = aa64900b274c4661a5c32e52922977f9
+SRC_URI[sha256sum] = 
7da88c169a56efccc516cebd9237da3fe518a343095a664607b368fe21df95b6
 S = ${WORKDIR}/pygobject-${PV}
 
 FILESPATH = ${FILE_DIRNAME}/python-pygobject:${FILE_DIRNAME}/files
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 05/12] gdb: upgrade to 7.7

2014-04-29 Thread Laurentiu Palcu
Removed the following patch(es):
 * gdb-fix-cygwin-check-in-configure-script.patch (changes included in release)

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../gdb/{gdb-7.6.2.inc = gdb-7.7.inc} |6 ++--
 ...canadian_7.6.2.bb = gdb-cross-canadian_7.7.bb} |0
 .../gdb/{gdb-cross_7.6.2.bb = gdb-cross_7.7.bb}   |0
 .../gdb-fix-cygwin-check-in-configure-script.patch |   38 
 .../gdb/{gdb_7.6.2.bb = gdb_7.7.bb}   |0
 5 files changed, 2 insertions(+), 42 deletions(-)
 rename meta/recipes-devtools/gdb/{gdb-7.6.2.inc = gdb-7.7.inc} (61%)
 rename meta/recipes-devtools/gdb/{gdb-cross-canadian_7.6.2.bb = 
gdb-cross-canadian_7.7.bb} (100%)
 rename meta/recipes-devtools/gdb/{gdb-cross_7.6.2.bb = gdb-cross_7.7.bb} 
(100%)
 delete mode 100644 
meta/recipes-devtools/gdb/gdb/gdb-fix-cygwin-check-in-configure-script.patch
 rename meta/recipes-devtools/gdb/{gdb_7.6.2.bb = gdb_7.7.bb} (100%)

diff --git a/meta/recipes-devtools/gdb/gdb-7.6.2.inc 
b/meta/recipes-devtools/gdb/gdb-7.7.inc
similarity index 61%
rename from meta/recipes-devtools/gdb/gdb-7.6.2.inc
rename to meta/recipes-devtools/gdb/gdb-7.7.inc
index ea69b39..62adcaa 100644
--- a/meta/recipes-devtools/gdb/gdb-7.6.2.inc
+++ b/meta/recipes-devtools/gdb/gdb-7.7.inc
@@ -4,9 +4,7 @@ LIC_FILES_CHKSUM = 
file://COPYING;md5=59530bdf33659b29e73d4adb9f9f6552 \
file://COPYING3.LIB;md5=6a6a8e020838b23406c81b19c1d46df6 \
file://COPYING.LIB;md5=9f604d8a4f8e74f4f5140845a21b6674
 
-SRC_URI +=  file://gdb-fix-cygwin-check-in-configure-script.patch 
-
 S = ${WORKDIR}/${BPN}-${PV}
 
-SRC_URI[md5sum] = 9ebf09fa76e4ca6034157086e9b20348
-SRC_URI[sha256sum] = 
17f7bcda1b24336aadcb1c5e703c31e59467e221c6f4353b720dffad73a7
+SRC_URI[md5sum] = 40051ff95b39bd57b14b1809e2c16152
+SRC_URI[sha256sum] = 
8814d98c2733639cb602b6ecd8d69e02498017e02b5724c9451c285b0e9ee173
diff --git a/meta/recipes-devtools/gdb/gdb-cross-canadian_7.6.2.bb 
b/meta/recipes-devtools/gdb/gdb-cross-canadian_7.7.bb
similarity index 100%
rename from meta/recipes-devtools/gdb/gdb-cross-canadian_7.6.2.bb
rename to meta/recipes-devtools/gdb/gdb-cross-canadian_7.7.bb
diff --git a/meta/recipes-devtools/gdb/gdb-cross_7.6.2.bb 
b/meta/recipes-devtools/gdb/gdb-cross_7.7.bb
similarity index 100%
rename from meta/recipes-devtools/gdb/gdb-cross_7.6.2.bb
rename to meta/recipes-devtools/gdb/gdb-cross_7.7.bb
diff --git 
a/meta/recipes-devtools/gdb/gdb/gdb-fix-cygwin-check-in-configure-script.patch 
b/meta/recipes-devtools/gdb/gdb/gdb-fix-cygwin-check-in-configure-script.patch
deleted file mode 100644
index 4e4647b..000
--- 
a/meta/recipes-devtools/gdb/gdb/gdb-fix-cygwin-check-in-configure-script.patch
+++ /dev/null
@@ -1,38 +0,0 @@
-Avoid false positives if the search pattern lose is found in path
-descriptions in comments generated by the preprocessor.
-
-See https://sourceware.org/bugzilla/show_bug.cgi?id=16152.

- gdb/configure| 2 +-
- gdb/configure.ac | 2 +-
- 3 files changed, 7 insertions(+), 2 deletions(-)
-
-diff --git a/gdb/configure b/gdb/configure
-index 5514b2f..b38e183 100755
 a/gdb/configure
-+++ b/gdb/configure
-@@ -12446,7 +12446,7 @@ lose
- #endif
- _ACEOF
- if (eval $ac_cpp conftest.$ac_ext) 25 |
--  $EGREP lose /dev/null 21; then :
-+  $EGREP ^lose$ /dev/null 21; then :
-   gdb_cv_os_cygwin=yes
- else
-   gdb_cv_os_cygwin=no
-diff --git a/gdb/configure.ac b/gdb/configure.ac
-index 9b73887..2947293 100644
 a/gdb/configure.ac
-+++ b/gdb/configure.ac
-@@ -1877,7 +1877,7 @@ AC_SUBST(WERROR_CFLAGS)
- 
- # In the Cygwin environment, we need some additional flags.
- AC_CACHE_CHECK([for cygwin], gdb_cv_os_cygwin,
--[AC_EGREP_CPP(lose, [
-+[AC_EGREP_CPP(^lose$, [
- #if defined (__CYGWIN__) || defined (__CYGWIN32__)
- lose
- #endif],[gdb_cv_os_cygwin=yes],[gdb_cv_os_cygwin=no])])
--- 
-1.8.4
-
diff --git a/meta/recipes-devtools/gdb/gdb_7.6.2.bb 
b/meta/recipes-devtools/gdb/gdb_7.7.bb
similarity index 100%
rename from meta/recipes-devtools/gdb/gdb_7.6.2.bb
rename to meta/recipes-devtools/gdb/gdb_7.7.bb
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 07/12] xrandr: upgrade to 1.4.2

2014-04-29 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../xorg-app/{xrandr_1.4.1.bb = xrandr_1.4.2.bb}  |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-graphics/xorg-app/{xrandr_1.4.1.bb = xrandr_1.4.2.bb} 
(72%)

diff --git a/meta/recipes-graphics/xorg-app/xrandr_1.4.1.bb 
b/meta/recipes-graphics/xorg-app/xrandr_1.4.2.bb
similarity index 72%
rename from meta/recipes-graphics/xorg-app/xrandr_1.4.1.bb
rename to meta/recipes-graphics/xorg-app/xrandr_1.4.2.bb
index 6ca6863..84f01b0 100644
--- a/meta/recipes-graphics/xorg-app/xrandr_1.4.1.bb
+++ b/meta/recipes-graphics/xorg-app/xrandr_1.4.2.bb
@@ -11,5 +11,5 @@ LIC_FILES_CHKSUM = 
file://COPYING;md5=fe1608bdb33cf8c62a4438f7d34679b3
 DEPENDS += libxrandr libxrender
 PE = 1
 
-SRC_URI[md5sum] = 52c3de0297bf45be6a189dc2e0515638
-SRC_URI[sha256sum] = 
67b554ab975652778bef587f86dab7fec8cb95dfd21c11d98a203dac5c241e50
+SRC_URI[md5sum] = 78fd973d9b532106f8777a3449176148
+SRC_URI[sha256sum] = 
b2e76ee92ff827f1c52ded7c666fe6f2704ca81cdeef882397da4e3e8ab490bc
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 03/12] xserver-xorg: upgrade to 1.15.1

2014-04-29 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 ...erver-xorg_1.15.0.bb = xserver-xorg_1.15.1.bb} |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-graphics/xorg-xserver/{xserver-xorg_1.15.0.bb = 
xserver-xorg_1.15.1.bb} (88%)

diff --git a/meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.0.bb 
b/meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.1.bb
similarity index 88%
rename from meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.0.bb
rename to meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.1.bb
index 1f9fa04..6a4168e 100644
--- a/meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.0.bb
+++ b/meta/recipes-graphics/xorg-xserver/xserver-xorg_1.15.1.bb
@@ -8,8 +8,8 @@ SRC_URI += file://crosscompile.patch \
 file://xorg-CVE-2013-6424.patch \

 
-SRC_URI[md5sum] = c2ace3697b32414094cf8c597c39d7d9
-SRC_URI[sha256sum] = 
613b2f2e7ee2a06d2f8e862c836dc70d319c52f1537749e027398f40086aabb8
+SRC_URI[md5sum] = e4c70262ed89764be8f8f5d699ed9227
+SRC_URI[sha256sum] = 
626db6882602ebe1ff81f7a4231c7ccc6ceb5032f2b5b3954bf749e1567221e2
 
 # These extensions are now integrated into the server, so declare the migration
 # path for in-place upgrades.
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 06/12] xproto: upgrade to 7.0.26

2014-04-29 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../{xproto_7.0.25.bb = xproto_7.0.26.bb} |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-graphics/xorg-proto/{xproto_7.0.25.bb = xproto_7.0.26.bb} 
(74%)

diff --git a/meta/recipes-graphics/xorg-proto/xproto_7.0.25.bb 
b/meta/recipes-graphics/xorg-proto/xproto_7.0.26.bb
similarity index 74%
rename from meta/recipes-graphics/xorg-proto/xproto_7.0.25.bb
rename to meta/recipes-graphics/xorg-proto/xproto_7.0.26.bb
index cf24125..0d591dc 100644
--- a/meta/recipes-graphics/xorg-proto/xproto_7.0.25.bb
+++ b/meta/recipes-graphics/xorg-proto/xproto_7.0.26.bb
@@ -15,6 +15,6 @@ SRC_URI += file://xproto_fix_for_x32.patch
 EXTRA_OECONF_append =  --enable-specs=no
 BBCLASSEXTEND = native nativesdk
 
-SRC_URI[md5sum] = 28311ef4edbbbf89f617a7f8a2e5648f
-SRC_URI[sha256sum] = 
92247485dc4ffc3611384ba84136591923da857212a7dc29f4ad7797e13909fe
+SRC_URI[md5sum] = 4dc2464bfeade23dab5de38da0f6b1b5
+SRC_URI[sha256sum] = 
636162c1759805a5a0114a369dffdeccb8af8c859ef6e1445f26a4e6e046514f
 
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 09/12] kexec-tools: upgrade to 2.0.6

2014-04-29 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../{kexec-tools_2.0.4.bb = kexec-tools_2.0.6.bb} |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-kernel/kexec/{kexec-tools_2.0.4.bb = 
kexec-tools_2.0.6.bb} (57%)

diff --git a/meta/recipes-kernel/kexec/kexec-tools_2.0.4.bb 
b/meta/recipes-kernel/kexec/kexec-tools_2.0.6.bb
similarity index 57%
rename from meta/recipes-kernel/kexec/kexec-tools_2.0.4.bb
rename to meta/recipes-kernel/kexec/kexec-tools_2.0.6.bb
index de44cf0..38a8947 100644
--- a/meta/recipes-kernel/kexec/kexec-tools_2.0.4.bb
+++ b/meta/recipes-kernel/kexec/kexec-tools_2.0.6.bb
@@ -2,8 +2,8 @@ require kexec-tools.inc
 export LDFLAGS = -L${STAGING_LIBDIR}
 EXTRA_OECONF =  --with-zlib=yes
 
-SRC_URI[md5sum] = b9f2a3ba0ba9c78625ee7a50532500d8
-SRC_URI[sha256sum] = 
6ba1872c58434b8e92506ff515c7ef64555671af54097bae51b833bda3f5126c
+SRC_URI[md5sum] = c39ea40a7598e49b6dc961ee7de38f57
+SRC_URI[sha256sum] = 
a9c6cd9adc8c1c37b3272782f581cb8b4b4070d0e3e921a558a9083f68dcf29a
 
 PACKAGES =+ kexec kdump
 
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 10/12] nasm: upgrade to 2.11.02

2014-04-29 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../nasm/{nasm_2.11.bb = nasm_2.11.02.bb} |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-devtools/nasm/{nasm_2.11.bb = nasm_2.11.02.bb} (81%)

diff --git a/meta/recipes-devtools/nasm/nasm_2.11.bb 
b/meta/recipes-devtools/nasm/nasm_2.11.02.bb
similarity index 81%
rename from meta/recipes-devtools/nasm/nasm_2.11.bb
rename to meta/recipes-devtools/nasm/nasm_2.11.02.bb
index 5f4b953..db84ec5 100644
--- a/meta/recipes-devtools/nasm/nasm_2.11.bb
+++ b/meta/recipes-devtools/nasm/nasm_2.11.02.bb
@@ -6,8 +6,8 @@ COMPATIBLE_HOST = '(x86_64|i.86).*-(linux|freebsd.*)'
 
 
 SRC_URI = http://www.nasm.us/pub/nasm/releasebuilds/${PV}/nasm-${PV}.tar.bz2 
-SRC_URI[md5sum] = 4cd558047ea5ed51fc2c7c94e249c7b8
-SRC_URI[sha256sum] = 
1ce7e897c67255a195367a60c739a90a0b33a4a73f058f7cda3253bcf975642b
+SRC_URI[md5sum] = 3bbc8ed83115b8caf7931f35ec3bc5e0
+SRC_URI[sha256sum] = 
ece26b5ef565f94d19a72756d05965e424d2e5ca55f88b949852da70dd62f0e0
 
 inherit autotools-brokensep
 
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 11/12] xauth: upgrade to 1.0.9

2014-04-29 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../xorg-app/{xauth_1.0.8.bb = xauth_1.0.9.bb}|4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-graphics/xorg-app/{xauth_1.0.8.bb = xauth_1.0.9.bb} (68%)

diff --git a/meta/recipes-graphics/xorg-app/xauth_1.0.8.bb 
b/meta/recipes-graphics/xorg-app/xauth_1.0.9.bb
similarity index 68%
rename from meta/recipes-graphics/xorg-app/xauth_1.0.8.bb
rename to meta/recipes-graphics/xorg-app/xauth_1.0.9.bb
index a2da49e..b173f4f 100644
--- a/meta/recipes-graphics/xorg-app/xauth_1.0.8.bb
+++ b/meta/recipes-graphics/xorg-app/xauth_1.0.9.bb
@@ -8,5 +8,5 @@ LIC_FILES_CHKSUM = 
file://COPYING;md5=5ec74dd7ea4d10c4715a7c44f159a40b
 DEPENDS += libxau libxext libxmu
 PE = 1
 
-SRC_URI[md5sum] = 50ee2ec0836c0186b05ec8fdcfd566d0
-SRC_URI[sha256sum] = 
a8696ae7a50c699d5fb3a41408b60d98843d19ea46e9f09e391cb98c8f7fd4f7
+SRC_URI[md5sum] = 7d6003f32838d5b688e2c8a131083271
+SRC_URI[sha256sum] = 
56ce1523eb48b1f8a4f4244fe1c3d8e6af1a3b7d4b0e6063582421b0b68dc28f
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 12/12] fontconfig: upgrade to 2.11.1

2014-04-29 Thread Laurentiu Palcu
Removed the following patch(es):
 * sysroot-arg.patch (changes included in release)

Correct the fccache.c license checksum and the line numbers because the
license snippet has moved and, also, at the previous upgrade the lines
were wrong (shifted a couple of lines).

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../fontconfig/fontconfig/sysroot-arg.patch|   32 
 .../{fontconfig_2.11.0.bb = fontconfig_2.11.1.bb} |8 ++---
 2 files changed, 4 insertions(+), 36 deletions(-)
 delete mode 100644 
meta/recipes-graphics/fontconfig/fontconfig/sysroot-arg.patch
 rename meta/recipes-graphics/fontconfig/{fontconfig_2.11.0.bb = 
fontconfig_2.11.1.bb} (83%)

diff --git a/meta/recipes-graphics/fontconfig/fontconfig/sysroot-arg.patch 
b/meta/recipes-graphics/fontconfig/fontconfig/sysroot-arg.patch
deleted file mode 100644
index abfdbc7..000
--- a/meta/recipes-graphics/fontconfig/fontconfig/sysroot-arg.patch
+++ /dev/null
@@ -1,32 +0,0 @@
-Upstream-Status: Submitted [https://bugs.freedesktop.org/show_bug.cgi?id=72044]
-Signed-off-by: Ross Burton ross.bur...@intel.com
-
-From f2ade764cc9f009e1fe25b856b24b7695f66a952 Mon Sep 17 00:00:00 2001
-From: Ross Burton ross.bur...@intel.com
-Date: Tue, 26 Nov 2013 17:18:25 +
-Subject: [PATCH] fc-cache: --sysroot option takes an argument
-
-The getopt_long option definitions say that sysroot doesn't take an argument,
-when it in fact does.
-
-Signed-off-by: Ross Burton ross.bur...@intel.com

- fc-cache/fc-cache.c | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/fc-cache/fc-cache.c b/fc-cache/fc-cache.c
-index 27c7513..bf3b6b4 100644
 a/fc-cache/fc-cache.c
-+++ b/fc-cache/fc-cache.c
-@@ -67,7 +67,7 @@
- const struct option longopts[] = {
- {force, 0, 0, 'f'},
- {really-force, 0, 0, 'r'},
--{sysroot, 0, 0, 'y'},
-+{sysroot, required_argument, 0, 'y'},
- {system-only, 0, 0, 's'},
- {version, 0, 0, 'V'},
- {verbose, 0, 0, 'v'},
--- 
-1.8.4.4
-
diff --git a/meta/recipes-graphics/fontconfig/fontconfig_2.11.0.bb 
b/meta/recipes-graphics/fontconfig/fontconfig_2.11.1.bb
similarity index 83%
rename from meta/recipes-graphics/fontconfig/fontconfig_2.11.0.bb
rename to meta/recipes-graphics/fontconfig/fontconfig_2.11.1.bb
index 48385a3..797b321 100644
--- a/meta/recipes-graphics/fontconfig/fontconfig_2.11.0.bb
+++ b/meta/recipes-graphics/fontconfig/fontconfig_2.11.1.bb
@@ -14,16 +14,16 @@ BUGTRACKER = 
https://bugs.freedesktop.org/enter_bug.cgi?product=fontconfig;
 LICENSE = MIT-style  MIT  PD
 LIC_FILES_CHKSUM = file://COPYING;md5=7a0449e9bc5370402a94c00204beca3d \
 
file://src/fcfreetype.c;endline=45;md5=5d9513e3196a1fbfdfa94051c09dfc84 \
-
file://src/fccache.c;beginline=1131;endline=1146;md5=754c7b855210ee746e5f0b840fad9a9f
+
file://src/fccache.c;beginline=1143;endline=1158;md5=0326cfeb4a7333dd4dd25fbbc4b9f27f
 
 SECTION = libs
 
 DEPENDS = expat freetype zlib
 
 SRC_URI = http://fontconfig.org/release/fontconfig-${PV}.tar.gz \
-   file://sysroot-arg.patch
-SRC_URI[md5sum] = 84278204cd7f36adbea7ad8094e039ac
-SRC_URI[sha256sum] = 
274c047487b90dacbaa55f4d70b8cdcd556944e7251ce9cf1de442c00a16343b
+   
+SRC_URI[md5sum] = e75e303b4f7756c2b16203a57ac87eba
+SRC_URI[sha256sum] = 
b6b066c7dce3f436fdc0dfbae9d36122b38094f4f53bd8dffd45e195b0540d8d
 
 PACKAGES =+ fontconfig-utils
 FILES_${PN} =+ ${datadir}/xml/*
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 08/12] xtrans: upgrade to 1.3.4

2014-04-29 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../xorg-lib/{xtrans_1.3.3.bb = xtrans_1.3.4.bb}  |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-graphics/xorg-lib/{xtrans_1.3.3.bb = xtrans_1.3.4.bb} 
(82%)

diff --git a/meta/recipes-graphics/xorg-lib/xtrans_1.3.3.bb 
b/meta/recipes-graphics/xorg-lib/xtrans_1.3.4.bb
similarity index 82%
rename from meta/recipes-graphics/xorg-lib/xtrans_1.3.3.bb
rename to meta/recipes-graphics/xorg-lib/xtrans_1.3.4.bb
index 9895ea3..f2b8921 100644
--- a/meta/recipes-graphics/xorg-lib/xtrans_1.3.3.bb
+++ b/meta/recipes-graphics/xorg-lib/xtrans_1.3.4.bb
@@ -20,5 +20,5 @@ inherit gettext
 
 BBCLASSEXTEND = native nativesdk
 
-SRC_URI[md5sum] = 2f14c31ab556fc91039f51a113b38aa2
-SRC_URI[sha256sum] = 
622db4adce224581a44fbe41321bbb0bdc0c78aec586ba83548f1f1c6e8a09bf
+SRC_URI[md5sum] = a615e17d9fee6f097fc3b716eacb3dca
+SRC_URI[sha256sum] = 
054d4ee3efd52508c753e9f7bc655ef185a29bd2850dd9e2fc2ccc33544f583a
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] relocate_sdk.py: Possible bug, /lib64/ld-linux-x86-64.so.2 not relocated

2014-04-03 Thread Laurentiu Palcu
Hi Stefan,

Can you please have a look at the binaries before relocation? Just to
make sure...

So, for that, run the installer with -R option:

./your_toolchain_installer.sh -R

Here, I'm interested which is the default suggested path (see below):

Enter target directory for SDK (default: /opt/poky/1.5+snapshot):

Then, run 'readelf -p .interp' on those binaries. They should all start
with the default prefix.

laurentiu


On Thu, Apr 03, 2014 at 09:59:34AM +0200, Stefan Agner wrote:
 Hi Laurentiu,
 
 Am 2014-03-03 09:38, schrieb Laurentiu Palcu:
  This is the correct behavior. We shouldn't relocate binaries that use
  host's dynamic loader.
  
  When I install the SDK with -S (copy the relocate scripts), and
  remove the condition around line 95, the binaries work as expected.
  Can you please run the installer script with -R so it doesn't perform
  any relocation on binaries, and then:
  
  readelf -p .interp
  /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc
  
  It looks like the default dynamic loader path of the toolchain binaries
  start with /lib* or /usr/lib* which is not quite right... It should be:
  ${SDKPATH}/sysroots/${SDK_SYS}.
 
 $ readelf -p .interp
 /usr/local/oecore-x86_64-non-reloc/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc
 
 String dump of section '.interp':
   [ 0]  /lib64/ld-linux-x86-64.so.2
 
 
 So this looks wrong then, right? 
 
 I also get the same for python and qmake:
 $ readelf -p .interp 
 /usr/local/oecore-x86_64-new/sysroots/x86_64-angstromsdk-linux/usr/bin/python2
 
 String dump of section '.interp':
   [ 0]  /lib64/ld-linux-x86-64.so.2
 
 $ readelf -p .interp 
 /usr/local/oecore-x86_64-new/sysroots/x86_64-angstromsdk-linux/usr/bin/qmake2
 
 
 String dump of section '.interp':
   [ 0]  /lib64/ld-linux-x86-64.so.2
 
 How can I make sure all those binaries get linked against the SDK link
 loader?
 
 --
 Stefan
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] image.bbclass: USE_DEVFS is now useless

2014-04-02 Thread Laurentiu Palcu
Hi Matthieu,

On Wed, Feb 26, 2014 at 11:11:28AM +, Matthieu CRAPET wrote:
 Hi,
 
  
 
 Since :
 
 http://cgit.openembedded.org/openembedded-core/commit/meta/classes/
 image.bbclass?id=a83144bac8d67704ff66f5dc0fc56f5b63979694
 
  
 
 USE_DEVFS is not considered anymore.
 
  
 
 Setting IMAGE_DEVICE_TABLE or IMAGE_DEVICE_TABLES to empty string in an image
 recipe works (makedevs is not called)
 
  
 
 Should be we drop USE_DEVFS variable (meta/classes/image.bbclass) or (re)add
 check in meta/lib/oe/rootfs.py?
 
 I can propose a patch.
Please do. It looks like I totally missed the USE_DEVFS check...

laurentiu
 
  
 
 Regards,
 
 Matthieu
 

 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 0/1] package_manager.py: delete RPM db locks after calling rpmresolve

2014-03-28 Thread Laurentiu Palcu
Put this on hold. I'll do some more investigations. I want to see what's
the deal with those rpm lock DBs and why we don't delete them only once,
after the rootfs generation has completely finished (after postprocess
commands are called, etc).

laurentiu

On Thu, Mar 27, 2014 at 06:18:52PM +0200, Laurentiu Palcu wrote:
 The following changes since commit c4eeaa8e35e926b6d1f633549f76d1ba9ed9278b:
 
   bitbake: knotty: Show a link to the logfile for failed setscene tasks 
 (2014-03-27 10:42:08 +)
 
 are available in the git repository at:
 
   git://git.yoctoproject.org/poky-contrib lpalcu/b6049_do_rootfs_crash
   
 http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b6049_do_rootfs_crash
 
 Laurentiu Palcu (1):
   package_manager.py: delete RPM db locks after calling rpmresolve
 
  meta/lib/oe/package_manager.py |   12 +---
  1 file changed, 9 insertions(+), 3 deletions(-)
 
 -- 
 1.7.9.5
 
 -- 
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/2] package_manager.py: leave the __db.00* files in place

2014-03-28 Thread Laurentiu Palcu
Do not delete the __db.00* files in the PackageManager class. Leave this
operation up to the client classes. One side effect of this deletion was
the following message appearing in the output of the next rpm command
executed:

rpmdb: BDB1540 configured environment flags incompatible with existing
environment

We might also gain some time here by not deleting/creating those files
very often.

[YOCTO #6049]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/package_manager.py |8 +---
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 764ab72..a8360fe 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -299,9 +299,6 @@ class RpmPkgsList(PkgsList):
 # bb.note(cmd)
 tmp_output = subprocess.check_output(cmd, 
stderr=subprocess.STDOUT, shell=True).strip()
 
-rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.rootfs_dir)
-for f in rpm_db_locks:
-bb.utils.remove(f, True)
 except subprocess.CalledProcessError as e:
 bb.fatal(Cannot get the installed packages list. Command '%s' 
  returned %d:\n%s % (cmd, e.returncode, e.output))
@@ -1101,7 +1098,6 @@ class RpmPM(PackageManager):
 output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, 
shell=True).strip()
 bb.note(output)
 os.chmod(saved_dir, 0755)
-self._unlock_rpm_db()
 except subprocess.CalledProcessError as e:
 bb.fatal(Invoke save_rpmpostinst failed. Command '%s' 
  returned %d:\n%s % (cmd, e.returncode, e.output))
@@ -1117,14 +1113,12 @@ class RpmPM(PackageManager):
 self._invoke_smart('flag --set ignore-recommends %s' % i)
 self._invoke_smart('channel --add rpmsys type=rpm-sys -y')
 
-self._unlock_rpm_db()
-
 '''
 The rpm db lock files were produced after invoking rpm to query on
 build system, and they caused the rpm on target didn't work, so we
 need to unlock the rpm db by removing the lock files.
 '''
-def _unlock_rpm_db(self):
+def unlock_rpm_db(self):
 # Remove rpm db lock files
 rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.target_rootfs)
 for f in rpm_db_locks:
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/2] Don't fool around with rpm __db.00* files

2014-03-28 Thread Laurentiu Palcu
We were deleting/creating those files too many times during the rootfs creation
process. That led to various rpmdb related issues. Instead, remove those files
at the end of the rootfs creation process which seems the most sane thing to do.

laurentiu

The following changes since commit 8c3eb5ee4582b6f6d489549290937657f37fc19e:

  packagegroup-toolset-native: Update after ocf-linux - cryptodev-linux change 
(2014-03-27 19:53:30 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/b6049_do_rootfs_crash
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b6049_do_rootfs_crash

Laurentiu Palcu (2):
  package_manager.py: leave the __db.00* files in place
  rootfs.py: add new cleanup method

 meta/lib/oe/package_manager.py |8 +---
 meta/lib/oe/rootfs.py  |   25 +
 2 files changed, 26 insertions(+), 7 deletions(-)

-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/2] rootfs.py: add new cleanup method

2014-03-28 Thread Laurentiu Palcu
This commit adds a new _cleanup() internal method that will be called at
the end of rootfs creation, so that each backend can delete various
files that were probably generated during rootfs postprocess execution,
etc.

[YOCTO #6049]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/rootfs.py |   25 +
 1 file changed, 25 insertions(+)

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index 0e6c8bc..3eac3c9 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -50,6 +50,15 @@ class Rootfs(object):
 def _handle_intercept_failure(self, failed_script):
 pass
 
+
+The _cleanup() method should be used to clean-up stuff that we don't really
+want to end up on target. For example, in the case of RPM, the DB locks.
+The method is called, once, at the end of create() method.
+
+@abstractmethod
+def _cleanup(self):
+pass
+
 def _exec_shell_cmd(self, cmd):
 fakerootcmd = self.d.getVar('FAKEROOT', True)
 if fakerootcmd is not None:
@@ -117,6 +126,8 @@ class Rootfs(object):
 
 self._generate_kernel_module_deps()
 
+self._cleanup()
+
 def _uninstall_uneeded(self):
 if base_contains(IMAGE_FEATURES, package-management,
  True, False, self.d):
@@ -358,6 +369,13 @@ class RpmRootfs(Rootfs):
 for pkg in registered_pkgs.split():
 self.pm.save_rpmpostinst(pkg)
 
+def _cleanup(self):
+# during the execution of postprocess commands, rpm is called several
+# times to get the files installed, dependencies, etc. This creates the
+# __db.00* (Berkeley DB files that hold locks, rpm specific environment
+# settings, etc.), that should not get into the final rootfs
+self.pm.unlock_rpm_db()
+
 
 class DpkgRootfs(Rootfs):
 def __init__(self, d, manifest_dir):
@@ -431,6 +449,9 @@ class DpkgRootfs(Rootfs):
 def _log_check(self):
 pass
 
+def _cleanup(self):
+pass
+
 
 class OpkgRootfs(Rootfs):
 def __init__(self, d, manifest_dir):
@@ -694,6 +715,10 @@ class OpkgRootfs(Rootfs):
 def _log_check(self):
 pass
 
+def _cleanup(self):
+pass
+
+
 def create_rootfs(d, manifest_dir=None):
 env_bkp = os.environ.copy()
 
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/1] populate_sdk_base: add dependency of do_package_write_* tasks

2014-03-28 Thread Laurentiu Palcu
The following changes since commit 5eceedf0326abaee632b32c7cb7ec74e6ba2d6d7:

  ocf-linux: remove recipe (2014-03-27 15:46:52 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/b5900_fix_adt_installer_issue
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5900_fix_adt_installer_issue

Laurentiu Palcu (1):
  populate_sdk_base: add dependency of do_package_write_* tasks

 meta/classes/populate_sdk_base.bbclass |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/1] populate_sdk_base: add dependency of do_package_write_* tasks

2014-03-28 Thread Laurentiu Palcu
nativesdk packages were created only for the first backend listed in
PACKAGE_CLASSES. Hence, if one had it set to package_rpm package_ipk
and did a 'bitbake -c populate_sdk core-image-something', the nativesdk
packages were created only for rpm.

This is particularily bad for adt-installer which is based on opkg
repos.

Credits go to richard.pur...@linuxfoundation.org who suggested me this
fix.

[YOCTO #5900]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/classes/populate_sdk_base.bbclass |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/populate_sdk_base.bbclass 
b/meta/classes/populate_sdk_base.bbclass
index 235d672..81da206 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -333,5 +333,5 @@ populate_sdk_log_check() {
 do_populate_sdk[dirs] = ${TOPDIR}
 do_populate_sdk[depends] += ${@' '.join([x + ':do_populate_sysroot' for x in 
d.getVar('SDK_DEPENDS', True).split()])}
 do_populate_sdk[rdepends] = ${@' '.join([x + ':do_populate_sysroot' for x in 
d.getVar('SDK_RDEPENDS', True).split()])}
-do_populate_sdk[recrdeptask] += do_packagedata
+do_populate_sdk[recrdeptask] += do_packagedata do_package_write_rpm 
do_package_write_ipk do_package_write_deb
 addtask populate_sdk
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/1] image.py: check file exists before deleting

2014-03-28 Thread Laurentiu Palcu
The following changes since commit 8c3eb5ee4582b6f6d489549290937657f37fc19e:

  packagegroup-toolset-native: Update after ocf-linux - cryptodev-linux change 
(2014-03-27 19:53:30 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/b6029_ignore_broken_symlinks
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b6029_ignore_broken_symlinks

Laurentiu Palcu (1):
  image.py: check file exists before deleting

 meta/lib/oe/image.py |3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/1] image.py: check file exists before deleting

2014-03-28 Thread Laurentiu Palcu
When RM_OLD_IMAGE = 1, we delete old images but we didn't check they
actually exist...

[YOCTO #6029]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/image.py |3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
index a03b73e..c9b9033 100644
--- a/meta/lib/oe/image.py
+++ b/meta/lib/oe/image.py
@@ -192,7 +192,8 @@ class Image(ImageDepGraph):
 if img.find(self.d.getVar('IMAGE_LINK_NAME', True)) == 0:
 img = os.path.join(deploy_dir, img)
 if os.path.islink(img):
-if self.d.getVar('RM_OLD_IMAGE', True) == 1:
+if self.d.getVar('RM_OLD_IMAGE', True) == 1 and \
+os.path.exists(os.path.realpath(img)):
 os.remove(os.path.realpath(img))
 
 os.remove(img)
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/1] run-postinsts: fix issue with checking IMAGE_FEATURES

2014-03-27 Thread Laurentiu Palcu
The old implementation was wrong. It was not very generic and it checked
IMAGE_FEATURES while building the recipe, which led to various issues
with the generation of the final script. That is, the run-postinsts
script was generated once, while building the package for the first
time. Hence, any other changes to IMAGE_FEATURES, like removing/adding
'package-management' did not reflect in the final script.

This commit makes run-postinsts script autodetect the backend used for
creating the image, making it generic.

[YOCTO #5666]
[YOCTO #5972]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../run-postinsts/run-postinsts/run-postinsts  |   30 +++-
 .../run-postinsts/run-postinsts_1.0.bb |2 --
 2 files changed, 23 insertions(+), 9 deletions(-)

diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts 
b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
index 08cfa9e..f547a7b 100755
--- a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
+++ b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
@@ -8,9 +8,29 @@
 # The following script will run all the scriptlets found in 
#SYSCONFDIR#/deb-postinsts,
 # #SYSCONFDIR#/ipk-postinsts or #SYSCONFDIR#/rpm-posinsts.
 
-pm=#IMAGE_PKGTYPE#
-pm_installed=#PM_INSTALLED#
-pi_dir=#SYSCONFDIR#/${pm}-postinsts
+# the order of this list is important, do not change!
+backend_list=rpm deb ipk
+
+pm_installed=false
+
+for pm in $backend_list; do
+   pi_dir=#SYSCONFDIR#/$pm-postinsts
+
+   [ -d $pi_dir ]  break
+
+   case $pm in
+   deb)
+   if [ -s /var/lib/dpkg/status ]; then
+   pm_installed=true
+   break
+   fi
+   ;;
+
+   ipk)
+   pm_installed=true
+   ;;
+   esac
+done
 
 remove_rcsd_link () {
if [ -n `which update-rc.d` ]; then
@@ -56,10 +76,6 @@ if $pm_installed; then
deb)
eval dpkg --configure -a $append_log
;;
-
-   rpm)
-   exec_postinst_scriptlets
-   ;;
esac
 else
exec_postinst_scriptlets
diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb 
b/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb
index e990c67..64f85c2 100644
--- a/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb
+++ b/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb
@@ -37,8 +37,6 @@ do_install() {
sed -i -e 's:#SYSCONFDIR#:${sysconfdir}:g' \
-e 's:#SBINDIR#:${sbindir}:g' \
-e 's:#BASE_BINDIR#:${base_bindir}:g' \
-   -e 's:#IMAGE_PKGTYPE#:${IMAGE_PKGTYPE}:g' \
-   -e 's:#PM_INSTALLED#:${@base_contains(IMAGE_FEATURES, 
package-management, true, false, d)}:g' \
${D}${sbindir}/run-postinsts \
${D}${systemd_unitdir}/system/run-postinsts.service
 }
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/1] Fix run-postinsts

2014-03-27 Thread Laurentiu Palcu
The following changes since commit 0150bc30d3674301631c2e9b6c64e01058fd1070:

  bitbake: runqueue: Really fix sigchld handling (2014-03-18 23:05:53 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/b5666_unique_run_postinsts
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5666_unique_run_postinsts

Laurentiu Palcu (1):
  run-postinsts: fix issue with checking IMAGE_FEATURES

 .../run-postinsts/run-postinsts/run-postinsts  |   30 +++-
 .../run-postinsts/run-postinsts_1.0.bb |2 --
 2 files changed, 23 insertions(+), 9 deletions(-)

-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/1] package_manager.py: delete RPM db locks after calling rpmresolve

2014-03-27 Thread Laurentiu Palcu
The following changes since commit c4eeaa8e35e926b6d1f633549f76d1ba9ed9278b:

  bitbake: knotty: Show a link to the logfile for failed setscene tasks 
(2014-03-27 10:42:08 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/b6049_do_rootfs_crash
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b6049_do_rootfs_crash

Laurentiu Palcu (1):
  package_manager.py: delete RPM db locks after calling rpmresolve

 meta/lib/oe/package_manager.py |   12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/1] package_manager.py: delete RPM db locks after calling rpmresolve

2014-03-27 Thread Laurentiu Palcu
If the locks are not removed, the output of the next rpm command
executed will contain the following string:

rpmdb: BDB1540 configured environment flags incompatible with
existing environment

And this will create various parsing issues.

[YOCTO #6049]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/package_manager.py |   12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 764ab72..0f22e46 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -242,6 +242,12 @@ class RpmPkgsList(PkgsList):
 self.ml_prefix_list, self.ml_os_list = \
 RpmIndexer(d, rootfs_dir).get_ml_prefix_and_os_list(arch_var, 
os_var)
 
+def _unlock_rpm_db(self):
+# Remove rpm db lock files
+rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.rootfs_dir)
+for f in rpm_db_locks:
+bb.utils.remove(f, True)
+
 '''
 Translate the RPM/Smart format names to the OE multilib format names
 '''
@@ -281,6 +287,8 @@ class RpmPkgsList(PkgsList):
 
 try:
 output = subprocess.check_output(cmd, 
stderr=subprocess.STDOUT).strip()
+
+self._unlock_rpm_db()
 except subprocess.CalledProcessError as e:
 bb.fatal(Cannot get the package dependencies. Command '%s' 
  returned %d:\n%s % (' '.join(cmd), e.returncode, 
e.output))
@@ -299,9 +307,7 @@ class RpmPkgsList(PkgsList):
 # bb.note(cmd)
 tmp_output = subprocess.check_output(cmd, 
stderr=subprocess.STDOUT, shell=True).strip()
 
-rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.rootfs_dir)
-for f in rpm_db_locks:
-bb.utils.remove(f, True)
+self._unlock_rpm_db()
 except subprocess.CalledProcessError as e:
 bb.fatal(Cannot get the installed packages list. Command '%s' 
  returned %d:\n%s % (cmd, e.returncode, e.output))
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] SDK build refers to existing installed SDK

2014-03-26 Thread Laurentiu Palcu
On Wed, Mar 26, 2014 at 07:40:26AM +, Mats Kärrman wrote:
 On Wednesday, March 26, 2014 4:23 AM, Khem Raj wrote:
   Hi,
  
   I just ran into an unexpected error while doing some multitasking.
   Project is based on OE-core Dora distroless.
  
   While my build machine was busy building a new SDK using -c 
   populate_sdk I simultaneously deleted an older SDK that was installed 
   under /usr/local and installed a different one. This resulted in the 
   build stopping with the following error:
  
   | x86_64-oesdk-linux-gcc  
   --sysroot=/home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/sysroots/x86_64-nativesdk-oesdk-linux
-DHAVE_CONFIG_H -I../ncurses 
   -I/home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/work/x86_64-nativesdk-oesdk-linux/nativesdk-ncurses/5.9-r15.1/ncurses-5.9/ncurses

   -isystem/home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/sysroots/x86_64-nativesdk-oesdk-linux/usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/include
-D_GNU_SOURCE -DNDEBUG -I. -I../include 
   -I/home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/work/x86_64-nativesdk-oesdk-linux/nativesdk-ncurses/5.9-r15.1/ncurses-5.9/ncurses/../include
-I/usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/include 
   -isystem/home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/sysroots/x86_64-nativesdk-oesdk-linux/usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/include
-O2 -pipe  --param max-inline-insns-single=1200 -fPIC -c 
   /home/makr/projects/xxx/svn/main/trunk/oe/build/tmp/work/x86
  _64-
   nativesdk-oesdk-linux/nativesdk-ncurses/5.9-r15.1/ncurses-5.9/ncurses/base/lib_beep.c
-o ../obj_s/lib_beep.o
   | cc1: error: 
   /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/include: 
   Permission denied
  
   The offending directory is given with an include directive which seems 
   strange, building a new SDK should not poke around in an old one...
  
   Config  log files available on request.
  
  Was the installed SDK in your path that fed into the OE build env which was 
  generating new SDK ?
 
 No, it was not.
 I have saved the output of bitbake nativesdk-ncurses -e if you're 
 interested.
I had a quick look at the generated Makefile for nativesdk-ncurses and,
apparently, it seems to have -I${includedir} added to the compilation
flags. This doesn't look like the right thing...

laurentiu

 
 BR // Mats
 -- 
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation

2014-03-26 Thread Laurentiu Palcu
On Wed, Mar 26, 2014 at 09:15:43AM +, Sarbu, Florin-Ionut (Florin) wrote:
 I get the same error on do_rootfs for a rpm based image:
 
 Exception: UnboundLocalError: local variable 'new_arch' referenced before 
 assignment
Does this happen on latest master? Do you have the 4 patches, starting with 
this one

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d91e35640d19471213122d36288315f071c37432

in your tree? Did you activate buildhistory? Can you provide more info
regarding the steps you followed to reproduce this?

laurentiu
 
 Florin
 
 From: openembedded-core-boun...@lists.openembedded.org 
 [openembedded-core-boun...@lists.openembedded.org] on behalf of Laurentiu 
 Palcu [laurentiu.pa...@intel.com]
 Sent: Friday, March 07, 2014 8:35 AM
 To: Richard Purdie
 Cc: openembedded-core@lists.openembedded.org
 Subject: Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files 
 creation
 
 On Thu, Mar 06, 2014 at 09:55:38PM +, Richard Purdie wrote:
  On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
   Call the new python routines.
  
   [YOCTO #5904]
  
   Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
   ---
meta/classes/buildhistory.bbclass |9 -
1 file changed, 8 insertions(+), 1 deletion(-)
  
   diff --git a/meta/classes/buildhistory.bbclass 
   b/meta/classes/buildhistory.bbclass
   index ef4135b..01b0082 100644
   --- a/meta/classes/buildhistory.bbclass
   +++ b/meta/classes/buildhistory.bbclass
   @@ -319,6 +319,12 @@ python buildhistory_list_installed() {
  
with open(pkgs_list_file, 'w') as pkgs_list:
pkgs_list.write(list_installed_packages(d, 'file'))
   +
   +pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
   +  bh_installed_pkgs_deps.txt)
   +
   +with open(pkgs_deps_file, 'w') as pkgs_deps:
   +pkgs_deps.write(list_installed_packages(d, 'deps'))
}
  
  
   @@ -338,7 +344,8 @@ buildhistory_get_installed() {
  
   # Produce dependency graph
   # First, quote each name to handle characters that cause issues for 
   dot
   -   rootfs_list_installed_depends | sed 's:\([^| ]*\):\1:g'  
   $1/depends.tmp
   +   cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):\1:g' 
$1/depends.tmp  \
   +   rm ${WORKDIR}/bh_installed_pkgs_deps.txt
   # Change delimiter from pipe to - and set style for recommend lines
   sed -i -e 's:|: - :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' 
   $1/depends.tmp
   # Add header, sorted and de-duped contents and footer and then delete 
   the temp file
 
  With this patch, a bitbake core-image-minimal -c populate_sdk resulted
 No, not quite. The bug appears to be in
 RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.
 
 CCing Hongxu.
 
 laurentiu
 
  in:
 
  ERROR: Error executing a python function in 
  /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
 
  The stack trace of python calls that resulted in this exception/failure was:
  File: 'buildhistory_list_installed', lineno: 18, function: module
   0014:with open(pkgs_deps_file, 'w') as pkgs_deps:
   0015:pkgs_deps.write(list_installed_packages(d, 'deps'))
   0016:
   0017:
   *** 0018:buildhistory_list_installed(d)
   0019:
  File: 'buildhistory_list_installed', lineno: 9, function: 
  buildhistory_list_installed
   0005:pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
   0006:  bh_installed_pkgs.txt)
   0007:
   0008:with open(pkgs_list_file, 'w') as pkgs_list:
   *** 0009:pkgs_list.write(list_installed_packages(d, 'file'))
   0010:
   0011:pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
   0012:  bh_installed_pkgs_deps.txt)
   0013:
  File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: 
  list_installed_packages
   0717:if img_type == rpm:
   0718:return RpmPM(d,
   0719: rootfs_dir,
   0720: d.getVar('TARGET_VENDOR', True)
   *** 0721: ).list_installed(format)
   0722:elif img_type == ipk:
   0723:return OpkgPM(d,
   0724:  rootfs_dir,
   0725:  d.getVar(IPKGCONF_TARGET, True),
  File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, 
  function: list_installed
   0850:pkg = line.split()[0]
   0851:arch = line.split()[1]
   0852:ver = line.split()[2]
   0853:pkgorigin = line.split()[3]
   *** 0854:new_pkg, new_arch = 
  self._pkg_translate_smart_to_oe(pkg, arch)
   0855:
   0856:if format == arch:
   0857:output.append('%s %s' % (new_pkg, new_arch))
   0858:elif format == file

Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation

2014-03-26 Thread Laurentiu Palcu
Please file a bug on this.

thanks,
laurentiu

On Wed, Mar 26, 2014 at 01:45:40PM +0200, Florin Sarbu wrote:
 On 03/26/2014 12:09 PM, Laurentiu Palcu wrote:
 On Wed, Mar 26, 2014 at 09:15:43AM +, Sarbu, Florin-Ionut (Florin) wrote:
 I get the same error on do_rootfs for a rpm based image:
 
 Exception: UnboundLocalError: local variable 'new_arch' referenced before 
 assignment
 Does this happen on latest master?
 Yes.
 Do you have the 4 patches, starting with this one
 
 http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d91e35640d19471213122d36288315f071c37432
 
 in your tree?
 Yes, I have that commit in poky.
   Did you activate buildhistory? Can you provide more info
 regarding the steps you followed to reproduce this?
 I am trying to build the gemini-image from the meta-ivi layer
 g...@git.yoctoproject.org:meta-ivi.git The build steps are those from
 the README.md file
 
 Florin
 
 laurentiu
 Florin
 
 From: openembedded-core-boun...@lists.openembedded.org 
 [openembedded-core-boun...@lists.openembedded.org] on behalf of Laurentiu 
 Palcu [laurentiu.pa...@intel.com]
 Sent: Friday, March 07, 2014 8:35 AM
 To: Richard Purdie
 Cc: openembedded-core@lists.openembedded.org
 Subject: Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency 
 files creation
 
 On Thu, Mar 06, 2014 at 09:55:38PM +, Richard Purdie wrote:
 On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
 Call the new python routines.
 
 [YOCTO #5904]
 
 Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
 ---
   meta/classes/buildhistory.bbclass |9 -
   1 file changed, 8 insertions(+), 1 deletion(-)
 
 diff --git a/meta/classes/buildhistory.bbclass 
 b/meta/classes/buildhistory.bbclass
 index ef4135b..01b0082 100644
 --- a/meta/classes/buildhistory.bbclass
 +++ b/meta/classes/buildhistory.bbclass
 @@ -319,6 +319,12 @@ python buildhistory_list_installed() {
 
   with open(pkgs_list_file, 'w') as pkgs_list:
   pkgs_list.write(list_installed_packages(d, 'file'))
 +
 +pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
 +  bh_installed_pkgs_deps.txt)
 +
 +with open(pkgs_deps_file, 'w') as pkgs_deps:
 +pkgs_deps.write(list_installed_packages(d, 'deps'))
   }
 
 
 @@ -338,7 +344,8 @@ buildhistory_get_installed() {
 
  # Produce dependency graph
  # First, quote each name to handle characters that cause issues for 
  dot
 -   rootfs_list_installed_depends | sed 's:\([^| ]*\):\1:g'  
 $1/depends.tmp
 +   cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):\1:g' 
  $1/depends.tmp  \
 +   rm ${WORKDIR}/bh_installed_pkgs_deps.txt
  # Change delimiter from pipe to - and set style for recommend lines
  sed -i -e 's:|: - :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' 
  $1/depends.tmp
  # Add header, sorted and de-duped contents and footer and then 
  delete the temp file
 With this patch, a bitbake core-image-minimal -c populate_sdk resulted
 No, not quite. The bug appears to be in
 RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.
 
 CCing Hongxu.
 
 laurentiu
 
 in:
 
 ERROR: Error executing a python function in 
 /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
 
 The stack trace of python calls that resulted in this exception/failure 
 was:
 File: 'buildhistory_list_installed', lineno: 18, function: module
   0014:with open(pkgs_deps_file, 'w') as pkgs_deps:
   0015:pkgs_deps.write(list_installed_packages(d, 'deps'))
   0016:
   0017:
   *** 0018:buildhistory_list_installed(d)
   0019:
 File: 'buildhistory_list_installed', lineno: 9, function: 
 buildhistory_list_installed
   0005:pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
   0006:  bh_installed_pkgs.txt)
   0007:
   0008:with open(pkgs_list_file, 'w') as pkgs_list:
   *** 0009:pkgs_list.write(list_installed_packages(d, 'file'))
   0010:
   0011:pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
   0012:  bh_installed_pkgs_deps.txt)
   0013:
 File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: 
 list_installed_packages
   0717:if img_type == rpm:
   0718:return RpmPM(d,
   0719: rootfs_dir,
   0720: d.getVar('TARGET_VENDOR', True)
   *** 0721: ).list_installed(format)
   0722:elif img_type == ipk:
   0723:return OpkgPM(d,
   0724:  rootfs_dir,
   0725:  d.getVar(IPKGCONF_TARGET, True),
 File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, 
 function: list_installed
   0850:pkg = line.split()[0]
   0851:arch = line.split()[1]
   0852:ver = line.split()[2]
   0853

[OE-core] [PATCH v2 0/4] SDK buildhistory fixes (cover letter only)

2014-03-19 Thread Laurentiu Palcu
v2:
 * I passed deploy_dir instead of target_rootfs to RpmPkgsList constructor...
   Strangely enough, this could be caught only when building core-image-minimal
   (which I didn't use in my tests).

The buildhistory dependency files for target/host SDK packages were not properly
created because the wrapper function called, list_installed_packages(), was 
always
looking in the image rootfs.

This patchset will rename the old wrapper function to 
image_list_installed_packages()
and create a new one, for SDK stuff, sdk_list_installed_packages().

The changes in package_manager.py, even though they appear to be lots, its the
same code moved around from one class to a newly created PkgsList class. So, the
logic remains the same.

Tested for all backends (buildhistory activated) with the following:

bitbake core-image-sato  bitbake -c populate_sdk core-image-sato

laurentiu

The following changes since commit 0150bc30d3674301631c2e9b6c64e01058fd1070:

  bitbake: runqueue: Really fix sigchld handling (2014-03-18 23:05:53 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/buildhistory_sdk_dep_files_fix
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/buildhistory_sdk_dep_files_fix

Laurentiu Palcu (4):
  package_manager.py: create separate class for installed packages
listing
  rootfs.py, sdk.py: adjust/create the wrappers for creating installed
packages list
  image.bbclass, license.bbclass: adjust the name of
list_installed_packages()
  buildhistory.bbclass: create proper dependency files for SDK

 meta/classes/buildhistory.bbclass |   40 ++--
 meta/classes/image.bbclass|4 +-
 meta/classes/license.bbclass  |4 +-
 meta/lib/oe/package_manager.py|  376 +
 meta/lib/oe/rootfs.py |   19 +-
 meta/lib/oe/sdk.py|   18 ++
 6 files changed, 264 insertions(+), 197 deletions(-)

-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/4] rootfs.py, sdk.py: adjust/create the wrappers for creating installed packages list

2014-03-18 Thread Laurentiu Palcu
Since we created a new PkgsList object that will deal with listing the
installed packages in a rootfs, use the new class both for images and
SDKs in the wrapper functions.

The old list_installed_packages() wrapper listed only the packages inside
an image rootfs. It didn't deal with target/host SDK rootfs's.

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/rootfs.py |   19 ---
 meta/lib/oe/sdk.py|   18 ++
 2 files changed, 22 insertions(+), 15 deletions(-)

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index 30a1321..0e6c8bc 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -709,28 +709,17 @@ def create_rootfs(d, manifest_dir=None):
 os.environ.update(env_bkp)
 
 
-def list_installed_packages(d, format=None, rootfs_dir=None):
+def image_list_installed_packages(d, format=None, rootfs_dir=None):
 if not rootfs_dir:
 rootfs_dir = d.getVar('IMAGE_ROOTFS', True)
 
 img_type = d.getVar('IMAGE_PKGTYPE', True)
 if img_type == rpm:
-return RpmPM(d,
- rootfs_dir,
- d.getVar('TARGET_VENDOR', True)
- ).list_installed(format)
+return RpmPkgsList(d, rootfs_dir).list(format)
 elif img_type == ipk:
-return OpkgPM(d,
-  rootfs_dir,
-  d.getVar(IPKGCONF_TARGET, True),
-  d.getVar(ALL_MULTILIB_PACKAGE_ARCHS, True)
-  ).list_installed(format)
+return OpkgPkgsList(d, rootfs_dir, d.getVar(IPKGCONF_TARGET, 
True)).list(format)
 elif img_type == deb:
-return DpkgPM(d,
-  rootfs_dir,
-  d.getVar('PACKAGE_ARCHS', True),
-  d.getVar('DPKG_ARCH', True)
-  ).list_installed(format)
+return DpkgPkgsList(d, rootfs_dir).list(format)
 
 if __name__ == __main__:
 
diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py
index 01a1807..5643199 100644
--- a/meta/lib/oe/sdk.py
+++ b/meta/lib/oe/sdk.py
@@ -289,6 +289,24 @@ class DpkgSdk(Sdk):
 bb.utils.remove(os.path.join(self.sdk_output, var), True)
 
 
+def sdk_list_installed_packages(d, target, format=None, rootfs_dir=None):
+if rootfs_dir is None:
+sdk_output = d.getVar('SDK_OUTPUT', True)
+target_path = d.getVar('SDKTARGETSYSROOT', True).strip('/')
+
+rootfs_dir = [sdk_output, os.path.join(sdk_output, 
target_path)][target is True]
+
+img_type = d.getVar('IMAGE_PKGTYPE', True)
+if img_type == rpm:
+arch_var = [SDK_PACKAGE_ARCHS, None][target is True]
+os_var = [SDK_OS, None][target is True]
+return RpmPkgsList(d, rootfs_dir, arch_var, os_var).list(format)
+elif img_type == ipk:
+conf_file_var = [IPKGCONF_SDK, IPKGCONF_Target][target is True]
+return OpkgPkgsList(d, rootfs_dir, d.getVar(conf_file_var, 
True)).list(format)
+elif img_type == deb:
+return DpkgPkgsList(d, rootfs_dir).list(format)
+
 def populate_sdk(d, manifest_dir=None):
 env_bkp = os.environ.copy()
 
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/4] package_manager.py: create separate class for installed packages listing

2014-03-18 Thread Laurentiu Palcu
This commit creates a new class that has the only purpose to generate
various listings of installed packages in the rootfs.

Basically, the methods involved in listing the installed packages, that
were part of each backend PM class implementation, were moved to this
new class.

This change avoids instantiating a new PM object just to get the list of
installed packages in a certain rootfs.

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/package_manager.py |  376 ++--
 1 file changed, 211 insertions(+), 165 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 1279b50..0426b23 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -213,6 +213,213 @@ class DpkgIndexer(Indexer):
 return(result)
 
 
+class PkgsList(object):
+__metaclass__ = ABCMeta
+
+def __init__(self, d, rootfs_dir):
+self.d = d
+self.rootfs_dir = rootfs_dir
+
+@abstractmethod
+def list(self, format=None):
+pass
+
+
+class RpmPkgsList(PkgsList):
+def __init__(self, d, rootfs_dir, arch_var=None, os_var=None):
+super(RpmPkgsList, self).__init__(d, rootfs_dir)
+
+self.rpm_cmd = bb.utils.which(os.getenv('PATH'), rpm)
+self.image_rpmlib = os.path.join(self.rootfs_dir, 'var/lib/rpm')
+
+self.ml_prefix_list, self.ml_os_list = \
+RpmIndexer(d, rootfs_dir).get_ml_prefix_and_os_list(arch_var, 
os_var)
+
+'''
+Translate the RPM/Smart format names to the OE multilib format names
+'''
+def _pkg_translate_smart_to_oe(self, pkg, arch):
+new_pkg = pkg
+fixed_arch = arch.replace('_', '-')
+found = 0
+for mlib in self.ml_prefix_list:
+for cmp_arch in self.ml_prefix_list[mlib]:
+fixed_cmp_arch = cmp_arch.replace('_', '-')
+if fixed_arch == fixed_cmp_arch:
+if mlib == 'default':
+new_pkg = pkg
+new_arch = cmp_arch
+else:
+new_pkg = mlib + '-' + pkg
+# We need to strip off the ${mlib}_ prefix on the arch
+new_arch = cmp_arch.replace(mlib + '_', '')
+
+# Workaround for bug 3565. Simply look to see if we
+# know of a package with that name, if not try again!
+filename = os.path.join(self.d.getVar('PKGDATA_DIR', True),
+'runtime-reverse',
+new_pkg)
+if os.path.exists(filename):
+found = 1
+break
+
+if found == 1 and fixed_arch == fixed_cmp_arch:
+break
+#bb.note('%s, %s - %s, %s' % (pkg, arch, new_pkg, new_arch))
+return new_pkg, new_arch
+
+def _list_pkg_deps(self):
+cmd = [bb.utils.which(os.getenv('PATH'), rpmresolve),
+   -t, self.image_rpmlib]
+
+try:
+output = subprocess.check_output(cmd, 
stderr=subprocess.STDOUT).strip()
+except subprocess.CalledProcessError as e:
+bb.fatal(Cannot get the package dependencies. Command '%s' 
+ returned %d:\n%s % (' '.join(cmd), e.returncode, 
e.output))
+
+return output
+
+def list(self, format=None):
+if format == deps:
+return self._list_pkg_deps()
+
+cmd = self.rpm_cmd + ' --root ' + self.rootfs_dir
+cmd += ' -D _dbpath /var/lib/rpm -qa'
+cmd +=  --qf '[%{NAME} %{ARCH} %{VERSION} %{PACKAGEORIGIN}\n]'
+
+try:
+# bb.note(cmd)
+tmp_output = subprocess.check_output(cmd, 
stderr=subprocess.STDOUT, shell=True).strip()
+
+rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.rootfs_dir)
+for f in rpm_db_locks:
+bb.utils.remove(f, True)
+except subprocess.CalledProcessError as e:
+bb.fatal(Cannot get the installed packages list. Command '%s' 
+ returned %d:\n%s % (cmd, e.returncode, e.output))
+
+output = list()
+for line in tmp_output.split('\n'):
+if len(line.strip()) == 0:
+continue
+pkg = line.split()[0]
+arch = line.split()[1]
+ver = line.split()[2]
+pkgorigin = line.split()[3]
+new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
+
+if format == arch:
+output.append('%s %s' % (new_pkg, new_arch))
+elif format == file:
+output.append('%s %s %s' % (new_pkg, pkgorigin, new_arch))
+elif format == ver:
+output.append('%s %s %s' % (new_pkg, new_arch, ver))
+else:
+output.append('%s' % (new_pkg))
+
+output.sort

[OE-core] [PATCH 3/4] image.bbclass, license.bbclass: adjust the name of list_installed_packages()

2014-03-18 Thread Laurentiu Palcu
The old wrapper got renamed to image_list_installed_packages().

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/classes/image.bbclass   |4 ++--
 meta/classes/license.bbclass |4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 51d16d7..9a04288 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -330,9 +330,9 @@ make_zimage_symlink_relative () {
 }
 
 python write_image_manifest () {
-from oe.rootfs import list_installed_packages
+from oe.rootfs import image_list_installed_packages
 with open(d.getVar('IMAGE_MANIFEST', True), 'w+') as image_manifest:
-image_manifest.write(list_installed_packages(d, 'ver'))
+image_manifest.write(image_list_installed_packages(d, 'ver'))
 }
 
 # Make login manager(s) enable automatic login.
diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index a0b877d..08f0665 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -20,9 +20,9 @@ python write_package_manifest() {
# Get list of installed packages
license_image_dir = d.expand('${LICENSE_DIRECTORY}/${IMAGE_NAME}')
bb.utils.mkdirhier(license_image_dir)
-   from oe.rootfs import list_installed_packages
+   from oe.rootfs import image_list_installed_packages
open(os.path.join(license_image_dir, 'package.manifest'),
- 'w+').write(list_installed_packages(d))
+ 'w+').write(image_list_installed_packages(d))
 }
 
 license_create_manifest() {
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/4] SDK buildhistory fixes

2014-03-18 Thread Laurentiu Palcu
The buildhistory dependency files for target/host SDK packages were not properly
created because the wrapper function called, list_installed_packages(), was 
always
looking in the image rootfs.

This patchset will rename the old wrapper function to 
image_list_installed_packages()
and create a new one, for SDK stuff, sdk_list_installed_packages().

The changes in package_manager.py, even though they appear to be lots, its the
same code moved around from one class to a newly created PkgsList class. So, the
logic remains the same.

Tested for all backends (buildhistory activated) with the following:

bitbake core-image-sato  bitbake -c populate_sdk core-image-sato

laurentiu

The following changes since commit 6bbb179cc526c86631dfcb140e3dd51a8c07a52d:

  bitbake: runqueue: More carefully handle the sigchld handler (2014-03-18 
10:23:13 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/buildhistory_sdk_dep_files_fix
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/buildhistory_sdk_dep_files_fix

Laurentiu Palcu (4):
  package_manager.py: create separate class for installed packages
listing
  rootfs.py, sdk.py: adjust/create the wrappers for creating installed
packages list
  image.bbclass, license.bbclass: adjust the name of
list_installed_packages()
  buildhistory.bbclass: create proper dependency files for SDK

 meta/classes/buildhistory.bbclass |   40 ++--
 meta/classes/image.bbclass|4 +-
 meta/classes/license.bbclass  |4 +-
 meta/lib/oe/package_manager.py|  376 +
 meta/lib/oe/rootfs.py |   19 +-
 meta/lib/oe/sdk.py|   18 ++
 6 files changed, 264 insertions(+), 197 deletions(-)

-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 4/4] buildhistory.bbclass: create proper dependency files for SDK

2014-03-18 Thread Laurentiu Palcu
The old functions were calling the list_installed_packages() wrapper
function that only listed the packages in an image rootfs. Even for
target/host SDK. Also, a python crash was possible if 'bitbake -c
populate_sdk core-image-*' was called without calling 'bitbake
core-image-*' first. That's because the wrapper was always looking into
the image rootfs...

This commit fixes the problem and calls the right wrapper for image/sdk.

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/classes/buildhistory.bbclass |   40 +
 1 file changed, 27 insertions(+), 13 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass 
b/meta/classes/buildhistory.bbclass
index 5d0a229..262095f 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -313,22 +313,36 @@ def write_pkghistory(pkginfo, d):
 if os.path.exists(filevarpath):
 os.unlink(filevarpath)
 
-python buildhistory_list_installed() {
-from oe.rootfs import list_installed_packages
+#
+# rootfs_type can be: image, sdk_target, sdk_host
+#
+def buildhistory_list_installed(d, rootfs_type=image):
+from oe.rootfs import image_list_installed_packages
+from oe.sdk import sdk_list_installed_packages
+
+process_list = [('file', 'bh_installed_pkgs.txt'),\
+('deps', 'bh_installed_pkgs_deps.txt')]
 
-pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
-  bh_installed_pkgs.txt)
+for output_type, output_file in process_list:
+output_file_full = os.path.join(d.getVar('WORKDIR', True), output_file)
 
-with open(pkgs_list_file, 'w') as pkgs_list:
-pkgs_list.write(list_installed_packages(d, 'file'))
+with open(output_file_full, 'w') as output:
+if rootfs_type == image:
+output.write(image_list_installed_packages(d, output_type))
+else:
+output.write(sdk_list_installed_packages(d, rootfs_type == 
sdk_target, output_type))
 
-pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
-  bh_installed_pkgs_deps.txt)
+python buildhistory_list_installed_image() {
+buildhistory_list_installed(d)
+}
 
-with open(pkgs_deps_file, 'w') as pkgs_deps:
-pkgs_deps.write(list_installed_packages(d, 'deps'))
+python buildhistory_list_installed_sdk_target() {
+buildhistory_list_installed(d, sdk_target)
 }
 
+python buildhistory_list_installed_sdk_host() {
+buildhistory_list_installed(d, sdk_host)
+}
 
 buildhistory_get_installed() {
mkdir -p $1
@@ -471,15 +485,15 @@ END
 }
 
 # By prepending we get in before the removal of packaging files
-ROOTFS_POSTPROCESS_COMMAND =+  buildhistory_list_installed ;\
+ROOTFS_POSTPROCESS_COMMAND =+  buildhistory_list_installed_image ;\
 buildhistory_get_image_installed ; 
 
 IMAGE_POSTPROCESS_COMMAND +=  buildhistory_get_imageinfo ; 
 
 # We want these to be the last run so that we get called after complementary 
package installation
-POPULATE_SDK_POST_TARGET_COMMAND_append =  buildhistory_list_installed ;\
+POPULATE_SDK_POST_TARGET_COMMAND_append =  
buildhistory_list_installed_sdk_target ;\
 
buildhistory_get_sdk_installed_target ; 
-POPULATE_SDK_POST_HOST_COMMAND_append =  buildhistory_list_installed ;\
+POPULATE_SDK_POST_HOST_COMMAND_append =  buildhistory_list_installed_sdk_host 
;\
   buildhistory_get_sdk_installed_host 
; 
 
 SDK_POSTPROCESS_COMMAND += buildhistory_get_sdkinfo ; 
-- 
1.7.9.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation

2014-03-10 Thread Laurentiu Palcu
I succeeded to replicate this. I'll take a look at it.

laurentiu

On Sat, Mar 08, 2014 at 04:23:22PM +0800, Hongxu Jia wrote:
 On 03/07/2014 02:35 PM, Laurentiu Palcu wrote:
 On Thu, Mar 06, 2014 at 09:55:38PM +, Richard Purdie wrote:
 On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
 Call the new python routines.
 
 [YOCTO #5904]
 
 Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
 ---
   meta/classes/buildhistory.bbclass |9 -
   1 file changed, 8 insertions(+), 1 deletion(-)
 
 diff --git a/meta/classes/buildhistory.bbclass 
 b/meta/classes/buildhistory.bbclass
 index ef4135b..01b0082 100644
 --- a/meta/classes/buildhistory.bbclass
 +++ b/meta/classes/buildhistory.bbclass
 @@ -319,6 +319,12 @@ python buildhistory_list_installed() {
   with open(pkgs_list_file, 'w') as pkgs_list:
   pkgs_list.write(list_installed_packages(d, 'file'))
 +
 +pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
 +  bh_installed_pkgs_deps.txt)
 +
 +with open(pkgs_deps_file, 'w') as pkgs_deps:
 +pkgs_deps.write(list_installed_packages(d, 'deps'))
   }
 
 @@ -338,7 +344,8 @@ buildhistory_get_installed() {
# Produce dependency graph
# First, quote each name to handle characters that cause issues for dot
 -  rootfs_list_installed_depends | sed 's:\([^| ]*\):\1:g'  
 $1/depends.tmp
 +  cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):\1:g'  
 $1/depends.tmp  \
 +  rm ${WORKDIR}/bh_installed_pkgs_deps.txt
# Change delimiter from pipe to - and set style for recommend lines
sed -i -e 's:|: - :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' 
  $1/depends.tmp
# Add header, sorted and de-duped contents and footer and then delete 
  the temp file
 With this patch, a bitbake core-image-minimal -c populate_sdk resulted
 No, not quite. The bug appears to be in
 RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.
 
 CCing Hongxu.
 
 Hi Laurentiu and Richard,
 
 The reason is pkg's arch was not found in the multilib arch list.
 
 I could not reproduce the defect on my local build with these
 patches applyed.
 
 The attach a patch that add some trace code to see which
 pkg's arch is not found in this situation.
 
 //Hongxu
 
 laurentiu
 
 in:
 
 ERROR: Error executing a python function in 
 /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
 
 The stack trace of python calls that resulted in this exception/failure was:
 File: 'buildhistory_list_installed', lineno: 18, function: module
   0014:with open(pkgs_deps_file, 'w') as pkgs_deps:
   0015:pkgs_deps.write(list_installed_packages(d, 'deps'))
   0016:
   0017:
   *** 0018:buildhistory_list_installed(d)
   0019:
 File: 'buildhistory_list_installed', lineno: 9, function: 
 buildhistory_list_installed
   0005:pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
   0006:  bh_installed_pkgs.txt)
   0007:
   0008:with open(pkgs_list_file, 'w') as pkgs_list:
   *** 0009:pkgs_list.write(list_installed_packages(d, 'file'))
   0010:
   0011:pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
   0012:  bh_installed_pkgs_deps.txt)
   0013:
 File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: 
 list_installed_packages
   0717:if img_type == rpm:
   0718:return RpmPM(d,
   0719: rootfs_dir,
   0720: d.getVar('TARGET_VENDOR', True)
   *** 0721: ).list_installed(format)
   0722:elif img_type == ipk:
   0723:return OpkgPM(d,
   0724:  rootfs_dir,
   0725:  d.getVar(IPKGCONF_TARGET, True),
 File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, 
 function: list_installed
   0850:pkg = line.split()[0]
   0851:arch = line.split()[1]
   0852:ver = line.split()[2]
   0853:pkgorigin = line.split()[3]
   *** 0854:new_pkg, new_arch = 
  self._pkg_translate_smart_to_oe(pkg, arch)
   0855:
   0856:if format == arch:
   0857:output.append('%s %s' % (new_pkg, new_arch))
   0858:elif format == file:
 File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, 
 function: _pkg_translate_smart_to_oe
   0472:
   0473:if found == 1 and fixed_arch == fixed_cmp_arch:
   0474:break
   0475:#bb.note('%s, %s - %s, %s' % (pkg, arch, new_pkg, 
  new_arch))
   *** 0476:return new_pkg, new_arch
   0477:
   0478:def _search_pkg_name_in_feeds(self, pkg, feed_archs):
   0479:for arch in feed_archs:
   0480:arch = arch.replace('-', '_')
 Exception: UnboundLocalError: local variable

[OE-core] [PATCH 1/1] xf86-video-intel: add recipe for 2.99.910, remove the git one

2014-03-07 Thread Laurentiu Palcu
Intel graphics stack releases = 2013Q3 need
xf86-video-intel = 2.99.902. However, keep the stable release around
too, in case people need it.

The git recipe is not really used. Remove.

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/conf/distro/include/default-versions.inc  |3 ++
 .../xorg-driver/xf86-video-intel_2.99.910.bb   |   31 
 .../xorg-driver/xf86-video-intel_git.bb|   23 ---
 3 files changed, 34 insertions(+), 23 deletions(-)
 create mode 100644 
meta/recipes-graphics/xorg-driver/xf86-video-intel_2.99.910.bb
 delete mode 100644 meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb

diff --git a/meta/conf/distro/include/default-versions.inc 
b/meta/conf/distro/include/default-versions.inc
index 53ec2e7..a9a931d 100644
--- a/meta/conf/distro/include/default-versions.inc
+++ b/meta/conf/distro/include/default-versions.inc
@@ -9,3 +9,6 @@ PREFERRED_VERSION_python-native ?= 2.7.3
 
 # Force the older version of liberation-fonts until we fix the fontforge issue
 PREFERRED_VERSION_liberation-fonts ?= 1.04
+
+# Intel video stack 2013Q3 and newer need a version =2.99.902
+PREFERRED_VERSION_xf86-video-intel ?= 2.99.910
diff --git a/meta/recipes-graphics/xorg-driver/xf86-video-intel_2.99.910.bb 
b/meta/recipes-graphics/xorg-driver/xf86-video-intel_2.99.910.bb
new file mode 100644
index 000..b74d174
--- /dev/null
+++ b/meta/recipes-graphics/xorg-driver/xf86-video-intel_2.99.910.bb
@@ -0,0 +1,31 @@
+require xorg-driver-video.inc
+
+SUMMARY = X.Org X server -- Intel integrated graphics chipsets driver
+
+DESCRIPTION = intel is an Xorg driver for Intel integrated graphics \
+chipsets. The driver supports depths 8, 15, 16 and 24. On some chipsets, \
+the driver supports hardware accelerated 3D via the Direct Rendering \
+Infrastructure (DRI).
+
+LIC_FILES_CHKSUM = file://COPYING;md5=8730ad58d11c7bbad9a7066d69f7808e
+
+DEPENDS += virtual/libx11 drm libpciaccess pixman
+
+SRC_URI += file://compat-api-Map-changes-of-DamageUnregister-API-in-1..patch \
+   
+
+PACKAGECONFIG ??= sna udev ${@base_contains('DISTRO_FEATURES', 'opengl', 
'dri', '', d)}
+
+PACKAGECONFIG[dri] = --enable-dri,--disable-dri,xf86driproto dri2proto
+PACKAGECONFIG[sna] = --enable-sna,--disable-sna
+PACKAGECONFIG[uxa] = --enable-uxa,--disable-uxa
+PACKAGECONFIG[udev] = --enable-udev,--disable-udev,udev
+PACKAGECONFIG[xvmc] = --enable-xvmc,--disable-xvmc,libxvmc
+
+# --enable-kms-only option is required by ROOTLESS_X
+EXTRA_OECONF += '${@base_conditional( ROOTLESS_X, 1,  --enable-kms-only, 
, d )}'
+
+COMPATIBLE_HOST = '(i.86|x86_64).*-linux'
+
+SRC_URI[md5sum] = a9a5c2c15766c06a024381efe0d724bb
+SRC_URI[sha256sum] = 
203d46064449da0e23a111418dfb189422ba96ea08707167c8dee463e2d745b1
diff --git a/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb 
b/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb
deleted file mode 100644
index d4c7654..000
--- a/meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb
+++ /dev/null
@@ -1,23 +0,0 @@
-require xorg-driver-video.inc
-
-SUMMARY = X.Org X server -- Intel integrated graphics chipsets driver
-
-DESCRIPTION = intel is an Xorg driver for Intel integrated graphics \
-chipsets. The driver supports depths 8, 15, 16 and 24. On some chipsets, \
-the driver supports hardware accelerated 3D via the Direct Rendering \
-Infrastructure (DRI).
-
-DEPENDS += virtual/libx11 drm dri2proto glproto \
-   virtual/libgl xineramaproto libpciaccess udev
-
-SRCREV = 87ea531c5dc5b39809395b277c330854f019
-PV = 2.10.0+git${SRCPV}
-PR = ${INC_PR}.1
-
-EXTRA_OECONF += --disable-xvmc --enable-dri --disable-static
-
-SRC_URI = git://anongit.freedesktop.org/git/xorg/driver/xf86-video-intel
-
-S = ${WORKDIR}/git
-
-COMPATIBLE_HOST = '(i.86|x86_64).*-linux'
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/1] xf86-video-intel: add recipe for 2.99.910, remove the git one

2014-03-07 Thread Laurentiu Palcu
The following changes since commit 2f9bf7bc97cb832f4db13df62ce09d96ce75e810:

  PR bumps to remove PRINC use in meta-fsl-arm (2014-03-07 13:39:11 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/b5705_q4_intel_graphics_stack
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5705_q4_intel_graphics_stack

Laurentiu Palcu (1):
  xf86-video-intel: add recipe for 2.99.910, remove the git one

 meta/conf/distro/include/default-versions.inc  |3 ++
 .../xorg-driver/xf86-video-intel_2.99.910.bb   |   31 
 .../xorg-driver/xf86-video-intel_git.bb|   23 ---
 3 files changed, 34 insertions(+), 23 deletions(-)
 create mode 100644 
meta/recipes-graphics/xorg-driver/xf86-video-intel_2.99.910.bb
 delete mode 100644 meta/recipes-graphics/xorg-driver/xf86-video-intel_git.bb

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/2] run-postinsts: use it for opkg/dpkg too

2014-03-06 Thread Laurentiu Palcu
Currently, opkg/dpkg have their own postinstalls that create a
run-postinsts script which is run at first boot.

This commit prepares the run-postinsts recipe/script to be used by
opkg/dpkg when DISTRO_FEATURES includes package-management.

[YOCTO #5666]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../run-postinsts/run-postinsts/run-postinsts  |   63 +---
 .../run-postinsts/run-postinsts_1.0.bb |2 +
 2 files changed, 42 insertions(+), 23 deletions(-)

diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts 
b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
index 11141ec..08cfa9e 100755
--- a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
+++ b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
@@ -8,13 +8,9 @@
 # The following script will run all the scriptlets found in 
#SYSCONFDIR#/deb-postinsts,
 # #SYSCONFDIR#/ipk-postinsts or #SYSCONFDIR#/rpm-posinsts.
 
-pi_dir=
-for pm in rpm deb ipk; do
-   if [ -d #SYSCONFDIR#/${pm}-postinsts ]; then
-   pi_dir=#SYSCONFDIR#/${pm}-postinsts
-   break
-   fi
-done
+pm=#IMAGE_PKGTYPE#
+pm_installed=#PM_INSTALLED#
+pi_dir=#SYSCONFDIR#/${pm}-postinsts
 
 remove_rcsd_link () {
if [ -n `which update-rc.d` ]; then
@@ -29,24 +25,45 @@ fi
 
 [ -e #SYSCONFDIR#/default/postinst ]  . #SYSCONFDIR#/default/postinst
 
-remove_pi_dir=1
-for i in `ls $pi_dir`; do
-   i=$pi_dir/$i
-   echo Running postinst $i...
-   [ $POSTINST_LOGGING = 1 ]  echo Running postinst $i...  
$LOGFILE
-   if [ -x $i ]; then
-   if [ $POSTINST_LOGGING = 1 ]; then
-   sh -c $i $LOGFILE 21
+if [ $POSTINST_LOGGING = 1 ]; then
+   rm -f $LOGFILE
+   append_log=$LOGFILE 21
+fi
+
+exec_postinst_scriptlets() {
+   for i in `ls $pi_dir`; do
+   i=$pi_dir/$i
+   echo Running postinst $i...
+   [ $POSTINST_LOGGING = 1 ]  eval echo Running postinst 
$i... $append_log
+   if [ -x $i ]; then
+   eval sh -c $i $append_log
+   rm $i
else
-   sh -c $i
+   echo ERROR: postinst $i failed.
+   [ $POSTINST_LOGGING = 1 ]  eval echo ERROR: 
postinst $i failed. $append_log
+   remove_pi_dir=0
fi
-   rm $i
-   else
-   echo ERROR: postinst $i failed.
-   [ $POSTINST_LOGGING = 1 ]  echo ERROR: postinst $i 
failed.  $LOGFILE
-   remove_pi_dir=0
-   fi
-done
+   done
+}
+
+remove_pi_dir=1
+if $pm_installed; then
+   case $pm in
+   ipk)
+   eval opkg-cl configure $append_log
+   ;;
+
+   deb)
+   eval dpkg --configure -a $append_log
+   ;;
+
+   rpm)
+   exec_postinst_scriptlets
+   ;;
+   esac
+else
+   exec_postinst_scriptlets
+fi
 
 # since all postinstalls executed successfully, remove the postinstalls 
directory
 # and the rcS.d link
diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb 
b/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb
index 64f85c2..e990c67 100644
--- a/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb
+++ b/meta/recipes-devtools/run-postinsts/run-postinsts_1.0.bb
@@ -37,6 +37,8 @@ do_install() {
sed -i -e 's:#SYSCONFDIR#:${sysconfdir}:g' \
-e 's:#SBINDIR#:${sbindir}:g' \
-e 's:#BASE_BINDIR#:${base_bindir}:g' \
+   -e 's:#IMAGE_PKGTYPE#:${IMAGE_PKGTYPE}:g' \
+   -e 's:#PM_INSTALLED#:${@base_contains(IMAGE_FEATURES, 
package-management, true, false, d)}:g' \
${D}${sbindir}/run-postinsts \
${D}${systemd_unitdir}/system/run-postinsts.service
 }
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/2] opkg/dpkg: remove the postinstalls

2014-03-06 Thread Laurentiu Palcu
Just use the run-postinsts recipe for running first boot postinstalls.

[YOCTO #5666]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/recipes-devtools/dpkg/dpkg.inc |   24 +---
 meta/recipes-devtools/opkg/opkg.inc |   28 +---
 2 files changed, 2 insertions(+), 50 deletions(-)

diff --git a/meta/recipes-devtools/dpkg/dpkg.inc 
b/meta/recipes-devtools/dpkg/dpkg.inc
index 471151e..eef7ce9 100644
--- a/meta/recipes-devtools/dpkg/dpkg.inc
+++ b/meta/recipes-devtools/dpkg/dpkg.inc
@@ -7,7 +7,7 @@ SRC_URI = ${DEBIAN_MIRROR}/main/d/dpkg/dpkg_${PV}.tar.xz \
 
 DEPENDS = zlib bzip2 perl ncurses
 DEPENDS_class-native = bzip2-replacement-native zlib-native 
virtual/update-alternatives-native gettext-native perl-native
-RDEPENDS_${PN} = ${VIRTUAL-RUNTIME_update-alternatives} xz
+RDEPENDS_${PN} = ${VIRTUAL-RUNTIME_update-alternatives} xz run-postinsts
 RDEPENDS_${PN}_class-native = xz-native
 
 S = ${WORKDIR}/${BPN}-${PV}
@@ -43,8 +43,6 @@ do_configure () {
 autotools_do_configure
 }
 
-DPKG_INIT_POSITION ?= 98
-
 do_install_append () {
if [ ${PN} = dpkg-native ]; then
# update-alternatives doesn't have an offline mode
@@ -66,26 +64,6 @@ do_install_append () {
fi
 }
 
-pkg_postinst_${PN} () {
-#!/bin/sh
-if ${@base_contains('DISTRO_FEATURES','sysvinit','true','false',d)}  \
-   [ x$D != x ]  [ -f $D/var/lib/dpkg/status ]; then
-   install -d $D${sysconfdir}/rcS.d
-
-   # this happens at S98 where our good 'ole packages script used to run
-   echo #!/bin/sh
-[ -e ${sysconfdir}/default/postinst ]  . ${sysconfdir}/default/postinst
-if [ \\$POSTINST_LOGGING\ = \1\ ]; then
-dpkg --configure -a \$LOGFILE 21
-else
-dpkg --configure -a
-fi
-rm -f ${sysconfdir}/rcS.d/S${DPKG_INIT_POSITION}run-postinsts
-  $D${sysconfdir}/rcS.d/S${DPKG_INIT_POSITION}run-postinsts
-   chmod 0755 $D${sysconfdir}/rcS.d/S${DPKG_INIT_POSITION}run-postinsts
-fi
-}
-
 PROV = virtual/update-alternatives
 PROV_class-native = 
 
diff --git a/meta/recipes-devtools/opkg/opkg.inc 
b/meta/recipes-devtools/opkg/opkg.inc
index 7bea26d..9f87df5 100644
--- a/meta/recipes-devtools/opkg/opkg.inc
+++ b/meta/recipes-devtools/opkg/opkg.inc
@@ -38,7 +38,7 @@ EXTRA_OECONF = \
   --with-opkglibdir=${OPKGLIBDIR} \
 
 
-RDEPENDS_${PN} = ${VIRTUAL-RUNTIME_update-alternatives} opkg-config-base
+RDEPENDS_${PN} = ${VIRTUAL-RUNTIME_update-alternatives} opkg-config-base 
run-postinsts
 RDEPENDS_${PN}_class-native = 
 RDEPENDS_${PN}_class-nativesdk = 
 RREPLACES_${PN} = opkg-nogpg
@@ -73,30 +73,4 @@ do_install_append() {
rm ${D}${bindir}/update-alternatives
 }
 
-pkg_postinst_${PN} () {
-#!/bin/sh
-if ${@base_contains('DISTRO_FEATURES','sysvinit','true','false',d)}  \
-   [ x$D != x ]  [ -f $D${OPKGLIBDIR}/opkg/status ]; then
-   install -d $D${sysconfdir}/rcS.d
-
-   # this happens at S98 where our good 'ole packages script used to run
-   echo #!/bin/sh
-[ -e ${sysconfdir}/default/postinst ]  . ${sysconfdir}/default/postinst
-if [ \\$POSTINST_LOGGING\ = \1\ ]; then
-opkg-cl configure \$LOGFILE 21
-else
-opkg-cl configure
-fi
-rm -f ${sysconfdir}/rcS.d/S${POSTINSTALL_INITPOSITION}run-postinsts
-  $D${sysconfdir}/rcS.d/S${POSTINSTALL_INITPOSITION}run-postinsts
-   chmod 0755 
$D${sysconfdir}/rcS.d/S${POSTINSTALL_INITPOSITION}run-postinsts
-fi
-}
-
 BBCLASSEXTEND = native nativesdk
-
-# Define a variable to allow distros to run configure earlier.
-# (for example, to enable loading of ethernet kernel modules before networking 
starts)
-# note: modifying name or default value for POSTINSTALL_INITPOSITION requires
-# changes in systemd-compat-units.bb
-POSTINSTALL_INITPOSITION ?= 98
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/2] Unique run-postinsts

2014-03-06 Thread Laurentiu Palcu
This patchset removes the opkg/dpkg postinstalls that were creating their own
run-postinsts script which, sometimes, confused people... Instead, use the
already available run-postinsts recipe in order to create the run-postinsts
init script. It will be used for both 'package-management'/no
'package-management' in DISTRO_FEATURES.

In my tests I tried to cover as much usecases as possible, in various
combinations:
 * rpm/ipk/deb;
 * package-management/no package-management;
 * delayed/no delayed postinstalls;
 * sysvinit/systemd;

I hope I covered most of them. If there are any other 'dirty' usecases I should
be aware of, don't be shy.

laurentiu

The following changes since commit a01af0202558e6ed9d16590b3a8d1dd1b95c0374:

  recipes: bump PRs (2014-03-05 17:36:37 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/b5666_unique_run_postinsts
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5666_unique_run_postinsts

Laurentiu Palcu (2):
  run-postinsts: use it for opkg/dpkg too
  opkg/dpkg: remove the postinstalls

 meta/recipes-devtools/dpkg/dpkg.inc|   24 +---
 meta/recipes-devtools/opkg/opkg.inc|   28 +
 .../run-postinsts/run-postinsts/run-postinsts  |   63 +---
 .../run-postinsts/run-postinsts_1.0.bb |2 +
 4 files changed, 44 insertions(+), 73 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation

2014-03-06 Thread Laurentiu Palcu
On Thu, Mar 06, 2014 at 09:55:38PM +, Richard Purdie wrote:
 On Wed, 2014-03-05 at 14:39 +0200, Laurentiu Palcu wrote:
  Call the new python routines.
  
  [YOCTO #5904]
  
  Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
  ---
   meta/classes/buildhistory.bbclass |9 -
   1 file changed, 8 insertions(+), 1 deletion(-)
  
  diff --git a/meta/classes/buildhistory.bbclass 
  b/meta/classes/buildhistory.bbclass
  index ef4135b..01b0082 100644
  --- a/meta/classes/buildhistory.bbclass
  +++ b/meta/classes/buildhistory.bbclass
  @@ -319,6 +319,12 @@ python buildhistory_list_installed() {
   
   with open(pkgs_list_file, 'w') as pkgs_list:
   pkgs_list.write(list_installed_packages(d, 'file'))
  +
  +pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
  +  bh_installed_pkgs_deps.txt)
  +
  +with open(pkgs_deps_file, 'w') as pkgs_deps:
  +pkgs_deps.write(list_installed_packages(d, 'deps'))
   }
   
  
  @@ -338,7 +344,8 @@ buildhistory_get_installed() {
   
  # Produce dependency graph
  # First, quote each name to handle characters that cause issues for dot
  -   rootfs_list_installed_depends | sed 's:\([^| ]*\):\1:g'  
  $1/depends.tmp
  +   cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):\1:g'  
  $1/depends.tmp  \
  +   rm ${WORKDIR}/bh_installed_pkgs_deps.txt
  # Change delimiter from pipe to - and set style for recommend lines
  sed -i -e 's:|: - :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' 
  $1/depends.tmp
  # Add header, sorted and de-duped contents and footer and then delete 
  the temp file
 
 With this patch, a bitbake core-image-minimal -c populate_sdk resulted
No, not quite. The bug appears to be in
RpmPM._pkg_translate_smart_to_oe(), which none of my patches touched.

CCing Hongxu.

laurentiu

 in:
 
 ERROR: Error executing a python function in 
 /media/build1/poky/meta/recipes-core/images/core-image-minimal.bb:
 
 The stack trace of python calls that resulted in this exception/failure was:
 File: 'buildhistory_list_installed', lineno: 18, function: module
  0014:with open(pkgs_deps_file, 'w') as pkgs_deps:
  0015:pkgs_deps.write(list_installed_packages(d, 'deps'))
  0016:
  0017:
  *** 0018:buildhistory_list_installed(d)
  0019:
 File: 'buildhistory_list_installed', lineno: 9, function: 
 buildhistory_list_installed
  0005:pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
  0006:  bh_installed_pkgs.txt)
  0007:
  0008:with open(pkgs_list_file, 'w') as pkgs_list:
  *** 0009:pkgs_list.write(list_installed_packages(d, 'file'))
  0010:
  0011:pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
  0012:  bh_installed_pkgs_deps.txt)
  0013:
 File: '/media/build1/poky/meta/lib/oe/rootfs.py', lineno: 721, function: 
 list_installed_packages
  0717:if img_type == rpm:
  0718:return RpmPM(d,
  0719: rootfs_dir,
  0720: d.getVar('TARGET_VENDOR', True)
  *** 0721: ).list_installed(format)
  0722:elif img_type == ipk:
  0723:return OpkgPM(d,
  0724:  rootfs_dir,
  0725:  d.getVar(IPKGCONF_TARGET, True),
 File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 854, 
 function: list_installed
  0850:pkg = line.split()[0]
  0851:arch = line.split()[1]
  0852:ver = line.split()[2]
  0853:pkgorigin = line.split()[3]
  *** 0854:new_pkg, new_arch = 
 self._pkg_translate_smart_to_oe(pkg, arch)
  0855:
  0856:if format == arch:
  0857:output.append('%s %s' % (new_pkg, new_arch))
  0858:elif format == file:
 File: '/media/build1/poky/meta/lib/oe/package_manager.py', lineno: 476, 
 function: _pkg_translate_smart_to_oe
  0472:
  0473:if found == 1 and fixed_arch == fixed_cmp_arch:
  0474:break
  0475:#bb.note('%s, %s - %s, %s' % (pkg, arch, new_pkg, 
 new_arch))
  *** 0476:return new_pkg, new_arch
  0477:
  0478:def _search_pkg_name_in_feeds(self, pkg, feed_archs):
  0479:for arch in feed_archs:
  0480:arch = arch.replace('-', '_')
 Exception: UnboundLocalError: local variable 'new_arch' referenced before 
 assignment
 
 ERROR: Function failed: buildhistory_list_installed
 ERROR: Logfile of failure stored in: 
 /media/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.d
 
 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/3] Fix issue with unpopulated buildhistory depends files

2014-03-05 Thread Laurentiu Palcu
The following changes since commit f03955041d0e44d377ca1c4def630982f24f1e8b:

  Revert ncurses: use ln -r to generate relative symlink (2014-03-03 15:55:27 
+)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib 
lpalcu/b5904_buildhistory_depends_empty
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5904_buildhistory_depends_empty

Laurentiu Palcu (3):
  package_manager.py: make list_installed() list pkg dependencies too
  buildhistory.bbclass: Fix dependency files creation
  populate_sdk_*.bbclass: remove old rootfs_list_installed_depends()

 meta/classes/buildhistory.bbclass |9 -
 meta/classes/populate_sdk_deb.bbclass |5 -
 meta/classes/populate_sdk_ipk.bbclass |4 
 meta/classes/populate_sdk_rpm.bbclass |4 
 meta/lib/oe/package_manager.py|   31 +++
 5 files changed, 39 insertions(+), 14 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/3] package_manager.py: make list_installed() list pkg dependencies too

2014-03-05 Thread Laurentiu Palcu
list_installed(deps) will now return the package dependencies.

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/package_manager.py |   31 +++
 1 file changed, 31 insertions(+)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index ff4f1de..f2c88bb 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -765,7 +765,22 @@ class RpmPM(PackageManager):
 self.image_rpmlib,
 symlinks=True)
 
+def _list_pkg_deps(self):
+cmd = [bb.utils.which(os.getenv('PATH'), rpmresolve),
+   -t, self.image_rpmlib]
+
+try:
+output = subprocess.check_output(cmd, 
stderr=subprocess.STDOUT).strip()
+except subprocess.CalledProcessError as e:
+bb.fatal(Cannot get the package dependencies. Command '%s' 
+ returned %d:\n%s % (' '.join(cmd), e.returncode, 
e.output))
+
+return output
+
 def list_installed(self, format=None):
+if format == deps:
+return self._list_pkg_deps()
+
 cmd = self.rpm_cmd + ' --root ' + self.target_rootfs
 cmd += ' -D _dbpath /var/lib/rpm -qa'
 cmd +=  --qf '[%{NAME} %{ARCH} %{VERSION} %{PACKAGEORIGIN}\n]'
@@ -1131,6 +1146,9 @@ class OpkgPM(PackageManager):
 elif format == ver:
 cmd = %s %s status | %s -v % \
 (self.opkg_cmd, self.opkg_args, opkg_query_cmd)
+elif format == deps:
+cmd = %s %s status | %s % \
+(self.opkg_cmd, self.opkg_args, opkg_query_cmd)
 else:
 cmd = %s %s list_installed | cut -d' ' -f1 % \
 (self.opkg_cmd, self.opkg_args)
@@ -1494,6 +1512,8 @@ class DpkgPM(PackageManager):
 cmd.append(-f=${Package} 
${Package}_${Version}_${Architecture}.deb ${PackageArch}\n)
 elif format == ver:
 cmd.append(-f=${Package} ${PackageArch} ${Version}\n)
+elif format == deps:
+cmd.append(-f=Package: ${Package}\nDepends: 
${Depends}\nRecommends: ${Recommends}\n\n)
 else:
 cmd.append(-f=${Package}\n)
 
@@ -1514,6 +1534,17 @@ class DpkgPM(PackageManager):
 tmp_output += %s %s %s\n % (pkg, pkg_file, pkg_arch)
 
 output = tmp_output
+elif format == deps:
+opkg_query_cmd = bb.utils.which(os.getenv('PATH'), 
opkg-query-helper.py)
+
+try:
+output = subprocess.check_output(echo -e '%s' | %s %
+ (output, opkg_query_cmd),
+ stderr=subprocess.STDOUT,
+ shell=True)
+except subprocess.CalledProcessError as e:
+bb.fatal(Cannot compute packages dependencies. Command '%s' 
+ returned %d:\n%s % (e.cmd, e.returncode, e.output))
 
 return output
 
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/3] buildhistory.bbclass: Fix dependency files creation

2014-03-05 Thread Laurentiu Palcu
Call the new python routines.

[YOCTO #5904]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/classes/buildhistory.bbclass |9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/meta/classes/buildhistory.bbclass 
b/meta/classes/buildhistory.bbclass
index ef4135b..01b0082 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -319,6 +319,12 @@ python buildhistory_list_installed() {
 
 with open(pkgs_list_file, 'w') as pkgs_list:
 pkgs_list.write(list_installed_packages(d, 'file'))
+
+pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
+  bh_installed_pkgs_deps.txt)
+
+with open(pkgs_deps_file, 'w') as pkgs_deps:
+pkgs_deps.write(list_installed_packages(d, 'deps'))
 }
 
 
@@ -338,7 +344,8 @@ buildhistory_get_installed() {
 
# Produce dependency graph
# First, quote each name to handle characters that cause issues for dot
-   rootfs_list_installed_depends | sed 's:\([^| ]*\):\1:g'  
$1/depends.tmp
+   cat ${WORKDIR}/bh_installed_pkgs_deps.txt | sed 's:\([^| ]*\):\1:g'  
$1/depends.tmp  \
+   rm ${WORKDIR}/bh_installed_pkgs_deps.txt
# Change delimiter from pipe to - and set style for recommend lines
sed -i -e 's:|: - :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' 
$1/depends.tmp
# Add header, sorted and de-duped contents and footer and then delete 
the temp file
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 3/3] populate_sdk_*.bbclass: remove old rootfs_list_installed_depends()

2014-03-05 Thread Laurentiu Palcu
We're using the python routines now.

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/classes/populate_sdk_deb.bbclass |5 -
 meta/classes/populate_sdk_ipk.bbclass |4 
 meta/classes/populate_sdk_rpm.bbclass |4 
 3 files changed, 13 deletions(-)

diff --git a/meta/classes/populate_sdk_deb.bbclass 
b/meta/classes/populate_sdk_deb.bbclass
index 462525f..833feca 100644
--- a/meta/classes/populate_sdk_deb.bbclass
+++ b/meta/classes/populate_sdk_deb.bbclass
@@ -10,8 +10,3 @@ do_populate_sdk[lockfiles] += ${DEPLOY_DIR_DEB}/deb.lock
 
 # This will of course only work after rootfs_deb_do_rootfs or populate_sdk_deb 
has been called
 DPKG_QUERY_COMMAND = ${STAGING_BINDIR_NATIVE}/dpkg-query 
--admindir=$INSTALL_ROOTFS_DEB/var/lib/dpkg
-
-rootfs_list_installed_depends() {
-   # Cheat here a little bit by using the opkg query helper util
-   ${DPKG_QUERY_COMMAND} -W -f='Package: ${Package}\nDepends: 
${Depends}\nRecommends: ${Recommends}\n\n' | opkg-query-helper.py
-}
diff --git a/meta/classes/populate_sdk_ipk.bbclass 
b/meta/classes/populate_sdk_ipk.bbclass
index 4c23f05..e2e3523 100644
--- a/meta/classes/populate_sdk_ipk.bbclass
+++ b/meta/classes/populate_sdk_ipk.bbclass
@@ -2,7 +2,3 @@ do_populate_sdk[depends] += opkg-native:do_populate_sysroot 
opkg-utils-native:d
 do_populate_sdk[recrdeptask] += do_package_write_ipk
 
 do_populate_sdk[lockfiles] += ${WORKDIR}/ipk.lock
-
-rootfs_list_installed_depends() {
-   opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py
-}
diff --git a/meta/classes/populate_sdk_rpm.bbclass 
b/meta/classes/populate_sdk_rpm.bbclass
index 06605d3..0fdef3b 100644
--- a/meta/classes/populate_sdk_rpm.bbclass
+++ b/meta/classes/populate_sdk_rpm.bbclass
@@ -16,7 +16,3 @@ do_populate_sdk[recrdeptask] += do_package_write_rpm
 rpmlibdir = /var/lib/rpm
 
 do_populate_sdk[lockfiles] += ${DEPLOY_DIR_RPM}/rpm.lock
-
-rootfs_list_installed_depends() {
-   rpmresolve -t $INSTALL_ROOTFS_RPM/${rpmlibdir}
-}
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] relocate_sdk.py: Possible bug, /lib64/ld-linux-x86-64.so.2 not relocated

2014-03-03 Thread Laurentiu Palcu
Hi Stefan,

On Sat, Mar 01, 2014 at 03:28:09PM +0100, Stefan Agner wrote:
 Hi,
 
 Using top of dylan branch, I generated a SDK using bitbake
 meta-toolchain. I'm running Arch Linux, but I also see similar issues on
 Ubuntu 12.04.4 LTS:
 
 Some binaries segfault when running them. I discovered, that the dynamic
 linker of the host is used:
 $ ldd
 /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc
 linux-vdso.so.1 (0x7fffc8f11000)
 libc.so.6 =
 /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/../../../lib/libc.so.6
 (0x7f4d85eb1000)
 /lib64/ld-linux-x86-64.so.2 (0x7f4d8625f000)
 
 When forcing the dynamic linker of the SDK, the binary works:
 $
 /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/lib/ld-linux-x86-64.so.2
 /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc
 arm-angstrom-linux-gnueabi-gcc: fatal error: no input files
 compilation terminated.
 
 Digging around I found this commit by Jason:
 3752a9c6d772b39bbe04d62ef4d3527b4c7198c1
 relocate_sdk.py: Fix corruption of sdk binaries
 
 It changes the behavior and don't relocates stuff which start with
 /lib64.
This is the correct behavior. We shouldn't relocate binaries that use
host's dynamic loader.

 When I install the SDK with -S (copy the relocate scripts), and
 remove the condition around line 95, the binaries work as expected. 
Can you please run the installer script with -R so it doesn't perform
any relocation on binaries, and then:

readelf -p .interp 
/usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/armv7ahf-vfp-neon-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc

It looks like the default dynamic loader path of the toolchain binaries
start with /lib* or /usr/lib* which is not quite right... It should be:
${SDKPATH}/sysroots/${SDK_SYS}.

laurentiu
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/2] fix package archs ordering for RPM

2014-03-03 Thread Laurentiu Palcu
This got messed up by me trying to fix a multilib problem using python sets.
Unfortunately sets do not keep the initial ordering and an image might end
up containing packages meant for other architectures... So, revert the offending
commit and fix the original issue by just checking if a channel was already 
added
or not.

laurentiu

The following changes since commit 8f36e1ef1afd10594391596391564e8904a840f2:

  poky.conf: add CentOS 6.5 to SANITY_TESTED_DISTROS (2014-02-28 18:05:47 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib 
lpalcu/b5890_rpm_pkg_arch_selection_logic_fix
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5890_rpm_pkg_arch_selection_logic_fix

Laurentiu Palcu (2):
  Revert lib/oe/package_manager.py: RpmPM: fix issue with multilib
builds
  package_manager.py: RpmPM: don't add smart channel if already added

 meta/lib/oe/package_manager.py |   19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/2] Revert lib/oe/package_manager.py: RpmPM: fix issue with multilib builds

2014-03-03 Thread Laurentiu Palcu
Converting a list to a set does not keep the order of the items, as they
were added, which might result in the wrong packages being installed in
the final image...

This reverts commit 12f47c23df8c109676f66d580d666a3147f3b046.

[YOCTO #5890]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/package_manager.py |   13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index ff4f1de..6c133c3 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -368,18 +368,19 @@ class RpmPM(PackageManager):
 self.ml_os_list['default'])
 
 # List must be prefered to least preferred order
-default_platform_extra = set()
-platform_extra = set()
+default_platform_extra = list()
+platform_extra = list()
 bbextendvariant = self.d.getVar('BBEXTENDVARIANT', True) or 
 for mlib in self.ml_os_list:
 for arch in self.ml_prefix_list[mlib]:
 plt = arch.replace('-', '_') + '-.*-' + self.ml_os_list[mlib]
 if mlib == bbextendvariant:
-default_platform_extra.add(plt)
+if plt not in default_platform_extra:
+default_platform_extra.append(plt)
 else:
-platform_extra.add(plt)
-
-platform_extra = platform_extra.union(default_platform_extra)
+if plt not in platform_extra:
+platform_extra.append(plt)
+platform_extra = default_platform_extra + platform_extra
 
 self._create_configs(platform, platform_extra)
 
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/2] package_manager.py: RpmPM: don't add smart channel if already added

2014-03-03 Thread Laurentiu Palcu
Make sure the channel was not already added, before attempting to add.

[YOCTO #5890]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/package_manager.py |6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 6c133c3..ee42952 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -604,11 +604,11 @@ class RpmPM(PackageManager):
 # self._invoke_smart('config --set rpm-log-level=debug')
 # cmd = 'config --set rpm-log-file=/tmp/smart-debug-logfile'
 # self._invoke_smart(cmd)
-
+ch_already_added = []
 for canonical_arch in platform_extra:
 arch = canonical_arch.split('-')[0]
 arch_channel = os.path.join(self.deploy_dir, arch)
-if os.path.exists(arch_channel):
+if os.path.exists(arch_channel) and not arch in ch_already_added:
 bb.note('Note: adding Smart channel %s (%s)' %
 (arch, channel_priority))
 self._invoke_smart('channel --add %s type=rpm-md baseurl=%s -y'
@@ -617,6 +617,8 @@ class RpmPM(PackageManager):
(arch, channel_priority))
 channel_priority -= 5
 
+ch_already_added.append(arch)
+
 bb.note('adding Smart RPM DB channel')
 self._invoke_smart('channel --add rpmsys type=rpm-sys -y')
 
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 2/2] package_manager.py: RpmPM: don't add smart channel if already added

2014-03-03 Thread Laurentiu Palcu
Hi Mark,

On Mon, Mar 03, 2014 at 09:02:35AM -0600, Mark Hatle wrote:
 On 3/3/14, 3:43 AM, Laurentiu Palcu wrote:
 Make sure the channel was not already added, before attempting to add.
 
 [YOCTO #5890]
 
 When generating the rootfs/etc/rpm/platform file, the order should
 be in the order of highest to lowest priority as well.
 
 Smart uses this order (via an RPM API) to help determine scores when
 two packages from repositories at the same priority (with different
 architectures) are selected.
Thanks for the info. This patchset should restore the behavior to the
one you describe. Python's sets messed up the order, so I reverted that
patch.

laurentiu
 
 --Mark
 
 Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
 ---
   meta/lib/oe/package_manager.py |6 --
   1 file changed, 4 insertions(+), 2 deletions(-)
 
 diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
 index 6c133c3..ee42952 100644
 --- a/meta/lib/oe/package_manager.py
 +++ b/meta/lib/oe/package_manager.py
 @@ -604,11 +604,11 @@ class RpmPM(PackageManager):
   # self._invoke_smart('config --set rpm-log-level=debug')
   # cmd = 'config --set rpm-log-file=/tmp/smart-debug-logfile'
   # self._invoke_smart(cmd)
 -
 +ch_already_added = []
   for canonical_arch in platform_extra:
   arch = canonical_arch.split('-')[0]
   arch_channel = os.path.join(self.deploy_dir, arch)
 -if os.path.exists(arch_channel):
 +if os.path.exists(arch_channel) and not arch in 
 ch_already_added:
   bb.note('Note: adding Smart channel %s (%s)' %
   (arch, channel_priority))
   self._invoke_smart('channel --add %s type=rpm-md 
  baseurl=%s -y'
 @@ -617,6 +617,8 @@ class RpmPM(PackageManager):
  (arch, channel_priority))
   channel_priority -= 5
 
 +ch_already_added.append(arch)
 +
   bb.note('adding Smart RPM DB channel')
   self._invoke_smart('channel --add rpmsys type=rpm-sys -y')
 
 
 
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/1] image.py, rootfs.py, package_manager.py: redirect stderr to stdout when calling check_output()

2014-03-03 Thread Laurentiu Palcu
The following changes since commit 54c2e993ec129563c7ae9f3fdee74ab60da28698:

  bsps: update H/W reference boards to v3.10.32 (2014-03-02 17:25:30 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/b5902_errors_not_displayed
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/b5902_errors_not_displayed

Laurentiu Palcu (1):
  image.py, rootfs.py, package_manager.py: redirect stderr to stdout
when calling check_output()

 meta/lib/oe/image.py   |2 +-
 meta/lib/oe/package_manager.py |   48 +---
 meta/lib/oe/rootfs.py  |2 +-
 3 files changed, 27 insertions(+), 25 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/1] image.py, rootfs.py, package_manager.py: redirect stderr to stdout when calling check_output()

2014-03-03 Thread Laurentiu Palcu
If a command executed with subprocess.check_output() fails, the
subprocess.CalledProcessError.output contains only STDOUT and the user
needs to check the log.do_rootfs to see any other details.

This commit forwards stderr to stdout so that, in case of failure, the
entire error output will be displayed in terminal.

[YOCTO #5902]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/image.py   |2 +-
 meta/lib/oe/package_manager.py |   48 +---
 meta/lib/oe/rootfs.py  |2 +-
 3 files changed, 27 insertions(+), 25 deletions(-)

diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
index 488683e..a03b73e 100644
--- a/meta/lib/oe/image.py
+++ b/meta/lib/oe/image.py
@@ -11,7 +11,7 @@ def generate_image(arg):
 (type, create_img_cmd))
 
 try:
-subprocess.check_output(create_img_cmd)
+subprocess.check_output(create_img_cmd, stderr=subprocess.STDOUT)
 except subprocess.CalledProcessError as e:
 return(Error: The image creation script '%s' returned %d:\n%s %
(e.cmd, e.returncode, e.output))
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index ff4f1de..90884cd 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -14,7 +14,7 @@ def create_index(arg):
 
 try:
 bb.note(Executing '%s' ... % index_cmd)
-subprocess.check_output(index_cmd, shell=True)
+subprocess.check_output(index_cmd, stderr=subprocess.STDOUT, 
shell=True)
 except subprocess.CalledProcessError as e:
 return(Index creation command '%s' failed with return code %d:\n%s %
(e.cmd, e.returncode, e.output))
@@ -298,7 +298,7 @@ class PackageManager(object):
globs]
 try:
 bb.note(Installing complementary packages ...)
-complementary_pkgs = subprocess.check_output(cmd)
+complementary_pkgs = subprocess.check_output(cmd, 
stderr=subprocess.STDOUT)
 except subprocess.CalledProcessError as e:
 bb.fatal(Could not compute complementary packages list. Command 
  '%s' returned %d:\n%s %
@@ -387,7 +387,9 @@ class RpmPM(PackageManager):
 cmd = %s %s %s % (self.smart_cmd, self.smart_opt, args)
 # bb.note(cmd)
 try:
-complementary_pkgs = subprocess.check_output(cmd, shell=True)
+complementary_pkgs = subprocess.check_output(cmd,
+ 
stderr=subprocess.STDOUT,
+ shell=True)
 # bb.note(complementary_pkgs)
 return complementary_pkgs
 except subprocess.CalledProcessError as e:
@@ -569,7 +571,7 @@ class RpmPM(PackageManager):
   self.rpm_cmd,
   self.target_rootfs)
 try:
-subprocess.check_output(cmd, shell=True)
+subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True)
 except subprocess.CalledProcessError as e:
 bb.fatal(Create rpm database failed. Command '%s' 
  returned %d:\n%s % (cmd, e.returncode, e.output))
@@ -691,7 +693,7 @@ class RpmPM(PackageManager):
 cmd = %s %s install --attempt -y %s % \
   (self.smart_cmd, self.smart_opt, ' '.join(pkgs))
 try:
-output = subprocess.check_output(cmd.split())
+output = subprocess.check_output(cmd.split(), 
stderr=subprocess.STDOUT)
 bb.note(output)
 except subprocess.CalledProcessError as e:
 bb.fatal(Unable to install packages. Command '%s' 
@@ -721,7 +723,7 @@ class RpmPM(PackageManager):
 
 try:
 bb.note(cmd)
-output = subprocess.check_output(cmd, shell=True)
+output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, 
shell=True)
 bb.note(output)
 except subprocess.CalledProcessError as e:
 bb.note(Unable to remove packages. Command '%s' 
@@ -772,7 +774,7 @@ class RpmPM(PackageManager):
 
 try:
 # bb.note(cmd)
-tmp_output = subprocess.check_output(cmd, shell=True).strip()
+tmp_output = subprocess.check_output(cmd, 
stderr=subprocess.STDOUT, shell=True).strip()
 self._unlock_rpm_db()
 except subprocess.CalledProcessError as e:
 bb.fatal(Cannot get the installed packages list. Command '%s' 
@@ -824,7 +826,7 @@ class RpmPM(PackageManager):
 # Disable rpmsys channel for the fake install
 self._invoke_smart('channel --disable rpmsys')
 
-subprocess.check_output(cmd, shell=True)
+subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True)
 with open(self.solution_manifest, 'r') as manifest:
 for pkg in manifest.read().split('\n'):
 if '@' in pkg

Re: [OE-core] [RFC][PATCH] Attempt to unify predefined package feed handling

2014-02-27 Thread Laurentiu Palcu
Hi David,

My comments/questions below.

laurentiu

On Thu, Feb 27, 2014 at 02:28:21PM +0100, David Nyström wrote:
 local.conf defines 
 PACKAGE_FEED_URIS = http://www.feed_repo.org/repo/;
 
 should we do it this way or some other way ?
 
 [Bug 5407]
 
 
 
 Signed-off-by: David Nyström david.nyst...@enea.com
 ---
  meta/lib/oe/package_manager.py | 76 
 +-
  meta/lib/oe/rootfs.py  | 15 +++--
  2 files changed, 87 insertions(+), 4 deletions(-)
 
 diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
 index d29adac..d4050c6 100644
 --- a/meta/lib/oe/package_manager.py
 +++ b/meta/lib/oe/package_manager.py
 @@ -223,6 +223,7 @@ class PackageManager(object):
  self.d = d
  self.deploy_dir = None
  self.deploy_lock = None
 +self.feed_uris = None
I guess you can do:
   self.feed_uris = self.d.getVar('PACKAGE_FEED_URIS', True) or 
since we're using the same variable for all backends.

  
  
  Update the package manager package database.
 @@ -262,6 +263,10 @@ class PackageManager(object):
  def list_installed(self, format=None):
  pass
  
 +@abstractmethod
 +def insert_feeds_uris(self):
 +pass
 +
  
  Install complementary packages based upon the list of currently installed
  packages e.g. locales, *-dev, *-dbg, etc. This will only attempt to 
 install
 @@ -339,6 +344,7 @@ class RpmPM(PackageManager):
  self.providename = providename
  self.fullpkglist = list()
  self.deploy_dir = self.d.getVar('DEPLOY_DIR_RPM', True)
 +self.feed_uris = self.d.getVar('PACKAGE_FEED_URIS', True)
can be removed, see above.

  self.etcrpm_dir = os.path.join(self.target_rootfs, etc/rpm)
  self.install_dir = os.path.join(self.target_rootfs, install)
  self.rpm_cmd = bb.utils.which(os.getenv('PATH'), rpm)
 @@ -358,6 +364,41 @@ class RpmPM(PackageManager):
  
  self.ml_prefix_list, self.ml_os_list = 
 self.indexer.get_ml_prefix_and_os_list(arch_var, os_var)
  
 +
 +def insert_feeds_uris(self):
shouldn't we check that self.feed_uris !=  before continuing?

 +# List must be prefered to least preferred order
 +channel_priority = 5
 +default_platform_extra = set()
 +platform_extra = set()
 +bbextendvariant = self.d.getVar('BBEXTENDVARIANT', True) or 
 +for mlib in self.ml_os_list:
 +for arch in self.ml_prefix_list[mlib]:
 +plt = arch.replace('-', '_') + '-.*-' + self.ml_os_list[mlib]
 +if mlib == bbextendvariant:
 +default_platform_extra.add(plt)
 +else:
 +platform_extra.add(plt)
 +channel_priority += 5
 +
 +platform_extra = platform_extra.union(default_platform_extra)
 +
 +for uri in self.feed_uris.split():
 +channel_priority *= 2
 +uri_iterator = 0
 +for canonical_arch in platform_extra:
 +arch = canonical_arch.split('-')[0]
 +if arch == noarch or arch == any:
 +continue
 +
 +bb.note('Note: adding Smart channel url%d%s (%s)' %
 +(uri_iterator, arch, channel_priority))
 +self._invoke_smart('channel --add url%d-%s type=rpm-md 
 baseurl=%s/rpm/%s -y'
 +   % (uri_iterator, arch, uri, arch))
 +self._invoke_smart('channel --set url%d-%s priority=%d' %
 +   (uri_iterator, arch, channel_priority))
 +channel_priority -= 5
 +uri_iterator += 1
 +
  '''
  Create configs for rpm and smart, and multilib is supported
  '''
 @@ -944,7 +985,7 @@ class OpkgPM(PackageManager):
  
  self.deploy_dir = self.d.getVar(DEPLOY_DIR_IPK, True)
  self.deploy_lock_file = os.path.join(self.deploy_dir, deploy.lock)
 -
 +self.feed_uris = self.d.getVar('PACKAGE_FEED_URIS', True)
can be removed, see above.

  self.opkg_cmd = bb.utils.which(os.getenv('PATH'), opkg-cl)
  self.opkg_args = -f %s -o %s  % (self.config_file, target_rootfs)
  self.opkg_args += self.d.getVar(OPKG_ARGS, True)
 @@ -1050,6 +1091,24 @@ class OpkgPM(PackageManager):
  config_file.write(src oe-%s file:%s\n %
(arch, pkgs_dir))
  
 +def insert_feeds_uris(self):
check self.feed_uris !=  here?

 +rootfs_config = os.path.join('%s/etc/opkg/base-feeds.conf'
 +  % self.target_rootfs)
 +
 +with open(rootfs_config, w+) as config_file:
 +priority = 5
apparently this is not used at all in the code below, can be removed.

 +uri_iterator = 0
 +for uri in self.feed_uris.split():
 +config_file.write(src/gz url-%d %s/ipk\n %
 +   

Re: [OE-core] [RFC][PATCH] Attempt to unify predefined package feed handling

2014-02-27 Thread Laurentiu Palcu
On Thu, Feb 27, 2014 at 03:35:50PM +0100, David Nyström wrote:
 On 2014-02-27 15:24, Laurentiu Palcu wrote:
 Hi David,
 
 My comments/questions below.
 
 Thank you for your detailed comments on the RFC.
 I will return with V2, where your comments are addressed when the
 general approach is Acked.
 
 What do you think about the general approach ?
I think the approach is ok. I don't know a simpler way than this to add
package feeds into the image.

 I guess it was you who added the insert_feeds_uris stubs in rootfs.py ?
Yes, I added the stubs in preparation for this. But I didn't know
when/if this will be done. Thanks for taking care of this.

laurentiu

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/2] do_rootfs: Added PACKAGE_FEED_URIS functionality

2014-02-27 Thread Laurentiu Palcu
Reviewed-by: Laurentiu Palcu laurentiu.pa...@intel.com

On Thu, Feb 27, 2014 at 09:20:37PM +0100, David Nyström wrote:
 Adding a common interface to add predefined package manager
 channels to prebuilt rootfs:es.
 
 Adding PACKAGE_FEED_URIS = http://myre.po/repo/, will
 assume repo directories named (rpm,ipk,deb) as subdirectories
 and statically add them to the rootfs, using the same PKG_ARCHs
 as the build which produced the images.
 
 Tested with RPM, IPK and DEB.
 
 deb feed functionality seem broken, is anyone using this ?
 
 Signed-off-by: David Nyström david.c.nyst...@gmail.com
 Signed-off-by: David Nyström david.nyst...@enea.com
 ---
  meta/lib/oe/package_manager.py | 89 
 +-
  meta/lib/oe/rootfs.py  | 16 ++--
  2 files changed, 92 insertions(+), 13 deletions(-)
 
 diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
 index ff4f1de..c930572 100644
 --- a/meta/lib/oe/package_manager.py
 +++ b/meta/lib/oe/package_manager.py
 @@ -223,6 +223,7 @@ class PackageManager(object):
  self.d = d
  self.deploy_dir = None
  self.deploy_lock = None
 +self.feed_uris = self.d.getVar('PACKAGE_FEED_URIS', True) or 
  
  
  Update the package manager package database.
 @@ -262,6 +263,10 @@ class PackageManager(object):
  def list_installed(self, format=None):
  pass
  
 +@abstractmethod
 +def insert_feeds_uris(self):
 +pass
 +
  
  Install complementary packages based upon the list of currently installed
  packages e.g. locales, *-dev, *-dbg, etc. This will only attempt to 
 install
 @@ -358,6 +363,46 @@ class RpmPM(PackageManager):
  
  self.ml_prefix_list, self.ml_os_list = 
 self.indexer.get_ml_prefix_and_os_list(arch_var, os_var)
  
 +
 +def insert_feeds_uris(self):
 +if self.feed_uris == :
 +return
 +
 +# List must be prefered to least preferred order
 +default_platform_extra = set()
 +platform_extra = set()
 +bbextendvariant = self.d.getVar('BBEXTENDVARIANT', True) or 
 +for mlib in self.ml_os_list:
 +for arch in self.ml_prefix_list[mlib]:
 +plt = arch.replace('-', '_') + '-.*-' + self.ml_os_list[mlib]
 +if mlib == bbextendvariant:
 +default_platform_extra.add(plt)
 +else:
 +platform_extra.add(plt)
 +
 +platform_extra = platform_extra.union(default_platform_extra)
 +
 +arch_list = []
 +for canonical_arch in platform_extra:
 +arch = canonical_arch.split('-')[0]
 +if not os.path.exists(os.path.join(self.deploy_dir, arch)):
 +continue
 +arch_list.append(arch)
 +
 +uri_iterator = 0
 +channel_priority = 10 + 5 * len(self.feed_uris.split()) * 
 len(arch_list)
 +
 +for uri in self.feed_uris.split():
 +for arch in arch_list:
 +bb.note('Note: adding Smart channel url%d%s (%s)' %
 +(uri_iterator, arch, channel_priority))
 +self._invoke_smart('channel --add url%d-%s type=rpm-md 
 baseurl=%s/rpm/%s -y'
 +   % (uri_iterator, arch, uri, arch))
 +self._invoke_smart('channel --set url%d-%s priority=%d' %
 +   (uri_iterator, arch, channel_priority))
 +channel_priority -= 5
 +uri_iterator += 1
 +
  '''
  Create configs for rpm and smart, and multilib is supported
  '''
 @@ -944,7 +989,6 @@ class OpkgPM(PackageManager):
  
  self.deploy_dir = self.d.getVar(DEPLOY_DIR_IPK, True)
  self.deploy_lock_file = os.path.join(self.deploy_dir, deploy.lock)
 -
  self.opkg_cmd = bb.utils.which(os.getenv('PATH'), opkg-cl)
  self.opkg_args = -f %s -o %s  % (self.config_file, target_rootfs)
  self.opkg_args += self.d.getVar(OPKG_ARGS, True)
 @@ -1050,6 +1094,29 @@ class OpkgPM(PackageManager):
  config_file.write(src oe-%s file:%s\n %
(arch, pkgs_dir))
  
 +def insert_feeds_uris(self):
 +if self.feed_uris == :
 +return
 +
 +rootfs_config = os.path.join('%s/etc/opkg/base-feeds.conf'
 +  % self.target_rootfs)
 +
 +with open(rootfs_config, w+) as config_file:
 +uri_iterator = 0
 +for uri in self.feed_uris.split():
 +config_file.write(src/gz url-%d %s/ipk\n %
 +  (uri_iterator, uri))
 +
 +for arch in self.pkg_archs.split():
 +if not os.path.exists(os.path.join(self.deploy_dir, 
 arch)):
 +continue
 +bb.note('Note: adding opkg channel url-%s-%d (%s)' %
 +(arch, uri_iterator, uri

[OE-core] [PATCH 1/1] adt_installer: fix issue with x86_64

2014-02-26 Thread Laurentiu Palcu
When the default tune for x86_64 was changed to core2-64, the
environment setup script name did not contain x86 anymore. Hence, the
adt_installer failed for x86_64.

This commit contains a generic fix and is supposed to work with any kind
of machine/tune setting. It's actually extracting the environment script
name using 'opkg files meta-environment-MACHINE'. So, no need to do any
other sort of searches. We know exactly which is the environment setup
script for the specified machine.

[YOCTO #5806]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 .../adt-installer/scripts/adt_installer_internal   |   61 +++-
 1 file changed, 35 insertions(+), 26 deletions(-)

diff --git 
a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal 
b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
index e3422f5..432ba41 100755
--- 
a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
+++ 
b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
@@ -64,6 +64,23 @@ get_sudo_app()
   fi
 }
 
+# this function accepts arch_type (x86, x86_64, arm, ppc, mips) as the first
+# argument, returning the location of the target rootfs
+get_target_rootfs_location() {
+   [ -z $1 ]  return
+
+   arch_type=$1
+   # rootfs extraction directory
+   target_sysroot_var=\$YOCTOADT_TARGET_SYSROOT_LOC_$arch_type
+   target_sysroot=`eval echo $target_sysroot_var`
+
+   if [ $target_sysroot ==  ]; then
+   return
+   else
+   echo `readlink -m $target_sysroot`
+   fi
+}
+
 
 #let us install a qemu-native firstly
 #installation step 2
@@ -134,6 +151,22 @@ for target_type in $YOCTOADT_TARGETS; do
   echo_info Installing cross canadian packages for $machine ...
   $OPKG_INSTALL_NATIVE_CMD packagegroup-cross-canadian-$machine  
$YOCTOADT_INSTALL_LOG_FILE
   check_result
+
+  target_sysroot=`get_target_rootfs_location $target_type`
+  [ -z $target_sysroot ]  continue
+
+  # get the environment setup script paths: original (the one before 
relocation)
+  # and relocated
+  env_script_original=`$OPKG_CMD -f $OPKG_CONFIG_FILE -o $NATIVE_INSTALL_DIR 
files meta-environment-$machine|\
+   grep environment-setup`
+  env_script_relocated=$INSTALL_FOLDER/${env_script_original##*/}
+
+  # opkg will not install packagegroup-cross-canadian package if it was already
+  # installed. So, the environment script is in one place or the other.
+  [ -e $env_script_original ]  env_script=$env_script_original
+  [ -e $env_script_relocated ]  env_script=$env_script_relocated
+
+  $SUDO sed -i -e s%##SDKTARGETSYSROOT##%$target_sysroot%g $env_script
 done
 
 if [ $YOCTOADT_QEMU == Y ] || [ $YOCTOADT_QEMU = y ]; then
@@ -232,15 +265,8 @@ echo_info \nSuccessfully installed selected native ADT!
 install_target()
 {
 
-# rootfs extraction directory
-target_sysroot_var=\$YOCTOADT_TARGET_SYSROOT_LOC_$1
-target_sysroot=`eval echo $target_sysroot_var`
-
-if [ $target_sysroot ==  ]; then
-  return 0
-else
-  target_sysroot=`readlink -m $target_sysroot`
-fi
+target_sysroot=`get_target_rootfs_location $1`
+[ -z $target_sysroot ]  return 0
 
 target_sysroot_image_var=\$YOCTOADT_TARGET_SYSROOT_IMAGE_$1
 target_sysroot_image=`eval echo $target_sysroot_image_var`
@@ -261,23 +287,6 @@ SUDO=$(get_sudo_app $target_sysroot)
 
 $SUDO scripts/extract_rootfs $sysroot_image_name $target_sysroot 
$OECORE_NATIVE_SYSROOT $user_inst_type
 check_result
-
-echo_info Updating environment script with target sysroot location.
-if [ $1 == x86  ]; then
-  env_filename=`ls $INSTALL_FOLDER/environment-setup-i586*`
-else
-  env_filename=`ls $INSTALL_FOLDER/environment-setup-$1*`
-fi
-
-if [ ! -z $env_filename ]; then
-  SUDO=$(get_sudo_app $INSTALL_FOLDER)
-  $SUDO sed -i -e s%##SDKTARGETSYSROOT##%$target_sysroot%g $env_filename
-else
-  echo_info [ADT_INST] Error: Failed to find environment script for arch: $1
-  return 1 
-fi
-
-
 }
 
 
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/1] adt_installer: fix issue with x86_64

2014-02-26 Thread Laurentiu Palcu
The following changes since commit e47afff7d812c74f8091d85facfdf34d6a1b698e:

  runqemu: Use readlink instead of realpath (2014-02-25 21:29:53 +)

are available in the git repository at:

  git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib 
lpalcu/b5806_adt_installer_fails_for_xf86_64

for you to fetch changes up to 72cf90ecf51d88becfaf037b6233c78e8c6e4a9c:

  adt_installer: fix issue with x86_64 (2014-02-26 15:36:44 +0200)


Laurentiu Palcu (1):
  adt_installer: fix issue with x86_64

 .../adt-installer/scripts/adt_installer_internal   |   61 +++-
 1 file changed, 35 insertions(+), 26 deletions(-)

Laurentiu Palcu (1):
  adt_installer: fix issue with x86_64

 .../adt-installer/scripts/adt_installer_internal   |   61 +++-
 1 file changed, 35 insertions(+), 26 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/1] package_manager.py: create index files for all backends in PACKAGE_CLASSES

2014-02-25 Thread Laurentiu Palcu
The following changes since commit 72576e2d04dc8b43ae0d7f31cddd74318f52103e:

  bitbake: hob: replace the use of hob-toolchain with populate_sdk task 
(2014-02-25 08:06:11 +)

are available in the git repository at:

  git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib 
lpalcu/b5827_package_index_fix

for you to fetch changes up to ae8627a6eb1d76b1535a222d9011093bdee79cdb:

  package_manager.py: create index files for all backends in PACKAGE_CLASSES 
(2014-02-25 14:25:35 +0200)


Laurentiu Palcu (1):
  package_manager.py: create index files for all backends in PACKAGE_CLASSES

 meta/lib/oe/package_manager.py |   24 +++-
 1 file changed, 15 insertions(+), 9 deletions(-)

Laurentiu Palcu (1):
  package_manager.py: create index files for all backends in
PACKAGE_CLASSES

 meta/lib/oe/package_manager.py |   24 +++-
 1 file changed, 15 insertions(+), 9 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/1] package_manager.py: create index files for all backends in PACKAGE_CLASSES

2014-02-25 Thread Laurentiu Palcu
The previous implementation was checking IMAGE_PKGTYPE and created the
index files just for the backend used to create the image. Apparently,
'bitbake package-index' should attempt to create the index files for all
backends specified in PACKAGE_CLASSES.

[YOCTO #5827]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/package_manager.py |   24 +++-
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index d29adac..ff4f1de 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -1519,19 +1519,25 @@ class DpkgPM(PackageManager):
 
 
 def generate_index_files(d):
-img_type = d.getVar('IMAGE_PKGTYPE', True)
+classes = d.getVar('PACKAGE_CLASSES', True).replace(package_, ).split()
+
+indexer_map = {
+rpm: (RpmIndexer, d.getVar('DEPLOY_DIR_RPM', True)),
+ipk: (OpkgIndexer, d.getVar('DEPLOY_DIR_IPK', True)),
+deb: (DpkgIndexer, d.getVar('DEPLOY_DIR_DEB', True))
+}
 
 result = None
 
-if img_type == rpm:
-result = RpmIndexer(d, d.getVar('DEPLOY_DIR_RPM', True)).write_index()
-elif img_type == ipk:
-result = OpkgIndexer(d, d.getVar('DEPLOY_DIR_IPK', True)).write_index()
-elif img_type == deb:
-result = DpkgIndexer(d, d.getVar('DEPLOY_DIR_DEB', True)).write_index()
+for pkg_class in classes:
+if not pkg_class in indexer_map:
+continue
+
+if os.path.exists(indexer_map[pkg_class][1]):
+result = indexer_map[pkg_class][0](d, 
indexer_map[pkg_class][1]).write_index()
 
-if result is not None:
-bb.fatal(result)
+if result is not None:
+bb.fatal(result)
 
 if __name__ == __main__:
 
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH V3 0/5] manifest.py/package_manager.py/rootfs.py: support ipk incremental image generation

2014-02-21 Thread Laurentiu Palcu
Reviewed-by: Laurentiu Palcu laurentiu.palcu.intel.com

laurentiu

On Fri, Feb 21, 2014 at 02:38:16PM +0800, Hongxu Jia wrote:
 Change in V2:
 - Move dump_install_solution to Manifest class as create_full;
 - Move load_old_install_solution to Manifest class as parse_full_manifest;
 - Handle the output of dummy install in Python rather than shell;
 - Fix typos
 - Rename _file_duplicate with _file_equal to avoid confusion;
 
 Change in V3:
 - While create_full calls dummy_install, pass an instance to PM
   as an argument rather than instantiate another PM object;
 - Rename _create_incremental with _remove_extra_packages;
 - Move the call of _remove_extra_packages below the call of
   self.pm.handle_bad_recommendations();
 - Remove redundant self.pm.update in _remove_extra_packages;
 - Rename _remove_existing_image with _remove_old_rootfs;
 
 Test Cases
 
 -
 Case 1: Rebuild in place
 1. Edit local.conf, enable multilib and ipk incremental image generation
 ...
  46 MACHINE ?= qemux86-64
 254 require conf/multilib.conf
 255 MULTILIBS = multilib:lib32
 256 DEFAULTTUNE_virtclass-multilib-lib32 = x86
 257 
 258 IMAGE_INSTALL_append =  lib32-gzip
 260 IMAGE_FEATURES_append =  package-management ssh-server-dropbear 
 262 
 263 INC_IPK_IMAGE_GEN = 1
 ...
 bb core-image-minimal
 
 2. Edit local.conf, add some spaces in IMAGE_INSTALL_append
 vim local.conf
 ...
 258 IMAGE_INSTALL_append =  lib32-gzip 
 ...
 
 3. bb core-image-minimal, the image should not be rebuilt
 vim log.do_rootfs:
 ...
  16 NOTE: Package packagegroup-core-ssh-dropbear (1.0-r1) installed in root 
 is up to date.
  17 Package opkg-collateral (1.0-r2) installed in root is up to date.
  18 Package run-postinsts (1.0-r9) installed in root is up to date.
  19 Package packagegroup-core-boot (1.0-r11) installed in root is up to date.
  20 Package opkg (1:0.2.0-r0) installed in root is up to date.
  21 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date.
  ...
  53 NOTE: Package lib32-gzip (1.6-r7) installed in root is up to date.
 ...
 
 vim installed_pkgs.txt
 ...
 lib32-gzip x86
 ...
 
 4. runqemu qemux86-64, ssh could work and lib32-gzip is installed
 On host:
 runqemu qemux86-64
 ssh root@192.168.7.2 and run 'opkg-cl list-installed'
 ...
 lib32-gzip - 1.6-r7
 ...
 
 -
 Case 2: Decremental Rebuild
 1. Edit local.conf, enable multilib and ipk incremental image generation
 ...
  46 MACHINE ?= qemux86-64
 254 require conf/multilib.conf
 255 MULTILIBS = multilib:lib32
 256 DEFAULTTUNE_virtclass-multilib-lib32 = x86
 257 
 258 IMAGE_INSTALL_append =  lib32-gzip openssl
 260 IMAGE_FEATURES_append =  package-management ssh-server-dropbear 
 262 
 263 INC_IPK_IMAGE_GEN = 1
 ...
 bb core-image-minimal
 
 2. Edit local.conf, remove 'lib32-gzip openssl'
 vim local.conf
 ...
 258 # IMAGE_INSTALL_append =  lib32-gzip openssl
 ...
 
 3. bb core-image-minimal, lib32-gzip and openssl should be removed.
 vim log.do_rootfs:
 ...
  20 NOTE: Removing package lib32-eglibc from root...
  21 Removing package lib32-gzip from root...
  22 Removing package lib32-update-alternatives-opkg from root...
  23 Removing package libcrypto1.0.0 from root...
  24 Removing package libssl1.0.0 from root...
  25 Removing package openssl from root...
  26 Removing package openssl-conf from root...
  ...
  31 Package opkg-collateral (1.0-r2) installed in root is up to date.
  32 Package run-postinsts (1.0-r9) installed in root is up to date.
  33 Package packagegroup-core-boot (1.0-r11) installed in root is up to date.
  34 Package opkg (1:0.2.0-r0) installed in root is up to date.
  35 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date.
 ...
 
 -
 Case 3: Incremental Rebuild
 1. Edit local.conf, enable multilib and ipk incremental image generation
 ...
  46 MACHINE ?= qemux86-64
 254 require conf/multilib.conf
 255 MULTILIBS = multilib:lib32
 256 DEFAULTTUNE_virtclass-multilib-lib32 = x86
 257 
 258 # IMAGE_INSTALL_append =  lib32-gzip openssl
 260 IMAGE_FEATURES_append =  package-management ssh-server-dropbear 
 262 
 263 INC_IPK_IMAGE_GEN = 1
 ...
 bb core-image-minimal
 
 2. Edit local.conf, append 'lib32-gzip openssl' to IMAGE_INSTALL
 vim local.conf
 ...
 258 IMAGE_INSTALL_append =  lib32-gzip openssl
 ...
 
 3. bb core-image-minimal, lib32-gzip and openssl should be added.
 vim log.do_rootfs:
 ...
  20 NOTE: Package packagegroup-core-ssh-dropbear (1.0-r1) installed in root 
 is up to date.
  21 Package opkg-collateral (1.0-r2) installed in root is up to date.
  22 Package run-postinsts (1.0-r9) installed in root is up to date.
  23 Installing openssl (1.0.1e-r15.0) to root...
  ...
  31 Package packagegroup-core-boot (1.0-r11) installed in root is up to date.
  32 Package opkg (1:0.2.0-r0) installed in root is up to date.
  33 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date.
  ...
  54 NOTE

Re: [OE-core] [PATCH 1/5] manifest.py: add create_full for OpkgManifest class

2014-02-20 Thread Laurentiu Palcu
On Thu, Feb 20, 2014 at 03:06:51PM +0800, Hongxu Jia wrote:
 The function create_full creates the manifest after the package in
 initial manifest has been dummy installed. It lists all *to be
 installed* packages. There is no real installation, just a test.
 
 [YOCTO #1894]
 Signed-off-by: Hongxu Jia hongxu@windriver.com
 ---
  meta/lib/oe/manifest.py | 65 
 -
  1 file changed, 64 insertions(+), 1 deletion(-)
 
 diff --git a/meta/lib/oe/manifest.py b/meta/lib/oe/manifest.py
 index a4bc04b..6e40a9f 100644
 --- a/meta/lib/oe/manifest.py
 +++ b/meta/lib/oe/manifest.py
 @@ -1,6 +1,8 @@
  from abc import ABCMeta, abstractmethod
 +from oe.package_manager import *
  import os
  import re
 +import bb
  
  
  class Manifest(object):
 @@ -69,6 +71,7 @@ class Manifest(object):
  
  self.initial_manifest = os.path.join(self.manifest_dir, 
 %s_initial_manifest % manifest_type)
  self.final_manifest = os.path.join(self.manifest_dir, 
 %s_final_manifest % manifest_type)
 +self.full_manifest = os.path.join(self.manifest_dir, 
 %s_full_manifest % manifest_type)
  
  # packages in the following vars will be split in 'must install' and
  # 'multilib'
 @@ -128,6 +131,15 @@ class Manifest(object):
  pass
  
  
 +This creates the manifest after the package in initial manifest has been
 +dummy installed. It lists all *to be installed* packages. There is no 
 real
 +installation, just a test.
 +
 +@abstractmethod
 +def create_full(self):
 +pass
 +
 +
  The following function parses an initial manifest and returns a 
 dictionary
  object with the must install, attempt only, multilib and language 
 packages.
  
 @@ -158,6 +170,22 @@ class Manifest(object):
  
  return pkgs
  
 +'''
 +This following function parses a full manifest and return a list
 +object with packages.
 +'''
 +def parse_full_manifest(self):
 +installed_pkgs = list()
 +if not os.path.exists(self.full_manifest):
 +bb.note('full manifest not exist')
 +return installed_pkgs
 +
 +with open(self.full_manifest, 'r') as manifest:
 +for pkg in manifest.read().split('\n'):
 +installed_pkgs.append(pkg.strip())
 +
 +return installed_pkgs
 +
  
  class RpmManifest(Manifest):
  
 @@ -202,10 +230,12 @@ class RpmManifest(Manifest):
  for pkg in pkgs[pkg_type].split():
  manifest.write(%s,%s\n % (pkg_type, pkg))
  
 -
  def create_final(self):
  pass
  
 +def create_full(self):
 +pass
 +
  
  class OpkgManifest(Manifest):
  
 @@ -253,6 +283,36 @@ class OpkgManifest(Manifest):
  def create_final(self):
  pass
  
 +def create_full(self):
 +if not os.path.exists(self.initial_manifest):
 +self.create_initial()
 +
 +initial_manifest = self.parse_initial_manifest()
 +pkgs_to_install = list()
 +for pkg_type in initial_manifest:
 +pkgs_to_install += initial_manifest[pkg_type]
 +if len(pkgs_to_install) == 0:
 +return
 +
 +image_rootfs = self.d.expand(${T}/opkg)
 +opkg_conf = self.d.getVar(IPKGCONF_TARGET, True)
 +pkg_archs = self.d.getVar(ALL_MULTILIB_PACKAGE_ARCHS, True)
 +pm = OpkgPM(self.d, image_rootfs, opkg_conf, pkg_archs)
Do we really need to instantiate another PM object here? Can't the
caller pass an instance to it? After all, dummy_install() is not
supposed to alter the metadata, am I wrong? So, in this case, I believe
the caller can safely pass an instance to PM as an argument to the
function.

 +
 +pm.update()
 +pm.handle_bad_recommendations()
 +output = pm.dummy_install(pkgs_to_install)
 +
 +with open(self.full_manifest, 'w+') as manifest:
 +pkg_re = re.compile('^Installing ([^ ]+) [^ ].*')
 +for line in set(output.split('\n')):
 +m = pkg_re.match(line)
 +if m:
 +manifest.write(m.group(1) + '\n')
 +
 +bb.utils.remove(image_rootfs, True)
 +return
 +
  
  class DpkgManifest(Manifest):
  def create_initial(self):
 @@ -272,6 +332,9 @@ class DpkgManifest(Manifest):
  def create_final(self):
  pass
  
 +def create_full(self):
 +pass
 +
  
  def create_manifest(d, final_manifest=False, manifest_dir=None,
  manifest_type=Manifest.MANIFEST_TYPE_IMAGE):
 -- 
 1.8.1.2
 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 2/5] package_manager.py: support ipk incremental image generation

2014-02-20 Thread Laurentiu Palcu
On Thu, Feb 20, 2014 at 03:06:52PM +0800, Hongxu Jia wrote:
 Add the following three functions to OpkgPM class:
 - The 'dummy_install' is used to dummy install pkgs, and returns the log
   of output;
 - The 'backup_packaging_data' is used to back up the current opkg
   database;
 - The 'recover_packaging_data' is used to recover the opkg database
   which backed up by the previous image creation;
 
 Tweak 'remove' function in OpkgPM class, which the options for remove
 with dependencies was incorrect.
 
 Tweak 'handle_bad_recommendations' function in OpkgPM class:
 - Fix none value check;
 - Add the existance check of opkg status file;
 - Fix the log format typo;
 
 [YOCTO #1894]
 Signed-off-by: Hongxu Jia hongxu@windriver.com
 ---
  meta/lib/oe/package_manager.py | 66 
 +-
  1 file changed, 59 insertions(+), 7 deletions(-)
 
 diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
 index 6dc8fbd..eb96727 100644
 --- a/meta/lib/oe/package_manager.py
 +++ b/meta/lib/oe/package_manager.py
 @@ -934,12 +934,13 @@ class RpmPM(PackageManager):
  
  
  class OpkgPM(PackageManager):
 -def __init__(self, d, target_rootfs, config_file, archs):
 +def __init__(self, d, target_rootfs, config_file, archs, 
 task_name='target'):
  super(OpkgPM, self).__init__(d)
  
  self.target_rootfs = target_rootfs
  self.config_file = config_file
  self.pkg_archs = archs
 +self.task_name = task_name
  
  self.deploy_dir = self.d.getVar(DEPLOY_DIR_IPK, True)
  self.deploy_lock_file = os.path.join(self.deploy_dir, deploy.lock)
 @@ -956,6 +957,10 @@ class OpkgPM(PackageManager):
  
  bb.utils.mkdirhier(self.opkg_dir)
  
 +self.saved_opkg_dir = self.d.expand('${T}/saved/%s' % self.task_name)
 +if not os.path.exists(self.d.expand('${T}/saved')):
 +bb.utils.mkdirhier(self.d.expand('${T}/saved'))
 +
  if (self.d.getVar('BUILD_IMAGES_FROM_FEEDS', True) or ) != 1:
  self._create_config()
  else:
 @@ -1075,7 +1080,9 @@ class OpkgPM(PackageManager):
  
  try:
  bb.note(Installing the following packages: %s % ' '.join(pkgs))
 -subprocess.check_output(cmd.split())
 +bb.note(cmd)
 +output = subprocess.check_output(cmd.split())
 +bb.note(output)
  except subprocess.CalledProcessError as e:
  (bb.fatal, bb.note)[attempt_only](Unable to install packages. 
Command '%s' returned 
 %d:\n%s %
 @@ -1083,14 +1090,16 @@ class OpkgPM(PackageManager):
  
  def remove(self, pkgs, with_dependencies=True):
  if with_dependencies:
 -cmd = %s %s remove %s % \
 +cmd = %s %s --force-depends --force-remove 
 --force-removal-of-dependent-packages remove %s % \
  (self.opkg_cmd, self.opkg_args, ' '.join(pkgs))
  else:
  cmd = %s %s --force-depends remove %s % \
  (self.opkg_cmd, self.opkg_args, ' '.join(pkgs))
  
  try:
 -subprocess.check_output(cmd.split())
 +bb.note(cmd)
 +output = subprocess.check_output(cmd.split())
 +bb.note(output)
  except subprocess.CalledProcessError as e:
  bb.fatal(Unable to remove packages. Command '%s' 
   returned %d:\n%s % (e.cmd, e.returncode, e.output))
 @@ -1146,12 +1155,17 @@ class OpkgPM(PackageManager):
  return output
  
  def handle_bad_recommendations(self):
 -bad_recommendations = self.d.getVar(BAD_RECOMMENDATIONS, True)
 -if bad_recommendations is None:
 +bad_recommendations = self.d.getVar(BAD_RECOMMENDATIONS, True) or 
 
 +if bad_recommendations.strip() == :
  return
  
  status_file = os.path.join(self.opkg_dir, status)
  
 +# If status file existed, it means the bad recommendations has 
 already
 +# been handled
 +if os.path.exists(status_file):
 +return
 +
  cmd = %s %s info  % (self.opkg_cmd, self.opkg_args)
  
  with open(status_file, w+) as status:
 @@ -1165,7 +1179,7 @@ class OpkgPM(PackageManager):
   returned %d:\n%s % (pkg_info, e.returncode, 
 e.output))
  
  if output == :
 -bb.note(Requested ignored recommendation $i is 
 +bb.note(Requested ignored recommendation %s is 
I already saw a patch fixing this on the mailing list. You might need to
rebase.

  not a package % pkg)
  continue
  
 @@ -1175,6 +1189,44 @@ class OpkgPM(PackageManager):
  else:
  status.write(line + \n)
  
 +'''
 +The following function dummy installs pkgs and returns the log of output.
 +'''
 +def dummy_install(self, pkgs):
 +

Re: [OE-core] [PATCH 3/5] rootfs.py: support ipk incremental image generation

2014-02-20 Thread Laurentiu Palcu
On Thu, Feb 20, 2014 at 03:06:53PM +0800, Hongxu Jia wrote:
 The incremental image generation is based on the previous existing
 image, adds new packages, upgrades existing packages, and removes unused
 packages.
 
 [YOCTO #1894]
 Signed-off-by: Hongxu Jia hongxu@windriver.com
 ---
  meta/lib/oe/rootfs.py | 52 
 ---
  1 file changed, 49 insertions(+), 3 deletions(-)
 
 diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
 index 3bcb812..93106c2 100644
 --- a/meta/lib/oe/rootfs.py
 +++ b/meta/lib/oe/rootfs.py
 @@ -438,13 +438,25 @@ class OpkgRootfs(Rootfs):
  def __init__(self, d, manifest_dir):
  super(OpkgRootfs, self).__init__(d)
  
 -bb.utils.remove(self.image_rootfs, True)
 -bb.utils.remove(self.d.getVar('MULTILIB_TEMP_ROOTFS', True), True)
  self.manifest = OpkgManifest(d, manifest_dir)
  self.opkg_conf = self.d.getVar(IPKGCONF_TARGET, True)
  self.pkg_archs = self.d.getVar(ALL_MULTILIB_PACKAGE_ARCHS, True)
  
 -self.pm = OpkgPM(d, self.image_rootfs, self.opkg_conf, 
 self.pkg_archs)
 +self.inc_opkg_image_gen = self.d.getVar('INC_IPK_IMAGE_GEN', True) 
 or 
 +if self.inc_opkg_image_gen != '1':
 +bb.utils.remove(self.image_rootfs, True)
 +self.pm = OpkgPM(d,
 + self.image_rootfs,
 + self.opkg_conf,
 + self.pkg_archs)
 +else:
 +self.pm = OpkgPM(d,
 + self.image_rootfs,
 + self.opkg_conf,
 + self.pkg_archs)
 +self.pm.recover_packaging_data()
 +
 +bb.utils.remove(self.d.getVar('MULTILIB_TEMP_ROOTFS', True), True)
  
  
  This function was reused from the old implementation.
 @@ -508,6 +520,34 @@ class OpkgRootfs(Rootfs):
  
  self._multilib_sanity_test(dirs)
  
 +'''
 +While ipk incremental image generation is enabled, it will remove the
 +unneeded pkgs by comparing the old full manifest in previous existing
 +image and the new full manifest in the current image.
 +'''
 +def _create_incremental(self, pkgs_initial_install):
Can we name this function _remove_extra_packages()? Because it doesn't
actually install anything... it just removes packages, correct?

 +if self.inc_opkg_image_gen == 1:
 +# Parse full manifest in previous existing image creation session
 +old_full_manifest = self.manifest.parse_full_manifest()
 +
 +# Create full manifest for the current image session, the old one
 +# will be replaced by the new one.
 +self.manifest.create_full()
 +
 +# Parse full manifest in current image creation session
 +new_full_manifest = self.manifest.parse_full_manifest()
 +
 +pkg_to_remove = list()
 +for pkg in old_full_manifest:
 +if pkg not in new_full_manifest:
 +pkg_to_remove.append(pkg)
 +
 +self.pm.update()
this update could be removed, if we move the function call below
self.pm.handle_bad_recommendations(). See below.

 +
 +if pkg_to_remove != []:
 +bb.note('decremental removed: %s' % ' '.join(pkg_to_remove))
 +self.pm.remove(pkg_to_remove)
 +
  def _create(self):
  pkgs_to_install = self.manifest.parse_initial_manifest()
  opkg_pre_process_cmds = self.d.getVar('OPKG_PREPROCESS_COMMANDS', 
 True)
 @@ -518,6 +558,9 @@ class OpkgRootfs(Rootfs):
  if (self.d.getVar('BUILD_IMAGES_FROM_FEEDS', True) or ) != 1:
  self.pm.write_index()
  
 +if self.inc_opkg_image_gen == 1:
 +self._create_incremental(pkgs_to_install)
Why not move this piece lower, below
self.pm.handle_bad_recommendations()? Correct me if I'm wrong, but I
believe the rootfs pre process commands should be executed before we do
anything on the rootfs generation.

 + execute_pre_post_process(self.d, opkg_pre_process_cmds)
  
  self.pm.update()
 @@ -540,6 +583,9 @@ class OpkgRootfs(Rootfs):
  execute_pre_post_process(self.d, opkg_post_process_cmds)
  execute_pre_post_process(self.d, rootfs_post_install_cmds)
  
 +if self.inc_opkg_image_gen == 1:
 +self.pm.backup_packaging_data()
 +
  def _get_delayed_postinsts(self):
  pkg_list = []
  status_file = os.path.join(self.image_rootfs,
 -- 
 1.8.1.2
 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 5/5] rootfs.py: support BAD_RECOMMENDATIONS for ipk incremental image generation

2014-02-20 Thread Laurentiu Palcu
On Thu, Feb 20, 2014 at 03:06:55PM +0800, Hongxu Jia wrote:
 While incremental image generation enabled and the previous image is
 existed, if BAD_RECOMMENDATIONS is changed, the operation on the
 existing image is complicated, so remove the existing image in this
 situation.
 
 The same with PACKAGE_EXCLUDE and NO_RECOMMENDATIONS.
 
 [YOCTO #1894]
 
 Signed-off-by: Hongxu Jia hongxu@windriver.com
 ---
  meta/lib/oe/rootfs.py | 29 -
  1 file changed, 28 insertions(+), 1 deletion(-)
 
 diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
 index 6120bce..80be490 100644
 --- a/meta/lib/oe/rootfs.py
 +++ b/meta/lib/oe/rootfs.py
 @@ -445,7 +445,7 @@ class OpkgRootfs(Rootfs):
  self.pkg_archs = self.d.getVar(ALL_MULTILIB_PACKAGE_ARCHS, True)
  
  self.inc_opkg_image_gen = self.d.getVar('INC_IPK_IMAGE_GEN', True) 
 or 
 -if self.inc_opkg_image_gen != '1':
 +if self._remove_existing_image():
  bb.utils.remove(self.image_rootfs, True)
  self.pm = OpkgPM(d,
   self.image_rootfs,
 @@ -595,6 +595,33 @@ class OpkgRootfs(Rootfs):
  bb.note('decremental removed: %s' % ' '.join(pkg_to_remove))
  self.pm.remove(pkg_to_remove)
  
 +'''
 +Compare with previous existing image creation, if some conditions
 +triggered, the previous existing image should be removed.
 +The conditions include any of 'PACKAGE_EXCLUDE, NO_RECOMMENDATIONS
 +and BAD_RECOMMENDATIONS' has been changed.
 +'''
 +def _remove_existing_image(self):
Since we're not removing the image, but the old rootfs, a better name
for this would be: _remove_old_rootfs().

 +if self.inc_opkg_image_gen != 1:
 +return True
 +
 +vars_list_file = self.d.expand('${T}/vars_list')
 +
 +old_vars_list = 
 +if os.path.exists(vars_list_file):
 +old_vars_list = open(vars_list_file, 'r+').read()
 +
 +new_vars_list = '%s:%s:%s\n' % \
 +((self.d.getVar('BAD_RECOMMENDATIONS', True) or '').strip(),
 + (self.d.getVar('NO_RECOMMENDATIONS', True) or '').strip(),
 + (self.d.getVar('PACKAGE_EXCLUDE', True) or '').strip())
 +open(vars_list_file, 'w+').write(new_vars_list)
 +
 +if old_vars_list != new_vars_list:
 +return True
 +
 +return False
 +
  def _create(self):
  pkgs_to_install = self.manifest.parse_initial_manifest()
  opkg_pre_process_cmds = self.d.getVar('OPKG_PREPROCESS_COMMANDS', 
 True)
 -- 
 1.8.1.2
 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/3] package_manager.py: fix installed package list creation for Opkg/Dpkg

2014-02-19 Thread Laurentiu Palcu
Small error in the package list creation routine. Buildhistory was
supposed to use this but was never called. Hence, it escaped tests...

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/package_manager.py |6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 6dc8fbd..a3c0a8e 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -1134,7 +1134,8 @@ class OpkgPM(PackageManager):
 
 if format == file:
 tmp_output = 
-for pkg, pkg_file, pkg_arch in tuple(output.split('\n')):
+for line in output.split('\n'):
+pkg, pkg_file, pkg_arch = line.split()
 full_path = os.path.join(self.deploy_dir, pkg_arch, pkg_file)
 if os.path.exists(full_path):
 tmp_output += %s %s %s\n % (pkg, full_path, pkg_arch)
@@ -1435,7 +1436,8 @@ class DpkgPM(PackageManager):
 
 if format == file:
 tmp_output = 
-for pkg, pkg_file, pkg_arch in tuple(output.split('\n')):
+for line in tuple(output.split('\n')):
+pkg, pkg_file, pkg_arch = line.split()
 full_path = os.path.join(self.deploy_dir, pkg_arch, pkg_file)
 if os.path.exists(full_path):
 tmp_output += %s %s %s\n % (pkg, full_path, pkg_arch)
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/3] Fix buildhistory problem

2014-02-19 Thread Laurentiu Palcu
Tested with all backends. Fixed some other issues in the process...
Buildhistory should be fine now.

laurentiu

The following changes since commit 54562006c1327c5b99daa4cc05a3ba7e38412da1:

  image_types.bbclass: Fix tar IMAGE_CMD to not change directories (2014-02-18 
08:38:52 +)

are available in the git repository at:

  git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib 
lpalcu/buildhistory_fix

for you to fetch changes up to 15ce4b01408e1ec91655103191c4c9436d894597:

  populate_sdk_*.bbclass: remove left over bash routines (2014-02-19 12:45:51 
+0200)


Laurentiu Palcu (3):
  package_manager.py: fix installed package list creation for Opkg/Dpkg
  buildhistory.bbclass: fix creation of installed packages list
  populate_sdk_*.bbclass: remove left over bash routines

 meta/classes/buildhistory.bbclass |   21 +
 meta/classes/populate_sdk_deb.bbclass |   21 -
 meta/classes/populate_sdk_ipk.bbclass |   20 
 meta/lib/oe/package_manager.py|6 --
 4 files changed, 21 insertions(+), 47 deletions(-)

Laurentiu Palcu (3):
  package_manager.py: fix installed package list creation for Opkg/Dpkg
  buildhistory.bbclass: fix creation of installed packages list
  populate_sdk_*.bbclass: remove left over bash routines

 meta/classes/buildhistory.bbclass |   21 +
 meta/classes/populate_sdk_deb.bbclass |   21 -
 meta/classes/populate_sdk_ipk.bbclass |   20 
 meta/lib/oe/package_manager.py|6 --
 4 files changed, 21 insertions(+), 47 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/3] buildhistory.bbclass: fix creation of installed packages list

2014-02-19 Thread Laurentiu Palcu
Call the new python routines to create the packages list.

[YOCTO #5831]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/classes/buildhistory.bbclass |   21 +
 1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass 
b/meta/classes/buildhistory.bbclass
index 0033b5a..ef4135b 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -311,13 +311,23 @@ def write_pkghistory(pkginfo, d):
 if os.path.exists(filevarpath):
 os.unlink(filevarpath)
 
+python buildhistory_list_installed() {
+from oe.rootfs import list_installed_packages
+
+pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
+  bh_installed_pkgs.txt)
+
+with open(pkgs_list_file, 'w') as pkgs_list:
+pkgs_list.write(list_installed_packages(d, 'file'))
+}
+
 
 buildhistory_get_installed() {
mkdir -p $1
 
# Get list of installed packages
pkgcache=$1/installed-packages.tmp
-   list_installed_packages file | sort  $pkgcache
+   cat ${WORKDIR}/bh_installed_pkgs.txt | sort  $pkgcache  rm 
${WORKDIR}/bh_installed_pkgs.txt
 
cat $pkgcache | awk '{ print $1 }'  $1/installed-package-names.txt
if [ -s $pkgcache ] ; then
@@ -452,13 +462,16 @@ END
 }
 
 # By prepending we get in before the removal of packaging files
-ROOTFS_POSTPROCESS_COMMAND =+ buildhistory_get_image_installed ; 
+ROOTFS_POSTPROCESS_COMMAND =+  buildhistory_list_installed ;\
+buildhistory_get_image_installed ; 
 
 IMAGE_POSTPROCESS_COMMAND +=  buildhistory_get_imageinfo ; 
 
 # We want these to be the last run so that we get called after complementary 
package installation
-POPULATE_SDK_POST_TARGET_COMMAND_append = 
buildhistory_get_sdk_installed_target ; 
-POPULATE_SDK_POST_HOST_COMMAND_append = buildhistory_get_sdk_installed_host ; 

+POPULATE_SDK_POST_TARGET_COMMAND_append =  buildhistory_list_installed ;\
+
buildhistory_get_sdk_installed_target ; 
+POPULATE_SDK_POST_HOST_COMMAND_append =  buildhistory_list_installed ;\
+  buildhistory_get_sdk_installed_host 
; 
 
 SDK_POSTPROCESS_COMMAND += buildhistory_get_sdkinfo ; 
 
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 3/3] populate_sdk_*.bbclass: remove left over bash routines

2014-02-19 Thread Laurentiu Palcu
The list_installed_packages bash routine is no longer needed here. We've
got a python replacement.

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/classes/populate_sdk_deb.bbclass |   21 -
 meta/classes/populate_sdk_ipk.bbclass |   20 
 2 files changed, 41 deletions(-)

diff --git a/meta/classes/populate_sdk_deb.bbclass 
b/meta/classes/populate_sdk_deb.bbclass
index d03b055..462525f 100644
--- a/meta/classes/populate_sdk_deb.bbclass
+++ b/meta/classes/populate_sdk_deb.bbclass
@@ -11,27 +11,6 @@ do_populate_sdk[lockfiles] += ${DEPLOY_DIR_DEB}/deb.lock
 # This will of course only work after rootfs_deb_do_rootfs or populate_sdk_deb 
has been called
 DPKG_QUERY_COMMAND = ${STAGING_BINDIR_NATIVE}/dpkg-query 
--admindir=$INSTALL_ROOTFS_DEB/var/lib/dpkg
 
-list_installed_packages() {
-   if [ $1 = arch ] ; then
-   # Here we want the PACKAGE_ARCH not the deb architecture
-   ${DPKG_QUERY_COMMAND} -W -f='${Package} ${PackageArch}\n'
-   elif [ $1 = file ] ; then
-   ${DPKG_QUERY_COMMAND} -W -f='${Package} 
${Package}_${Version}_${Architecture}.deb ${PackageArch}\n' | while read pkg 
pkgfile pkgarch
-   do
-   fullpath=`find ${DEPLOY_DIR_DEB} -name $pkgfile || 
true`
-   if [ $fullpath =  ] ; then
-   echo $pkg $pkgfile $pkgarch
-   else
-   echo $pkg $fullpath $pkgarch
-   fi
-   done
-   elif [ $1 = ver ] ; then
-   ${DPKG_QUERY_COMMAND} -W -f='${Package} ${PackageArch} 
${Version}\n'
-   else
-   ${DPKG_QUERY_COMMAND} -W -f='${Package}\n'
-   fi
-}
-
 rootfs_list_installed_depends() {
# Cheat here a little bit by using the opkg query helper util
${DPKG_QUERY_COMMAND} -W -f='Package: ${Package}\nDepends: 
${Depends}\nRecommends: ${Recommends}\n\n' | opkg-query-helper.py
diff --git a/meta/classes/populate_sdk_ipk.bbclass 
b/meta/classes/populate_sdk_ipk.bbclass
index f51a22c..4c23f05 100644
--- a/meta/classes/populate_sdk_ipk.bbclass
+++ b/meta/classes/populate_sdk_ipk.bbclass
@@ -3,26 +3,6 @@ do_populate_sdk[recrdeptask] += do_package_write_ipk
 
 do_populate_sdk[lockfiles] += ${WORKDIR}/ipk.lock
 
-list_installed_packages() {
-   if [ $1 = arch ] ; then
-   opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py -a
-   elif [ $1 = file ] ; then
-   opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py -f | while 
read pkg pkgfile pkgarch
-   do
-   fullpath=`find ${DEPLOY_DIR_IPK} -name $pkgfile || 
true`
-   if [ $fullpath =  ] ; then
-   echo $pkg $pkgfile $pkgarch
-   else
-   echo $pkg $fullpath $pkgarch
-   fi
-   done
-   elif [ $1 = ver ] ; then
-   opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py -v
-   else
-   opkg-cl ${OPKG_ARGS} list_installed | awk '{ print $1 }'
-   fi
-}
-
 rootfs_list_installed_depends() {
opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py
 }
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH] update-rc.d.bbclass: fix inhibit check

2014-02-19 Thread Laurentiu Palcu
On Wed, Feb 19, 2014 at 06:59:26PM +0800, Kai Kang wrote:
 In update-rc.d.bbclass it checks variable INITSCRIPT_PACKAGES to avoid
 inherit this class. But it is wrong logic to check INITSCRIPT_PACKAGES.
 When 'sysvinit' is in 'DISTRO_FEATURES', INITSCRIPT_PACKAGES will not be
 checked.
s/INITSCRIPT_PACKAGES/INHIBIT_UPDATERCD_BBCLASS/ above will make more
sense, I guess.

Because, as I see it, that 'if' will evaluate to True if 'sysvinit' is in
DISTRO_FEATURES, irrespective of how INHIBIT_UPDATERCD_BBCLASS is set...
which is a little bit wrong.

So, I think you just need to adjust the commit message.

laurentiu

 
 Replace 'or' with 'and' to fix it.
 
 Signed-off-by: Kai Kang kai.k...@windriver.com
 ---
  meta/classes/update-rc.d.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/meta/classes/update-rc.d.bbclass 
 b/meta/classes/update-rc.d.bbclass
 index 4b92d8d..0ac2af7 100644
 --- a/meta/classes/update-rc.d.bbclass
 +++ b/meta/classes/update-rc.d.bbclass
 @@ -117,7 +117,7 @@ python populate_packages_updatercd () {
  
  # Check that this class isn't being inhibited (generally, by
  # systemd.bbclass) before doing any work.
 -if oe.utils.contains('DISTRO_FEATURES', 'sysvinit', True, False, d) or \
 +if oe.utils.contains('DISTRO_FEATURES', 'sysvinit', True, False, d) and \
 not d.getVar(INHIBIT_UPDATERCD_BBCLASS, True):
  pkgs = d.getVar('INITSCRIPT_PACKAGES', True)
  if pkgs == None:
 -- 
 1.8.1.2
 
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 3/6] rootfs.py: Check for LDCONFIGDEPEND being empty string

2014-02-18 Thread Laurentiu Palcu
Hi Khem,

On Mon, Feb 17, 2014 at 11:34:15AM -0800, Khem Raj wrote:
 We override LDCONFIGDEPEND to be empty string for uclibc
 however the current check is for it being None as a result
 the function is still executed but ldconfig-native is not
 built as dependency for rootfs when building with uclibc
 
 Fixes errors like below
 
 File:
 '/home/kraj/work/angstrom-repo/sources/openembedded-core/meta/lib/oe/rootfs.py',
 lineno: 191, function: _run_ldconfig
  0187:def _run_ldconfig(self):
  0188:if self.d.getVar('LDCONFIGDEPEND', True) is not None:
  0189:bb.note(Executing: ldconfig -r +
 self.image_rootfs + -c new -v)
  0190:self._exec_shell_cmd(['ldconfig', '-r',
 self.image_rootfs, '-c',
  *** 0191:  'new', '-v'])
 
 Signed-off-by: Khem Raj raj.k...@gmail.com
 ---
  meta/lib/oe/rootfs.py |2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
 index d149ca3..3bcb812 100644
 --- a/meta/lib/oe/rootfs.py
 +++ b/meta/lib/oe/rootfs.py
 @@ -185,7 +185,7 @@ class Rootfs(object):
  self._handle_intercept_failure(registered_pkgs)
  
  def _run_ldconfig(self):
 -if self.d.getVar('LDCONFIGDEPEND', True) is not None:
 +if self.d.getVar('LDCONFIGDEPEND', True) != :
May I suggest:
   if (self.d.getVar('LDCONFIGDEPEND', True) or ) != :
here? Otherwise, the condition will be true even if LDCONFIGDEPEND is
None.

laurentiu
  bb.note(Executing: ldconfig -r + self.image_rootfs + -c new 
 -v)
  self._exec_shell_cmd(['ldconfig', '-r', self.image_rootfs, '-c',
'new', '-v'])
 -- 
 1.7.10.4
 
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/2] lib/oe/image.py: add image dependency mechanism

2014-02-18 Thread Laurentiu Palcu
This commit adds a dependency mechanism to image creation, so that we can
split the images creation execution in groups, that can be executed in
parallel, having the dependencies satisfied in the same time. The old
code didn't need this since everything was serialized.

Technically, it adds a dependency graph topological sort class that the
main Image class can use to sort out the dependencies.

Images that have dependencies have to declare them using the NEW
IMAGE_TYPEDEP variable, like in the example below:

For:
IMAGE_FSTYPES = i1 i2 i3 i4 i5
IMAGE_TYPEDEP_i4 = i2
IMAGE_TYPEDEP_i5 = i6 i4
IMAGE_TYPEDEP_i6 = i7
IMAGE_TYPEDEP_i7 = i2

We'll get the following image groups, sorted out by their dependencies:

[['i1', 'i3', 'i2'], ['i4', 'i7'], ['i6'], ['i5']]

The algorithm can probably be optimized but, given the small size of the
graphs, it'll do.

[YOCTO #5830]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/image.py |  282 ++
 1 file changed, 191 insertions(+), 91 deletions(-)

diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
index c15296f..488683e 100644
--- a/meta/lib/oe/image.py
+++ b/meta/lib/oe/image.py
@@ -19,9 +19,124 @@ def generate_image(arg):
 return None
 
 
-class Image(object):
+
+This class will help compute IMAGE_FSTYPE dependencies and group them in 
batches
+that can be executed in parallel.
+
+The next example is for illustration purposes, highly unlikely to happen in 
real life.
+It's just one of the test cases I used to test the algorithm:
+
+For:
+IMAGE_FSTYPES = i1 i2 i3 i4 i5
+IMAGE_TYPEDEP_i4 = i2
+IMAGE_TYPEDEP_i5 = i6 i4
+IMAGE_TYPEDEP_i6 = i7
+IMAGE_TYPEDEP_i7 = i2
+
+We get the following list of batches that can be executed in parallel, having 
the
+dependencies satisfied:
+
+[['i1', 'i3', 'i2'], ['i4', 'i7'], ['i6'], ['i5']]
+
+class ImageDepGraph(object):
 def __init__(self, d):
 self.d = d
+self.graph = dict()
+self.deps_array = dict()
+
+def _construct_dep_graph(self, image_fstypes):
+graph = dict()
+
+def add_node(node):
+deps = (self.d.getVar('IMAGE_TYPEDEP_' + node, True) or )
+if deps != :
+graph[node] = deps
+
+for dep in deps.split():
+if not dep in graph:
+add_node(dep)
+else:
+graph[node] = 
+
+for fstype in image_fstypes:
+add_node(fstype)
+
+return graph
+
+def _clean_graph(self):
+# Live and VMDK images will be processed via inheriting
+# bbclass and does not get processed here. Remove them from the fstypes
+# graph. Their dependencies are already added, so no worries here.
+remove_list = (self.d.getVar('IMAGE_TYPES_MASKED', True) or ).split()
+
+for item in remove_list:
+self.graph.pop(item, None)
+
+def _compute_dependencies(self):
+
+returns dict object of nodes with [no_of_depends_on, no_of_depended_by]
+for each node
+
+deps_array = dict()
+for node in self.graph:
+deps_array[node] = [0, 0]
+
+for node in self.graph:
+deps = self.graph[node].split()
+deps_array[node][0] += len(deps)
+for dep in deps:
+deps_array[dep][1] += 1
+
+return deps_array
+
+def _sort_graph(self):
+sorted_list = []
+group = []
+for node in self.graph:
+if node not in self.deps_array:
+continue
+
+depends_on = self.deps_array[node][0]
+
+if depends_on == 0:
+group.append(node)
+
+if len(group) == 0 and len(self.deps_array) != 0:
+bb.fatal(possible fstype circular dependency...)
+
+sorted_list.append(group)
+
+# remove added nodes from deps_array
+for item in group:
+for node in self.graph:
+if item in self.graph[node]:
+self.deps_array[node][0] -= 1
+
+self.deps_array.pop(item, None)
+
+if len(self.deps_array):
+# recursive call, to find the next group
+sorted_list += self._sort_graph()
+
+return sorted_list
+
+def group_fstypes(self, image_fstypes):
+self.graph = self._construct_dep_graph(image_fstypes)
+
+self._clean_graph()
+
+self.deps_array = self._compute_dependencies()
+
+alltypes = [node for node in self.graph]
+
+return (alltypes, self._sort_graph())
+
+
+class Image(ImageDepGraph):
+def __init__(self, d):
+self.d = d
+
+super(Image, self).__init__(d)
 
 def _get_rootfs_size(self):
 compute the rootfs size
@@ -82,66 +197,44 @@ class Image(object):
 
 os.remove(img)
 
+
+This function will just filter out the compressed image types from the
+fstype groups

[OE-core] [PATCH 0/2] Add image creation dependency mechanism

2014-02-18 Thread Laurentiu Palcu
The image creation is done now in parallel. However, we didn't have any
dependency handling in place since the old code did that serial. This patchset
adds such dependency mechanism and all images types are split in groups,
according to dependency, which can be done in parallel.

The users, however, have to declare the image dependency using the 
IMAGE_TYPEDEP variable:

Example:
IMAGE_TYPEDEP_rpi-sdimg = ext3

laurentiu

The following changes since commit 54562006c1327c5b99daa4cc05a3ba7e38412da1:

  image_types.bbclass: Fix tar IMAGE_CMD to not change directories (2014-02-18 
08:38:52 +)

are available in the git repository at:

  git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib 
lpalcu/image_deps_fix

for you to fetch changes up to e7feddd1dafc5eb02b80b8d43aca504b17151e5c:

  lib/oe/image.py: add image dependency mechanism (2014-02-18 15:16:21 +0200)


Laurentiu Palcu (2):
  image*.bbclass, bootimg.bbclass: add image type dependencies
  lib/oe/image.py: add image dependency mechanism

 meta/classes/bootimg.bbclass |4 +
 meta/classes/image-live.bbclass  |3 +
 meta/classes/image-vmdk.bbclass  |3 +
 meta/classes/image_types.bbclass |5 +
 meta/lib/oe/image.py |  282 ++
 5 files changed, 206 insertions(+), 91 deletions(-)

Laurentiu Palcu (2):
  image*.bbclass, bootimg.bbclass: add image type dependencies
  lib/oe/image.py: add image dependency mechanism

 meta/classes/bootimg.bbclass |4 +
 meta/classes/image-live.bbclass  |3 +
 meta/classes/image-vmdk.bbclass  |3 +
 meta/classes/image_types.bbclass |5 +
 meta/lib/oe/image.py |  282 ++
 5 files changed, 206 insertions(+), 91 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/2] image*.bbclass, bootimg.bbclass: add image type dependencies

2014-02-18 Thread Laurentiu Palcu
The following dependencies were manually added in the image creation
code. However, in order to have an image dependency mechanism in place,
use a new variable, IMAGE_TYPEDEP, to declare that an image type depends
on another being already created.

The following dependencies are added by this commit:

elf - cpio.gz
live - ext3
vmdk - ext3
iso - ext3
hddimg - ext3

This commit adds also another new variable: IMAGE_TYPES_MASKED. Currently,
masking out certain types from IMAGE_FSTYPES was hardcoded in the image
creation code.

[YOCTO #5830]

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/classes/bootimg.bbclass |4 
 meta/classes/image-live.bbclass  |3 +++
 meta/classes/image-vmdk.bbclass  |3 +++
 meta/classes/image_types.bbclass |5 +
 4 files changed, 15 insertions(+)

diff --git a/meta/classes/bootimg.bbclass b/meta/classes/bootimg.bbclass
index 395085d..c370bd6 100644
--- a/meta/classes/bootimg.bbclass
+++ b/meta/classes/bootimg.bbclass
@@ -232,4 +232,8 @@ python do_bootimg() {
 bb.build.exec_func('build_iso', d)
 }
 
+IMAGE_TYPEDEP_iso = ext3
+IMAGE_TYPEDEP_hddimg = ext3
+IMAGE_TYPES_MASKED += iso hddimg
+
 addtask bootimg before do_build
diff --git a/meta/classes/image-live.bbclass b/meta/classes/image-live.bbclass
index bfb59f8..c7e6937 100644
--- a/meta/classes/image-live.bbclass
+++ b/meta/classes/image-live.bbclass
@@ -13,3 +13,6 @@ do_bootimg[depends] += ${INITRD_IMAGE}:do_rootfs
 do_bootimg[depends] += ${PN}:do_rootfs
 
 inherit bootimg
+
+IMAGE_TYPEDEP_live = ext3
+IMAGE_TYPES_MASKED += live
diff --git a/meta/classes/image-vmdk.bbclass b/meta/classes/image-vmdk.bbclass
index 6983e5c..6a98f17 100644
--- a/meta/classes/image-vmdk.bbclass
+++ b/meta/classes/image-vmdk.bbclass
@@ -16,6 +16,9 @@ ROOTFS ?= 
${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext3
 #inherit image-live
 inherit boot-directdisk
 
+IMAGE_TYPEDEP_vmdk = ext3
+IMAGE_TYPES_MASKED += vmdk
+
 create_vmdk_image () {
qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hdddirect 
${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk
ln -sf ${IMAGE_NAME}.vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.vmdk
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 944e6db..200e2e7 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -84,6 +84,7 @@ IMAGE_CMD_elf () {
test -f ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.elf  rm -f 
${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.elf
mkelfImage --kernel=${ELF_KERNEL} 
--initrd=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio.gz 
--output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.elf --append='${ELF_APPEND}' 
${EXTRA_IMAGECMD}
 }
+IMAGE_TYPEDEP_elf = cpio.gz
 
 UBI_VOLNAME ?= ${MACHINE}-rootfs
 
@@ -150,3 +151,7 @@ DEPLOYABLE_IMAGE_TYPES ?= hddimg iso
 
 # Use IMAGE_EXTENSION_xxx to map image type 'xxx' with real image file 
extension name(s) for Hob
 IMAGE_EXTENSION_live = hddimg iso
+
+# The IMAGE_TYPES_MASKED variable will be used to mask out from the 
IMAGE_FSTYPES,
+# images that will not be built at do_rootfs time: vmdk, hddimg, iso, etc.
+IMAGE_TYPES_MASKED ?= 
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/5] package_manager.py: support ipk incremental image generation

2014-02-18 Thread Laurentiu Palcu
On Tue, Feb 18, 2014 at 05:42:24PM +0800, Hongxu Jia wrote:
 While incremental image generation enabled, 'load_old_install_solution'
 is used to determine what we've got in the previous (existed) image and
 'dump_install_solution' to determine what we need to install in the
 current image.
 
 The 'backup_packaging_data' is used to back up the current opkg database.
 
 The 'recovery_packaging_data' is used to recover the opkg database which
 backed up by the previous image creation.
 
 Tweak 'remove' function in OpkgPM class, which the options for remove
 with dependencies was incorrect.
 
 [YOCTO #1894]
 
 Signed-off-by: Hongxu Jia hongxu@windriver.com
 ---
  meta/lib/oe/package_manager.py | 106 
 +++--
  1 file changed, 102 insertions(+), 4 deletions(-)
 
 diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
 index 6dc8fbd..4ea9677 100644
 --- a/meta/lib/oe/package_manager.py
 +++ b/meta/lib/oe/package_manager.py
 @@ -934,12 +934,13 @@ class RpmPM(PackageManager):
  
  
  class OpkgPM(PackageManager):
 -def __init__(self, d, target_rootfs, config_file, archs):
 +def __init__(self, d, target_rootfs, config_file, archs, 
 task_name='target'):
  super(OpkgPM, self).__init__(d)
  
  self.target_rootfs = target_rootfs
  self.config_file = config_file
  self.pkg_archs = archs
 +self.task_name = task_name
  
  self.deploy_dir = self.d.getVar(DEPLOY_DIR_IPK, True)
  self.deploy_lock_file = os.path.join(self.deploy_dir, deploy.lock)
 @@ -956,6 +957,13 @@ class OpkgPM(PackageManager):
  
  bb.utils.mkdirhier(self.opkg_dir)
  
 +self.solution_manifest = self.d.expand('${T}/saved/%s_solution' %
 +   self.task_name)
 +self.saved_opkg_dir = self.d.expand('${T}/saved/%s' % self.task_name)
 +if not os.path.exists(self.d.expand('${T}/saved')):
 +bb.utils.mkdirhier(self.d.expand('${T}/saved'))
 +
 +
  if (self.d.getVar('BUILD_IMAGES_FROM_FEEDS', True) or ) != 1:
  self._create_config()
  else:
 @@ -1075,7 +1083,9 @@ class OpkgPM(PackageManager):
  
  try:
  bb.note(Installing the following packages: %s % ' '.join(pkgs))
 -subprocess.check_output(cmd.split())
 +bb.note(cmd)
 +output = subprocess.check_output(cmd.split())
 +bb.note(output)
  except subprocess.CalledProcessError as e:
  (bb.fatal, bb.note)[attempt_only](Unable to install packages. 
Command '%s' returned 
 %d:\n%s %
 @@ -1083,14 +1093,16 @@ class OpkgPM(PackageManager):
  
  def remove(self, pkgs, with_dependencies=True):
  if with_dependencies:
 -cmd = %s %s remove %s % \
 +cmd = %s %s --force-depends --force-remove 
 --force-removal-of-dependent-packages remove %s % \
  (self.opkg_cmd, self.opkg_args, ' '.join(pkgs))
  else:
  cmd = %s %s --force-depends remove %s % \
  (self.opkg_cmd, self.opkg_args, ' '.join(pkgs))
  
  try:
 -subprocess.check_output(cmd.split())
 +bb.note(cmd)
 +output = subprocess.check_output(cmd.split())
 +bb.note(output)
  except subprocess.CalledProcessError as e:
  bb.fatal(Unable to remove packages. Command '%s' 
   returned %d:\n%s % (e.cmd, e.returncode, e.output))
 @@ -1175,6 +1187,92 @@ class OpkgPM(PackageManager):
  else:
  status.write(line + \n)
  
 +'''
 +If incremental install, we need to determine what we've got,
 +what we need to add, and what to remove...
 +The dump_install_solution will dump and save the new install
 +solution.
 +'''
 +def dump_install_solution(self, pkgs):
Why not have this function in the Manifest class? We have an API in
place: Manifest.create_final() that should probably take care of
anything related to manifest(s) creation after all packages are installed.

 +bb.note('creating new install solution for incremental install')
 +if len(pkgs) == 0:
 +return
 +
 +install_pkgs = list()
 +
 +# Create an temp dir as opkg root for simulating the installation
 +temp_rootfs = self.d.expand('${T}/opkg')
 +temp_opkg_dir = os.path.join(temp_rootfs, 'var/lib/opkg')
 +bb.utils.mkdirhier(temp_opkg_dir)
 +
 +opkg_args = -f %s -o %s  % (self.config_file, temp_rootfs)
 +opkg_args += self.d.getVar(OPKG_ARGS, True)
 +
 +cmd = %s %s update % (self.opkg_cmd,
 +opkg_args)
 +try:
 +subprocess.check_output(cmd, shell=True)
 +except subprocess.CalledProcessError as e:
 +bb.note(Unable to dump install packages. Command '%s' 
 +  

Re: [OE-core] [PATCH 4/5] rootfs.py: fix BAD_RECOMMENDATIONS for ipk incremental image generation

2014-02-18 Thread Laurentiu Palcu
On Tue, Feb 18, 2014 at 05:42:27PM +0800, Hongxu Jia wrote:
 While incremental image generation enabled and the previous image existed,
 if BAD_RECOMMENDATIONS is changed, the operation on the existed image is
 complicated, so remove the existed image in this situation.
 
 The same with PACKAGE_EXCLUDE and NO_RECOMMENDATIONS.
 
 [YOCTO #1894]
 Signed-off-by: Hongxu Jia hongxu@windriver.com
 ---
  meta/lib/oe/rootfs.py | 30 +-
  1 file changed, 29 insertions(+), 1 deletion(-)
 
 diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
 index 5561bb9..3d7adf9 100644
 --- a/meta/lib/oe/rootfs.py
 +++ b/meta/lib/oe/rootfs.py
 @@ -443,7 +443,7 @@ class OpkgRootfs(Rootfs):
  self.pkg_archs = self.d.getVar(ALL_MULTILIB_PACKAGE_ARCHS, True)
  
  self.inc_opkg_image_gen = self.d.getVar('INC_IPK_IMAGE_GEN', True)
 -if self.inc_opkg_image_gen != 1:
 +if self._remove_existed_image():
  bb.utils.remove(self.image_rootfs, True)
  self.pm = OpkgPM(d,
   self.image_rootfs,
 @@ -546,6 +546,34 @@ class OpkgRootfs(Rootfs):
  bb.note('incremental removed: %s' % ' '.join(pkg_to_remove))
  self.pm.remove(pkg_to_remove)
  
 +'''
 +Compare with previous (existed) image creation, if some conditions
 +triggered, the previous (existed) image should be removed.
 +The conditions included any of 'PACKAGE_EXCLUDE, NO_RECOMMENDATIONS
 +and BAD_RECOMMENDATIONS' has been changed.
 +'''
 +def _remove_existed_image(self):
s/existed/existing/

Also in the comment above.

 +# Incremental image creation is not enable
 +if self.inc_opkg_image_gen != 1:
 +return True
 +
 +vars_list_dir = self.d.expand('${T}/vars_list')
s/vars_list_dir/vars_list_file/ in the line above and the code below.
Leaving this variable with a 'dir' suffix while we're opening a file for
reading/writing, is a little misleading.

 +
 +old_vars_list = 
 +if os.path.exists(vars_list_dir):
 +old_vars_list = open(vars_list_dir, 'r+').read()
 +
 +new_vars_list = '%s:%s:%s\n' % \
 +((self.d.getVar('BAD_RECOMMENDATIONS', True) or '').strip(),
 + (self.d.getVar('NO_RECOMMENDATIONS', True) or '').strip(),
 + (self.d.getVar('PACKAGE_EXCLUDE', True) or '').strip())
 +open(vars_list_dir, 'w+').write(new_vars_list)
 +
 +if old_vars_list != new_vars_list:
 +return True
 +
 +return False
 +
  def _create(self):
  pkgs_to_install = self.manifest.parse_initial_manifest()
  opkg_pre_process_cmds = self.d.getVar('OPKG_PREPROCESS_COMMANDS', 
 True)
 -- 
 1.8.1.2
 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 5/5] rootfs.py: tweak _multilib_sanity_test for ipk incremental image generation

2014-02-18 Thread Laurentiu Palcu
On Tue, Feb 18, 2014 at 05:42:28PM +0800, Hongxu Jia wrote:
 The _multilib_sanity_test installs multilib packages in a temporary
 root fs, and compare with the current image to figure out duplicated
 file that comes from different packages.
 
 While incremental image generation enabled and the previous image
 existed, there was a Multilib check error:
 ...
 ERROR: Multilib check error: duplicate files tmp/work/qemux86_64-poky-
 linux/core-image-minimal/1.0-r0/multilib/lib32/lib/libc.so.6 tmp/work/
 qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/lib/libc.so.6
 is not the same
 ...
 
 The reason is the file in current image has been prelinked in previous
 image generation and the file in a temporary root fs is not prelinked,
 even though the files come from the same package, the Multilib check
 considers they are different.
 
 [YOCTO #1894]
 Signed-off-by: Hongxu Jia hongxu@windriver.com
 ---
  meta/lib/oe/rootfs.py | 56 
 +++
  1 file changed, 52 insertions(+), 4 deletions(-)
 
 diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
 index 3d7adf9..5054d1e 100644
 --- a/meta/lib/oe/rootfs.py
 +++ b/meta/lib/oe/rootfs.py
 @@ -3,6 +3,8 @@ from oe.utils import execute_pre_post_process
  from oe.utils import contains as base_contains
  from oe.package_manager import *
  from oe.manifest import *
 +import oe.path
 +import filecmp
  import shutil
  import os
  import subprocess
 @@ -458,13 +460,61 @@ class OpkgRootfs(Rootfs):
  
  bb.utils.remove(self.d.getVar('MULTILIB_TEMP_ROOTFS', True), True)
  
 +def _prelink_file(self, root_dir, filename):
 +bb.note('prelink %s in %s' % (filename, root_dir))
 +prelink_cfg = oe.path.join(root_dir,
 +   
 self.d.expand('${sysconfdir}/prelink.conf'))
 +if not os.path.exists(prelink_cfg):
 +
 shutil.copy(self.d.expand('${STAGING_DIR_NATIVE}${sysconfdir_native}/prelink.conf'),
 +prelink_cfg)
 +
 +cmd_prelink = 
 self.d.expand('${STAGING_DIR_NATIVE}${sbindir_native}/prelink')
 +self._exec_shell_cmd([cmd_prelink,
 +  '--root',
 +  root_dir,
 +  '-amR',
 +  '-N',
 +  '-c',
 +  self.d.expand('${sysconfdir}/prelink.conf')])
 +
 +'''
 +Compare two files with the same key twice to see if they came
 +from the same package. If they are not same, they are duplicated
 +and come from different packages.
I'm kind of confused by this comment. Doesn't same = duplicate? There
might be a small confusion of terms here because the function's behavior
is not as the name implies.

 +1st: Comapre them directly;
 +2nd: While incremental image creation is enabled, one of the
 + files could be probaly prelinked in the previous image
 + creation and the file has been changed, so we need to
 + prelink the other one and compare them.
 +'''
 +def _file_duplicate(self, key, f1, f2):
Shouldn't be better to rename this function to something else in order
to avoid confusion? Let's say: _files_are_equal() ?

 +
 +if not os.path.exists(f1) or not os.path.exists(f2):
 +return False
 +
 +# f1 is the same with f2, both of them were not prelinked
 +if filecmp.cmp(f1, f2):
 +return False
filecmp.cmp() returns True if files are equal. Hence the confusion:
_file_duplicate() returns False here, if files are equal... I think the logic
is a little bit the other way around! :)

 +
 +if self.image_rootfs not in f1:
 +self._prelink_file(f1.replace(key, ''), f1)
 +
 +if self.image_rootfs not in f2:
 +self._prelink_file(f2.replace(key, ''), f2)
 +
 +# f1 is the same with f2, both of them were prelinked
 +if filecmp.cmp(f1, f2):
 +return False
 +
 +# Duplicated
 +return True
here the return value matches the comment and the function name! :)
 +
  
  This function was reused from the old implementation.
  See commit: image.bbclass: Added variables for multilib support. by
  Lianhao Lu.
  
  def _multilib_sanity_test(self, dirs):
 -import filecmp
  
  allow_replace = self.d.getVar(MULTILIBRE_ALLOW_REP, True)
  if allow_replace is None:
 @@ -486,9 +536,7 @@ class OpkgRootfs(Rootfs):
  if allow_rep.match(key):
  valid = True
  else:
 -if os.path.exists(files[key]) and \
 -   os.path.exists(item) and \
 -   not filecmp.cmp(files[key], item):
 +if self._file_duplicate(key, files[key], item):
  valid = False
  

Re: [OE-core] [PATCH 3/7] lsbinitscripts: Update to 9.52

2014-02-17 Thread Laurentiu Palcu
Hi Saul,

On Mon, Feb 17, 2014 at 12:22:58AM -0800, Saul Wold wrote:
 Signed-off-by: Saul Wold s...@linux.intel.com
 ---
  meta/recipes-extended/lsb/lsbinitscripts_9.52.bb | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/meta/recipes-extended/lsb/lsbinitscripts_9.52.bb 
 b/meta/recipes-extended/lsb/lsbinitscripts_9.52.bb
 index 11deae2..bc3e620 100644
 --- a/meta/recipes-extended/lsb/lsbinitscripts_9.52.bb
 +++ b/meta/recipes-extended/lsb/lsbinitscripts_9.52.bb
 @@ -6,7 +6,7 @@ DEPENDS = popt glib-2.0
  LIC_FILES_CHKSUM = file://COPYING;md5=ebf4e8b49780ab187d51bd26aaa022c6
  
  S=${WORKDIR}/initscripts-${PV}
 -SRC_URI = 
 http://pkgs.fedoraproject.org/repo/pkgs/initscripts/initscripts-${PV}.tar.bz2/2453811ec27a781a77f309f356663d9e/initscripts-${PV}.tar.bz2
  \
 +SRC_URI = 
 http://pkgs.fedoraproject.org/repo/pkgs/initscripts/initscripts-9.52.tar.bz2/2453811ec27a781a77f309f356663d9e/initscripts-9.52.tar.bz2
  \
This doesn't look like an upgrade... Besides, I believe you already
upgraded this recipe to 9.52:

Author: Saul Wold s...@linux.intel.com
Date:   Mon Jan 20 18:54:30 2014 -0800

lsbinitscripts: Update to 9.52

Am I missing something?

laurentiu


 file://functions.patch \
 
  
 -- 
 1.8.3.1
 
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 2/3] package-index.bb: use the new python indexing routines

2014-02-14 Thread Laurentiu Palcu
Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/recipes-core/meta/package-index.bb |7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-core/meta/package-index.bb 
b/meta/recipes-core/meta/package-index.bb
index 3cc1239..27b6d8e 100644
--- a/meta/recipes-core/meta/package-index.bb
+++ b/meta/recipes-core/meta/package-index.bb
@@ -21,10 +21,9 @@ do_populate_sysroot[noexec] = 1
 do_package_index[nostamp] = 1
 do_package_index[depends] += ${PACKAGEINDEXDEPS}
 
-do_package_index() {
-   set -ex
-   ${PACKAGEINDEXES}
-   set +ex
+python do_package_index() {
+from oe.rootfs import generate_index_files
+generate_index_files(d)
 }
 addtask do_package_index before do_build
 EXCLUDE_FROM_WORLD = 1
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 3/3] package_*.bbclass: remove references to the old bash indexing routines

2014-02-14 Thread Laurentiu Palcu
Package indexing is done in python and package-index.bb uses the new
routines.

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/classes/package_deb.bbclass |1 -
 meta/classes/package_ipk.bbclass |1 -
 meta/classes/package_rpm.bbclass |1 -
 3 files changed, 3 deletions(-)

diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
index a0a31bd..b9d7978 100644
--- a/meta/classes/package_deb.bbclass
+++ b/meta/classes/package_deb.bbclass
@@ -309,6 +309,5 @@ do_package_write_deb[umask] = 022
 addtask package_write_deb before do_package_write after do_packagedata 
do_package
 
 
-PACKAGEINDEXES += [ ! -e ${DEPLOY_DIR_DEB} ] || package_update_index_deb;
 PACKAGEINDEXDEPS += dpkg-native:do_populate_sysroot
 PACKAGEINDEXDEPS += apt-native:do_populate_sysroot
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 516ec4b..7cf2c8a 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -255,6 +255,5 @@ do_package_write_ipk[cleandirs] = ${PKGWRITEDIRIPK}
 do_package_write_ipk[umask] = 022
 addtask package_write_ipk before do_package_write after do_packagedata 
do_package
 
-PACKAGEINDEXES += [ ! -e ${DEPLOY_DIR_IPK} ] || package_update_index_ipk;
 PACKAGEINDEXDEPS += opkg-utils-native:do_populate_sysroot
 PACKAGEINDEXDEPS += opkg-native:do_populate_sysroot
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 82ea187..f267b8a 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -734,6 +734,5 @@ do_package_write_rpm[cleandirs] = ${PKGWRITEDIRRPM}
 do_package_write_rpm[umask] = 022
 addtask package_write_rpm before do_package_write after do_packagedata 
do_package
 
-PACKAGEINDEXES += [ ! -e ${DEPLOY_DIR_RPM} ] || package_update_index_rpm;
 PACKAGEINDEXDEPS += rpm-native:do_populate_sysroot
 PACKAGEINDEXDEPS += createrepo-native:do_populate_sysroot
-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 0/3] fix package-index build issue

2014-02-14 Thread Laurentiu Palcu
After rootfs code refactoring, 'bitbake package-index' fails because it's
calling the old bash indexing routines. This patchset should fix that.

laurentiu

The following changes since commit b60ed2d0fd86a3ae5ce7facbb044677aa7ec2889:

  bitbake: runqueue: Ensure we do run 'masked' setscene tasks if specified as 
targets (2014-02-13 17:57:20 +)

are available in the git repository at:

  git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib 
lpalcu/package_index_recipe_fix

for you to fetch changes up to db1e6488ef483ab76f7eca961b214bd8f8758b2e:

  package_*.bbclass: remove references to the old bash indexing routines 
(2014-02-14 12:47:53 +0200)


Laurentiu Palcu (3):
  package_manager.py, rootfs.py, sdk.py: add Indexer class
  package-index.bb: use the new python indexing routines
  package_*.bbclass: remove references to the old bash indexing routines

 meta/classes/package_deb.bbclass|1 -
 meta/classes/package_ipk.bbclass|1 -
 meta/classes/package_rpm.bbclass|1 -
 meta/lib/oe/package_manager.py  |  339 ---
 meta/lib/oe/rootfs.py   |   58 --
 meta/lib/oe/sdk.py  |   48 +
 meta/recipes-core/meta/package-index.bb |7 +-
 7 files changed, 224 insertions(+), 231 deletions(-)

Laurentiu Palcu (3):
  package_manager.py, rootfs.py, sdk.py: add Indexer class
  package-index.bb: use the new python indexing routines
  package_*.bbclass: remove references to the old bash indexing
routines

 meta/classes/package_deb.bbclass|1 -
 meta/classes/package_ipk.bbclass|1 -
 meta/classes/package_rpm.bbclass|1 -
 meta/lib/oe/package_manager.py  |  339 ---
 meta/lib/oe/rootfs.py   |   58 --
 meta/lib/oe/sdk.py  |   48 +
 meta/recipes-core/meta/package-index.bb |7 +-
 7 files changed, 224 insertions(+), 231 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/3] package_manager.py, rootfs.py, sdk.py: add Indexer class

2014-02-14 Thread Laurentiu Palcu
Because the package-index.bb needs to create package indexes outside
do_rootfs environment, move the indexing capability out of
PackageManager class to a smaller Indexer class.

This commit:
 * simply moves the indexing functions for ipk/deb with no changes;
 * rewrites the RPM indexing function so that it can be easily moved out
   of the PackageManager class;
 * removes some RPM duplicate code, moves it into a method inside
   RpmPM class and changes the RpmPM constructor so that the new method
   is effective;

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/package_manager.py |  339 +---
 meta/lib/oe/rootfs.py  |   58 ---
 meta/lib/oe/sdk.py |   48 +-
 3 files changed, 221 insertions(+), 224 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 9884c3a..af14d5a 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -22,6 +22,144 @@ def create_index(arg):
 return None
 
 
+class Indexer(object):
+__metaclass__ = ABCMeta
+
+def __init__(self, d, deploy_dir):
+self.d = d
+self.deploy_dir = deploy_dir
+
+@abstractmethod
+def write_index(self):
+pass
+
+
+class RpmIndexer(Indexer):
+def write_index(self):
+sdk_pkg_archs = (self.d.getVar('SDK_PACKAGE_ARCHS', True) or 
).replace('-', '_').split()
+mlb_prefix_list = (self.d.getVar('MULTILIB_PREFIX_LIST', True) or 
).replace('-', '_').split()
+all_mlb_pkg_archs = (self.d.getVar('ALL_MULTILIB_PACKAGE_ARCHS', True) 
or ).replace('-', '_').split()
+
+archs = set()
+for item in mlb_prefix_list:
+archs = archs.union(set(item.split(':')[1:]))
+
+if len(archs) == 0:
+archs = archs.union(set(all_mlb_pkg_archs))
+
+archs = archs.union(set(sdk_pkg_archs))
+
+rpm_createrepo = bb.utils.which(os.getenv('PATH'), createrepo)
+index_cmds = []
+rpm_dirs_found = False
+for arch in archs:
+arch_dir = os.path.join(self.deploy_dir, arch)
+if not os.path.isdir(arch_dir):
+continue
+
+index_cmds.append(%s --update -q %s % (rpm_createrepo, arch_dir))
+
+rpm_dirs_found = True
+
+if not rpm_dirs_found:
+return(There are no packages in %s % self.deploy_dir)
+
+nproc = multiprocessing.cpu_count()
+pool = bb.utils.multiprocessingpool(nproc)
+results = list(pool.imap(create_index, index_cmds))
+pool.close()
+pool.join()
+
+for result in results:
+if result is not None:
+return(result)
+
+
+class OpkgIndexer(Indexer):
+def write_index(self):
+arch_vars = [ALL_MULTILIB_PACKAGE_ARCHS,
+ SDK_PACKAGE_ARCHS,
+ MULTILIB_ARCHS]
+
+opkg_index_cmd = bb.utils.which(os.getenv('PATH'), opkg-make-index)
+
+if not os.path.exists(os.path.join(self.deploy_dir, Packages)):
+open(os.path.join(self.deploy_dir, Packages), w).close()
+
+index_cmds = []
+for arch_var in arch_vars:
+archs = self.d.getVar(arch_var, True)
+if archs is None:
+continue
+
+for arch in archs.split():
+pkgs_dir = os.path.join(self.deploy_dir, arch)
+pkgs_file = os.path.join(pkgs_dir, Packages)
+
+if not os.path.isdir(pkgs_dir):
+continue
+
+if not os.path.exists(pkgs_file):
+open(pkgs_file, w).close()
+
+index_cmds.append('%s -r %s -p %s -m %s' %
+  (opkg_index_cmd, pkgs_file, pkgs_file, 
pkgs_dir))
+
+if len(index_cmds) == 0:
+return(There are no packages in %s! % self.deploy_dir)
+
+nproc = multiprocessing.cpu_count()
+pool = bb.utils.multiprocessingpool(nproc)
+results = list(pool.imap(create_index, index_cmds))
+pool.close()
+pool.join()
+
+for result in results:
+if result is not None:
+return(result)
+
+
+class DpkgIndexer(Indexer):
+def write_index(self):
+pkg_archs = self.d.getVar('PACKAGE_ARCHS', True)
+if pkg_archs is not None:
+arch_list = pkg_archs.split()
+sdk_pkg_archs = self.d.getVar('SDK_PACKAGE_ARCHS', True)
+if sdk_pkg_archs is not None:
+arch_list += sdk_pkg_archs.split()
+
+dpkg_scanpackages = bb.utils.which(os.getenv('PATH'), 
dpkg-scanpackages)
+gzip = bb.utils.which(os.getenv('PATH'), gzip)
+
+index_cmds = []
+deb_dirs_found = False
+for arch in arch_list:
+arch_dir = os.path.join(self.deploy_dir, arch)
+if not os.path.isdir(arch_dir):
+continue
+
+with open(os.path.join(arch_dir, Release), w

[OE-core] [PATCH 0/1] package_manager.py: move multilib prefix list computation function to RpmIndexer

2014-02-14 Thread Laurentiu Palcu
I tested this with the following:

rm -rf tmp/  bb core-image-sato  bb -c populate_sdk core-image-sato 
bb lib32-core-image-minimal  rm -rf tmp/  bb lib32-core-image-minimal

laurentiu

The following changes since commit b38c7cb7a6ab787d7be2b2c295ba58a98a954cd5:

  socat: upgrade to 1.7.2.3 (2014-02-14 12:31:10 +)

are available in the git repository at:

  git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib 
lpalcu/rpm_ml_build_fix

for you to fetch changes up to 4e62efc1659a4be94b6437049a771ec444515600:

  package_manager.py: move multilib prefix list computation function to 
RpmIndexer (2014-02-14 20:59:17 +0200)


Laurentiu Palcu (1):
  package_manager.py: move multilib prefix list computation function to 
RpmIndexer

 meta/lib/oe/package_manager.py |  105 
 1 file changed, 53 insertions(+), 52 deletions(-)

Laurentiu Palcu (1):
  package_manager.py: move multilib prefix list computation function to
RpmIndexer

 meta/lib/oe/package_manager.py |  105 
 1 file changed, 53 insertions(+), 52 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH 1/1] package_manager.py: move multilib prefix list computation function to RpmIndexer

2014-02-14 Thread Laurentiu Palcu
Since the code from anonymous function in rootfs_rpm.bbclass has been
removed, MULTILIB_PREFIX_LIST variable was never set. Hence not all
directories got indexed.

This commit will move the multilib prefix list computation function from
RpmPM class to RpmIndexer, since the indexer needs it too. I was hoping
to avoid this but, unfortunately, I couldn't.

Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
---
 meta/lib/oe/package_manager.py |  105 
 1 file changed, 53 insertions(+), 52 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index af14d5a..2faf422 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -35,14 +35,64 @@ class Indexer(object):
 
 
 class RpmIndexer(Indexer):
+def get_ml_prefix_and_os_list(self, arch_var=None, os_var=None):
+package_archs = {
+'default': [],
+}
+
+target_os = {
+'default': ,
+}
+
+if arch_var is not None and os_var is not None:
+package_archs['default'] = self.d.getVar(arch_var, True).split()
+package_archs['default'].reverse()
+target_os['default'] = self.d.getVar(os_var, True).strip()
+else:
+package_archs['default'] = self.d.getVar(PACKAGE_ARCHS, 
True).split()
+# arch order is reversed.  This ensures the -best- match is
+# listed first!
+package_archs['default'].reverse()
+target_os['default'] = self.d.getVar(TARGET_OS, True).strip()
+multilibs = self.d.getVar('MULTILIBS', True) or 
+for ext in multilibs.split():
+eext = ext.split(':')
+if len(eext)  1 and eext[0] == 'multilib':
+localdata = bb.data.createCopy(self.d)
+default_tune_key = DEFAULTTUNE_virtclass-multilib- + 
eext[1]
+default_tune = localdata.getVar(default_tune_key, False)
+if default_tune:
+localdata.setVar(DEFAULTTUNE, default_tune)
+bb.data.update_data(localdata)
+package_archs[eext[1]] = 
localdata.getVar('PACKAGE_ARCHS',
+  True).split()
+package_archs[eext[1]].reverse()
+target_os[eext[1]] = localdata.getVar(TARGET_OS,
+  True).strip()
+
+ml_prefix_list = dict()
+for mlib in package_archs:
+if mlib == 'default':
+ml_prefix_list[mlib] = package_archs[mlib]
+else:
+ml_prefix_list[mlib] = list()
+for arch in package_archs[mlib]:
+if arch in ['all', 'noarch', 'any']:
+ml_prefix_list[mlib].append(arch)
+else:
+ml_prefix_list[mlib].append(mlib + _ + arch)
+
+return (ml_prefix_list, target_os)
+
 def write_index(self):
 sdk_pkg_archs = (self.d.getVar('SDK_PACKAGE_ARCHS', True) or 
).replace('-', '_').split()
-mlb_prefix_list = (self.d.getVar('MULTILIB_PREFIX_LIST', True) or 
).replace('-', '_').split()
 all_mlb_pkg_archs = (self.d.getVar('ALL_MULTILIB_PACKAGE_ARCHS', True) 
or ).replace('-', '_').split()
 
+mlb_prefix_list = self.get_ml_prefix_and_os_list()[0]
+
 archs = set()
 for item in mlb_prefix_list:
-archs = archs.union(set(item.split(':')[1:]))
+archs = archs.union(set(i.replace('-', '_') for i in 
mlb_prefix_list[item]))
 
 if len(archs) == 0:
 archs = archs.union(set(all_mlb_pkg_archs))
@@ -303,56 +353,7 @@ class RpmPM(PackageManager):
 
 self.indexer = RpmIndexer(self.d, self.deploy_dir)
 
-self.ml_prefix_list, self.ml_os_list = 
self._get_prefix_and_os_list(arch_var, os_var)
-
-def _get_prefix_and_os_list(self, arch_var, os_var):
-package_archs = {
-'default': [],
-}
-
-target_os = {
-'default': ,
-}
-
-if arch_var is not None and os_var is not None:
-package_archs['default'] = self.d.getVar(arch_var, True).split()
-package_archs['default'].reverse()
-target_os['default'] = self.d.getVar(os_var, True).strip()
-else:
-package_archs['default'] = self.d.getVar(PACKAGE_ARCHS, 
True).split()
-# arch order is reversed.  This ensures the -best- match is
-# listed first!
-package_archs['default'].reverse()
-target_os['default'] = self.d.getVar(TARGET_OS, True).strip()
-multilibs = self.d.getVar('MULTILIBS', True) or 
-for ext in multilibs.split():
-eext = ext.split(':')
-if len(eext)  1 and eext[0] == 'multilib

Re: [OE-core] [PATCH 1/1] lib/oe/rootfs.py: fix RPM multilib issue

2014-02-13 Thread Laurentiu Palcu
Hi Mark,

On Wed, Feb 12, 2014 at 01:46:31PM -0600, Mark Hatle wrote:
 On 2/12/14, 12:54 PM, Laurentiu Palcu wrote:
 For some odd reason (at least I couldn't find an explanation to this,
 yet), if a multilib version of a package is installed after the main one
 (that is: in a different smart session), the main package binaries are
 not overwritten.
 
 For two packages with the same name, but different architectures --
 the non-ELF files must be identical (or different file names in each
 package.)
 
 The ELF files are identified by type and there is a resolution
 mechanism within RPM to determine which is the 'preferred' version.
 
 We do NOT have the resolution mechanism set, so the first package
 installed 'wins'.  So if you do two packages in two separate
 transactions, the first version installed will be kept.  If you do
 it in one transaction, then I believe it ends up being the last
 version [or maybe it's random due to sort order?]
 
 There is a Yocto Project defect to configure the RPM multilib
 settings, but so far it's just sat there, as nobody has either
 needed it -- or cared enough to implement it.
There are several rpm related multilib issues in bugzilla left hanging,
unfortunately. :|

 The reality is
 installed two packages with the same /bin, /sbin binaries is rare
 these days.. it's much more useful to install the libraries.
Apparently, the multilib test we currently have on AB is to install
lib32-connman-gnome and test the ELF class of the
/usr/bin/connman-applet binary...

 
 The code you have below may turn out to be a performance
 improvement.. (each smart/rpm transaction takes setup and cleanup
 time, so multiple transactions are slower then one larger
 transaction.)  But it's likely not fixing the actual problem you
 have.
True, this problem should be properly fixed. This patch, however, just
restores the behavior we already used in the old bash code which people
seemed to have used for a long time now.

 
 #   The default transaction color. This value is a set of bits to
 #   determine file and dependency affinity for this arch.
 #   0   uncolored (i.e. use only arch as install hint)
 #   1   Elf32 permitted
 #   2   Elf64 permitted
 #   4   MIPS reserved
 %_transaction_color 3
 
 (4 BTW is MIPS64 - n32)
 
 _transaction_color of 3 indicates we allow Elf32 and Elf64 to be installed..
 
 There is a second item:
 
 _prefer_color that is used to define which item should be installed
 when there is that ELF conflict.  The numbers above follow there as
 well.  By default I believe it sets itself to '2' when doing a
 single transaction installed, preferring Elf64.
If these parameters (_transaction_color and _prefer_color) can be easily
set before each transaction, I suppose we could alter them accordingly,
depending on the type of packages we're going to install. Doesn't seem
very complicated, in theory...

laurentiu

 
 --Mark
 
 This commit restores the functionality to the original one, before
 migrating to python: feed all the packages to smart, apart from attempt
 only ones which are installed separately.
 
 Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
 ---
   meta/lib/oe/rootfs.py |   16 
   1 file changed, 12 insertions(+), 4 deletions(-)
 
 diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
 index b6baf77..9162e52 100644
 --- a/meta/lib/oe/rootfs.py
 +++ b/meta/lib/oe/rootfs.py
 @@ -317,10 +317,18 @@ class RpmRootfs(Rootfs):
 
   self.pm.update()
 
 -for pkg_type in self.install_order:
 -if pkg_type in pkgs_to_install:
 -self.pm.install(pkgs_to_install[pkg_type],
 -[False, True][pkg_type == aop])
 +pkgs = []
 +pkgs_attempt = []
 +for pkg_type in pkgs_to_install:
 +if pkg_type == Manifest.PKG_TYPE_ATTEMPT_ONLY:
 +pkgs_attempt = pkgs_to_install[pkg_type]
 +continue
 +
 +pkgs += pkgs_to_install[pkg_type]
 +
 +self.pm.install(pkgs)
 +
 +self.pm.install(pkgs_attempt, True)
 
   self.pm.install_complementary()
 
 
 
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


Re: [OE-core] [PATCH 1/1] lib/oe/rootfs.py: fix RPM multilib issue

2014-02-13 Thread Laurentiu Palcu
On Thu, Feb 13, 2014 at 11:32:36AM +, Richard Purdie wrote:
 On Wed, 2014-02-12 at 20:54 +0200, Laurentiu Palcu wrote:
  For some odd reason (at least I couldn't find an explanation to this,
  yet), if a multilib version of a package is installed after the main one
  (that is: in a different smart session), the main package binaries are
  not overwritten.
  
  This commit restores the functionality to the original one, before
  migrating to python: feed all the packages to smart, apart from attempt
  only ones which are installed separately.
  
  Signed-off-by: Laurentiu Palcu laurentiu.pa...@intel.com
  ---
   meta/lib/oe/rootfs.py |   16 
   1 file changed, 12 insertions(+), 4 deletions(-)
  
  diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
  index b6baf77..9162e52 100644
  --- a/meta/lib/oe/rootfs.py
  +++ b/meta/lib/oe/rootfs.py
  @@ -317,10 +317,18 @@ class RpmRootfs(Rootfs):
   
   self.pm.update()
   
  -for pkg_type in self.install_order:
  -if pkg_type in pkgs_to_install:
  -self.pm.install(pkgs_to_install[pkg_type],
  -[False, True][pkg_type == aop])
  +pkgs = []
  +pkgs_attempt = []
  +for pkg_type in pkgs_to_install:
  +if pkg_type == Manifest.PKG_TYPE_ATTEMPT_ONLY:
  +pkgs_attempt = pkgs_to_install[pkg_type]
 
 Should this be a += ?
Indeed, I guess I was thinking that there can be only one attempt only
list. Nevertheless, should be += there. Thanks!

 
 This might be clearer as an if: else:
Agreed.

Incoming v2...

laurentiu
 
  +continue
  +
  +pkgs += pkgs_to_install[pkg_type]
  +
  +self.pm.install(pkgs)
  +
  +self.pm.install(pkgs_attempt, True)
   
   self.pm.install_complementary()
   
 
 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


[OE-core] [PATCH v2 0/1] lib/oe/rootfs.py: fix RPM multilib issue

2014-02-13 Thread Laurentiu Palcu
v2:
 * address RP's comments;

laurentiu

The following changes since commit b70a10ca3ee4a0add90d387e2c8288e531aaa47c:

  systemd-serialgetty: Update to match version in recent systemd (2014-02-11 
22:59:38 +)

are available in the git repository at:

  git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib 
lpalcu/rootfs_refactoring_rpm_multilib_fix

for you to fetch changes up to 453395e793d0d5c30d4e8092ee6faf80484ad447:

  lib/oe/rootfs.py: fix RPM multilib issue (2014-02-13 13:52:38 +0200)


Laurentiu Palcu (1):
  lib/oe/rootfs.py: fix RPM multilib issue

 meta/lib/oe/rootfs.py |   15 +++
 1 file changed, 11 insertions(+), 4 deletions(-)

Laurentiu Palcu (1):
  lib/oe/rootfs.py: fix RPM multilib issue

 meta/lib/oe/rootfs.py |   15 +++
 1 file changed, 11 insertions(+), 4 deletions(-)

-- 
1.7.9.5

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


  1   2   3   4   5   6   7   8   >