[OE-core] [PATCH 2/5] Move check that all installed files are shipped into insane.bbclass

2011-12-18 Thread Dmitry Eremin-Solenikov 
Checking that all installed files are shipped is in reality a QA check.
It would benefit from mechanisms like ERROR_QA/WARNING_QA. So move it
into insane.bbclass. If some of the files are installed but should not
be shipped for some reasons, one can add them to the variable
IGNORE_UNSHIPPED_FILES.

Signed-off-by: Dmitry Eremin-Solenikov dbarysh...@gmail.com
---
 meta/classes/insane.bbclass  |   50 -
 meta/classes/package.bbclass |   15 
 2 files changed, 48 insertions(+), 17 deletions(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 5726e69..41b815c 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -100,7 +100,7 @@ def package_qa_get_machine_dict():
 
 
 # Currently not being used by default desktop
-WARN_QA ?= ldflags useless-rpaths rpaths
+WARN_QA ?= ldflags useless-rpaths rpaths unshipped
 ERROR_QA ?= dev-so debug-deps dev-deps debug-files arch la2 pkgconfig la 
perms
 
 def package_qa_clean_path(path,d):
@@ -485,6 +485,49 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, d):
 
 return sane
 
+IGNORE_UNSHIPPED_FILES ??= 
+
+def packages_qa_unshipped_files(d):
+warn = (d.getVar('WARN_QA', True) or ).split()
+err = (d.getVar('ERROR_QA', True) or ).split()
+if not unshipped in warn + err:
+return True
+
+seen = d.getVar('IGNORE_UNSHIPPED_FILES', True).split()
+unshipped = []
+dvar = d.getVar('PKGD', True)
+destvar = d.getVar('PKGDEST', True)
+packages = d.getVar('PACKAGES', True).split()
+for p in packages:
+pdir = os.path.join(destvar, p)
+for root, dirs, files in os.walk(pdir):
+dir = root[len(pdir):]
+if not dir:
+dir = os.sep
+for f in (files + dirs):
+path = os.path.join(dir, f)
+if path not in seen:
+seen.append(path)
+
+for root, dirs, files in os.walk(dvar):
+dir = root[len(dvar):]
+if not dir:
+dir = os.sep
+for f in (files + dirs):
+path = os.path.join(dir, f)
+if path not in seen:
+unshipped.append(path)
+
+pn = d.getVar('PN', True)
+
+if unshipped == []:
+return True
+
+ret = package_qa_handle_error(unshipped, For recipe %s, the following 
files/directories were installed but not shipped in any package: % pn, d)
+for f in unshipped:
+package_qa_handle_error(unshipped, f, d)
+return ret
+
 # The PACKAGE FUNC to scan each package
 python do_package_qa () {
 bb.note(DO PACKAGE QA)
@@ -522,6 +565,7 @@ python do_package_qa () {
 g = globals()
 walk_sane = True
 rdepends_sane = True
+shipped_sane = True
 for package in packages.split():
 skip = (d.getVar('INSANE_SKIP_' + package, True) or ).split()
 if skip:
@@ -546,8 +590,10 @@ python do_package_qa () {
 if not package_qa_check_rdepends(package, pkgdest, skip, d):
 rdepends_sane = False
 
+if not packages_qa_unshipped_files(d):
+shipped_sane = False
 
-if not walk_sane or not rdepends_sane:
+if not walk_sane or not rdepends_sane or not shipped_sane:
 bb.fatal(QA run found fatal errors. Please consider fixing them.)
 bb.note(DONE with PACKAGE QA)
 }
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 39c1d4b..fbea9c6 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -957,21 +957,6 @@ python populate_packages () {
del localdata
os.chdir(workdir)
 
-   unshipped = []
-   for root, dirs, files in os.walk(dvar):
-   dir = root[len(dvar):]
-   if not dir:
-   dir = os.sep
-   for f in (files + dirs):
-   path = os.path.join(dir, f)
-   if ('.' + path) not in seen:
-   unshipped.append(path)
-
-   if unshipped != []:
-   bb.warn(For recipe %s, the following files/directories were 
installed but not shipped in any package: % pn)
-   for f in unshipped:
-   bb.warn(   + f)
-
bb.build.exec_func(package_name_hook, d)
 
for pkg in package_list:
-- 
1.7.7.3


___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] Move check that all installed files are shipped into insane.bbclass

2011-12-18 Thread Khem Raj 
On Sunday, December 18, 2011, Dmitry Eremin-Solenikov dbarysh...@gmail.com
wrote:
 Checking that all installed files are shipped is in reality a QA check.
 It would benefit from mechanisms like ERROR_QA/WARNING_QA. So move it
 into insane.bbclass. If some of the files are installed but should not
 be shipped for some reasons, one can add them to the variable
 IGNORE_UNSHIPPED_FILES.


This needs to be documented somewhere too and may be added to
sample.conf.extended

 Signed-off-by: Dmitry Eremin-Solenikov dbarysh...@gmail.com
 ---
  meta/classes/insane.bbclass  |   50
-
  meta/classes/package.bbclass |   15 
  2 files changed, 48 insertions(+), 17 deletions(-)

 diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
 index 5726e69..41b815c 100644
 --- a/meta/classes/insane.bbclass
 +++ b/meta/classes/insane.bbclass
 @@ -100,7 +100,7 @@ def package_qa_get_machine_dict():


  # Currently not being used by default desktop
 -WARN_QA ?= ldflags useless-rpaths rpaths
 +WARN_QA ?= ldflags useless-rpaths rpaths unshipped
  ERROR_QA ?= dev-so debug-deps dev-deps debug-files arch la2 pkgconfig
la perms

  def package_qa_clean_path(path,d):
 @@ -485,6 +485,49 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, d):

 return sane

 +IGNORE_UNSHIPPED_FILES ??= 
 +
 +def packages_qa_unshipped_files(d):
 +warn = (d.getVar('WARN_QA', True) or ).split()
 +err = (d.getVar('ERROR_QA', True) or ).split()
 +if not unshipped in warn + err:
 +return True
 +
 +seen = d.getVar('IGNORE_UNSHIPPED_FILES', True).split()
 +unshipped = []
 +dvar = d.getVar('PKGD', True)
 +destvar = d.getVar('PKGDEST', True)
 +packages = d.getVar('PACKAGES', True).split()
 +for p in packages:
 +pdir = os.path.join(destvar, p)
 +for root, dirs, files in os.walk(pdir):
 +dir = root[len(pdir):]
 +if not dir:
 +dir = os.sep
 +for f in (files + dirs):
 +path = os.path.join(dir, f)
 +if path not in seen
 +seen.append(path)
 +
 +for root, dirs, files in os.walk(dvar):
 +dir = root[len(dvar):]
 +if not dir:
 +dir = os.sep
 +for f in (files + dirs):
 +path = os.path.join(dir, f)
 +if path not in seen:
 +unshipped.append(path)
 +
 +pn = d.getVar('PN', True)
 +
 +if unshipped == []:
 +return True
 +
 +ret = package_qa_handle_error(unshipped, For recipe %s, the
following files/directories were installed but not shipped in any package:
% pn, d)
 +for f in unshipped:
 +package_qa_handle_error(unshipped, f, d)
 +return ret
 +
  # The PACKAGE FUNC to scan each package
  python do_package_qa () {
 bb.note(DO PACKAGE QA)
 @@ -522,6 +565,7 @@ python do_package_qa () {
 g = globals()
 walk_sane = True
 rdepends_sane = True
 +shipped_sane = True
 for package in packages.split():
 skip = (d.getVar('INSANE_SKIP_' + package, True) or ).split()
 if skip:
 @@ -546,8 +590,10 @@ python do_package_qa () {
 if not package_qa_check_rdepends(package, pkgdest, skip, d):
 rdepends_sane = False

 +if not packages_qa_unshipped_files(d):
 +shipped_sane = False

 -if not walk_sane or not rdepends_sane:
 +if not walk_sane or not rdepends_sane or not shipped_sane:
 bb.fatal(QA run found fatal errors. Please consider fixing
them.)
 bb.note(DONE with PACKAGE QA)
  }
 diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
 index 39c1d4b..fbea9c6 100644
 --- a/meta/classes/package.bbclass
 +++ b/meta/classes/package.bbclass
 @@ -957,21 +957,6 @@ python populate_packages () {
del localdata
os.chdir(workdir)

 -   unshipped = []
 -   for root, dirs, files in os.walk(dvar):
 -   dir = root[len(dvar):]
 -   if not dir:
 -   dir = os.sep
 -   for f in (files + dirs):
 -   path = os.path.join(dir, f)
 -   if ('.' + path) not in seen:
 -   unshipped.append(path)
 -
 -   if unshipped != []:
 -   bb.warn(For recipe %s, the following files/directories
were installed but not shipped in any package: % pn)
 -   for f in unshipped:
 -   bb.warn(   + f)
 -
bb.build.exec_func(package_name_hook, d)

for pkg in package_list:
 --
 1.7.7.3


 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] Move check that all installed files are shipped into insane.bbclass

2011-12-18 Thread Richard Purdie 
On Sun, 2011-12-18 at 15:29 -0800, Khem Raj wrote:
 On Sunday, December 18, 2011, Dmitry Eremin-Solenikov
 dbarysh...@gmail.com wrote:
  Checking that all installed files are shipped is in reality a QA
 check.
  It would benefit from mechanisms like ERROR_QA/WARNING_QA. So move
 it
  into insane.bbclass. If some of the files are installed but should
 not
  be shipped for some reasons, one can add them to the variable
  IGNORE_UNSHIPPED_FILES.
 
 This needs to be documented somewhere too and may be added to
 sample.conf.extended

This doesn't make sense for sample.conf.extended. We're talking about a
recipe specific configuration variable, not distro/machine/conf config
which is more appropriate there.

I'd suggest adding to documentation.conf and/or the various manuals.

Cheers,

Richard


___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] Move check that all installed files are shipped into insane.bbclass

2011-12-18 Thread Khem Raj 
On Sun, Dec 18, 2011 at 4:02 PM, Richard Purdie
richard.pur...@linuxfoundation.org wrote:
 On Sun, 2011-12-18 at 15:29 -0800, Khem Raj wrote:
 On Sunday, December 18, 2011, Dmitry Eremin-Solenikov
 dbarysh...@gmail.com wrote:
  Checking that all installed files are shipped is in reality a QA
 check.
  It would benefit from mechanisms like ERROR_QA/WARNING_QA. So move
 it
  into insane.bbclass. If some of the files are installed but should
 not
  be shipped for some reasons, one can add them to the variable
  IGNORE_UNSHIPPED_FILES.
 
 This needs to be documented somewhere too and may be added to
 sample.conf.extended

 This doesn't make sense for sample.conf.extended. We're talking about a
 recipe specific configuration variable, not distro/machine/conf config
 which is more appropriate there.

yes forgot that


 I'd suggest adding to documentation.conf and/or the various manuals.

 Cheers,

 Richard


 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core