Re: [gentoo-portage-dev] [PATCH] bin/estrip: avoid copying directories in FEATURES=installsources
On Sat, 17 Jul 2021 15:34:12 -0700 Zac Medico wrote: > On 7/17/21 12:59 PM, Sergei Trofimovich wrote: > > Initially problem is noticed on gcc-11 as a full ${WORKDIR} syncing > > into /usr/src/debug. It happens because `debug.sources` sometimes > > contains directory. For example on bash-5 it has: > > > > $ grep -zv '/<[^/>]*>$' debug.sources | LANG=C sort -z -u | sed -e > > 's/\x00/\n/g' > > bash-5.0/ > > bash-5.0/alias.c > > ... > > > > This causes syncing object files, config.log, final binaries > > and other unexpected data. The change avoids syncking paths > > that end with '/'. > > > > Signed-off-by: Sergei Trofimovich > > --- > > bin/estrip | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/bin/estrip b/bin/estrip > > index 7ef1ec35c..6cca0d04b 100755 > > --- a/bin/estrip > > +++ b/bin/estrip > > @@ -464,7 +464,10 @@ if [[ -s ${tmpdir}/debug.sources ]] && \ > > then > > __vecho "installsources: rsyncing source files" > > [[ -d ${D%/}/${prepstrip_sources_dir#/} ]] || mkdir -p > > "${D%/}/${prepstrip_sources_dir#/}" > > + # skip installation of ".../" (system headers? why inner slashes > > are forbidden?) > > + # skip syncing of ".../foo/" (complete directories) > > grep -zv '/<[^/>]*>$' "${tmpdir}"/debug.sources | \ > > + grep -zv '/$' | \ > > (cd "${WORKDIR}"; LANG=C sort -z -u | \ > > rsync -tL0 --chmod=ugo-st,a+r,go-w,Da+x,Fa-x --files-from=- > > "${WORKDIR}/" "${D%/}/${prepstrip_sources_dir#/}/" ) > > > > > > Looks good. Merged with both grep calls combined via grep -e. Thanks! > > https://gitweb.gentoo.org/proj/portage.git/commit/?id=e083c8bf20d8488d329e3dccd643c28429e6fe30 TIL 'grep -e'! Thank you! -- Sergei pgprH8NkjP86o.pgp Description: Цифровая подпись OpenPGP
[gentoo-portage-dev] [PATCH] bin/estrip: avoid copying directories in FEATURES=installsources
Initially problem is noticed on gcc-11 as a full ${WORKDIR} syncing into /usr/src/debug. It happens because `debug.sources` sometimes contains directory. For example on bash-5 it has: $ grep -zv '/<[^/>]*>$' debug.sources | LANG=C sort -z -u | sed -e 's/\x00/\n/g' bash-5.0/ bash-5.0/alias.c ... This causes syncing object files, config.log, final binaries and other unexpected data. The change avoids syncking paths that end with '/'. Signed-off-by: Sergei Trofimovich --- bin/estrip | 3 +++ 1 file changed, 3 insertions(+) diff --git a/bin/estrip b/bin/estrip index 7ef1ec35c..6cca0d04b 100755 --- a/bin/estrip +++ b/bin/estrip @@ -464,7 +464,10 @@ if [[ -s ${tmpdir}/debug.sources ]] && \ then __vecho "installsources: rsyncing source files" [[ -d ${D%/}/${prepstrip_sources_dir#/} ]] || mkdir -p "${D%/}/${prepstrip_sources_dir#/}" + # skip installation of ".../" (system headers? why inner slashes are forbidden?) + # skip syncing of ".../foo/" (complete directories) grep -zv '/<[^/>]*>$' "${tmpdir}"/debug.sources | \ + grep -zv '/$' | \ (cd "${WORKDIR}"; LANG=C sort -z -u | \ rsync -tL0 --chmod=ugo-st,a+r,go-w,Da+x,Fa-x --files-from=- "${WORKDIR}/" "${D%/}/${prepstrip_sources_dir#/}/" ) -- 2.32.0
[gentoo-portage-dev] [PATCH] emerge: drop FEATURES=distcc-pump support, bug #702146
'distcc' dustributes code generation for preprocessed files. 'pump' distributes preprocessing and code generation of files and imposes very strict requirement: """ Note that distcc's pump-mode assumes that sources files will not be modified during the lifetime of the include server, so modifying source files during a build may cause inconsistent results. """ `src_configure()` (where we used to start include server before this change) almost always violates that requirement. It is not uncommon to generate more intermediate source files as a package builds (`bison`, `flex`, child `./configure` calls from `make`) and thus quite unsafe to use `pump`. This change drops `FEATURES=distcc-pump` and leaves only FEATURES=distcc. This way all the proprocessing happens as expected and only code generation is offloaded. Closes: https://bugs.gentoo.org/702146 Signed-off-by: Sergei Trofimovich --- bin/phase-functions.sh | 17 - lib/_emerge/EbuildPhase.py | 2 +- lib/portage/const.py | 1 - 3 files changed, 1 insertion(+), 19 deletions(-) diff --git a/bin/phase-functions.sh b/bin/phase-functions.sh index 92fcd3929..73f8cee9b 100644 --- a/bin/phase-functions.sh +++ b/bin/phase-functions.sh @@ -403,19 +403,6 @@ __dyn_prepare() { trap - SIGINT SIGQUIT } -# @FUNCTION: __start_distcc -# @DESCRIPTION: -# Start distcc-pump if necessary. -__start_distcc() { - if has distcc $FEATURES && has distcc-pump $FEATURES ; then - if [[ -z $INCLUDE_SERVER_PORT ]] || [[ ! -w $INCLUDE_SERVER_PORT ]] ; then - # adding distcc to PATH repeatedly results in fatal distcc recursion :) - eval $(pump --startup | grep -v PATH) - trap "pump --shutdown >/dev/null" EXIT - fi - fi -} - __dyn_configure() { if [[ -e $PORTAGE_BUILDDIR/.configured ]] ; then @@ -435,7 +422,6 @@ __dyn_configure() { fi trap __abort_configure SIGINT SIGQUIT - __start_distcc __ebuild_phase pre_src_configure @@ -469,7 +455,6 @@ __dyn_compile() { fi trap __abort_compile SIGINT SIGQUIT - __start_distcc __ebuild_phase pre_src_compile @@ -493,7 +478,6 @@ __dyn_test() { fi trap "__abort_test" SIGINT SIGQUIT - __start_distcc if [[ -d ${S} ]]; then cd "${S}" @@ -541,7 +525,6 @@ __dyn_install() { return 0 fi trap "__abort_install" SIGINT SIGQUIT - __start_distcc # Handle setting QA_* based on QA_PREBUILT # Those variables shouldn't be needed before src_install() diff --git a/lib/_emerge/EbuildPhase.py b/lib/_emerge/EbuildPhase.py index 4104cefa7..50e3dd1f4 100644 --- a/lib/_emerge/EbuildPhase.py +++ b/lib/_emerge/EbuildPhase.py @@ -49,7 +49,7 @@ class EbuildPhase(CompositeTask): # FEATURES displayed prior to setup phase _features_display = ( - "ccache", "compressdebug", "distcc", "distcc-pump", "fakeroot", + "ccache", "compressdebug", "distcc", "fakeroot", "installsources", "keeptemp", "keepwork", "network-sandbox", "network-sandbox-proxy", "nostrip", "preserve-libs", "sandbox", "selinux", "sesandbox", "splitdebug", "suidctl", "test", diff --git a/lib/portage/const.py b/lib/portage/const.py index 36b33af92..e95039fd5 100644 --- a/lib/portage/const.py +++ b/lib/portage/const.py @@ -142,7 +142,6 @@ SUPPORTED_FEATURES = frozenset([ "config-protect-if-modified", "digest", "distcc", - "distcc-pump", "distlocks", "downgrade-backup", "ebuild-locks", -- 2.24.0
Re: [gentoo-portage-dev] [PATCH] repoman: add --include-profiles=PROFILES
On Mon, 18 Nov 2019 16:45:58 -0800 Zac Medico wrote: > On 11/18/19 4:21 PM, Sergei Trofimovich wrote: > > repoman slows down ~linearly with amount of profiles being scanned. > > In case of amd64 we have 28 stable profiles. > > > > To speed up processing and fit into time budged of various CIs we can > > split the work across different processes that handle different profiles. > > > > Example benchmark on ::haskell overlay: > > $ ./repoman full --include-arches=amd64 > > ~65 minutes > > $ ./repoman full --include-profiles=default/linux/amd64/17.0 > > ~4 minutes > > This allows for a crude sharding of work across processes and allows for > > cheap tree-wide scans for early failures. > > > > Bug: https://bugs.gentoo.org/700456 > > Signed-off-by: Sergei Trofimovich > > --- > > repoman/lib/repoman/actions.py | 4 > > repoman/lib/repoman/argparser.py| 7 +++ > > repoman/lib/repoman/modules/scan/depend/__init__.py | 3 ++- > > repoman/lib/repoman/modules/scan/depend/profile.py | 9 +++-- > > repoman/lib/repoman/scanner.py | 5 + > > repoman/man/repoman.1 | 4 > > 6 files changed, 29 insertions(+), 3 deletions(-) > > > > diff --git a/repoman/lib/repoman/actions.py b/repoman/lib/repoman/actions.py > > index 1c9989a72..92d4d4e94 100644 > > --- a/repoman/lib/repoman/actions.py > > +++ b/repoman/lib/repoman/actions.py > > @@ -412,6 +412,10 @@ the whole commit message to abort. > > report_options.append( > > "--include-arches=\"%s\"" % > > " ".join(sorted(self.scanner.include_arches))) > > + if self.scanner.include_profiles is not None: > > + report_options.append( > > + "--include-profiles=\"%s\"" % > > + " ".join(sorted(self.scanner.include_profiles))) > > > > if portage_version is None: > > sys.stderr.write("Failed to insert portage version in > > message!\n") > > diff --git a/repoman/lib/repoman/argparser.py > > b/repoman/lib/repoman/argparser.py > > index fa0e6ff90..670a0e91d 100644 > > --- a/repoman/lib/repoman/argparser.py > > +++ b/repoman/lib/repoman/argparser.py > > @@ -164,6 +164,13 @@ def parse_args(argv, repoman_default_opts): > > 'A space separated list of arches used to ' > > 'filter the selection of profiles for dependency > > checks')) > > > > + parser.add_argument( > > + '--include-profiles', > > + dest='include_profiles', metavar='PROFILES', action='append', > > + help=( > > + 'A space separated list of profiles used to ' > > + 'define the selection of profiles for dependency > > checks')) > > + > > parser.add_argument( > > '-d', '--include-dev', dest='include_dev', action='store_true', > > default=False, > > diff --git a/repoman/lib/repoman/modules/scan/depend/__init__.py > > b/repoman/lib/repoman/modules/scan/depend/__init__.py > > index c3cc0ddeb..9068760bb 100644 > > --- a/repoman/lib/repoman/modules/scan/depend/__init__.py > > +++ b/repoman/lib/repoman/modules/scan/depend/__init__.py > > @@ -19,7 +19,8 @@ module_spec = { > > 'func_desc': { > > }, > > 'mod_kwargs': ['qatracker', 'portdb', 'profiles', > > 'options', > > - 'repo_metadata', 'repo_settings', > > 'include_arches', 'caches', > > + 'repo_metadata', 'repo_settings', > > 'include_arches', > > + 'include_profiles', 'caches', > > 'repoman_incrementals', 'env', 'have', > > 'dev_keywords' > > ], > > 'func_kwargs': { > > diff --git a/repoman/lib/repoman/modules/scan/depend/profile.py > > b/repoman/lib/repoman/modules/scan/depend/profile.py > > index d980f4eca..0b1d74483 100644
[gentoo-portage-dev] [PATCH] repoman: add --include-profiles=PROFILES
repoman slows down ~linearly with amount of profiles being scanned. In case of amd64 we have 28 stable profiles. To speed up processing and fit into time budged of various CIs we can split the work across different processes that handle different profiles. Example benchmark on ::haskell overlay: $ ./repoman full --include-arches=amd64 ~65 minutes $ ./repoman full --include-profiles=default/linux/amd64/17.0 ~4 minutes This allows for a crude sharding of work across processes and allows for cheap tree-wide scans for early failures. Bug: https://bugs.gentoo.org/700456 Signed-off-by: Sergei Trofimovich --- repoman/lib/repoman/actions.py | 4 repoman/lib/repoman/argparser.py| 7 +++ repoman/lib/repoman/modules/scan/depend/__init__.py | 3 ++- repoman/lib/repoman/modules/scan/depend/profile.py | 9 +++-- repoman/lib/repoman/scanner.py | 5 + repoman/man/repoman.1 | 4 6 files changed, 29 insertions(+), 3 deletions(-) diff --git a/repoman/lib/repoman/actions.py b/repoman/lib/repoman/actions.py index 1c9989a72..92d4d4e94 100644 --- a/repoman/lib/repoman/actions.py +++ b/repoman/lib/repoman/actions.py @@ -412,6 +412,10 @@ the whole commit message to abort. report_options.append( "--include-arches=\"%s\"" % " ".join(sorted(self.scanner.include_arches))) + if self.scanner.include_profiles is not None: + report_options.append( + "--include-profiles=\"%s\"" % + " ".join(sorted(self.scanner.include_profiles))) if portage_version is None: sys.stderr.write("Failed to insert portage version in message!\n") diff --git a/repoman/lib/repoman/argparser.py b/repoman/lib/repoman/argparser.py index fa0e6ff90..670a0e91d 100644 --- a/repoman/lib/repoman/argparser.py +++ b/repoman/lib/repoman/argparser.py @@ -164,6 +164,13 @@ def parse_args(argv, repoman_default_opts): 'A space separated list of arches used to ' 'filter the selection of profiles for dependency checks')) + parser.add_argument( + '--include-profiles', + dest='include_profiles', metavar='PROFILES', action='append', + help=( + 'A space separated list of profiles used to ' + 'define the selection of profiles for dependency checks')) + parser.add_argument( '-d', '--include-dev', dest='include_dev', action='store_true', default=False, diff --git a/repoman/lib/repoman/modules/scan/depend/__init__.py b/repoman/lib/repoman/modules/scan/depend/__init__.py index c3cc0ddeb..9068760bb 100644 --- a/repoman/lib/repoman/modules/scan/depend/__init__.py +++ b/repoman/lib/repoman/modules/scan/depend/__init__.py @@ -19,7 +19,8 @@ module_spec = { 'func_desc': { }, 'mod_kwargs': ['qatracker', 'portdb', 'profiles', 'options', - 'repo_metadata', 'repo_settings', 'include_arches', 'caches', + 'repo_metadata', 'repo_settings', 'include_arches', + 'include_profiles', 'caches', 'repoman_incrementals', 'env', 'have', 'dev_keywords' ], 'func_kwargs': { diff --git a/repoman/lib/repoman/modules/scan/depend/profile.py b/repoman/lib/repoman/modules/scan/depend/profile.py index d980f4eca..0b1d74483 100644 --- a/repoman/lib/repoman/modules/scan/depend/profile.py +++ b/repoman/lib/repoman/modules/scan/depend/profile.py @@ -33,6 +33,7 @@ class ProfileDependsChecks(ScanBase): @param options: cli options @param repo_settings: repository settings instance @param include_arches: set + @param include_profiles: set @param caches: dictionary of our caches @param repoman_incrementals: tuple @param env: the environment @@ -46,6 +47,7 @@ class ProfileDependsChecks(ScanBase): self.options = kwargs.get('options') self.repo_settings = kwargs.get('repo_settings') self.include_arches = kwargs.get('include_arches') + self.include_profiles = kwargs.get
Re: [gentoo-portage-dev] Avoid stripping ld.so ?
On Thu, 30 Aug 2018 07:37:16 + Joakim Tjernlund wrote: > > > Tweaking ebuild to expand STRIP_MASK for ld.so might help. > > > > OK, can I use STRIP_MASK outside ebuilds, like in /etc/portage/make.conf or > > in my profile? Something like STRIP_MASK="/lib64/ld-* /lib/ld-*" I believe you can't. You would have to resort to /etc/portage/bashrc hacks to override ebuilds' value. Requesting new feature from dev-portage@ sounds reasonable. For example, add STRIP_MASK_USER to be controlled by user. STRIP_MASK is normally used for cases when software breaks when stripped (examples: not-quite-ELF files, not-yet-linked .o files). Not something for convenience or niche purposes. STRIP_MASK_USER might be a good way to fine-grain desired policy of stripping package on top of existing FEATURES=nostrip knob. -- Sergei
Re: [gentoo-portage-dev] Avoid stripping ld.so ?
On Wed, 29 Aug 2018 20:10:54 + Joakim Tjernlund wrote: > Is there a way for portage to avoid strippning debug syms for ld.so ? Tweaking ebuild to expand STRIP_MASK for ld.so might help. > I would like to avoid building all of glibc with debug syms. s/building/installing, right? You will have to build all (or large part) of glibc with debug symbols anyways as ld.so is linked of multiple source files. -- Sergei
Re: [gentoo-portage-dev] [PATCH v2] portage/dep/depcheck.py: Whitespace, add indentation level after newline
On Sat, 18 Jun 2016 20:26:20 +0100 Sergei Trofimovich wrote: > From: Sergei Trofimovich > > CC: dol...@gentoo.org > Signed-off-by: Sergei Trofimovich > --- > V2: updated subject > pym/portage/dep/dep_check.py | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/pym/portage/dep/dep_check.py b/pym/portage/dep/dep_check.py > index 9af4e65..76e5d82 100644 > --- a/pym/portage/dep/dep_check.py > +++ b/pym/portage/dep/dep_check.py > @@ -688,8 +688,8 @@ def dep_check(depstring, mydbapi, mysettings, use="yes", > mode=None, myuse=None, > else: > try: > mysplit = use_reduce(depstring, uselist=myusesplit, > - masklist=mymasks, matchall=(use=="all"), > excludeall=useforce, > - opconvert=True, token_class=Atom, eapi=eapi) > + masklist=mymasks, matchall=(use=="all"), > excludeall=useforce, > + opconvert=True, token_class=Atom, eapi=eapi) > except InvalidDependString as e: > return [0, "%s" % (e,)] > > -- > 2.9.0 > > Ping. -- Sergei pgp6VKNe_I_7L.pgp Description: Цифровая подпись OpenPGP
[gentoo-portage-dev] Re: [PATCH v2] repoman: new QA error: slot operator under '||' alternative
On Sat, 18 Jun 2016 14:15:54 -0700 Zac Medico wrote: > On 06/18/2016 12:27 PM, Sergei Trofimovich wrote: > > + if in_any_of and atom.slot_operator: > > + qatracker.add_error("dependency.badslotop", > > relative_path + > > + ": %s: '%s' uses ':=' slot operator under '||' > > dep clause." % > > + (mytype, atom)) > > It should be like this: > > if in_any_of and atom.slot_operator == '=': > > Because the case of atom.slot_operator == '*' is not forbidden. Nice catch! Sent V4. -- Sergei pgpyl5mQIj5Eq.pgp Description: Цифровая подпись OpenPGP
[gentoo-portage-dev] [PATCH v4] repoman: new QA error: slot operator under '||' alternative
From: Sergei Trofimovich A few links discussing what can go wrong with slot operator under '||': https://archives.gentoo.org/gentoo-dev/message/81a4e1a1f5767e20009628ecd33da8d4 https://archives.gentoo.org/gentoo-dev/message/66ff016a055e57b6027abcd36734e0e3 The main problem here is how vdb gets updated when you have a dependency of style: RDEPEND="|| ( foo:= bar:= )" depending on what you have installed in system: only 'foo'/'bar' or both 'foo' and 'bar'. I'm about to add actual test for some examples to gen-b0rk. New repoman complains on them as follows: RDEPEND="|| ( =not-broken/pkg1-subslot-0:= =not-broken/pkg1-subslot-1:0= )" Yields: dependency.badslotop [fatal] 2 ebuild-test/RDEPEND-any-of-slotop/RDEPEND-any-of-slotop-0.ebuild: RDEPEND: '=not-broken/pkg1-subslot-0:=' uses ':=' slot operator under '||' dep clause. ebuild-test/RDEPEND-any-of-slotop/RDEPEND-any-of-slotop-0.ebuild: RDEPEND: '=not-broken/pkg1-subslot-1:0=' uses ':=' slot operator under '||' dep clause. CC: dol...@gentoo.org CC: q...@gentoo.org CC: mgo...@gentoo.org Signed-off-by: Sergei Trofimovich --- V4: forbid only ':=' style slot operator. Zac pointed out ':*' is fine here. V3: unbaked tree traversal back to _traverse_tree, moved dependency parser out to check_slotop() helper V2: made check_slotop function local, baked tree traversal in it .../repoman/modules/scan/depend/_depend_checks.py | 45 ++ repoman/pym/repoman/qa_data.py | 2 + 2 files changed, 47 insertions(+) diff --git a/repoman/pym/repoman/modules/scan/depend/_depend_checks.py b/repoman/pym/repoman/modules/scan/depend/_depend_checks.py index 4e1d216..3f6c93e 100644 --- a/repoman/pym/repoman/modules/scan/depend/_depend_checks.py +++ b/repoman/pym/repoman/modules/scan/depend/_depend_checks.py @@ -3,11 +3,52 @@ from _emerge.Package import Package +from portage.dep import Atom + from repoman.check_missingslot import check_missingslot # import our initialized portage instance from repoman._portage import portage from repoman.qa_data import suspect_virtual, suspect_rdepend +def check_slotop(depstr, is_valid_flag, badsyntax, mytype, + qatracker, relative_path): + '''Checks if RDEPEND uses ':=' slot operator + in '||' style dependencies.''' + + try: + # to find use of ':=' in '||' we preserve + # tree structure of dependencies + my_dep_tree = portage.dep.use_reduce( + depstr, + flat=False, + matchall=1, + is_valid_flag=is_valid_flag, + opconvert=True, + token_class=portage.dep.Atom) + except portage.exception.InvalidDependString as e: + my_dep_tree = None + badsyntax.append(str(e)) + + def _traverse_tree(dep_tree, in_any_of): + # leaf + if isinstance(dep_tree, Atom): + atom = dep_tree + if in_any_of and atom.slot_operator == '=': + qatracker.add_error("dependency.badslotop", + "%s: %s: '%s' uses ':=' slot operator under '||' dep clause." % + (relative_path, mytype, atom)) + + # branches + if isinstance(dep_tree, list): + if len(dep_tree) == 0: + return + # entering any-of + if dep_tree[0] == '||': + _traverse_tree(dep_tree[1:], in_any_of=True) + else: + for branch in dep_tree: + _traverse_tree(branch, in_any_of=in_any_of) + _traverse_tree(my_dep_tree, False) def _depend_checks(ebuild, pkg, portdb, qatracker, repo_metadata): '''Checks the ebuild dependencies for errors @@ -117,6 +158,10 @@ def _depend_checks(ebuild, pkg, portdb, qatracker, repo_metadata): type_list.extend([mytype] * (len(badsyntax) - len(type_list))) + if runtime: + check_slotop(mydepstr, pkg.iuse.is_valid_flag, + badsyntax, mytype, qatracker, ebuild.relative_path) + for m, b in zip(type_list, badsyntax): if m.endswith("DEPEND"): qacat = "dependency.syntax" diff --git a/repoman/pym/repoman/qa_data.py b/repoman/pym/repoman/qa_data.py index b9475e8..48ab389 100644 --- a/r
[gentoo-portage-dev] [PATCH] repoman: new QA error: slot operator under '||' alternative
From: Sergei Trofimovich A few links discussing what can go wrong with slot operator under '||': https://archives.gentoo.org/gentoo-dev/message/81a4e1a1f5767e20009628ecd33da8d4 https://archives.gentoo.org/gentoo-dev/message/66ff016a055e57b6027abcd36734e0e3 The main problem here is how vdb gets updated when you have a dependency of style: RDEPEND="|| ( foo:= bar:= )" depending on what you have installed in system: only 'foo'/'bar' or both 'foo' and 'bar'. I'm about to add actual test for some examples to gen-b0rk. New repoman complains on them as follows: RDEPEND="|| ( =not-broken/pkg1-subslot-0:= =not-broken/pkg1-subslot-1:0= )" Yields: dependency.badslotop [fatal] 2 ebuild-test/RDEPEND-any-of-slotop/RDEPEND-any-of-slotop-0.ebuild: RDEPEND: '=not-broken/pkg1-subslot-0:=' uses ':=' slot operator under '||' dep clause. ebuild-test/RDEPEND-any-of-slotop/RDEPEND-any-of-slotop-0.ebuild: RDEPEND: '=not-broken/pkg1-subslot-1:0=' uses ':=' slot operator under '||' dep clause. CC: dol...@gentoo.org CC: q...@gentoo.org CC: mgo...@gentoo.org Signed-off-by: Sergei Trofimovich --- V3: unbaked tree traversal back to _traverse_tree, moved dependency parser out to check_slotop() helper V2: made check_slotop function local, baked tree traversal in it .../repoman/modules/scan/depend/_depend_checks.py | 45 ++ repoman/pym/repoman/qa_data.py | 2 + 2 files changed, 47 insertions(+) diff --git a/repoman/pym/repoman/modules/scan/depend/_depend_checks.py b/repoman/pym/repoman/modules/scan/depend/_depend_checks.py index 4e1d216..64a35f4 100644 --- a/repoman/pym/repoman/modules/scan/depend/_depend_checks.py +++ b/repoman/pym/repoman/modules/scan/depend/_depend_checks.py @@ -3,11 +3,52 @@ from _emerge.Package import Package +from portage.dep import Atom + from repoman.check_missingslot import check_missingslot # import our initialized portage instance from repoman._portage import portage from repoman.qa_data import suspect_virtual, suspect_rdepend +def check_slotop(depstr, is_valid_flag, badsyntax, mytype, + qatracker, relative_path): + '''Checks if RDEPEND uses ':=' slot operator + in '||' style dependencies.''' + + try: + # to find use of ':=' in '||' we preserve + # tree structure of dependencies + my_dep_tree = portage.dep.use_reduce( + depstr, + flat=False, + matchall=1, + is_valid_flag=is_valid_flag, + opconvert=True, + token_class=portage.dep.Atom) + except portage.exception.InvalidDependString as e: + my_dep_tree = None + badsyntax.append(str(e)) + + def _traverse_tree(dep_tree, in_any_of): + # leaf + if isinstance(dep_tree, Atom): + atom = dep_tree + if in_any_of and atom.slot_operator: + qatracker.add_error("dependency.badslotop", + "%s: %s: '%s' uses ':=' slot operator under '||' dep clause." % + (relative_path, mytype, atom)) + + # branches + if isinstance(dep_tree, list): + if len(dep_tree) == 0: + return + # entering any-of + if dep_tree[0] == '||': + _traverse_tree(dep_tree[1:], in_any_of=True) + else: + for branch in dep_tree: + _traverse_tree(branch, in_any_of=in_any_of) + _traverse_tree(my_dep_tree, False) def _depend_checks(ebuild, pkg, portdb, qatracker, repo_metadata): '''Checks the ebuild dependencies for errors @@ -117,6 +158,10 @@ def _depend_checks(ebuild, pkg, portdb, qatracker, repo_metadata): type_list.extend([mytype] * (len(badsyntax) - len(type_list))) + if runtime: + check_slotop(mydepstr, pkg.iuse.is_valid_flag, + badsyntax, mytype, qatracker, ebuild.relative_path) + for m, b in zip(type_list, badsyntax): if m.endswith("DEPEND"): qacat = "dependency.syntax" diff --git a/repoman/pym/repoman/qa_data.py b/repoman/pym/repoman/qa_data.py index b9475e8..48ab389 100644 --- a/repoman/pym/repoman/qa_data.py +++ b/repoman/pym/repoman/qa_data.py @@ -58,6 +58,8 @@ qahelp = { "Ebuild has a dependency t
[gentoo-portage-dev] [PATCH v2] repoman: new QA error: slot operator under '||' alternative
From: Sergei Trofimovich A few links discussing what can go wrong with slot operator under '||': https://archives.gentoo.org/gentoo-dev/message/81a4e1a1f5767e20009628ecd33da8d4 https://archives.gentoo.org/gentoo-dev/message/66ff016a055e57b6027abcd36734e0e3 The main problem here is how vdb gets updated when you have a dependency of style: RDEPEND="|| ( foo:= bar:= )" depending on what you have installed in system: only 'foo'/'bar' or both 'foo' and 'bar'. I'm about to add actual test for some examples to gen-b0rk. New repoman complains on them as follows: RDEPEND="|| ( =not-broken/pkg1-subslot-0:= =not-broken/pkg1-subslot-1:0= )" Yields: dependency.badslotop [fatal] 2 ebuild-test/RDEPEND-any-of-slotop/RDEPEND-any-of-slotop-0.ebuild: RDEPEND: '=not-broken/pkg1-subslot-0:=' uses ':=' slot operator under '||' dep clause. ebuild-test/RDEPEND-any-of-slotop/RDEPEND-any-of-slotop-0.ebuild: RDEPEND: '=not-broken/pkg1-subslot-1:0=' uses ':=' slot operator under '||' dep clause. CC: dol...@gentoo.org CC: q...@gentoo.org CC: mgo...@gentoo.org Signed-off-by: Sergei Trofimovich --- V2: made check_slotop function local, baked tree traversal in it .../repoman/modules/scan/depend/_depend_checks.py | 42 ++ repoman/pym/repoman/qa_data.py | 2 ++ 2 files changed, 44 insertions(+) diff --git a/repoman/pym/repoman/modules/scan/depend/_depend_checks.py b/repoman/pym/repoman/modules/scan/depend/_depend_checks.py index 4e1d216..2ee7e9c 100644 --- a/repoman/pym/repoman/modules/scan/depend/_depend_checks.py +++ b/repoman/pym/repoman/modules/scan/depend/_depend_checks.py @@ -3,11 +3,37 @@ from _emerge.Package import Package +from portage.dep import Atom + from repoman.check_missingslot import check_missingslot # import our initialized portage instance from repoman._portage import portage from repoman.qa_data import suspect_virtual, suspect_rdepend +def check_slotop(dep_tree, mytype, qatracker, relative_path, in_any_of=False): + '''Checks if RDEPEND uses ':=' slot operator + in '||' style dependencies.''' + + # leaf + if isinstance(dep_tree, Atom): + atom = dep_tree + if in_any_of and atom.slot_operator: + qatracker.add_error("dependency.badslotop", relative_path + + ": %s: '%s' uses ':=' slot operator under '||' dep clause." % + (mytype, atom)) + + # branches + if isinstance(dep_tree, list): + if len(dep_tree) == 0: + return + # entering any-of + if dep_tree[0] == '||': + check_slotop(dep_tree[1:], mytype, qatracker, relative_path, + in_any_of=True) + else: + for branch in dep_tree: + check_slotop(branch, mytype, qatracker, relative_path, + in_any_of=in_any_of) def _depend_checks(ebuild, pkg, portdb, qatracker, repo_metadata): '''Checks the ebuild dependencies for errors @@ -117,6 +143,22 @@ def _depend_checks(ebuild, pkg, portdb, qatracker, repo_metadata): type_list.extend([mytype] * (len(badsyntax) - len(type_list))) + if runtime: + try: + # to find use of ':=' in '||' we preserve + # tree structure of dependencies + hier_atoms = portage.dep.use_reduce( + mydepstr, + flat=False, + matchall=1, + is_valid_flag=pkg.iuse.is_valid_flag, + opconvert=True, + token_class=token_class) + except portage.exception.InvalidDependString as e: + hier_atoms = None + badsyntax.append(str(e)) + check_slotop(hier_atoms, mytype, qatracker, ebuild.relative_path) + for m, b in zip(type_list, badsyntax): if m.endswith("DEPEND"): qacat = "dependency.syntax" diff --git a/repoman/pym/repoman/qa_data.py b/repoman/pym/repoman/qa_data.py index b9475e8..48ab389 100644 --- a/repoman/pym/repoman/qa_data.py +++ b/repoman/pym/repoman/qa_data.py @@ -58,6 +58,8 @@ qahelp = { "Ebuild has a dependency that refers to an unknown package" &qu
[gentoo-portage-dev] [PATCH v2] repoman/modules/.../profile.py: typo in docstring s/dependancy/dependency/
From: Sergei Trofimovich Signed-off-by: Sergei Trofimovich --- V2: updated subject repoman/pym/repoman/modules/scan/depend/profile.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repoman/pym/repoman/modules/scan/depend/profile.py b/repoman/pym/repoman/modules/scan/depend/profile.py index a714a93..cf3d9a8 100644 --- a/repoman/pym/repoman/modules/scan/depend/profile.py +++ b/repoman/pym/repoman/modules/scan/depend/profile.py @@ -51,7 +51,7 @@ class ProfileDependsChecks(ScanBase): self.repo_metadata = kwargs.get('repo_metadata') def check(self, **kwargs): - '''Perform profile dependant dependancy checks + '''Perform profile dependant dependency checks @param arches: @param pkg: Package in which we check (object). -- 2.9.0
[gentoo-portage-dev] [PATCH v2] portage/dep/depcheck.py: Whitespace, add indentation level after newline
From: Sergei Trofimovich CC: dol...@gentoo.org Signed-off-by: Sergei Trofimovich --- V2: updated subject pym/portage/dep/dep_check.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pym/portage/dep/dep_check.py b/pym/portage/dep/dep_check.py index 9af4e65..76e5d82 100644 --- a/pym/portage/dep/dep_check.py +++ b/pym/portage/dep/dep_check.py @@ -688,8 +688,8 @@ def dep_check(depstring, mydbapi, mysettings, use="yes", mode=None, myuse=None, else: try: mysplit = use_reduce(depstring, uselist=myusesplit, - masklist=mymasks, matchall=(use=="all"), excludeall=useforce, - opconvert=True, token_class=Atom, eapi=eapi) + masklist=mymasks, matchall=(use=="all"), excludeall=useforce, + opconvert=True, token_class=Atom, eapi=eapi) except InvalidDependString as e: return [0, "%s" % (e,)] -- 2.9.0
[gentoo-portage-dev] [PATCH] repoman: whitespace: add indentation level after newline
From: Sergei Trofimovich Signed-off-by: Sergei Trofimovich --- pym/portage/dep/dep_check.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pym/portage/dep/dep_check.py b/pym/portage/dep/dep_check.py index 9af4e65..76e5d82 100644 --- a/pym/portage/dep/dep_check.py +++ b/pym/portage/dep/dep_check.py @@ -688,8 +688,8 @@ def dep_check(depstring, mydbapi, mysettings, use="yes", mode=None, myuse=None, else: try: mysplit = use_reduce(depstring, uselist=myusesplit, - masklist=mymasks, matchall=(use=="all"), excludeall=useforce, - opconvert=True, token_class=Atom, eapi=eapi) + masklist=mymasks, matchall=(use=="all"), excludeall=useforce, + opconvert=True, token_class=Atom, eapi=eapi) except InvalidDependString as e: return [0, "%s" % (e,)] -- 2.9.0
[gentoo-portage-dev] [PATCH] repoman: new QA error: slot operator under '||' alternative
From: Sergei Trofimovich A few links discussing what can go wrong with slot operator under '||': https://archives.gentoo.org/gentoo-dev/message/81a4e1a1f5767e20009628ecd33da8d4 https://archives.gentoo.org/gentoo-dev/message/66ff016a055e57b6027abcd36734e0e3 The main problem here is how vdb gets updated when you have a dependency of style: RDEPEND="|| ( foo:= bar:= )" depending on what you have installed in system: only 'foo'/'bar' or both 'foo' and 'bar'. I'm about to add actual test for some examples to gen-b0rk. New repoman complains on them as follows: RDEPEND="|| ( =not-broken/pkg1-subslot-0:= =not-broken/pkg1-subslot-1:0= )" Yields: dependency.badslotop [fatal] 2 ebuild-test/RDEPEND-any-of-slotop/RDEPEND-any-of-slotop-0.ebuild: RDEPEND: '=not-broken/pkg1-subslot-0:=' uses ':=' slot operator under '||' dep clause. ebuild-test/RDEPEND-any-of-slotop/RDEPEND-any-of-slotop-0.ebuild: RDEPEND: '=not-broken/pkg1-subslot-1:0=' uses ':=' slot operator under '||' dep clause. CC: q...@gentoo.org CC: mgo...@gentoo.org Signed-off-by: Sergei Trofimovich --- repoman/pym/repoman/check_slotop.py| 32 ++ .../repoman/modules/scan/depend/_depend_checks.py | 17 repoman/pym/repoman/qa_data.py | 2 ++ 3 files changed, 51 insertions(+) create mode 100644 repoman/pym/repoman/check_slotop.py diff --git a/repoman/pym/repoman/check_slotop.py b/repoman/pym/repoman/check_slotop.py new file mode 100644 index 000..3c3aec1 --- /dev/null +++ b/repoman/pym/repoman/check_slotop.py @@ -0,0 +1,32 @@ +# -*- coding:utf-8 -*- +# repoman: missing slot check +# Copyright 2016 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 + +"""This module contains the check used to find ':=' slot operator +uses in '||' style dependencies.""" + +from portage.dep import Atom + +def check_slotop(dep_tree, mytype, qatracker, relative_path): + def _traverse_tree(dep_tree, is_under_any_of): + # leaf + if isinstance(dep_tree, Atom): + atom = dep_tree + if is_under_any_of and atom.slot_operator: + qatracker.add_error("dependency.badslotop", relative_path + + ": %s: '%s' uses ':=' slot operator under '||' dep clause." % + (mytype, atom)) + + # branches + if isinstance(dep_tree, list): + if len(dep_tree) == 0: + return + # any-of + if dep_tree[0] == '||': + _traverse_tree(dep_tree[1:], True) + else: + for branch in dep_tree: + _traverse_tree(branch, is_under_any_of) + + _traverse_tree(dep_tree, False) diff --git a/repoman/pym/repoman/modules/scan/depend/_depend_checks.py b/repoman/pym/repoman/modules/scan/depend/_depend_checks.py index 4e1d216..1fd69d4 100644 --- a/repoman/pym/repoman/modules/scan/depend/_depend_checks.py +++ b/repoman/pym/repoman/modules/scan/depend/_depend_checks.py @@ -4,6 +4,7 @@ from _emerge.Package import Package from repoman.check_missingslot import check_missingslot +from repoman.check_slotop import check_slotop # import our initialized portage instance from repoman._portage import portage from repoman.qa_data import suspect_virtual, suspect_rdepend @@ -117,6 +118,22 @@ def _depend_checks(ebuild, pkg, portdb, qatracker, repo_metadata): type_list.extend([mytype] * (len(badsyntax) - len(type_list))) + if runtime: + try: + # to find use of ':=' in '||' we preserve + # tree structure of dependencies + hier_atoms = portage.dep.use_reduce( + mydepstr, + flat=False, + matchall=1, + is_valid_flag=pkg.iuse.is_valid_flag, + opconvert=True, + token_class=token_class) + except portage.exception.InvalidDependString as e: + hier_atoms = None + badsyntax.append(str(e)) + check_slotop(hier_atoms, mytype, qatracker, ebuild.relative_path) + for m, b in zip(type_list, badsyntax): if m.endswith(&qu
[gentoo-portage-dev] [PATCH] repoman: typo in docstring s/dependancy/dependency/
From: Sergei Trofimovich Signed-off-by: Sergei Trofimovich --- repoman/pym/repoman/modules/scan/depend/profile.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repoman/pym/repoman/modules/scan/depend/profile.py b/repoman/pym/repoman/modules/scan/depend/profile.py index a714a93..cf3d9a8 100644 --- a/repoman/pym/repoman/modules/scan/depend/profile.py +++ b/repoman/pym/repoman/modules/scan/depend/profile.py @@ -51,7 +51,7 @@ class ProfileDependsChecks(ScanBase): self.repo_metadata = kwargs.get('repo_metadata') def check(self, **kwargs): - '''Perform profile dependant dependancy checks + '''Perform profile dependant dependency checks @param arches: @param pkg: Package in which we check (object). -- 2.9.0
Re: [gentoo-portage-dev] [PATCH 2/2] runtests: fix output for skipped pythons
On Sun, 20 Mar 2016 14:53:01 +0100 Alexander Berntsen wrote: > Nice! This LGMT too, and I'll commit and push it Monday with the other > one unless there are any objections. Thanks :) -- Sergei pgpKDeifVM2Uk.pgp Description: Цифровая подпись OpenPGP
[gentoo-portage-dev] [PATCH 2/2] runtests: fix output for skipped pythons
From: Sergei Trofimovich I have Pythons 2.7 and 3.4 installed. Before the patch I had the following output: $ ./runtests pym/portage/tests/resolver/test_blocker.py Testing with Python 2.7... Testing with Python 3.4... Summary: | Version| Status | | 2.7| PASS | 3.3| PASS After the patch: $ ./runtests pym/portage/tests/resolver/test_blocker.py Testing with Python 2.7... Skip Python 3.3... Testing with Python 3.4... Skip Python pypy... Skip Python 3.5... Summary: | Version| Status | | 2.7| PASS | 3.4| PASS Signed-off-by: Sergei Trofimovich --- runtests | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/runtests b/runtests index d492bc8..9135768 100755 --- a/runtests +++ b/runtests @@ -42,7 +42,7 @@ class Colors(object): _COLORS_YES = ('y', 'yes', 'true') _COLORS_NO = ('n', 'no', 'false') - GOOD = BAD = NORMAL = '' + WARN = GOOD = BAD = NORMAL = '' def __init__(self, colorize=None): if colorize is None: @@ -63,6 +63,7 @@ class Colors(object): raise ValueError('--colors is invalid: %s' % colorize) if colorize: + self.WARN = '\033[1;33m' self.GOOD = '\033[1;32m' self.BAD = '\033[1;31m' self.NORMAL = '\033[0m' @@ -135,11 +136,14 @@ def main(argv): if os.access(prog, os.X_OK): print('%sTesting with Python %s...%s' % (colors.GOOD, ver, colors.NORMAL)) - statuses.append(subprocess.call(cmd)) + statuses.append((ver, subprocess.call(cmd))) elif not ignore_missing: print('%sCould not find requested Python %s%s' % (colors.BAD, ver, colors.NORMAL)) - statuses.append(1) + statuses.append((ver, 1)) + else: + print('%sSkip Python %s...%s' % + (colors.WARN, ver, colors.NORMAL)) print() finally: if tempdir is not None: @@ -154,7 +158,9 @@ def main(argv): width = 10 header = '| %-*s | %s' % (width, 'Version', 'Status') print('%s\n|%s' % (header, '-' * (len(header) - 1))) - for ver, status in zip(pyversions, statuses): + exit_status = 0 + for ver, status in statuses: + exit_status += status if status: color = colors.BAD msg = 'FAIL' @@ -163,7 +169,7 @@ def main(argv): msg = 'PASS' print('| %s%-*s%s | %s%s%s' % (color, width, ver, colors.NORMAL, color, msg, colors.NORMAL)) - exit(sum(statuses)) + exit(exit_status) if __name__ == '__main__': -- 2.7.4
[gentoo-portage-dev] [PATCH 1/2] cleanup: fix spaces to tabs in depgraph.py
From: Sergei Trofimovich Signed-off-by: Sergei Trofimovich --- pym/_emerge/depgraph.py | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py index b58fbda..cfaafa3 100644 --- a/pym/_emerge/depgraph.py +++ b/pym/_emerge/depgraph.py @@ -5126,9 +5126,9 @@ class depgraph(object): break writemsg("\nemerge: there are no %s to satisfy " % -("binary packages" if - self._frozen_config.myopts.get("--usepkgonly", "y") == True - else "ebuilds") + green(xinfo) + ".\n", noiselevel=-1) + ("binary packages" if + self._frozen_config.myopts.get("--usepkgonly", "y") == True + else "ebuilds") + green(xinfo) + ".\n", noiselevel=-1) if isinstance(myparent, AtomArg) and \ not cp_exists and \ self._frozen_config.myopts.get( @@ -6937,7 +6937,7 @@ class depgraph(object): for root in implicit_libc_roots: vardb = self._frozen_config.trees[root]["vartree"].dbapi for atom in self._expand_virt_from_graph(root, - portage.const.LIBC_PACKAGE_ATOM): + portage.const.LIBC_PACKAGE_ATOM): if atom.blocker: continue for pkg in self._dynamic_config._package_tracker.match(root, atom): -- 2.7.4
Re: [gentoo-portage-dev] [PATCH] repoman: Finally deprecate base.eclass
On Sun, 11 Oct 2015 09:23:20 +0200 Michał Górny wrote: > Contributors are repeatedly adding base.eclass uses, so we should > finally make the deprecation formal, even at the cost of adding warnings > for some frequently used eclasses. > > Signed-off-by: Michał Górny > --- > pym/repoman/checks/ebuilds/checks.py | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/pym/repoman/checks/ebuilds/checks.py > b/pym/repoman/checks/ebuilds/checks.py > index ae6d96e..a00d518 100644 > --- a/pym/repoman/checks/ebuilds/checks.py > +++ b/pym/repoman/checks/ebuilds/checks.py > @@ -409,6 +409,7 @@ class InheritDeprecated(LineCheck): > > # deprecated eclass : new eclass (False if no new eclass) > deprecated_eclasses = { > + "base": False, > "bash-completion": "bash-completion-r1", > "boost-utils": False, > "distutils": "distutils-r1", > -- > 2.6.1 I would also suggest: - adding a @DEPRECATED documentation stanza right into base.eclass with a migration strategy link - and a ewarn/eqawarn call to flag all offenders and prevent new ebuilds to creep dependency back -- Sergei signature.asc Description: PGP signature
[gentoo-portage-dev] [PATCH] Switch to new git '# $Id$' header format
From: Sergei Trofimovich Currently repoman complains on every ebuild in ::gentoo. Signed-off-by: Sergei Trofimovich --- pym/repoman/checks.py | 7 +++ pym/repoman/errors.py | 2 +- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py index 5f37648..4ab23d1 100644 --- a/pym/repoman/checks.py +++ b/pym/repoman/checks.py @@ -81,7 +81,7 @@ class EbuildHeader(LineCheck): # Why a regex here, use a string match # gentoo_license = re.compile(r'^# Distributed under the terms of the GNU General Public License v2$') gentoo_license = '# Distributed under the terms of the GNU General Public License v2' - cvs_header = re.compile(r'^# \$Header: .*\$$') + id_header = '# $Id$' ignore_comment = False def new(self, pkg): @@ -100,9 +100,8 @@ class EbuildHeader(LineCheck): return errors.COPYRIGHT_ERROR elif num == 1 and line.rstrip('\n') != self.gentoo_license: return errors.LICENSE_ERROR - elif num == 2: - if not self.cvs_header.match(line): - return errors.CVS_HEADER_ERROR + elif num == 2 and line.rstrip('\n') != self.id_header: + return errors.ID_HEADER_ERROR class EbuildWhitespace(LineCheck): diff --git a/pym/repoman/errors.py b/pym/repoman/errors.py index 3833be6..74a5959 100644 --- a/pym/repoman/errors.py +++ b/pym/repoman/errors.py @@ -6,7 +6,7 @@ from __future__ import unicode_literals COPYRIGHT_ERROR = 'Invalid Gentoo Copyright on line: %d' LICENSE_ERROR = 'Invalid Gentoo/GPL License on line: %d' -CVS_HEADER_ERROR = 'Malformed CVS Header on line: %d' +ID_HEADER_ERROR = 'Malformed Id header on line: %d' LEADING_SPACES_ERROR = 'Ebuild contains leading spaces on line: %d' TRAILING_WHITESPACE_ERROR = 'Trailing whitespace error on line: %d' READONLY_ASSIGNMENT_ERROR = 'Ebuild contains assignment to read-only variable on line: %d' -- 2.5.0
[gentoo-portage-dev] [PATCH] bin/eapi.sh: default to EAPI=0 when EAPI is unset
Observed as a breakage on binutils ebuilds: > ERROR: sys-devel/binutils-2.24-r3::gentoo failed (depend phase): > use() calls are not allowed in global scope >Call stack: >ebuild.sh, line 584: Called source 'binutils-2.24-r3.ebuild, >ebuild.sh, line 7: Called inherit 'toolchain-binutils' >ebuild.sh, line 280: Called __qa_source > '/gentoo-32k/gentoo-x86/eclass/toolchain-binutils.e >ebuild.sh, line 80: Called source > '/gentoo-32k/gentoo-x86/eclass/toolchain-binutils.eclass > toolchain-binutils.eclass, line 106: Called use 'multislot' > ebuild.sh, line 47: Called die > The specific snippet of code: > # These functions die because calls to them during the "depend" phase Zac confirms it's a safe thing to use ${EAPI-0} here: > ebuild.sh unsets the EAPI just before it sources the ebuild, > and then it does this after it sources the ebuild: > > [ "${EAPI+set}" = set ] || EAPI=0 > > So, for any code that is called while the ebuild is being sourced, using > ${EAPI-0} would be correct. For consistency and defense against future copy/paste errors converted all uses of ${EAPI} for ${EAPI-0} in 'bin/eapi.sh'. Signed-off-by: Sergei Trofimovich --- bin/eapi.sh | 90 ++--- 1 file changed, 45 insertions(+), 45 deletions(-) diff --git a/bin/eapi.sh b/bin/eapi.sh index 7e7b54b..f1c677e 100644 --- a/bin/eapi.sh +++ b/bin/eapi.sh @@ -5,189 +5,189 @@ # PHASES ___eapi_has_pkg_pretend() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2|3)$ ]] } ___eapi_has_src_prepare() { - [[ ! ${1-${EAPI}} =~ ^(0|1)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1)$ ]] } ___eapi_has_src_configure() { - [[ ! ${1-${EAPI}} =~ ^(0|1)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1)$ ]] } ___eapi_default_src_test_disables_parallel_jobs() { - [[ ${1-${EAPI}} =~ ^(0|1|2|3|4|4-python|4-slot-abi)$ ]] + [[ ${1-${EAPI-0}} =~ ^(0|1|2|3|4|4-python|4-slot-abi)$ ]] } ___eapi_has_S_WORKDIR_fallback() { - [[ ${1-${EAPI}} =~ ^(0|1|2|3)$ ]] + [[ ${1-${EAPI-0}} =~ ^(0|1|2|3)$ ]] } # VARIABLES ___eapi_has_prefix_variables() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2)$ || " ${FEATURES} " == *" force-prefix "* ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2)$ || " ${FEATURES} " == *" force-prefix "* ]] } ___eapi_has_HDEPEND() { - [[ ${1-${EAPI}} =~ ^(5-hdepend)$ ]] + [[ ${1-${EAPI-0}} =~ ^(5-hdepend)$ ]] } ___eapi_has_RDEPEND_DEPEND_fallback() { - [[ ${1-${EAPI}} =~ ^(0|1|2|3)$ ]] + [[ ${1-${EAPI-0}} =~ ^(0|1|2|3)$ ]] } # HELPERS PRESENCE ___eapi_has_dohard() { - [[ ${1-${EAPI}} =~ ^(0|1|2|3)$ ]] + [[ ${1-${EAPI-0}} =~ ^(0|1|2|3)$ ]] } ___eapi_has_dosed() { - [[ ${1-${EAPI}} =~ ^(0|1|2|3)$ ]] + [[ ${1-${EAPI-0}} =~ ^(0|1|2|3)$ ]] } ___eapi_has_einstall() { - [[ ${1-${EAPI}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] + [[ ${1-${EAPI-0}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] } ___eapi_has_dohtml_deprecated() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] } ___eapi_has_docompress() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2|3)$ ]] } ___eapi_has_nonfatal() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2|3)$ ]] } ___eapi_has_doheader() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3|4|4-python|4-slot-abi)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2|3|4|4-python|4-slot-abi)$ ]] } ___eapi_has_usex() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3|4|4-python|4-slot-abi)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2|3|4|4-python|4-slot-abi)$ ]] } ___eapi_has_get_libdir() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] } ___eapi_has_einstalldocs() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] } ___eapi_has_eapply() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] } ___eapi_has_eapply_user() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] + [[ ! ${1-${EAPI-0}} =~ ^(0|1|2|3|4|4-python|4-slot-abi|5|5-hdepend|5-progress)$ ]] } ___eapi_has_in_iuse() { - [[ ! ${1-${EAPI}} =~ ^(0|1|2|3|4|4-pytho
Re: [gentoo-portage-dev] [PATCH 16/18] Disallow helpers in global scope in EAPI 6
On Mon, 1 Dec 2014 22:28:34 +0100 Michał Górny wrote: > Disallow calling most of the ebuild helpers in global scope since they > are meaningless in that context. Most of them are also prohibited by PMS > for all EAPIs, so EAPI 6 sounds like a good moment to finally enforce > that restriction. > --- > bin/eapi.sh | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/bin/eapi.sh b/bin/eapi.sh > index e0ade02..5ab92f4 100644 > --- a/bin/eapi.sh > +++ b/bin/eapi.sh > @@ -159,7 +159,7 @@ ___eapi_helpers_can_die() { > } > > ___eapi_disallows_helpers_in_global_scope() { > - [[ ${1-${EAPI}} =~ ^(4-python|5-progress)$ ]] > + [[ ! ${1-${EAPI}} =~ ^(0|1|2|3|4|4-slot-abi|5|5-hdepend)$ ]] > } > > ___eapi_unpack_is_case_sensitive() { Looks like it cries on ebuilds with unset EAPI in main tree (attached). Can be easily fixed in tree by adding EAPI=0, but better handle it here: ${EAPI-0} (untested!) It also changed binutils: it used to be slotted only in presence of USE flag. Now it's always slotted even with USE=-multislot (at least eix thinks so). -- Sergei [31;01m*[0m ERROR: dev-java/ibm-jdk-bin-1.6.0.9_p1::gentoo failed (depend phase): [31;01m*[0m use() calls are not allowed in global scope [31;01m*[0m [31;01m*[0m Call stack: [31;01m*[0m ebuild.sh, line 584: Called source '/gentoo-32k/gentoo-x86/dev-java/ibm-jdk-bin/ibm-jdk-bin-1.6.0.9_p1.ebuild' [31;01m*[0m ERROR: dev-java/ibm-jdk-bin-1.6.0.9_p2::gentoo failed (depend phase): [31;01m*[0m use() calls are not allowed in global scope [31;01m*[0m ibm-jdk-bin-1.6.0.9_p1.ebuild, line 50: Called use 'x86' [31;01m*[0m [31;01m*[0m Call stack: [31;01m*[0m ebuild.sh, line 47: Called die [31;01m*[0m The specific snippet of code: [31;01m*[0m ebuild.sh, line 584: Called source '/gentoo-32k/gentoo-x86/dev-java/ibm-jdk-bin/ibm-jdk-bin-1.6.0.9_p2.ebuild' [31;01m*[0m # These functions die because calls to them during the "depend" phase [31;01m*[0m ERROR: dev-java/ibm-jre-bin-1.6.0.9_p1::gentoo failed (depend phase): [31;01m*[0m [31;01m*[0m use() calls are not allowed in global scope [31;01m*[0m [31;01m*[0m ibm-jdk-bin-1.6.0.9_p2.ebuild, line 50: Called use 'x86' [31;01m*[0m Call stack: [31;01m*[0m If you need support, post the output of `emerge --info '=dev-java/ibm-jdk-bin-1.6.0.9_p1::gentoo'`, [31;01m*[0m the complete build log and the output of `emerge -pqv '=dev-java/ibm-jdk-bin-1.6.0.9_p1::gentoo'`. [31;01m*[0m ebuild.sh, line 47: Called die [31;01m*[0m The specific snippet of code: [31;01m*[0m Working directory: '/usr/lib64/python3.3/site-packages' [31;01m*[0m ERROR: dev-java/ibm-jre-bin-1.6.0.9_p2::gentoo failed (depend phase): [31;01m*[0m S: '/tmp/portage-tmpdir/portage/dev-java/ibm-jdk-bin-1.6.0.9_p1/work/ibm-jdk-bin-1.6.0.9_p1' [31;01m*[0m use() calls are not allowed in global scope [31;01m*[0m [31;01m*[0m # These functions die because calls to them during the "depend" phase [31;01m*[0m Call stack: [31;01m*[0m [31;01m*[0m If you need support, post the output of `emerge --info '=dev-java/ibm-jdk-bin-1.6.0.9_p2::gentoo'`, [31;01m*[0m the complete build log and the output of `emerge -pqv '=dev-java/ibm-jdk-bin-1.6.0.9_p2::gentoo'`. [31;01m*[0m ebuild.sh, line 584: Called source '/gentoo-32k/gentoo-x86/dev-java/ibm-jre-bin/ibm-jre-bin-1.6.0.9_p2.ebuild' [31;01m*[0m ebuild.sh, line 584: Called source '/gentoo-32k/gentoo-x86/dev-java/ibm-jre-bin/ibm-jre-bin-1.6.0.9_p1.ebuild' [31;01m*[0m Working directory: '/usr/lib64/python3.3/site-packages' [31;01m*[0m S: '/tmp/portage-tmpdir/portage/dev-java/ibm-jdk-bin-1.6.0.9_p2/work/ibm-jdk-bin-1.6.0.9_p2' [31;01m*[0m ibm-jre-bin-1.6.0.9_p2.ebuild, line 31: Called use 'x86' [31;01m*[0m ibm-jre-bin-1.6.0.9_p1.ebuild, line 31: Called use 'x86' [31;01m*[0m ebuild.sh, line 47: Called die [31;01m*[0m ebuild.sh, line 47: Called die [31;01m*[0m The specific snippet of code: [31;01m*[0m The specific snippet of code: [31;01m*[0m # These functions die because calls to them during the "depend" phase [31;01m*[0m # These functions die because calls to them during the "depend" phase [31;01m*[0m [31;01m*[0m [31;01m*[0m If you need support, post the output of `emerge --info '=dev-java/ibm-jre-bin-1.6.0.9_p2::gentoo'`, [31;01m*[0m If you need support, post the output of `emerge --info '=dev-java/ibm-jre-bin-1.6.0.9_p1::gentoo'`, [31;01m*[0m the complete build log and the output of `emerge -pqv '=dev-java/ibm-jre-bin-1.6.0.9_p2::gentoo'`. [31;01m*[0m the complete build log and the output of `emerge -pqv '=dev-java/ibm-jre-bin-1.6.0.9_p1::gentoo'`. [31;01m*[0m Working directory: '/usr/
[gentoo-portage-dev] Re: [PATCH] emake: pass EXTRA_EMAKE after user's "$@", not before
On Fri, 25 Jul 2014 01:11:47 +0300 Sergei Trofimovich wrote: ping :] > Patch allows easier override or ebuild things, like > #foo-.ebuild: > src_compile() { > emake V=1 > } > > EXTRA_EMAKE="V=0" emerge =foo- > > That way it's easier to eyeball upstream builds for new warnings. > > This behaviour is more in line with EXTRA_ECONF variable. > > Signed-off-by: Sergei Trofimovich > --- > bin/ebuild-helpers/emake | 4 ++-- > bin/phase-helpers.sh | 8 > 2 files changed, 6 insertions(+), 6 deletions(-) > > diff --git a/bin/ebuild-helpers/emake b/bin/ebuild-helpers/emake > index 69d836f..4618053 100755 > --- a/bin/ebuild-helpers/emake > +++ b/bin/ebuild-helpers/emake > @@ -13,7 +13,7 @@ source > "${PORTAGE_BIN_PATH:-/usr/lib/portage/bin}"/isolated-functions.sh > > if [[ $PORTAGE_QUIET != 1 ]] ; then > ( > - for arg in ${MAKE:-make} $MAKEOPTS $EXTRA_EMAKE "$@" ; do > + for arg in ${MAKE:-make} $MAKEOPTS "$@" $EXTRA_EMAKE ; do > [[ ${arg} == *" "* ]] \ > && printf "'%s' " "${arg}" \ > || printf "%s " "${arg}" > @@ -22,7 +22,7 @@ if [[ $PORTAGE_QUIET != 1 ]] ; then > ) >&2 > fi > > -${MAKE:-make} ${MAKEOPTS} ${EXTRA_EMAKE} "$@" > +${MAKE:-make} ${MAKEOPTS} "$@" ${EXTRA_EMAKE} > ret=$? > [[ $ret -ne 0 ]] && __helpers_die "${0##*/} failed" > exit $ret > diff --git a/bin/phase-helpers.sh b/bin/phase-helpers.sh > index 412decb..47bd843 100644 > --- a/bin/phase-helpers.sh > +++ b/bin/phase-helpers.sh > @@ -617,8 +617,8 @@ einstall() { > mandir="${ED}usr/share/man" \ > sysconfdir="${ED}etc" \ > ${LOCAL_EXTRA_EINSTALL} \ > - ${MAKEOPTS} ${EXTRA_EMAKE} -j1 \ > - "$@" install > + ${MAKEOPTS} -j1 \ > + "$@" ${EXTRA_EMAKE} install > fi > ${MAKE:-make} prefix="${ED}usr" \ > datadir="${ED}usr/share" \ > @@ -627,8 +627,8 @@ einstall() { > mandir="${ED}usr/share/man" \ > sysconfdir="${ED}etc" \ > ${LOCAL_EXTRA_EINSTALL} \ > - ${MAKEOPTS} ${EXTRA_EMAKE} -j1 \ > - "$@" install || die "einstall failed" > + ${MAKEOPTS} -j1 \ > + "$@" ${EXTRA_EMAKE} install || die "einstall failed" > else > die "no Makefile found" > fi > -- > 2.0.0 > -- Sergei signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] QA: use pngfix to find broken PNG files.
On Sat, 26 Jul 2014 08:50:43 +0200 Michał Górny wrote: > + if [[ -z ${next} ]] ; then > + eqawarn "QA Notice: broken .png files > found:" I guess you planned to plug 'next=yes' here, right? > + fi -- Sergei signature.asc Description: PGP signature
[gentoo-portage-dev] [PATCH] repoman: warn when herd's email appears in section
Manuel Rüger noticed that most of haskell packages's 'metadata.xml' contain duplicate information: haskell hask...@gentoo.org I've added a check against 'herds.xml's email aliases. Now repoman warns about such redundancy: metadata.warning 1 dev-haskell/text/metadata.xml: use haskell instead of maintainer 'hask...@gentoo.org' Quick scan [1] of tree revealed a lot of non-haskell packages having redundancy: samba, xemacs, sci, gpe, etc, etc. [1]: https://github.com/trofi/gentoo-qa/blob/master/check_herd.sh Run in the root tree of gentoo-x86: gentoo-x86 $ ~/portage/gentoo-qa/check_herd.sh | wc -l 571 # some examples: gentoo-x86 $ ~/portage/gentoo-qa/check_herd.sh ./app-admin/haskell-updater/metadata.xml: hask...@gentoo.org ./app-editors/xemacs/metadata.xml:xem...@gentoo.org ./app-i18n/ibus-table-chinese/metadata.xml: c...@gentoo.org ./app-portage/fquery/metadata.xml: hask...@gentoo.org ./app-text/glosung/metadata.xml: theol...@gentoo.org ... # a lot of haskell ./dev-lang/tcl/metadata.xml:tc...@gentoo.org ./dev-libs/Ice/metadata.xml:c...@gentoo.org ./dev-libs/cloog/metadata.xml: toolch...@gentoo.org ./dev-libs/cvector/metadata.xml:s...@gentoo.org ./dev-libs/iniparser/metadata.xml: sa...@gentoo.org ./dev-libs/isl/metadata.xml:toolch...@gentoo.org ... Signed-off-by: Sergei Trofimovich --- bin/repoman | 13 + pym/repoman/herdbase.py | 21 + pym/repoman/utilities.py | 19 +++ 3 files changed, 49 insertions(+), 4 deletions(-) diff --git a/bin/repoman b/bin/repoman index c36ace1..74a4d44 100755 --- a/bin/repoman +++ b/bin/repoman @@ -1762,6 +1762,19 @@ for x in effective_scanlist: fails["metadata.bad"].append("%s/metadata.xml: %s" % (x, e)) del e + # check if 'metadata.xml' contains redundant + # 'some-h...@gentoo.org' + # email address. Instead it should contain + # 'some-herd' + if herd_base is not None: + for m_email in utilities.get_maintainer_emails_from_metadata(_metadata_xml): + herd_name = herd_base.herd_by_herd_email(m_email) + if herd_name is not None: + stats["metadata.warning"] += 1 + fails["metadata.warning"].append("%s/metadata.xml:" + " use %s instead of maintainer '%s'" % \ + (x, herd_name, m_email)) + # Only carry out if in package directory or check forced if xmllint_capable and not metadata_bad: # xmlint can produce garbage output even on success, so only dump diff --git a/pym/repoman/herdbase.py b/pym/repoman/herdbase.py index c5b88ff..ae7c4e4 100644 --- a/pym/repoman/herdbase.py +++ b/pym/repoman/herdbase.py @@ -34,9 +34,10 @@ def _make_email(nick_name): class HerdBase(object): - def __init__(self, herd_to_emails, all_emails): + def __init__(self, herd_to_emails, all_emails, herd_email_to_herd): self.herd_to_emails = herd_to_emails self.all_emails = all_emails + self.herd_email_to_herd = herd_email_to_herd def known_herd(self, herd_name): return herd_name in self.herd_to_emails @@ -47,6 +48,9 @@ class HerdBase(object): def maintainer_in_herd(self, nick_name, herd_name): return _make_email(nick_name) in self.herd_to_emails[herd_name] + def herd_by_herd_email(self, email): + return self.herd_email_to_herd.get(email) + class _HerdsTreeBuilder(xml.etree.ElementTree.TreeBuilder): """ Implements doctype() as required to avoid deprecation warnings with @@ -57,6 +61,7 @@ class _HerdsTreeBuilder(xml.etree.ElementTree.TreeBuilder): def make_herd_base(filename): herd_to_emails = dict() + herd_email_to_herd = dict() all_emails = set() try: @@ -82,6 +87,11 @@ def make_herd_base(filename): herd_name = _herd_name.text.strip() del _herd_name + _herd_email = h.find('email') + herd_email = _herd_email.text.strip() + del _herd_email + herd_email_to_herd[herd_email] = herd_name + maintainers = h.findall('maintainer') herd_emails = set() for m
[gentoo-portage-dev] [PATCH] emake: pass EXTRA_EMAKE after user's "$@", not before
Patch allows easier override or ebuild things, like #foo-.ebuild: src_compile() { emake V=1 } EXTRA_EMAKE="V=0" emerge =foo- That way it's easier to eyeball upstream builds for new warnings. This behaviour is more in line with EXTRA_ECONF variable. Signed-off-by: Sergei Trofimovich --- bin/ebuild-helpers/emake | 4 ++-- bin/phase-helpers.sh | 8 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/bin/ebuild-helpers/emake b/bin/ebuild-helpers/emake index 69d836f..4618053 100755 --- a/bin/ebuild-helpers/emake +++ b/bin/ebuild-helpers/emake @@ -13,7 +13,7 @@ source "${PORTAGE_BIN_PATH:-/usr/lib/portage/bin}"/isolated-functions.sh if [[ $PORTAGE_QUIET != 1 ]] ; then ( - for arg in ${MAKE:-make} $MAKEOPTS $EXTRA_EMAKE "$@" ; do + for arg in ${MAKE:-make} $MAKEOPTS "$@" $EXTRA_EMAKE ; do [[ ${arg} == *" "* ]] \ && printf "'%s' " "${arg}" \ || printf "%s " "${arg}" @@ -22,7 +22,7 @@ if [[ $PORTAGE_QUIET != 1 ]] ; then ) >&2 fi -${MAKE:-make} ${MAKEOPTS} ${EXTRA_EMAKE} "$@" +${MAKE:-make} ${MAKEOPTS} "$@" ${EXTRA_EMAKE} ret=$? [[ $ret -ne 0 ]] && __helpers_die "${0##*/} failed" exit $ret diff --git a/bin/phase-helpers.sh b/bin/phase-helpers.sh index 412decb..47bd843 100644 --- a/bin/phase-helpers.sh +++ b/bin/phase-helpers.sh @@ -617,8 +617,8 @@ einstall() { mandir="${ED}usr/share/man" \ sysconfdir="${ED}etc" \ ${LOCAL_EXTRA_EINSTALL} \ - ${MAKEOPTS} ${EXTRA_EMAKE} -j1 \ - "$@" install + ${MAKEOPTS} -j1 \ + "$@" ${EXTRA_EMAKE} install fi ${MAKE:-make} prefix="${ED}usr" \ datadir="${ED}usr/share" \ @@ -627,8 +627,8 @@ einstall() { mandir="${ED}usr/share/man" \ sysconfdir="${ED}etc" \ ${LOCAL_EXTRA_EINSTALL} \ - ${MAKEOPTS} ${EXTRA_EMAKE} -j1 \ - "$@" install || die "einstall failed" + ${MAKEOPTS} -j1 \ + "$@" ${EXTRA_EMAKE} install || die "einstall failed" else die "no Makefile found" fi -- 2.0.0
[gentoo-portage-dev] [PATCH] repoman: warn when herd's email appears in section
Manuel Rüger noticed that most of haskell packages's 'metadata.xml' contain duplicate information: haskell hask...@gentoo.org I've added a check against 'herds.xml's email aliases. Now repoman warns about such redundancy: metadata.warning 1 dev-haskell/text/metadata.xml: use haskell instead of maintainer 'hask...@gentoo.org' Quick scan [1] of tree revealed a lot of non-haskell packages having redundancy: samba, xemacs, sci, gpe, etc, etc. [1]: https://github.com/trofi/gentoo-qa/blob/master/check_herd.sh Run in the root tree of gentoo-x86: gentoo-x86 $ ~/portage/gentoo-qa/check_herd.sh | wc -l 571 # some examples: gentoo-x86 $ ~/portage/gentoo-qa/check_herd.sh ./app-admin/haskell-updater/metadata.xml: hask...@gentoo.org ./app-editors/xemacs/metadata.xml:xem...@gentoo.org ./app-i18n/ibus-table-chinese/metadata.xml: c...@gentoo.org ./app-portage/fquery/metadata.xml: hask...@gentoo.org ./app-text/glosung/metadata.xml: theol...@gentoo.org ... # a lot of haskell ./dev-lang/tcl/metadata.xml:tc...@gentoo.org ./dev-libs/Ice/metadata.xml:c...@gentoo.org ./dev-libs/cloog/metadata.xml: toolch...@gentoo.org ./dev-libs/cvector/metadata.xml:s...@gentoo.org ./dev-libs/iniparser/metadata.xml: sa...@gentoo.org ./dev-libs/isl/metadata.xml:toolch...@gentoo.org ... Signed-off-by: Sergei Trofimovich --- bin/repoman | 13 + pym/repoman/herdbase.py | 21 + pym/repoman/utilities.py | 19 +++ 3 files changed, 49 insertions(+), 4 deletions(-) diff --git a/bin/repoman b/bin/repoman index c36ace1..74a4d44 100755 --- a/bin/repoman +++ b/bin/repoman @@ -1762,6 +1762,19 @@ for x in effective_scanlist: fails["metadata.bad"].append("%s/metadata.xml: %s" % (x, e)) del e + # check if 'metadata.xml' contains redundant + # 'some-h...@gentoo.org' + # email address. Instead it should contain + # 'some-herd' + if herd_base is not None: + for m_email in utilities.get_maintainer_emails_from_metadata(_metadata_xml): + herd_name = herd_base.herd_by_herd_email(m_email) + if herd_name is not None: + stats["metadata.warning"] += 1 + fails["metadata.warning"].append("%s/metadata.xml:" + " use %s instead of maintainer '%s'" % \ + (x, herd_name, m_email)) + # Only carry out if in package directory or check forced if xmllint_capable and not metadata_bad: # xmlint can produce garbage output even on success, so only dump diff --git a/pym/repoman/herdbase.py b/pym/repoman/herdbase.py index c5b88ff..ae7c4e4 100644 --- a/pym/repoman/herdbase.py +++ b/pym/repoman/herdbase.py @@ -34,9 +34,10 @@ def _make_email(nick_name): class HerdBase(object): - def __init__(self, herd_to_emails, all_emails): + def __init__(self, herd_to_emails, all_emails, herd_email_to_herd): self.herd_to_emails = herd_to_emails self.all_emails = all_emails + self.herd_email_to_herd = herd_email_to_herd def known_herd(self, herd_name): return herd_name in self.herd_to_emails @@ -47,6 +48,9 @@ class HerdBase(object): def maintainer_in_herd(self, nick_name, herd_name): return _make_email(nick_name) in self.herd_to_emails[herd_name] + def herd_by_herd_email(self, email): + return self.herd_email_to_herd.get(email) + class _HerdsTreeBuilder(xml.etree.ElementTree.TreeBuilder): """ Implements doctype() as required to avoid deprecation warnings with @@ -57,6 +61,7 @@ class _HerdsTreeBuilder(xml.etree.ElementTree.TreeBuilder): def make_herd_base(filename): herd_to_emails = dict() + herd_email_to_herd = dict() all_emails = set() try: @@ -82,6 +87,11 @@ def make_herd_base(filename): herd_name = _herd_name.text.strip() del _herd_name + _herd_email = h.find('email') + herd_email = _herd_email.text.strip() + del _herd_email + herd_email_to_herd[herd_email] = herd_name + maintainers = h.findall('maintainer') herd_emails = set() for m
[gentoo-portage-dev] [PATCH] emake: pass EXTRA_EMAKE after user's "$@", not before
Patch allows easier override or ebuild things, like #foo-.ebuild: src_compile() { emake V=1 } EXTRA_EMAKE="V=0" emerge =foo- That way it's easier to eyeball upstream builds for new warnings. This behaviour is more in line with EXTRA_ECONF variable. Signed-off-by: Sergei Trofimovich --- bin/ebuild-helpers/emake | 4 ++-- bin/phase-helpers.sh | 8 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/bin/ebuild-helpers/emake b/bin/ebuild-helpers/emake index 69d836f..4618053 100755 --- a/bin/ebuild-helpers/emake +++ b/bin/ebuild-helpers/emake @@ -13,7 +13,7 @@ source "${PORTAGE_BIN_PATH:-/usr/lib/portage/bin}"/isolated-functions.sh if [[ $PORTAGE_QUIET != 1 ]] ; then ( - for arg in ${MAKE:-make} $MAKEOPTS $EXTRA_EMAKE "$@" ; do + for arg in ${MAKE:-make} $MAKEOPTS "$@" $EXTRA_EMAKE ; do [[ ${arg} == *" "* ]] \ && printf "'%s' " "${arg}" \ || printf "%s " "${arg}" @@ -22,7 +22,7 @@ if [[ $PORTAGE_QUIET != 1 ]] ; then ) >&2 fi -${MAKE:-make} ${MAKEOPTS} ${EXTRA_EMAKE} "$@" +${MAKE:-make} ${MAKEOPTS} "$@" ${EXTRA_EMAKE} ret=$? [[ $ret -ne 0 ]] && __helpers_die "${0##*/} failed" exit $ret diff --git a/bin/phase-helpers.sh b/bin/phase-helpers.sh index 412decb..47bd843 100644 --- a/bin/phase-helpers.sh +++ b/bin/phase-helpers.sh @@ -617,8 +617,8 @@ einstall() { mandir="${ED}usr/share/man" \ sysconfdir="${ED}etc" \ ${LOCAL_EXTRA_EINSTALL} \ - ${MAKEOPTS} ${EXTRA_EMAKE} -j1 \ - "$@" install + ${MAKEOPTS} -j1 \ + "$@" ${EXTRA_EMAKE} install fi ${MAKE:-make} prefix="${ED}usr" \ datadir="${ED}usr/share" \ @@ -627,8 +627,8 @@ einstall() { mandir="${ED}usr/share/man" \ sysconfdir="${ED}etc" \ ${LOCAL_EXTRA_EINSTALL} \ - ${MAKEOPTS} ${EXTRA_EMAKE} -j1 \ - "$@" install || die "einstall failed" + ${MAKEOPTS} -j1 \ + "$@" ${EXTRA_EMAKE} install || die "einstall failed" else die "no Makefile found" fi -- 2.0.0
Re: [gentoo-portage-dev] [PATCH] Introduce depgraph._extend_slot_operator_conflicts
On Thu, 27 Feb 2014 20:08:49 +0100 Sebastian Luther wrote: > This function allows emerge to solve more slot conflicts without > backtracking. > > It does this by creating more conflicts for packages with built slot > operator dependencies. This gives more freedom to > depgraph._solve_non_slot_operator_slot_conflicts, which is then able > to solve conflicts it wouldn't have otherwise. > --- Nice patch! For haskell packages it makes huge difference. My typical workflow is: - bump some package foo and it's subslot - emerge -1 foo -j9 - see what will get rebuilt/broken by subslot rebuild If package would trigger 500 rebuilds portage had no chance to find a solution with backtracks. Now sometimes it does. Thanks! > pym/_emerge/depgraph.py| 483 > +++-- > pym/_emerge/resolver/output.py | 5 +- > 2 files changed, 415 insertions(+), 73 deletions(-) > > diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py > index 835bd6b..970a9c7 100644 > --- a/pym/_emerge/depgraph.py > +++ b/pym/_emerge/depgraph.py > @@ -427,6 +427,12 @@ class _dynamic_depgraph_config(object): > # Track missed updates caused by solved conflicts. > self._conflict_missed_update = collections.defaultdict(dict) > > + # Rebuilds caused by slot conflicts. > + self._slot_conflict_rebuilds = {} > + # Rebuilds caused by missed slot operator updates or > + # slot conflicts. > + self._forced_rebuilds = None > + > for myroot in depgraph._frozen_config.trees: > self.sets[myroot] = _depgraph_sets() > vardb = > depgraph._frozen_config.trees[myroot]["vartree"].dbapi > @@ -614,6 +620,9 @@ class depgraph(object): > Fill self._forced_rebuilds with packages that cause rebuilds. > """ > > + if self._dynamic_config._forced_rebuilds is not None: > + return > + > debug = "--debug" in self._frozen_config.myopts > > # Get all atoms that might have caused a forced rebuild. > @@ -687,19 +696,23 @@ class depgraph(object): > writemsg_level("\n\n", > level=logging.DEBUG, noiselevel=-1) > > - self._forced_rebuilds = forced_rebuilds > + for child, parents in > self._dynamic_config._slot_conflict_rebuilds.items(): > + forced_rebuilds.setdefault(child.root, > {}).setdefault(child, set()).update(parents) > + > + self._dynamic_config._forced_rebuilds = forced_rebuilds > > def _show_abi_rebuild_info(self): > > - if not self._forced_rebuilds: > + forced_rebuilds = self._dynamic_config._forced_rebuilds > + if not forced_rebuilds: > return > > writemsg_stdout("\nThe following packages are causing > rebuilds:\n\n", noiselevel=-1) > > - for root in self._forced_rebuilds: > - for child in self._forced_rebuilds[root]: > + for root in forced_rebuilds: > + for child in forced_rebuilds[root]: > writemsg_stdout(" %s causes rebuilds for:\n" % > (child,), noiselevel=-1) > - for parent in > self._forced_rebuilds[root][child]: > + for parent in forced_rebuilds[root][child]: > writemsg_stdout("%s\n" % (parent,), > noiselevel=-1) > > def _show_ignored_binaries(self): > @@ -968,16 +981,16 @@ class depgraph(object): > writemsg(line + '\n', noiselevel=-1) > writemsg('\n', noiselevel=-1) > > - def _solve_non_slot_operator_slot_conflicts(self): > + > + def _extend_slot_operator_conflicts(self): > """ > - This function solves slot conflicts which can > - be solved by simply choosing one of the conflicting > - and removing all the other ones. > - It is able to solve somewhat more complex cases where > - conflicts can only be solved simultaniously. > + This function searches for packages that cause > + slot conflicts by dependening on conflict packages > + with built slot operator deps. If such a package > + is found an alternative package is pulled in with > + the hope that the alternative package would not > + cuase the slot conflict. > """ > - debug = "--debug" in self._frozen_config.myopts > - > # List all conflicts. Ignore those that involve slot operator > rebuilds > # as the logic there needs special slot conflict behavior which > isn't > # provided by this function. > @@ -990,9 +1003,133 @@ class depgraph(object): > if not conflicts
Re: [gentoo-portage-dev] [PATCH v2] prepman: do not compress files <=128 bytes
On Thu, 21 Mar 2013 07:51:49 -0400 James Cloos wrote: > > "MF" == Mike Frysinger writes: > > MF> +SIZE_LIMIT='128' > > The limit probably should be larger, given that nothing which fits into > one filesystem block will use less space compressed. While it's usually true btrfs does not store things in blocks and allocates space in byte ranges. All filesystem (data or meta) modifications are copy-on-write thus it does not needs "growth holes" by design. -- Sergei signature.asc Description: PGP signature