Re: [gentoo-portage-dev] [PATCH] repoman: filter out duplicate dependencies in error messages
On 04 Jan 2016 19:17, Brian Dolbec wrote: > I immediately want to say REJECT!, REJECT!, REJECT!,... > ... > I would much prefer you re-base your patch on the rewrite code. reject != delay ... i don't mind waiting and rebasing -mike signature.asc Description: Digital signature
Re: [gentoo-portage-dev] [PATCH] repoman: filter out duplicate dependencies in error messages
On Mon, 4 Jan 2016 19:17:41 -0800 Brian Dolbecwrote: > On Mon, 4 Jan 2016 16:30:30 -0500 > Mike Frysinger wrote: > > > Some packages list the same atom multiple times (e.g. behind diff > > USE flags). If one of them throws an error, we end up listing it > > more than once, and the output can get verbose/useless. > > --- > > pym/repoman/scanner.py | 13 +++-- > > 1 file changed, 11 insertions(+), 2 deletions(-) > > > > diff --git a/pym/repoman/scanner.py b/pym/repoman/scanner.py > > index d1c10d7..94ada90 100644 > > --- a/pym/repoman/scanner.py > > +++ b/pym/repoman/scanner.py > > @@ -704,13 +704,22 @@ class Scanner(object): > > > > # > > we have some unsolvable deps # remove ! deps, which always show up > > as unsatisfiable > > - > > atoms = [ > > + > > all_atoms = [ str(atom.unevaluated_atom) > > for > > atom in atoms if not atom.blocker] > > # > > if we emptied out our list, continue: > > - if > > not atoms: > > + if > > not all_atoms: continue > > + > > + # > > Filter out duplicates. We do this by hand (rather > > + # > > than use a set) so the order is stable and better > > + # > > matches the order that's in the ebuild itself. > > + > > atoms = [] > > + for > > atom in all_atoms: > > + > > if atom not in atoms: > > + > > atoms.append(atom) + > > if > > self.options.output_style in ['column']: > > self.qatracker.add_error(mykey, "%s: %s: %s(%s) %s" > > > I immediately want to say REJECT!, REJECT!, REJECT!,... > > I just spent a marathon week working on stage2 of the repoman rewrite. > looks like this is now the pym/repoman/modules/scan/depend/profile.py check -- Brian Dolbec
[gentoo-portage-dev] [PATCH] repoman: filter out duplicate dependencies in error messages
Some packages list the same atom multiple times (e.g. behind diff USE flags). If one of them throws an error, we end up listing it more than once, and the output can get verbose/useless. --- pym/repoman/scanner.py | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/pym/repoman/scanner.py b/pym/repoman/scanner.py index d1c10d7..94ada90 100644 --- a/pym/repoman/scanner.py +++ b/pym/repoman/scanner.py @@ -704,13 +704,22 @@ class Scanner(object): # we have some unsolvable deps # remove ! deps, which always show up as unsatisfiable - atoms = [ + all_atoms = [ str(atom.unevaluated_atom) for atom in atoms if not atom.blocker] # if we emptied out our list, continue: - if not atoms: + if not all_atoms: continue + + # Filter out duplicates. We do this by hand (rather + # than use a set) so the order is stable and better + # matches the order that's in the ebuild itself. + atoms = [] + for atom in all_atoms: + if atom not in atoms: + atoms.append(atom) + if self.options.output_style in ['column']: self.qatracker.add_error(mykey, "%s: %s: %s(%s) %s" -- 2.6.2
Re: [gentoo-portage-dev] [PATCH] repoman: filter out duplicate dependencies in error messages
On Mon, 4 Jan 2016 16:30:30 -0500 Mike Frysingerwrote: > Some packages list the same atom multiple times (e.g. behind diff USE > flags). If one of them throws an error, we end up listing it more > than once, and the output can get verbose/useless. > --- > pym/repoman/scanner.py | 13 +++-- > 1 file changed, 11 insertions(+), 2 deletions(-) > > diff --git a/pym/repoman/scanner.py b/pym/repoman/scanner.py > index d1c10d7..94ada90 100644 > --- a/pym/repoman/scanner.py > +++ b/pym/repoman/scanner.py > @@ -704,13 +704,22 @@ class Scanner(object): > > # we > have some unsolvable deps # remove ! deps, which always show up as > unsatisfiable > - > atoms = [ > + > all_atoms = [ str(atom.unevaluated_atom) > for > atom in atoms if not atom.blocker] > # if > we emptied out our list, continue: > - if > not atoms: > + if > not all_atoms: continue > + > + # > Filter out duplicates. We do this by hand (rather > + # > than use a set) so the order is stable and better > + # > matches the order that's in the ebuild itself. > + > atoms = [] > + for > atom in all_atoms: > + > if atom not in atoms: > + > atoms.append(atom) + > if > self.options.output_style in ['column']: > self.qatracker.add_error(mykey, "%s: %s: %s(%s) %s" I immediately want to say REJECT!, REJECT!, REJECT!,... I just spent a marathon week working on stage2 of the repoman rewrite. I have all the checks and vcs related code in 2 plugin systems. I have to move the vcs plugins to their final destination path still (minor move) I am going to start cleaning up the commits, do some rebasing and unifying of them all now that I have it split up and working. Still some more testing/debugging to do. Hopefully be the end of this week it'll be ready for review and merge soon. If this is applied to current repoman, it may cause some rebase hell. https://github.com/dol-sen/portage/tree/repoman That is the repoman branch with the individualized checks that run in 3 small loops in scanner.py now. There is no code in scanner that does any checks. Those are all in pym/repoman/modules/scan/*/*.py. Some modules contain several different files and class definitions. There are a bunch of new ones that I created from the code that still remained in scanner.py's _scan_ebuilds(). I'll push the changes to teh main gentoo portage repo's repoman branch once I have it cleaned up. I would much prefer you re-base your patch on the rewrite code. I will make a wiki page for the module definition requirements, with a section on the vcs system as well. But the modules are quite simple, only small changes from the initial code split we did already. So new modules are easy to create and add in to the sequence of checks to perform. You just have to be careful where you insert checks. As the dynamic_data used and updated by the modules varies as it progresses through the sequence. I have yet to document the data changed/required by each of the modules. But they are quite clear looking at the code. -- Brian Dolbec
Re: [gentoo-portage-dev] [PATCH] repoman: filter out duplicate dependencies in error messages
On 01/04/2016 01:30 PM, Mike Frysinger wrote: > + # Filter out > duplicates. We do this by hand (rather > + # than use a > set) so the order is stable and better > + # matches the > order that's in the ebuild itself. > + atoms = [] > + for atom in > all_atoms: > + if atom > not in atoms: > + > atoms.append(atom) > + Alternative implementation using OrderedDict which has O(1) average complexity for containment checks, rather than O(N): atoms = OrderedDict() for atom in all_atoms: atoms.setdefault(atom, atom) atoms = list(atoms) -- Thanks, Zac