Portage code is horrid about not properly using iterators/generators. Check to ensure -all- calling code can handle the returned iterator, and it's a possibility for stable if other portage devs agree.
Meanwhile, it will be go into head in a way... the db rewrite I'm working on is built from the ground up to (ab|)use generators/iterators :) ~Brian On Wed, Feb 16, 2005 at 06:36:38PM +0100, J?rgen H?tzel wrote: > Hi, > > Because portage searches took so long on my notebook i started > profiling and found the bottleneck in > > class portdbapi::cp_all (/usr/lib/portage/pym/portage.py) > > a huge list is generated and returned. Its better to yield each single > entry. Generators are part of python since 2.3.0. I wonder why they > are not used in portage. Look at the performance improvement of my patch: > > bash-2.05b# time emerge search python-mode > Searching... > [ Results for search key : python-mode ] > [ Applications found : 2 ] > .. > real 0m6.845s > user 0m6.536s > sys 0m0.137s > > After applying the patch: > > bash-2.05b# time emerge search python-mode > Searching... > [ Results for search key : python-mode ] > [ Applications found : 2 ] > .. > real 0m2.323s > user 0m1.914s > sys 0m0.142s > > Should i submit my patch too bugs.gentoo.org? > > Juergen > --- portage_old.py 2005-02-16 17:15:39.731629944 +0100 > +++ portage.py 2005-02-16 16:57:28.755483528 +0100 > @@ -5367,15 +5367,15 @@ > return 0 > > def cp_all(self): > - "returns a list of all keys in our tree" > - biglist=[] > + "yield all keys in our tree" > for x in self.mysettings.categories: > + biglist=[] > for oroot in self.porttrees: > for y in > listdir(oroot+"/"+x,EmptyOnError=1,ignorecvs=1): > mykey=x+"/"+y > if not mykey in biglist: > biglist.append(mykey) > - return biglist > + yield(mykey) > > def p_list(self,mycp): > returnme=[] > > -- > [email protected] mailing list -- [email protected] mailing list
