Hi Chun-Yu, On 26-06-2020 23:34:12 -0700, Chun-Yu Shei wrote: > Hi, > > I was recently interested in whether portage could be speed up, since > dependency resolution can sometimes take a while on slower machines. > After generating some flame graphs with cProfile and vmprof, I found 3 > functions which seem to be called extremely frequently with the same > arguments: catpkgsplit, use_reduce, and match_from_list. In the first > two cases, it was simple to cache the results in dicts, while > match_from_list was a bit trickier, since it seems to be a requirement > that it return actual entries from the input "candidate_list". I also > ran into some test failures if I did the caching after the > mydep.unevaluated_atom.use and mydep.repo checks towards the end of the > function, so the caching is only done up to just before that point. > > The catpkgsplit change seems to definitely be safe, and I'm pretty sure > the use_reduce one is too, since anything that could possibly change the > result is hashed. I'm a bit less certain about the match_from_list one, > although all tests are passing. > > With all 3 patches together, "emerge -uDvpU --with-bdeps=y @world" > speeds up from 43.53 seconds to 30.96 sec -- a 40.6% speedup. "emerge > -ep @world" is just a tiny bit faster, going from 18.69 to 18.22 sec > (2.5% improvement). Since the upgrade case is far more common, this > would really help in daily use, and it shaves about 30 seconds off > the time you have to wait to get to the [Yes/No] prompt (from ~90s to > 60s) on my old Sandy Bridge laptop when performing normal upgrades. > > Hopefully, at least some of these patches can be incorporated, and please > let me know if any changes are necessary.
This sounds like a good job to me! Do you have any idea what the added memory pressure for these changes are? Thanks, Fabian > > Thanks, > Chun-Yu > > > -- Fabian Groffen Gentoo on a different level
signature.asc
Description: PGP signature
