On 02/26/2018 10:16 AM, Alec Warner wrote: > > > On Mon, Feb 26, 2018 at 1:09 PM, Zac Medico <zmed...@gentoo.org > <mailto:zmed...@gentoo.org>> wrote: > > On 02/26/2018 04:29 AM, Michał Górny wrote: > > W dniu nie, 25.02.2018 o godzinie 17∶50 -0800, użytkownik Zac Medico > > napisał: > >> Add async_aux_get method that returns a Future and otherwise > >> behaves identically to aux_get. Use async_aux_get to implement > >> the synchronous aux_get method. > >> > >> Bug: https://bugs.gentoo.org/648790 > >> --- > >> pym/portage/dbapi/porttree.py | 91 > +++++++++++++++++++++++++++++++++---------- > >> 1 file changed, 70 insertions(+), 21 deletions(-) > >> > > > > What's the exact use case? What's the gain, in numbers? > > I'm planning to use the for repoman, since it commonly has to generate > metadata for multiple packages a once, when dealing with ebuild or > eclass modifications. > > The gain in numbers is that it scales linearly with the number of > available cores. > > > Daniel, any data on real world gains from the Funtoo implementation?
There's no doubt that it scales linearly, it uses exactly the same code as egencache --jobs. > > This seems like a lot of added complexity. > > It doesn't really add complexity, asynchronous code or this sort is > already used throughout portage internals. Asynchronous methods like > these are quite standard today, thanks to asyncio. > > > With Portage being practically unmaintainable, > > Any reasonably complex codebase appears to be unmaintainable to those > who don't maintain it. > > > I mean I'm generally with mgorny, but I don't think this change is that > complicated. > > The 'hard to maintain' stuff for me comes from: > > 1) really long functions (e.g >100 lines) with poor naming (x, my* > variables, etc.) > 2) portions of the codebase with no tests (more than I'd like...) > 3) The random spawl and layout of some components that just makes it > annoying to modify specific bits. > > While I agree that these are all a problem (and I've tried at various > times to tilt at these windmills to no avail) > I don't think this makes any of those worse. Agreed. > > I'm against any changes that make things worse without > > clearly defined major benefit. > > Transitioning to asynchronous interfaces like this is unavoidable as we > move into the realm of asyncio compatibility. > > > So to me this implies we will be adding even more asyncIO to try to > improve the performance of portage by utilizing more cores? Correct, and frankly any other python project that can benefit from multiprocessing should do the same thing. -- Thanks, Zac
signature.asc
Description: OpenPGP digital signature
