Zac Medico posted on Mon, 14 Nov 2011 07:22:19 -0800 as excerpted: > On 11/14/2011 12:47 AM, Dirkjan Ochtman wrote: >> On Mon, Nov 14, 2011 at 02:59, Zac Medico <[email protected]> wrote: >>> Well, it's much easier to gather interest and get feedback if we >>> deploy the change and ask questions later. >> >> I like the new output, but find it kind of annoying that there's very >> little feedback on how far the progress is within a single job. Perhaps >> we could show the currently executing ebuild phase in order to give a >> little more feedback? Maybe most packages are completely dominated by >> src_compile(), but for smaller packages it would be helpful, IMO. > > I think that would be an interesting option. I imagine that it would be > too much information for many people, so I don't think that we'd want to > enable it by default.
Thanks for thinking "out of the box". =:^) This is one thing that I've found frustrating with parallel emerging, myself. The first thing I discovered were that some phases (at least the merge phase, and I believe install) aren't counted, so the numbers don't entirely add up (complete + running + not-started + failed <> total). Thus, some packages appear to be simply sitting there doing nothing. When a whole bunch of packages are in that state, it appears emerge is doing nothing but consuming cycles and real-time, for no visible reason at all. Since for a number of packages (I seem to notice it most on kde packages, but perhaps it's all C++ or most CMAKE or some such, plus of course primarily data packages that don't have a significant build phase but install many or large files) the install phase can be as intense as the build phase if not more so, so if some phases aren't counted in "running", then I'd definitely prefer they be counted /somewhere/. Of course, listing the number of packages in each phase (which in single- mode would by definition list the phase the single package is in) would cure the above issue at the same time. Another problem, but one I'm not sure how to fix (tho the interactive display would offer opportunities here), is that it'd be nice to get a list of completed packages, in-process packages, and still not started packages. That doesn't so easily fit in a neat summary, but as noted, the interactive proposal introduces quite some opportunity, here. Alternatively, since often the real info desired is the status of a particular package, it'd be useful to have either an interactive command or a simple separate querying command, that can be run to query the status of a particular package. *** Which brings up an alternative to the whole interactive emerges idea as well -- what about a separate command, say "emerging", that when run, would simply output a bunch of detail on any pending emerges? This would certainly offer a safer initial implementation, as well, since it's less likely to break the current setup due to bugs, etc. Another noted benefit is that it leaves the existing defaults alone, so it's not going to be threatening anyone's sacred cows (or scripted assumptions) in terms of how portage has always behaved. =:^) So, perhaps improve the current "quiet" display marginally, either including all phases in "running" or adding another listing so the numbers add up, but more importantly... *** Add a new "emerging" (name up for bikeshedding) command, that displays a nice multiline summary, perhaps something like this (the second emerge command was for a failure in the first, with a second attempt made before completion of the main emerge @world has completed): Current emerge status: 2 emerge command(s) running. 7 emerge jobs outstanding. Commands: ** emerge --upgrade --deep --newuse --keep-going @world Status: 123 of 257 jobs complete, 1 failed, 6 running, 120 not yet started, 7 dependencies removed due to failures. Running (6): 1 pkgsetup (cat-egory/package-ver) 1 configure (cat-egory/package-ver, cat-egory/package-ver) 2 build (cat-egory/package-ver, cat-egory/package-ver) 1 install (cat-egory/package-ver) 1 merge (cat-egory/package-ver) Failed (1): dev-libs/boost-1.47.0-r1 Complete (123): <list in nice columns> ... Not started (120): <list in nice columns> ... Removed dependencies due to failure (7): <list in nice columns> ... ** emerge -u1 boost Status: 0 of 1 jobs complete, 1 running. Running (1): 1 build (dev-libs/boost-1.47.0-r1) Obviously the first implementation might not have all the information above, and there might have been some nice information I missed and other information that could be displayed better, but it's a very cool idea even if I /do/ say so myself. =:^) Note that zeroed-out listings aren't displayed, so the the --oneshot doesn't list failed, not yet started, etc, and inactive phases are also not displayed. Also note how easy this sort of detailed display would make it to retry failed jobs and the resulting removed dependencies, once the failure has been resolved. "emerging" could have a "repeat" mode as well as one-shot, with a polling- delay parameter set to something sane like 30 or 60 seconds by default. That way, I wouldn't have to feed it to "watch" =:^) Because this is a separate command, worries about TMI/TLI should be eliminated. Additionally, it should go some way toward easing the whole quiet/noisy debate as well, since there's now another command available with an intermediate level of information. Something like this: edisplaylog If we then throw in one more tool, I'll call it edisplaylog for lack of a better name, that reads make.conf to find the elog dir, and can automatically pick out the latest log for a package, thus eliminating the trouble of doing it manually, that should help eliminate another objection to quiet-by-default. So... edisplaylog boost ... would find the last elog for boost and display it. To make things easy for the user, simply ... edisplaylog ... could be made to filetime search and display the last active log in tail -f mode. If after say a couple seconds of no activity, it re-polled the log dir and switched to displaying a different log (have it list the log it's displaying at the top) if it was newer, that could pretty much replace "noisy" mode entirely. =:^) Obviously running simply "edisplaylog" by itself during a parallel-jobs emerge would be nearly as confusing as portage not automatically going into quiet mode for parallel emerging would be, tho not quite since it would display the logfile name at the top, but "edisplaylog <pkg>" would still be very useful, ESPECIALLY for those using --keep-going, so they could track down and fix whatever failed a build before the parallel emerge spit out the results at the end. Talking about which... an "edisplaylog failed" mode, which would automatically find the last failure and displayed the log for it, could be very helpful as well. =:^) Obviously all these ideas entail a lot of coding, and I realize it's easy to sit here and make requests without doing the coding, but implementing them as separate commands first and incremental implementation should help a lot, and I hadn't seen anything like this proposed yet, so... I'll close with a thanks to Zac and everyone else who has already made portage the great tool it is today, because without them, we'd obviously not be having this discussion in the first place. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
