#20884: line buffering in sage-logger causes "hang" due to invisible prompt when
installing experimental packages
-------------------------------------+-------------------------------------
Reporter: mkoeppe | Owner:
Type: defect | Status: new
Priority: critical | Milestone: sage-7.3
Component: build | Resolution:
Keywords: | Merged in:
Authors: Erik Bray | Reviewers: Dima Pasechnik,
| Matthias Koeppe
Report Upstream: N/A | Work issues:
Branch: | Commit:
705573623266f1017aa4a433e816c0e878960836|
Dependencies: | Stopgaps:
-------------------------------------+-------------------------------------
Comment (by leif):
Replying to [comment:53 embray]:
> > How about simply changing the scripts to output a newline when
prompting the user for something, at least when the sage-logger is used?
>
> Those scripts don't necessarily have any way to know that `sage-logger`
is being used, nor should they.
While you could set some env var in "non-interactive" mode (or when the
sage-logger is used), what's the problem with having a newline after `"...
[Y/n]?"`?
\\
> I think the test of just `cat`ting is not quite fair either. In a real
build the overhead of the commands being run far outweighs the overhead of
the logger.
The timings I listed are (the lower bound of) ''consumed CPU time you get
in addition'' with every build from scratch (not taking into account a GCC
build btw.); on a single-core that's what adds to the wall time (plus
presumably the overhead of more context switches), and regardless of
whether you build any experimental package (or any other question is
asked).
\\
> There's no need to emphasize ''shell loop'' since it's using only shell
built-ins, or should be,
You were not using bash's built-in test (`[[ ... ]]`), but the speed-up
with the latter isn't huge either:
{{{
#!sh
$ time ./sage-logger-20884-bt -p "cat logs/install.log" test-nw2.log
>/dev/null
real 21m51.249s
user 21m15.210s
sys 0m36.170s
}}}
(i.e., ~22 minutes vs. previously ~25 minutes, saving about 3.5 minutes,
and all tests were performed on the same, a ''relatively fast'' machine;
note that `sys` though dropped by nearly one third.)
\\
> so that shouldn't be a significant source of overhead. Ultimately `sed`
is looping over one character at a time too.
But it's not an interpreter, it compiles the regular expressions instead
of rereading and interpreting them over and over, and btw. operates on
lines of input -- you've seen the difference (orders of magnitude).
\\
> [...] I might prefer to go with my earlier instinct and just rewrite it
in Python. I think using `sed` just to prepend to a line also feels like
a blunt instrument.
Please don't, `sed` is IMHO pretty adequate in lack of other portable
tools for this trivial task (prepending something to each line); regarding
that we meanwhile have separate log files for each package it's a bit
superfluous though, even for parallel builds. But ''as is'' (with `sed`),
it doesn't hurt much.
(If at all, I'd personally rewrite the whole sage-logger in C.)
--
Ticket URL: <https://trac.sagemath.org/ticket/20884#comment:61>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/sage-trac.
For more options, visit https://groups.google.com/d/optout.
