[gentoo-dev] Re: rfc: escape sequences in logs
William Hubbs writes: > On Mon, Sep 02, 2013 at 09:41:28PM +0200, Michał Górny wrote: >> That is a bug in pybugz and not an argument, you know. > > I said "things like pybugz". > > Bugzilla allowing control characters in the xml is the issue. The python > xmlrpc library raises an exception for malformed xml because of it, so > there isn't much pybugz, or anything that uses that library, can do > about it. Right. The bug is in bugzilla itself, not pybugz. I did some research. Escape and all of the other non-whitespace ASCII control characters are illegal in XML. It is also not valid to escape them with entities, like . However, Bugzilla's XML-RPC interface sends them anyway. This is only a problem for build logs included inline in bug comments. Attached logs are fine, with or without the color codes. Maybe the codes could be stripped by pybugz before processing the XML? -- Chris
[gentoo-dev] Re: rfc: escape sequences in logs
Ulrich Mueller posted on Tue, 03 Sep 2013 15:01:28 +0200 as excerpted: > [Please check your e-mail agent. The "References:" header in your > posting was broken.] Thanks. Upstream is aware of the bug but hasn't patched it yet. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
[gentoo-dev] Re: rfc: escape sequences in logs
> On Tue, 3 Sep 2013, Duncan wrote: [Please check your e-mail agent. The "References:" header in your posting was broken.] > Zac mentioned in a different subthread that portage's logger is > single-threaded ATM, and could become a bottleneck if it had to do > too much output processing. > How feasible might it be to make something like this a (possibly > optional but presumably enabled by default) portage dep, and set > portage's default logging to use it when writing the log file (as > opposed to the screen output)? Presumably that would pipe to another > app to do the actual logfile filtering and writing, so the threading > issue would disappear. > Actually, presuming there's already a python implementation to > parallel that perl one, it's possible the only thing that would need > changed would be portage's default logging command. Pythonistas? Isn't it just a matter of regexp matching? For example, Emacs' ansi-color.el discards any string that matches \[\([ABCDsuK]\|2J\|=[0-9]+[hI]\|[0-9;]*[Hfm]\) and it seems to work in the common cases. Ulrich
[gentoo-dev] Re: rfc: escape sequences in logs
Kent Fredric posted on Tue, 03 Sep 2013 11:59:13 +1200 as excerpted: > And it is quite simple to remove escape sequences from log files if I > desire it. > > using: app-text/ansifilter > > zcat > /var/log/portage/build/app-accessibility/at-spi2- core-2.6.3:20130825-143158.log.gz > | ansifilter > > using perl and Term::ANSIColor ( standard issue with Perl itself ) > > zcat > /var/log/portage/build/app-accessibility/at-spi2- core-2.6.3:20130825-143158.log.gz > | perl -MTerm::ANSIColor=colorstrip -ple '$_ = colorstrip($_)' This has long bothered me, but not enough to be worth investigating myself... Zac mentioned in a different subthread that portage's logger is single- threaded ATM, and could become a bottleneck if it had to do too much output processing. How feasible might it be to make something like this a (possibly optional but presumably enabled by default) portage dep, and set portage's default logging to use it when writing the log file (as opposed to the screen output)? Presumably that would pipe to another app to do the actual logfile filtering and writing, so the threading issue would disappear. Actually, presuming there's already a python implementation to parallel that perl one, it's possible the only thing that would need changed would be portage's default logging command. Pythonistas? -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
