Re: [gentoo-dev] [RFC] Portage einfo, elog... output format change
On 2021-09-28, Ulrich Mueller wrote: > > On Tue, 28 Sep 2021, Michał Górny wrote: > > On Tue, 2021-09-28 at 18:26 +0200, Ulrich Mueller wrote: > >> Markers: (--) probed, (**) from config file, (==) default > >> setting, > >> (++) from command line, (!!) notice, (II) informational, > >> (WW) warning, (EE) error, (NI) not implemented, (??) unknown. > >> > >> So, maybe change einfo from [--] to [II]? > > > Nah, the whole point is to avoid letters since it's not very > > important. > > Alternatively to '[--]' I could make it look down '[..]' or maybe even > > without eyes entirely '[ ]'. > > Yeah, anything but [--]. The version with the dots is nice. I think any change from only-color would be an improvement; mgorny's first version looks nice. I can see why symmetry with Xorg EE, WW, etc. has an appeal but also the downsides of a) since things don't actually line up 1:1 it could be misleading to try; b) it's also somewhat language-biased. Something a lot of tools have started doing is [?] prefixes to their output; I don't know if there's even a semi-formal spec on it, but what seems to me to map well would be: [!] fatal error (die / eerror?) [*] important but nonfatal (ewarn?) [+] info, maybe memorable, but not harmful (elog?) [.] status/verbose message (einfo?) Could double that if preserving a 4-char-wide tag is preferred. That doesn't cover all needed, like eqawarn, but there are more to choose from. It's unfortunate that many of these are regex metacharacters, making slightly more (human) overhead when grepping, but we already have that with the [] delimeters. > > or maybe even without eyes entirely '[ ]'. For no reason I can articulate, the empty one bugs me. I would get over it though ;) Thanks, -- Hank Leininger 9606 3BF9 B593 4CBC E31A A384 6200 F6E3 781E 3DD7 signature.asc Description: Digital signature
[gentoo-dev] Package up for grabs: >=app-emulation/docker-compose-2.0.0 (1.x.x was Python, 2.0.0 is Go)
Hi! docker-compose upstream has apparently re-written docker-compose in Golang from scratch. While I'm happy to keep maintaining the current python-based =2.0.0 in Gentoo. Thanks in advance! Best Sebastian
Re: [gentoo-dev] [RFC] Portage einfo, elog... output format change
> On Tue, 28 Sep 2021, Michał Górny wrote: > On Tue, 2021-09-28 at 18:26 +0200, Ulrich Mueller wrote: >> Markers: (--) probed, (**) from config file, (==) default setting, >> (++) from command line, (!!) notice, (II) informational, >> (WW) warning, (EE) error, (NI) not implemented, (??) unknown. >> >> So, maybe change einfo from [--] to [II]? > Nah, the whole point is to avoid letters since it's not very important. > Alternatively to '[--]' I could make it look down '[..]' or maybe even > without eyes entirely '[ ]'. Yeah, anything but [--]. The version with the dots is nice. Ulrich signature.asc Description: PGP signature
Re: [gentoo-dev] [RFC] Portage einfo, elog... output format change
On Tue, 2021-09-28 at 18:26 +0200, Ulrich Mueller wrote: > > > > > > On Tue, 28 Sep 2021, Michał Górny wrote: > > > The proposed new format distinguished message types both using > > colors > > and strings. This is roughly inspired by Xorg logs. > > Using the same markers as Xorg (especially [--]) but with different > meaning may be confusing though. Xorg has these: > > Markers: (--) probed, (**) from config file, (==) default setting, > (++) from command line, (!!) notice, (II) informational, > (WW) warning, (EE) error, (NI) not implemented, (??) unknown. > > So, maybe change einfo from [--] to [II]? Nah, the whole point is to avoid letters since it's not very important. Alternatively to '[--]' I could make it look down '[..]' or maybe even without eyes entirely '[ ]'. -- Best regards, Michał Górny
Re: [gentoo-dev] [RFC] Portage einfo, elog... output format change
> On Tue, 28 Sep 2021, Michał Górny wrote: > The proposed new format distinguished message types both using colors > and strings. This is roughly inspired by Xorg logs. Using the same markers as Xorg (especially [--]) but with different meaning may be confusing though. Xorg has these: Markers: (--) probed, (**) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. So, maybe change einfo from [--] to [II]? signature.asc Description: PGP signature
Re: [gentoo-dev] [RFC] Portage einfo, elog... output format change
I love it - this is similar to the xorg log output, and i think it makes the portage output much cleaner and easier to read. El mar., 28 de septiembre de 2021 11:36 a. m., Michał Górny < mgo...@gentoo.org> escribió: > Hi, everyone. > > I know I'm going to regret asking this... but I've prepared a change to > the Portage output format and I think it asks for a wider discussion > than internally in Portage team. > > The primary problem with the current output format is that different > kinds of messages differ only in color. This makes them > indistinguishable without colors and hard to grep. ago's been asking > for a better way to grep for QA warnings and this is pretty much what > motivated me to do this. > > The proposed new format distinguished message types both using colors > and strings. This is roughly inspired by Xorg logs. For example, > instead of: > > * some message > * other message > * hell if i know what this is > > You get: > > [WW] some message > [EE] other message > [QA] hell if i know what this is > > I've also added more colors to explicitly distinguish einfo from elog, > and ewarn from eqawarn. Then, I've replaced most of '>>>' and '!!!' > used by Portage with four-character versions to keep messages aligned. > The 'zings' used for merging files remain three-character, so now it's > easily possible to distinguish messages from installed file list. > > The PR doing this is: https://github.com/gentoo/portage/pull/759 > > Example screenshot: > > https://user-images.githubusercontent.com/110765/135119090-16e9599d-1b0f-41b8-a965-a55577183ffd.png > > -- > Best regards, > Michał Górny > > > >
Re: [gentoo-portage-dev] [PATCH 0/4] Output rewrite for better clarify and greppability
On Tue, 2021-09-28 at 11:28 -0400, Mike Gilbert wrote: > On Tue, Sep 28, 2021 at 10:20 AM Michał Górny wrote: > > > > Hi, > > > > Ok, so it's more major than I originally intended but I think it's > > a good direction (once you get used to it). Roughly: > > > > 1. All bash color vars are now prefixed with `PORTAGE_COLOR_` to avoid > >accidental collisions with ebuild vars (e.g. ebuild setting > >`GOOD=foo` broke `elog` before). > > 2. There are specific color vars for all kinds of output functions, > >and now `einfo` messages use distinct color (dark green) from `elog`, > >and `eqawarn` (brown) from `ewarn`. > > 3. Messages are now prefixed by their kind, making it possible to > >distinguish them without colors and grep for specific kind of logs: > >- `[--]` for einfo & ebegin > >- `[II]` for elog > >- `[WW]` for ewarn > >- `[QA]` for eqawarn > >- `[EE]` for eerror > > 4. Finally, I've replaced most of `>>>` and `!!!` in Portage output with > >four `` and `` to align the output again. > > I like it. > > Maybe add a reference to this bug? https://bugs.gentoo.org/728046 > Oh right, I also need to work on removing redundant 'QA Notice:' thingies. -- Best regards, Michał Górny
[gentoo-dev] [RFC] Portage einfo, elog... output format change
Hi, everyone. I know I'm going to regret asking this... but I've prepared a change to the Portage output format and I think it asks for a wider discussion than internally in Portage team. The primary problem with the current output format is that different kinds of messages differ only in color. This makes them indistinguishable without colors and hard to grep. ago's been asking for a better way to grep for QA warnings and this is pretty much what motivated me to do this. The proposed new format distinguished message types both using colors and strings. This is roughly inspired by Xorg logs. For example, instead of: * some message * other message * hell if i know what this is You get: [WW] some message [EE] other message [QA] hell if i know what this is I've also added more colors to explicitly distinguish einfo from elog, and ewarn from eqawarn. Then, I've replaced most of '>>>' and '!!!' used by Portage with four-character versions to keep messages aligned. The 'zings' used for merging files remain three-character, so now it's easily possible to distinguish messages from installed file list. The PR doing this is: https://github.com/gentoo/portage/pull/759 Example screenshot: https://user-images.githubusercontent.com/110765/135119090-16e9599d-1b0f-41b8-a965-a55577183ffd.png -- Best regards, Michał Górny
Re: [gentoo-portage-dev] [PATCH 0/4] Output rewrite for better clarify and greppability
On Tue, Sep 28, 2021 at 10:20 AM Michał Górny wrote: > > Hi, > > Ok, so it's more major than I originally intended but I think it's > a good direction (once you get used to it). Roughly: > > 1. All bash color vars are now prefixed with `PORTAGE_COLOR_` to avoid >accidental collisions with ebuild vars (e.g. ebuild setting >`GOOD=foo` broke `elog` before). > 2. There are specific color vars for all kinds of output functions, >and now `einfo` messages use distinct color (dark green) from `elog`, >and `eqawarn` (brown) from `ewarn`. > 3. Messages are now prefixed by their kind, making it possible to >distinguish them without colors and grep for specific kind of logs: >- `[--]` for einfo & ebegin >- `[II]` for elog >- `[WW]` for ewarn >- `[QA]` for eqawarn >- `[EE]` for eerror > 4. Finally, I've replaced most of `>>>` and `!!!` in Portage output with >four `` and `` to align the output again. I like it. Maybe add a reference to this bug? https://bugs.gentoo.org/728046
[gentoo-portage-dev] [PATCH 3/4] Use verbose prefixes for output messages
Use verbose prefixes for output functions einfo, elog... in order to
make the message types distinguishable on non-color terminals,
and to enable the possibility of grepping them. The prefixes are
roughly inspired by Xorg logs, and are:
- `[--]` for einfo (and ebegin)
- `[II]` for elog
- `[WW]` for ewarn
- `[QA]` for eqawarn
- `[EE]` for eerror
Signed-off-by: Michał Górny
---
bin/isolated-functions.sh | 12 ++--
lib/portage/output.py | 12 ++--
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/bin/isolated-functions.sh b/bin/isolated-functions.sh
index df806d403..c7fda6784 100644
--- a/bin/isolated-functions.sh
+++ b/bin/isolated-functions.sh
@@ -270,7 +270,7 @@ eqawarn() {
__elog_base QA "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " ${PORTAGE_COLOR_QAWARN}*${PORTAGE_COLOR_NORMAL}
${REPLY}" >&2
+ echo "${PORTAGE_COLOR_QAWARN}[QA]${PORTAGE_COLOR_NORMAL}
${REPLY}" >&2
done
LAST_E_CMD="eqawarn"
return 0
@@ -280,7 +280,7 @@ elog() {
__elog_base LOG "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " ${PORTAGE_COLOR_LOG}*${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
+ echo "${PORTAGE_COLOR_LOG}[II]${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
done
LAST_E_CMD="elog"
return 0
@@ -290,7 +290,7 @@ einfo() {
__elog_base INFO "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " ${PORTAGE_COLOR_INFO}*${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
+ echo "${PORTAGE_COLOR_INFO}[--]${PORTAGE_COLOR_NORMAL}
${REPLY}" >&2
done
LAST_E_CMD="einfo"
return 0
@@ -299,7 +299,7 @@ einfo() {
einfon() {
__elog_base INFO "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
- echo -ne " ${PORTAGE_COLOR_INFO}*${PORTAGE_COLOR_NORMAL} $*" >&2
+ echo -ne "${PORTAGE_COLOR_INFO}[--]${PORTAGE_COLOR_NORMAL} $*" >&2
LAST_E_CMD="einfon"
return 0
}
@@ -308,7 +308,7 @@ ewarn() {
__elog_base WARN "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " ${PORTAGE_COLOR_WARN}*${PORTAGE_COLOR_NORMAL}
${RC_INDENTATION}${REPLY}" >&2
+ echo "${PORTAGE_COLOR_WARN}[WW]${PORTAGE_COLOR_NORMAL}
${RC_INDENTATION}${REPLY}" >&2
done
LAST_E_CMD="ewarn"
return 0
@@ -318,7 +318,7 @@ eerror() {
__elog_base ERROR "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " ${PORTAGE_COLOR_ERR}*${PORTAGE_COLOR_NORMAL}
${RC_INDENTATION}${REPLY}" >&2
+ echo "${PORTAGE_COLOR_ERR}[EE]${PORTAGE_COLOR_NORMAL}
${RC_INDENTATION}${REPLY}" >&2
done
LAST_E_CMD="eerror"
return 0
diff --git a/lib/portage/output.py b/lib/portage/output.py
index 42f487f8a..deae207fc 100644
--- a/lib/portage/output.py
+++ b/lib/portage/output.py
@@ -680,7 +680,7 @@ class EOutput:
if not self.quiet:
if self.__last_e_cmd == "ebegin":
self._write(out, "\n")
-self._write(out, colorize("ERR", " * ") + msg + "\n")
+self._write(out, colorize("ERR", "[EE] ") + msg + "\n")
self.__last_e_cmd = "eerror"
def einfo(self, msg):
@@ -694,7 +694,7 @@ class EOutput:
if not self.quiet:
if self.__last_e_cmd == "ebegin":
self._write(out, "\n")
-self._write(out, colorize("INFO", " * ") + msg + "\n")
+self._write(out, colorize("INFO", "[--] ") + msg + "\n")
self.__last_e_cmd = "einfo"
def einfon(self, msg):
@@ -708,7 +708,7 @@ class EOutput:
if not self.quiet:
if self.__last_e_cmd == "ebegin":
self._write(out, "\n")
-self._write(out, colorize("INFO", " * ") + msg)
+self._write(out, colorize("INFO", "[--] ") + msg)
self.__last_e_cmd = "einfon"
def eqawarn(self, msg):
@@ -722,7 +722,7 @@ class EOutput:
if not self.quiet:
if self.__last_e_cmd == "ebegin":
self._write(out, "\n")
-self._write(out, colorize("QAWARN", " * ") + msg + "\n")
+self._write(out, colorize("QAWARN", "[QA] ") + msg + "\n")
self.__last_e_cmd = "ewarn"
def elog(self, msg):
@@ -736,7 +736,7 @@ class EOutput:
if not self.quiet:
if self.__last_e_cmd == "ebegin":
self._write(out, "\n")
-self._write(out, colorize("LOG", " * ") + msg + "\n")
+self._write(out, colorize("LOG", "[II] ") + msg + "\n")
[gentoo-portage-dev] [PATCH 2/4] Use distinct colors for output msg types
Introduce distinct colors per output function. For elog and eerror
the colors remain the same but the names change. For einfo and eqawarn,
use darker colors to distinguish them.
Signed-off-by: Michał Górny
---
bin/isolated-functions.sh| 30 +++---
bin/save-ebuild-env.sh | 6 ++--
lib/portage/elog/messages.py | 10 +++---
lib/portage/elog/mod_echo.py | 4 +--
lib/portage/output.py| 59 +++-
5 files changed, 81 insertions(+), 28 deletions(-)
diff --git a/bin/isolated-functions.sh b/bin/isolated-functions.sh
index 3e79ea6e3..df806d403 100644
--- a/bin/isolated-functions.sh
+++ b/bin/isolated-functions.sh
@@ -270,7 +270,7 @@ eqawarn() {
__elog_base QA "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " ${PORTAGE_COLOR_WARN}*${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
+ echo " ${PORTAGE_COLOR_QAWARN}*${PORTAGE_COLOR_NORMAL}
${REPLY}" >&2
done
LAST_E_CMD="eqawarn"
return 0
@@ -280,7 +280,7 @@ elog() {
__elog_base LOG "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " ${PORTAGE_COLOR_GOOD}*${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
+ echo " ${PORTAGE_COLOR_LOG}*${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
done
LAST_E_CMD="elog"
return 0
@@ -290,7 +290,7 @@ einfo() {
__elog_base INFO "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " ${PORTAGE_COLOR_GOOD}*${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
+ echo " ${PORTAGE_COLOR_INFO}*${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
done
LAST_E_CMD="einfo"
return 0
@@ -299,7 +299,7 @@ einfo() {
einfon() {
__elog_base INFO "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
- echo -ne " ${PORTAGE_COLOR_GOOD}*${PORTAGE_COLOR_NORMAL} $*" >&2
+ echo -ne " ${PORTAGE_COLOR_INFO}*${PORTAGE_COLOR_NORMAL} $*" >&2
LAST_E_CMD="einfon"
return 0
}
@@ -318,7 +318,7 @@ eerror() {
__elog_base ERROR "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " ${PORTAGE_COLOR_BAD}*${PORTAGE_COLOR_NORMAL}
${RC_INDENTATION}${REPLY}" >&2
+ echo " ${PORTAGE_COLOR_ERR}*${PORTAGE_COLOR_NORMAL}
${RC_INDENTATION}${REPLY}" >&2
done
LAST_E_CMD="eerror"
return 0
@@ -378,12 +378,16 @@ __unset_colors() {
COLS=80
ENDCOL=
- PORTAGE_COLOR_GOOD=
- PORTAGE_COLOR_WARN=
PORTAGE_COLOR_BAD=
+ PORTAGE_COLOR_BRACKET=
+ PORTAGE_COLOR_ERR=
+ PORTAGE_COLOR_GOOD=
PORTAGE_COLOR_HILITE=
+ PORTAGE_COLOR_INFO=
+ PORTAGE_COLOR_LOG=
PORTAGE_COLOR_NORMAL=
- PORTAGE_COLOR_BRACKET=
+ PORTAGE_COLOR_QAWARN=
+ PORTAGE_COLOR_WARN=
}
__set_colors() {
@@ -403,12 +407,16 @@ __set_colors() {
if [ -n "${PORTAGE_COLORMAP}" ] ; then
eval ${PORTAGE_COLORMAP}
else
- PORTAGE_COLOR_GOOD=$'\e[32;01m'
- PORTAGE_COLOR_WARN=$'\e[33;01m'
PORTAGE_COLOR_BAD=$'\e[31;01m'
- PORTAGE_COLOR_HILITE=$'\e[36;01m'
PORTAGE_COLOR_BRACKET=$'\e[34;01m'
+ PORTAGE_COLOR_ERR=$'\e[31;01m'
+ PORTAGE_COLOR_GOOD=$'\e[32;01m'
+ PORTAGE_COLOR_HILITE=$'\e[36;01m'
+ PORTAGE_COLOR_INFO=$'\e[32m'
+ PORTAGE_COLOR_LOG=$'\e[32;01m'
PORTAGE_COLOR_NORMAL=$'\e[0m'
+ PORTAGE_COLOR_QAWARN=$'\e[33m'
+ PORTAGE_COLOR_WARN=$'\e[33;01m'
fi
}
diff --git a/bin/save-ebuild-env.sh b/bin/save-ebuild-env.sh
index 8d2ec67ed..98808814b 100644
--- a/bin/save-ebuild-env.sh
+++ b/bin/save-ebuild-env.sh
@@ -102,8 +102,10 @@ __save_ebuild_env() {
LAST_E_CMD LAST_E_LEN LD_PRELOAD MISC_FUNCTIONS_ARGS MOPREFIX \
NOCOLOR PKGDIR PKGUSE PKG_LOGDIR PKG_TMPDIR \
PORTAGE_BASHRC_FILES PORTAGE_BASHRCS_SOURCED \
- PORTAGE_COLOR_BAD PORTAGE_COLOR_BRACKET PORTAGE_COLOR_GOOD \
- PORTAGE_COLOR_HILITE PORTAGE_COLOR_NORMAL PORTAGE_COLOR_WARN \
+ PORTAGE_COLOR_BAD PORTAGE_COLOR_BRACKET PORTAGE_COLOR_ERR \
+ PORTAGE_COLOR_GOOD PORTAGE_COLOR_HILITE PORTAGE_COLOR_INFO \
+ PORTAGE_COLOR_LOG PORTAGE_COLOR_NORMAL PORTAGE_COLOR_QAWARN \
+ PORTAGE_COLOR_WARN \
PORTAGE_COMPRESS PORTAGE_COMPRESS_EXCLUDE_SUFFIXES \
PORTAGE_DOHTML_UNWARNED_SKIPPED_EXTENSIONS \
PORTAGE_DOHTML_UNWARNED_SKIPPED_FILES \
diff --git a/lib/portage/elog/messages.py
[gentoo-portage-dev] [PATCH 1/4] Prefix color vars with "PORTAGE_COLOR_"
Rename color variables used by einfo etc. to use "PORTAGE_COLOR_"
prefix. Currently these variables are prone to being accidentally
ovewritten e.g. if an ebuild uses GOOD or BAD variables for some
purpose. Using PORTAGE prefix should keep us safe.
As an extra benefit, this makes it trivial to grep for all uses of color
variables.
Signed-off-by: Michał Górny
---
bin/install-qa-check.d/10ignored-flags | 4 +--
bin/isolated-functions.sh | 42 +-
bin/save-ebuild-env.sh | 16 +-
lib/portage/output.py | 4 +--
4 files changed, 34 insertions(+), 32 deletions(-)
diff --git a/bin/install-qa-check.d/10ignored-flags
b/bin/install-qa-check.d/10ignored-flags
index 89706cd4c..7cd073578 100644
--- a/bin/install-qa-check.d/10ignored-flags
+++ b/bin/install-qa-check.d/10ignored-flags
@@ -52,7 +52,7 @@ ignored_flag_check() {
f=$(<"${T}"/scanelf-ignored-CFLAGS.log)
if [[ -n ${f} ]] ; then
__vecho -ne '\n'
- eqawarn "${BAD}QA Notice: Files built without
respecting CFLAGS have been detected${NORMAL}"
+ eqawarn "${PORTAGE_COLOR_BAD}QA Notice: Files
built without respecting CFLAGS have been detected${PORTAGE_COLOR_NORMAL}"
eqawarn " Please include the following list of
files in your report:"
eqawarn "${f}"
__vecho -ne '\n'
@@ -82,7 +82,7 @@ ignored_flag_check() {
f=$(<"${T}"/scanelf-ignored-LDFLAGS.log)
if [[ -n ${f} ]] ; then
__vecho -ne '\n'
- eqawarn "${BAD}QA Notice: Files built without
respecting LDFLAGS have been detected${NORMAL}"
+ eqawarn "${PORTAGE_COLOR_BAD}QA Notice: Files
built without respecting LDFLAGS have been detected${PORTAGE_COLOR_NORMAL}"
eqawarn " Please include the following list of
files in your report:"
eqawarn "${f}"
__vecho -ne '\n'
diff --git a/bin/isolated-functions.sh b/bin/isolated-functions.sh
index 5630dcf4c..3e79ea6e3 100644
--- a/bin/isolated-functions.sh
+++ b/bin/isolated-functions.sh
@@ -255,7 +255,7 @@ __elog_base() {
shift
;;
*)
- __vecho -e " ${BAD}*${NORMAL} Invalid use of internal
function __elog_base(), next message will not be logged"
+ __vecho -e "
${PORTAGE_COLOR_BAD}*${PORTAGE_COLOR_NORMAL} Invalid use of internal function
__elog_base(), next message will not be logged"
return 1
;;
esac
@@ -270,7 +270,7 @@ eqawarn() {
__elog_base QA "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " $WARN*$NORMAL $REPLY" >&2
+ echo " ${PORTAGE_COLOR_WARN}*${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
done
LAST_E_CMD="eqawarn"
return 0
@@ -280,7 +280,7 @@ elog() {
__elog_base LOG "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " $GOOD*$NORMAL $REPLY" >&2
+ echo " ${PORTAGE_COLOR_GOOD}*${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
done
LAST_E_CMD="elog"
return 0
@@ -290,7 +290,7 @@ einfo() {
__elog_base INFO "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " $GOOD*$NORMAL $REPLY" >&2
+ echo " ${PORTAGE_COLOR_GOOD}*${PORTAGE_COLOR_NORMAL} ${REPLY}"
>&2
done
LAST_E_CMD="einfo"
return 0
@@ -299,7 +299,7 @@ einfo() {
einfon() {
__elog_base INFO "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
- echo -ne " ${GOOD}*${NORMAL} $*" >&2
+ echo -ne " ${PORTAGE_COLOR_GOOD}*${PORTAGE_COLOR_NORMAL} $*" >&2
LAST_E_CMD="einfon"
return 0
}
@@ -308,7 +308,7 @@ ewarn() {
__elog_base WARN "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " $WARN*$NORMAL $RC_INDENTATION$REPLY" >&2
+ echo " ${PORTAGE_COLOR_WARN}*${PORTAGE_COLOR_NORMAL}
${RC_INDENTATION}${REPLY}" >&2
done
LAST_E_CMD="ewarn"
return 0
@@ -318,7 +318,7 @@ eerror() {
__elog_base ERROR "$*"
[[ ${RC_ENDCOL} != "yes" && ${LAST_E_CMD} == "ebegin" ]] && echo >&2
echo -e "$@" | while read -r ; do
- echo " $BAD*$NORMAL $RC_INDENTATION$REPLY" >&2
+ echo "
[gentoo-portage-dev] [PATCH 0/4] Output rewrite for better clarify and greppability
Hi, Ok, so it's more major than I originally intended but I think it's a good direction (once you get used to it). Roughly: 1. All bash color vars are now prefixed with `PORTAGE_COLOR_` to avoid accidental collisions with ebuild vars (e.g. ebuild setting `GOOD=foo` broke `elog` before). 2. There are specific color vars for all kinds of output functions, and now `einfo` messages use distinct color (dark green) from `elog`, and `eqawarn` (brown) from `ewarn`. 3. Messages are now prefixed by their kind, making it possible to distinguish them without colors and grep for specific kind of logs: - `[--]` for einfo & ebegin - `[II]` for elog - `[WW]` for ewarn - `[QA]` for eqawarn - `[EE]` for eerror 4. Finally, I've replaced most of `>>>` and `!!!` in Portage output with four `` and `` to align the output again. PR (includes screenshot): https://github.com/gentoo/portage/pull/759 Michał Górny (4): Prefix color vars with "PORTAGE_COLOR_" Use distinct colors for output msg types Use verbose prefixes for output messages Use "" and "" for output prefixes bin/clean_locks | 2 +- bin/dohtml.py | 2 +- bin/doins.py | 2 +- bin/ebuild| 8 +- bin/ebuild-helpers/dobin | 2 +- bin/ebuild-helpers/dodoc | 2 +- bin/ebuild-helpers/doexe | 2 +- bin/ebuild-helpers/doinfo | 2 +- bin/ebuild-helpers/dolib | 2 +- bin/ebuild-helpers/doman | 2 +- bin/ebuild-helpers/domo | 2 +- bin/ebuild-helpers/dosbin | 2 +- bin/ebuild-helpers/dosed | 4 +- bin/ebuild-helpers/keepdir| 2 +- bin/ebuild-helpers/newins | 4 +- bin/env-update| 2 +- bin/fixpackages | 4 +- bin/glsa-check| 14 +- bin/install-qa-check.d/10ignored-flags| 4 +- bin/install.py| 2 +- bin/isolated-functions.sh | 50 --- bin/misc-functions.sh | 22 +-- bin/phase-functions.sh| 64 - bin/phase-helpers.sh | 18 +-- bin/portageq | 6 +- bin/save-ebuild-env.sh| 18 ++- cnf/repo.postsync.d/example | 4 +- lib/_emerge/Binpkg.py | 10 +- lib/_emerge/BinpkgExtractorAsync.py | 4 +- lib/_emerge/BinpkgVerifier.py | 12 +- lib/_emerge/BlockerCache.py | 2 +- lib/_emerge/BlockerDB.py | 2 +- lib/_emerge/EbuildBuild.py| 2 +- lib/_emerge/EbuildIpcDaemon.py| 2 +- lib/_emerge/EbuildPhase.py| 2 +- lib/_emerge/JobStatusDisplay.py | 2 +- lib/_emerge/MergeListItem.py | 2 +- lib/_emerge/PackageUninstall.py | 6 +- lib/_emerge/Scheduler.py | 20 +-- lib/_emerge/SpawnProcess.py | 2 +- lib/_emerge/SubProcess.py | 2 +- lib/_emerge/actions.py| 126 ++ lib/_emerge/countdown.py | 4 +- lib/_emerge/depgraph.py | 121 + lib/_emerge/main.py | 6 +- lib/_emerge/post_emerge.py| 4 +- lib/_emerge/resolver/output_helpers.py| 4 +- lib/_emerge/resolver/slot_collision.py| 16 +-- lib/_emerge/show_invalid_depstring_notice.py | 2 +- lib/_emerge/unmerge.py| 44 +++--- lib/portage/__init__.py | 24 ++-- lib/portage/_selinux.py | 4 +- lib/portage/_sets/shell.py| 2 +- lib/portage/binrepo/config.py | 4 +- lib/portage/data.py | 2 +- lib/portage/dbapi/__init__.py | 2 +- lib/portage/dbapi/bintree.py | 45 --- lib/portage/dbapi/porttree.py | 12 +- lib/portage/dbapi/vartree.py | 94 ++--- lib/portage/dep/__init__.py | 48 +++ lib/portage/elog/__init__.py | 2 +- lib/portage/elog/messages.py | 15 ++- lib/portage/elog/mod_custom.py| 4 +- lib/portage/elog/mod_echo.py | 4 +- lib/portage/elog/mod_mail_summary.py | 2 +- lib/portage/elog/mod_save_summary.py | 2 +- lib/portage/emaint/modules/binhost/binhost.py | 3 +- lib/portage/env/loaders.py|
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On Mon, 2021-09-27 at 15:44 -0400, Mike Gilbert wrote: > On Mon, Sep 27, 2021 at 3:11 PM Peter Stuge wrote: > > > > Mike Gilbert wrote: > > > It's a waste of time and effort to pepper random ebuilds with checks > > > for options that everyone should have enabled in the first place. > > > > It's not for you to say what everyone should have enabled in their kernel. > > Do you not agree that there are some options that should always be > enabled, or at least that we can assume are enabled? > There is a distro/Kconfig in gentoo-sources with, config GENTOO_LINUX bool "Gentoo Linux support" ... that could be extended to include a set of assumed features; otherwise a sub-setting (default enabled) could be added.
Re: [gentoo-portage-dev] [PATCH] install-qa-check.d: Add a QA check for installing xattrs
On Tue, Sep 28, 2021 at 2:25 AM Michał Górny wrote:
>
> On Mon, 2021-09-27 at 21:09 -0400, Mike Gilbert wrote:
> > On Mon, Sep 27, 2021 at 1:20 PM Michał Górny wrote:
> > > + eqawarn
> > > + eqawarn "It is impossible to reliably guarantee that the
> > > extended attributes"
> > > + eqawarn "will be reliably preserved while merging.
> > > Please ensure that any"
> > > + eqawarn "extended metadata necessary is applied in
> > > pkg_postinst() phase,"
> > > + eqawarn "and that the implementation includes a fallback
> > > if necessary."
> >
> > This message suggests that applying xattrs in pkg_postinst is
> > acceptable. However, your patch offers no way to disable the QA
> > warning for ebuilds that do so.
>
> We'll cross that bridge when we get there. Ideally, we wouldn't need to
> silence the check because no packages would do that. If they do, then
> we'll probably want to work on an eclass like fcaps.eclas.
We need a way to silence this thing when false positives pop up and/or
ebuilds are adjusted. That needs to be there from day 1, not when we
cross some bridge later.
An immediate example: packages that call pax-mark in src_compile
because the need to disable MPROTECT on binary that is called a
compile time will end up with extended attributes in ${D} due to
install-xattr. We can adjust them to also call pax-mark in
pkg_postinst, but that won't magically make them go away in ${D}.
[gentoo-portage-dev] [PATCH v2] install-qa-check.d: Add a QA check for installing xattrs
Warn the developers if ebuilds install files with xattrs to ${ED}.
The xattrs may or may not be preserved when installing the package,
making them unreliable on one hand, and somewhat suprising in other
cases (e.g. when they unintentionally leak from developer's system).
This is the first step towards restoring PMS compliance and *not*
preserving extended metadata.
Signed-off-by: Michał Górny
---
bin/install-qa-check.d/95xattr | 54 ++
1 file changed, 54 insertions(+)
create mode 100644 bin/install-qa-check.d/95xattr
Changes in v2:
- added a link to the tracker in place of in-place explanation
diff --git a/bin/install-qa-check.d/95xattr b/bin/install-qa-check.d/95xattr
new file mode 100644
index 0..4bf6631d3
--- /dev/null
+++ b/bin/install-qa-check.d/95xattr
@@ -0,0 +1,54 @@
+# Check for xattrs. See https://bugs.gentoo.org/815220.
+
+xattr_check() {
+ type -P getfattr >/dev/null || return
+
+ pushd "${ED}" >/dev/null || die
+ local x file= keys
+ local -A data=()
+ while read -r x; do
+ case ${x} in
+ "# file: "*)
+ file=${x#*: }
+ file=/${file#.}
+ ;;
+ btrfs.*)
+ # ignore btrfs xattrs, they're implicit fs
metadata
+ ;;
+ security.capability)
+ # don't report caps if we have fcaps.eclass
inherited
+ if ! has fcaps ${INHERITED}; then
+ data[${file}]+=" ${x}"
+ fi
+ ;;
+ ?*)
+ data[${file}]+=" ${x}"
+ ;;
+ esac
+ done < <(getfattr -R -h -m - . 2>/dev/null)
+ popd >/dev/null || die
+
+ if [[ ${data[@]} ]]; then
+ eqawarn "One or more files in \${ED} include extended
attributes."
+ eqawarn
+
+ for file in "${!data[@]}"; do
+ keys=( ${data[${file}]} )
+ for x in "${keys[@]}"; do
+ eqatag xattr "key=${x}" "${file}"
+ done
+ eqawarn " ${file} (${keys[*]})"
+ done
+
+ eqawarn
+ eqawarn "It is impossible to reliably guarantee that the
extended attributes"
+ eqawarn "will be reliably preserved while merging. Please file
a bug"
+ eqawarn "and make it block the tracker
https://bugs.gentoo.org/815220.;
+ eqawarn "More information about the problem can also be found
on the tracker."
+ fi
+}
+
+xattr_check
+: # guarantee successful exit
+
+# vim:ft=sh
--
2.33.0
Re: [gentoo-portage-dev] [PATCH] install-qa-check.d: Add a QA check for installing xattrs
On Mon, 2021-09-27 at 21:09 -0400, Mike Gilbert wrote: > On Mon, Sep 27, 2021 at 1:20 PM Michał Górny wrote: > > + eqawarn > > + eqawarn "It is impossible to reliably guarantee that the > > extended attributes" > > + eqawarn "will be reliably preserved while merging. Please > > ensure that any" > > + eqawarn "extended metadata necessary is applied in > > pkg_postinst() phase," > > + eqawarn "and that the implementation includes a fallback if > > necessary." > > This message suggests that applying xattrs in pkg_postinst is > acceptable. However, your patch offers no way to disable the QA > warning for ebuilds that do so. We'll cross that bridge when we get there. Ideally, we wouldn't need to silence the check because no packages would do that. If they do, then we'll probably want to work on an eclass like fcaps.eclas. -- Best regards, Michał Górny
Re: [gentoo-portage-dev] [PATCH] install-qa-check.d: Add a QA check for installing xattrs
On Mon, 2021-09-27 at 21:03 -0400, Mike Gilbert wrote:
> On Mon, Sep 27, 2021 at 1:20 PM Michał Górny wrote:
> >
> > Warn the developers if ebuilds install files with xattrs to ${ED}.
> > The xattrs may or may not be preserved when installing the package,
> > making them unreliable on one hand, and somewhat suprising in other
> > cases (e.g. when they unintentionally leak from developer's system).
> >
> > This is the first step towards restoring PMS compliance and *not*
> > preserving extended metadata.
>
> How does preserving xattrs conflict with PMS?
The PMS doesn't specify that xattrs, ACLs, caps etc. are preserved.
By doing that, Portage allows developers to commit ebuilds that are not
going to work reliably without even realizing it. In fact, this can't
even work reliably inside Portage itself, depending on the filesystem
used for $D.
Furthermore, doexe preserving stuff goes contrary to common sense. Why
would helpers preserve xattrs when they are supposed to reset things
like mode and ownership by design?
> Is there a bug report you could reference?
It starts with https://bugs.gentoo.org/814857.
--
Best regards,
Michał Górny
