Re: [gentoo-dev] Add Hooks to Eselect
On Tue, Aug 22, 2023, 02:50 Ulrich Mueller, wrote: > > On Mon, 21 Aug 2023, konsolebox wrote: > > > This actually allows users to virtually extend an eselect module without > > needing to fork it. The things people can do are endless. > > This isn't what eselect modules were designed for. They are specialised > tools that are supposed to do one thing, and one thing only. So let's say modules weren't initially designed to be extensive. What's stopping those from becoming one now? Tradition? It's like > suggesting that a command like cat or mv would need configuration or > some extension mechanism. > Bad comparison and a stretch. Also, given that eselect modules are normally run as root and the damage > that can result from bugs, I'd rather keep things simple, instead of > introducing "endless" possibilities. This approach has worked very well > during the last 18 years. > And when I said unnecessary restriction this was it. You're trying to restrict what users can do in a free system like Gentoo because they might damage their system which is ridiculous. If you want to keep things simple for yourself don't touch /etc/eselect/hooks. And if you think third party code might abuse it, QA it. Or maybe just make eselect read /usr/local/etc/eselect/hooks instead. > (If anything, a hook mechanism would look very different from what was > proposed here. First, changing a low-level function like check_do() is > really a no-no, because this function is documented in the eselect > developer guide and third-party modules may rely on it. Second, instead > of executing a separate script for every hook, there would be a file > defining shell functions for all subactions of a given module. But I > haven't thought about any details, because as I said, I don't see much > incentive for such a thing.) > I just checked. Check_do is currently documented to be only used internally so no modules should use it but if you really think check_do shouldn't be modified then the answer is simple. Just make the function that calls the hooks the wrapper instead. As for declaring functions to represent as hooks yes I thought about that but that's a terrible idea. Not only is it hard to declare declaratively it also taints eselect's environment. Come think of it, hooks should be source'd from a subshell. I.e., "( source ... )". >
Re: [gentoo-dev] Add Hooks to Eselect
On 2023-08-21 20:50:39, Ulrich Mueller wrote:
> First, changing a low-level function like check_do() is really a no-no,
> because this function is documented in the eselect developer guide and
> third-party modules may rely on it.
The documentation, as far as I can tell, states only
> check_do
> The check_do utility function checks that the first parameter is a
> function, and then calls it with any additional parameters as its
> arguments. If the function does not exist, die is called.
> Again, this is mostly internal.
The behavior here is not modified by adding the hooks. If anything the
implementation ensures that if one eselect action calls another one, hooks
still apply as they should.
I actually just noticed making $params a local variable works too, so that
further ensures nothing in behavior changes for anything that happens to
use check_do, even if that is documented as being "mostly internal", and
not having the "PUBLIC" that for example die() has in its comment.
Calling the function with local variable "${params[@]}" instead of "$@" is
not something that's even possible to detect in bash as far as I know,
neither inside the called function nor to something calling check_do.
> [...] Given that eselect modules are normally run as root and the damage
> that can result from bugs, I'd rather keep things simple, instead of
> introducing "endless" possibilities. This approach has worked very well
> during the last 18 years.
One of the nice things about hooks is their low profile. If you don't know
about them, don't create their directory, you don't have to care. It is
not just all the possibilities of their use, it is that the price for them
is very low, for those not using them and for the maintainers of eselect.
The actions eselect does normally require root, so naturally the things a
user might want to add would too. If they used a wrapper script instead,
they would most likely run that as root too.
> Second, instead of executing a separate script for every hook, there
> [should] be a file defining shell functions for all subactions of a
> given module. But I haven't thought about any details, because as I
> said, I don't see much incentive for such a thing.
I don't see any advantage to this. Spreading hooks out into their own
directories is a common implementation, that allows one to have multiple
independent hooks with their own version management or any other number
of differences. I have for example now split my elselect kernel set hook,
so that if I want to disable defaulting to the most recent kernel version
when no version is specified I can chmod -x
kernel/set/pre/defaultVersion while keeping the migration of .config.
From experience I usually prefer cron.hourly over an entry in cron.d
and prefer that over a line in crontab. I realize me disliking crontab is
in part because I have been burned by heavy handed distros (looking at
you synology DSM) that like to overwrite changes in configuration files,
but even a well behaved distribution like gentoo still recently migrated
ssh_config and sshd_config to ssh_config.d and sshd_config.d.
I am open for arguments to the contrary but I am of the belief that a
directory approach where two independently written hooks can almost
certainly be merged on a file level without conflicts is the right
solution here and for hooks in general.
> If there was, it would have been suggested previously
> since eselect was created in 2005.
> This isn't what eselect modules were designed for. They are specialised
> tools that are supposed to do one thing, and one thing only. It's like
> suggesting that a command like cat or mv would need configuration or
> some extension mechanism.
It seems to me there might have been some drift in what eselect is.
As I have experienced it, eselect is a uniform interface to a variety of
"selection decisions" that otherwise would require learning a lot of
different quirks and implementations. When you call eselect, the action
it does internally might be simple or complex, change over time, or
otherwise be annoying to manually implement yourself. What cat or mv do is
predictable, uniform - eselect is not. If I want to modify what mv does
(outside of its flags) I use a different command, I assemble what I
intend to do from other smaller commands because replacing mv or calling
it from somewhere else is easy.
Eselect has a certain bulk to it, which from your position I would assume
it might not have had initially. Eselect modules have accumulated a lot of
quirks, specific behavior, some are truly complex looking through them now,
to the point where writing my own neither seems like something I want to do,
nor like something I should. Even the eselect kernel set command has an
entire page of sanity checks and niceties.
So we have a uniform interface to a set of actions too complex to want
to do with anything else, to me this feels more like a package manager than
a posix command, eselect to me always had the same "
Re: [gentoo-dev] Add Hooks to Eselect
> On Mon, 21 Aug 2023, konsolebox wrote: > This actually allows users to virtually extend an eselect module without > needing to fork it. The things people can do are endless. This isn't what eselect modules were designed for. They are specialised tools that are supposed to do one thing, and one thing only. It's like suggesting that a command like cat or mv would need configuration or some extension mechanism. Also, given that eselect modules are normally run as root and the damage that can result from bugs, I'd rather keep things simple, instead of introducing "endless" possibilities. This approach has worked very well during the last 18 years. (If anything, a hook mechanism would look very different from what was proposed here. First, changing a low-level function like check_do() is really a no-no, because this function is documented in the eselect developer guide and third-party modules may rely on it. Second, instead of executing a separate script for every hook, there would be a file defining shell functions for all subactions of a given module. But I haven't thought about any details, because as I said, I don't see much incentive for such a thing.)
Re: [gentoo-dev] Add Hooks to Eselect
On Mon, Aug 21, 2023, 18:00 Ulrich Mueller, wrote: > Sorry, but I don't see much incentive for adding such a hook mechanism. > This actually allows users to virtually extend an eselect module without needing to fork it. The things people can do are endless. Even now I just thought about using it to auto update Ruby shebangs in $HOME/bin in a manner that stubs are consistent with existing user gems for the newly selected version. I wouldn't need to wait for Gentoo Ruby to change anything. If there was, it would have been suggested previously since eselect was > created in 2005. Also, by design, eselect itself doesn't rely on any > configuration in /etc so this would be a somewhat intrusive change. > Weak arguments in my opinion. It promotes unnecessary restriction.
Re: [gentoo-dev] Re: Add Hooks to Eselect
On Mon, Aug 21, 2023, 08:15 Duncan, <1i5t5.dun...@cox.net> wrote:
> Suggestion: A more flexible approach would make pre/post directories,
>
Or allow both file and directories.
> do [[
> # maybe skip either the executable or README test?
> # or perhaps only match *.sh filenames instead
> # to reflect the in-shell sourcing?
> -x $hookfile &&
> $hookfile == ${hookfile#README} &&
> $hookfile == ${hookfile#.} &&
> $hookfile == ${hookfile%.bak} &&
> $hookfile == ${hookfile%\~}
> ]] && source "$hookfile"
> done
>
Just checking if file has +x is better. Easier to disable scripts that
way.
[gentoo-dev] [PATCH] metadata: Add stabilisation group for Ansible
Signed-off-by: Marek Szuba --- metadata/stabilization-groups/ansible | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 metadata/stabilization-groups/ansible diff --git a/metadata/stabilization-groups/ansible b/metadata/stabilization-groups/ansible new file mode 100644 index 000..b259400a0fc --- /dev/null +++ b/metadata/stabilization-groups/ansible @@ -0,0 +1,2 @@ +app-admin/ansible +app-admin/ansible-core -- 2.41.0
[gentoo-dev] [RFC PATCH] Add Ansible package-stabilisation group
This is very much for future reference, seeing as stabilisation groups are still very much work in progress. Anyway, the point here is to make it easier to avoid in the future the fairly regular occurrence of Portage complaining about having to hold back newly stabilised Ansible Core owing to version restrictions set in Ansible ebuilds. -- Marecki
Re: [gentoo-dev] Add Hooks to Eselect
> On Mon, 21 Aug 2023, Redjard wrote:
> I did consider doing so, however with that approach my effective fork
> of the kernel.eselect module would get continually outdated, while a
> wrapper module requires a different command [...]
Not necessarily. It could use the same name as the original module,
after adjusting the modules path:
ESELECT_MODULES_PATH=( "${ESELECT_DEFAULT_MODULES_PATH}" )
Still, I'd not recommend using the same name when its behaviour is
different from the original module.
Re: [gentoo-dev] Add Hooks to Eselect
On 2023-08-21 12:00:18, Ulrich Mueller wrote:
In addition to user patches, you can also put your own modules in the
${HOME}/.eselect/modules/ directory.
For example, you could either copy kernel.eselect to there and modify
it. Or, you could have a mykernel.eselect module, along these lines:
I did consider doing so, however with that approach my effective fork of
the kernel.eselect module would get continually outdated, while a
wrapper module requires a different command and isn't very expressive on
what it is, making future maintenance harder. The wrapper would also be
a lot of boilerplate, having to mention all actions, not just modified
ones. Both solutions didn't feel right to me.
I expect to use and modify eselect more in the future, and want an
approach that scales well while being low maintenance. A patch is the
correct solution for me, whether it gets upstreamed or not.
Also, by design, eselect itself doesn't rely on any configuration in
/etc so this would be a somewhat intrusive change.
> # l /etc/eselect/
> total 4.0K
> drwxr-xr-x 1 root root 48 2023-08-15 19:59:07 .
> drwxr-xr-x 1 root root 3.0K 2023-08-21 12:07:30 ..
> drwxrwxr-x 1 root root 12 2023-08-15 19:59:31 hooks
> -rw-r--r-- 1 root root 1.6K 2023-05-19 16:57:34 repository.conf
> drwxr-xr-x 1 root root 120 2023-08-10 03:09:50 wine
This is what my /etc/eselect dir looks like right now. I have hardly
used eselect in the past, and my system is fairly recent.
Given the apparent existing practice of putting eselect related elements
into /etc/eselect and that the hooks are entirely optional, I see no
difference to what has already been established.
Without knowledge of the feature, the directory doesn't exist and no
action is ever taken.
When someone hears of and intends to use the hooks, they create the
correct directory structure.
As a side note, your previously posted patch wouldn't work as-is:
- ${function} "$@"
+ ${function} "${params}"
Using a scalar variable instead of "$@" (which is an array) would break
quite a few modules.
Thanks, I did not know that. The updated patch below should handle those
cases now.
--- a/libs/core.bash.in
+++ b/libs/core.bash.in
@@ -20,10 +20,24 @@
-check_do() {
+check_do() {
local function=$1
- shift
+ shift; params=("${@}")
if is_function "${function}" ; then
+ run_hook "${ESELECT_MODULE_NAME}" "${function##do_}" pre
- ${function} "$@"
+ ${function} "${params[@]}"
+ run_hook "${ESELECT_MODULE_NAME}" "${function##do_}" post
else
die "No function ${function}"
fi
}
+# Redjard patch: call hooks
+run_hook() {
+ local action=$1
+ local subaction=$2
+ local hookscriptdir=$3
+ for hookfile in
"/etc/eselect/hooks/${action}/${subaction}/${hookscriptdir}"/* ; do [[
+ `basename "$hookfile"` =~ ^[a-zA-Z0-9_-][a-zA-Z0-9._-]+$
+ ]] &&
+ source "$hookfile"
+ done
+}
+
[gentoo-dev] Re: Add Hooks to Eselect
On 2023-08-21 02:15:08, Duncan wrote:
Suggestion: A more flexible approach would make pre/post directories, such
that "any" file therein (exceptions for backups, etc) would be sourced.
In run_hook something like (as written assumes bashisms OK, didn't verify
if that's valid for eselect or not):
local action=$1
local subaction=$2
local hookscriptdir=$3
for $hookfile in
/etc/eselect/hooks/${action}/${subaction}/${hookscriptdir}/*
do [[
# maybe skip either the executable or README test?
# or perhaps only match *.sh filenames instead
# to reflect the in-shell sourcing?
-x $hookfile &&
$hookfile == ${hookfile#README} &&
$hookfile == ${hookfile#.} &&
$hookfile == ${hookfile%.bak} &&
$hookfile == ${hookfile%\~}
]] && source "$hookfile"
done
eselect as a whole is a bash-script, so bashisms should indeed be ok.
I agree that a directory as the last level is the right choice for a
system the user didn't write themselves.
The filename requirements didn't seem right to me, so I dug through the
source
of run-parts in sys-apps/debianutils and found the default behavior as
regcomp(&classicalre, "^[a-zA-Z0-9_-][a-zA-Z0-9._-]+$",REG_EXTENDED | REG_NOSUB)
which curiously appears to be different from upstream "^[a-zA-Z0-9_-]+$",
which would not match script.sh for example.
Using the gentoo behavior of run-parts seems like the least
controversial implementation to me.
An executable check seems unnecessary, the directory is clearly marked
as containing hooks.
--- a/libs/core.bash.in
+++ b/libs/core.bash.in
@@ -20,10 +20,24 @@
-check_do() {
+check_do() {
local function=$1
- shift
+ shift; params="$@"
if is_function "${function}" ; then
- ${function} "$@"
+ run_hook "${ESELECT_MODULE_NAME}" "${function##do_}" pre
+ ${function} "${params}"
+ run_hook "${ESELECT_MODULE_NAME}" "${function##do_}" post
else
die "No function ${function}"
fi
}
+# Redjard patch: call hooks
+run_hook() {
+ local action=$1
+ local subaction=$2
+ local hookscriptdir=$3
+ for hookfile in
"/etc/eselect/hooks/${action}/${subaction}/${hookscriptdir}"/* ; do [[
+ `basename "$hookfile"` =~ ^[a-zA-Z0-9_-][a-zA-Z0-9._-]+$
+ ]] &&
+ source "$hookfile"
+ done
+}
+
Re: [gentoo-dev] Add Hooks to Eselect
> On Mon, 21 Aug 2023, Redjard wrote:
> To roughly summarize, I was asking for a method to hook into eselect,
> to modify the behavior of eselect kernel set.
> I was pointed in the direction of a user patch by konsolebox, and
> consequently wrote the patch.
In addition to user patches, you can also put your own modules in the
${HOME}/.eselect/modules/ directory.
For example, you could either copy kernel.eselect to there and modify
it. Or, you could have a mykernel.eselect module, along these lines:
do_set() {
... # execute "pre" stuff
do_action kernel set "$@"
... # execute "post" stuff
}
The unchanged subactions would be trivial functions like this:
do_list() { do_action kernel list "$@"; }
> I provided an example for using the patched hooks, which I will repeat
> below.
Sorry, but I don't see much incentive for adding such a hook mechanism.
If there was, it would have been suggested previously since eselect was
created in 2005. Also, by design, eselect itself doesn't rely on any
configuration in /etc so this would be a somewhat intrusive change.
As a side note, your previously posted patch wouldn't work as-is:
>>> -check_do() {
>>> +check_do() {
>>> local function=$1
>>> - shift
>>> + shift; params="$@"
>>> if is_function "${function}" ; then
>>> - ${function} "$@"
>>> + run_hook "${ESELECT_MODULE_NAME}" "${function##do_}" pre
>>> + ${function} "${params}"
Using a scalar variable instead of "$@" (which is an array) would break
quite a few modules.
>>> + run_hook "${ESELECT_MODULE_NAME}" "${function##do_}" post
>>> else
>>> die "No function ${function}"
>>> fi
>>> }
Ulrich
[gentoo-dev] [PATCH 1/1] secureboot.eclass: error out early if signing key/cert not readable
Hi all,
This patch is similar to the one I sent earlier for kernel-build.eclass.
Instead of checking if the signing key/cert exists we now also check if
we can actually read it. Also removed an extra "#" that prevents the
documentation from rendering as intended.
Best regards,
Andrew
diff --git a/eclass/secureboot.eclass b/eclass/secureboot.eclass
index 477722a83bb3..f95b7295dd06 100644
--- a/eclass/secureboot.eclass
+++ b/eclass/secureboot.eclass
@@ -52,7 +52,7 @@ BDEPEND="secureboot? ( app-crypt/sbsigntools )"
# @DESCRIPTION:
# Used with USE=secureboot. Should be set to the path of the private
# key in PEM format to use, or a PKCS#11 URI.
-#
+
# @ECLASS_VARIABLE: SECUREBOOT_SIGN_CERT
# @USER_VARIABLE
# @DEFAULT_UNSET
@@ -75,11 +75,11 @@ _secureboot_die_if_unset() {
if [[ -z ${SECUREBOOT_SIGN_KEY} || -z ${SECUREBOOT_SIGN_CERT} ]]; then
die "USE=secureboot enabled but SECUREBOOT_SIGN_KEY and/or
SECUREBOOT_SIGN_CERT not set."
fi
- if [[ ! ${SECUREBOOT_SIGN_KEY} == pkcs11:* && ! -f
${SECUREBOOT_SIGN_KEY} ]]; then
- die "SECUREBOOT_SIGN_KEY=${SECUREBOOT_SIGN_KEY} not found"
+ if [[ ! ${SECUREBOOT_SIGN_KEY} == pkcs11:* && ! -r
${SECUREBOOT_SIGN_KEY} ]]; then
+ die "SECUREBOOT_SIGN_KEY=${SECUREBOOT_SIGN_KEY} not found or not
readable"
fi
- if [[ ! -f ${SECUREBOOT_SIGN_CERT} ]];then
- die "SECUREBOOT_SIGN_CERT=${SECUREBOOT_SIGN_CERT} not found"
+ if [[ ! -r ${SECUREBOOT_SIGN_CERT} ]];then
+ die "SECUREBOOT_SIGN_CERT=${SECUREBOOT_SIGN_CERT} not found or not
readable"
fi
}
Re: [gentoo-dev] Add Hooks to Eselect
On 2023-08-19 06:58:10, Michał Górny wrote:
You have to provide the rationale here. You can't expect people to sign
up for some shady third-party service to read it.
My bad, not much of value is locked in the proprietary service but I
should have stated as such.
To roughly summarize, I was asking for a method to hook into eselect, to
modify the behavior of eselect kernel set.
I was pointed in the direction of a user patch by konsolebox, and
consequently wrote the patch.
I provided an example for using the patched hooks, which I will repeat
below.
>>> /etc/eselect/hooks/kernel/set/pre:
if [ "$EUID" -ne 0 ]; then die "run as root please"; fi
local -n target=params
if [ -z "$target" ]; then
target=`ls -1v /usr/src/ | grep '^linux-.*-gentoo$' | tail -n1`
else
if [ `echo "$target" | grep -P "^\d+(\.\d+){2}$"` ]; then
target=linux-"$1"-gentoo
elif [ `echo "${target%/}" | grep -P
"^linux-\d+(\.\d+){2}-gentoo$"` ]; then
target="${target%/}"
fi
fi
if [ "$target" == `basename "$(readlink /usr/src/linux)"` ]; then
die "kernel $target already selected"
fi
cp /usr/src/linux/.config /tmp/redjard_linux_kernel_eselect_config
echo "selecting kernel $target"
>>> /etc/eselect/hooks/kernel/set/post:
# (re)generate /etc/portage/sets/active_kernels which selects the
active, the installed, and the selected kernel
/opt/startup/lock-active-kernel-versions
# stop if .config is already present,
if [[ -e /usr/src/linux/.config ]];then
die "Warning: .config already exists. Stopping post hook
execution, you're on your own"
fi
echo "migrating .config"
mv -n /tmp/redjard_linux_kernel_eselect_config /usr/src/linux/.config
oldfile=`sha512sum /usr/src/linux/.config`
#echo "use make oldconfig to migrate the kernel configuration"
echo "running make oldconfig to migrate .config"
(cd /usr/src/linux && make oldconfig)
if [[ "$oldfile" != `sha512sum /usr/src/linux/.config` ]]; then
/opt/kernel/backup_config
fi
echo "use kernel-configure to edit the kernel configuration"
echo "use kernel-make or kernel-install to build and install the
configured kernel"
[gentoo-dev] [PATCH 1/1]: kernel-build.eclass: error out early if signing key not readable
Hi all,
Currently if the MODULES_SIGN_KEY exists but is not readable the kernel
will fail to compile. The kernel build system does not produce a clear
error when this happens so it is not immediately obvious what is wrong.
This small patch adjusts our src_prepare logic to check if the signing
key is readable, this makes the build error out sooner and with a
clearer error message.
Best regards,
Andrew
diff --git a/eclass/kernel-build.eclass b/eclass/kernel-build.eclass
index d498dd2e705b..5b324e036c5f 100644
--- a/eclass/kernel-build.eclass
+++ b/eclass/kernel-build.eclass
@@ -421,11 +421,11 @@ kernel-build_merge_configs() {
cat "${MODULES_SIGN_CERT}" "${MODULES_SIGN_KEY}" > "${T}/kernel_key.pem"
|| die
MODULES_SIGN_KEY="${T}/kernel_key.pem"
fi
- if [[ ${MODULES_SIGN_KEY} == pkcs11:* || -e
${MODULES_SIGN_KEY} ]]; then
+ if [[ ${MODULES_SIGN_KEY} == pkcs11:* || -r
${MODULES_SIGN_KEY} ]]; then
echo "CONFIG_MODULE_SIG_KEY=\"${MODULES_SIGN_KEY}\"" \
>> "${WORKDIR}/modules-sign.config"
elif [[ -n ${MODULES_SIGN_KEY} ]]; then
- die
"MODULES_SIGN_KEY=${MODULES_SIGN_KEY} not found!"
+ die
"MODULES_SIGN_KEY=${MODULES_SIGN_KEY} not found or not readable!"
fi
merge_configs+=( "${WORKDIR}/modules-sign.config" )
