[gentoo-portage-dev] [PATCH v4] env-update: create systemd env configuration if required
Portage's env-update currently transforms the environment information from /etc/env.d into /etc/profile.env, which is typically sourced by every user session, setting up its environment. However, /etc/profile.env is not sourced by systemd user services. Instead, for the definition of a systemd user session environment, the 'environment.d' machinery exists. Unfortunately, up to now, env-update does not produce a profile.env equivalent for this machinery, causing issues for systemd user services. For example, an emacs daemon run a user systemd service does not have a complete PATH (bug #704412 [1]), because some PATH components are injected by packages via /etc/env.d. For example, an LLVM ebuild may set PATH="/usr/lib/llvm/9/bin". This commit changes env-update so that, after profile.env has was generated, a systemd user session environment configuration file named /etc/environment.d/10-gentoo-profile-env.conf is created. Thanks to Michael 'veremitz' Everitt, Arfrever Frehtes Taifersar Arahesis, Ulrich Müller, and Joakim Tjernlund for the useful feedback. 1: https://bugs.gentoo.org/704412 Closes: https://bugs.gentoo.org/704416 Signed-off-by: Florian Schmaus --- Notes: - Prefix generated conf with "10-", for easier overwriting (thanks Joakim) lib/portage/util/env_update.py | 38 +++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/lib/portage/util/env_update.py b/lib/portage/util/env_update.py index f130b6f6bacb..8ba86c0bfd47 100644 --- a/lib/portage/util/env_update.py +++ b/lib/portage/util/env_update.py @@ -333,14 +333,16 @@ def _env_update(makelinks, target_root, prev_mtimes, contents, env, del specials["LDPATH"] - penvnotice = "# THIS FILE IS AUTOMATICALLY GENERATED BY env-update.\n" - penvnotice += "# DO NOT EDIT THIS FILE. CHANGES TO STARTUP PROFILES\n" + notice = "# THIS FILE IS AUTOMATICALLY GENERATED BY env-update.\n" + notice += "# DO NOT EDIT THIS FILE." + penvnotice = notice + " CHANGES TO STARTUP PROFILES\n" cenvnotice = penvnotice[:] penvnotice += "# GO INTO /etc/profile NOT /etc/profile.env\n\n" cenvnotice += "# GO INTO /etc/csh.cshrc NOT /etc/csh.env\n\n" #create /etc/profile.env for bash support - outfile = atomic_ofstream(os.path.join(eroot, "etc", "profile.env")) + profile_env_path = os.path.join(eroot, "etc", "profile.env") + outfile = atomic_ofstream(profile_env_path) outfile.write(penvnotice) env_keys = [x for x in env if x != "LDPATH"] @@ -353,6 +355,36 @@ def _env_update(makelinks, target_root, prev_mtimes, contents, env, outfile.write("export %s='%s'\n" % (k, v)) outfile.close() + # Create the systemd user environment configuration file + # /etc/environment.d/gentoo-profile-env.conf with the + # environment variables of /etc/profile.env. + systemd_environment_dir = os.path.join(eroot, "etc", "environment.d") + if not os.path.isdir(systemd_environment_dir): + os.mkdir(systemd_environment_dir) + + systemd_profile_env_path = os.path.join(systemd_environment_dir, +"10-gentoo-profile-env.conf") + with open(systemd_profile_env_path, "w") as systemd_profile_env: + senvnotice = notice + "\n\n" + systemd_profile_env.write(senvnotice) + + for env_key in env_keys: + env_key_value = env[env_key] + + # Skip variables with the empty string + # as value. Those sometimes appear in + # profile.env (e.g. "export GCC_SPECS=''"), + # but are invalid in systemd's syntax. + if not env_key_value: + continue + + # Transform into systemd environment.d + # conf syntax, basically shell variable + # assignment (without "export "). + line = f"{env_key}={env_key_value}\n" + + systemd_profile_env.write(line) + #create /etc/csh.env for (t)csh support outfile = atomic_ofstream(os.path.join(eroot, "etc", "csh.env")) outfile.write(cenvnotice) -- 2.26.2
Re: [gentoo-portage-dev] [PATCH v3] env-update: create systemd env configuration if required
On Thu, 2020-09-03 at 15:06 +0200, Florian Schmaus wrote: > CAUTION: This email originated from outside of the organization. Do not click > links or open attachments unless you recognize the sender and know the > content is safe. > > > Portage's env-update currently transforms the environment information > from /etc/env.d into /etc/profile.env, which is typically sourced by > every user session, setting up its environment. > > However, /etc/profile.env is not sourced by systemd user > services. Instead, for the definition of a systemd user session > environment, the 'environment.d' machinery exists. Unfortunately, up > to now, env-update does not produce a profile.env equivalent for this > machinery, causing issues for systemd user services. For example, > an emacs daemon run a user systemd service does not have a complete > PATH (bug #704412 [1]), because some PATH components are injected by > packages via /etc/env.d. For example, an LLVM ebuild may set > PATH="/usr/lib/llvm/9/bin". > > This commit changes env-update so that, after profile.env has was > generated, a systemd user session environment configuration file named > > /etc/environment.d/gentoo-profile-env.conf Are files /etc/environment.d/ ordered in anyway? If so it might be a good idea to name the file 10-gentoo-profile-env.conf or similar. Jocke
[gentoo-portage-dev] [PATCH v3] env-update: create systemd env configuration if required
Portage's env-update currently transforms the environment information from /etc/env.d into /etc/profile.env, which is typically sourced by every user session, setting up its environment. However, /etc/profile.env is not sourced by systemd user services. Instead, for the definition of a systemd user session environment, the 'environment.d' machinery exists. Unfortunately, up to now, env-update does not produce a profile.env equivalent for this machinery, causing issues for systemd user services. For example, an emacs daemon run a user systemd service does not have a complete PATH (bug #704412 [1]), because some PATH components are injected by packages via /etc/env.d. For example, an LLVM ebuild may set PATH="/usr/lib/llvm/9/bin". This commit changes env-update so that, after profile.env has was generated, a systemd user session environment configuration file named /etc/environment.d/gentoo-profile-env.conf is created. Thanks to Michael 'veremitz' Everitt, Arfrever Frehtes Taifersar Arahesis and Ulrich Müller for the useful feedback. 1: https://bugs.gentoo.org/704412 Closes: https://bugs.gentoo.org/704416 Signed-off-by: Florian Schmaus --- Notes: - Place generated file in /etc/environment.d (Thanks ulm) lib/portage/util/env_update.py | 38 +++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/lib/portage/util/env_update.py b/lib/portage/util/env_update.py index f130b6f6bacb..0baca8a98676 100644 --- a/lib/portage/util/env_update.py +++ b/lib/portage/util/env_update.py @@ -333,14 +333,16 @@ def _env_update(makelinks, target_root, prev_mtimes, contents, env, del specials["LDPATH"] - penvnotice = "# THIS FILE IS AUTOMATICALLY GENERATED BY env-update.\n" - penvnotice += "# DO NOT EDIT THIS FILE. CHANGES TO STARTUP PROFILES\n" + notice = "# THIS FILE IS AUTOMATICALLY GENERATED BY env-update.\n" + notice += "# DO NOT EDIT THIS FILE." + penvnotice = notice + " CHANGES TO STARTUP PROFILES\n" cenvnotice = penvnotice[:] penvnotice += "# GO INTO /etc/profile NOT /etc/profile.env\n\n" cenvnotice += "# GO INTO /etc/csh.cshrc NOT /etc/csh.env\n\n" #create /etc/profile.env for bash support - outfile = atomic_ofstream(os.path.join(eroot, "etc", "profile.env")) + profile_env_path = os.path.join(eroot, "etc", "profile.env") + outfile = atomic_ofstream(profile_env_path) outfile.write(penvnotice) env_keys = [x for x in env if x != "LDPATH"] @@ -353,6 +355,36 @@ def _env_update(makelinks, target_root, prev_mtimes, contents, env, outfile.write("export %s='%s'\n" % (k, v)) outfile.close() + # Create the systemd user environment configuration file + # /etc/environment.d/gentoo-profile-env.conf with the + # environment variables of /etc/profile.env. + systemd_environment_dir = os.path.join(eroot, "etc", "environment.d") + if not os.path.isdir(systemd_environment_dir): + os.mkdir(systemd_environment_dir) + + systemd_profile_env_path = os.path.join(systemd_environment_dir, +"gentoo-profile-env.conf") + with open(systemd_profile_env_path, "w") as systemd_profile_env: + senvnotice = notice + "\n\n" + systemd_profile_env.write(senvnotice) + + for env_key in env_keys: + env_key_value = env[env_key] + + # Skip variables with the empty string + # as value. Those sometimes appear in + # profile.env (e.g. "export GCC_SPECS=''"), + # but are invalid in systemd's syntax. + if not env_key_value: + continue + + # Transform into systemd environment.d + # conf syntax, basically shell variable + # assignment (without "export "). + line = f"{env_key}={env_key_value}\n" + + systemd_profile_env.write(line) + #create /etc/csh.env for (t)csh support outfile = atomic_ofstream(os.path.join(eroot, "etc", "csh.env")) outfile.write(cenvnotice) -- 2.26.2
Re: [gentoo-portage-dev] [PATCH v2] env-update: create systemd env configuration if required
On 03.09.20 14:43, Ulrich Mueller wrote: >> On Thu, 03 Sep 2020, Florian Schmaus wrote: > >> It's not really maintaining the information twice. The information is >> maintained at a single point: /etc/env.d >> And from there is is transformed by env-update already into two >> different formats: >> - /etc/profile.env >> - /etc/csh.env > >> And with that change additionally into >> - /usr/lib/environment.d/gentoo-profile-env.conf > > Sorry for another nitpick, but it's changing a file in /usr at runtime> Also, > does the file belong to any package, or is it an orphan? It's an orphan. > Maybe it would be cleaner to generate the file in /etc like the others, > if necessary with a symlink in /usr/lib/environment.d? Good point. A symlink wont be necessary. We could also generate the file in /etc/environment.d I guess /etc/environment.d is preferred over /usr/lib/environment.d? I'll make the according changes to the patch. - Florian signature.asc Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH v2] env-update: create systemd env configuration if required
> On Thu, 03 Sep 2020, Florian Schmaus wrote: > It's not really maintaining the information twice. The information is > maintained at a single point: /etc/env.d > And from there is is transformed by env-update already into two > different formats: > - /etc/profile.env > - /etc/csh.env > And with that change additionally into > - /usr/lib/environment.d/gentoo-profile-env.conf Sorry for another nitpick, but it's changing a file in /usr at runtime? Also, does the file belong to any package, or is it an orphan? Maybe it would be cleaner to generate the file in /etc like the others, if necessary with a symlink in /usr/lib/environment.d? The symlink could belong to some package, maybe even sys-apps/systemd itself. Ulrich signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH v2] env-update: create systemd env configuration if required
On 03.09.20 13:30, Ulrich Mueller wrote: >> On Thu, 03 Sep 2020, Florian Schmaus wrote: > >> This commit changes env-update so that, after profile.env has was >> generated, a systemd user session environment configuration file named > >> /usr/lib/environment.d/gentoo-profile-env.conf > >> is created, if the directory /usr/lib/environment.d exists. > > Maybe a stupid question, but can't this file just source /etc/profile? Akin to what 96e0294f0892 ("Add env gen to inject full Gentoo PATH to services") in gentoo-systemd-integration does? Unfortunately the answer is 'no', because gentoo-profile-env.conf is not an interactive script, it is just a systemd configuration file. > Maintaining the same information twice doesn't look like the right thing > to do. It's not really maintaining the information twice. The information is maintained at a single point: /etc/env.d And from there is is transformed by env-update already into two different formats: - /etc/profile.env - /etc/csh.env And with that change additionally into - /usr/lib/environment.d/gentoo-profile-env.conf - Florian signature.asc Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH v2] env-update: create systemd env configuration if required
> On Thu, 03 Sep 2020, Florian Schmaus wrote: > This commit changes env-update so that, after profile.env has was > generated, a systemd user session environment configuration file named > /usr/lib/environment.d/gentoo-profile-env.conf > is created, if the directory /usr/lib/environment.d exists. Maybe a stupid question, but can't this file just source /etc/profile? Maintaining the same information twice doesn't look like the right thing to do. Ulrich signature.asc Description: PGP signature
[gentoo-portage-dev] [PATCH v2] env-update: create systemd env configuration if required
Portage's env-update currently transforms the environment information from /etc/env.d into /etc/profile.env, which is typically sourced by every user session, setting up its environment. However, /etc/profile.env is not sourced by systemd user services. Instead, for the definition of a systemd user session environment, the 'environment.d' machinery exists. Unfortunately, up to now, env-update does not produce a profile.env equivalent for this machinery, causing issues for systemd user services. For example, an emacs daemon run a user systemd service does not have a complete PATH (bug #704412 [1]), because some PATH components are injected by packages via /etc/env.d. For example, an LLVM ebuild may set PATH="/usr/lib/llvm/9/bin". This commit changes env-update so that, after profile.env has was generated, a systemd user session environment configuration file named /usr/lib/environment.d/gentoo-profile-env.conf is created, if the directory /usr/lib/environment.d exists. Thanks to Michael 'veremitz' Everitt and Arfrever Frehtes Taifersar Arahesis for the useful feedback. 1: https://bugs.gentoo.org/704412 Closes: https://bugs.gentoo.org/704416 Signed-off-by: Florian Schmaus --- Notes: - Fix typo s/profile.enf/profile.env/ (thanks Michael) - Use env_keys to generate gentoo-profile-env.conf (thanks Arfrever) - Add "file is auto generated" notice header to generated conf lib/portage/util/env_update.py | 37 +++--- 1 file changed, 34 insertions(+), 3 deletions(-) diff --git a/lib/portage/util/env_update.py b/lib/portage/util/env_update.py index f130b6f6bacb..0aace8f4c5f4 100644 --- a/lib/portage/util/env_update.py +++ b/lib/portage/util/env_update.py @@ -333,14 +333,16 @@ def _env_update(makelinks, target_root, prev_mtimes, contents, env, del specials["LDPATH"] - penvnotice = "# THIS FILE IS AUTOMATICALLY GENERATED BY env-update.\n" - penvnotice += "# DO NOT EDIT THIS FILE. CHANGES TO STARTUP PROFILES\n" + notice = "# THIS FILE IS AUTOMATICALLY GENERATED BY env-update.\n" + notice += "# DO NOT EDIT THIS FILE." + penvnotice = notice + " CHANGES TO STARTUP PROFILES\n" cenvnotice = penvnotice[:] penvnotice += "# GO INTO /etc/profile NOT /etc/profile.env\n\n" cenvnotice += "# GO INTO /etc/csh.cshrc NOT /etc/csh.env\n\n" #create /etc/profile.env for bash support - outfile = atomic_ofstream(os.path.join(eroot, "etc", "profile.env")) + profile_env_path = os.path.join(eroot, "etc", "profile.env") + outfile = atomic_ofstream(profile_env_path) outfile.write(penvnotice) env_keys = [x for x in env if x != "LDPATH"] @@ -353,6 +355,35 @@ def _env_update(makelinks, target_root, prev_mtimes, contents, env, outfile.write("export %s='%s'\n" % (k, v)) outfile.close() + # Create the systemd user environment configuration file + # /usr/lib/environment.d/gentoo-profile-env.conf with the + # environment variables of /etc/profile.env if + # /usr/lib/environment.d exists. + systemd_environment_dir = os.path.join(eroot, "usr", "lib", "environment.d") + if os.path.isdir(systemd_environment_dir): + systemd_profile_env_path = os.path.join(systemd_environment_dir, +"gentoo-profile-env.conf") + with open(systemd_profile_env_path, "w") as systemd_profile_env: + senvnotice = notice + "\n\n" + systemd_profile_env.write(senvnotice) + + for env_key in env_keys: + env_key_value = env[env_key] + + # Skip variables with the empty string + # as value. Those sometimes appear in + # profile.env (e.g. "export GCC_SPECS=''"), + # but are invalid in systemd's syntax. + if not env_key_value: + continue + + # Transform into systemd environment.d + # conf syntax, basically shell variable + # assignment (without "export "). + line = f"{env_key}={env_key_value}\n" + + systemd_profile_env.write(line) + #create /etc/csh.env for (t)csh support outfile = atomic_ofstream(os.path.join(eroot, "etc", "csh.env")) outfile.write(cenvnotice) -- 2.26.2