Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
* Marvin Renich [221115 12:57]: > TEMPDIR, on the other hand, is for _specific_ cases, and can have ^ et al Of course, that should be TMPDIR, not TEMPDIR. Apologies. ...Marvin
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
* Robie Basak [221113 14:15]: > On Sun, Nov 13, 2022 at 05:46:00PM +0100, Marco d'Itri wrote: > > On Nov 13, Robie Basak wrote: > > > > > This seems inconsistent to me. Where is the expectation that TMPDIR must > > > be unset if dropping privileges coming from? Obviously for users of > > Where is the expectation that $TMPDIR is writable by any user but the > > current one? > > I do not believe that it is expected that if a user creates a directory > > and points $TMPDIR to it then they also have to make it sticky, so this > > has nothing to do with libpam-tmpdir. > > I understand the traditional semantics of TMPDIR to be exactly the same > as /tmp. So that includes the sticky bit, or at least behaviour that is > equivalent under all circumstances. Or, alternatively, that someone who > sets TMPDIR without setting the sticky bit is certain that it will be > used in a way that does not rely on that. This is an incorrect assumption and may be what is driving your resistance to others' guidance on how to resolve these bugs. /tmp has world-writable, sticky-bit semantics specifically because it was necessary in order for the system to give a single _common_ place where all users could write temporary files, not because a temporary directory for a given process needs those semantics. TEMPDIR, on the other hand, is for _specific_ cases, and can have whatever semantics are appropriate for the environment in which it is set. TEMPDIR is specifically for use when /tmp has properties that are not appropriate for a process for any reason. The inappropriate property may be available disk space, inappropriate permissions (privacy), or anything else. When TEMPDIR is set, you specifically cannot assume that it is world-writable or has a specific amount of disk space. Also, you cannot count on automatic cleaning of the TEMPDIR directory, while you can with /tmp (at some indeterminate point in the future). I think Simon's analysis is the most thorough, well-thought-out, and technically correct that I have seen in this thread. His reasoning should be used to determine how these bugs are resolved. ...Marvin
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
> "Otto" == Otto Kekäläinen writes: Otto> Instead of manually trying to manage TMPDIR env variable in Otto> various places, we should have a standardized way to run Otto> maintainer scripts in clean shell sessions that have all env Otto> variables set automatically correctly. I think trusting TMPDIR when running a maintainer script as root is fine.\ The sanitization should happen by sudo (or su or sshd) which is what gates you into root privilege. The issue with the mysql/mariadb scripts is that they are taking root's environment and applying it to the mysql user. So, those scripts need to do additional sanitization/trimming of the environment. But that comes up because those scripts are introducing a uid transition.
Bug#1022994: Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Sun, Nov 13, 2022 at 05:46:00PM +0100, Marco d'Itri wrote: > On Nov 13, Robie Basak wrote: > > > This seems inconsistent to me. Where is the expectation that TMPDIR must > > be unset if dropping privileges coming from? Obviously for users of > Where is the expectation that $TMPDIR is writable by any user but the > current one? > I do not believe that it is expected that if a user creates a directory > and points $TMPDIR to it then they also have to make it sticky, so this > has nothing to do with libpam-tmpdir. I understand the traditional semantics of TMPDIR to be exactly the same as /tmp. So that includes the sticky bit, or at least behaviour that is equivalent under all circumstances. Or, alternatively, that someone who sets TMPDIR without setting the sticky bit is certain that it will be used in a way that does not rely on that. libpam-tmpdir breaks those semantics in a way that breaks in edge cases like the situation raised in this (and other) bug reports. So on the contrary, I think it has everything to do with libpam-tmpdir, and anything else that sets TMPDIR to something that doesn't match the traditional semantics. To be clear, if it's better that we change the semantics to improve the system as a whole, then I don't have a particular objection to that. But I'd prefer that it be done deliberately, with consensus across all developers, and be well-defined and documented. Rather than have a change of semantics exist in a multitude of individual fixes for individual bug reports that potentially end up solving the issue differently, inconsistently or incompletely, and without regard for the bigger picture (eg. if the conclusion is that it should be done somewhere other than maintainer scripts or in common tooling). There's also the risk that we swap one problem for another - for example if there are use cases which rely on maintainer scripts honouring TMPDIR, including when they drop privileges. signature.asc Description: PGP signature
Bug#1022994: Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Nov 13, Robie Basak wrote: > This seems inconsistent to me. Where is the expectation that TMPDIR must > be unset if dropping privileges coming from? Obviously for users of Where is the expectation that $TMPDIR is writable by any user but the current one? I do not believe that it is expected that if a user creates a directory and points $TMPDIR to it then they also have to make it sticky, so this has nothing to do with libpam-tmpdir. -- ciao, Marco signature.asc Description: PGP signature
Bug#1022994: Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Sun, Nov 13, 2022 at 04:16:29PM +0100, Marco d'Itri wrote: > And I think that it would be wrong to have dpkg generally unset $TMPDIR, > because if root sets it then it would be reasonable to expect that also > dpkg and the maintainer scripts use it (as long as they are not dropping > privileges). This seems inconsistent to me. Where is the expectation that TMPDIR must be unset if dropping privileges coming from? Obviously for users of libpam-tmpdir that's a problem. But in the default case, it's something that would be entirely reasonable to inherit through a drop of privileges, for the same reason that I think you find that setting TMPDIR for maintainer scripts to use would be useful. signature.asc Description: PGP signature
Bug#1022994: Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Sun, Nov 13, 2022 at 02:58:47PM +, Simon McVittie wrote: > If the maintainer script is *dropping* privileges from root down to a > system user, then I think the maintainer script is/should be responsible > for doing that privilege drop in a way that works... Agreed, but amongst various other things, I don't think it's at all clear whether TMPDIR is one of the things that the maintainer script should be expected to unset in this case. For libpam-tmpdir's use of setting TMPDIR, it's obviously the required behaviour. But what about in general? What's the "environment variable" interface to maintainer scripts? Clearly they are expected to honor _some_ environment variables. Is TMPDIR one of them? Where's the complete list defined, so that we can be sure that the semantics are reasonable for all the use cases we can think of, and so that we can all write maintainer scripts correctly? Why is it that dropping privileges requires the unsetting of environment variables in the case of maintainer script invocations anyway? They are always run from a trusted environment and unsetting variables removes the ability to pass configuration through. Sure, I can't rely on some expectations holding (eg. HOME), but if I don't rely on this, there's no problem. So then, what about TMPDIR? What are its actual semantics? Tying TMPDIR to the uid that uses it is not the default, nor the tradition. This entanglement is something that libpam-tmpdir adds. Maybe that means that we need to consider TMPDIR's semantics changed, because people find this kind of behaviour more useful. But that's a discussion that hasn't yet happened. For example, is there a user who expects to be able to use TMPDIR to tell a maintainer script where there is enough space for its task, only to find that it doesn't have any effect because the maintainer script drops privileges and unsets it before doing its task? signature.asc Description: PGP signature
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
]] Sunil Mohan Adapa > During today's FreedomBox meet, we have discussed that systemd'd > PrivateTmp= is a better solution than libpam-tmpdir for FreedomBox at > least as systemd makes a cleaner mount isolation between processes > instead of managing directories and permissions. > > For this reason, we believe that we can stop using libpam-tmpdir if > most of the daemons on the system use PrivateTmp=yes. For a while now, > FreedomBox has been forcefully adding systemd security features to > daemons that don't enable them. Without upstream blessing, we can only > do this for smaller applications than something like MariaDB/MySQL due > the testing effort needed. They solve completely different problems, though. One handles PAM sessions, the other handles services. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
]] Robie Basak > On Thu, Nov 10, 2022 at 05:37:53PM +0100, Tollef Fog Heen wrote: > > I think it's more wide than that: If you change UID, you need to > > sanitise the environment. Your HOME is likely to be wrong. PATH might > > very well be pointing at directories which are not appropriate for the > > user you're changing the UID to, etc. > > I don't think that this is necessarily obviously the case in general. > For example, I often use "sudo -s" and *don't* want HOME reset. It > depends on the purpose of taking different privileges as to what is > appropriate to reset. I don't think we're disagreeing here. > > I'm not sure this is libpam-tmpdir specific, but rather a bit more > > general: what are the expectations that maintainer scripts can have > > about the environment they're running in, and how do we make those > > expectations hold? This should probably then be documented in policy. > > Agreed, but also, we need a specific answer for TMPDIR. We pass things > into maintainer scripts because we want to change their behaviour (eg. > DEBIAN_FRONTEND). So which specific variables are required to be reset > by maintainer scripts and under what circumstances? In the specific case of changing users, I'd say any that might influence the behaviour of what you're executing, whether it's PATH, TMP, TMPDIR, XDG_DATA_DIRS, PERL5LIB or something else. I can see arguments both for and against dpkg ensuring that maintainer scripts run with a sanitised environment. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are
Bug#1022994: Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Nov 13, Simon McVittie wrote: > I think you can both be right. The symptom here is a maintainer script > failing, but if I'm understanding Marco's argument correctly, he's > saying that the root cause is that when you switch between execution > environments, not all of the environment variables inherited from the old > execution environment are appropriate for the new one. (A similar thing > can happen for other inheritable process parameters like resource limits.) Yes, but this is a general issue and not relevant for this specific case. Because let's consider an environment in which: - there is no relevant elevation of privileges, so discussions about what should libpam-something, sudo, etc... do are not applicable - the root environmente legitimately has TMPDIR=/tmp/root-only/ mode 700 - a maintainer script does something as an unprivileged user which needs to access $TMPDIR so you correctly noted that: > If the maintainer script is *dropping* privileges from root down to a > system user, then I think the maintainer script is/should be responsible > for doing that privilege drop in a way that works - but ideally the > maintainer script should be delegating that responsibility to a common > tool rather than open-coding it itself, either by launching a one-shot > system service that the init infrastructure can run in a predictable > execution environment, or using a common privilege-level-switching tool > like runuser, su or sudo, or using an IPC-based "run this command in a > more controllable enviroment" tool like systemd-run. And I think that it would be wrong to have dpkg generally unset $TMPDIR, because if root sets it then it would be reasonable to expect that also dpkg and the maintainer scripts use it (as long as they are not dropping privileges). -- ciao, Marco signature.asc Description: PGP signature
Bug#1022994: Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Sun, 13 Nov 2022 at 11:38:08 +, Robie Basak wrote: > On Sun, Nov 13, 2022 at 02:21:58AM +0100, Marco d'Itri wrote: > > On Nov 12, Otto Kekäläinen wrote: > > > Instead of manually trying to manage TMPDIR env variable in various > > > places, we should have a standardized way to run maintainer scripts in > > > clean shell sessions that have all env variables set automatically > > > correctly. > > > > This is not about maintainer scripts, but about programs which change > > the UID without sanitizing the environment. > > No, it's absolutely about maintainer scripts, since that's the bug > reported, and the specific fix suggested implies that all relevant > maintainer scripts need changing. I think you can both be right. The symptom here is a maintainer script failing, but if I'm understanding Marco's argument correctly, he's saying that the root cause is that when you switch between execution environments, not all of the environment variables inherited from the old execution environment are appropriate for the new one. (A similar thing can happen for other inheritable process parameters like resource limits.) The specific bug we're discussing in mysql-server-8.0 seems like it potentially has two separate privilege transitions: one elevating privileges from a sysadmin up to root (via `sudo apt upgrade` or equivalent), and one dropping privileges from root down to a system user to run a preparation step in the maintainer script. I think some of the people replying to the thread on -devel might be conflating those two privilege transitions, which means we end up talking at cross purposes because a valid point made about one transition can be invalid for the other. For the privilege elevation from a sysadmin up to root, this could in principle be addressed anywhere within the inheritance chain: your shell -> sudo/su/pkexec -> apt -> dpkg -> maintainer script -> ... but I think it's reasonable to say that it's sudo/su/pkexec/... that is doing an unusual change to the execution environment, so it should also be sudo/su/pkexec/... that is responsible for making sure the new execution environment is internally consistent. Making each maintainer script responsible for solving this problem for itself would scale poorly, so if someone's suggested fix involves changing all maintainer scripts, I'd very much prefer to look at whether that can be avoided. If the maintainer script is *dropping* privileges from root down to a system user, then I think the maintainer script is/should be responsible for doing that privilege drop in a way that works - but ideally the maintainer script should be delegating that responsibility to a common tool rather than open-coding it itself, either by launching a one-shot system service that the init infrastructure can run in a predictable execution environment, or using a common privilege-level-switching tool like runuser, su or sudo, or using an IPC-based "run this command in a more controllable enviroment" tool like systemd-run. smcv
Bug#1022994: Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Sun, Nov 13, 2022 at 02:21:58AM +0100, Marco d'Itri wrote: > On Nov 12, Otto Kekäläinen wrote: > > > Instead of manually trying to manage TMPDIR env variable in various > > places, we should have a standardized way to run maintainer scripts in > > clean shell sessions that have all env variables set automatically > > correctly. > This is not about maintainer scripts, but about programs which change > the UID without sanitizing the environment. No, it's absolutely about maintainer scripts, since that's the bug reported, and the specific fix suggested implies that all relevant maintainer scripts need changing. You have generalised this, but I don't think it's at all clear that such a broad generalisation is appropriate here. signature.asc Description: PGP signature
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Thu, Nov 10, 2022 at 10:46:55PM +, brian m. carlson wrote: > > I think it's more wide than that: If you change UID, you need to > > sanitise the environment. Your HOME is likely to be wrong. PATH might > > very well be pointing at directories which are not appropriate for the > > user you're changing the UID to, etc. > > I believe this is the best practice. For example, sudo typically passes > through only a handful of environment variables, such as TERM, to avoid > things like insecure PATH entries. For example, if MySQL invoked a > binary in PATH and I had a custom script named the same thing that had > insecure behaviour when invoked as another user, that would be bad. > OpenSSH also sanitizes the environment passed over the connection. Taking your example, if we decide we cannot trust PATH, then dpkg should reset it before invoking maintainer scripts. It doesn't make sense to say that we should not trust the supplied PATH under these circumstances but then also require maintainer scripts to individually reset it. signature.asc Description: PGP signature
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Thu, Nov 10, 2022 at 05:37:53PM +0100, Tollef Fog Heen wrote: > I think it's more wide than that: If you change UID, you need to > sanitise the environment. Your HOME is likely to be wrong. PATH might > very well be pointing at directories which are not appropriate for the > user you're changing the UID to, etc. I don't think that this is necessarily obviously the case in general. For example, I often use "sudo -s" and *don't* want HOME reset. It depends on the purpose of taking different privileges as to what is appropriate to reset. > I'm not sure this is libpam-tmpdir specific, but rather a bit more > general: what are the expectations that maintainer scripts can have > about the environment they're running in, and how do we make those > expectations hold? This should probably then be documented in policy. Agreed, but also, we need a specific answer for TMPDIR. We pass things into maintainer scripts because we want to change their behaviour (eg. DEBIAN_FRONTEND). So which specific variables are required to be reset by maintainer scripts and under what circumstances? signature.asc Description: PGP signature
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Thu, Nov 10, 2022 at 12:08:55PM +0100, Marco d'Itri wrote: > > But are you in essence saying that libpam-tmpdir requires that *every > > maintainer script* that runs things as non-root, or starts processes > > that do that, unset TMPDIR first? > This would not be right, because it is totally valid to set $TMPDIR for > the root user too. > The real issue here is that TMPDIR, like some other variables, should > not be propagated when switching privileges from the user to root. > > But here we have ANOTHER issue: whatever ends up initialising mysql does > not run as root, but still uses $TMPDIR provided by the root environment. > Since there is no guarantee at all that $TMPDIR can be accessed (not > just be writeable!) by other users then in this case it is correct to > request that the package ignores $TMPDIR. I think this statement is in violent agreement with the statement I made above? I agree that there is now no guarantee that $TMPDIR can be accessed, because of what libpam-tmpdir is doing. However, if you were to ask an expert from the nineties, that was a reasonable assumption. So what changed, and where and how precisely is this change supposed to be accomodated? Every relevant maintainer script? dpkg? Or somewhere else? signature.asc Description: PGP signature
Bug#1022994: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Nov 12, Otto Kekäläinen wrote: > Instead of manually trying to manage TMPDIR env variable in various > places, we should have a standardized way to run maintainer scripts in > clean shell sessions that have all env variables set automatically > correctly. This is not about maintainer scripts, but about programs which change the UID without sanitizing the environment. -- ciao, Marco signature.asc Description: PGP signature
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On 11/12/22 14:09, Daniel Black wrote: So what Fedora does is a prep script called at StartPre on their systemd service. https://src.fedoraproject.org/rpms/mariadb/blob/rawhide/f/mariadb-prepare-db-dir.sh Which even recently was seen as bloated (https://lists.launchpad.net/maria-discuss/msg06376.html). What could be done is a oneshot (https://www.redhat.com/sysadmin/systemd-oneshot-service) service before MariaDB/MySQL that does the installation. Either installation or startup triggered. At least on TMPDIR side, the systemd side PrivateTmp is default (https://www.freedesktop.org/software/systemd/man/systemd.exec.html#PrivateTmp=) for at least the oneshot service. The option for loading files into MariaDB was the only reason this wasn't set in the default MariaDB systemd file. During today's FreedomBox meet, we have discussed that systemd'd PrivateTmp= is a better solution than libpam-tmpdir for FreedomBox at least as systemd makes a cleaner mount isolation between processes instead of managing directories and permissions. For this reason, we believe that we can stop using libpam-tmpdir if most of the daemons on the system use PrivateTmp=yes. For a while now, FreedomBox has been forcefully adding systemd security features to daemons that don't enable them. Without upstream blessing, we can only do this for smaller applications than something like MariaDB/MySQL due the testing effort needed. How User= systemd directives work with lbpam-tmpdir I'm not sure, however without a setuid there shouldn't be an invalid TMPDIR env variable there. libpam-tmpdir does not seem to effect systemd's process execution. See the following session on system with libpam-tmpdir installed: $ sudo --user mysql /usr/bin/bash -c 'echo TMPDIR=$TMPDIR' TMPDIR=/tmp/user/119 $ sudo systemd-run --pipe --uid=mysql /usr/bin/bash -c 'echo TMPDIR=$TMPDIR' Running as unit: run-u30.service TMPDIR= $ sudo systemd-run --pipe --property=PrivateTmp=yes --uid=mysql /usr/bin/bash -c 'echo TMPDIR=$TMPDIR' Running as unit: run-u31.service TMPDIR= -- Sunil OpenPGP_0x36C361440C9BC971.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Bug#1022994: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
So what Fedora does is a prep script called at StartPre on their systemd service. https://src.fedoraproject.org/rpms/mariadb/blob/rawhide/f/mariadb-prepare-db-dir.sh Which even recently was seen as bloated (https://lists.launchpad.net/maria-discuss/msg06376.html). What could be done is a oneshot (https://www.redhat.com/sysadmin/systemd-oneshot-service) service before MariaDB/MySQL that does the installation. Either installation or startup triggered. At least on TMPDIR side, the systemd side PrivateTmp is default (https://www.freedesktop.org/software/systemd/man/systemd.exec.html#PrivateTmp=) for at least the oneshot service. The option for loading files into MariaDB was the only reason this wasn't set in the default MariaDB systemd file. How User= systemd directives work with lbpam-tmpdir I'm not sure, however without a setuid there shouldn't be an invalid TMPDIR env variable there. Also not perfect, but maybe viable. On Sun, Nov 13, 2022 at 8:14 AM Otto Kekäläinen wrote: > > > > I think the answer to this should probably be established by the > > > libpam-tmpdir maintainer and documented first, for fear of someone else > > > later coming along and saying that the maintainer script incorrectly > > > ignores TMPDIR because we started ignoring it to resolve this bug. So I > > > copied debian-devel@ for comment. > > > > I'm not sure this is libpam-tmpdir specific, but rather a bit more > > general: what are the expectations that maintainer scripts can have > > about the environment they're running in, and how do we make those > > expectations hold? This should probably then be documented in policy. > > This MySQL 8.0 in Sid tmpdir issue is identical to MariaDB 10.6 in Sid > tmpdir issue, as they are based on historically the same maintainer > script. > > MariaDB duplicate of this has been filed as > https://jira.mariadb.org/browse/MDEV-29910 and > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1022994 > > The suggested solution in > https://salsa.debian.org/mariadb-team/mariadb-server/-/merge_requests/23 > to set an empty TMPDIR does not seem to work - or at least CI shows > that even a basic install fails after that. > > Instead of manually trying to manage TMPDIR env variable in various > places, we should have a standardized way to run maintainer scripts in > clean shell sessions that have all env variables set automatically > correctly. > > More proposals on how to handle this are welcome at > https://salsa.debian.org/mariadb-team/mariadb-server/-/merge_requests > :) > > - Otto
Bug#1022994: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
> > I think the answer to this should probably be established by the > > libpam-tmpdir maintainer and documented first, for fear of someone else > > later coming along and saying that the maintainer script incorrectly > > ignores TMPDIR because we started ignoring it to resolve this bug. So I > > copied debian-devel@ for comment. > > I'm not sure this is libpam-tmpdir specific, but rather a bit more > general: what are the expectations that maintainer scripts can have > about the environment they're running in, and how do we make those > expectations hold? This should probably then be documented in policy. This MySQL 8.0 in Sid tmpdir issue is identical to MariaDB 10.6 in Sid tmpdir issue, as they are based on historically the same maintainer script. MariaDB duplicate of this has been filed as https://jira.mariadb.org/browse/MDEV-29910 and https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1022994 The suggested solution in https://salsa.debian.org/mariadb-team/mariadb-server/-/merge_requests/23 to set an empty TMPDIR does not seem to work - or at least CI shows that even a basic install fails after that. Instead of manually trying to manage TMPDIR env variable in various places, we should have a standardized way to run maintainer scripts in clean shell sessions that have all env variables set automatically correctly. More proposals on how to handle this are welcome at https://salsa.debian.org/mariadb-team/mariadb-server/-/merge_requests :) - Otto
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On 2022-11-10 at 16:37:53, Tollef Fog Heen wrote: > ]] Robie Basak > > > But are you in essence saying that libpam-tmpdir requires that *every > > maintainer script* that runs things as non-root, or starts processes > > that do that, unset TMPDIR first? > > I think it's more wide than that: If you change UID, you need to > sanitise the environment. Your HOME is likely to be wrong. PATH might > very well be pointing at directories which are not appropriate for the > user you're changing the UID to, etc. I believe this is the best practice. For example, sudo typically passes through only a handful of environment variables, such as TERM, to avoid things like insecure PATH entries. For example, if MySQL invoked a binary in PATH and I had a custom script named the same thing that had insecure behaviour when invoked as another user, that would be bad. OpenSSH also sanitizes the environment passed over the connection. Without getting into a debate about systemd, this is one the pieces of functionality it provides, since it runs binaries in a clean environment. You can probably get most of that in a sysvinit script by using `env -i` with a handful of environment variables specifically overridden if necessary. Then it would be very obvious what dependencies the binary had on the outside environment. > > I think the answer to this should probably be established by the > > libpam-tmpdir maintainer and documented first, for fear of someone else > > later coming along and saying that the maintainer script incorrectly > > ignores TMPDIR because we started ignoring it to resolve this bug. So I > > copied debian-devel@ for comment. > > I'm not sure this is libpam-tmpdir specific, but rather a bit more > general: what are the expectations that maintainer scripts can have > about the environment they're running in, and how do we make those > expectations hold? This should probably then be documented in policy. I agree documentation here is helpful. For reference, this was an invocation from `sudo aptitude` as part of a normal package upgrade, which I think is a relatively common situation to be in. Possibly also an initscript helper which enables developers to do the right thing automatically, or a flag to start-stop-daemon, or other tooling would be beneficial to help people easily solve this problem without thinking too much about it. -- brian m. carlson (he/him or they/them) Toronto, Ontario, CA signature.asc Description: PGP signature
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
]] Robie Basak > But are you in essence saying that libpam-tmpdir requires that *every > maintainer script* that runs things as non-root, or starts processes > that do that, unset TMPDIR first? I think it's more wide than that: If you change UID, you need to sanitise the environment. Your HOME is likely to be wrong. PATH might very well be pointing at directories which are not appropriate for the user you're changing the UID to, etc. > I think the answer to this should probably be established by the > libpam-tmpdir maintainer and documented first, for fear of someone else > later coming along and saying that the maintainer script incorrectly > ignores TMPDIR because we started ignoring it to resolve this bug. So I > copied debian-devel@ for comment. I'm not sure this is libpam-tmpdir specific, but rather a bit more general: what are the expectations that maintainer scripts can have about the environment they're running in, and how do we make those expectations hold? This should probably then be documented in policy. Cheers, -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
On Nov 10, Robie Basak wrote: > Thank you for the report. Adding debian-devel@ and the libpam-tmpdir > maintainer for wider discussion. > > On Thu, Nov 10, 2022 at 12:54:34AM +, brian m. carlson wrote: > > On my systems, I use libpam-tmpdir, which provides each user with a > > private temporary directory owned and accessible only by them under > > /tmp/user/UID (e.g., /tmp/user/1000). PAM sets the TMPDIR variable to > > this value upon creating a session. > > > > When I upgrade mysql-server-8.0, it is obviously as root, so TMPDIR is > > set to /tmp/user/0. This value does not work since MySQL doesn't run as > > root, and so MySQL fails to start after upgrade in such a configuration, > > like so: > > I think I understand the problem. > > But are you in essence saying that libpam-tmpdir requires that *every > maintainer script* that runs things as non-root, or starts processes > that do that, unset TMPDIR first? This would not be right, because it is totally valid to set $TMPDIR for the root user too. The real issue here is that TMPDIR, like some other variables, should not be propagated when switching privileges from the user to root. But here we have ANOTHER issue: whatever ends up initialising mysql does not run as root, but still uses $TMPDIR provided by the root environment. Since there is no guarantee at all that $TMPDIR can be accessed (not just be writeable!) by other users then in this case it is correct to request that the package ignores $TMPDIR. -- ciao, Marco signature.asc Description: PGP signature
Bug#1023778: TMPDIR behaviour in maintainer scripts [was: Re: Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir]
Thank you for the report. Adding debian-devel@ and the libpam-tmpdir maintainer for wider discussion. On Thu, Nov 10, 2022 at 12:54:34AM +, brian m. carlson wrote: > On my systems, I use libpam-tmpdir, which provides each user with a > private temporary directory owned and accessible only by them under > /tmp/user/UID (e.g., /tmp/user/1000). PAM sets the TMPDIR variable to > this value upon creating a session. > > When I upgrade mysql-server-8.0, it is obviously as root, so TMPDIR is > set to /tmp/user/0. This value does not work since MySQL doesn't run as > root, and so MySQL fails to start after upgrade in such a configuration, > like so: I think I understand the problem. But are you in essence saying that libpam-tmpdir requires that *every maintainer script* that runs things as non-root, or starts processes that do that, unset TMPDIR first? Wouldn't ignoring TMPDIR make it less useful for users who wish to set it for other purposes, since maintainer scripts would stop respecting this? Or, if this universally is the right thing to do, then maybe dpkg should be doing it rather than individual maintainer script? I'd be happy to unset TMPDIR if there's clear consensus that this is what such maintainer scripts should always do, but it isn't clear to me that this is correct. Note that in this case it's the database initialization that's affected, which happens to be done by mysqld running as a daemon, but in a way that is equivalent to it not being a daemon - the task will be complete, with no lingering processes, before the maintainer script exits. In searching all I could find was https://lists.debian.org/debian-devel/2005/02/msg00374.html which is a similar question but doesn't look like it ever concluded. I think the answer to this should probably be established by the libpam-tmpdir maintainer and documented first, for fear of someone else later coming along and saying that the maintainer script incorrectly ignores TMPDIR because we started ignoring it to resolve this bug. So I copied debian-devel@ for comment. Thanks, Robie signature.asc Description: PGP signature
Bug#1023778: mysql-server-8.0: fails to restart on upgrade with libpam-tmpdir
Package: mysql-server-8.0 Version: 8.0.30-1+b2 Severity: normal On my systems, I use libpam-tmpdir, which provides each user with a private temporary directory owned and accessible only by them under /tmp/user/UID (e.g., /tmp/user/1000). PAM sets the TMPDIR variable to this value upon creating a session. When I upgrade mysql-server-8.0, it is obviously as root, so TMPDIR is set to /tmp/user/0. This value does not work since MySQL doesn't run as root, and so MySQL fails to start after upgrade in such a configuration, like so: 2022-09-28T19:02:43.113663Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.30-1) starting as process 134423 2022-09-28T19:02:43.156068Z 1 [System] [MY-011012] [Server] Starting upgrade of data directory. 2022-09-28T19:02:43.156091Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. mysqld: Can't create/write to file '/tmp/user/0/ibafhFAQ' (OS errno 13 - Permission denied) 2022-09-28T19:02:44.224787Z 1 [ERROR] [MY-012576] [InnoDB] Unable to create temporary file inside "/tmp/user/0"; errno: 13 2022-09-28T19:02:44.224823Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error. 2022-09-28T19:02:44.224856Z 1 [ERROR] [MY-011013] [Server] Failed to initialize DD Storage Engine. 2022-09-28T19:02:44.225102Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed. 2022-09-28T19:02:44.225217Z 0 [ERROR] [MY-010119] [Server] Aborting 2022-09-28T19:02:44.226017Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.30-1) (Debian). 2022-10-17T13:26:08.726136Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.30-1+b1) starting as process 36399 2022-10-17T13:26:08.755895Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. mysqld: Can't create/write to file '/tmp/user/0/ibJujAnM' (OS errno 13 - Permission denied) 2022-10-17T13:26:09.299586Z 1 [ERROR] [MY-012576] [InnoDB] Unable to create temporary file inside "/tmp/user/0"; errno: 13 2022-10-17T13:26:09.299605Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error. 2022-10-17T13:26:09.299625Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine 2022-10-17T13:26:09.299737Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed. 2022-10-17T13:26:09.299751Z 0 [ERROR] [MY-010119] [Server] Aborting 2022-10-17T13:26:09.300393Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.30-1+b1) (Debian). 2022-11-09T21:40:40.722163Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.30-1+b2) starting as process 110969 2022-11-09T21:40:40.731948Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. mysqld: Can't create/write to file '/tmp/user/0/iby1UOtJ' (OS errno 13 - Permission denied) 2022-11-09T21:40:41.516120Z 1 [ERROR] [MY-012576] [InnoDB] Unable to create temporary file inside "/tmp/user/0"; errno: 13 2022-11-09T21:40:41.516155Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error. 2022-11-09T21:40:41.516191Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine 2022-11-09T21:40:41.516337Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed. 2022-11-09T21:40:41.516362Z 0 [ERROR] [MY-010119] [Server] Aborting 2022-11-09T21:40:41.517553Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.30-1+b2) (Debian). It's probably appropriate to sanitize the environment somewhat before starting MySQL to avoid this problem. For example, in my testing, `sudo env -u TMPDIR dpkg-reconfigure mysql-server-8.0` causes the server to restart properly, but there may be other environment variables that may result in undesired behaviour. Note that all of this is from another system, so none of the usual information is relevant below. -- System Information: Debian Release: bookworm/sid APT prefers stable-security APT policy: (500, 'stable-security'), (500, 'unstable'), (500, 'stable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 6.0.0-2-amd64 (SMP w/20 CPU threads; PREEMPT) Kernel taint flags: TAINT_USER, TAINT_WARN Locale: LANG=en_CA.UTF-8, LC_CTYPE=en_CA.UTF-8 (charmap=UTF-8), LANGUAGE=en_CA:en Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages mysql-server-8.0 depends on: ii adduser3.129 ii debconf [debconf-2.0] 1.5.79 pn mysql-client-8.0 ii mysql-common 5.8+1.0.8 pn mysql-server-core-8.0 ii passwd 1:4.12.3+dfsg1-3 ii perl 5.36.0-4 ii psmisc 23.5-3 ii sysvinit-utils [lsb-base] 3.05-6 Versions of packages mysql-server-8.0 recommends: pn libhtml-template-perl pn mecab-ipadic-utf8 Versions of packages
