Re: [CentOS] systemd private tmp dirs

[Les Mikesell]

On Thu, Apr 16, 2015 at 9:25 AM, Matthew Miller  wrote:
> On Thu, Apr 16, 2015 at 07:44:21AM -0500, Les Mikesell wrote:
>> > The issue here really isn't systemd or the PrivateTmp feature but the
>> > fact that some applications don't properly distinguish between temporary
>> > files and data files.
>> Maybe, but if an application wants a private directory for temporary
>> files, shouldn't it create and manage that directory itself instead of
>> being second-guessed by the default configuration of the OS?
>
> This one I have a clear answer for: no. It's the distribution's job to
> help regularize application practices, especially when they don't
> follow good practices for security.

Really?  I would have expected that it was the distribution's job to
not surprise coders or administrators.  Particularly for 'enterprise'
operating systems where the point is to keep the same application
working the same way, often for the life of a company.

> Ideally, we work with upstreams on
> this, but sometimes where it's just a matter of configuration, we
> choose to exercise options to make everything fit together.

I typically have many web 'applications' running on the same system
under the same apache instance, distinguished only by the top level
directory in the url.   Even if it made sense to someone to surprise
these applications by remapping the filesystem for some reason, why
would it make sense for them to share what the system thinks it is
making private?

-- 
   Les Mikesell
 lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Matthew Miller]

On Wed, Apr 15, 2015 at 08:52:16PM -0500, Les Mikesell wrote:
> Mostly I'm interested in avoiding surprises and having code that isn't
> married to the weirdness of any particular version of any particular
> distribution.  And I found this to be pretty surprising, given that I

That's always difficult, as distributions all have their own quirks. We
have some semblance of a standard in the LSB, but it's not strongly
adhered to by any distro. For file locations in specific, there is the
FHS (see http://www.linuxbase.org/betaspecs/fhs/fhs.html for latest
draft), and while I think it could be a little more clear on /tmp,
there are no promises of anything other than that /tmp must be
available to programs — and that programs can't count on data there to
be preserved. I think it's clear that /var/lib/{something} is a better
match.

Of course, with some degree of irony, providing a defacto standard base
across Linux distributions is, I think, one of the goals of the
upstream project — and, whatever you may think of it, as measured by
distribution adoption, that seems to be _more_ successful in practice
than any previous standarization effort. So, with an eye to the future,
"do what systemd suggests" is, really, not your worst option.


> could see the file in /tmp and could read the code that was looking
> there.   So, from the point of view of writing portable code, how
> should something handle this to run on any unix-like system?

Do exactly what twiki does — provide a configuration option.

-- 
Matthew Miller

Fedora Project Leader
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Matthew Miller]

On Thu, Apr 16, 2015 at 07:44:21AM -0500, Les Mikesell wrote:
> > The issue here really isn't systemd or the PrivateTmp feature but the
> > fact that some applications don't properly distinguish between temporary
> > files and data files.
> Maybe, but if an application wants a private directory for temporary
> files, shouldn't it create and manage that directory itself instead of
> being second-guessed by the default configuration of the OS?

This one I have a clear answer for: no. It's the distribution's job to
help regularize application practices, especially when they don't
follow good practices for security. Ideally, we work with upstreams on
this, but sometimes where it's just a matter of configuration, we
choose to exercise options to make everything fit together.

> filesystem.And as far as what the default location should be -
> what would be correct for portable code?   Isn't /var/lib/something
> kind of linux-centric?  Where can an application expect to be able to
> write?

Linux-centric? Linux/Unix-centric, maybe. I mean, that's not gonna work
on VMS or MS Windows — but then, neither is /tmp.

-- 
Matthew Miller

Fedora Project Leader
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Les Mikesell]

On Thu, Apr 16, 2015 at 6:58 AM, Dennis Jacobfeuerborn
 wrote:
> >>
>> No, systemd actually remaps /tmp from apache - and apparently most
>> other daemons - to private directories  below /tmp with configs as
>> shipped.  The command line tool wrote the file to /tmp as expected.
>> The perl code running under httpd reading what it thought was /tmp was
>> actually looking under /tmp/systemd-private-something.  I'm beginning
>> to see why so much of EPEL isn't included in epel7 yet.
>
> The issue here really isn't systemd or the PrivateTmp feature but the
> fact that some applications don't properly distinguish between temporary
> files and data files.

Maybe, but if an application wants a private directory for temporary
files, shouldn't it create and manage that directory itself instead of
being second-guessed by the default configuration of the OS?

> Temporary files are files the application generates temporarily for
> internal processing and that are not to be touched by anybody else.
> If as in the twiki backup case the files generated are to be used by
> somebody else after twiki is done generating them then these are regular
> data files and not temporary files.

This is very fuzzy...  It is really all the application code
creating/reading the files, and they are intended to be created at
least daily with timestamps in the name and not live forever.

> The application should have a configuration option to set its data
> directory and it should default to /var/lib/.
> In cases where this option is not available and the application "abuses"
> the tmp directory as data directory there is probably no other option
> than to the set PrivateTmp=false in the service file.

It does that - the issue is just that it is handy (and common) to use
cron to do the scheduled runs and what the application sees as
absolute file paths are perverted by the system into something
surprising. The 'modern' approach might be to provide a rest type
interface in the web application so the cron job could use wget/curl
to access a URL instead of running the perl code itself. But that's
also kind of weird to have to do to get a consistent view of the
filesystem.And as far as what the default location should be -
what would be correct for portable code?   Isn't /var/lib/something
kind of linux-centric?  Where can an application expect to be able to
write?

-- 
   Les Mikesell
 lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Dennis Jacobfeuerborn]

On 16.04.2015 04:15, Les Mikesell wrote:
> On Wed, Apr 15, 2015 at 9:00 PM, John R Pierce  wrote:
>> On 4/15/2015 6:52 PM, Les Mikesell wrote:
>>>
>>> Mostly I'm interested in avoiding surprises and having code that isn't
>>> married to the weirdness of any particular version of any particular
>>> distribution.  And I found this to be pretty surprising, given that I
>>> could see the file in /tmp and could read the code that was looking
>>> there.   So, from the point of view of writing portable code, how
>>> should something handle this to run on any unix-like system?
>>
>>
>> you sure this had nothing to do with selinux not letting perl running as the
>> http user write there?
>>
> 
> No, systemd actually remaps /tmp from apache - and apparently most
> other daemons - to private directories  below /tmp with configs as
> shipped.  The command line tool wrote the file to /tmp as expected.
> The perl code running under httpd reading what it thought was /tmp was
> actually looking under /tmp/systemd-private-something.  I'm beginning
> to see why so much of EPEL isn't included in epel7 yet.

The issue here really isn't systemd or the PrivateTmp feature but the
fact that some applications don't properly distinguish between temporary
files and data files.
Temporary files are files the application generates temporarily for
internal processing and that are not to be touched by anybody else.
If as in the twiki backup case the files generated are to be used by
somebody else after twiki is done generating them then these are regular
data files and not temporary files.
The application should have a configuration option to set its data
directory and it should default to /var/lib/.
In cases where this option is not available and the application "abuses"
the tmp directory as data directory there is probably no other option
than to the set PrivateTmp=false in the service file.

Regards,
  Dennis
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Les Mikesell]

On Wed, Apr 15, 2015 at 9:00 PM, John R Pierce  wrote:
> On 4/15/2015 6:52 PM, Les Mikesell wrote:
>>
>> Mostly I'm interested in avoiding surprises and having code that isn't
>> married to the weirdness of any particular version of any particular
>> distribution.  And I found this to be pretty surprising, given that I
>> could see the file in /tmp and could read the code that was looking
>> there.   So, from the point of view of writing portable code, how
>> should something handle this to run on any unix-like system?
>
>
> you sure this had nothing to do with selinux not letting perl running as the
> http user write there?
>

No, systemd actually remaps /tmp from apache - and apparently most
other daemons - to private directories  below /tmp with configs as
shipped.  The command line tool wrote the file to /tmp as expected.
The perl code running under httpd reading what it thought was /tmp was
actually looking under /tmp/systemd-private-something.  I'm beginning
to see why so much of EPEL isn't included in epel7 yet.

-- 
   Les Mikesell
lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[John R Pierce]

On 4/15/2015 6:52 PM, Les Mikesell wrote:

Mostly I'm interested in avoiding surprises and having code that isn't
married to the weirdness of any particular version of any particular
distribution.  And I found this to be pretty surprising, given that I
could see the file in /tmp and could read the code that was looking
there.   So, from the point of view of writing portable code, how
should something handle this to run on any unix-like system?


you sure this had nothing to do with selinux not letting perl running as 
the http user write there?




--
john r pierce, recycling bits in santa cruz

___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Les Mikesell]

On Wed, Apr 15, 2015 at 6:48 PM, Matthew Miller  wrote:
> On Wed, Apr 15, 2015 at 05:31:52PM -0500, Les Mikesell wrote:
>> Thanks - I can see how those would work once you understand what is
>> broken on the target system and why, but is there a way that programs
>> 'should' be written to run with/without systemd?   That just happened
>> to be the first thing I've tried to move over that wasn't already
>> packaged and adapted - I expect to hit many more.
>
> This isn't really a systemd thing. It's a standard Linux kernel
> feature, which could also be enabled with (for example) pam_namespace.
> Systemd happens to make it easy, so we started enabling it for services
> which would benefit on Fedora, and that was inherited into RHEL and
> CentOS. See the change page for this
> .
>
> If you're really interested in learning every possible thing about
> systemd, you could of course go through the author's blog post series
> "systemd for administrators" — see
> . It's
> pretty useful.
>
> Or, if you're mostly interested in packaging something up to run in a
> nice way in the Fedora/RHEL/CentOS-ecosystem, the Fedora packaging
> guidelines for systemd might help; those are at
> . I notice that
> private temp dirs aren't mentioned there (not a bad thing to add,
> really) but you'll find some other advice that might be helpful. (Take
> a look at private devices and networking for a related issue.)

Mostly I'm interested in avoiding surprises and having code that isn't
married to the weirdness of any particular version of any particular
distribution.  And I found this to be pretty surprising, given that I
could see the file in /tmp and could read the code that was looking
there.   So, from the point of view of writing portable code, how
should something handle this to run on any unix-like system?

-- 
   Les Mikesell
 lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Matthew Miller]

On Wed, Apr 15, 2015 at 05:31:52PM -0500, Les Mikesell wrote:
> Thanks - I can see how those would work once you understand what is
> broken on the target system and why, but is there a way that programs
> 'should' be written to run with/without systemd?   That just happened
> to be the first thing I've tried to move over that wasn't already
> packaged and adapted - I expect to hit many more.

This isn't really a systemd thing. It's a standard Linux kernel
feature, which could also be enabled with (for example) pam_namespace.
Systemd happens to make it easy, so we started enabling it for services
which would benefit on Fedora, and that was inherited into RHEL and
CentOS. See the change page for this
.

If you're really interested in learning every possible thing about
systemd, you could of course go through the author's blog post series
"systemd for administrators" — see
. It's
pretty useful. 

Or, if you're mostly interested in packaging something up to run in a
nice way in the Fedora/RHEL/CentOS-ecosystem, the Fedora packaging
guidelines for systemd might help; those are at 
. I notice that
private temp dirs aren't mentioned there (not a bad thing to add,
really) but you'll find some other advice that might be helpful. (Take
a look at private devices and networking for a related issue.)


-- 
Matthew Miller

Fedora Project Leader
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Les Mikesell]

On Wed, Apr 15, 2015 at 5:01 PM, Matthew Miller  wrote:
> On Wed, Apr 15, 2015 at 04:15:23PM -0500, Les Mikesell wrote:
>> > Why does this directory have to be /tmp rather than a specific
>> > directory belonging to twiki?
>> Twiki is a perl web application run under apache.  It doesn't have its
>> own uid.  It doesn't 'have' to be anywhere in particular but that is
>> the way it was written and thus has very confusing results when trying
>> to move it to CentOS 7.  Is there some generic approach to fixing this
>> kind of breakage (that is, to make it work and not confusing, not to
>> say it was broken as designed)?To function as a backup, it
>> probably shouldn't default to being in the same directory as the files
>> it backs up.
>
> There are two (sane) options, I think.
>
> The first, and I think the best, is to configure twiki to share files
> in some specific location rather than /tmp. It doesn't have to be the
> same directory as the files being backed up — maybe something under
> /var/lib/twiki (or /var/local/twiki).
>
> If the twiki backup plugin didn't allow this to be configured, I would
> argue that it _is_ broken by design. But a quick Google search leads me
> to , which
> shows that it is indeed configurable, so I'm just going to call it a
> questionable default. :)
>
> If you want to keep that default, though, the second approach would be
> to configure Apache to not use a private namespace, which I don't
> recommend because you lose the security benefit. To do that, put
>
> [Service]
> PrivateTmp=false
>
> in /etc/systemd/system/httpd.service (which may not exist).
>

Thanks - I can see how those would work once you understand what is
broken on the target system and why, but is there a way that programs
'should' be written to run with/without systemd?   That just happened
to be the first thing I've tried to move over that wasn't already
packaged and adapted - I expect to hit many more.

-- 
   Les Mikesell
 lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Matthew Miller]

On Wed, Apr 15, 2015 at 04:15:23PM -0500, Les Mikesell wrote:
> > Why does this directory have to be /tmp rather than a specific
> > directory belonging to twiki?
> Twiki is a perl web application run under apache.  It doesn't have its
> own uid.  It doesn't 'have' to be anywhere in particular but that is
> the way it was written and thus has very confusing results when trying
> to move it to CentOS 7.  Is there some generic approach to fixing this
> kind of breakage (that is, to make it work and not confusing, not to
> say it was broken as designed)?To function as a backup, it
> probably shouldn't default to being in the same directory as the files
> it backs up.

There are two (sane) options, I think.

The first, and I think the best, is to configure twiki to share files
in some specific location rather than /tmp. It doesn't have to be the
same directory as the files being backed up — maybe something under
/var/lib/twiki (or /var/local/twiki).

If the twiki backup plugin didn't allow this to be configured, I would
argue that it _is_ broken by design. But a quick Google search leads me
to , which
shows that it is indeed configurable, so I'm just going to call it a
questionable default. :)

If you want to keep that default, though, the second approach would be
to configure Apache to not use a private namespace, which I don't
recommend because you lose the security benefit. To do that, put

[Service]
PrivateTmp=false

in /etc/systemd/system/httpd.service (which may not exist).

-- 
Matthew Miller

Fedora Project Leader
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Les Mikesell]

On Wed, Apr 15, 2015 at 4:07 PM, Matthew Miller  wrote:
> On Wed, Apr 15, 2015 at 03:55:34PM -0500, Les Mikesell wrote:
>> Is there a generic way that processes written to share files with
>> (say) apache in /tmp can figure out that they are running on an OS
>> with systemd and in that case, where the daemon in question thinks
>> /tmp is?
>>
>> For example, twiki has a backup/restore add-in where the backup part
>> is normally done from cron with a command line script but the
>> resulting archives that go in /tmp  are supposed to be seen in the web
>> interface where you can choose and restore from them.  How should that
>> have been written so the file lands where systemd has remapped /tmp
>> for httpd if it happens to be running on a host with systemd?
>
> Why does this directory have to be /tmp rather than a specific
> directory belonging to twiki?

Twiki is a perl web application run under apache.  It doesn't have its
own uid.  It doesn't 'have' to be anywhere in particular but that is
the way it was written and thus has very confusing results when trying
to move it to CentOS 7.  Is there some generic approach to fixing this
kind of breakage (that is, to make it work and not confusing, not to
say it was broken as designed)?To function as a backup, it
probably shouldn't default to being in the same directory as the files
it backs up.

-- 
   Les Mikesell
 lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] systemd private tmp dirs

[Matthew Miller]

On Wed, Apr 15, 2015 at 03:55:34PM -0500, Les Mikesell wrote:
> Is there a generic way that processes written to share files with
> (say) apache in /tmp can figure out that they are running on an OS
> with systemd and in that case, where the daemon in question thinks
> /tmp is?
> 
> For example, twiki has a backup/restore add-in where the backup part
> is normally done from cron with a command line script but the
> resulting archives that go in /tmp  are supposed to be seen in the web
> interface where you can choose and restore from them.  How should that
> have been written so the file lands where systemd has remapped /tmp
> for httpd if it happens to be running on a host with systemd?

Why does this directory have to be /tmp rather than a specific
directory belonging to twiki?


-- 
Matthew Miller

Fedora Project Leader
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

[CentOS] systemd private tmp dirs

[Les Mikesell]

Is there a generic way that processes written to share files with
(say) apache in /tmp can figure out that they are running on an OS
with systemd and in that case, where the daemon in question thinks
/tmp is?

For example, twiki has a backup/restore add-in where the backup part
is normally done from cron with a command line script but the
resulting archives that go in /tmp  are supposed to be seen in the web
interface where you can choose and restore from them.  How should that
have been written so the file lands where systemd has remapped /tmp
for httpd if it happens to be running on a host with systemd?

-- 
   Les Mikesell
 lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos