Re: [systemd-devel] Why did you set MountFlags=slave in systemd-udevd.service.in

[Hongzhi, Song]

Thank you very much. It dose work.

Would you tell me that how the '--no-block' works,

I doesn't deeply understand the interpretation in

https://www.freedesktop.org/software/systemd/man/systemd-mount.html#SYSTEMD_MOUNT_OPTIONS=


And why systemd-mount can solve the problem brought by MountFlag=slave.


Best Regard,

Hongzhi


On 2018年02月28日 03:02, Andrei Borzenkov wrote:

27.02.2018 17:20, Hongzhi, Song пишет:

Hi,

thank for your help, but I still have some question.


My current linux system init uses systemd and udev, with
'automount.rules' and 'mount.sh' in /etc/udev/,

to manage device. But owning to MountFlags=slave, hotpluggable media
(e.g., /dev/sda1 )

can be mounted again in host, whereas can't be formatted by mkfs.ext4 in
host with error

message '/dev/sda1 is apparently in use by the system; will not make a
filesystem here!'.


In your reply, you told me to invoke "systemd-mount" from udev rules. Do
you mean that

I should replace /bin/mount with /usr/bin/systemd-mount in mount.sh, or add

"RUN+='/usr/bin/systemd-mount $env{DEVNAME}'" to automount.rules?

1）

I  replaced /bin/mount with /usr/bin/systemd-mount in mount.sh.

     /usr/bin/systemd-mount $DEVNAME "/run/media/$name"

it prompted that

     systemd[1]: dev-sda.device: Job dev-sda.device/start timed out.
     systemd[1]: Timed out waiting for device dev-sda.device.
     systemd[1]: Dependency failed for /run/media/sda.
     systemd[1]: run-media-sda.mount: Job run-media-sda.mount/start
failed with result 'dependency'.
     systemd[1]: Dependency failed for File System Check on /dev/sda.
     systemd[1]: systemd-fsck@dev-sda.service: Job
systemd-fsck@dev-sda.service/start failed with result 'dependency'.
     systemd[1]: Startup finished in 16.692s (kernel) + 1min 32.605s
(userspace) = 1min 49.298s.
     systemd[1]: dev-sda.device: Job dev-sda.device/start failed with
result 'timeout'.

     ...

in /var/log/syslog.


Try

systemd-mount --no-block ...
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel


___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Best practice for prepopulating the CacheDirectory of dynamic users

[Antoine Pietri]

On Wed, Feb 28, 2018 at 6:59 PM, Lennart Poettering
 wrote:
> Not sure I follow. Why do you let the service hang around? If all you
> want to do is have it create the directory for you you could just run:
>
> # systemd-run -P -p DynamicUser=yes -p CacheDirectory=mywrapper --wait true
>
> That would be synchronous, would set up the dir and immediately
> return.

My bad, for some reason my understanding was that there was some
teardown happening after the service was run, but the problem was just
somewhere else.
This works great, thanks!

-- 
Antoine Pietri
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] custom var in sd_notify

[Lennart Poettering]

On Mo, 26.02.18 08:09, Mantas Mikulėnas (graw...@gmail.com) wrote:

> > Daemons can choose to send additional variables. However, it is recommended
> > to prefix variable names not listed above with X_.
> > So naturally i tried
> >
> > sd_notify(0, "X_ANSWER=42")
> >
> > and apparently systemd has no problem with that, but my questions are 2
> > now:
> >
> > 1) What does systemd do with this information?
> >
> 
> Nothing. The documentation just says in other words that "the init system
> must not reject packets with unknown variables", but doesn't say that
> systemd or any other init will store all of them anywhere.

Currently we indeed ignore those unknown variables entirely. It has
been requested that we expose these variables on the bus somehow. I
think that would be an OK addition, but we need to think about the
lifecycle of those vars then so that the vars we collect don't grow
without bounds.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Best practice for prepopulating the CacheDirectory of dynamic users

[Lennart Poettering]

On Mi, 28.02.18 18:22, Antoine Pietri (antoine.piet...@gmail.com) wrote:

> On Wed, Feb 28, 2018 at 6:13 PM, Lennart Poettering
>  wrote:
> >> Okay! Does that also apply to the other possible approach I sent in my
> >> second mail? (Running a `dummy sh -c read` service with systemd-run,
> >> do the setup and kill it when the setup is done). The advantage of
> >> that one is that you can do any arbitrary processing while staying in
> >> the Python code.
> >
> > I am not sure I follow?
> 
> If you run from the script:
> 
> systemd-run -P -p DynamicUser=yes -p CacheDirectory=mywrapper sh -c read
> 
> This will do all the setup with the symlink to /var/cache/private, and
> then just hang. While the process is hanging, you can do the
> processing you need in the cache directory, including populating it
> with whatever you want.
> 
> Once your processing is over, you can kill the systemd-run process. On
> subsequent calls to systemd-run, the files you added will just be
> recursively chmod()ed by systemd, so you should just get back the
> directory populated with your files with the correct permissions.

Not sure I follow. Why do you let the service hang around? If all you
want to do is have it create the directory for you you could just run:

# systemd-run -P -p DynamicUser=yes -p CacheDirectory=mywrapper --wait true

That would be synchronous, would set up the dir and immediately
return.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Best practice for prepopulating the CacheDirectory of dynamic users

[Lennart Poettering]

On Mi, 28.02.18 17:38, Antoine Pietri (antoine.piet...@gmail.com) wrote:

> On Wed, Feb 28, 2018 at 5:24 PM, Lennart Poettering
>  wrote:
> > Does it have to be a writable copy? if not you could just do '-p
> > BindReadOnlyPaths=/path/to/my/source:/var/cache/mywrapper'
> 
> Yes it does, the build happens in place.
> 
> > That said, maybe we should add a concept of TemplateCacheDirectory= or
> > so, which would allow prepopulating the dir from some external
> > source.
> 
> That would be cool, although if we can treat /var/cache/private as an
> API, it might be redundant with just using /var/cache/private as the
> template cache directory directly?

Well, if we'd have TemplateCacheDirectory= then you could do fun stuff
like having a single template dir, but multiple instances, and each
time you start a new instance it gets its own private copy
transparently and magically. 

> 
> >> - My current workaround is to shell-out to `systemd-run -p
> >> DynamicUser=yes ...` first to do a mkdir -p, then for a cp -R. This
> >> solution requires a lot of boilerplate from the Python wrapper and
> >> takes more time for no good reason, so I think it's not ideal.
> >
> > This sounds OK to me tbh.
> 
> Okay! Does that also apply to the other possible approach I sent in my
> second mail? (Running a `dummy sh -c read` service with systemd-run,
> do the setup and kill it when the setup is done). The advantage of
> that one is that you can do any arbitrary processing while staying in
> the Python code.

I am not sure I follow?

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Best practice for prepopulating the CacheDirectory of dynamic users

[Antoine Pietri]

On Wed, Feb 28, 2018 at 6:13 PM, Lennart Poettering
 wrote:
>> Okay! Does that also apply to the other possible approach I sent in my
>> second mail? (Running a `dummy sh -c read` service with systemd-run,
>> do the setup and kill it when the setup is done). The advantage of
>> that one is that you can do any arbitrary processing while staying in
>> the Python code.
>
> I am not sure I follow?

If you run from the script:

systemd-run -P -p DynamicUser=yes -p CacheDirectory=mywrapper sh -c read

This will do all the setup with the symlink to /var/cache/private, and
then just hang. While the process is hanging, you can do the
processing you need in the cache directory, including populating it
with whatever you want.

Once your processing is over, you can kill the systemd-run process. On
subsequent calls to systemd-run, the files you added will just be
recursively chmod()ed by systemd, so you should just get back the
directory populated with your files with the correct permissions.

-- 
Antoine Pietri
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Best practice for prepopulating the CacheDirectory of dynamic users

[Antoine Pietri]

On Wed, Feb 28, 2018 at 5:13 PM, aleivag  wrote:
> do everything in python and use `pystemd` (pip install pystemd, just have
> libsystemd installed and you should be fine)

This is not an option for me as one of our requirements is to have
everything packaged in the repos of Archlinux. But thanks for making
me aware of this library, and your advice also applies to shelling out
to systemd-run.

> and also ditch the
> cachedirectory in favor of PrivateTmp, that is always new when you start
> your unit, and always goes away with your unit.

I do need persistent data here to avoid recloning the repo at each
build, so it's not an option either. I'm not sure that changes
anything, though?

> so my recommendation, create a custom script with your build process and
> call it using pystemd.run.

This is kind of what we're doing right now with the same downsides,
but yeah that also works.

-- 
Antoine Pietri
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Best practice for prepopulating the CacheDirectory of dynamic users

[Antoine Pietri]

On Wed, Feb 28, 2018 at 5:24 PM, Lennart Poettering
 wrote:
> Does it have to be a writable copy? if not you could just do '-p
> BindReadOnlyPaths=/path/to/my/source:/var/cache/mywrapper'

Yes it does, the build happens in place.

> That said, maybe we should add a concept of TemplateCacheDirectory= or
> so, which would allow prepopulating the dir from some external
> source.

That would be cool, although if we can treat /var/cache/private as an
API, it might be redundant with just using /var/cache/private as the
template cache directory directly?

>> - My current workaround is to shell-out to `systemd-run -p
>> DynamicUser=yes ...` first to do a mkdir -p, then for a cp -R. This
>> solution requires a lot of boilerplate from the Python wrapper and
>> takes more time for no good reason, so I think it's not ideal.
>
> This sounds OK to me tbh.

Okay! Does that also apply to the other possible approach I sent in my
second mail? (Running a `dummy sh -c read` service with systemd-run,
do the setup and kill it when the setup is done). The advantage of
that one is that you can do any arbitrary processing while staying in
the Python code.

Thanks,

-- 
Antoine Pietri
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Best practice for prepopulating the CacheDirectory of dynamic users

[aleivag]

Hi Antoine:

2 disclosure before reading this:

1) i'm not part of systemd-devel team, and
2) this is also a shameless plug because i'm talking about a lib i created.

with that out of the way, here is my advice/solution.

do everything in python and use `pystemd` (pip install pystemd, just have
libsystemd installed and you should be fine), and also ditch the
cachedirectory in favor of PrivateTmp, that is always new when you start
your unit, and always goes away with your unit.

pystemd is  apython wrapper around a few libsystemd-dev, and it has a nice
module name pystemd.run, here is a example

import sys
import pystemd.run

pscript = """
import os
import shutil
print(shutil.copytree('/var/cache/dnf', '/tmp/dnf'))
print(os.listdir('/tmp/dnf'))
"""

pystemd.run(
['/usr/bin/python3', '-c', pscript],
stdout=sys.stdout, stderr=sys.stderr, wait=True,
env={'PYTHONUSERBASE': '/dev/null'},
extra={'DynamicUser': True, 'PrivateTmp': True},
)

this would output: something like
/tmp/dnf
['last_makecache', '.gpgkeyschecked.yum', 'rawhide-2d95c80a1fa0a67d',
'google-chrome-filenames.solvx', 'tempfiles.json',
'rawhide-filenames.solvx', 'google-chrome.solv', 'expired_repos.json',
'rawhide.solv', 'packages.db', 'google-chrome-eb0d6f10ccbdafba']


so my recommendation, create a custom script with your build process and
call it using pystemd.run. Now, you could also use systemd-run to run your
script, but then it would not have been a shameless plug, right?

hope it helps


Alvaro Leiva

On Wed, Feb 28, 2018 at 7:03 AM, Antoine Pietri 
wrote:

> On Tue, Feb 27, 2018 at 2:37 PM, Antoine Pietri
>  wrote:
> > - My current workaround is to shell-out to `systemd-run -p
> > DynamicUser=yes ...` first to do a mkdir -p, then for a cp -R. This
> > solution requires a lot of boilerplate from the Python wrapper and
> > takes more time for no good reason, so I think it's not ideal.
> >
> > - I believe another solution would be to modify /var/cache/private
> > directly, but I'm not sure it's a good practice to do so because I
> > don't know if this path is reliable or just an implementation detail.
> > Plus, it requires a weird special case compared to when I don't run
> > makepkg with systemd-run, as I have to insert something in the middle
> > of the copy destination path.
> >
> > - Maybe there's something else I'm missing that would allow me to do
> > this more cleanly?
>
> We came up with a third option, which looks a bit weird at first but
> should work:
>
> 1) systemd-run -P \
>   -p DynamicUser=yes \
>   -p CacheDirectory=mywrapper \
>   sh -c read
>
> 2) do the file operations in the Python code
> 3) send a "\n" or just kill() the systemd-run process when the setup is
> done.
>
> I am still not satisfied with any of the three options, so I would
> love to know what you think would be best. :-)
>
> --
> Antoine Pietri
>
> On Tue, Feb 27, 2018 at 2:37 PM, Antoine Pietri
>  wrote:
> > Hi!
> >
> > To experiment with systemd dynamic users, I started working on a
> > wrapper around a program that builds user packages (Archlinux makepkg)
> > and that refuses to be launched as root (for very good reasons). The
> > idea is that the wrapper just calls:
> >
> > systemd-run --pipe \
> >   -p DynamicUser=yes \
> >   -p CacheDirectory=mywrapper \
> >   -p WorkingDirectory=/var/cache/mywrapper/build/repo \
> >   makepkg
> >
> > However, to be able to run makepkg, its cache directory has to be
> > pre-populated with a clone of the package to build. So, from my
> > wrapper, I just did:
> >
> >   os.makedirs(os.path.dirname(build_dir), exist_ok=True)
> >   shutil.copytree(repo_path, build_dir)
> >
> > to copy the content of the repo to the build directory. But it fails
> with:
> >
> >   run-u63.service: Failed to set up special execution directory in
> > /var/cache: File exists
> >
> > This makes sense, because of the symbolic link shenanigans to
> > /var/cache/private that systemd uses to keep the filesystem readonly.
> > So, now I'm wondering what would be the best practice to prepopulate
> > this directory:
> >
> > - My current workaround is to shell-out to `systemd-run -p
> > DynamicUser=yes ...` first to do a mkdir -p, then for a cp -R. This
> > solution requires a lot of boilerplate from the Python wrapper and
> > takes more time for no good reason, so I think it's not ideal.
> >
> > - I believe another solution would be to modify /var/cache/private
> > directly, but I'm not sure it's a good practice to do so because I
> > don't know if this path is reliable or just an implementation detail.
> > Plus, it requires a weird special case compared to when I don't run
> > makepkg with systemd-run, as I have to insert something in the middle
> > of the copy destination path.
> >
> > - Maybe there's something else I'm missing that would allow me to do
> > this more cleanly?
> >
> > Thanks,
> >
> > --
> > Antoine Pietri
>
>
>
> --
> Antoine Pietri
> 

Re: [systemd-devel] Best practice for prepopulating the CacheDirectory of dynamic users

[Lennart Poettering]

On Di, 27.02.18 14:37, Antoine Pietri (antoine.piet...@gmail.com) wrote:

> Hi!
> 
> To experiment with systemd dynamic users, I started working on a
> wrapper around a program that builds user packages (Archlinux makepkg)
> and that refuses to be launched as root (for very good reasons). The
> idea is that the wrapper just calls:
> 
> systemd-run --pipe \
>   -p DynamicUser=yes \
>   -p CacheDirectory=mywrapper \
>   -p WorkingDirectory=/var/cache/mywrapper/build/repo \
>   makepkg
> 
> However, to be able to run makepkg, its cache directory has to be
> pre-populated with a clone of the package to build. So, from my
> wrapper, I just did:

Does it have to be a writable copy? if not you could just do '-p
BindReadOnlyPaths=/path/to/my/source:/var/cache/mywrapper'

i.e. there's no need to actually use the CacheDirectory= logic if the
semantics aren't right for you.

That said, maybe we should add a concept of TemplateCacheDirectory= or
so, which would allow prepopulating the dir from some external
source. 

> - My current workaround is to shell-out to `systemd-run -p
> DynamicUser=yes ...` first to do a mkdir -p, then for a cp -R. This
> solution requires a lot of boilerplate from the Python wrapper and
> takes more time for no good reason, so I think it's not ideal.

This sounds OK to me tbh.

> - I believe another solution would be to modify /var/cache/private
> directly, but I'm not sure it's a good practice to do so because I
> don't know if this path is reliable or just an implementation detail.
> Plus, it requires a weird special case compared to when I don't run
> makepkg with systemd-run, as I have to insert something in the middle
> of the copy destination path.

I think it's safe to treat /var/cache/private/ as API. We document it
already at various places, and the semantics aren't overly
complex. Hence this approach is OK too, as long as you create the
relevant dirs if they are missing with the right perms.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Best practice for prepopulating the CacheDirectory of dynamic users

[Antoine Pietri]

On Tue, Feb 27, 2018 at 2:37 PM, Antoine Pietri
 wrote:
> - My current workaround is to shell-out to `systemd-run -p
> DynamicUser=yes ...` first to do a mkdir -p, then for a cp -R. This
> solution requires a lot of boilerplate from the Python wrapper and
> takes more time for no good reason, so I think it's not ideal.
>
> - I believe another solution would be to modify /var/cache/private
> directly, but I'm not sure it's a good practice to do so because I
> don't know if this path is reliable or just an implementation detail.
> Plus, it requires a weird special case compared to when I don't run
> makepkg with systemd-run, as I have to insert something in the middle
> of the copy destination path.
>
> - Maybe there's something else I'm missing that would allow me to do
> this more cleanly?

We came up with a third option, which looks a bit weird at first but
should work:

1) systemd-run -P \
  -p DynamicUser=yes \
  -p CacheDirectory=mywrapper \
  sh -c read

2) do the file operations in the Python code
3) send a "\n" or just kill() the systemd-run process when the setup is done.

I am still not satisfied with any of the three options, so I would
love to know what you think would be best. :-)

-- 
Antoine Pietri

On Tue, Feb 27, 2018 at 2:37 PM, Antoine Pietri
 wrote:
> Hi!
>
> To experiment with systemd dynamic users, I started working on a
> wrapper around a program that builds user packages (Archlinux makepkg)
> and that refuses to be launched as root (for very good reasons). The
> idea is that the wrapper just calls:
>
> systemd-run --pipe \
>   -p DynamicUser=yes \
>   -p CacheDirectory=mywrapper \
>   -p WorkingDirectory=/var/cache/mywrapper/build/repo \
>   makepkg
>
> However, to be able to run makepkg, its cache directory has to be
> pre-populated with a clone of the package to build. So, from my
> wrapper, I just did:
>
>   os.makedirs(os.path.dirname(build_dir), exist_ok=True)
>   shutil.copytree(repo_path, build_dir)
>
> to copy the content of the repo to the build directory. But it fails with:
>
>   run-u63.service: Failed to set up special execution directory in
> /var/cache: File exists
>
> This makes sense, because of the symbolic link shenanigans to
> /var/cache/private that systemd uses to keep the filesystem readonly.
> So, now I'm wondering what would be the best practice to prepopulate
> this directory:
>
> - My current workaround is to shell-out to `systemd-run -p
> DynamicUser=yes ...` first to do a mkdir -p, then for a cp -R. This
> solution requires a lot of boilerplate from the Python wrapper and
> takes more time for no good reason, so I think it's not ideal.
>
> - I believe another solution would be to modify /var/cache/private
> directly, but I'm not sure it's a good practice to do so because I
> don't know if this path is reliable or just an implementation detail.
> Plus, it requires a weird special case compared to when I don't run
> makepkg with systemd-run, as I have to insert something in the middle
> of the copy destination path.
>
> - Maybe there's something else I'm missing that would allow me to do
> this more cleanly?
>
> Thanks,
>
> --
> Antoine Pietri



-- 
Antoine Pietri
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel