Re: [systemd-devel] kdbus, udev and systemd in the initramfs

[Tom Gundersen]

On Wed, Jul 29, 2015 at 5:40 AM, Tom Gundersen  wrote:
> On Wed, Jul 29, 2015 at 1:21 AM, Michael Biebl  wrote:
>> something I was wondering regarding kdbus and udev.
>> If udev wants to drop the netlink transport and instead rely on kdbus,
>> would this mean, systemd becomes mandatory in the initramfs to setup
>> kdbus before udev is run?
>
> _If_ udev drops netlink entirely, then yes someone would need to set
> up kdbus. How this will all look has not been finally decided yet
> though.

Actually, the problem might not even be netlink (as you may not have
anything using libudev in your intrd), but then the problem will be
the custom udev control protocol which would surely also be swapped
out if netlink is.

>> Will it still be possible in the future to run udev  without systemd
>> in the initramfs?
>
> Maybe, but I, for one, don't see this as a long-term goal. Having the
> same software in the initrd as on the real system seems much more
> reasonable to me.
>
> Cheers,
>
> Tom
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] kdbus, udev and systemd in the initramfs

[Tom Gundersen]

On Wed, Jul 29, 2015 at 1:21 AM, Michael Biebl  wrote:
> something I was wondering regarding kdbus and udev.
> If udev wants to drop the netlink transport and instead rely on kdbus,
> would this mean, systemd becomes mandatory in the initramfs to setup
> kdbus before udev is run?

_If_ udev drops netlink entirely, then yes someone would need to set
up kdbus. How this will all look has not been finally decided yet
though.

> Will it still be possible in the future to run udev  without systemd
> in the initramfs?

Maybe, but I, for one, don't see this as a long-term goal. Having the
same software in the initrd as on the real system seems much more
reasonable to me.

Cheers,

Tom
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] kdbus, udev and systemd in the initramfs

[Michael Biebl]

Hi,

something I was wondering regarding kdbus and udev.
If udev wants to drop the netlink transport and instead rely on kdbus,
would this mean, systemd becomes mandatory in the initramfs to setup
kdbus before udev is run?

Will it still be possible in the future to run udev  without systemd
in the initramfs?


Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Cannot mount anything after recovering and redoing boot mbr

[ccox]

> On Mon, 27.07.15 16:35, c...@endlessnow.com (c...@endlessnow.com) wrote:
>
>> > On Mon, 27.07.15 01:18, Christopher Cox (c...@endlessnow.com) wrote:
>> >
>> >> I suspect that somebody here knows why, but all mounts now fail...
>> well
>> >> all but /.
>> >>
>> >> Has anyone run across this before?  What did I miss?
>> >>
>> >> I accidentally messed up my boot mbr. and I did a rescue cd and
>> chroot
>> >> in
>> >> order to rerun grub2-mkconfig and do a grub2-install.
>> >>
>> >> Now the system boots to a grub menu and tries to boot, root fileystem
>> >> mounts
>> >> but all other mounts fail so goes into "emergency" mode.  From that
>> >> shell I
>> >> cannot seem to mount anything, they all fail saying that whatever I'm
>> >> trying
>> >> to mount is already mounted or it's "in use".  I can't fsck umounted
>> >> filesystems either.  They all say "in use".
>> >
>> > Maybe your changed the order of your partitions or changed their
>> > partition UUID? If so, then /etc/fstab will reference incorrect
>> > partitions now. Make sure bring /etc/fstab into sync with your actual
>> > partitions.
>>
>> A "root" is getting mounted and I figure it's the same but will double
>> check (away from system right now).  Would some kind of root getting
>> mounted at startup and being different from root in /etc/fstab make some
>> sort of difference?
>
> Nope, mounted is mounted. systemd doesn't really care where something
> is mounted from, it only cares whether it is mounted at all. And the
> mount source it will only use if it needs to mount something because
> nobody else has mounted it yet.
>
>> Would that cause manual mounts of old style nonportable dev
>> shortnames (e.g. mount /dev/sda7 /mnt) to fail with the error of
>> "busy" when done at the command line? (from emergency shell).
>
> Well if you use references such as /dev/sda7 then you are of course
> very vulnerable to partition renumbering if you redoo your partition
> table. Use /dev/disks/by-uuid/ and you should be safe regarding that.
>
> Lennart

Thanks Lennart.  It did turn out that the upgrades to this host over the
years...  that the switch from "ata" names to "scsi" names happened (not
sure why it worked before though).  Once I changed the names from
"ata-blah" to "scsi-blah" in /etc/fstab, all came back to normal.  So
thanks for the tip.

But why would having a failed mount due to a bad name in /etc/fstab cause
nothing to be mountable?  Not even foreign objects could be mounted...
(that is things the system hadn't seen before).

It was an adventure to be sure...   Very cryptic.  Not like
troubleshooting from a couple of years ago.





___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] systemd-tmpfiles for the user instance of systemd

[Dan Tihelka]

On Tuesday 28 of July 2015 03:31:08 Lennart Poettering wrote:
> On Sat, 04.07.15 13:23, Tomasz Torcz (to...@pipebreaker.pl) wrote:
> > On Fri, Jul 03, 2015 at 08:31:42PM +0200, Lennart Poettering wrote:
> > > On Wed, 01.07.15 12:35, Daniel Tihelka (dtihe...@gmail.com) wrote:
> > > > Hello,
> > > > does anyone have an experience with the use of systemd-tmpfiles for
> > > > the user instance of systemd.
> > > 
> > > This is currently not nicely supported. And I am not sure it
> > > should. Note that much of what tmpfiles supports is only necessary
> > > for:
> > > 
> > > - aging (automatic time-based clean-up of files). Doesn't really apply
> > > 
> > >   to user sessions, since /tmp and /var/tmp are already cleaned up by
> > >   the system instance of tmpfiles
> >   
> >   /var and /tmp are not only aged files.  I'm using tmpfiles for removing
> > 
> > – files in ~/Downloads/* older than 1 year
> > – emails in ~/Mail/.spam/cur/* older than 1 month
> > 
> >   Out of neccessity I have cleanup configured in system instance for my
> > 
> > specific user only.
> 
> My recommendation would be to clean ~/Downloads up from the root
> tmpfiles instances.

Well, it does not work for encfs (FUSE-based) encrypted directories (root does 
not have to have an access to it). An option is to link Downloads (or any 
other) to an unencrypted dir, but it somehow violates the purpose of 
encryption ... And of course, one must have root access to configure it.

> 
> The simple issue is that aging for dirs cannot work, since iterating
> through them will bump the atime, which defeats the aging. You hence
> break the aging by aging... THis can only work if you have root privs,
> since then we can turn off the atime bumping...

Yes, it is documented. But the aging still works for files, doesn't it? Could 
it be solved by (optional) deletition of those dirs which stay empty after 
files cleaning?

Dan

> 
> Lennart

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] udev and dm inotify problems

[Oleksii Shevchuk]

>
> Most likely you built your LVM/dm userspace without proper udev
> support, or left support in there that creates device nodes on its
> own.
>
> Nowadays with devtmpfs device nodes are created exclusively by the
> kernel and userspace should never create a single device node. If your
> LVM/DM tools still do, then they have been built with incorrect
> build-time settings.
>
> Note that LVM/DM issues are probably better discussed at the LVM/DM
> mls than here...
>
> Lennart

The problem was that I had outdated libudev library in /lib. God knows
how that was happend. 

Thanks to all for support.

//wbr
// Oleksii Shevchuk
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Completion of error handling

[SF Markus Elfring]

> I have no understanding of Coccinelle. I do not understand what you
> are saying.

The application of a few scripts in the semantic patch language
can occasionally help to improve some software, can't it?
Now I'll try again to present more detailed source code analysis results
according to specific software metrics.

I assumed that each source file (*.c) provides only functions which
unique names.

╔╤╗
║│║
║│   incidents║
║ overview   │   │║
║│ total │ ≠ 0║
║│   │ ≠ NULL ║
╟┼───┼╢
║│   │║
║ non-empty  │   │║
║ return │ 20633 │ 15477  ║
║ statements │   │║
║│   │║
╟┼───┼╢
║│   │║
║ non-void   │  5990 │  5183  ║
║ functions  │   │║
║│   │║
╚╧═══╧╝


Does such a table indicate that there are some function implementations
left over which will provide only the return value "zero" (or "NULL")?

A few specific examples:
* client_timeout_resend_expire
* config_parse_memory_limit
* transfer_on_log
* udev_rules_unref
* writer_free


Is this information worth for further considerations?

Regards,
Markus
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Suggestion for a lowlevel fsnotify change daemon.

[Stef Bon]

2015-07-28 19:20 GMT+02:00 Simon McVittie :

> On 28/07/15 17:28, Mantas Mikulėnas wrote:
> > At first look, this seems very similar to FAM (which even supported
> > NFSv3, using custom notifications over SunRPC).
> >
> > Later I remember GNOME replaced it with Gamin and finally with
> > local-only inotify inside glib/gvfs.


No, what I propose is a fs notify change daemon which is able to forward a
watch to individual filesystems,
and can listen to events coming from these filesystems from the backend.

Gamin can't do that. It's not maintained anymore and difficult to
understand.


>

What GLib actually uses is an abstraction with multiple backends,
> including inotify (the default on Linux) and FAM; so in principle it
> could have a backend for some new thing, or even use inotify for "normal
> local filesystems" and the new backend for other mounts.
>

I'm afraid I do not understand you here.

>> - the FUSE kernel module in VFS has to trigger fsnotify call when

> >> events are pushed to the VFS by the userspace daemon.
>
> If you're adding a monitoring/notification call to FUSE, would it be out
> of the question for the user-space API to it to be exactly "use
> inotify"? (Or fanotify, or whatever is believed to be the right
> user-space API for file monitoring these days.)
>

I've written that I've tried this, and also that I stopped this idea, cause
it's not the way to go.
But for your understanding, the underlying subsystem is fsnotify, and it
does handle inotify and/or fanotify, whatever is used.
So userspace should use the fsnotify api.


>
> >> It should also be able to "forward" a watch to a filesystem like
> >> FUSE and cifs and nfs, so that they "know" a watch has been set.
> >> They can act then on it, by forwarding the watch to the backend. SMB
> >> does upport this, NFS4 also, and you can make FUSE also support
> >> it(depending the protocol).
>
> If the in-kernel implementation of NFS or CIFS isn't enhanced to support
> monitoring, this can't work.
>

At this moment this can't work, but for CIFS in the past it worked (with
dnotiify). See:

See line 6459 in cifssmb.c in /fs/cifs in the kernel. It is disabled for
now.
But support in SMB (SMB servers do support it).

I know that the NFS4 protocol also supports it.

You must understand that the protocols do support it, but it does not work
with Linux, cause
nobody has tried it yet, and fsnotify does not let the individual
filesystems know that there is a watch set.

There are network filesystems/protocols like webdav which do not support
the setting of a watch and getting
fsevents from the backend. Webdav is build upon HTTP, and the current
version does not support the pushing of events
from the server to the client. We have to wait for version 2 of the HTTP
protocol, as the main developer of the webdav proto
told me Joe Orton.


> If the in-kernel implementation of NFS or CIFS *is* enhanced to support
> monitoring, is there any reason for the kernel not to present the
> resulting information to user-space via inotify? In other words, is
> there a reason why a user-space service is necessary?
>

Like I mentioned in the first post and here again, the filesystems like
FUSE, NFS and CIFS (and other)
are not "contacted" by fsnotify about a watch. This is by design.

So if the kernel does not do this (and beleive me there are good reasons
for), you have to do this in userspace.

Stef Bon



>
> (I realise that one possible reason for a user-space service is so that
> it can aggregate all the periodic polling, on filesystems that don't
> have anything better you can do - that's why FAM had a daemon, if I
> remember correctly.)
>

Yes that's one reason.  But there are more reasons to do this in userspace,
see above.

Stef
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Suggestion for a lowlevel fsnotify change daemon.

[Simon McVittie]

On 28/07/15 17:28, Mantas Mikulėnas wrote:
> At first look, this seems very similar to FAM (which even supported
> NFSv3, using custom notifications over SunRPC).
> 
> Later I remember GNOME replaced it with Gamin and finally with
> local-only inotify inside glib/gvfs.

What GLib actually uses is an abstraction with multiple backends,
including inotify (the default on Linux) and FAM; so in principle it
could have a backend for some new thing, or even use inotify for "normal
local filesystems" and the new backend for other mounts.

However...

> On Jul 28, 2015 18:46, "Stef Bon"  wrote:
>> - I had to add some calls to the fuse library to "push" changes to
>> the VFS where there is no direct related call from the VFS. (files
>> are added and/or files are changed)
>> - the FUSE kernel module in VFS has to trigger fsnotify call when
>> events are pushed to the VFS by the userspace daemon.

If you're adding a monitoring/notification call to FUSE, would it be out
of the question for the user-space API to it to be exactly "use
inotify"? (Or fanotify, or whatever is believed to be the right
user-space API for file monitoring these days.)

>> It should also be able to "forward" a watch to a filesystem like
>> FUSE and cifs and nfs, so that they "know" a watch has been set.
>> They can act then on it, by forwarding the watch to the backend. SMB
>> does upport this, NFS4 also, and you can make FUSE also support
>> it(depending the protocol).

If the in-kernel implementation of NFS or CIFS isn't enhanced to support
monitoring, this can't work.

If the in-kernel implementation of NFS or CIFS *is* enhanced to support
monitoring, is there any reason for the kernel not to present the
resulting information to user-space via inotify? In other words, is
there a reason why a user-space service is necessary?

(I realise that one possible reason for a user-space service is so that
it can aggregate all the periodic polling, on filesystems that don't
have anything better you can do - that's why FAM had a daemon, if I
remember correctly.)

-- 
Simon McVittie
Collabora Ltd. 

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] persistent network device names

[Keller, Jacob E]

On Tue, 2015-07-28 at 06:22 +0300, Andrei Borzenkov wrote:
> В Mon, 27 Jul 2015 19:05:37 +
> "Keller, Jacob E"  пишет:
> 
> > On Mon, 2015-07-27 at 21:53 +0300, Andrei Borzenkov wrote:
> > > В Mon, 27 Jul 2015 18:29:57 +
> > > "Keller, Jacob E"  пишет:
> > > 
> > > > On Mon, 2015-07-27 at 21:12 +0300, Andrei Borzenkov wrote:
> > > > > В Mon, 27 Jul 2015 17:52:59 +
> > > > > "Keller, Jacob E"  пишет:
> > > > > 
> > > > > > Hello,
> > > > > > 
> > > > > > I am working with a network device that can create virtual 
> > > > > > function
> > > > > > devices. When I create a large (>8) vfs for this device I 
> > > > > > get 
> > > > > > some
> > > > > > weird device names,
> > > > > > 
> > > > > > If I create 64 vfs, I see something like:
> > > > > > 
> > > > > > ens8
> > > > > > ens8f[1-7
> > > > > > en8sfs1-7]]
> > > 
> > > Are you sure? I do not see where such names would be generated.
> > > 
> > 
> > Yes. These are generated via the hotplug section of udev-builtin
> > -net_id.c rules
> > 
> > on line: 230 or so.
> > 
> 
> I do not see where it generates double 's'.
> 

Sorry that is a typo it looks like

ens8f[1-7]

and then

enp8s1f[1-7]

Basically, sometimes it uses the hotplug slot format and sometimes it
uses the full format, since the hotplug slot is

:08:00.0

and the SR-IOV devices show up as part slot part func.

every slot in my platform has "00" for the slot part of the PCI path,
and some number for the bus.

It seems wrong that devices plugged into the hotplug slot don't always
get found as "in" that slot, but maybe this is because slots are able
to actually show up as domain:bus:slot with a non-zero value for slot?
none of my platforms have non-zero slot for their hotplug slots...

But the current naming scheme for hotplug slots assumes they can be non
-zero.

> 
> > > > It looks like my issue comes from the "hotplug_slot" stuff, 
> > > > which
> > > > overwrites the "enp8s0f0" into "ens8"... that is what causes 
> > > > this 
> > > > dual
> > > > -name scheme confusion.
> > > > 
> > > 
> > > You can disable hotplug slot-based naming by setting suitable
> > > NamePolicy for network links (drop "slot"). 
> > 
> > How can I define my own version of the net_id which does what I 
> > would
> > like and run that in my udev rule?
> 
> 
> man systemd.link
> 
> specifically description of NamePolicy
 I see. Thank you.

Regards,
Jake
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Suggestion for a lowlevel fsnotify change daemon.

[Mantas Mikulėnas]

At first look, this seems very similar to FAM (which even supported NFSv3,
using custom notifications over SunRPC).

Later I remember GNOME replaced it with Gamin and finally with local-only
inotify inside glib/gvfs.

It might be useful to revive it, both inotify and fanotify have problems.
But I guess security would be a problem (how to determine which users may
receive which events).

-- 
Mantas Mikulėnas
On Jul 28, 2015 18:46, "Stef Bon"  wrote:

> Hi all,
>
> for some time I have been looking at the issue why fsnotify does not work
> with network filesystems and FUSE (with a shared backend).
>
> I've found out that changes initiated on the localhost, on the filesystem
> are supported by the fs change subsystems on Linux, and events initiated at
> the backend (from another host with network fs)  are not detected. This is
> because the filesystem are not "aware" a watch has been set on an inode,
> and thus cannot act on it.
> (if they act if they are aware is another question).
>
> I've tried to tackle this in the kernel. I've made this working with a
> FUSE:
> - when a watch is set on a FUSE fs, a message is forwarded to the
> userspace daemon containing the inode and the mask. I had to add a opcode
> FUSE_FSNOTIFY.
> - the fuse fs has to react in it, by setting a watch on the backend. I
> wrote a simple overlay fs, and setting a watch on the backend is simple
> - I had to add some calls to the fuse library to "push" changes to the VFS
> where there is no direct related call from the VFS. (files are added and/or
> files are changed)
> - the FUSE kernel module in VFS has to trigger fsnotify call when events
> are pushed to the VFS by the userspace daemon.
>
> This worked but is I think not the best way to deal with it.
>
> My suggestion it to write a fs notify change service which does all the
> watching for clients, like there are already services for desktops right
> now.
>
> This service should also work with a console app like mc, but also with
> desktop environments like Gnome and KDE.
>
> It should also be able to "forward" a watch to a filesystem like FUSE and
> cifs and nfs, so that they "know" a watch has been set.
> They can act then on it, by forwarding the watch to the backend. SMB does
> upport this, NFS4 also, and you can make FUSE also support it(depending the
> protocol).
> When the fs receives an event, it can send it back to the fs notify change
> service, which informs the client(s). This way the filesystem also stays up
> to date.
>
> To forward a watch and to read to incoming fsevents, a
> socket/filedescriptor is required. A FUSE fs can easily connect to it at
> startup, the in kernel filesystems need some extra. Via mountoptions parse
> the fd to the kernel?
>
> Is this something what can be added to systemd? Please let me know what
> you think of it.
>
> Stef
>
>
> ___
> systemd-devel mailing list
> systemd-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>
>
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Suggestion for a lowlevel fsnotify change daemon.

[Stef Bon]

Hi all,

for some time I have been looking at the issue why fsnotify does not work
with network filesystems and FUSE (with a shared backend).

I've found out that changes initiated on the localhost, on the filesystem
are supported by the fs change subsystems on Linux, and events initiated at
the backend (from another host with network fs)  are not detected. This is
because the filesystem are not "aware" a watch has been set on an inode,
and thus cannot act on it.
(if they act if they are aware is another question).

I've tried to tackle this in the kernel. I've made this working with a FUSE:
- when a watch is set on a FUSE fs, a message is forwarded to the userspace
daemon containing the inode and the mask. I had to add a opcode
FUSE_FSNOTIFY.
- the fuse fs has to react in it, by setting a watch on the backend. I
wrote a simple overlay fs, and setting a watch on the backend is simple
- I had to add some calls to the fuse library to "push" changes to the VFS
where there is no direct related call from the VFS. (files are added and/or
files are changed)
- the FUSE kernel module in VFS has to trigger fsnotify call when events
are pushed to the VFS by the userspace daemon.

This worked but is I think not the best way to deal with it.

My suggestion it to write a fs notify change service which does all the
watching for clients, like there are already services for desktops right
now.

This service should also work with a console app like mc, but also with
desktop environments like Gnome and KDE.

It should also be able to "forward" a watch to a filesystem like FUSE and
cifs and nfs, so that they "know" a watch has been set.
They can act then on it, by forwarding the watch to the backend. SMB does
upport this, NFS4 also, and you can make FUSE also support it(depending the
protocol).
When the fs receives an event, it can send it back to the fs notify change
service, which informs the client(s). This way the filesystem also stays up
to date.

To forward a watch and to read to incoming fsevents, a
socket/filedescriptor is required. A FUSE fs can easily connect to it at
startup, the in kernel filesystems need some extra. Via mountoptions parse
the fd to the kernel?

Is this something what can be added to systemd? Please let me know what you
think of it.

Stef
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel