Re: [systemd-devel] sd-bus connections & authentication timeout

[Lennart Poettering]

On Mon, 20.03.17 16:37, Stanislav Angelovič (angelovi...@gmail.com) wrote:

> Hi,
> 
> We use sd-bus for DBus IPC in our own applications. Some applications of
> ours cause the dbus daemon to issue "Connection has not authenticated soon
> enough, closing it" message, leading to the
> org.freedesktop.DBus.Error.Timeout error at the peer side when the
> connection is used for an IPC call.
> 
> The situation is the following:
> 1. We have a DBus client application.
> 2. A connection is created using sd_bus_open_system() on application
> startup.
> 3. The connection is kept open and unused (i.e. no other sd-bus functions
> invoked on it).
> 4. After 30 seconds, we get the above-mentioned authentication timeout
> message from the dbus daemon, and any attempt to issue calls against a
> service via this connection fails with immediate timeout error as mentioned
> above. However, if the call is made before the 30 seconds authentication
> timeout, the dbus daemon prints nothing and the call succeeds, just like
> any subsequent call, even if issued after 1 hour from the first one.
> 
> Is that correct behavior? We would like to open dbus connections at
> application startup, and use it for communication at any time later,
> depending on user actions. Is there something we are missing?

So, as you already found out, if you explicitly flush the connection
after creating it you can make sure the timeout is not hit.

But do note that creating a connection and not processing it for
indefinite time is problematic: other peers might send you messages,
and if you don't read them in time and your input buffer hence runs
over for too long, then dbus-daemon will assume you#re stuck and
disconnect you.

Hence: if your app runs for a longer time, then either process the
connection from time to time, or disconnect it whenever you don't need
it anymore.

Or to say this differently: if you rely on keeping bg connection
objects around you can do this safely only as long as you do this for
"short-lived" operations.

> And a side suggestion: If we want to close a connection that has not been
> used for any IPC call at all, after some struggling we have realized that
> sd_bus_unref() is not enough, as there is a lingering DBus.Hello reply
> which keeps the reference count on that connection, leading to lingering
> connections. sd_bus_flush_close_unref() is needed instead. That behavior
> could be mentioned in the documentation, e.g. at
> http://0pointer.net/blog/the-new-sd-bus-api-of-systemd.html in the
> reference client implementation, to save others from pain.

Yeah, this has come up before, and we really need better docs for
this, indeed.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sd-bus connections & authentication timeout

[Lennart Poettering]

On Mon, 20.03.17 19:40, Stanislav Angelovič (angelovi...@gmail.com) wrote:

> Hi Jan,
> 
> thanks for quick response. Instead of sd_bus_process(), we could perhaps
> use sd_bus_flush() after creating the connection, as that one actually
> processes the requests until the connection changes state to 'running'. I
> tried this and it helped.
> 
> Regarding polling, I understand, but I fear this way we could get race
> condition on the connection instance, since we'd have a thread doing the
> polling and processing in a loop on the connection, and some other thread
> (say handling the UI, or an external system event, or incoming DBus call
> from another connection or whatever else) issuing a DBus call on the same
> connection. I think the connection instance is not thread-safe, as once I
> experienced such data race related problems. Maybe there is another way of
> doing that? One that comes to my mind is having just one single thread
> handling the connection and communication through it, but then it would
> have to poll on other fd's as well, to be able to communicate with other
> threads, for example to get DBus call request from the UI thread and
> perform it on the connection. This involves asynchronous programming
> paradigm for quite a simple thing, IMHO. What do you think?

sd-bus is not thread-safe, you'd need to lock around this yourself. It
is written in a threads-aware style however, which permits you doing
it and it keeps very little static state, and the one it keeps is
either managed per-thread implicitly or protected by mutexes.

My recommendation would be to either run a proper event loop in your
threads (for example sd-event, or whatever you like), or only keep the
connection around for as long as you need, and close it right-away.

Async programming (i.e. going for a proper event loop) of course gets
you the best performance.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sd-bus connections & authentication timeout

[Simon McVittie]

On Mon, 20 Mar 2017 at 19:40:04 +0100, Stanislav Angelovič wrote:
> Regarding polling, I understand, but I fear this way we could get race
> condition on the connection instance, since we'd have a thread doing the
> polling and processing in a loop on the connection, and some other thread (say
> handling the UI, or an external system event, or incoming DBus call from
> another connection or whatever else) issuing a DBus call on the same
> connection.

Don't do this. The sd-bus connection is not designed to be safe to
access from multiple threads.

If you need thread-safe D-Bus, then you will need a design where one thread
"dispatches" the D-Bus connection (with sd-event or the GLib
main loop or whatever abstraction around epoll/poll/select you prefer),
and all other threads submit requests to it and get events back from it.
This is essentially what GLib's GDBus does - all the public APIs just
submit requests to a worker thread that is not directly accessible by
user code.

libdbus (the reference implementation of D-Bus) did try to support the
situation you are concerned about, but doing that turns out to be basically
unmaintainable; we've fixed the worst issues, but it has never been very
reliable, and we strongly recommend not doing it. For best results, choose
either always-threaded (GDBus' design) or never-threaded (sd-bus' design,
and also the recommended pattern for libdbus for the last few years)
and stick to it.

> This involves asynchronous programming paradigm for quite a simple
> thing, IMHO.

D-Bus is fundamentally an asynchronous message-passing system, in the
same sorts of ways that X11, Wayland and most network protocols are.
If asynchronous message passing is not acceptable to you, then D-Bus is
probably not going to be the right solution to your requirements.

If you stop reading from your D-Bus socket, the dbus-daemon will
eventually have to disconnect you, because it cannot distinguish
between that and a denial of service attack. The timeout that you
encountered is precisely there to stop malicious processes from carrying
out a denial of service attack on the system dbus-daemon instance
by tying up all of its file descriptors with unauthenticated connections,
and there are similar limits to stop a malicious process from carrying
out a DoS attack on the system dbus-daemon instance by making it
consume unbounded amounts of memory.

In general we put these arbitrary limits in for good reasons, not
just to make your life difficult :-) In this case the relevant
security vulnerability was CVE-2014-3639.

S
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sd-bus connections & authentication timeout

[Jan Alexander Steffens]

On Mon, Mar 20, 2017 at 7:40 PM Stanislav Angelovič 
wrote:

Hi Jan,

thanks for quick response. Instead of sd_bus_process(), we could perhaps
use sd_bus_flush() after creating the connection, as that one actually
processes the requests until the connection changes state to 'running'. I
tried this and it helped.

Regarding polling, I understand, but I fear this way we could get race
condition on the connection instance, since we'd have a thread doing the
polling and processing in a loop on the connection, and some other thread
(say handling the UI, or an external system event, or incoming DBus call
from another connection or whatever else) issuing a DBus call on the same
connection. I think the connection instance is not thread-safe, as once I
experienced such data race related problems. Maybe there is another way of
doing that? One that comes to my mind is having just one single thread
handling the connection and communication through it, but then it would
have to poll on other fd's as well, to be able to communicate with other
threads, for example to get DBus call request from the UI thread and
perform it on the connection. This involves asynchronous programming
paradigm for quite a simple thing, IMHO. What do you think?

Thank you,

Stanislav.


Yes, sd-bus was designed with asynchronous programming in mind. It prefers
the sd-event event loop, but you can also integrate it into another event
loop. The blocking calls all loop internally.

Your programming language and/or framework might have a preferred DBus
implementation already. For example,  if you have a GLib/GTK event loop, I
would recommend using GIO (GDBus) instead of sd-bus. Similarly, Qt has
QDBus.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sd-bus connections & authentication timeout

[Stanislav Angelovič]

Hi Jan,

thanks for quick response. Instead of sd_bus_process(), we could perhaps
use sd_bus_flush() after creating the connection, as that one actually
processes the requests until the connection changes state to 'running'. I
tried this and it helped.

Regarding polling, I understand, but I fear this way we could get race
condition on the connection instance, since we'd have a thread doing the
polling and processing in a loop on the connection, and some other thread
(say handling the UI, or an external system event, or incoming DBus call
from another connection or whatever else) issuing a DBus call on the same
connection. I think the connection instance is not thread-safe, as once I
experienced such data race related problems. Maybe there is another way of
doing that? One that comes to my mind is having just one single thread
handling the connection and communication through it, but then it would
have to poll on other fd's as well, to be able to communicate with other
threads, for example to get DBus call request from the UI thread and
perform it on the connection. This involves asynchronous programming
paradigm for quite a simple thing, IMHO. What do you think?

Thank you,

Stanislav.


On Mon, Mar 20, 2017 at 5:19 PM, Jan Alexander Steffens <
jan.steff...@gmail.com> wrote:

> On Mon, Mar 20, 2017, 17:14 Jan Alexander Steffens 
> wrote:
>
>>
>> You could try calling sd_bus_process(bus, NULL) in a loop while it
>> returns >0 so that the initial hello is handled.
>>
>
> Actually, never mind, this is not reliable. IIRC the initial handshake has
> multiple steps so this can return zero before everything is done. Can't get
> around polling to do this properly.
>
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sd-bus connections & authentication timeout

[Jan Alexander Steffens]

On Mon, Mar 20, 2017, 17:14 Jan Alexander Steffens 
wrote:

>
> You could try calling sd_bus_process(bus, NULL) in a loop while it returns
> >0 so that the initial hello is handled.
>

Actually, never mind, this is not reliable. IIRC the initial handshake has
multiple steps so this can return zero before everything is done. Can't get
around polling to do this properly.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] sd-bus connections & authentication timeout

[Jan Alexander Steffens]

On Mon, Mar 20, 2017, 16:37 Stanislav Angelovič 
wrote:

> Hi,
>
> We use sd-bus for DBus IPC in our own applications. Some applications of
> ours cause the dbus daemon to issue "Connection has not authenticated soon
> enough, closing it" message, leading to the
> org.freedesktop.DBus.Error.Timeout error at the peer side when the
> connection is used for an IPC call.
>
> The situation is the following:
> 1. We have a DBus client application.
> 2. A connection is created using sd_bus_open_system() on application
> startup.
> 3. The connection is kept open and unused (i.e. no other sd-bus functions
> invoked on it).
> 4. After 30 seconds, we get the above-mentioned authentication timeout
> message from the dbus daemon, and any attempt to issue calls against a
> service via this connection fails with immediate timeout error as mentioned
> above. However, if the call is made before the 30 seconds authentication
> timeout, the dbus daemon prints nothing and the call succeeds, just like
> any subsequent call, even if issued after 1 hour from the first one.
>
> Is that correct behavior? We would like to open dbus connections at
> application startup, and use it for communication at any time later,
> depending on user actions. Is there something we are missing?
>
> And a side suggestion: If we want to close a connection that has not been
> used for any IPC call at all, after some struggling we have realized that
> sd_bus_unref() is not enough, as there is a lingering DBus.Hello reply
> which keeps the reference count on that connection, leading to lingering
> connections. sd_bus_flush_close_unref() is needed instead. That behavior
> could be mentioned in the documentation, e.g. at
> http://0pointer.net/blog/the-new-sd-bus-api-of-systemd.html in the
> reference client implementation, to save others from pain.
>
> Thanks a lot,
>
> Stanislav.
>
> ___
> systemd-devel mailing list
> systemd-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/systemd-devel


You could try calling sd_bus_process(bus, NULL) in a loop while it returns
>0 so that the initial hello is handled.

However, that only gets you past the hello. To be a good D-Bus client, you
should poll the connection and process any unexpected incoming messages.
The best approach depends on what your app and your use of sd-bus looks
like.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel