Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wed, Nov 26, 2014 at 01:25:18AM +0100, Lennart Poettering wrote: On Tue, 25.11.14 12:01, Thiago Macieira (thi...@kde.org) wrote: [...] === KDBUS_ATTACH_NAMES === Documentation for metadata says that userspace must cope with some metadata not being delivered. Can we at least require that KDBUS_ATTACH_NAMES be delivered if requested? If the cookie in the match rule isn't provided in the message reception, having the source's names would help solve the problem of the signal delivery. The timestamp should also be mandatory. Yes, they are mandatory. process credentials might be suppressed hwover, for example if they cannot be translated due to namespaces. Thanks. Could you clarify in the docs? Daniel, David? Could you add a note about this? Ok pushed a note about namespace issues, thanks! -- Djalal Harouni http://opendz.org ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Tue, 25.11.14 18:32, Thiago Macieira (thi...@kde.org) wrote: On Wednesday 26 November 2014 01:25:18 Lennart Poettering wrote: Thinking of non-system buses here. If the variable is empty, I agree that it should have an equivalent of an autostart mechanism, but I disagree on the solution and I also disagree that distros should leave it empty. Oh, no. No autostart please. No such concept exists in kdbus, and systemd/sd-bus will not support that either. In fact I refuse to support that even on dbus1 in sd-bus. Autostart is a kludge for systems where dbus is just an add-on, but that's completely out-of-focus for kdbus, systemd and sd-bus. I didn't actually mean automatically starting the bus, sorry for the confusion. I meant automatic discovery only. Currently on dbus1, autostart: as a transport protocol means both auto discovery and automatic starting if the bus isn't running. Please also note that the autostart solution has a valid use-case which is when a D-Bus application is launched in an environment where no bus had been started before. I understand this is out-of-scope for kdbus, since after all a regular user won't be able to create a kdbus bus if one wasn't provided by a privileged process before. In an environment where a kdbus bus wasn't provided, the only alternative is to fallback to dbus1. Hmm? Creating busses requires no privileges. Users create their own busses without requiring any kind of privilege for that. You are free to implement autostart behaviour for dbus1 in your own library, but this concept is not defined for kdbus. No, we don't support weird setups where kdbusfs mounted elsewhere. This is a bew API we introduce here, and we can very much make decisions where stuff is to be mounted. You may not support it in systemd, but from reading the kernel API that could happen with another implementation. Well, you can also mount /sys to /foobar and /proc to /waldo. No app supports that though and the place where kdbusfs is mounted is simply part of the kdbus API definition, and if you mount it anywhere else you get to keep the pieces. Note that we the kernel driver explicitly creates a dir /sys/fs/kdbus in order to have a place where to mount it. The location is really not configurable. We initially tried to support that, but it's awfully racy, since the driver calls and calls to other services wouldn't be executed in strict order anymore... We removed this again after figuring out and decided that emulation can only happen client side, synchronous to the message stream if we want to guarantee correct ordering. I'm not asking for AddMatch and connection control mechanisms. The one I really want is StartServiceByName, since it can't be emulated. Moreover, starting services is systemd's raison-d'être, so I feel it should be no problem for you to provide such a service. My recommendation for StartServiceByName() is to convert it into a Ping() method call to the respective service and wait for it to return. This is at least what the compat proxy does. It would be nice if UpdateActivationEnvironment worked. This functionality was added for people who need to update variables like XDG_DATA_DIRS after starting the bus. If this one isn't present, we can report not implemented and be fine with it. We'll just have to tell people to configure their systemd user session environments properly. The same goes for ReloadConfig, but I'd prefer to know whether that failed (no config reloading is possible) or whether it happens automatically whether the call was made or not. ReloadConfig is important when there are new activatable services on a user's bus, such as newly-installed applications. If you really want to support this, then call into systemd for it, and make it graceful to support non-systemd systems. Return a notsupported error if the systemd service is not reachable. The client side emulation can choose to either forward ReloadConifg and UpdateActivationEnvironment to the respect systemd calls, or just return som not supported error. Can't do that. What if it's a kdbus system that is not systemd? Well, again, return not supported then. I mean, currently there is no kdbus userspace implementation beyond kdbus, we cannot really discuss something that doesn't exist... I assume you meant beyond systemd there. Oops. Sorry, yes. Note that on dbus1 systemd systems we actually never provided UpdateActivationEnvironment correctly (since services got forked off PID 1, instead of dbus-daemon but the call would alter dbus-daemon's env block, not systemd's one), but nobody ever noticed. I really think you should just return some not supported error or make it a NOP if you don't want to pass this on to systemd. I *want* to pass this to systemd, somehow. So the first question is whether we can expect there to be a connection by systemd in
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Tue, 25.11.14 18:46, Thiago Macieira (thi...@kde.org) wrote: On Wednesday 26 November 2014 00:46:50 David Herrmann wrote: We had systemd-bus-driverd, which implemented org.freedesktop.DBus as normal service. However, this didn't work out as many dbus clients rely on this services to not be re-ordered in regard to external requests. In particular, if gdbus runs AddMatch(), it assumes the match takes effect immediately. If it sends a method call to another service after installing the match, and this triggers a signal, gdbus assumes the AddMatch() call to have succeeded (without waiting for the reply). However, if org.freedesktop.DBus is not implemented by the bus, but by an external service, you cannot guarantee that messages targetted at different receivers don't get re-ordered, and there're no guarantees which process gets scheduled first. This is a real bug and we couldn't figure out a way to fix it. Current DBus applications depend on org.freedesktop.DBus to be handled by the bus entity _in-order_. Therefore, we dropped systemd-bus-driverd and all the kdbus ioctls that we added to support this. Aside from the connection-control mechanisms (AddMatch, RemoveMatch), did you see any problems? It was primarily about that, but it is easy to construct races with this for most of the other driver calls as well. The race is very real btw. Because a frequent pattern is to establish a match, then invoke some method on some random service. However, with doing the driver in a separate service it might happen that the method call is processed before the driver finds the time to process the add match request. And then, any signals sent out by the service processing the method call might not be subject to the added match... User code usually shouldn't be doing AddMatch and RemoveMatch (the only case I can think of is dconf, which does its own parsing of the message payload). That's the domain of the binding, which will know that it connected to kdbus and won't be making those calls. Well, I am sure that in many cases apps want to install their own matches, beyond the object model of the used framework. For example, a graphical UI for systemd might want to subscribe to all of systemd's signals without actually matching on any specific object. Most framework object models don#t cover that. Or think of a tool like d-feet that wants to subscribe even more liberally, per service... Custom endpoints do _not_ create new buses. Really. You could create a custom bus and use it for just 2 connections, but then you could also just use socketpair(2). Note that there was some discussion on anonymous buses, which would allow to create such buses on the fly. But again, this will not be part of the initial kdbus draft. If anyone cares, submit it as patches once kdbus is upstream. As far as I can tell, an anonymous bus in the sense of a bus with a randomised name is doable by having the proper interface in the systemd1 system bus service. An application would request the bus and receive as a reply the address of the bus, the file descriptor for the control, and a file descriptor to a privileged connection so it can control the policies. No need to involve systemd. Creating additional busses requires no priviliges. You can just invoke the ioctls directly. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On 25/11/14 23:46, David Herrmann wrote: In particular, if gdbus runs AddMatch(), it assumes the match takes effect immediately. If it sends a method call to another service after installing the match, and this triggers a signal, gdbus assumes the AddMatch() call to have succeeded (without waiting for the reply). Argument in favour of this being sane: if AddMatch() failed, what would it do about it? The only error that can happen without programmer error is running out of match rule slots, which can't be recovered from in traditional D-Bus (but now that https://code.google.com/p/d-bus/issues/detail?id=10 has been fixed, kdbus *can* recover). I strongly recommend to either drop support for org.freedesktop.DBus on any kdbus-aware DBus APIs, or fake it in the library. sd-bus doesn't support it, and IIRC Ryan didn't want to fake it in gdbus either. For sd-bus, a new and non-API-stable library, that's reasonable. For existing libraries, this sounds like an argument in favour of having a separate DBUS_BUS_SESSION_KDBUS (or whatever you want to call it, but basically it means allowed to be kdbus), to be able to opt-in to potentially breaking changes: lack of ACLs (definitely a breaking change on the system bus but arguably not a breaking change on the session bus), and lack of support for direct o.fd.DBus calls (a breaking change on both buses). There are several o.fd.DBus calls for which libdbus doesn't have high-level API at all, so libdbus users have no option but to use direct calls. GDBus might be more complete in that respect? S ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wednesday 26 November 2014 12:27:16 Lennart Poettering wrote: Please also note that the autostart solution has a valid use-case which is when a D-Bus application is launched in an environment where no bus had been started before. I understand this is out-of-scope for kdbus, since after all a regular user won't be able to create a kdbus bus if one wasn't provided by a privileged process before. In an environment where a kdbus bus wasn't provided, the only alternative is to fallback to dbus1. Hmm? Creating busses requires no privileges. Users create their own busses without requiring any kind of privilege for that. How is that secure? That would imply that any process, with any UID, can issue KDBUS_CMD_BUS_MAKE for a bus name that another UID would expect to use. I must be misunderstanding something. My recommendation for StartServiceByName() is to convert it into a Ping() method call to the respective service and wait for it to return. This is at least what the compat proxy does. Hmm... I had forgotten about Ping(). That sounds like a solution. It would be nice if UpdateActivationEnvironment worked. This functionality was added for people who need to update variables like XDG_DATA_DIRS after starting the bus. If this one isn't present, we can report not implemented and be fine with it. We'll just have to tell people to configure their systemd user session environments properly. The same goes for ReloadConfig, but I'd prefer to know whether that failed (no config reloading is possible) or whether it happens automatically whether the call was made or not. ReloadConfig is important when there are new activatable services on a user's bus, such as newly-installed applications. If you really want to support this, then call into systemd for it, and make it graceful to support non-systemd systems. Return a notsupported error if the systemd service is not reachable. Well, that's exactly what I want, but see below. Note that on dbus1 systemd systems we actually never provided UpdateActivationEnvironment correctly (since services got forked off PID 1, instead of dbus-daemon but the call would alter dbus-daemon's env block, not systemd's one), but nobody ever noticed. I really think you should just return some not supported error or make it a NOP if you don't want to pass this on to systemd. I *want* to pass this to systemd, somehow. So the first question is whether we can expect there to be a connection by systemd in the buses it creates. On the system bus, there's org.freedesktop.systemd1 so I expect that to continue. Can we expect a similar service on systemd-user buses? Yes. systemd creates the bus and then acquires org.freedesktop.systemd1 on it, and does this for both the system and the user bus the same way. My argument was for a more generic name, such as something in the org.freedesktop.DBus hierarchy. That way, if someone else wants to provide kdbus-powered buses, they'd implement that interface instead of trying to emulate systemd and playing catch-up and possibly bug compatibility. Further, if it does acquire the org.freedesktop.DBus name itself, libraries won't have to emulate most of the compat behaviour by intercepting outgoing messages. Intercepting means EVERY message needs to be inspected at one point or another, instead of letting it fallback when the call actually happens. In other words, providing a fallback service optimises for the right use-case. Since systemd will connect to the bus anyway and will acquire a name, it can acquire another. When creating the bus the creator can pass policy to the kernel so that there is no time window where the bus is accessible and open to manipulation from untrusted clients. How can you update the policies then? There's an ioctl for that too, that the creator can then call on the fd it has open. I guess I got confused by the wording of what you can do with what file descriptors. I was under the impression that you'd need three file descriptors for a bus or an endpoint to make any useful use 1) one fd that opened the control file (to create a bus) or to the bus (to create the endpoint) 2) one fd that you'll want to open the bus/endpoint on so you can later send updates to the policies 3) one fd to actually do talking on My confusion was that the doc says that after CMD_xxx_MAKE the file descriptor can't be used again. 5.3 says ... is file descriptor will manage the newly created endpoint resource. It cannot be used to manage further resources. I guess I read further resources as including changing policies on this bus/endpoint. Thanks for the clarification. I recommend adding a section on what steps to do to create a bus (including setting up policies, activators), one on creating an endpoint, and then one on how a controlling process should operate to manage the bus/endpoint properly. What's the usecase? I mean
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wed, 26.11.14 10:04, Thiago Macieira (thi...@kde.org) wrote: On Wednesday 26 November 2014 12:27:16 Lennart Poettering wrote: Please also note that the autostart solution has a valid use-case which is when a D-Bus application is launched in an environment where no bus had been started before. I understand this is out-of-scope for kdbus, since after all a regular user won't be able to create a kdbus bus if one wasn't provided by a privileged process before. In an environment where a kdbus bus wasn't provided, the only alternative is to fallback to dbus1. Hmm? Creating busses requires no privileges. Users create their own busses without requiring any kind of privilege for that. How is that secure? That would imply that any process, with any UID, can issue KDBUS_CMD_BUS_MAKE for a bus name that another UID would expect to use. I must be misunderstanding something. The kernel enforces that each bus name is prefixed with $UID-. This is why the system bus is /sys/fs/kdbus/0-system rather than just /sys/fs/kdbus/system. This makes sure that users cannot play games with other users. Note that on dbus1 systemd systems we actually never provided UpdateActivationEnvironment correctly (since services got forked off PID 1, instead of dbus-daemon but the call would alter dbus-daemon's env block, not systemd's one), but nobody ever noticed. I really think you should just return some not supported error or make it a NOP if you don't want to pass this on to systemd. I *want* to pass this to systemd, somehow. So the first question is whether we can expect there to be a connection by systemd in the buses it creates. On the system bus, there's org.freedesktop.systemd1 so I expect that to continue. Can we expect a similar service on systemd-user buses? Yes. systemd creates the bus and then acquires org.freedesktop.systemd1 on it, and does this for both the system and the user bus the same way. My argument was for a more generic name, such as something in the org.freedesktop.DBus hierarchy. That way, if someone else wants to provide kdbus-powered buses, they'd implement that interface instead of trying to emulate systemd and playing catch-up and possibly bug compatibility. I am really not keen in defining a new abstract bus interface for two functions of which at least one is very questionnable anyway, and where it's not clear that there will ever be a second implementation of... I mean, the two operations are not even really abstractable since both the update-env stuff and the reload stuff have different semantics on systemd: they are much more comprehensive and apply to all services at once instead of just the bus services. I mean, it sounds OK to me to pass this to systemd for compat reasons in a quirky way. But it if we define an abstract, well-defined, standardized interface for this with inherently unclean semantics this is really nothing I want to do. Further, if it does acquire the org.freedesktop.DBus name itself, libraries won't have to emulate most of the compat behaviour by intercepting outgoing messages. Intercepting means EVERY message needs to be inspected at one point or another, instead of letting it fallback when the call actually happens. In other words, providing a fallback service optimises for the right use-case. This kind of half/half solution sounds messy to me. Either emulate it or don't. But having some of the calls emulated on the client-side and some elsewhere sounds messy. There's an ioctl for that too, that the creator can then call on the fd it has open. I guess I got confused by the wording of what you can do with what file descriptors. I was under the impression that you'd need three file descriptors for a bus or an endpoint to make any useful use 1) one fd that opened the control file (to create a bus) or to the bus (to create the endpoint) 2) one fd that you'll want to open the bus/endpoint on so you can later send updates to the policies 3) one fd to actually do talking on My confusion was that the doc says that after CMD_xxx_MAKE the file descriptor can't be used again. 5.3 says ... is file descriptor will manage the newly created endpoint resource. It cannot be used to manage further resources. I guess I read further resources as including changing policies on this bus/endpoint. Thanks for the clarification. I recommend adding a section on what steps to do to create a bus (including setting up policies, activators), one on creating an endpoint, and then one on how a controlling process should operate to manage the bus/endpoint properly. To create a bus you open the control file, issue an ioctl (MAKE_BUS) on the fd, and the fd is then *converted* into a reference to the bus. You can use it then only for updating policy and for eventually closing it to make the bus go away. To create a bus connection you open the bus file,
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wed, 26.11.14 10:10, Thiago Macieira (thi...@kde.org) wrote: On Wednesday 26 November 2014 12:35:33 Lennart Poettering wrote: Aside from the connection-control mechanisms (AddMatch, RemoveMatch), did you see any problems? It was primarily about that, but it is easy to construct races with this for most of the other driver calls as well. [snip] Can you explain, if you remember, how the race could happen for the other driver calls? Is this one: GetConnectionUnixProcessID could return stale information because the process has exited by the time that the reply arrived. If so, this race already existed in dbus1. Here's are two examples: First: AcquireName(): you issue issue an async request for acquiring foo.bar, and immediately send a message to it. On dbus1 this will work: the bus will first process the AcquireName(), and then deliver the message back to you given that you now have the name. In a kdbus design with an extern driver process the AcquireName() call would be passed to the driver, and the next message most likely be processed before the driver processes it, thus most likely resulting in a delivery failure of the second message. Now, that's the reason why acquirename/releasename need to be implemented client side. With that knowledge we can punch holes in the other calls too. For example, the GetConnectionUnixProcessID() call you brought up: let's now issue a GetConnectionUnixProcessID() async first, immediately followed by an AcquireName() for the same name. On dbus this will tell you the old owner, and then acquire the name for you. On kdbus with an external driver this would be different though. While the GetConnecitonUnixProcessID() is queued and waiting to be processed the AcquireName() might already have been executed (because it needs to be an ioctl as we have shown above), hence you would get your own PID returned, not the old one! hence, this calls really should be sync local ioctl()s, not half-half... Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wednesday 26 November 2014 19:30:16 Lennart Poettering wrote: I must be misunderstanding something. The kernel enforces that each bus name is prefixed with $UID-. This is why the system bus is /sys/fs/kdbus/0-system rather than just /sys/fs/kdbus/system. This makes sure that users cannot play games with other users. Thanks for the clarification. That tidbit is nowhere in the kdbus.txt docs. It should probably be mentioned in the section about creating buses. Also, shouldn't CAP_IPC_OWNER-privileged processes be able to create a bus with an arbitrary name? My argument was for a more generic name, such as something in the org.freedesktop.DBus hierarchy. That way, if someone else wants to provide kdbus-powered buses, they'd implement that interface instead of trying to emulate systemd and playing catch-up and possibly bug compatibility. I am really not keen in defining a new abstract bus interface for two functions of which at least one is very questionnable anyway, and where it's not clear that there will ever be a second implementation of... I understand that reluctance. I mean, the two operations are not even really abstractable since both the update-env stuff and the reload stuff have different semantics on systemd: they are much more comprehensive and apply to all services at once instead of just the bus services. I mean, it sounds OK to me to pass this to systemd for compat reasons in a quirky way. But it if we define an abstract, well-defined, standardized interface for this with inherently unclean semantics this is really nothing I want to do. Well, whatever it is, we need a sane API. I'm willing to standardise on the systemd behaviour if the current one is quirky anyway. If we do that, we'll need a new interface name anyway. Further, if it does acquire the org.freedesktop.DBus name itself, libraries won't have to emulate most of the compat behaviour by intercepting outgoing messages. Intercepting means EVERY message needs to be inspected at one point or another, instead of letting it fallback when the call actually happens. In other words, providing a fallback service optimises for the right use-case. This kind of half/half solution sounds messy to me. Either emulate it or don't. But having some of the calls emulated on the client-side and some elsewhere sounds messy. I don't want to emulate any in the library. Any calls that are made are sent to the service. AddMatch and RemoveMatch would reply with error, but most of the others should just work. But given the other email, I guess I am only looking at my own problem. I didn't know that other applications, using other bindings, had legitimate uses for calling AddMatch directly. Thanks for the clarification. I recommend adding a section on what steps to do to create a bus (including setting up policies, activators), one on creating an endpoint, and then one on how a controlling process should operate to manage the bus/endpoint properly. To create a bus you open the control file, issue an ioctl (MAKE_BUS) on the fd, and the fd is then *converted* into a reference to the bus. You can use it then only for updating policy and for eventually closing it to make the bus go away. To create a bus connection you open the bus file, issue an ioctl (HELLO) on the fd, and the fd is then *converted* into a connection fd. You can then use that for send/recv messages, or acquiring/releasing names. To create a bus EP you open the bus file, issue an ioctl (MAKE_EP) on the fd, and the fd is then *converted* into a reference to the EP. You can use it then only to update the EP's policy, and for eventually closing it to make the EP go away. The systemd PID 1 instance has only two fds open hence: one bus reference, and one bus connection. A sandboxing app executor has one, maybe two fds open: the EP reference, and maybe a bus connection in case it wants to offer some services... Understood. My recommendation for clarification in kdbus.txt remains. Well, but in those cases you can just use the user bus or system bus, there's no need to create a new bus just for the purpose of P2P... That implies changes to the user application. I want to provide a replacement API for DBusServer. So, if this is about compat with old apps then continue to use AF_UNIX for that... No need to push them to something new... Well, one thing I've just realised is that the other app needs to know about kdbus too. So we have a double compat issue. Still, I think that a binding *can* provide a P2P-style bus by abusing endpoints with just a bit of convention rules. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 ___ systemd-devel mailing list
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wednesday 26 November 2014 19:44:46 Lennart Poettering wrote: Now, that's the reason why acquirename/releasename need to be implemented client side. With that knowledge we can punch holes in the other calls too. For example, the GetConnectionUnixProcessID() call you brought up: let's now issue a GetConnectionUnixProcessID() async first, immediately followed by an AcquireName() for the same name. On dbus this will tell you the old owner, and then acquire the name for you. On kdbus with an external driver this would be different though. While the GetConnecitonUnixProcessID() is queued and waiting to be processed the AcquireName() might already have been executed (because it needs to be an ioctl as we have shown above), hence you would get your own PID returned, not the old one! hence, this calls really should be sync local ioctl()s, not half-half... Ok, thanks. I understand the issue now. Given that and the fact that there are legitimate uses for AddMatch, I am in agreement that bindings need to provide the interception themselves. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wed, 26.11.14 11:08, Thiago Macieira (thi...@kde.org) wrote: On Wednesday 26 November 2014 19:30:16 Lennart Poettering wrote: I must be misunderstanding something. The kernel enforces that each bus name is prefixed with $UID-. This is why the system bus is /sys/fs/kdbus/0-system rather than just /sys/fs/kdbus/system. This makes sure that users cannot play games with other users. Thanks for the clarification. That tidbit is nowhere in the kdbus.txt docs. It should probably be mentioned in the section about creating buses. Added now. Also, shouldn't CAP_IPC_OWNER-privileged processes be able to create a bus with an arbitrary name? We thought about that too, but then stepped away from it, in order to avoid potential namespace clashes with additional files we might want to place in the kdbusfs file system in the future. Given that there's a strict regime now how the userspace objects are named we have freedom picking names for new files we want to introduce from the kernel-side, as long as we don't name thew new files $SOMENUMBER-$SOMETHINGELSE... Also, it's another thing where being strict initially is a good thing, since opening stuff up later is easier than closing it down later. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Mon, 24.11.14 18:40, Thiago Macieira (thi...@kde.org) wrote: I'm wondering if the same solution should be applied to the session bus. That would have the unfortunate effect that applications that aren't ported to know about kdbus will always fallback to proxy functionality. It would be unfortunate because the number of applications that need policy decisions on the session bust must be asymptotically close to zero. I figure this is up to the library implementation. I'd probably simplify this and avoid duplicating the session bus name too. Session bus access-control policy = In principle, people could configure the session bus to do the same elaborate access-control as the system bus. In practice, this is not a particularly useful thing to do, because there are many ways for processes running under the same uid to subvert each other, particularly if a LSM like SELinux or AppArmor is not used. kdbus does not appear to make any attempt to protect a uid from itself: the uid that created a bus is considered to be privileged on that bus. I assume this means that the intention is that app sandboxing will use a separate Unix uid, like it does on Android? Unless there's an outcry from people who like LSMs, I'm inclined to say that protecting same-uid session processes from each other is doomed to failure, and hence that it's OK for DBUS_BUS_SESSION to connect to kdbus without special precautions. I don't understand this domain enough to be able to offer an opinion. I know that Tizen will want SMACK security applied even between processes of the same UID. I just don't know whether that maps to what Lennart said about labels. SMACK and SELinux will have the chance to make stricter decisions thang the baseline policy. The SMACK folks have posted patches that add the right hooks to kdbus to make this possible. That's over 25% of the limit. Can this be made runtime-configurable? Yes, that's the plan. THough currently they are compiled in. I have now increased this to 1K. In kdbus, each connection may own up to 64 well-known names; the system dbus-daemon defaults to 512, and the session to 50 000. 64 is *probably* enough, but I could potentially see this becoming an issue for services that allocate one well-known name per $app_specific_thing, like Telepathy (one name per Connection). Also be made configurable, but please raise the default to 256 or 512. Same, also increased now. To 256. https://code.google.com/p/d-bus/source/detail?r=20ce3cfa9f65fc6a0be052ec64d9d796626f6630 A couple of other items to discuss: == DBUS__BUS_ADDRESS == We probably discussed this. Should we specify that the address on the environment variable should be of the form: kdbus:path=/sys/fs/kdbus/,uuid=uuid from hello[;fall back addresses] Well, we don't need any env var really, as we enforce that the UID of the user is included in the name of their bussess, and the busses are cleaned up when the registrar dies. We don't have the risk of leaving old busses around, or even by other users, hence all code can just imply the path to use is kernel:path=/sys/fs/kdbus/0-system and kernel:path=/sys/fs/kdbus/$UID-user and all is good, without ever having to deal with env vars at all. (of course, if env-vars are set they should be used, but the normal codepaths in the distros should work without them.) == org.freedesktop.DBus connection == Will systemd-kdbus provide that name on the bus so applications that make calls directly be able to continue working? I imagine the following methods would be interesting to have: No, this is not supported in the current versions of kdbus anymore. Emulation of these calls must happen client side if it shall be supported. org.freedesktop.DBus.GetAdtAuditSessionData org.freedesktop.DBus.GetConnectionCredentials org.freedesktop.DBus.GetConnectionSELinuxSecurityContext org.freedesktop.DBus.GetConnectionUnixProcessID org.freedesktop.DBus.GetConnectionUnixUser org.freedesktop.DBus.GetId org.freedesktop.DBus.GetNameOwner org.freedesktop.DBus.ListActivatableNames org.freedesktop.DBus.ListNames org.freedesktop.DBus.ListQueuedOwners org.freedesktop.DBus.NameHasOwner org.freedesktop.DBus.ReloadConfig org.freedesktop.DBus.StartServiceByName org.freedesktop.DBus.UpdateActivationEnvironment Most of those would be just convenience for other, existing kdbus low-level calls, but ReloadConfig and UpdateActivationEnvironment are not available anywhere else. It's true that there's nothing stopping more CAP_IPC_OWNER connections from installing more activators, but the question is whether systemd will provide those for the activations it holds. The client side emulation can choose to either forward ReloadConifg and UpdateActivationEnvironment to the respect systemd calls, or just return som not supported error. == Kernel API == === Custom endpoints === The
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Tuesday 25 November 2014 17:11:36 Lennart Poettering wrote: [snip] Thanks for raising the resource limits. == DBUS__BUS_ADDRESS == We probably discussed this. Should we specify that the address on the environment variable should be of the form: kdbus:path=/sys/fs/kdbus/,uuid=uuid from hello[;fall back addresses] Well, we don't need any env var really, as we enforce that the UID of the user is included in the name of their bussess, and the busses are cleaned up when the registrar dies. We don't have the risk of leaving old busses around, or even by other users, hence all code can just imply the path to use is kernel:path=/sys/fs/kdbus/0-system and kernel:path=/sys/fs/kdbus/$UID-user and all is good, without ever having to deal with env vars at all. (of course, if env-vars are set they should be used, but the normal codepaths in the distros should work without them.) Thinking of non-system buses here. If the variable is empty, I agree that it should have an equivalent of an autostart mechanism, but I disagree on the solution and I also disagree that distros should leave it empty. For one thing, the fallback address is expected to be there if there's a proxy bus running. The current autostart mechanism relies on X being present, so the fallback won't be found unless X is running and something registered the proxy's socket address there. For another, it's good practice to have it set and not depend on autostart. For a third, hardcoding kernel paths in userspace sounds like a poor idea. The kdbus mountpoint may be elsewhere and whatever is creating buses may not do it per user, but per session or other creation rule it may have. So we should make sure code works when the env vars are missing, but we should recommend that they always be set. == org.freedesktop.DBus connection == Will systemd-kdbus provide that name on the bus so applications that make calls directly be able to continue working? I imagine the following methods would be interesting to have: No, this is not supported in the current versions of kdbus anymore. Emulation of these calls must happen client side if it shall be supported. That wouldn't be kdbus, but systemd doing it. Since systemd is the one that opens the bus, it can register the first connection and claim the org.freedesktop.DBus service name, providing compatibility. So this isn't a feature request for kdbus but a feature request for systemd. By the way, is there a way to ensure that a given connection is the first connection? As soon as the bus creator is able to connect to the /sys/fs/kdbus path, so is another process and therefore this other process could maliciously acquire names it shouldn't. org.freedesktop.DBus.ReloadConfig org.freedesktop.DBus.StartServiceByName org.freedesktop.DBus.UpdateActivationEnvironment Most of those would be just convenience for other, existing kdbus low-level calls, but ReloadConfig and UpdateActivationEnvironment are not available anywhere else. It's true that there's nothing stopping more CAP_IPC_OWNER connections from installing more activators, but the question is whether systemd will provide those for the activations it holds. The client side emulation can choose to either forward ReloadConifg and UpdateActivationEnvironment to the respect systemd calls, or just return som not supported error. Can't do that. What if it's a kdbus system that is not systemd? I don't mind forwarding to a well-known bus name, as long as we establish that there is such a service running on the bus that will accept those calls. But if such a service exists, why can't it claim the org.freedesktop.DBus name? == Kernel API == === Custom endpoints === The docs say To create a custom endpoint, use the KDBUS_CMD_ENDPOINT_MAKE ioctl. On what file descriptor? The one for the control file? Or can it be sent on any kdbus endpoint? I'm asking because I'm not sure what the permissions of the control file will be -- will any process be allowed to open it and create endpoints? if you want to create a new endpoint for an existing bus, then invoke that ioctl on the bus fd. The control file after all is unrelated to any bus, and thus wouldn#t know which bus you mean if we'd allow invoking that ioctl on it. Ok, so any application that connected to the bus bus can then create custom endpoints. Correct? How does one get to install policies or activators on this custom bus if the opening connection is a regular, non-privileged process? But if that's the case, how would one implement a peer-to-peer connection? Or should it simply be a convention that P2P connections are really regular buses, except that no one owns any names, there are no policy restrictions and that the only two connections are :1.1 and :1.2? kdbus is not for peer-to-peer connections. If you want that use AF_UNIX. Why? There's really no need for peer-to-peer
Re: [systemd-devel] Compatibility between D-Bus and kdbus
Hi Thiago On Tue, Nov 25, 2014 at 9:01 PM, Thiago Macieira thi...@kde.org wrote: On Tuesday 25 November 2014 17:11:36 Lennart Poettering wrote: == org.freedesktop.DBus connection == Will systemd-kdbus provide that name on the bus so applications that make calls directly be able to continue working? I imagine the following methods would be interesting to have: No, this is not supported in the current versions of kdbus anymore. Emulation of these calls must happen client side if it shall be supported. That wouldn't be kdbus, but systemd doing it. Since systemd is the one that opens the bus, it can register the first connection and claim the org.freedesktop.DBus service name, providing compatibility. So this isn't a feature request for kdbus but a feature request for systemd. We had systemd-bus-driverd, which implemented org.freedesktop.DBus as normal service. However, this didn't work out as many dbus clients rely on this services to not be re-ordered in regard to external requests. In particular, if gdbus runs AddMatch(), it assumes the match takes effect immediately. If it sends a method call to another service after installing the match, and this triggers a signal, gdbus assumes the AddMatch() call to have succeeded (without waiting for the reply). However, if org.freedesktop.DBus is not implemented by the bus, but by an external service, you cannot guarantee that messages targetted at different receivers don't get re-ordered, and there're no guarantees which process gets scheduled first. This is a real bug and we couldn't figure out a way to fix it. Current DBus applications depend on org.freedesktop.DBus to be handled by the bus entity _in-order_. Therefore, we dropped systemd-bus-driverd and all the kdbus ioctls that we added to support this. I strongly recommend to either drop support for org.freedesktop.DBus on any kdbus-aware DBus APIs, or fake it in the library. sd-bus doesn't support it, and IIRC Ryan didn't want to fake it in gdbus either. Applications are required to use explicit add_match/remove_match library calls, instead of sending messages to org.freedesktop.DBus. Note that for legacy applications, we emulate org.freedesktop.DBus in our proxy. So this is really just about applications that want to use kdbus directly. By the way, is there a way to ensure that a given connection is the first connection? As soon as the bus creator is able to connect to the /sys/fs/kdbus path, so is another process and therefore this other process could maliciously acquire names it shouldn't. Acquiring names requires matching policies. If you setup your policies in a way that two applications can acquire the same name, you're doing something wrong. Or maybe I don't understand your use-case? == Kernel API == === Custom endpoints === The docs say To create a custom endpoint, use the KDBUS_CMD_ENDPOINT_MAKE ioctl. On what file descriptor? The one for the control file? Or can it be sent on any kdbus endpoint? I'm asking because I'm not sure what the permissions of the control file will be -- will any process be allowed to open it and create endpoints? if you want to create a new endpoint for an existing bus, then invoke that ioctl on the bus fd. The control file after all is unrelated to any bus, and thus wouldn#t know which bus you mean if we'd allow invoking that ioctl on it. Ok, so any application that connected to the bus bus can then create custom endpoints. Correct? No, custom endpoints can only be created by privileged users. On the system-bus, this means root, on the user-bus this means processes of the user itself. How does one get to install policies or activators on this custom bus if the opening connection is a regular, non-privileged process? Policy holders and activators are privileged operations, like creating custom endpoints. You need to open an endpoint and pass POLICY_HOLDER or ACTIVATOR in KDBUS_CMD_HELLO to become a policy-holder or activator. You will not be an ordinary connection, so you will not be announced on the bus, nor can you send messages. But if that's the case, how would one implement a peer-to-peer connection? Or should it simply be a convention that P2P connections are really regular buses, except that no one owns any names, there are no policy restrictions and that the only two connections are :1.1 and :1.2? kdbus is not for peer-to-peer connections. If you want that use AF_UNIX. Why? kdbus implements bus-based IPC. If you want P2P IPC, use one of the established transports. Yes, kdbus has some handy features people like to see on unix-sockets (like flexible metadata transports), but our current policy is fix unix sockets!, and this can optionally be implemented later on. There is no plan to support P2P connections in the initial kdbus draft. There's really no need for peer-to-peer connections really, at least performance-wise. The need is that we can avoid loading the code that does AF_UNIX
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Tue, 25.11.14 12:01, Thiago Macieira (thi...@kde.org) wrote: Well, we don't need any env var really, as we enforce that the UID of the user is included in the name of their bussess, and the busses are cleaned up when the registrar dies. We don't have the risk of leaving old busses around, or even by other users, hence all code can just imply the path to use is kernel:path=/sys/fs/kdbus/0-system and kernel:path=/sys/fs/kdbus/$UID-user and all is good, without ever having to deal with env vars at all. (of course, if env-vars are set they should be used, but the normal codepaths in the distros should work without them.) Thinking of non-system buses here. If the variable is empty, I agree that it should have an equivalent of an autostart mechanism, but I disagree on the solution and I also disagree that distros should leave it empty. Oh, no. No autostart please. No such concept exists in kdbus, and systemd/sd-bus will not support that either. In fact I refuse to support that even on dbus1 in sd-bus. Autostart is a kludge for systems where dbus is just an add-on, but that's completely out-of-focus for kdbus, systemd and sd-bus. Note that even on systemd we will set $DBUS_SESSION_BUS_ADDRESS, simply because classic libdbus and gdbus won't work without it. However, we will actually set it to a fixed value. For one thing, the fallback address is expected to be there if there's a proxy bus running. The current autostart mechanism relies on X being present, so the fallback won't be found unless X is running and something registered the proxy's socket address there. For another, it's good practice to have it set and not depend on autostart. For a third, hardcoding kernel paths in userspace sounds like a poor idea. The kdbus mountpoint may be elsewhere and whatever is creating buses may not do it per user, but per session or other creation rule it may have. No, we don't support weird setups where kdbusfs mounted elsewhere. This is a bew API we introduce here, and we can very much make decisions where stuff is to be mounted. Env vars are a hack, due to the awful inheritence logic, and we should really avoid using them, except where necessary for compat, and that's precisely to which level we'll support them in systemd. would be interesting to have: No, this is not supported in the current versions of kdbus anymore. Emulation of these calls must happen client side if it shall be supported. That wouldn't be kdbus, but systemd doing it. Since systemd is the one that opens the bus, it can register the first connection and claim the org.freedesktop.DBus service name, providing compatibility. So this isn't a feature request for kdbus but a feature request for systemd. We initially tried to support that, but it's awfully racy, since the driver calls and calls to other services wouldn't be executed in strict order anymore... We removed this again after figuring out and decided that emulation can only happen client side, synchronous to the message stream if we want to guarantee correct ordering. By the way, is there a way to ensure that a given connection is the first connection? As soon as the bus creator is able to connect to the /sys/fs/kdbus path, so is another process and therefore this other process could maliciously acquire names it shouldn't. When creating the bus the creator can pass policy to the kernel so that there is no time window where the bus is accessible and open to manipulation from untrusted clients. org.freedesktop.DBus.ReloadConfig org.freedesktop.DBus.StartServiceByName org.freedesktop.DBus.UpdateActivationEnvironment Most of those would be just convenience for other, existing kdbus low-level calls, but ReloadConfig and UpdateActivationEnvironment are not available anywhere else. It's true that there's nothing stopping more CAP_IPC_OWNER connections from installing more activators, but the question is whether systemd will provide those for the activations it holds. The client side emulation can choose to either forward ReloadConifg and UpdateActivationEnvironment to the respect systemd calls, or just return som not supported error. Can't do that. What if it's a kdbus system that is not systemd? Well, again, return not supported then. I mean, currently there is no kdbus userspace implementation beyond kdbus, we cannot really discuss something that doesn't exist... I don't mind forwarding to a well-known bus name, as long as we establish that there is such a service running on the bus that will accept those calls. But if such a service exists, why can't it claim the org.freedesktop.DBus name? Note that on dbus1 systemd systems we actually never provided UpdateActivationEnvironment correctly (since services got forked off PID 1, instead of dbus-daemon but the call would alter dbus-daemon's env block, not systemd's one), but nobody ever noticed. I really
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wed, 26.11.14 00:46, David Herrmann (dh.herrm...@gmail.com) wrote: Custom endpoints do _not_ create new buses. Really. You could create a custom bus and use it for just 2 connections, but then you could also just use socketpair(2). Note that there was some discussion on anonymous buses, which would allow to create such buses on the fly. But again, this will not be part of the initial kdbus draft. If anyone cares, submit it as patches once kdbus is upstream. I gave up on the anonymous bus idea btw. With the current kdbusfs-based approach creating a bus is so cheap that the benefit of being anonymous is realy gone. We can implement a scheme like this fully in userspace now, by simply using a randomized bus name... Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wednesday 26 November 2014 01:25:18 Lennart Poettering wrote: Thinking of non-system buses here. If the variable is empty, I agree that it should have an equivalent of an autostart mechanism, but I disagree on the solution and I also disagree that distros should leave it empty. Oh, no. No autostart please. No such concept exists in kdbus, and systemd/sd-bus will not support that either. In fact I refuse to support that even on dbus1 in sd-bus. Autostart is a kludge for systems where dbus is just an add-on, but that's completely out-of-focus for kdbus, systemd and sd-bus. I didn't actually mean automatically starting the bus, sorry for the confusion. I meant automatic discovery only. Currently on dbus1, autostart: as a transport protocol means both auto discovery and automatic starting if the bus isn't running. Please also note that the autostart solution has a valid use-case which is when a D-Bus application is launched in an environment where no bus had been started before. I understand this is out-of-scope for kdbus, since after all a regular user won't be able to create a kdbus bus if one wasn't provided by a privileged process before. In an environment where a kdbus bus wasn't provided, the only alternative is to fallback to dbus1. Note that even on systemd we will set $DBUS_SESSION_BUS_ADDRESS, simply because classic libdbus and gdbus won't work without it. However, we will actually set it to a fixed value. That's all I asked for. Whether the value is constant or not is not relevant, as long as it gets set. For one thing, the fallback address is expected to be there if there's a proxy bus running. The current autostart mechanism relies on X being present, so the fallback won't be found unless X is running and something registered the proxy's socket address there. For another, it's good practice to have it set and not depend on autostart. For a third, hardcoding kernel paths in userspace sounds like a poor idea. The kdbus mountpoint may be elsewhere and whatever is creating buses may not do it per user, but per session or other creation rule it may have. No, we don't support weird setups where kdbusfs mounted elsewhere. This is a bew API we introduce here, and we can very much make decisions where stuff is to be mounted. You may not support it in systemd, but from reading the kernel API that could happen with another implementation. Env vars are a hack, due to the awful inheritence logic, and we should really avoid using them, except where necessary for compat, and that's precisely to which level we'll support them in systemd. Env vars actually match pretty well the concept of session, except when nested sessions happen without resetting the necessary environment variables (e.g., a screen(1), tmux(1) or Xnest sub-session). I know you're designing systemd so that it won't provide session buses, which is why it feels like a kludge to you. I won't argue here. I'm satisfied that the env var gets set. would be interesting to have: No, this is not supported in the current versions of kdbus anymore. Emulation of these calls must happen client side if it shall be supported. That wouldn't be kdbus, but systemd doing it. Since systemd is the one that opens the bus, it can register the first connection and claim the org.freedesktop.DBus service name, providing compatibility. So this isn't a feature request for kdbus but a feature request for systemd. We initially tried to support that, but it's awfully racy, since the driver calls and calls to other services wouldn't be executed in strict order anymore... We removed this again after figuring out and decided that emulation can only happen client side, synchronous to the message stream if we want to guarantee correct ordering. I'm not asking for AddMatch and connection control mechanisms. The one I really want is StartServiceByName, since it can't be emulated. Moreover, starting services is systemd's raison-d'être, so I feel it should be no problem for you to provide such a service. It would be nice if UpdateActivationEnvironment worked. This functionality was added for people who need to update variables like XDG_DATA_DIRS after starting the bus. If this one isn't present, we can report not implemented and be fine with it. We'll just have to tell people to configure their systemd user session environments properly. The same goes for ReloadConfig, but I'd prefer to know whether that failed (no config reloading is possible) or whether it happens automatically whether the call was made or not. ReloadConfig is important when there are new activatable services on a user's bus, such as newly-installed applications. The client side emulation can choose to either forward ReloadConifg and UpdateActivationEnvironment to the respect systemd calls, or just return som not supported error. Can't do that. What if it's a kdbus system that
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wednesday 26 November 2014 00:46:50 David Herrmann wrote: We had systemd-bus-driverd, which implemented org.freedesktop.DBus as normal service. However, this didn't work out as many dbus clients rely on this services to not be re-ordered in regard to external requests. In particular, if gdbus runs AddMatch(), it assumes the match takes effect immediately. If it sends a method call to another service after installing the match, and this triggers a signal, gdbus assumes the AddMatch() call to have succeeded (without waiting for the reply). However, if org.freedesktop.DBus is not implemented by the bus, but by an external service, you cannot guarantee that messages targetted at different receivers don't get re-ordered, and there're no guarantees which process gets scheduled first. This is a real bug and we couldn't figure out a way to fix it. Current DBus applications depend on org.freedesktop.DBus to be handled by the bus entity _in-order_. Therefore, we dropped systemd-bus-driverd and all the kdbus ioctls that we added to support this. Aside from the connection-control mechanisms (AddMatch, RemoveMatch), did you see any problems? User code usually shouldn't be doing AddMatch and RemoveMatch (the only case I can think of is dconf, which does its own parsing of the message payload). That's the domain of the binding, which will know that it connected to kdbus and won't be making those calls. As I said in the reply to Lennart, I'm specifically looking for StartServiceByName. Everything else is emulatable anyway or nonsense in kdbus. By the way, is there a way to ensure that a given connection is the first connection? As soon as the bus creator is able to connect to the /sys/fs/kdbus path, so is another process and therefore this other process could maliciously acquire names it shouldn't. Acquiring names requires matching policies. If you setup your policies in a way that two applications can acquire the same name, you're doing something wrong. Or maybe I don't understand your use-case? No, it was my misunderstanding. The endpoint creator should set up a policy so that it itself can acquire the name(s) it needs, then deny any other connections owning names or TALKing to each other. This effectively creates a many-to-one bus system. E.g., a fake P2P. Ok, so any application that connected to the bus bus can then create custom endpoints. Correct? No, custom endpoints can only be created by privileged users. On the system-bus, this means root, on the user-bus this means processes of the user itself. Sorry, I meant application connected to the a user's bus bus. Like you said, that means any application with the same UID, which means all applications that were allowed to connect to bus in the first place. kdbus implements bus-based IPC. If you want P2P IPC, use one of the established transports. Yes, kdbus has some handy features people like to see on unix-sockets (like flexible metadata transports), but our current policy is fix unix sockets!, and this can optionally be implemented later on. There is no plan to support P2P connections in the initial kdbus draft. I'm not asking for much more for P2P connections in kdbus. I think that it's already doable by restricting an endpoint to a many-to-one system. Custom endpoints do _not_ create new buses. Really. You could create a custom bus and use it for just 2 connections, but then you could also just use socketpair(2). Note that there was some discussion on anonymous buses, which would allow to create such buses on the fly. But again, this will not be part of the initial kdbus draft. If anyone cares, submit it as patches once kdbus is upstream. As far as I can tell, an anonymous bus in the sense of a bus with a randomised name is doable by having the proper interface in the systemd1 system bus service. An application would request the bus and receive as a reply the address of the bus, the file descriptor for the control, and a file descriptor to a privileged connection so it can control the policies. This does not require kernel-side modifications, only in systemd. Understood, but coming from userspace's perspective this seems ill-advised and optimising for the wrong use-case. The common case is that syscalls are not interrupted: with a timeout, there's one clock_gettime() call *after* the interruption if any; with the timestamp, there's always one before the syscall. Use clock_gettime(CLOCK_MONOTONIC_COARSE). This reads the time from the VDSO, which is a simple memory read. No trap/syscall is involved. Downside is that you loose precision, but that should really be fine for this use-case. I need to read a little more on the coarse monotonic clock to see what cases it's applicable for and when you should avoid it. In this specific case, precision is not required, as a 25-second timeout is really long. Currently, QtDBus on dbus1 may get that wrong for as much as
Re: [systemd-devel] Compatibility between D-Bus and kdbus
Sorry for the delay in replying. I didn't have time until after KLF... On Wednesday 01 October 2014 14:33:01 Simon McVittie wrote: System bus access-control policy [snip] If I remember correctly, the least bad solution anyone could think of was to introduce a new pseudo-bus-type alongside DBUS_BUS_SYSTEM (and its equivalent in other libraries like GDBus), perhaps called DBUS_BUS_SYSTEM_UNTRUSTED or something (better names welcome), with its own shared connection: connections to that bus type are not assumed to filter messages by their payload, and method-call recipients are expected to use Polkit or similar, or do their own simplistic access-controls like must be uid 0 by calling GetConnectionUnixUser or GetConnectionCredentials on the sender's unique name. I'm wondering if the same solution should be applied to the session bus. That would have the unfortunate effect that applications that aren't ported to know about kdbus will always fallback to proxy functionality. It would be unfortunate because the number of applications that need policy decisions on the session bust must be asymptotically close to zero. An alternative solution would be for the trusted connection to check if there are any files in /etc/dbus-1/session.d. If there aren't, it can assume that trusted == untrusted. PS: we should also find a name that conveys that you *want* the new bus type. I'm also very tempted to propose a syntax for an opt-in kdbus-like security model (which would take precedence over system.conf/system.d) via adding lines to .service files, so that individual services can have a sane security model on non-kdbus or non-Linux systems, and systemd's systemd-dbus1-generator could use those lines as input. If we get far enough with moving system services to that, maybe the transition can be easier. I like this idea, with the proviso that Lennart pointed out: we need to first include the metadata in the dbus1 stream messages so the application can sort out the policy decisions like the kdbus implementation would. Session bus access-control policy = In principle, people could configure the session bus to do the same elaborate access-control as the system bus. In practice, this is not a particularly useful thing to do, because there are many ways for processes running under the same uid to subvert each other, particularly if a LSM like SELinux or AppArmor is not used. kdbus does not appear to make any attempt to protect a uid from itself: the uid that created a bus is considered to be privileged on that bus. I assume this means that the intention is that app sandboxing will use a separate Unix uid, like it does on Android? Unless there's an outcry from people who like LSMs, I'm inclined to say that protecting same-uid session processes from each other is doomed to failure, and hence that it's OK for DBUS_BUS_SESSION to connect to kdbus without special precautions. I don't understand this domain enough to be able to offer an opinion. I know that Tizen will want SMACK security applied even between processes of the same UID. I just don't know whether that maps to what Lennart said about labels. Resource limits === [snip] In kdbus, each uid may create up to 16 buses. In dbus-daemon there is no limit. I do not anticipate this being a problem: the reference implementation of kdbus' user-space, in systemd, seems to be using a per-uid user bus instead of a session bus. Also, even if we continued to use a session bus per login, 16 sounds like a reasonable number. Agreed. Most people I know that start more than one bus for the same user are doing it for nested sessions, usually for testing purposes or to keep separate implementations of equivalent services (early KDE Frameworks 5 releases recommended being separate from KDE 4, not so any more). In kdbus, each uid may connect to each bus up to 256 times. I think this is actually somewhat likely to be a practical problem: I currently have 46 connections to my session bus, so I'm only an order of magnitude away from the session breaking. https://code.google.com/p/d-bus/issues/detail?id=9 Agreed again. $ qdbus | grep -c \: 72 That's over 25% of the limit. Can this be made runtime-configurable? In kdbus, each connection may own up to 64 well-known names; the system dbus-daemon defaults to 512, and the session to 50 000. 64 is *probably* enough, but I could potentially see this becoming an issue for services that allocate one well-known name per $app_specific_thing, like Telepathy (one name per Connection). Also be made configurable, but please raise the default to 256 or 512. In kdbus, each connection may have 256 bloom filter entries, which AIUI are slightly less expressive than match rules (one match rule maps to one or more match rules). The system dbus-daemon defaults to allowing 512 match rules, and the session to 50 000.
Re: [systemd-devel] Compatibility between D-Bus and kdbus
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 1/10/2014 15:33, Simon McVittie wrote: [cut] Resource limits === Some resource limits are lower in kdbus than in dbus-daemon. In kdbus, the number of unread messages per recipient is limited to 256, with up to 16 per uid; subsequent broadcasts are silently dropped, and subsequent unicast messages cause the sender to block. The message header (fixed-length header and header fields) is limited to 2 MiB in kdbus, whereas on dbus-daemon it may be up to a configurable limit, by default 32 MiB for the system bus or 128 MiB for the session bus. Broadcast messages (header + body) have the same 2 MiB limit, but unicast message bodies may be any size: kdbus itself does not impose any limit. I don't know whether anything sends broadcasts as large as 2 MiB (Tracker perhaps?): if you do, please share. Tracker's GraphUpdated will 'compress' its payload by sending numerical ID's that represent the graph, resource, property and sometimes value to signal inserts and deletes on a graph. It has two triggers to invoke it (which also flushes it) When the amount of events is larger than a certain number: https://git.gnome.org/browse/tracker/tree/src/tracker-store/tracker-resources.vala#n25 https://git.gnome.org/browse/tracker/tree/src/tracker-store/tracker-resources.vala#n312 And timer based as configured as graphupdated_delay (dconf config): https://git.gnome.org/browse/tracker/tree/src/tracker-store/tracker-resources.vala#n297 Some more explanation here: http://pvanhoof.be/blog/index.php/2010/08/24/trackers-new-class-signal-system-being-developed Of greater concern is an API which I always considered to be for debugging purposes only. That's the Query API on Resources1 of Tracker: libtracker-sparql will not ever use it, the library will either use a direct connection with sqlite's meta.db using WAL journaling for read queries, or fall back to using FD passing with as you mentioned in your mail pipe() and vmsplice(). That's the Steroids1 D-Bus interface of Tracker's SPARQL endpoint. Although I consider the Query API on Resources1 to be a private or debug D-Bus API, it can indeed cause a very large DBusMessage. This was once put in place to guard ourselves against it: https://git.gnome.org/browse/tracker/tree/src/tracker-store/tracker-resources.vala#n50 https://git.gnome.org/browse/tracker/tree/src/tracker-store/tracker-resources.vala#n111 Some explanation here: https://git.gnome.org/browse/tracker/tree/src/tracker-store/tracker-resources.vala#n27 [cut] Comments, corrections? (To the dbus list, please.) Kind regards, Philip -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (MingW32) iQEcBAEBAgAGBQJUSCi9AAoJEEP2NSGEz4aDDKEH/RDRuIHtmUljzdYX7XQqGA6N I/0DAaCHRCg0D2az9MbzgEnrFrvVmhACtGrFcWA15hfasNadcCmPnIBRJk84WOld rELimMayQ8wyCsKlCBszoAjQKVe4Qr5KVw3zVNBTPvfs6xsqdeGFSaNbFDrrCZtC 5SmwnlI9ezenJ3+EjAbTqPUDi7rMciA4BvDWI0HSa4TQD/ggtnHnVyikff6POPX7 BQcHp65vBSCvz+mwVPUawTI04l3MC8UXzMOYw/I+UduESnptbD7ZyJ0GzUY7rxxI Se/Et+ijSBCdvb3cuBnDk14kZ+slJFEBrYLsukZVdQ8hqzm1NmU0/XBsMK3eN+c= =GA7y -END PGP SIGNATURE- ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Compatibility between D-Bus and kdbus
(Cc'd to the systemd mailing list because sd-bus is the reference implementation of the user-space side of kdbus, but please join the dbus list and follow-up there if you are interested in D-Bus.) I've recently been looking at kdbus as a transport for D-Bus messages, and how compatible or otherwise it is with traditional stream-based D-Bus (and in particular, dbus-daemon, the reference implementation of stream-based D-Bus). My intention here is to replace dbus-daemon on Linux with something that does not have dbus-daemon's limitations, but avoid making existing software non-functional or insecure in the process. See https://bugs.freedesktop.org/show_bug.cgi?id=84188 for my attempts to document the current state of kdbus in the D-Bus Specification. I have not reviewed quality-of-implementation except where I happened to notice things as I went past, and I have not yet looked at Samsung/Tizen's patches adding a kdbus transport to libdbus and GDBus; for now I'm using sd-bus as my only reference for the user-space parts. Here are the likely compatibility issues that I can see in my first pass through: System bus access-control policy I think this is the biggest point of incompatibility. In dbus-daemon there is a needlessly elaborate access-control language; in kdbus, there is a much simpler and more realistic access-control language, which specifically does not look into message payloads. This is far easier to reason about than what dbus-daemon does, but it's a problem for any system service that assumes that its existing per-interface or per-method access-control will be applied, such as Avahi denying access to SetHostName. It is not acceptable for such services to get instant security flaws as a result of their D-Bus implementation being upgraded from a version that does not support kdbus to a version that does. Unfortunately, last time we discussed this, we didn't have a particularly good solution. If I remember correctly, the least bad solution anyone could think of was to introduce a new pseudo-bus-type alongside DBUS_BUS_SYSTEM (and its equivalent in other libraries like GDBus), perhaps called DBUS_BUS_SYSTEM_UNTRUSTED or something (better names welcome), with its own shared connection: connections to that bus type are not assumed to filter messages by their payload, and method-call recipients are expected to use Polkit or similar, or do their own simplistic access-controls like must be uid 0 by calling GetConnectionUnixUser or GetConnectionCredentials on the sender's unique name. On non-kdbus systems, it would just be the same thing as DBUS_BUS_SYSTEM; on kdbus systems, DBUS_BUS_SYSTEM would refuse to use the kdbus transport, and fall back to stream-based D-Bus compatibility mechanisms like systemd-bus-proxyd. That would enable individual libraries and applications to opt-in to using this new shared connection when they have been audited for safety, with the goal being that everything eventually moved to it, and nothing connected to DBUS_BUS_SYSTEM any more. I'm also very tempted to propose a syntax for an opt-in kdbus-like security model (which would take precedence over system.conf/system.d) via adding lines to .service files, so that individual services can have a sane security model on non-kdbus or non-Linux systems, and systemd's systemd-dbus1-generator could use those lines as input. If we get far enough with moving system services to that, maybe the transition can be easier. Session bus access-control policy = In principle, people could configure the session bus to do the same elaborate access-control as the system bus. In practice, this is not a particularly useful thing to do, because there are many ways for processes running under the same uid to subvert each other, particularly if a LSM like SELinux or AppArmor is not used. kdbus does not appear to make any attempt to protect a uid from itself: the uid that created a bus is considered to be privileged on that bus. I assume this means that the intention is that app sandboxing will use a separate Unix uid, like it does on Android? Unless there's an outcry from people who like LSMs, I'm inclined to say that protecting same-uid session processes from each other is doomed to failure, and hence that it's OK for DBUS_BUS_SESSION to connect to kdbus without special precautions. Resource limits === Some resource limits are lower in kdbus than in dbus-daemon. In kdbus, the number of unread messages per recipient is limited to 256, with up to 16 per uid; subsequent broadcasts are silently dropped, and subsequent unicast messages cause the sender to block. The message header (fixed-length header and header fields) is limited to 2 MiB in kdbus, whereas on dbus-daemon it may be up to a configurable limit, by default 32 MiB for the system bus or 128 MiB for the session bus. Broadcast messages (header + body) have the same 2 MiB limit, but unicast message bodies
Re: [systemd-devel] Compatibility between D-Bus and kdbus
Hi Simon, On 10/01/2014 03:33 PM, Simon McVittie wrote: (Cc'd to the systemd mailing list because sd-bus is the reference implementation of the user-space side of kdbus, but please join the dbus list and follow-up there if you are interested in D-Bus.) I've recently been looking at kdbus as a transport for D-Bus messages, and how compatible or otherwise it is with traditional stream-based D-Bus (and in particular, dbus-daemon, the reference implementation of stream-based D-Bus). My intention here is to replace dbus-daemon on Linux with something that does not have dbus-daemon's limitations, but avoid making existing software non-functional or insecure in the process. See https://bugs.freedesktop.org/show_bug.cgi?id=84188 for my attempts to document the current state of kdbus in the D-Bus Specification. Thanks a lot for doing this! Much appreciated. I have not reviewed quality-of-implementation except where I happened to notice things as I went past, and I have not yet looked at Samsung/Tizen's patches adding a kdbus transport to libdbus and GDBus; for now I'm using sd-bus as my only reference for the user-space parts. Let me just state that we didn't post kernel patches to the D-Bus list in the past, as we didn't intend to discuss the internals of the kernel code on primarily user-space mailing lists. That, of course, doesn't mean we didn't want to discuss the external userspace API. We absolutely want to make sure that kdbus is capable of serving as transportation level interface that allows us to implement the same set of features that we have now, eventually, and it seems like a good time to get into the details. While in the past, we were saying that kdbus is not ready yet and people should not yet look at it closely, we're now at a stage in which we consider the interface more or less stable. Some internal details might still change, however. So, for everyone interested, please have a look at the following git repository: https://code.google.com/p/d-bus/ There's a fairly comprehensive documentation in kdbus.txt on the public kernel API, which is itself defined in kdbus.h. If you have any questions that are not answered by the existing documentation, please feel free to ask. We also accept patches, of course. Since a long time, it's also possible to boot up machines with systemd when --enable-kdbus is passed in configure, the module is built and installed, and the verb 'kdbus' is added to the kernel command line. System bus access-control policy I think this is the biggest point of incompatibility. In dbus-daemon there is a needlessly elaborate access-control language; in kdbus, there is a much simpler and more realistic access-control language, which specifically does not look into message payloads. Exactly. In contrast to the traditional dbus-daemon, the transport layer in kdbus has no understanding of what it actually transports in its data blobs. There's no code in the kernel that is able to marshal or demarshal the payload, that's entirely left to the users and the libraries they use. This is far easier to reason about than what dbus-daemon does, but it's a problem for any system service that assumes that its existing per-interface or per-method access-control will be applied, such as Avahi denying access to SetHostName. It is not acceptable for such services to get instant security flaws as a result of their D-Bus implementation being upgraded from a version that does not support kdbus to a version that does. Unfortunately, last time we discussed this, we didn't have a particularly good solution. Yes, that needs to be addressed. However, we won't add support to kdbus for method filtering or anything alike, so the kernel won't be able to help enforce such policies. If I remember correctly, the least bad solution anyone could think of was to introduce a new pseudo-bus-type alongside DBUS_BUS_SYSTEM (and its equivalent in other libraries like GDBus), perhaps called DBUS_BUS_SYSTEM_UNTRUSTED or something (better names welcome), with its own shared connection: connections to that bus type are not assumed to filter messages by their payload, and method-call recipients are expected to use Polkit or similar, or do their own simplistic access-controls like must be uid 0 by calling GetConnectionUnixUser or GetConnectionCredentials on the sender's unique name. On non-kdbus systems, it would just be the same thing as DBUS_BUS_SYSTEM; on kdbus systems, DBUS_BUS_SYSTEM would refuse to use the kdbus transport, and fall back to stream-based D-Bus compatibility mechanisms like systemd-bus-proxyd. That would enable individual libraries and applications to opt-in to using this new shared connection when they have been audited for safety, with the goal being that everything eventually moved to it, and nothing connected to DBUS_BUS_SYSTEM any more. Exactly, we would fall back to legacy translators for such client.
Re: [systemd-devel] Compatibility between D-Bus and kdbus
On Wed, 01.10.14 14:33, Simon McVittie (simon.mcvit...@collabora.co.uk) wrote: Thanks a ton for reviewing kdbus and its concepts! (Also, sorry for not responding earlier to all the dbus/kdbus traffic in the last month, I was travelling.) System bus access-control policy I think this is the biggest point of incompatibility. In dbus-daemon there is a needlessly elaborate access-control language; in kdbus, there is a much simpler and more realistic access-control language, which specifically does not look into message payloads. This is far easier to reason about than what dbus-daemon does, but it's a problem for any system service that assumes that its existing per-interface or per-method access-control will be applied, such as Avahi denying access to SetHostName. It is not acceptable for such services to get instant security flaws as a result of their D-Bus implementation being upgraded from a version that does not support kdbus to a version that does. Unfortunately, last time we discussed this, we didn't have a particularly good solution. If I remember correctly, the least bad solution anyone could think of was to introduce a new pseudo-bus-type alongside DBUS_BUS_SYSTEM (and its equivalent in other libraries like GDBus), perhaps called DBUS_BUS_SYSTEM_UNTRUSTED or something (better names welcome), with its own shared connection: connections to that bus type are not assumed to filter messages by their payload, and method-call recipients are expected to use Polkit or similar, or do their own simplistic access-controls like must be uid 0 by calling GetConnectionUnixUser or GetConnectionCredentials on the sender's unique name. Yes, this is indeed the solution I have in mind, and I think Ryan is mostly OK with this too for GLib. On non-kdbus systems, it would just be the same thing as DBUS_BUS_SYSTEM; on kdbus systems, DBUS_BUS_SYSTEM would refuse to use the kdbus transport, and fall back to stream-based D-Bus compatibility mechanisms like systemd-bus-proxyd. That would enable individual libraries and applications to opt-in to using this new shared connection when they have been audited for safety, with the goal being that everything eventually moved to it, and nothing connected to DBUS_BUS_SYSTEM any more. I'm also very tempted to propose a syntax for an opt-in kdbus-like security model (which would take precedence over system.conf/system.d) via adding lines to .service files, so that individual services can have a sane security model on non-kdbus or non-Linux systems, and systemd's systemd-dbus1-generator could use those lines as input. If we get far enough with moving system services to that, maybe the transition can be easier. If packages want compatibility with both kdbus/systemd and classic dbus1, then I'd suggest to simply continue shipping the old XML policy file fragments, plus the new systemd .busname unit. On dbus1 systems the old policy would be used, on new kdbus systems the new policy would be used. The app would of course be updated to use the new bus name to connect to to acknowledge that it is responsible for doing auth on its own. In this scheme compatbility should be easy: just add the new stuff on top, and leave the old stuff in place, and things should be good. No need to add anything new that would be a stop-gap and little else. Session bus access-control policy = In principle, people could configure the session bus to do the same elaborate access-control as the system bus. In practice, this is not a particularly useful thing to do, because there are many ways for processes running under the same uid to subvert each other, particularly if a LSM like SELinux or AppArmor is not used. Yes, Linux access control is built around UIDs, GIDs and labels, and I am pretty strongly of the idea that we should do access control based on exactly these credentials, but not come up with new access control schemes betweend processes, that does not run along these lines. kdbus does not appear to make any attempt to protect a uid from itself: the uid that created a bus is considered to be privileged on that bus. I assume this means that the intention is that app sandboxing will use a separate Unix uid, like it does on Android? Well, our idea about sandboxing for apps that it is mostly about visibility, not just mere access control. Or in other words: if a sandboxed app shall not be able to manipulate other processes then these processes should not be visible to it at all (via PID namespacing). Similar, if it is not supposed to be able to manipulate per-user kernel settings, then this should also be solved by visibility (via user namespacing) and so on. For kdbus this translates to the custom endpoint logic, where you can add additional bus device nodes for existing busses, that carry a second, stricter policy that is applied on top of the main bus policy. This allows us to
