Re: APIs for persistent remote access and headless/hybrid sessions
A remote session would need the following four pieces: 1. An API for the remote access tool to request a remote display and authenticate. This could involve a headless greeter session for remote login or direct authentication of some sort. 2. A way for the login manager to launch a session in headless mode. 3. A way for the login manager to query whether an existing session is headless or can dynamically transition to being headless (and trigger the transition if so). 4. A way for the remote desktop tool to connect to and control a headless session (both the headless greeter, if used, and the user session). I think all of these parts can, and arguably should, be completely independent of a standardized org.freedesktop.DisplayManager: (1) should arguably be a separate interface than that for local display managers. Discussion on the wlroots tracker suggests many users of wlroots-based compositors prefer to have a simple local login manager like ly or greetd. Having a separate interface allows a separate package to offer remote login functionality on the system bus. A full featured display manager like GDM or SDDM could still offer both interfaces itself for better integration. (2) could be as simple as a property in the session's .desktop file providing a command-line for headless launch. (3) would presumably be an interface provided on the user bus by the compositor itself. (4) would also be a compositor-provided interface, which could either be a new or expanded Portal API or a distinct API like that proposed by Jonas Ådahl. I believe ConsoleKit and logind already provide a mechanism for a local login manager to switch to and unlock an existing session, so the main piece of functionality more integration or a standardized display manager API would provide would be to provide a way for the session to kick back to the login screen when curtained, instead of staying in the foreground with the physical inputs and outputs disabled. Please let me know if I'm missing something, though. I'm still relatively new to this area. > Also, it looks like there's a draft spec being worked on by Jonas > Ådahl about a remote desktop protocol (presumably based on what GNOME > has been working on). Indeed. (I mentioned it in my initial message.) However, I do not believe it is directly related to the remote login support Joan Torres is working on for GNOME Remote Desktop, which relies on GNOME-specific unstable APIs, but is rather a separate effort to provide a standard interface for what I call piece (4) above.
Re: APIs for persistent remote access and headless/hybrid sessions
On Fri, Mar 15, 2024 at 7:52 PM Erik Jensen wrote: > > Thinking about this more, I'm not really sure extending the existing > Portal API makes sense. > > Given that remote assistance (well served by the existing API) wants > > * Explicit user consent to share > * To allow the local user to select what to share > * To mirror the selected existing displays / windows > > while remote assistance wants > > * Persistent access > * To ensure the session is curtained or headless to prevent > observation and interaction from the local console > * Full control over the virtual monitor layout and the ability to > capture all of it > > It seems like a unified API would have enough special cases depending > on which access mode is in use that it wouldn't really be worth it, > especially given that the actual capture (PipeWire) and input > injection (libei) would be the same either way. E.g., even with > persistent permission tokens, the existing ScreenCast portal doesn't > really fit the remote access use case, and a separate API to control > the layout and get the resulting PipeWire streams without user monitor > selection likely makes more sense. > > > I suspect we might need Plasma Login Manager to exist first before > > we can achieve this, though. The kind of integration that GDM and > > GNOME Shell have allowed them to pull off what they did there, I'm > > not sure how to do it without that integration. > > Ideally, I'd like to have a standard protocol that is both lightweight > and flexible enough that it wouldn't require any deep integration > between the login manager and desktop environment, but could be > implemented by even lightweight login managers. I would like that too, but we currently don't even have a standard specification for display managers themselves. I thought we did, but then I did the research about the protocol that SDDM implements[1], and it turns out that it was created by LightDM over a decade ago[2] and was never documented as a standard in the first place. GDM's protocol is quite a bit more sophisticated (but also not standardized). So that's a problem that we might need to address first. Also, it looks like there's a draft spec[3] being worked on by Jonas Ådahl about a remote desktop protocol (presumably based on what GNOME has been working on). [1]: https://github.com/sddm/sddm/commit/069f1d7d91bca55673d78cbace448942d46965d6 [2]: https://github.com/canonical/lightdm/commits/main/src/display-manager.xml [3]: https://gitlab.freedesktop.org/jadahl/xdg-specs/-/merge_requests/1 -- 真実はいつも一つ！/ Always, there's only one truth!
Re: APIs for persistent remote access and headless/hybrid sessions
Thinking about this more, I'm not really sure extending the existing Portal API makes sense. Given that remote assistance (well served by the existing API) wants * Explicit user consent to share * To allow the local user to select what to share * To mirror the selected existing displays / windows while remote assistance wants * Persistent access * To ensure the session is curtained or headless to prevent observation and interaction from the local console * Full control over the virtual monitor layout and the ability to capture all of it It seems like a unified API would have enough special cases depending on which access mode is in use that it wouldn't really be worth it, especially given that the actual capture (PipeWire) and input injection (libei) would be the same either way. E.g., even with persistent permission tokens, the existing ScreenCast portal doesn't really fit the remote access use case, and a separate API to control the layout and get the resulting PipeWire streams without user monitor selection likely makes more sense. > I suspect we might need Plasma Login Manager to exist first before > we can achieve this, though. The kind of integration that GDM and > GNOME Shell have allowed them to pull off what they did there, I'm > not sure how to do it without that integration. Ideally, I'd like to have a standard protocol that is both lightweight and flexible enough that it wouldn't require any deep integration between the login manager and desktop environment, but could be implemented by even lightweight login managers.
Re: APIs for persistent remote access and headless/hybrid sessions
> We are definitely interested in working in this area. We have a lot of > the proposed core infrastructure that's listed in that proposed > specification; pipewire, creation of virtual screens, libei access is > coming soon. It's "just" a case of gluing everything together. That's great to hear! > The current portal API is expanding with a concept of tokens to allow > creation of streams without user prompts and access. I was hoping this > would suffice if it was coupled with some mechanism to get these > tokens ahead of time? That should work as long as there is a mechanism to provision the tokens in an automated fashion that grants access to virtual monitor configuration and clipboard in addition to capture and input injection. (One of our use cases is being able to set up CRD during the automated provisioning process for a VM, so the user can connect immediately after provisioning.) Indeed, since our remote assistance flow will be using the Portal APIs in any event, having less divergence would not be unwelcome. (The proposed dedicated API still uses PipeWire and libei, though, so it wouldn't be a big deal either way.) I could imagine a flow that looks like the following: * The remote desktop tool connects to the login manager on the system bus using some TBD API. * The two negotiate an authentication method. (Graphical greeter, username & password, PAM conversation, Kerberos, et cetera.) * If a graphical greeter is negotiated, a token is provided to the remote desktop tool to connect to it. (How? Is an implementation of the needed portal interfaces provided by the greeter on the system DBUS, protected by the token?) * When the user logs in, a shared token is passed to the desktop environment that is being started in or transitioned to headless mode. * The login manager provides the remote desktop tool with the token and some kind of handle to the resulting session. * The remote desktop tool spins up a process running as the target user, which uses the token to connect to the compositor via the Portal APIs on the user bus. * The remote desktop tool hands off the connection from the system process to the new user process. (Note that I'm not 100% on how everything works, so apologies if any aspects of that don't make sense.) That said, there are enough differences between the current Portal API use case and what is wanted by a persistent remote access tool that I can definitely see the argument for a separate API like that proposed by Jonas Ådahl. E.g., having methods to modify the monitory layout in the Portal API that are in practice only used via persistent remote access tools via a token and which users would rarely, if ever, want to grant via the UI to ephemeral remote assistance tools feels weird. Having a separate API that's only available in headless mode would also help persistent remote access tools ensure that the connection wasn't visible from the local workstation. > Does Chrome Remote Desktop provide the server part that runs on the > client computer? Or is it using some network API to connect to talk to > the existing gnome/kde remote-desktop server? Chrome Remote Desktop provides its own server components. It uses WebRTC to establish a direct peer-to-peer connection between the process running on the host machine (the machine being remotely accessed) and a web-based client running in the user's web browser on the client machine.
Re: APIs for persistent remote access and headless/hybrid sessions
Hello, Thanks for reaching out and keeping us in the loop. It's better now than after something has been implemented. We are definitely interested in working in this area. We have a lot of the proposed core infrastructure that's listed in that proposed specification; pipewire, creation of virtual screens, libei access is coming soon. It's "just" a case of gluing everything together. The current portal API is expanding with a concept of tokens to allow creation of streams without user prompts and access. I was hoping this would suffice if it was coupled with some mechanism to get these tokens ahead of time? Does Chrome Remote Desktop provide the server part that runs on the client computer? Or is it using some network API to connect to talk to the existing gnome/kde remote-desktop server? David
Re: APIs for persistent remote access and headless/hybrid sessions
On Tue, Feb 27, 2024 at 2:52 PM Erik Jensen wrote: > > I'm on the Chrome Remote Desktop team at Google, and I am working to > add support for Wayland-based desktop environments. While our initial > focus will be on getting it to work with GNOME, we'd love the > functionally to be based on standardized APIs that can be used across > desktop environments, so I'm reaching out here to see what interest > there might be on the KDE side. > > There's been a bit of discussion[1] on the GNOME Discourse, but to > summarize, while the existing Portal APIs are great for remote > assistance use cases where there is a local user to approve the > session and mirroring the local display(s) makes sense, they are > lacking functionality needed for the persistent remote access use > case, where: > > * Connection should be possible immediately after boot. > * For security, the session should operate in a headless mode while a > remote connection is active and the physical seat should remain > locked. > * Virtual monitor layout should be configurable by the remote desktop > tool to match the client, rather than mirroring the physical layout of > the host. > > GNOME is currently pursuing an approach where the remote access tool > runs as a system service, and connecting presents a login screen where > the user can select a session and log in. Some handoff mechanism is > then used for a process running in the resulting session to take over > the connection. > > The current GNOME implementation is based on unstable, GNOME-specific > APIs. As far as I understand it, there are three main pieces that > would need to be standardized for this approach to work in a > desktop-agnostic fashion: > > * An expanded remote desktop API for the compositor. Unlike the > Portals API, this API would only be exposed to trusted processes and > would not require a prompt for each session. Additionally, it would > provide support for configuring virtual monitor resolutions, layout, > and DPI, as well as support for accessing the clipboard. Jonas Ådahl > has been working on creating a standard spec for this functionality, a > draft of which is available at [2]. > * An API for requesting a new remote display from the greeter, which > the remote desktop tool could then attach to using the above API to > allow the user to log in. This would also provide a method for the > remote access tool to identify the resulting session. > * A protocol by which the greeter can instruct the desktop > environment to launch in headless mode for remote access, or > (eventually) transition into/out of headless mode so the user can > attach to the same session locally or remotely. Desktop Environments > would presumably signal their support for headless / hybrid sesssions > via an entry in their respective .desktop file. > > Of course, there are other approaches one could take, such as > expecting the remote desktop tool to handle PAM authentication and > session creation directly, rather than relying on the system greeter, > or allowing individual users to set up a remote desktop tool as a user > service that can only launch sessions on behalf of that user (which > would require linger and wouldn't work with encrypted home > directories, but would be more similar to how Chrome Remote Desktop > currently works). > > Is such persistent remote access functionality something KDE would be > interested in having? Do folks have thoughts on what they'd like to > see such APIs look like? > > Thanks! > > [1]: https://discourse.gnome.org/t/persistent-remote-desktop-access-api/19415 > [2]: https://gitlab.freedesktop.org/jadahl/xdg-specs/-/merge_requests/1 (Sorry if you have two, I sent with the wrong address initially...) I can't speak for everyone in KDE, but I am certainly interested in seeing this in KDE Plasma. Last year, I held a talk with David Duncan at Flock discussing the concept of "Fedora Cloud KDE"[1]. To that end, I've been playing around with KRdp[2] and other things. It became very apparent that we have some gaps, and I think they align quite closely to what you've observed. What I would like to see is a specification that display managers could implement to support headless/virtual remote desktop. Then we don't have to do weird things with PAM, and more importantly, we can support either protocols that do login directly (RDP) and those that don't (VNC). This would also likely require extending the wayland-sessions desktop file format to offer attributes to declare support for headless login and specific protocols. I will point out that the pre-authorization problem with portals[3] is at this point backend-specific (and I think the KDE portal backend automatically authorizes RemoteDesktop portal for system installed software). What we need is to push for the frontend-side (xdg-desktop-portal) to provide a way to consistently apply pre-authorization policy. It's a major usability problem and will affect future portals too (such as the accessibility portal
APIs for persistent remote access and headless/hybrid sessions
I'm on the Chrome Remote Desktop team at Google, and I am working to add support for Wayland-based desktop environments. While our initial focus will be on getting it to work with GNOME, we'd love the functionally to be based on standardized APIs that can be used across desktop environments, so I'm reaching out here to see what interest there might be on the KDE side. There's been a bit of discussion[1] on the GNOME Discourse, but to summarize, while the existing Portal APIs are great for remote assistance use cases where there is a local user to approve the session and mirroring the local display(s) makes sense, they are lacking functionality needed for the persistent remote access use case, where: * Connection should be possible immediately after boot. * For security, the session should operate in a headless mode while a remote connection is active and the physical seat should remain locked. * Virtual monitor layout should be configurable by the remote desktop tool to match the client, rather than mirroring the physical layout of the host. GNOME is currently pursuing an approach where the remote access tool runs as a system service, and connecting presents a login screen where the user can select a session and log in. Some handoff mechanism is then used for a process running in the resulting session to take over the connection. The current GNOME implementation is based on unstable, GNOME-specific APIs. As far as I understand it, there are three main pieces that would need to be standardized for this approach to work in a desktop-agnostic fashion: * An expanded remote desktop API for the compositor. Unlike the Portals API, this API would only be exposed to trusted processes and would not require a prompt for each session. Additionally, it would provide support for configuring virtual monitor resolutions, layout, and DPI, as well as support for accessing the clipboard. Jonas Ådahl has been working on creating a standard spec for this functionality, a draft of which is available at [2]. * An API for requesting a new remote display from the greeter, which the remote desktop tool could then attach to using the above API to allow the user to log in. This would also provide a method for the remote access tool to identify the resulting session. * A protocol by which the greeter can instruct the desktop environment to launch in headless mode for remote access, or (eventually) transition into/out of headless mode so the user can attach to the same session locally or remotely. Desktop Environments would presumably signal their support for headless / hybrid sesssions via an entry in their respective .desktop file. Of course, there are other approaches one could take, such as expecting the remote desktop tool to handle PAM authentication and session creation directly, rather than relying on the system greeter, or allowing individual users to set up a remote desktop tool as a user service that can only launch sessions on behalf of that user (which would require linger and wouldn't work with encrypted home directories, but would be more similar to how Chrome Remote Desktop currently works). Is such persistent remote access functionality something KDE would be interested in having? Do folks have thoughts on what they'd like to see such APIs look like? Thanks! [1]: https://discourse.gnome.org/t/persistent-remote-desktop-access-api/19415 [2]: https://gitlab.freedesktop.org/jadahl/xdg-specs/-/merge_requests/1
