On 3/5/26 10:51 AM, Stefano Garzarella wrote:
> On Wed, Mar 04, 2026 at 11:00:27PM +0000, Alexander Graf wrote:
>> When no H2G transport is loaded, vsock currently routes all CIDs to the
>> G2H transport (commit 65b422d9b61b ("vsock: forward all packets to the
>> host when no H2G is registered"). Extend that existing behavior: when
>> an H2G transport is loaded but does not claim a given CID, the
>> connection falls back to G2H in the same way.
>>
>> This matters in environments like Nitro Enclaves, where an instance may
>> run nested VMs via vhost-vsock (H2G) while also needing to reach sibling
>> enclaves at higher CIDs through virtio-vsock-pci (G2H). With the old
>> code, any CID > 2 was unconditionally routed to H2G when vhost was
>> loaded, making those enclaves unreachable without setting
>> VMADDR_FLAG_TO_HOST explicitly on every connect.
>>
>> Requiring every application to set VMADDR_FLAG_TO_HOST creates friction:
>> tools like socat, iperf, and others would all need to learn about it.
>> The flag was introduced 6 years ago and I am still not aware of any tool
>> that supports it. Even if there was support, it would be cumbersome to
>> use. The most natural experience is a single CID address space where H2G
>> only wins for CIDs it actually owns, and everything else falls through to
>> G2H, extending the behavior that already exists when H2G is absent.
>>
>> To give user space at least a hint that the kernel applied this logic,
>> automatically set the VMADDR_FLAG_TO_HOST on the remote address so it
>> can determine the path taken via getpeername().
>>
>> Add a per-network namespace sysctl net.vsock.g2h_fallback (default 1).
>> At 0 it forces strict routing: H2G always wins for CID > VMADDR_CID_HOST,
>> or ENODEV if H2G is not loaded.
>>
>> Signed-off-by: Alexander Graf <[email protected]>
>> Tested-by: [email protected]
>>
>> ---
>>
>> v1 -> v2:
>>
>> - Rebase on 7.0, include namespace support
>> - Add net.vsock.g2h_fallback sysctl
>> - Rework description
>> - Set VMADDR_FLAG_TO_HOST automatically
>> - Add VMCI support
>> - Update vsock_assign_transport() comment
>>
>> v2 -> v3:
>>
>> - Use has_remote_cid() on G2H transport to gate the fallback. This is
>> used by VMCI to indicate that it never takes G2H CIDs > 2.
>> - Move g2h_fallback into struct netns_vsock to enable namespaces
>> and fix syzbot warning
>> - Gate the !transport_h2g case on g2h_fallback as well, folding the
>> pre-existing no-H2G fallback into the new logic
>> - Remove has_remote_cid() from VMCI again. Instead implement it in
>> virtio.
>>
>> v3 -> v4:
>>
>> - Fix commit reference format (checkpatch)
>> - vhost: use !!vhost_vsock_get() instead of != NULL (checkpatch)
>> - Add braces around final else branch (checkpatch)
>> - Replace 'vhost' with 'H2G transport' (Stefano)
>> ---
>> Documentation/admin-guide/sysctl/net.rst | 28 +++++++++++++++++++
>> drivers/vhost/vsock.c | 13 +++++++++
>> include/net/af_vsock.h | 9 ++++++
>> include/net/netns/vsock.h | 2 ++
>> net/vmw_vsock/af_vsock.c | 35 ++++++++++++++++++++----
>> net/vmw_vsock/virtio_transport.c | 7 +++++
>> 6 files changed, 89 insertions(+), 5 deletions(-)
>>
>> diff --git a/Documentation/admin-guide/sysctl/net.rst
>> b/Documentation/admin-guide/sysctl/net.rst
>> index 3b2ad61995d4..0724a793798f 100644
>> --- a/Documentation/admin-guide/sysctl/net.rst
>> +++ b/Documentation/admin-guide/sysctl/net.rst
>> @@ -602,3 +602,31 @@ it does not modify the current namespace or any
>> existing children.
>>
>> A namespace with ``ns_mode`` set to ``local`` cannot change
>> ``child_ns_mode`` to ``global`` (returns ``-EPERM``).
>> +
>> +g2h_fallback
>> +------------
>> +
>> +Controls whether connections to CIDs not owned by the host-to-guest (H2G)
>> +transport automatically fall back to the guest-to-host (G2H) transport.
>> +
>> +When enabled, if a connect targets a CID that the H2G transport (e.g.
>> +vhost-vsock) does not serve, or if no H2G transport is loaded at all, the
>> +connection is routed via the G2H transport (e.g. virtio-vsock) instead. This
>> +allows a host running both nested VMs (via vhost-vsock) and sibling VMs
>> +reachable through the hypervisor (e.g. Nitro Enclaves) to address both using
>> +a single CID space, without requiring applications to set
>> +``VMADDR_FLAG_TO_HOST``.
>> +
>> +When the fallback is taken, ``VMADDR_FLAG_TO_HOST`` is automatically set on
>> +the remote address so that userspace can determine the path via
>> +``getpeername()``.
>> +
>> +Note: With this sysctl enabled, user space that attempts to talk to a guest
>> +CID which is not implemented by the H2G transport will create host vsock
>> +traffic. Environments that rely on H2G-only isolation should set it to 0.
>> +
>> +Values:
>> +
>> + - 0 - Connections to CIDs <= 2 or with VMADDR_FLAG_TO_HOST use G2H;
>> + all others use H2G (or fail with ENODEV if H2G is not loaded).
>> + - 1 - Connections to CIDs not owned by H2G fall back to G2H. (default)
>> diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
>> index 054f7a718f50..1d8ec6bed53e 100644
>> --- a/drivers/vhost/vsock.c
>> +++ b/drivers/vhost/vsock.c
>> @@ -91,6 +91,18 @@ static struct vhost_vsock *vhost_vsock_get(u32 guest_cid,
>> struct net *net)
>> return NULL;
>> }
>>
>> +static bool vhost_transport_has_remote_cid(struct vsock_sock *vsk, u32 cid)
>> +{
>> + struct sock *sk = sk_vsock(vsk);
>> + struct net *net = sock_net(sk);
>> + bool found;
>> +
>> + rcu_read_lock();
>> + found = !!vhost_vsock_get(cid, net);
>> + rcu_read_unlock();
>> + return found;
>> +}
>> +
>> static void
>> vhost_transport_do_send_pkt(struct vhost_vsock *vsock,
>> struct vhost_virtqueue *vq)
>> @@ -424,6 +436,7 @@ static struct virtio_transport vhost_transport = {
>> .module = THIS_MODULE,
>>
>> .get_local_cid = vhost_transport_get_local_cid,
>> + .has_remote_cid = vhost_transport_has_remote_cid,
>>
>> .init = virtio_transport_do_socket_init,
>> .destruct = virtio_transport_destruct,
>> diff --git a/include/net/af_vsock.h b/include/net/af_vsock.h
>> index 533d8e75f7bb..4e40063adab4 100644
>> --- a/include/net/af_vsock.h
>> +++ b/include/net/af_vsock.h
>> @@ -179,6 +179,15 @@ struct vsock_transport {
>> /* Addressing. */
>> u32 (*get_local_cid)(void);
>>
>> + /* Check if this transport serves a specific remote CID.
>> + * For H2G transports: return true if the CID belongs to a registered
>> + * guest. If not implemented, all CIDs > VMADDR_CID_HOST go to H2G.
>> + * For G2H transports: return true if the transport can reach arbitrary
>> + * CIDs via the hypervisor (i.e. supports the fallback overlay). VMCI
>> + * does not implement this as it only serves CIDs 0 and 2.
>> + */
>> + bool (*has_remote_cid)(struct vsock_sock *vsk, u32 remote_cid);
>> +
>> /* Read a single skb */
>> int (*read_skb)(struct vsock_sock *, skb_read_actor_t);
>>
>> diff --git a/include/net/netns/vsock.h b/include/net/netns/vsock.h
>> index dc8cbe45f406..7f84aad92f57 100644
>> --- a/include/net/netns/vsock.h
>> +++ b/include/net/netns/vsock.h
>> @@ -20,5 +20,7 @@ struct netns_vsock {
>>
>> /* 0 = unlocked, 1 = locked to global, 2 = locked to local */
>> int child_ns_mode_locked;
>> +
>> + int g2h_fallback;
>> };
>> #endif /* __NET_NET_NAMESPACE_VSOCK_H */
>> diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c
>> index 2f7d94d682cb..50843a977878 100644
>> --- a/net/vmw_vsock/af_vsock.c
>> +++ b/net/vmw_vsock/af_vsock.c
>> @@ -545,9 +545,13 @@ static void vsock_deassign_transport(struct vsock_sock
>> *vsk)
>> * The vsk->remote_addr is used to decide which transport to use:
>> * - remote CID == VMADDR_CID_LOCAL or g2h->local_cid or VMADDR_CID_HOST if
>> * g2h is not loaded, will use local transport;
>> - * - remote CID <= VMADDR_CID_HOST or h2g is not loaded or remote flags
>> field
>> - * includes VMADDR_FLAG_TO_HOST flag value, will use guest->host
>> transport;
>> - * - remote CID > VMADDR_CID_HOST will use host->guest transport;
>> + * - remote CID <= VMADDR_CID_HOST or remote flags field includes
>> + * VMADDR_FLAG_TO_HOST, will use guest->host transport;
>> + * - remote CID > VMADDR_CID_HOST and h2g is loaded and h2g claims that
>> CID,
>> + * will use host->guest transport;
>> + * - h2g not loaded or h2g does not claim that CID and g2h claims the CID
>> via
>> + * has_remote_cid, will use guest->host transport (when g2h_fallback=1)
>> + * - anything else goes to h2g or returns -ENODEV if no h2g is available
>> */
>> int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk)
>> {
>> @@ -581,11 +585,21 @@ int vsock_assign_transport(struct vsock_sock *vsk,
>> struct vsock_sock *psk)
>> case SOCK_SEQPACKET:
>> if (vsock_use_local_transport(remote_cid))
>> new_transport = transport_local;
>> - else if (remote_cid <= VMADDR_CID_HOST || !transport_h2g ||
>> + else if (remote_cid <= VMADDR_CID_HOST ||
>> (remote_flags & VMADDR_FLAG_TO_HOST))
>> new_transport = transport_g2h;
>> - else
>> + else if (transport_h2g &&
>> + (!transport_h2g->has_remote_cid ||
>> + transport_h2g->has_remote_cid(vsk, remote_cid)))
>> + new_transport = transport_h2g;
>> + else if (sock_net(sk)->vsock.g2h_fallback &&
>> + transport_g2h && transport_g2h->has_remote_cid &&
>> + transport_g2h->has_remote_cid(vsk, remote_cid)) {
>> + vsk->remote_addr.svm_flags |= VMADDR_FLAG_TO_HOST;
>> + new_transport = transport_g2h;
>> + } else {
>> new_transport = transport_h2g;
>> + }
>> break;
>> default:
>> ret = -ESOCKTNOSUPPORT;
>> @@ -2879,6 +2893,15 @@ static struct ctl_table vsock_table[] = {
>> .mode = 0644,
>> .proc_handler = vsock_net_child_mode_string
>> },
>> + {
>> + .procname = "g2h_fallback",
>> + .data = &init_net.vsock.g2h_fallback,
>> + .maxlen = sizeof(int),
>> + .mode = 0644,
>> + .proc_handler = proc_dointvec_minmax,
>> + .extra1 = SYSCTL_ZERO,
>> + .extra2 = SYSCTL_ONE,
>> + },
>> };
>>
>> static int __net_init vsock_sysctl_register(struct net *net)
>> @@ -2894,6 +2917,7 @@ static int __net_init vsock_sysctl_register(struct net
>> *net)
>>
>> table[0].data = &net->vsock.mode;
>> table[1].data = &net->vsock.child_ns_mode;
>> + table[2].data = &net->vsock.g2h_fallback;
>> }
>>
>> net->vsock.sysctl_hdr = register_net_sysctl_sz(net, "net/vsock", table,
>> @@ -2928,6 +2952,7 @@ static void vsock_net_init(struct net *net)
>> net->vsock.mode =
>> vsock_net_child_mode(current->nsproxy->net_ns);
>>
>> net->vsock.child_ns_mode = net->vsock.mode;
>> + net->vsock.g2h_fallback = 1;
>
> My last concern is what I mentioned in v3 [1].
> Let me quote it here as well:
>
> @Michael @Paolo @Jakub
> I don't know what the sysctl policy is in general in net or virtio.
> Is this fine or should we inherit this from the parent and set the
> default only for init_ns?
AFAICT, there is no geneal policy; depending on the specific value it
should be inherited or be available to configuration on per netns case.
Usually the inherited values are constraints to system-wide resources,
i.e. max memory allocated.
In this specific case I feel like allowing per netns configuration is
correct.
/P
