On 2/6/26 12:21 PM, Maurice Klein wrote: [snip]
> I think I didn't explain properly about that. > Basically the whole Idea is to have a gateway IP like 192.0.2.1/32 on > the pve host on that bridge and not have a /24 or so route then. Those are just local to the node for routing, the /24 wouldn't get announced - only the /32 routes for the additional IPs. But I guess with that setup you could do without it as well. It shouldn't be an issue to create a 'subnet' as /32 and then give the PVE host the only IP as gateway IP and configure it that way. Layer-2 zones for instance (VLAN, QinQ, VXLAN) don't even need a subnet configured at all - so I don't see a blocker there. > Guests then also have addresses whatever they might look like. > For example a guest could have 1.1.1.1/32 but usually always /32, > although I guess for some use cases it could be beneficial to be able to > have a guest that gets more then a /32 but let's put that aside for now. would be quite interesting for IPv6 actually. > Now there is no need/reason to define which subnet a guest is on and no > need to be in the same with the host. > > The guest would configure it's ip statically inside and it would be > a /32 usually. Yeah, most implementations I've seen usually have a 'cluster IP' that is in RFC 1918 range for local cluster communication, but with containers that works a lot easier since you can control the network configuration of them whereas with VMs you cannot and would need to update the configuration on every move - or use the same subnet across every node instead of having a dedicated subnet per node/rack. [snip] > Now the biggest thing this enables us to do is in pve clusters if we > build for example a ibgp full mesh the routes get shared. > There could be any topology now and routing would adapt. > just as an example while that is a shity topology it can illustrate the > point.: > > GW-1 GW-2 > | \ / | > | \ / | > | \ / | > pve1--pve3 > \ / > \ / > pve2 > > Any pve can fail and there would still be everything reachable. > Always the shortest path will be chosen. > Any link can Fail. > Any Gateway can Fail. > Even multiple links failing is ok. > No chance for loops because every link is p2p. > Much like at the full mesh ceph setup with ospf or openfabric. > > That can be archived with evpn/vxlan and anycast gateway and multiple > exit nodes. > Problem is the complexity and by giving bigger routes then /24 to > gateways they will not always use the optimal path thus increasing > latancy and putting unnesisary routing load on hosts where the vm isn't > living right now. > And all that to have one L2 domain which often brings more disadvantages > then advantages. > > I hope I explained it well now, if not feel free to ask anything, I > could also provide some bigger documentation with screenshots of > everything. Yes that makes sense. The way I described it in my previous mail should be like that, since it decouples the IP configuration + route creation (which would then be handled by the zone / vnet) from the announcement of that route (which would be handled by fabrics). As a start we could just utilize the default routing table. I'm planning on adding VRF + Route Redistribution + Route Map support mid-term, so the new zone could then profit from those without having to implement anything of the sort for now. It's a bit of an awkward timing, since I'm still working on implementing several features that this plugin would benefit quite heavily from and I don't want to do any duplicate work / code ourselves into a corner by basically implementing all that functionality but only specific to that plugin and then having to migrate everything over while maintaining backwards compatibility. [snip] > I also feel like it would make sense in the network device, since it is > part of specific configuration for that vm but I get why you are > reluctant to that. > This honestly makes me reconsider the sdn approach a little bit. > I have an Idea here that could be something workable. > What if we add a field not saying guest ip, what if we instead call id > routes. > Essentially that is what it is and might have extra use cases apart from > what I'm trying to archive. > That way for this use case you can use those fields to add the > needed /32 host routes. > It wouldn't be specific to the sdn feature we build. > The SDN feature could then be more about configuring the bridge with the > right addresses and fetures and enable us to later distribute the routes > via bgp and other ways. > I looked into the hotplug scenarios as well and that way those would be > solved. Yeah, I think VM configuration is the best bet. It should be tied to the network device imo, so I guess adding a property that allows configuring a CIDR there should be fine for starting out. Adding the route is handled by the respective tap_plug / veth_create functions in pve-network and the new zone plugin then. [snip]
