Am Mon, Nov 16, 2026 at 12:46:18PM +0100 schrieb Dumitru Ceara:
> Hi everyone,

Hi Dumitru,

thanks for the summary here.

> 
> First of all, thank you to those who could attend the conference and the
> "ovn-local: A new proposed architecture for OVN" discussion panel:
> 
> https://www.openvswitch.org/support/ovscon2026/#t26
> 
> Some references, also for those who couldn't make it live (or online):
> 
> - Slides presented at the conference:
> https://drive.google.com/file/d/1TBDrqoYk9EHseqSaif6xKt4ePYncSocb
> 
> - Recording of Mark's presentation (shared at the conference):
> https://drive.google.com/file/d/1mwLmZQmUpZMZ3T-DMtkvo6xQaq867Ebc
> 
> - A snapshot of the document that lead after discussion between Red Hat
> internal OVN team members to the proposal we shared at the conference:
> https://drive.google.com/file/d/1w7IwB6UX-KbzUChBZb9yGi42yPIP7NZH/
> 
> As usual, it takes some time after the conference until we have
> everything ready to share publicly so we don't (yet) have the recording
> of the actual panel.  But I'll try to summarize as best as I can (at the
> end of this email) what was discussed during the actual session but also
> during the ad-hoc/hallway discussions after the official session.
> 
> I think at this point quite a few members of the community have
> expressed interest in the initiative and it might be a good idea to
> start planning some of the work that might be required for this.  As
> you'll see below, I don't think there were any unsolvable technical
> issues but there will definitely be challenges.
> 
> In order to get things going, I'd like to ask folks to think about
> timelines regarding this new initiative.
> 
> I personally am quite confident that we can try to deliver some of the
> changes suggested in the ovn-local document (e.g., phase 1) as early as
> the next development cycle (27.09) and that there's a high chance to
> implement the migration to the new architecture before our next LTS
> release (28.03).  But that's my personal opinion, I'd like to hear
> others too. :)
> 
> Please read further for the notes I took during the conference.
> 
> Best regards,
> Dumitru
> ---
> 
> My notes from the panel and follow up discussion:
> -------------------------------------------------
> DISCLAIMER: I wrote this down after the fact so there might be inaccuracies.
> 
> The main take away from the discussion was that OVN users in the
> audience were interested in moving towards such an architecture.  It’s
> also in-line with other talks that were presented at the conference
> (e.g., Crusoe is moving towards a custom ovn-ic deployment similar to
> the OpenShift/ovn-kubernetes one - essentially one OVN per node, Stackit
> has issues due to the large size of the SB database in their classic
> centralized deployment).
> 
> There were a few questions/remarks from the audience and afterwards
> during the ad-hoc discussion.  I'll list everything (including
> answers/comments) here:
> 
> 1. How can we deal with the "synchronization work that ovn-northd
> currently does, e.g., allocating SB datapath and port_binding tunnel keys?

This also includes allocating ip and mac addresses.

> 
> There were some options mentioned during the session:
> a. fully distributed and predictable/consistent generation of the keys
> by individual ovn-local instances.  This has IMO the risk of
> complicating upgrades if we ever decide to change the algorithm used to
> generate keys.
> 
> b. adding support to ovsdb to "run stored procedures" and move the key
> allocation logic there
> 
> c. mimic what ovn-kubernetes does and have a "cluster manager", a
> centralized component, essentially a way simpler ovn-northd that still
> manages keys even for ovn-local deployments

I think this is the only option that actually will work well for ip and
mac addresses. Since they need more logic than just having a unique
number in some way.
In addition that allows users where the CMS already does all the
required work to skip this component.

> 
> 2. Would moving to ovn-local shift the performance issues from the
> central nodes to the chassis?  That’s because ovn-local would now
> perform the work that ovn-northd does in the current deployment
> (centralized).
> 
> I think we can't provide a definitive answer before experimenting with
> the proposed new architecture but we expect that in the final ovn-local
> deployment only locally relevant logical switches/routers would be
> processed (reducing compute time to values likely similar to what
> ovn-controller has to process today); the large SB contents would be
> gone, reducing time to process those in the IDL (json handling).

I think this is directly related to how efficientlly the new ovn-local
can find out what NB contents are relevant for it. The less it needs to
filter the better we are probably.

However we need to also account for the released intermediate steps. I
see there especially the part concerning that would write the SB content
to NB as well. I think that is one of the steps where we need to find
out if we would even want to release that as the only option. Or if this
is a development path we need to take, but that no user would actually
need to see.

> 
> 3. Would moving to ovn-local cause upgrade incompatibility?  I.e., in
> very large clusters upgrades can take days/months.
> 
> This has two aspects:
> a. the first upgrade, from centralized OVN to ovn-local
> b. subsequent upgrades, from ovn-local version x to ovn-local version x+1
> 
> "a" was discussed separately quite extensively during the ad-hoc session
> after the conference, I'll detail that later.
> 
> for "b”, taking the ovn-kubernetes example, we probably have a chance of
> incompatibility even with today’s architecture.  That’s because whenever
> a packet needs to be tunneled to a remote chassis (i.e., it has been
> received on a logical port on chassis-1 and needs to be sent out on a
> logical port on chassis-2) we are splitting that datapath’s logical
> pipeline across two nodes: ingress pipeline is executed on chassis-1
> while egress pipeline of the same switch/router is executed on chassis-2.
> 
> So, if a change happens that involves, for example, moving stages from
> one pipeline (ingress or egress) to the other then these may cause
> inconsistent packet forwarding.
> 
> In general, this is handled today through feature flags (each chassis
> advertises its "supported” feature set and ovn-northd only generates
> logical flows that are compatible with the "oldest” ovn-controller).
> Moving to ovn-local, we’d have to be even more careful with these kinds
> of situations but, IMO, nothing new here.

That might be something we can explicitly build tests for. So that we
can validate the forwarding for these half-upgraded states.

> 
> 4. What about the shared state that’s currently stored in the Southbound
> (pushed up by ovn-controllers)?  E.g., mac-binding, fdb, learned routes.
> 
> That’s still needed in the ovn-local mode; either stored as new tables
> in the (new) NB database or as a "Status/Operational/etc” database.

I would not see a benefit of having it in the same database. People can
still just run one ovsdb cluster and run both DBs on that. That would
give everyone more flexibility and we would not loose anything.

But for me the biggest question there would probably be the Chassis (and
related) tables. Since the CMS needs to build e.g. HA_Chassis_Groups and
similar things for them, maybe they make more sense in NB than in the
other db.

> 
> 5. Numan suggested reviving his old RFC work that moved logical flow
> generation to ovn-controller or at least first focusing on removing
> Logical_Flow from the "new” ovn-local database.
> 
> https://mail.openvswitch.org/pipermail/ovs-dev/2021-June/384488.html
> 
> I think the ovn-local "Phase 2" part of the proposal presented at the
> conference covers that and we could likely start with removing Logical
> Flows but it's my personal opinion that it would require less work
> overall if we first implement "Phase 1" of the proposal ("Delete the
> Southbound database").
> 
> 6. Is one of the problems with the current architecture that we end up
> pushing "too detailed” information to the chassis from the central OVN node?
> 
> Yes, most likely, and that’s inline with other targeted optimizations we
> did in the past, e.g., moving part of the LB flow generation directly to
> ovn-controllers - those did come at a cost of duplicating some NB config
> information (LBs synced to the SB).
> 
> NOTE: The following items were discussed in the ad-hoc session, after
> the official end of the conference:
> 
> 7. The "first" upgrade scenario: part of the cluster using ovn-local and
> part of the cluster (including "old” central nodes) running the legacy
> mode.  Some options were discussed:
> 
> a. Ilya suggested running two sets of databases: the "old" SB for
> chassis that haven’t upgraded yet and the "new"
> Status/Operational/Whatever database ovn-local needs moving forward.
> 
> In his suggestion ovn-northd (upgraded) would be the intermediate actor
> forwarding contents written by the upgraded ovn-locals in the new
> databases to the legacy SB
> 
> b. Another option would be to run a "SB forwarder/proxy" on the upgraded
> nodes: ovn-local would be unaware of its existence, it would only write
> to the new "Status" database; the "SB forwarder" would monitor the
> "Status" and propagate the information to the "legacy SB" that’s still
> running on the central node for the not-yet-upgraded chassis.

Maybe to add an option c:
If the "Status" database only contains tables compatible with the SB
then there would be no need to actually sync anything or so.

> 
> 8. Taking things a step further and adding support for using other
> databases for OVN config/status storage (e.g., etcd).
> 
> Numan argued that we may anyway need a translation layer in the new
> ovn-local during one of the transition phases described in the talk in
> order to abstract out database access (e.g., the OF generation part in
> ovn-controller currently needs logical flows to be parsed in memory); we
> could consider keeping that in place even after ovn-local is the only
> way of deploying OVN; we would repurpose it to provide an abstraction
> for fetching/pushing data to the NB/Status databases.
> 
> 9. From my perspective one of the most important requirement for this
> whole effort is maintaining OVN backwards compatibility from CMS
> perspective.  There might be some small (hopefully) changes the CMS
> would have to put in place for the first upgrade to ovn-local but
> ideally we should avoid needing other changes in the CMS. This might
> also mean:
> 
> a. transparently making ovn-sbctl work with ovn-local deployments
> b. transparently making ovn-trace work with ovn-local deployments

In addition we have Openstack that talks to the southbound to discover
chassis and i think port statuses. So we might to at least clarify what
to do with such behaviour. Potentially there we need a CMS change. 

> 
> 10. It was raised that this might be a good moment to start using a
> different programming language (probably Rust) for implementing the new
> parts that will be required for ovn-local.

Rust is always nice :)

> 
> 11. There was also a small debate whether we should keep logical flows
> or not:
> a. e.g., for debugging (easier to read than OpenFlow)
> b. OTOH this would incur extra computation cost

We could return them in an appctl command and only compute them on
demand. Then we have them as debugging features but they do not spend
time/space on them otherwise.

> 

Thanks for starting this,
Felix
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Reply via email to