On 10/18/23 17:14, Vladislav Odintsov wrote: > Hi Ilya, Terry, > >> On 7 Mar 2023, at 14:03, Ilya Maximets <i.maxim...@ovn.org> wrote: >> >> On 3/7/23 00:15, Vladislav Odintsov wrote: >>> Hi Ilya, >>> >>> I’m wondering whether there are possible configuration parameters for ovsdb >>> relay -> main ovsdb server inactivity probe timer. >>> My cluster experiencing issues where relay disconnects from main cluster >>> due to 5 sec. inactivity probe timeout. >>> Main cluster has quite big database and a bunch of daemons, which connects >>> to it and it makes difficult to maintain connections in time. >>> >>> For ovsdb relay as a remote I use in-db configuration (to provide >>> inactivity probe and rbac configuration for ovn-controllers). >>> For ovsdb-server, which serves SB, I just set --remote=pssl:<port>. >>> >>> I’d like to configure remote for ovsdb cluster via DB to set inactivity >>> probe setting, but I’m not sure about the correct way for that. >>> >>> For now I see only two options: >>> 1. Setup custom database scheme with connection table, serve it in same SB >>> cluster and specify this connection when start ovsdb sb server. >> >> There is a ovsdb/local-config.ovsschema shipped with OVS that can be >> used for that purpose. But you'll need to craft transactions for it >> manually with ovsdb-client. >> >> There is a control tool prepared by Terry: >> >> https://patchwork.ozlabs.org/project/openvswitch/patch/20220713030250.2634491-1-twil...@redhat.com/ >> >> But it's not in the repo yet (I need to get back to reviews on that >> topic at some point). The tool itself should be fine, but maybe name >> will change. > > I want to step back to this thread. > The mentioned patch is archived with "Changes Requested" state, but there is > no review comments in this patch. > If there is no ongoing work with it, I can take it over to finalise. > For now it needs a small rebase, so I can do it and resend, but before want > to hear your thoughts on this. > > Internally we use this patch to work with Local_Config DB for almost 6 months > and it works fine. > On each OVS update we have to re-apply it and sometimes solve conflicts, so > would be nice to have this patch in upstream.
Hi, I'm currently in the middle of re-working the ovsdb-server configuration for a different approach that will replace command-line and appctl configs with a config file (cmdline and appctls will be preserved for backward compatibility, but there will be a new way of setting things up). This should be much more flexible and user-friendly than working with a local-config database. That should also address most of the concerns raised by Terry regarding usability of local-config (having way too many ways of configuring the same thing mainly, and requirement to use special tools to modify the configuration). I'm planning to post the first version of the change relatively soon. I can Cc you on the patches. Best regards, Ilya Maximets. > >> >>> 2. Setup second connection in ovn sb database to be used for ovsdb cluster >>> and deploy cluster separately from ovsdb relay, because they both start >>> same connections and conflict on ports. (I don’t use docker here, so I need >>> a separate server for that). >> >> That's an easy option available right now, true. If they are deployed >> on different nodes, you may even use the same connection record. >> >>> >>> Anyway, if I configure ovsdb remote for ovsdb cluster with specified >>> inactivity probe (say, to 60k), I guess it’s still not enough to have ovsdb >>> pings every 60 seconds. Inactivity probe must be the same from both ends - >>> right? From the ovsdb relay process. >> >> Inactivity probes don't need to be the same. They are separate for each >> side of a connection and so configured separately. >> >> You can set up inactivity probe for the server side of the connection via >> database. So, server will probe the relay every 60 seconds, but today >> it's not possible to set inactivity probe for the relay-to-server direction. >> So, relay will probe the server every 5 seconds. >> >> The way out from this situation is to allow configuration of relays via >> database as well, e.g. relay:db:Local_Config,Config,relays. This will >> require addition of a new table to the Local_Config database and allowing >> relay config to be parsed from the database in the code. That wasn't >> implemented yet. >> >>> I saw your talk on last ovscon about this topic, and the solution was in >>> progress there. But maybe there were some changes from that time? I’m ready >>> to test it if any. Or, maybe there’s any workaround? >> >> Sorry, we didn't move forward much on that topic since the presentation. >> There are few unanswered questions around local config database. Mainly >> regarding upgrades from cmdline/main db -based configuration to a local >> config -based. But I hope we can figure that out in the current release >> time frame, i.e. before 3.2 release. >> >> There is also this workaround: >> >> https://patchwork.ozlabs.org/project/openvswitch/patch/an2a4qcpihpcfukyt1uomqre.1.1641782536691.hmail.wentao....@easystack.cn/ >> It simply takes the server->relay inactivity probe value and applies it >> to the relay->server connection. But it's not a correct solution, because >> it relies on certain database names. >> >> Out of curiosity, what kind of poll intervals you see on your main server >> setup that triggers inactivity probe failures? Can upgrade to OVS 3.1 >> solve some of these issues? 3.1 should be noticeably faster than 2.17, >> and also parallel compaction introduced in 3.0 removes one of the big >> reasons for large poll intervals. OVN upgrade to 22.09+ or even 23.03 >> should also help with database sizes. >> >> Best regards, Ilya Maximets. > > > Regards, > Vladislav Odintsov > _______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
