Hi Feng, My main question is - Do you think it¹s feasible to use Cumulus MI patch and 6WIND VRF patch to see a somewhat generic N = M x P model? (If not complete for all combinations right away, at least making way in the future to be completed)
More comments inline.. On 12/15/14, 12:41 AM, "Feng Lu" <[email protected]> wrote: >On 12/13/2014 04:22 AM, Vipin Kumar wrote: >> >> While we all are agreeing (mostly) that the ultimate solution to scale >> better and get the most out of the system is to provide N-in-1 and >> 1-in-1 and also both together. So, lets see how the desired model may >> look like if we also want to utilize the submitted patches to the best >> of their capabilities :). >> >> if user wants.. >> >> N = M x P >> N = total number of VRFs >> P = total number of processes >> M = total number of (VRF)instances per processes. >> >> (M could vary per process too) >Hi Vipin, > >Please allow me to show it in another way: > >zebra + > >Process A >VRF Aa >Instance Aa1 >Instance Aa2 >Instance ... >VRF Ab >Instance Ab1 >Instance Ab2 >Instance ... >VRF ... >Process B >VRF ... >Instance ... This is somewhat similar to how I depicted earlier with the additions of multiple instances within one VRF, which I assume is pending or future work planned by 6WIND, correct? > >All instances in the same VRF serve the same forwarding table (their >routes are comparable, by distance). > >The instances in different VRFs serve the different forwarding tables. >Their routes are incomparable. > >And using multiple processes here, is to avoid too busy in one process >since it is handling lots of instances. > >Cumulus patch provides an instance per process, while multiple >processes serve the same non-VRF table. > >6WIND patch provides an instance per VRF, while multiple VRFs are >handled in a single process. > >The multi-VRF patch provided by 6WIND is just the first step. The >next step is to let multiple instances work in a same VRF (not >implemented yet). We may add multi-daemon support if the single >process did not have enough performance to meet users needs. All understood. But do you see a possibility of using Cumulus MI patch to create multiple processes, and 6WIND patch to create VRFs within them? > >> If we keep things simple, and continue with single zebra daemon >> approach (submitted by 6WIND) >As far as I see, only one zebra process seems enough. I agree. > >> and see how protocols (OSPF/BGP/etc.) can be made to work in the above >> model. >Yes, this is what we want to achieve. > >> >> Multi-instance OSPF (submitted by Cumulus) can be used to create >> multiple protocol daemons. >> (I am not sure if 6WIND or someone else has a working model of >> creating processes based on the config read by vtysh, didn¹t explore >> this). >I did not see in Cumulus patch that multiple OSPF daemons can be started >by vtysh. Had I lost? I'll check again. No, Cumulus patch doesn¹t do it :) Actually I was asking if 6WIND did this by adding functionality into vtysh. > >> >> Few relevant things to note about the Cumulus patch: >> >> - It already registers with zebra with process-instance-identifier >> (coming from 'router ospf <id>¹) and by default reads/writes to the >> current non-VRF table. >> - Once daemons are up, then all configuration/debug/show commands are >> from the vtysh just like before. >Yes, this is in principle. > >6WIND patch appends VRF ID to the commands, while the default behavior >(non-VRF case) is not changed. I assume once we think of this N = MxP model, we will have to find ways in adding a keyword before the VRF ID. So the process-instance ID can be given without conflicting with each others. > >> - There is one TO-DO, like some registering with the IP stack so one >> process receives packets only coming on interfaces its interested in. >I see the approach of Cumulus patch on this point. It disables the >"network PREFIX area ID" command in case of multiple instances. > >While I think, this is in fact a management issue. We are not able >to avoid the mis-configurations. It should be the responsibility >of the administrator to ensure the correctness in the configurations. Right, Cumulus patch simplified it by no allowing network command in the instanced bring-up. So proper mutual exclusion of interfaces among instances can be offered. While we can leave it up to the user, this when misconfigured can lead to serious adjacency issues, like a mis-config can affect the existing adjacency, hence we thought just compromising on network config was well worth it. > >> Once inside vtsyh, any config is allowed for those daemons. So, at >> that point, VRFs can be configured within these OSPF daemons and map >> to different tables for route-updates/redistribution purpose. >> >> | OSPFd1 < vrf1, vrf2, .. >> | >> Zebra (VRF LIB) --| OSPFd2 < vrfs >> | >> | OSPFd3 < vrfs >> | >> >> >> From the VRFs requirement point of view, the key difference from the >> MI OSPF is that for VRFs, OSPF (instances/processes) should be writing >> to different tables in zebra, I assume 6WIND design/patches take care >> of that. >Yes. > >> >> So, as far as I see, with ospf process-id specification, writing to >> the same non-vrf table, and VRFs mapping to different VRF tables, >> should cover what we are trying to achieve. >> For example: >> - if someone wants separate process for each VRF, they just configure >> one VRF per process. >> - if someone wants all VRFs per process, that¹s what they can configure. >> - and a mix can be imagined too. >> - Multi-instance OSPF when users wants separate process for each >> instance would work as is >Yes. The configuration of N-in-1 + 1-in-1 mode would be interesting. > >> >> Except.. >> >> .. if we want to create multiple OSPF (non-vrf) instances within one >> process. >> >> Was 6WIND planning to create generic separations within OSPF, and then >> give flexibility to have a way to map those to either the same non-VRF >> table or separate VRF tables ? >Yes as I explained at the beginning, if our customers need (presently >still in one daemon). > >> >> I think if we have this last one also taken care of, then we can head >> towards a very generic model. And all we need to provide are >> initialization/configuration methods. >> >> Regards, >> Vipin >> PS: I tried to read all the comments, so I don¹t repeat whats already >> been discussed/discarded, still its possible there is some redundancy >> to my text. >> Also, I agree in general that individual processes for each VRFs will >> have memory overhead, but I think 90M is probably too much, may be >> some more insight is needed, it might be coming from static >> allocations, which can be improved to make it need based. >> >As I explained in my original email, I was using 6WIND version quagga. >Zebra + ospfd totally occupied about 90M, each using 45M. > >While on the VM (ubuntu image), I found each daemon of quagga-0.99.23.1 >needs about 23M. The value is half of that of 6WIND version. > >But I think this is not the point. My point is the memory overhead. >I believe the memory consumption will increase when: >- more and more routes injected; >- more and more peers formed; >- more and more featured supported; >- more memory allocation made by users in their own extensions. Memory used/needed based on the above points should be the same in any model. The extra overhead because of the 1-in-1 model is whatever is used by a process as it comes up without a single vrf or adjacency. And that¹s where I think it could be improved. So at least may be up to (lets say 25 or so) OSPF daemons can still be conceivable. > >Hoo, seems I am repeating something, sorry :) >Please let me know if I'm incorrect. > >Thanks and best regards, >Feng Lu _______________________________________________ Quagga-dev mailing list [email protected] https://lists.quagga.net/mailman/listinfo/quagga-dev
