That is it, as I understand it Josh. you basically need to turn your switch in to X seperate switches so each nodes nic, is running on a "seperate" network.
if you were to do the same thing physically without any config, with 3 nodes, you would need to have as many seperate switches as you wanted nics in the balance-rr. I understand mikrotik support balance-rr, but tbh I don't even count them as a normal switch manufacturer. Their game is routers.... I don't know any other switches which have support for balance-rr? as for the 3Gbps limit I mentioned earlier with balance-rr (no matter how many nics you have)... I don't know if that was just an issue of the day as cheap 10Gbps came along and the need evaporated for me. I would love to know if anyone has a test setup to try it though. Cheers On Sat, 25 Aug 2018, 00:15 Josh Knight, <[email protected]> wrote: > Just guessing here, if the switch doesn't support rr on its port channels, > then using separate VLANs instead of bundles on the switch is essentially > wiring nodeA to nodeB. That way you don't hit the port channel hashing on > the switch and you keep the rr as-is from A to B. > > I would also try using UDP mode on iperf to see if it's TCP retransmission > that's preventing you from getting closer to 4Gbps. Another useful tool is > maisezahn for traffic generation, though it is more complex to run. > > On Fri, Aug 24, 2018, 4:59 PM Gilberto Nunes <[email protected]> > wrote: > > > I can get 3 gbps. At least 1.3 gbps. > > Don't know why! > > Em 24/08/2018 17:36, "mj" <[email protected]> escreveu: > > > > > Hi Mark, > > > > > > On 08/24/2018 06:20 PM, Mark Adams wrote: > > > > > >> also, balance-rr through a switch requires each nic to be on a > seperate > > >> vlan. You probably need to remove your lacp config also but this > depends > > >> on > > >> switch model and configuration. so safest idea is remove it. > > >> > > > > > > <snip> > > > > > > then I belive your iperf test will return ~3Gbps... i seem to remember > > >> performance doesnt get much better than this but I cant remember why. > > >> > > >> Also can't say if this is a good setup for ceph performance.. > > >> > > > > > > This is really interesting info, i did not know this. Someone has tried > > > this with ceph? Any experiences to share..? > > > > > > Strange that performence turns out to be ~3Gbps, instead of the > expected > > > 4... > > > > > > Anyone with more information on this subject? > > > > > > Have a nice weekend all! > > > > > > MJ > > > _______________________________________________ > > > pve-user mailing list > > > [email protected] > > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > _______________________________________________ > > pve-user mailing list > > [email protected] > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > _______________________________________________ > pve-user mailing list > [email protected] > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > _______________________________________________ pve-user mailing list [email protected] https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
