Hi, just to update you, I am using the TM example app (which I indeed works fine, doing traffic prioritization) to find the exact setting missing in my config. I will share my findings...
-- Oriol Arcas Software Engineer Starflow Networks On Wed, May 3, 2017 at 3:35 PM, Bala Manoharan <[email protected]> wrote: > On 3 May 2017 at 18:29, Oriol Arcas <[email protected]> wrote: > > Hi, I use a simple: > > > > int res = odp_tm_enq(q, pkt); > > > > Our app chooses the queue to which enqueue the packets. Mostly it's C++ > > wrappers around ODP objects. > > > > The packets have different sizes, but they belong to the same pool. > > > > Does this answer your questions? > > Can you set rate limitation (CIR) on the node to limit the rate on the > pktio interface and then run the test. > We have a suspicion that the weights are not being effected since > there will not be any packet on the backlog due to the speed of > transmission. > > Regards, > Bala > > > > -- > > Oriol Arcas > > Software Engineer > > Starflow Networks > > > > On Tue, May 2, 2017 at 5:27 PM, Bala Manoharan < > [email protected]> > > wrote: > >> > >> Can you please share the code as to how are you enqueing packets to > >> odp_tm_queue() are you creating packets of same size? > >> I would like to know how these packets are enqueued to these 3 queues? > >> > >> Regards, > >> Bala > >> > >> > >> On 2 May 2017 at 18:50, Oriol Arcas <[email protected]> wrote: > >> > Bill, Barry, Bala, > >> > > >> > I've been thinking about this problem, and maybe it's caused by the > >> > mismatch > >> > in throughput between the PKTIOs and the CPU for high speed BW (eg, > >> > localhost connections). That is, both the PKTIN and the PKTOUT are > >> > faster > >> > than the TM thread. This would lead to a situation where the > >> > input_work_queue gets full and tm_enqueue() rejects packets, while the > >> > PKTOUT never gets full and the TM thread doesn't need to enqueue > >> > packets. > >> > > >> > So probably with a more realistic testbed, where the TM thread is > faster > >> > than the PKTIOs, it should work as expected. > >> > > >> > Anyway, the list of config calls is similar to this: > >> > > >> > odp_tm_requirements_init(&tm_req); > >> > tm_req.num_levels = 1; > >> > tm_req.max_tm_queues = 4; > >> > odp_tm_egress_init(&egress); > >> > egress.egress_kind = ODP_TM_EGRESS_PKT_IO; > >> > egress.pktio = pktio; > >> > tm = odp_tm_create("TM-0", &tm_req, &egress); > >> > > >> > odp_tm_node_params_init(&node_params); > >> > node_params.level = 1; > >> > node_params.max_fanin = 4; > >> > node = odp_tm_node_create(tm, "TM-NODE-0", &node_params) > >> > > >> > odp_tm_threshold_params_init(&th_params); > >> > th_params.max_pkts = 1000; > >> > th_params.enable_max_pkts = 1; > >> > th = odp_tm_threshold_create("TM-TH-0", &th_params); > >> > > >> > /* Queue 1, prio 0, weight 0.3 */ > >> > odp_tm_queue_params_init(&queue_params); > >> > queue_params.priority = 0; > >> > queue_params.threshold_profile = th; > >> > q = odp_tm_queue_create(tm, &queue_params); > >> > odp_tm_queue_connect(q, node) > >> > odp_tm_sched_params_init(&sch_params); > >> > for (i = 0; i < ODP_TM_MAX_PRIORITIES; ++i) { > >> > sch_params.sched_modes[i] = mode; > >> > sch_params.sched_weights[i] = 0.3 * 254 + 1; > >> > } > >> > sch = odp_tm_sched_create("TM-SCH-0", &sch_params); > >> > odp_tm_node_sched_config(node, q, sch); > >> > > >> > /* Queue 2, prio 0, weight 0.7 */ > >> > odp_tm_queue_params_init(&queue_params); > >> > queue_params.priority = 0; > >> > queue_params.threshold_profile = th; > >> > q = odp_tm_queue_create(tm, &queue_params); > >> > odp_tm_queue_connect(q, node) > >> > odp_tm_sched_params_init(&sch_params); > >> > for (i = 0; i < ODP_TM_MAX_PRIORITIES; ++i) { > >> > sch_params.sched_modes[i] = mode; > >> > sch_params.sched_weights[i] = 0.7 * 254 + 1; > >> > } > >> > sch = odp_tm_sched_create("TM-SCH-1", &sch_params); > >> > odp_tm_node_sched_config(node, q, sch); > >> > > >> > /* Queue 3, prio 1, weight 1 (unused) */ > >> > odp_tm_queue_params_init(&queue_params); > >> > queue_params.priority = 1; > >> > queue_params.threshold_profile = th; > >> > q = odp_tm_queue_create(tm, &queue_params); > >> > odp_tm_queue_connect(q, node) > >> > odp_tm_sched_params_init(&sch_params); > >> > for (i = 0; i < ODP_TM_MAX_PRIORITIES; ++i) { > >> > sch_params.sched_modes[i] = mode; > >> > sch_params.sched_weights[i] = 255; > >> > } > >> > sch = odp_tm_sched_create("TM-SCH-2", &sch_params); > >> > odp_tm_node_sched_config(node, q, sch); > >> > > >> > /* Queue 4, prio 2, weight 1 */ > >> > odp_tm_queue_params_init(&queue_params); > >> > queue_params.priority = 2; > >> > queue_params.threshold_profile = th; > >> > q = odp_tm_queue_create(tm, &queue_params); > >> > odp_tm_queue_connect(q, node) > >> > odp_tm_sched_params_init(&sch_params); > >> > for (i = 0; i < ODP_TM_MAX_PRIORITIES; ++i) { > >> > sch_params.sched_modes[i] = mode; > >> > sch_params.sched_weights[i] = 255; > >> > } > >> > sch = odp_tm_sched_create("TM-SCH-3", &sch_params); > >> > odp_tm_node_sched_config(node, q, sch); > >> > > >> > -- > >> > Oriol Arcas > >> > Software Engineer > >> > Starflow Networks > >> > > >> > On Mon, May 1, 2017 at 8:33 PM, Bill Fischofer > >> > <[email protected]> > >> > wrote: > >> >> > >> >> +cc Barry, who is the expert in this area, for his thoughts. Do you > >> >> have > >> >> the complete list of TM config calls you issue to set this up? I know > >> >> Bala > >> >> (+cc) was looking to propose a "streamlined" config API for TM since > >> >> this is > >> >> can be tricky as there are a number of "knobs" the TM provides to > >> >> control > >> >> these things. > >> >> > >> >> You are correct that the TM only really gets involved when output > >> >> classes > >> >> become over-subscribed as its job is to throttle output based either > on > >> >> physical or configured transmission limits. So without seeing the > >> >> complete > >> >> config it's difficult to assess what might be going on here. > >> >> > >> >> On Fri, Apr 28, 2017 at 8:57 AM, Oriol Arcas > >> >> <[email protected]> > >> >> wrote: > >> >>> > >> >>> Hello, > >> >>> > >> >>> I'm trying to use the Traffic Manager to implement priorities and > >> >>> weights. > >> >>> But it is not applying these policies. > >> >>> > >> >>> I have observed from the traces that the TM is not queuing the > >> >>> packets, > >> >>> but > >> >>> directly ouputing them. For instance, see this final stats from the > >> >>> TM: > >> >>> > >> >>> odp_tm_stats_print - tm_system=0x56347C6E6790 tm_idx=0 > >> >>> input_work_queue size=256 current cnt=0 peak cnt=256 > >> >>> input_work_queue enqueues=1225767 dequeues=1225767 fail_cnt=1034 > >> >>> green_cnt=0 yellow_cnt=0 red_cnt=0 > >> >>> * queue_num=1 priority=0 rcvd=408650 enqueued=0 dequeued=0 > >> >>> consumed=408650 > >> >>> * queue_num=2 priority=0 rcvd=409041 enqueued=0 dequeued=0 > >> >>> consumed=409041 > >> >>> * queue_num=4 priority=2 rcvd=408076 enqueued=0 dequeued=0 > >> >>> consumed=408076 > >> >>> > >> >>> I have two priorities (0 and 2) and 3 queues. Two queues go to > >> >>> priority > >> >>> 0, > >> >>> with 30% and 70% weights. I transmit 1GB through each queue with > TCP. > >> >>> There > >> >>> is no prioritization, nor weighted scheduling. > >> >>> > >> >>> My understanding is that if the TM doesn't enqueue the packets in > its > >> >>> internal queues (enqueued and dequeued stats are 0), it doesn't > apply > >> >>> any > >> >>> of the algorithms (priority encoder, WFQ, etc.). > >> >>> > >> >>> I am using taps in my laptop, not a real network, so the BW is quite > >> >>> high > >> >>> (~3 Gbps). Could it be that if the network can absorb the BW > processed > >> >>> by > >> >>> the CPU, then the PKTOUT doesn't get congested and no queing is > done, > >> >>> so > >> >>> no > >> >>> TM either. > >> >>> > >> >>> However, to reject this hypothesis, I reduced the main input queue > to > >> >>> 256 > >> >>> packets. Now, it gets full, because the TM rejects 1034 packets. So > it > >> >>> still doesn't apply queuing (thus priorities and weights), but it is > >> >>> clearly congested because it rejects packets. > >> >>> > >> >>> Any thoughts on this? > >> >>> > >> >>> Thank you in advance. > >> >>> > >> >>> -- > >> >>> Oriol Arcas > >> >>> Software Engineer > >> >>> Starflow Networks > >> >> > >> >> > >> > > > > > >
