OK, I just submitted (i) PMD Perf refactor v10. /Jan > -----Original Message----- > From: Stokes, Ian [mailto:[email protected]] > Sent: Monday, 15 January, 2018 12:37 > To: Kevin Traynor <[email protected]>; Jan Scheurich > <[email protected]>; [email protected]; > [email protected]; O Mahony, Billy <[email protected]>; > [email protected]; [email protected] > Subject: RE: [PATCH v2 1/2] dpif-netdev: Add percentage of pmd/core used by > each rxq. > > > On 01/15/2018 08:26 AM, Jan Scheurich wrote: > > > Hi Kevin, > > > > > > This patch doesn't apply on top of my "Refactor PMD stats and cycle > > counting" v9 (as we discussed when outlining the strategy for merging > > contributions 1&2 and 3a-c) nor does the my series apply easily on top of > > yours. Both cases seem to require non-trivial rebasing. > > > > > > > Hi Jan - not sure if you looked through the code changes needed, but IIUC > > it's just a one line change. > > > > @@ -6051,5 +6051,5 @@ dp_netdev_pmd_try_optimize(struct > > dp_netdev_pmd_thread *pmd, > > 0); > > } > > - curr_tsc = cycles_counter(); > > + curr_tsc = cycles_counter_update(&pmd->perf_stats); > > > > > > > > > Would it be possible for you to rebase your patches on my v9 to make use > > of the refactored cycle counting? I believe that makes more sense than > > refactoring your cycle counting afterwards. > > > > > > > I don't agree that it makes sense to send new patches that don't apply on > > master anymore but do apply after a chain of patches which might be > > merged. Apart from that one line of code, the code is independent of each > > other, so a rebase for either patches should be trivial IMHO. I'm happy to > > help with rebasing any of the patches, if it helps things. > > I'd agree with above, let's keep Kevin's patch separate for them moment, I > can review today and rebase it to DPDK_MERGE when > required, I'd like to see these patches go to DPDK_MERGE in the following > order > > (i) PMD Refactor v10. > (ii)Tx_Time_Batching v11. > (iii) Percentage PMD core (Possibly v3). > > Ian > > > > thanks, > > Kevin. > > > > > BR, Jan > > > > > >> -----Original Message----- > > >> From: Kevin Traynor [mailto:[email protected]] > > >> Sent: Thursday, 11 January, 2018 15:32 > > >> To: [email protected]; Jan Scheurich <[email protected]>; > > >> [email protected]; [email protected]; > > >> [email protected]; [email protected]; [email protected] > > >> Cc: Kevin Traynor <[email protected]> > > >> Subject: [PATCH v2 1/2] dpif-netdev: Add percentage of pmd/core used by > > each rxq. > > >> > > >> It is based on the length of history that is stored about an rxq > > >> (currently 1 min). > > >> > > >> $ ovs-appctl dpif-netdev/pmd-rxq-show pmd thread numa_id 0 core_id 4: > > >> isolated : false > > >> port: dpdkphy1 queue-id: 0 pmd usage: 70 % > > >> port: dpdkvhost0 queue-id: 0 pmd usage: 0 % > > >> pmd thread numa_id 0 core_id 6: > > >> isolated : false > > >> port: dpdkphy0 queue-id: 0 pmd usage: 64 % > > >> port: dpdkvhost1 queue-id: 0 pmd usage: 0 % > > >> > > >> These values are what would be used as part of rxq to pmd assignment > > >> due to a reconfiguration event e.g. adding pmds, adding rxqs or with > > >> the command: > > >> > > >> ovs-appctl dpif-netdev/pmd-rxq-rebalance > > >> > > >> Signed-off-by: Jan Scheurich <[email protected]> > > >> Co-authored-by: Jan Scheurich <[email protected]> > > >> Signed-off-by: Kevin Traynor <[email protected]> > > >> --- > > >> > > >> V2: > > >> - Re-worked to calculate and store cycles needed for stats directly in > > pmd. > > >> > > >> Documentation/howto/dpdk.rst | 11 +++++++++ > > >> NEWS | 1 + > > >> lib/dpif-netdev.c | 53 +++++++++++++++++++++++++++++++++--- > > -------- > > >> tests/pmd.at | 51 +++++++++++++++++++++++++++++++----- > > ------ > > >> 4 files changed, 90 insertions(+), 26 deletions(-) > > >> > > >> diff --git a/Documentation/howto/dpdk.rst > > >> b/Documentation/howto/dpdk.rst index 587aaed..40f9d96 100644 > > >> --- a/Documentation/howto/dpdk.rst > > >> +++ b/Documentation/howto/dpdk.rst > > >> @@ -140,4 +140,15 @@ Core 7: Q4 (70%) | Q5 (10%) core 8: Q3 (60%) | > > >> Q0 (30%) > > >> > > >> +To see the current measured usage history of pmd core cycles for each > > rxq:: > > >> + > > >> + $ ovs-appctl dpif-netdev/pmd-rxq-show > > >> + > > >> +.. note:: > > >> + > > >> + A history of one minute is recorded and shown for each rxq to > > >> + allow for traffic pattern spikes. An rxq's pmd core cycles usage > > >> + changes due to traffic pattern or reconfig changes will take one > > >> + minute before they are fully reflected in the stats. > > >> + > > >> Rxq to pmds assignment takes place whenever there are configuration > > >> changes or can be triggered by using:: > > >> diff --git a/NEWS b/NEWS > > >> index aff0671..088dca3 100644 > > >> --- a/NEWS > > >> +++ b/NEWS > > >> @@ -41,4 +41,5 @@ Post-v2.8.0 > > >> - ovs-ofctl dump-ports command now prints new of set custom > > statistics > > >> if available (for OpenFlow 1.4+). > > >> + * Add rxq utilization of pmd to appctl 'dpif-netdev/pmd-rxq- > > show'. > > >> - vswitchd: > > >> * Datapath IDs may now be specified as 0x1 (etc.) instead of 16 > > digits. > > >> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index > > >> 57f6570..d960a3a 100644 > > >> --- a/lib/dpif-netdev.c > > >> +++ b/lib/dpif-netdev.c > > >> @@ -602,4 +602,9 @@ struct dp_netdev_pmd_thread { > > >> struct dp_netdev_pmd_cycles cycles; > > >> > > >> + /* Last interval timestamp. */ > > >> + uint64_t intrvl_tsc_prev; > > >> + /* Last interval cycles. */ > > >> + atomic_ullong intrvl_cycles; > > >> + > > >> /* Current context of the PMD thread. */ > > >> struct dp_netdev_pmd_thread_ctx ctx; @@ -996,7 +1001,7 @@ > > >> pmd_info_show_rxq(struct ds *reply, struct dp_netdev_pmd_thread *pmd) > > >> { > > >> if (pmd->core_id != NON_PMD_CORE_ID) { > > >> - const char *prev_name = NULL; > > >> struct rxq_poll *list; > > >> - size_t i, n; > > >> + size_t n_rxq; > > >> + uint64_t total_cycles = 0; > > >> > > >> ds_put_format(reply, > > >> @@ -1006,20 +1011,32 @@ pmd_info_show_rxq(struct ds *reply, struct > > >> dp_netdev_pmd_thread *pmd) > > >> > > >> ovs_mutex_lock(&pmd->port_mutex); > > >> - sorted_poll_list(pmd, &list, &n); > > >> - for (i = 0; i < n; i++) { > > >> - const char *name = netdev_rxq_get_name(list[i].rxq->rx); > > >> + sorted_poll_list(pmd, &list, &n_rxq); > > >> > > >> - if (!prev_name || strcmp(name, prev_name)) { > > >> - if (prev_name) { > > >> - ds_put_cstr(reply, "\n"); > > >> - } > > >> - ds_put_format(reply, "\tport: %s\tqueue-id:", name); > > >> + /* Get the total pmd cycles for an interval. */ > > >> + atomic_read_relaxed(&pmd->intrvl_cycles, &total_cycles); > > >> + /* Estimate the cycles to cover all intervals. */ > > >> + total_cycles *= PMD_RXQ_INTERVAL_MAX; > > >> + > > >> + for (int i = 0; i < n_rxq; i++) { > > >> + struct dp_netdev_rxq *rxq = list[i].rxq; > > >> + const char *name = netdev_rxq_get_name(rxq->rx); > > >> + uint64_t proc_cycles = 0; > > >> + > > >> + for (int j = 0; j < PMD_RXQ_INTERVAL_MAX; j++) { > > >> + proc_cycles += dp_netdev_rxq_get_intrvl_cycles(rxq, > > >> + j); > > >> } > > >> - ds_put_format(reply, " %d", > > >> + ds_put_format(reply, "\tport: %16s\tqueue-id: %2d", > > >> + name, > > >> netdev_rxq_get_queue_id(list[i].rxq->rx)); > > >> - prev_name = name; > > >> + ds_put_format(reply, "\tpmd usage: "); > > >> + if (total_cycles) { > > >> + ds_put_format(reply, "%2"PRIu64"", > > >> + proc_cycles * 100 / total_cycles); > > >> + ds_put_cstr(reply, " %"); > > >> + } else { > > >> + ds_put_format(reply, "%s", "NOT AVAIL"); > > >> + } > > >> + ds_put_cstr(reply, "\n"); > > >> } > > >> ovs_mutex_unlock(&pmd->port_mutex); > > >> - ds_put_cstr(reply, "\n"); > > >> free(list); > > >> } > > >> @@ -4157,4 +4174,6 @@ reload: > > >> } > > >> > > >> + pmd->intrvl_tsc_prev = 0; > > >> + atomic_store_relaxed(&pmd->intrvl_cycles, 0); > > >> cycles_count_start(pmd); > > >> for (;;) { > > >> @@ -6138,4 +6157,5 @@ dp_netdev_pmd_try_optimize(struct > > >> dp_netdev_pmd_thread *pmd, > > >> > > >> if (pmd->ctx.now > pmd->rxq_next_cycle_store) { > > >> + uint64_t curr_tsc; > > >> /* Get the cycles that were used to process each queue and > > store. */ > > >> for (unsigned i = 0; i < poll_cnt; i++) { @@ -6146,4 > > >> +6166,11 @@ dp_netdev_pmd_try_optimize(struct dp_netdev_pmd_thread > > *pmd, > > >> 0); > > >> } > > >> + curr_tsc = cycles_counter(); > > >> + if (pmd->intrvl_tsc_prev) { > > >> + /* There is a prev timestamp, store a new intrvl cycle > > count. */ > > >> + atomic_store_relaxed(&pmd->intrvl_cycles, > > >> + curr_tsc - pmd->intrvl_tsc_prev); > > >> + } > > >> + pmd->intrvl_tsc_prev = curr_tsc; > > >> /* Start new measuring interval */ > > >> pmd->rxq_next_cycle_store = pmd->ctx.now + > > >> PMD_RXQ_INTERVAL_LEN; diff --git a/tests/pmd.at b/tests/pmd.at index > > >> fcb007c..09d09cb 100644 > > >> --- a/tests/pmd.at > > >> +++ b/tests/pmd.at > > >> @@ -7,5 +7,13 @@ m4_divert_push([PREPARE_TESTS]) # port_name rxq_id > > >> numa_id core_id parse_pmd_rxq_show () { > > >> - awk '/pmd/ {numa=$4; core=substr($6, 1, length($6) - 1)} /\t/{for > > (i=4; i<=NF; i++) print $2, $i, numa, core}' | sort > > >> + awk '/pmd thread/ {numa=$4; core=substr($6, 1, length($6) - 1)} > > >> +/\tport:/ {print $2, $4, numa, core}' | sort } > > >> + > > >> +# Given the output of `ovs-appctl dpif-netdev/pmd-rxq-show`, # and > > >> +with queues for each core on one line, prints the rxqs # of the core > > >> +on one line # 'port:' port_name 'queue_id:' rxq_id rxq_id rxq_id > > >> +rxq_id parse_pmd_rxq_show_group () { > > >> + awk '/port:/ {print $1, $2, $3, $4, $12, $20, $28}' > > >> } > > >> > > >> @@ -54,5 +62,5 @@ m4_define([CHECK_PMD_THREADS_CREATED], [ > > >> > > >> m4_define([SED_NUMA_CORE_PATTERN], ["s/\(numa_id \)[[0-9]]*\( > > >> core_id \)[[0-9]]*:/\1<cleared>\2<cleared>:/"]) > > >> -m4_define([SED_NUMA_CORE_QUEUE_PATTERN], ["s/\(numa_id \)[[0-9]]*\( > > core_id \)[[0-9]]*:/\1<cleared>\2<cleared>:/;s/\(queue-id: > > >> \)1 2 5 6/\1<cleared>/;s/\(queue-id: \)0 3 4 7/\1<cleared>/"]) > > >> +m4_define([SED_NUMA_CORE_QUEUE_PATTERN], ["s/1 2 5 6/<group>/;s/0 3 > > >> +4 7/<group>/"]) > > >> m4_define([DUMMY_NUMA], [--dummy-numa="0,0,0,0"]) > > >> > > >> @@ -66,5 +74,5 @@ AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed > > >> SED_NUMA_CORE_PATTERN], [0], pmd thread numa_id <cleared> core_id > > <cleared>: > > >> isolated : false > > >> - port: p0 queue-id: 0 > > >> + port: p0 queue-id: 0 pmd usage: NOT AVAIL > > >> ]) > > >> > > >> @@ -97,5 +105,12 @@ AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | > > >> sed SED_NUMA_CORE_PATTERN], [0], pmd thread numa_id <cleared> core_id > > <cleared>: > > >> isolated : false > > >> - port: p0 queue-id: 0 1 2 3 4 5 6 7 <sip:01234567> > > >> + port: p0 queue-id: 0 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 1 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 2 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 3 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 4 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 5 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 6 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 7 pmd usage: NOT AVAIL > > >> ]) > > >> > > >> @@ -121,5 +136,12 @@ AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | > > >> sed SED_NUMA_CORE_PATTERN], [0], pmd thread numa_id <cleared> core_id > > <cleared>: > > >> isolated : false > > >> - port: p0 queue-id: 0 1 2 3 4 5 6 7 <sip:01234567> > > >> + port: p0 queue-id: 0 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 1 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 2 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 3 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 4 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 5 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 6 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 7 pmd usage: NOT AVAIL > > >> ]) > > >> > > >> @@ -128,11 +150,7 @@ AT_CHECK([ovs-vsctl set Open_vSwitch . > > >> other_config:pmd-cpu-mask=0x3]) CHECK_PMD_THREADS_CREATED([2], [], > > >> [+$TMP]) > > >> > > >> -AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed > > >> SED_NUMA_CORE_QUEUE_PATTERN], [0], [dnl -pmd thread numa_id <cleared> > > core_id <cleared>: > > >> - isolated : false > > >> - port: p0 queue-id: <cleared> > > >> -pmd thread numa_id <cleared> core_id <cleared>: > > >> - isolated : false > > >> - port: p0 queue-id: <cleared> > > >> +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed > > >> +':a;/AVAIL$/{N;s/\n//;ba}' | parse_pmd_rxq_show_group | sed > > >> SED_NUMA_CORE_QUEUE_PATTERN], [0], [dnl > > >> +port: p0 queue-id: <group> > > >> +port: p0 queue-id: <group> > > >> ]) > > >> > > >> @@ -144,5 +162,12 @@ AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | > > >> sed SED_NUMA_CORE_PATTERN], [0], pmd thread numa_id <cleared> core_id > > <cleared>: > > >> isolated : false > > >> - port: p0 queue-id: 0 1 2 3 4 5 6 7 <sip:01234567> > > >> + port: p0 queue-id: 0 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 1 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 2 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 3 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 4 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 5 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 6 pmd usage: NOT AVAIL > > >> + port: p0 queue-id: 7 pmd usage: NOT AVAIL > > >> ]) > > >> > > >> -- > > >> 1.8.3.1 > > >
_______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
