On Mon, Aug 18, 2014 at 08:41:45AM +0100, Santosh Shukla wrote: > - Disable default odp thread cpuset. Use is-cpu-isolated.sh script to create > cpuset for dplane, cplane for odp. > - Use odp-on-isolated-cpu.sh script to test this application. > - test script location [1] > > - Refer readme for instruction to use. > > [1] > https://git.linaro.org/people/santosh.shukla/test-definitions.git/shortlog/refs/heads/isol-v2 > > Signed-off-by: Santosh Shukla <[email protected]> > --- > example/isolation/odp_isolation.c | 5 +++++ > example/isolation/readme | 20 ++++++++++++++++++++ > platform/linux-generic/odp_linux.c | 12 ++++++++++-- > 3 files changed, 35 insertions(+), 2 deletions(-) > create mode 100644 example/isolation/readme > > diff --git a/example/isolation/odp_isolation.c > b/example/isolation/odp_isolation.c > index 8d8fffc..1ade4c7 100644 > --- a/example/isolation/odp_isolation.c > +++ b/example/isolation/odp_isolation.c > @@ -214,6 +214,11 @@ int main(int argc, char *argv[]) > thr_id = odp_thread_create(0); > odp_init_local(thr_id); > > + /* > + * create dummy dp thread and don't pin them on any core, > + * use odp-on-isolated-cpu.sh script to define dplane cpuset > + * and to pin, run app on them, Refer readme. > + */ > for (i=0; i<num_workers; i++) { > > first_core = atoi(args.core_name[i]); > diff --git a/example/isolation/readme b/example/isolation/readme > new file mode 100644 > index 0000000..d49d61a > --- /dev/null > +++ b/example/isolation/readme > @@ -0,0 +1,20 @@ > + > +README on How-to run odp app on isolated core for no_hz_full kernel mode. > + > +- Disbale timer init in odp application. > +- avoid using odp's cpuset. Let odp thread launch unpinned. > +- use odp-on-isolated-cpu.sh script to pin odp dp thread to > + dplane cpuset. > + > +- download my local test-definition repo link [1] > +- copy odp_isolation binary to /usr/local/bin > +- run ./common/scripts/odp-on-isolated-cpu.sh > +Or > +- ./common/scripts/odp-on-isolated-cpu.sh 1,2 "odp_isolation -l 1,2" > +- Above script shows isolation duration, before / after interrupt count. > + > +Known issue : > +- x86 hw generates spurious ipi lead to break no_hz_full isolation (todo). > + > +[1] > https://git.linaro.org/people/santosh.shukla/test-definitions.git/shortlog/refs/heads/isol-v2 > + > diff --git a/platform/linux-generic/odp_linux.c > b/platform/linux-generic/odp_linux.c > index 6e2b448..7e853c7 100644 > --- a/platform/linux-generic/odp_linux.c > +++ b/platform/linux-generic/odp_linux.c > @@ -22,6 +22,11 @@ > #include <odp_debug.h> > > > +#define NO_HZ_FULL_ISOL /* > + * don't use odp cpuset. > + * enable this for odp isolation mode > + */ > + > typedef struct { > int thr_id; > void *(*start_routine) (void *); > @@ -49,7 +54,9 @@ void odp_linux_pthread_create(odp_linux_pthread_t > *thread_tbl, int num, > int first_core, void *(*start_routine) (void *), void *arg) > { > int i; > +#ifndef NO_HZ_FULL_ISOL > cpu_set_t cpu_set; > +#endif > odp_start_args_t *start_args; > int core_count; > int cpu; > @@ -62,16 +69,17 @@ void odp_linux_pthread_create(odp_linux_pthread_t > *thread_tbl, int num, > memset(thread_tbl, 0, num * sizeof(odp_linux_pthread_t)); > > for (i = 0; i < num; i++) { > + cpu = (first_core + i) % core_count; > +#ifndef NO_HZ_FULL_ISOL > pthread_attr_init(&thread_tbl[i].attr); > > CPU_ZERO(&cpu_set); > > - cpu = (first_core + i) % core_count; > CPU_SET(cpu, &cpu_set); > > pthread_attr_setaffinity_np(&thread_tbl[i].attr, > sizeof(cpu_set_t), &cpu_set); > - > +#endif
There's no need for this to be a compile time decision - this changes the behaviour of all applications linked against this library. I would rather leave the affinity setting in odp_linux_pthread_create and let the script just ensure that the specified cores are isolated. Is there anything actually preventing that from being possible? As it is, what happens if you build with NO_HZ_FULL_ISOL but don't run the application via the odp-on-isolated-cpu.sh script? and if you do run via the script, what's happening during the time between the application launching the threads and the script moving those threads onto the correct cores? -- Stuart. _______________________________________________ lng-odp mailing list [email protected] http://lists.linaro.org/mailman/listinfo/lng-odp
