Re: [lng-odp] [API-NEXT PATCH v9 0/6] A scalable software scheduler

[Bill Fischofer]

I explained that my review was premature. I had forgotten to check make
distcheck. This needs to be fixed prior to merge.

On Wed, Jun 21, 2017 at 4:22 PM Honnappa Nagarahalli <
honnappa.nagaraha...@linaro.org> wrote:

> Just wondering here. Why is this being enforced now? Why was this not
> enforced when the patch was approved?
>
> On 21 June 2017 at 15:26, Bill Fischofer 
> wrote:
> > On Wed, Jun 21, 2017 at 3:17 PM, Brian Brooks 
> wrote:
> >> On 06/20 19:38:59, Bill Fischofer wrote:
> >>> Now that master has been merged back into api-next I can confirm that
> >>> make distcheck fails for 64-bit systems as well, so I need to withdraw
> >>> my earlier reviewed-and-tested-by until that is fixed.
> >>
> >> This appears to be a problem in pristine upstream as well. "make
> distcheck"
> >> is broken on ARM (and likely all non-x86 archs).
> >>
> >> The issue is that in platform/linux-generic/Makefile.am "if ARCH_IS_X86"
> >> is always true. I have not yet found a workaround..
> >
> > Good to know, but with this patch it's also broken for x86. Sounds
> > like we have two issues to address.
> >
> >>
> >>> On Mon, Jun 19, 2017 at 6:13 PM, Bill Fischofer
> >>>  wrote:
> >>> > Looks like I posted a wee bit too soon. On a 32-bit system:
> >>> >
> >>> > bill@Ub16-32:~/linaro/armschedv9$ make distcheck
> >>> > make  dist-gzip am__post_remove_distdir='@:'
> >>> > make[1]: Entering directory '/home/bill/linaro/armschedv9'
> >>> > if test -d "opendataplane-1.14.0.0"; then find
> >>> > "opendataplane-1.14.0.0" -type d ! -perm -200 -exec chmod u+w {} ';'
> >>> > && rm -rf "opendataplane-1.14.0.0" || { sleep 5 && rm -rf
> >>> > "opendataplane-1.14.0.0"; }; else :; fi
> >>> > test -d "opendataplane-1.14.0.0" || mkdir "opendataplane-1.14.0.0"
> >>> >  (cd platform/linux-generic && make
> >>> > top_distdir=../../opendataplane-1.14.0.0
> >>> > distdir=../../opendataplane-1.14.0.0/platform/linux-generic \
> >>> >  am__remove_distdir=: am__skip_length_check=:
> am__skip_mode_fix=: distdir)
> >>> > make[2]: Entering directory
> >>> > '/home/bill/linaro/armschedv9/platform/linux-generic'
> >>> > make[2]: *** No rule to make target 'arch/x86/odp_atomic.h', needed
> by
> >>> > 'distdir'.  Stop.
> >>> > make[2]: Leaving directory
> '/home/bill/linaro/armschedv9/platform/linux-generic'
> >>> > Makefile:603: recipe for target 'distdir' failed
> >>> > make[1]: *** [distdir] Error 1
> >>> > make[1]: Leaving directory '/home/bill/linaro/armschedv9'
> >>> > Makefile:702: recipe for target 'dist' failed
> >>> > make: *** [dist] Error 2
> >>> >
> >>> > On Mon, Jun 19, 2017 at 6:11 PM, Bill Fischofer
> >>> >  wrote:
> >>> >> For the v9 series:
> >>> >>
> >>> >> Reviewed-and-tested-by: Bill Fischofer 
> >>> >>
> >>> >> I also verified that there are no conflicts between this series and
> >>> >> Petri's queue cleanup patch, so this can apply and run just fine on
> >>> >> top of it. Maxim should be able to merge both tomorrow.
> >>> >>
> >>> >> On Mon, Jun 19, 2017 at 2:12 PM, Brian Brooks 
> wrote:
> >>> >>> This work derives from Ola Liljedahl's prototype [1] which
> introduced a
> >>> >>> scalable scheduler design based on primarily lock-free algorithms
> and
> >>> >>> data structures designed to decrease contention. A thread searches
> >>> >>> through a data structure containing only queues that are both
> non-empty
> >>> >>> and allowed to be scheduled to that thread. Strict priority
> scheduling is
> >>> >>> respected, and (W)RR scheduling may be used within queues of the
> same priority.
> >>> >>> Lastly, pre-scheduling or stashing is not employed since it is
> optional
> >>> >>> functionality that can be implemented in the application.
> >>> >>>
> >>> >>> In addition to scalable ring buffers, the algorithm also uses
> unbounded
> >>> >>> concurrent queues. LL/SC and CAS variants exist in cases where
> absense of
> >>> >>> ABA problem cannot be proved, and also in cases where the
> compiler's atomic
> >>> >>> built-ins may not be lowered to the desired instruction(s).
> Finally, a version
> >>> >>> of the algorithm that uses locks is also provided.
> >>> >>>
> >>> >>> Use --enable-schedule-scalable to conditionally compile this
> scheduler
> >>> >>> into the library.
> >>> >>>
> >>> >>> [1]
> https://lists.linaro.org/pipermail/lng-odp/2016-September/025682.html
> >>> >>>
> >>> >>> On checkpatch.pl:
> >>> >>>  - [2/6] and [5/6] have checkpatch.pl issues that are superfluous
> >>> >>>
> >>> >>> v9:
> >>> >>>  - Include patch to enable scalable scheduler in Travis CI
> >>> >>>  - Fix 'make distcheck'
> >>> >>>
> >>> >>> v8:
> >>> >>>  - Reword commit messages
> >>> >>>
> >>> >>> v7:
> >>> >>>  - Rebase against new modular queue interface
> >>> >>>  - Duplicate arch files under mips64 and powerpc
> >>> >>>  - Fix sched->order_lock()
> >>> >>>  - Loop until all deferred events have been 

Re: [lng-odp] [API-NEXT PATCH v9 0/6] A scalable software scheduler

[Honnappa Nagarahalli]

Just wondering here. Why is this being enforced now? Why was this not
enforced when the patch was approved?

On 21 June 2017 at 15:26, Bill Fischofer  wrote:
> On Wed, Jun 21, 2017 at 3:17 PM, Brian Brooks  wrote:
>> On 06/20 19:38:59, Bill Fischofer wrote:
>>> Now that master has been merged back into api-next I can confirm that
>>> make distcheck fails for 64-bit systems as well, so I need to withdraw
>>> my earlier reviewed-and-tested-by until that is fixed.
>>
>> This appears to be a problem in pristine upstream as well. "make distcheck"
>> is broken on ARM (and likely all non-x86 archs).
>>
>> The issue is that in platform/linux-generic/Makefile.am "if ARCH_IS_X86"
>> is always true. I have not yet found a workaround..
>
> Good to know, but with this patch it's also broken for x86. Sounds
> like we have two issues to address.
>
>>
>>> On Mon, Jun 19, 2017 at 6:13 PM, Bill Fischofer
>>>  wrote:
>>> > Looks like I posted a wee bit too soon. On a 32-bit system:
>>> >
>>> > bill@Ub16-32:~/linaro/armschedv9$ make distcheck
>>> > make  dist-gzip am__post_remove_distdir='@:'
>>> > make[1]: Entering directory '/home/bill/linaro/armschedv9'
>>> > if test -d "opendataplane-1.14.0.0"; then find
>>> > "opendataplane-1.14.0.0" -type d ! -perm -200 -exec chmod u+w {} ';'
>>> > && rm -rf "opendataplane-1.14.0.0" || { sleep 5 && rm -rf
>>> > "opendataplane-1.14.0.0"; }; else :; fi
>>> > test -d "opendataplane-1.14.0.0" || mkdir "opendataplane-1.14.0.0"
>>> >  (cd platform/linux-generic && make
>>> > top_distdir=../../opendataplane-1.14.0.0
>>> > distdir=../../opendataplane-1.14.0.0/platform/linux-generic \
>>> >  am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: 
>>> > distdir)
>>> > make[2]: Entering directory
>>> > '/home/bill/linaro/armschedv9/platform/linux-generic'
>>> > make[2]: *** No rule to make target 'arch/x86/odp_atomic.h', needed by
>>> > 'distdir'.  Stop.
>>> > make[2]: Leaving directory 
>>> > '/home/bill/linaro/armschedv9/platform/linux-generic'
>>> > Makefile:603: recipe for target 'distdir' failed
>>> > make[1]: *** [distdir] Error 1
>>> > make[1]: Leaving directory '/home/bill/linaro/armschedv9'
>>> > Makefile:702: recipe for target 'dist' failed
>>> > make: *** [dist] Error 2
>>> >
>>> > On Mon, Jun 19, 2017 at 6:11 PM, Bill Fischofer
>>> >  wrote:
>>> >> For the v9 series:
>>> >>
>>> >> Reviewed-and-tested-by: Bill Fischofer 
>>> >>
>>> >> I also verified that there are no conflicts between this series and
>>> >> Petri's queue cleanup patch, so this can apply and run just fine on
>>> >> top of it. Maxim should be able to merge both tomorrow.
>>> >>
>>> >> On Mon, Jun 19, 2017 at 2:12 PM, Brian Brooks  
>>> >> wrote:
>>> >>> This work derives from Ola Liljedahl's prototype [1] which introduced a
>>> >>> scalable scheduler design based on primarily lock-free algorithms and
>>> >>> data structures designed to decrease contention. A thread searches
>>> >>> through a data structure containing only queues that are both non-empty
>>> >>> and allowed to be scheduled to that thread. Strict priority scheduling 
>>> >>> is
>>> >>> respected, and (W)RR scheduling may be used within queues of the same 
>>> >>> priority.
>>> >>> Lastly, pre-scheduling or stashing is not employed since it is optional
>>> >>> functionality that can be implemented in the application.
>>> >>>
>>> >>> In addition to scalable ring buffers, the algorithm also uses unbounded
>>> >>> concurrent queues. LL/SC and CAS variants exist in cases where absense 
>>> >>> of
>>> >>> ABA problem cannot be proved, and also in cases where the compiler's 
>>> >>> atomic
>>> >>> built-ins may not be lowered to the desired instruction(s). Finally, a 
>>> >>> version
>>> >>> of the algorithm that uses locks is also provided.
>>> >>>
>>> >>> Use --enable-schedule-scalable to conditionally compile this scheduler
>>> >>> into the library.
>>> >>>
>>> >>> [1] 
>>> >>> https://lists.linaro.org/pipermail/lng-odp/2016-September/025682.html
>>> >>>
>>> >>> On checkpatch.pl:
>>> >>>  - [2/6] and [5/6] have checkpatch.pl issues that are superfluous
>>> >>>
>>> >>> v9:
>>> >>>  - Include patch to enable scalable scheduler in Travis CI
>>> >>>  - Fix 'make distcheck'
>>> >>>
>>> >>> v8:
>>> >>>  - Reword commit messages
>>> >>>
>>> >>> v7:
>>> >>>  - Rebase against new modular queue interface
>>> >>>  - Duplicate arch files under mips64 and powerpc
>>> >>>  - Fix sched->order_lock()
>>> >>>  - Loop until all deferred events have been enqueued
>>> >>>  - Implement ord_enq_multi()
>>> >>>  - Fix ordered_lock/unlock
>>> >>>  - Revert stylistic changes
>>> >>>  - Add default xfactor
>>> >>>  - Remove changes to odp_sched_latency
>>> >>>  - Remove ULL suffix to alleviate Clang build
>>> >>>
>>> >>> v6:
>>> >>>  - Move conversions into scalable scheduler to alleviate #ifdefs
>>> >>>  - Remove 

Re: [lng-odp] [API-NEXT PATCH v9 0/6] A scalable software scheduler

[Bill Fischofer]

On Wed, Jun 21, 2017 at 3:17 PM, Brian Brooks  wrote:
> On 06/20 19:38:59, Bill Fischofer wrote:
>> Now that master has been merged back into api-next I can confirm that
>> make distcheck fails for 64-bit systems as well, so I need to withdraw
>> my earlier reviewed-and-tested-by until that is fixed.
>
> This appears to be a problem in pristine upstream as well. "make distcheck"
> is broken on ARM (and likely all non-x86 archs).
>
> The issue is that in platform/linux-generic/Makefile.am "if ARCH_IS_X86"
> is always true. I have not yet found a workaround..

Good to know, but with this patch it's also broken for x86. Sounds
like we have two issues to address.

>
>> On Mon, Jun 19, 2017 at 6:13 PM, Bill Fischofer
>>  wrote:
>> > Looks like I posted a wee bit too soon. On a 32-bit system:
>> >
>> > bill@Ub16-32:~/linaro/armschedv9$ make distcheck
>> > make  dist-gzip am__post_remove_distdir='@:'
>> > make[1]: Entering directory '/home/bill/linaro/armschedv9'
>> > if test -d "opendataplane-1.14.0.0"; then find
>> > "opendataplane-1.14.0.0" -type d ! -perm -200 -exec chmod u+w {} ';'
>> > && rm -rf "opendataplane-1.14.0.0" || { sleep 5 && rm -rf
>> > "opendataplane-1.14.0.0"; }; else :; fi
>> > test -d "opendataplane-1.14.0.0" || mkdir "opendataplane-1.14.0.0"
>> >  (cd platform/linux-generic && make
>> > top_distdir=../../opendataplane-1.14.0.0
>> > distdir=../../opendataplane-1.14.0.0/platform/linux-generic \
>> >  am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: 
>> > distdir)
>> > make[2]: Entering directory
>> > '/home/bill/linaro/armschedv9/platform/linux-generic'
>> > make[2]: *** No rule to make target 'arch/x86/odp_atomic.h', needed by
>> > 'distdir'.  Stop.
>> > make[2]: Leaving directory 
>> > '/home/bill/linaro/armschedv9/platform/linux-generic'
>> > Makefile:603: recipe for target 'distdir' failed
>> > make[1]: *** [distdir] Error 1
>> > make[1]: Leaving directory '/home/bill/linaro/armschedv9'
>> > Makefile:702: recipe for target 'dist' failed
>> > make: *** [dist] Error 2
>> >
>> > On Mon, Jun 19, 2017 at 6:11 PM, Bill Fischofer
>> >  wrote:
>> >> For the v9 series:
>> >>
>> >> Reviewed-and-tested-by: Bill Fischofer 
>> >>
>> >> I also verified that there are no conflicts between this series and
>> >> Petri's queue cleanup patch, so this can apply and run just fine on
>> >> top of it. Maxim should be able to merge both tomorrow.
>> >>
>> >> On Mon, Jun 19, 2017 at 2:12 PM, Brian Brooks  
>> >> wrote:
>> >>> This work derives from Ola Liljedahl's prototype [1] which introduced a
>> >>> scalable scheduler design based on primarily lock-free algorithms and
>> >>> data structures designed to decrease contention. A thread searches
>> >>> through a data structure containing only queues that are both non-empty
>> >>> and allowed to be scheduled to that thread. Strict priority scheduling is
>> >>> respected, and (W)RR scheduling may be used within queues of the same 
>> >>> priority.
>> >>> Lastly, pre-scheduling or stashing is not employed since it is optional
>> >>> functionality that can be implemented in the application.
>> >>>
>> >>> In addition to scalable ring buffers, the algorithm also uses unbounded
>> >>> concurrent queues. LL/SC and CAS variants exist in cases where absense of
>> >>> ABA problem cannot be proved, and also in cases where the compiler's 
>> >>> atomic
>> >>> built-ins may not be lowered to the desired instruction(s). Finally, a 
>> >>> version
>> >>> of the algorithm that uses locks is also provided.
>> >>>
>> >>> Use --enable-schedule-scalable to conditionally compile this scheduler
>> >>> into the library.
>> >>>
>> >>> [1] https://lists.linaro.org/pipermail/lng-odp/2016-September/025682.html
>> >>>
>> >>> On checkpatch.pl:
>> >>>  - [2/6] and [5/6] have checkpatch.pl issues that are superfluous
>> >>>
>> >>> v9:
>> >>>  - Include patch to enable scalable scheduler in Travis CI
>> >>>  - Fix 'make distcheck'
>> >>>
>> >>> v8:
>> >>>  - Reword commit messages
>> >>>
>> >>> v7:
>> >>>  - Rebase against new modular queue interface
>> >>>  - Duplicate arch files under mips64 and powerpc
>> >>>  - Fix sched->order_lock()
>> >>>  - Loop until all deferred events have been enqueued
>> >>>  - Implement ord_enq_multi()
>> >>>  - Fix ordered_lock/unlock
>> >>>  - Revert stylistic changes
>> >>>  - Add default xfactor
>> >>>  - Remove changes to odp_sched_latency
>> >>>  - Remove ULL suffix to alleviate Clang build
>> >>>
>> >>> v6:
>> >>>  - Move conversions into scalable scheduler to alleviate #ifdefs
>> >>>  - Remove unnecessary prefetch
>> >>>  - Fix ARMv8 build
>> >>>
>> >>> v5:
>> >>>  - Allocate cache aligned memory using shm pool APIs
>> >>>  - Move more code to scalable scheduler specific files
>> >>>  - Remove CONFIG_SPLIT_READWRITE
>> >>>  - Fix 'make distcheck' issue
>> >>>
>> >>> v4:
>> >>>  - Fix a couple more checkpatch.pl 

Re: [lng-odp] [API-NEXT PATCH v9 0/6] A scalable software scheduler

[Brian Brooks]

On 06/20 19:38:59, Bill Fischofer wrote:
> Now that master has been merged back into api-next I can confirm that
> make distcheck fails for 64-bit systems as well, so I need to withdraw
> my earlier reviewed-and-tested-by until that is fixed.

This appears to be a problem in pristine upstream as well. "make distcheck"
is broken on ARM (and likely all non-x86 archs).

The issue is that in platform/linux-generic/Makefile.am "if ARCH_IS_X86"
is always true. I have not yet found a workaround..

> On Mon, Jun 19, 2017 at 6:13 PM, Bill Fischofer
>  wrote:
> > Looks like I posted a wee bit too soon. On a 32-bit system:
> >
> > bill@Ub16-32:~/linaro/armschedv9$ make distcheck
> > make  dist-gzip am__post_remove_distdir='@:'
> > make[1]: Entering directory '/home/bill/linaro/armschedv9'
> > if test -d "opendataplane-1.14.0.0"; then find
> > "opendataplane-1.14.0.0" -type d ! -perm -200 -exec chmod u+w {} ';'
> > && rm -rf "opendataplane-1.14.0.0" || { sleep 5 && rm -rf
> > "opendataplane-1.14.0.0"; }; else :; fi
> > test -d "opendataplane-1.14.0.0" || mkdir "opendataplane-1.14.0.0"
> >  (cd platform/linux-generic && make
> > top_distdir=../../opendataplane-1.14.0.0
> > distdir=../../opendataplane-1.14.0.0/platform/linux-generic \
> >  am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: 
> > distdir)
> > make[2]: Entering directory
> > '/home/bill/linaro/armschedv9/platform/linux-generic'
> > make[2]: *** No rule to make target 'arch/x86/odp_atomic.h', needed by
> > 'distdir'.  Stop.
> > make[2]: Leaving directory 
> > '/home/bill/linaro/armschedv9/platform/linux-generic'
> > Makefile:603: recipe for target 'distdir' failed
> > make[1]: *** [distdir] Error 1
> > make[1]: Leaving directory '/home/bill/linaro/armschedv9'
> > Makefile:702: recipe for target 'dist' failed
> > make: *** [dist] Error 2
> >
> > On Mon, Jun 19, 2017 at 6:11 PM, Bill Fischofer
> >  wrote:
> >> For the v9 series:
> >>
> >> Reviewed-and-tested-by: Bill Fischofer 
> >>
> >> I also verified that there are no conflicts between this series and
> >> Petri's queue cleanup patch, so this can apply and run just fine on
> >> top of it. Maxim should be able to merge both tomorrow.
> >>
> >> On Mon, Jun 19, 2017 at 2:12 PM, Brian Brooks  wrote:
> >>> This work derives from Ola Liljedahl's prototype [1] which introduced a
> >>> scalable scheduler design based on primarily lock-free algorithms and
> >>> data structures designed to decrease contention. A thread searches
> >>> through a data structure containing only queues that are both non-empty
> >>> and allowed to be scheduled to that thread. Strict priority scheduling is
> >>> respected, and (W)RR scheduling may be used within queues of the same 
> >>> priority.
> >>> Lastly, pre-scheduling or stashing is not employed since it is optional
> >>> functionality that can be implemented in the application.
> >>>
> >>> In addition to scalable ring buffers, the algorithm also uses unbounded
> >>> concurrent queues. LL/SC and CAS variants exist in cases where absense of
> >>> ABA problem cannot be proved, and also in cases where the compiler's 
> >>> atomic
> >>> built-ins may not be lowered to the desired instruction(s). Finally, a 
> >>> version
> >>> of the algorithm that uses locks is also provided.
> >>>
> >>> Use --enable-schedule-scalable to conditionally compile this scheduler
> >>> into the library.
> >>>
> >>> [1] https://lists.linaro.org/pipermail/lng-odp/2016-September/025682.html
> >>>
> >>> On checkpatch.pl:
> >>>  - [2/6] and [5/6] have checkpatch.pl issues that are superfluous
> >>>
> >>> v9:
> >>>  - Include patch to enable scalable scheduler in Travis CI
> >>>  - Fix 'make distcheck'
> >>>
> >>> v8:
> >>>  - Reword commit messages
> >>>
> >>> v7:
> >>>  - Rebase against new modular queue interface
> >>>  - Duplicate arch files under mips64 and powerpc
> >>>  - Fix sched->order_lock()
> >>>  - Loop until all deferred events have been enqueued
> >>>  - Implement ord_enq_multi()
> >>>  - Fix ordered_lock/unlock
> >>>  - Revert stylistic changes
> >>>  - Add default xfactor
> >>>  - Remove changes to odp_sched_latency
> >>>  - Remove ULL suffix to alleviate Clang build
> >>>
> >>> v6:
> >>>  - Move conversions into scalable scheduler to alleviate #ifdefs
> >>>  - Remove unnecessary prefetch
> >>>  - Fix ARMv8 build
> >>>
> >>> v5:
> >>>  - Allocate cache aligned memory using shm pool APIs
> >>>  - Move more code to scalable scheduler specific files
> >>>  - Remove CONFIG_SPLIT_READWRITE
> >>>  - Fix 'make distcheck' issue
> >>>
> >>> v4:
> >>>  - Fix a couple more checkpatch.pl issues
> >>>
> >>> v3:
> >>>  - Only conditionally compile scalable scheduler and queue
> >>>  - Move some code to arch/ dir
> >>>  - Use a single shm block for queues instead of block-per-queue
> >>>  - De-interleave odp_llqueue.h
> >>>  - Use compiler macros to determine ATOM_BITSET_SIZE
> >>>  - 

Re: [lng-odp] [API-NEXT PATCH v9 0/6] A scalable software scheduler

[Bill Fischofer]

Now that master has been merged back into api-next I can confirm that
make distcheck fails for 64-bit systems as well, so I need to withdraw
my earlier reviewed-and-tested-by until that is fixed.

On Mon, Jun 19, 2017 at 6:13 PM, Bill Fischofer
 wrote:
> Looks like I posted a wee bit too soon. On a 32-bit system:
>
> bill@Ub16-32:~/linaro/armschedv9$ make distcheck
> make  dist-gzip am__post_remove_distdir='@:'
> make[1]: Entering directory '/home/bill/linaro/armschedv9'
> if test -d "opendataplane-1.14.0.0"; then find
> "opendataplane-1.14.0.0" -type d ! -perm -200 -exec chmod u+w {} ';'
> && rm -rf "opendataplane-1.14.0.0" || { sleep 5 && rm -rf
> "opendataplane-1.14.0.0"; }; else :; fi
> test -d "opendataplane-1.14.0.0" || mkdir "opendataplane-1.14.0.0"
>  (cd platform/linux-generic && make
> top_distdir=../../opendataplane-1.14.0.0
> distdir=../../opendataplane-1.14.0.0/platform/linux-generic \
>  am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
> make[2]: Entering directory
> '/home/bill/linaro/armschedv9/platform/linux-generic'
> make[2]: *** No rule to make target 'arch/x86/odp_atomic.h', needed by
> 'distdir'.  Stop.
> make[2]: Leaving directory 
> '/home/bill/linaro/armschedv9/platform/linux-generic'
> Makefile:603: recipe for target 'distdir' failed
> make[1]: *** [distdir] Error 1
> make[1]: Leaving directory '/home/bill/linaro/armschedv9'
> Makefile:702: recipe for target 'dist' failed
> make: *** [dist] Error 2
>
> On Mon, Jun 19, 2017 at 6:11 PM, Bill Fischofer
>  wrote:
>> For the v9 series:
>>
>> Reviewed-and-tested-by: Bill Fischofer 
>>
>> I also verified that there are no conflicts between this series and
>> Petri's queue cleanup patch, so this can apply and run just fine on
>> top of it. Maxim should be able to merge both tomorrow.
>>
>> On Mon, Jun 19, 2017 at 2:12 PM, Brian Brooks  wrote:
>>> This work derives from Ola Liljedahl's prototype [1] which introduced a
>>> scalable scheduler design based on primarily lock-free algorithms and
>>> data structures designed to decrease contention. A thread searches
>>> through a data structure containing only queues that are both non-empty
>>> and allowed to be scheduled to that thread. Strict priority scheduling is
>>> respected, and (W)RR scheduling may be used within queues of the same 
>>> priority.
>>> Lastly, pre-scheduling or stashing is not employed since it is optional
>>> functionality that can be implemented in the application.
>>>
>>> In addition to scalable ring buffers, the algorithm also uses unbounded
>>> concurrent queues. LL/SC and CAS variants exist in cases where absense of
>>> ABA problem cannot be proved, and also in cases where the compiler's atomic
>>> built-ins may not be lowered to the desired instruction(s). Finally, a 
>>> version
>>> of the algorithm that uses locks is also provided.
>>>
>>> Use --enable-schedule-scalable to conditionally compile this scheduler
>>> into the library.
>>>
>>> [1] https://lists.linaro.org/pipermail/lng-odp/2016-September/025682.html
>>>
>>> On checkpatch.pl:
>>>  - [2/6] and [5/6] have checkpatch.pl issues that are superfluous
>>>
>>> v9:
>>>  - Include patch to enable scalable scheduler in Travis CI
>>>  - Fix 'make distcheck'
>>>
>>> v8:
>>>  - Reword commit messages
>>>
>>> v7:
>>>  - Rebase against new modular queue interface
>>>  - Duplicate arch files under mips64 and powerpc
>>>  - Fix sched->order_lock()
>>>  - Loop until all deferred events have been enqueued
>>>  - Implement ord_enq_multi()
>>>  - Fix ordered_lock/unlock
>>>  - Revert stylistic changes
>>>  - Add default xfactor
>>>  - Remove changes to odp_sched_latency
>>>  - Remove ULL suffix to alleviate Clang build
>>>
>>> v6:
>>>  - Move conversions into scalable scheduler to alleviate #ifdefs
>>>  - Remove unnecessary prefetch
>>>  - Fix ARMv8 build
>>>
>>> v5:
>>>  - Allocate cache aligned memory using shm pool APIs
>>>  - Move more code to scalable scheduler specific files
>>>  - Remove CONFIG_SPLIT_READWRITE
>>>  - Fix 'make distcheck' issue
>>>
>>> v4:
>>>  - Fix a couple more checkpatch.pl issues
>>>
>>> v3:
>>>  - Only conditionally compile scalable scheduler and queue
>>>  - Move some code to arch/ dir
>>>  - Use a single shm block for queues instead of block-per-queue
>>>  - De-interleave odp_llqueue.h
>>>  - Use compiler macros to determine ATOM_BITSET_SIZE
>>>  - Incorporated queue size changes
>>>  - Dropped 'ODP_' prefix on config and moved to other files
>>>  - Dropped a few patches that were send independently to the list
>>>
>>> v2:
>>>  - Move ARMv8 issues and other fixes into separate patches
>>>  - Abstract away some #ifdefs
>>>  - Fix some checkpatch.pl warnings
>>>
>>> Brian Brooks (5):
>>>   test: odp_pktio_ordered: add queue size
>>>   linux-gen: sched scalable: add arch files
>>>   linux-gen: sched scalable: add a bitset
>>>   linux-gen: sched scalable: add a 

Re: [lng-odp] [API-NEXT PATCH v9 0/6] A scalable software scheduler

[Bill Fischofer]

Looks like I posted a wee bit too soon. On a 32-bit system:

bill@Ub16-32:~/linaro/armschedv9$ make distcheck
make  dist-gzip am__post_remove_distdir='@:'
make[1]: Entering directory '/home/bill/linaro/armschedv9'
if test -d "opendataplane-1.14.0.0"; then find
"opendataplane-1.14.0.0" -type d ! -perm -200 -exec chmod u+w {} ';'
&& rm -rf "opendataplane-1.14.0.0" || { sleep 5 && rm -rf
"opendataplane-1.14.0.0"; }; else :; fi
test -d "opendataplane-1.14.0.0" || mkdir "opendataplane-1.14.0.0"
 (cd platform/linux-generic && make
top_distdir=../../opendataplane-1.14.0.0
distdir=../../opendataplane-1.14.0.0/platform/linux-generic \
 am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[2]: Entering directory
'/home/bill/linaro/armschedv9/platform/linux-generic'
make[2]: *** No rule to make target 'arch/x86/odp_atomic.h', needed by
'distdir'.  Stop.
make[2]: Leaving directory '/home/bill/linaro/armschedv9/platform/linux-generic'
Makefile:603: recipe for target 'distdir' failed
make[1]: *** [distdir] Error 1
make[1]: Leaving directory '/home/bill/linaro/armschedv9'
Makefile:702: recipe for target 'dist' failed
make: *** [dist] Error 2

On Mon, Jun 19, 2017 at 6:11 PM, Bill Fischofer
 wrote:
> For the v9 series:
>
> Reviewed-and-tested-by: Bill Fischofer 
>
> I also verified that there are no conflicts between this series and
> Petri's queue cleanup patch, so this can apply and run just fine on
> top of it. Maxim should be able to merge both tomorrow.
>
> On Mon, Jun 19, 2017 at 2:12 PM, Brian Brooks  wrote:
>> This work derives from Ola Liljedahl's prototype [1] which introduced a
>> scalable scheduler design based on primarily lock-free algorithms and
>> data structures designed to decrease contention. A thread searches
>> through a data structure containing only queues that are both non-empty
>> and allowed to be scheduled to that thread. Strict priority scheduling is
>> respected, and (W)RR scheduling may be used within queues of the same 
>> priority.
>> Lastly, pre-scheduling or stashing is not employed since it is optional
>> functionality that can be implemented in the application.
>>
>> In addition to scalable ring buffers, the algorithm also uses unbounded
>> concurrent queues. LL/SC and CAS variants exist in cases where absense of
>> ABA problem cannot be proved, and also in cases where the compiler's atomic
>> built-ins may not be lowered to the desired instruction(s). Finally, a 
>> version
>> of the algorithm that uses locks is also provided.
>>
>> Use --enable-schedule-scalable to conditionally compile this scheduler
>> into the library.
>>
>> [1] https://lists.linaro.org/pipermail/lng-odp/2016-September/025682.html
>>
>> On checkpatch.pl:
>>  - [2/6] and [5/6] have checkpatch.pl issues that are superfluous
>>
>> v9:
>>  - Include patch to enable scalable scheduler in Travis CI
>>  - Fix 'make distcheck'
>>
>> v8:
>>  - Reword commit messages
>>
>> v7:
>>  - Rebase against new modular queue interface
>>  - Duplicate arch files under mips64 and powerpc
>>  - Fix sched->order_lock()
>>  - Loop until all deferred events have been enqueued
>>  - Implement ord_enq_multi()
>>  - Fix ordered_lock/unlock
>>  - Revert stylistic changes
>>  - Add default xfactor
>>  - Remove changes to odp_sched_latency
>>  - Remove ULL suffix to alleviate Clang build
>>
>> v6:
>>  - Move conversions into scalable scheduler to alleviate #ifdefs
>>  - Remove unnecessary prefetch
>>  - Fix ARMv8 build
>>
>> v5:
>>  - Allocate cache aligned memory using shm pool APIs
>>  - Move more code to scalable scheduler specific files
>>  - Remove CONFIG_SPLIT_READWRITE
>>  - Fix 'make distcheck' issue
>>
>> v4:
>>  - Fix a couple more checkpatch.pl issues
>>
>> v3:
>>  - Only conditionally compile scalable scheduler and queue
>>  - Move some code to arch/ dir
>>  - Use a single shm block for queues instead of block-per-queue
>>  - De-interleave odp_llqueue.h
>>  - Use compiler macros to determine ATOM_BITSET_SIZE
>>  - Incorporated queue size changes
>>  - Dropped 'ODP_' prefix on config and moved to other files
>>  - Dropped a few patches that were send independently to the list
>>
>> v2:
>>  - Move ARMv8 issues and other fixes into separate patches
>>  - Abstract away some #ifdefs
>>  - Fix some checkpatch.pl warnings
>>
>> Brian Brooks (5):
>>   test: odp_pktio_ordered: add queue size
>>   linux-gen: sched scalable: add arch files
>>   linux-gen: sched scalable: add a bitset
>>   linux-gen: sched scalable: add a concurrent queue
>>   linux-gen: sched scalable: add scalable scheduler
>>
>> Honnappa Nagarahalli (1):
>>   travis: add scalable scheduler in CI
>>
>>  .travis.yml|1 +
>>  configure.ac   |1 +
>>  platform/linux-generic/Makefile.am |   17 +
>>  platform/linux-generic/arch/arm/odp_atomic.h   |  210 +++
>>  

Re: [lng-odp] [API-NEXT PATCH v9 0/6] A scalable software scheduler

[Bill Fischofer]

For the v9 series:

Reviewed-and-tested-by: Bill Fischofer 

I also verified that there are no conflicts between this series and
Petri's queue cleanup patch, so this can apply and run just fine on
top of it. Maxim should be able to merge both tomorrow.

On Mon, Jun 19, 2017 at 2:12 PM, Brian Brooks  wrote:
> This work derives from Ola Liljedahl's prototype [1] which introduced a
> scalable scheduler design based on primarily lock-free algorithms and
> data structures designed to decrease contention. A thread searches
> through a data structure containing only queues that are both non-empty
> and allowed to be scheduled to that thread. Strict priority scheduling is
> respected, and (W)RR scheduling may be used within queues of the same 
> priority.
> Lastly, pre-scheduling or stashing is not employed since it is optional
> functionality that can be implemented in the application.
>
> In addition to scalable ring buffers, the algorithm also uses unbounded
> concurrent queues. LL/SC and CAS variants exist in cases where absense of
> ABA problem cannot be proved, and also in cases where the compiler's atomic
> built-ins may not be lowered to the desired instruction(s). Finally, a version
> of the algorithm that uses locks is also provided.
>
> Use --enable-schedule-scalable to conditionally compile this scheduler
> into the library.
>
> [1] https://lists.linaro.org/pipermail/lng-odp/2016-September/025682.html
>
> On checkpatch.pl:
>  - [2/6] and [5/6] have checkpatch.pl issues that are superfluous
>
> v9:
>  - Include patch to enable scalable scheduler in Travis CI
>  - Fix 'make distcheck'
>
> v8:
>  - Reword commit messages
>
> v7:
>  - Rebase against new modular queue interface
>  - Duplicate arch files under mips64 and powerpc
>  - Fix sched->order_lock()
>  - Loop until all deferred events have been enqueued
>  - Implement ord_enq_multi()
>  - Fix ordered_lock/unlock
>  - Revert stylistic changes
>  - Add default xfactor
>  - Remove changes to odp_sched_latency
>  - Remove ULL suffix to alleviate Clang build
>
> v6:
>  - Move conversions into scalable scheduler to alleviate #ifdefs
>  - Remove unnecessary prefetch
>  - Fix ARMv8 build
>
> v5:
>  - Allocate cache aligned memory using shm pool APIs
>  - Move more code to scalable scheduler specific files
>  - Remove CONFIG_SPLIT_READWRITE
>  - Fix 'make distcheck' issue
>
> v4:
>  - Fix a couple more checkpatch.pl issues
>
> v3:
>  - Only conditionally compile scalable scheduler and queue
>  - Move some code to arch/ dir
>  - Use a single shm block for queues instead of block-per-queue
>  - De-interleave odp_llqueue.h
>  - Use compiler macros to determine ATOM_BITSET_SIZE
>  - Incorporated queue size changes
>  - Dropped 'ODP_' prefix on config and moved to other files
>  - Dropped a few patches that were send independently to the list
>
> v2:
>  - Move ARMv8 issues and other fixes into separate patches
>  - Abstract away some #ifdefs
>  - Fix some checkpatch.pl warnings
>
> Brian Brooks (5):
>   test: odp_pktio_ordered: add queue size
>   linux-gen: sched scalable: add arch files
>   linux-gen: sched scalable: add a bitset
>   linux-gen: sched scalable: add a concurrent queue
>   linux-gen: sched scalable: add scalable scheduler
>
> Honnappa Nagarahalli (1):
>   travis: add scalable scheduler in CI
>
>  .travis.yml|1 +
>  configure.ac   |1 +
>  platform/linux-generic/Makefile.am |   17 +
>  platform/linux-generic/arch/arm/odp_atomic.h   |  210 +++
>  platform/linux-generic/arch/arm/odp_cpu.h  |   65 +
>  platform/linux-generic/arch/arm/odp_cpu_idling.h   |   51 +
>  platform/linux-generic/arch/arm/odp_llsc.h |  249 +++
>  platform/linux-generic/arch/default/odp_cpu.h  |   41 +
>  platform/linux-generic/arch/mips64/odp_cpu.h   |   41 +
>  platform/linux-generic/arch/powerpc/odp_cpu.h  |   41 +
>  platform/linux-generic/arch/x86/odp_cpu.h  |   41 +
>  .../include/odp/api/plat/schedule_types.h  |4 +-
>  platform/linux-generic/include/odp_bitset.h|  210 +++
>  .../linux-generic/include/odp_config_internal.h|   17 +-
>  platform/linux-generic/include/odp_llqueue.h   |  309 +++
>  .../include/odp_queue_scalable_internal.h  |  102 +
>  platform/linux-generic/include/odp_schedule_if.h   |2 +-
>  .../linux-generic/include/odp_schedule_scalable.h  |  137 ++
>  .../include/odp_schedule_scalable_config.h |   55 +
>  .../include/odp_schedule_scalable_ordered.h|  132 ++
>  platform/linux-generic/m4/odp_schedule.m4  |   55 +-
>  platform/linux-generic/odp_queue_if.c  |8 +
>  platform/linux-generic/odp_queue_scalable.c| 1020 ++
>  platform/linux-generic/odp_schedule_if.c   |6 +
>  platform/linux-generic/odp_schedule_scalable.c | 1978 
> 
>  

[lng-odp] [API-NEXT PATCH v9 0/6] A scalable software scheduler

[Brian Brooks]

This work derives from Ola Liljedahl's prototype [1] which introduced a
scalable scheduler design based on primarily lock-free algorithms and
data structures designed to decrease contention. A thread searches
through a data structure containing only queues that are both non-empty
and allowed to be scheduled to that thread. Strict priority scheduling is
respected, and (W)RR scheduling may be used within queues of the same priority.
Lastly, pre-scheduling or stashing is not employed since it is optional
functionality that can be implemented in the application.

In addition to scalable ring buffers, the algorithm also uses unbounded
concurrent queues. LL/SC and CAS variants exist in cases where absense of
ABA problem cannot be proved, and also in cases where the compiler's atomic
built-ins may not be lowered to the desired instruction(s). Finally, a version
of the algorithm that uses locks is also provided.

Use --enable-schedule-scalable to conditionally compile this scheduler
into the library.

[1] https://lists.linaro.org/pipermail/lng-odp/2016-September/025682.html

On checkpatch.pl:
 - [2/6] and [5/6] have checkpatch.pl issues that are superfluous

v9:
 - Include patch to enable scalable scheduler in Travis CI
 - Fix 'make distcheck'

v8:
 - Reword commit messages

v7:
 - Rebase against new modular queue interface
 - Duplicate arch files under mips64 and powerpc
 - Fix sched->order_lock()
 - Loop until all deferred events have been enqueued
 - Implement ord_enq_multi()
 - Fix ordered_lock/unlock
 - Revert stylistic changes
 - Add default xfactor
 - Remove changes to odp_sched_latency
 - Remove ULL suffix to alleviate Clang build

v6:
 - Move conversions into scalable scheduler to alleviate #ifdefs
 - Remove unnecessary prefetch
 - Fix ARMv8 build

v5:
 - Allocate cache aligned memory using shm pool APIs
 - Move more code to scalable scheduler specific files
 - Remove CONFIG_SPLIT_READWRITE
 - Fix 'make distcheck' issue

v4:
 - Fix a couple more checkpatch.pl issues

v3:
 - Only conditionally compile scalable scheduler and queue
 - Move some code to arch/ dir
 - Use a single shm block for queues instead of block-per-queue
 - De-interleave odp_llqueue.h
 - Use compiler macros to determine ATOM_BITSET_SIZE
 - Incorporated queue size changes
 - Dropped 'ODP_' prefix on config and moved to other files
 - Dropped a few patches that were send independently to the list

v2:
 - Move ARMv8 issues and other fixes into separate patches
 - Abstract away some #ifdefs
 - Fix some checkpatch.pl warnings

Brian Brooks (5):
  test: odp_pktio_ordered: add queue size
  linux-gen: sched scalable: add arch files
  linux-gen: sched scalable: add a bitset
  linux-gen: sched scalable: add a concurrent queue
  linux-gen: sched scalable: add scalable scheduler

Honnappa Nagarahalli (1):
  travis: add scalable scheduler in CI

 .travis.yml|1 +
 configure.ac   |1 +
 platform/linux-generic/Makefile.am |   17 +
 platform/linux-generic/arch/arm/odp_atomic.h   |  210 +++
 platform/linux-generic/arch/arm/odp_cpu.h  |   65 +
 platform/linux-generic/arch/arm/odp_cpu_idling.h   |   51 +
 platform/linux-generic/arch/arm/odp_llsc.h |  249 +++
 platform/linux-generic/arch/default/odp_cpu.h  |   41 +
 platform/linux-generic/arch/mips64/odp_cpu.h   |   41 +
 platform/linux-generic/arch/powerpc/odp_cpu.h  |   41 +
 platform/linux-generic/arch/x86/odp_cpu.h  |   41 +
 .../include/odp/api/plat/schedule_types.h  |4 +-
 platform/linux-generic/include/odp_bitset.h|  210 +++
 .../linux-generic/include/odp_config_internal.h|   17 +-
 platform/linux-generic/include/odp_llqueue.h   |  309 +++
 .../include/odp_queue_scalable_internal.h  |  102 +
 platform/linux-generic/include/odp_schedule_if.h   |2 +-
 .../linux-generic/include/odp_schedule_scalable.h  |  137 ++
 .../include/odp_schedule_scalable_config.h |   55 +
 .../include/odp_schedule_scalable_ordered.h|  132 ++
 platform/linux-generic/m4/odp_schedule.m4  |   55 +-
 platform/linux-generic/odp_queue_if.c  |8 +
 platform/linux-generic/odp_queue_scalable.c| 1020 ++
 platform/linux-generic/odp_schedule_if.c   |6 +
 platform/linux-generic/odp_schedule_scalable.c | 1978 
 .../linux-generic/odp_schedule_scalable_ordered.c  |  347 
 test/common_plat/performance/odp_pktio_ordered.c   |4 +
 27 files changed, 5122 insertions(+), 22 deletions(-)
 create mode 100644 platform/linux-generic/arch/arm/odp_atomic.h
 create mode 100644 platform/linux-generic/arch/arm/odp_cpu.h
 create mode 100644 platform/linux-generic/arch/arm/odp_cpu_idling.h
 create mode 100644 platform/linux-generic/arch/arm/odp_llsc.h
 create mode 100644 platform/linux-generic/arch/default/odp_cpu.h
 create mode 100644 platform/linux-generic/arch/mips64/odp_cpu.h
 create