On 2014-05-21 16:00, Joel Sherrill wrote:
Hi
We have an SMP behavioral decision to make and it isn't getting
enough discussion.
With cluster scheduling, there are potentially multiple scheduler
instances associated with non-overlapping subsets of cores.
With affinity, a thread can be restricted to execute on a subset
of the cores associated with a scheduler instance.
There are operations to change the scheduler associated with
a thread and the affinity of a thread.
The question is whether changing affinity should be able to
implicitly change the scheduler instance?
I lean to no because affinity and scheduler changes should
should be so rare in practical use that nothing should be
implicit.
The pthread_setaffinity_np() is the only way to set the scheduler via the
non-portable POSIX API. So I would keep the ability to change the scheduler
with it.
How would you change the scheduler with the non-portable POSIX API?
Consider this scenario:
Scheduler A: cores 0-1
Scheduler B: cores 2-3
Thread 1 is associated with Scheduler B and with affinity 2-3
can run on either processor scheduled by B.
Thread 1 changes affinity 1-3. Should this change the scheduler,
be an error, or just have an affinity for a core in the system
that is not scheduled by this scheduler instance?
This is currently an error:
http://git.rtems.org/rtems/tree/testsuites/smptests/smpscheduler02/init.c#n141
If you look at the current code for _Scheduler_Set_affinity(),
it looks like the current behavior is none of the above and
appears to just be broken. Scheduler A's set affinity operation
is invoked and Scheduler B is not informed that it no longer
has control of Thread 1.
It is informed in case _Scheduler_default_Set_affinity_body() is used. What is
broken is the _Scheduler_priority_affinity_SMP_Set_affinity() function.
Please have a look at the attached patch which I already sent to a similar
thread.
--
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
>From 4c7a22c3b110856b3654b3ad8d29acc98448b284 Mon Sep 17 00:00:00 2001
From: Sebastian Huber <sebastian.hu...@embedded-brains.de>
Date: Wed, 21 May 2014 09:07:51 +0200
Subject: [PATCH] score: _Scheduler_Set_affinity()
Check that the CPU set doesn't overlap two or more scheduler instances
in _Scheduler_Set_affinity() and remove this check in
_Scheduler_default_Set_affinity().
Allow that CPUs are set outside of the processor set of the system in
_Scheduler_default_Set_affinity(). This avoids errors in case an
application is used on systems with different processor sets.
---
cpukit/score/include/rtems/score/schedulerimpl.h | 28 ++++------
cpukit/score/src/schedulersetaffinity.c | 61 +++++++++++++---------
2 files changed, 48 insertions(+), 41 deletions(-)
diff --git a/cpukit/score/include/rtems/score/schedulerimpl.h b/cpukit/score/include/rtems/score/schedulerimpl.h
index 15a4207..401cef9 100644
--- a/cpukit/score/include/rtems/score/schedulerimpl.h
+++ b/cpukit/score/include/rtems/score/schedulerimpl.h
@@ -419,35 +419,31 @@ RTEMS_INLINE_ROUTINE bool _Scheduler_default_Set_affinity_body(
const cpu_set_t *cpuset
)
{
- size_t cpu_max = _CPU_set_Maximum_CPU_count( cpusetsize );
+#if defined(RTEMS_SMP)
uint32_t cpu_count = _SMP_Get_processor_count();
uint32_t cpu_index;
bool ok = true;
+ /* Check that the CPU set covers all processors of this scheduler instance */
for ( cpu_index = 0 ; cpu_index < cpu_count ; ++cpu_index ) {
-#if defined(RTEMS_SMP)
const Scheduler_Control *scheduler_of_cpu =
_Scheduler_Get_by_CPU_index( cpu_index );
- ok = ok
- && ( ( CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset )
- && scheduler == scheduler_of_cpu )
- || ( !CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset )
- && scheduler != scheduler_of_cpu ) );
-#else
- (void) scheduler;
-
- ok = ok && CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset );
-#endif
+ if ( !CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset ) ) {
+ ok = ok && scheduler != scheduler_of_cpu;
+ }
}
- for ( ; cpu_index < cpu_max ; ++cpu_index ) {
- ok = ok && !CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset );
+ if ( ok ) {
+ _Scheduler_Set( scheduler, the_thread );
}
- _Scheduler_Set( scheduler, the_thread );
-
return ok;
+#else
+ (void) scheduler;
+
+ return CPU_ISSET_S( 0, cpusetsize, cpuset );
+#endif
}
bool _Scheduler_Set_affinity(
diff --git a/cpukit/score/src/schedulersetaffinity.c b/cpukit/score/src/schedulersetaffinity.c
index a20888b..b4fea3c 100644
--- a/cpukit/score/src/schedulersetaffinity.c
+++ b/cpukit/score/src/schedulersetaffinity.c
@@ -26,45 +26,56 @@ bool _Scheduler_Set_affinity(
const cpu_set_t *cpuset
)
{
- bool ok;
-
- if ( _CPU_set_Is_large_enough( cpusetsize ) ) {
#if defined(RTEMS_SMP)
- uint32_t cpu_count = _SMP_Get_processor_count();
- uint32_t cpu_index;
+ uint32_t cpu_count = _SMP_Get_processor_count();
+ uint32_t cpu_index;
+ const Scheduler_Control *scheduler_of_cpuset = NULL;
+ bool ok = false;
+#endif
- ok = false;
+ if ( !_CPU_set_Is_large_enough( cpusetsize ) ) {
+ return false;
+ }
- for ( cpu_index = 0 ; cpu_index < cpu_count ; ++cpu_index ) {
- if ( CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset ) ) {
- const Scheduler_Control *scheduler_of_cpu =
- _Scheduler_Get_by_CPU_index( cpu_index );
+#if defined(RTEMS_SMP)
+ /*
+ * Check that the CPU set doesn't overlap with two or more scheduler
+ * instances.
+ */
+ for ( cpu_index = 0 ; cpu_index < cpu_count ; ++cpu_index ) {
+ if ( CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset ) ) {
+ const Scheduler_Control *scheduler_of_cpu =
+ _Scheduler_Get_by_CPU_index( cpu_index );
- if ( scheduler_of_cpu != NULL ) {
- ok = ( *scheduler_of_cpu->Operations.set_affinity )(
- scheduler_of_cpu,
- the_thread,
- cpusetsize,
- cpuset
- );
+ if ( scheduler_of_cpu != NULL ) {
+ if ( scheduler_of_cpuset == NULL ) {
+ scheduler_of_cpuset = scheduler_of_cpu;
+ ok = true;
+ } else {
+ ok = ok && scheduler_of_cpu == scheduler_of_cpuset;
}
-
- break;
}
}
-#else
- ok = _Scheduler_default_Set_affinity_body(
- _Scheduler_Get( the_thread ),
+ }
+
+ if ( ok ) {
+ ok = ( *scheduler_of_cpuset->Operations.set_affinity )(
+ scheduler_of_cpuset,
the_thread,
cpusetsize,
cpuset
);
-#endif
- } else {
- ok = false;
}
return ok;
+#else
+ return _Scheduler_default_Set_affinity_body(
+ _Scheduler_Get( the_thread ),
+ the_thread,
+ cpusetsize,
+ cpuset
+ );
+#endif
}
#endif /* defined(__RTEMS_HAVE_SYS_CPUSET_H__) */
--
1.7.7
_______________________________________________
rtems-devel mailing list
rtems-devel@rtems.org
http://www.rtems.org/mailman/listinfo/rtems-devel
