Re: [PATCH] Fix migration of SCHED_DEADLINE tasks

[Juri Lelli]

Hi,

On 16/10/15 09:03, Luca Abeni wrote:
> On 10/15/2015 06:40 PM, Juri Lelli wrote:
>> On 15/10/15 12:09, Luca Abeni wrote:
>>> Commit 9d5142624256 ("sched/deadline: Reduce rq lock contention by
>>> eliminating locking of non-feasible target") broke select_task_rq_dl()
> [...]
>>> -dl_time_before(p->dl.deadline,
>>> -cpu_rq(target)->dl.earliest_dl.curr))
>>> +(dl_time_before(p->dl.deadline,
>>> +cpu_rq(target)->dl.earliest_dl.curr) ||
>>> +(cpu_rq(target)->dl.earliest_dl.curr == 0)))
>>
>> Can't we actually use dl.dl_nr_running here and below, so
>> that we won't incur any wraparound problem?
> Ok, I tested the patch with dl.dl_nr_running and if works for me...
> 
> I am going to send the updated patch in few minutes.
> 

Thanks!

> BTW, should we also use "dl_rq->dl_nr_running == 0" instead of
> "dl_rq->earliest_dl.curr == 0" in inc_dl_deadline(), and remove the
> comment from init_dl_rq()? If you think it is a good idea, I'll test this
> additional change and send a patch in next week.
> 

Yeah, it seems we need that fix too.

Best,

- Juri

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Fix migration of SCHED_DEADLINE tasks

[Luca Abeni]

On 10/15/2015 06:40 PM, Juri Lelli wrote:

On 15/10/15 12:09, Luca Abeni wrote:

Commit 9d5142624256 ("sched/deadline: Reduce rq lock contention by
eliminating locking of non-feasible target") broke select_task_rq_dl()

[...]

-   dl_time_before(p->dl.deadline,
-   cpu_rq(target)->dl.earliest_dl.curr))
+   (dl_time_before(p->dl.deadline,
+   cpu_rq(target)->dl.earliest_dl.curr) ||
+   (cpu_rq(target)->dl.earliest_dl.curr == 0)))


Can't we actually use dl.dl_nr_running here and below, so
that we won't incur any wraparound problem?

Ok, I tested the patch with dl.dl_nr_running and if works for me...

I am going to send the updated patch in few minutes.

BTW, should we also use "dl_rq->dl_nr_running == 0" instead of
"dl_rq->earliest_dl.curr == 0" in inc_dl_deadline(), and remove the
comment from init_dl_rq()? If you think it is a good idea, I'll test this
additional change and send a patch in next week.



Thanks,
Luca
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Fix migration of SCHED_DEADLINE tasks

[Luca Abeni]

On 10/15/2015 06:40 PM, Juri Lelli wrote:

On 15/10/15 12:09, Luca Abeni wrote:

Commit 9d5142624256 ("sched/deadline: Reduce rq lock contention by
eliminating locking of non-feasible target") broke select_task_rq_dl()

[...]

-   dl_time_before(p->dl.deadline,
-   cpu_rq(target)->dl.earliest_dl.curr))
+   (dl_time_before(p->dl.deadline,
+   cpu_rq(target)->dl.earliest_dl.curr) ||
+   (cpu_rq(target)->dl.earliest_dl.curr == 0)))


Can't we actually use dl.dl_nr_running here and below, so
that we won't incur any wraparound problem?

Ok, I tested the patch with dl.dl_nr_running and if works for me...

I am going to send the updated patch in few minutes.

BTW, should we also use "dl_rq->dl_nr_running == 0" instead of
"dl_rq->earliest_dl.curr == 0" in inc_dl_deadline(), and remove the
comment from init_dl_rq()? If you think it is a good idea, I'll test this
additional change and send a patch in next week.



Thanks,
Luca
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Fix migration of SCHED_DEADLINE tasks

[Juri Lelli]

Hi,

On 16/10/15 09:03, Luca Abeni wrote:
> On 10/15/2015 06:40 PM, Juri Lelli wrote:
>> On 15/10/15 12:09, Luca Abeni wrote:
>>> Commit 9d5142624256 ("sched/deadline: Reduce rq lock contention by
>>> eliminating locking of non-feasible target") broke select_task_rq_dl()
> [...]
>>> -dl_time_before(p->dl.deadline,
>>> -cpu_rq(target)->dl.earliest_dl.curr))
>>> +(dl_time_before(p->dl.deadline,
>>> +cpu_rq(target)->dl.earliest_dl.curr) ||
>>> +(cpu_rq(target)->dl.earliest_dl.curr == 0)))
>>
>> Can't we actually use dl.dl_nr_running here and below, so
>> that we won't incur any wraparound problem?
> Ok, I tested the patch with dl.dl_nr_running and if works for me...
> 
> I am going to send the updated patch in few minutes.
> 

Thanks!

> BTW, should we also use "dl_rq->dl_nr_running == 0" instead of
> "dl_rq->earliest_dl.curr == 0" in inc_dl_deadline(), and remove the
> comment from init_dl_rq()? If you think it is a good idea, I'll test this
> additional change and send a patch in next week.
> 

Yeah, it seems we need that fix too.

Best,

- Juri

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Fix migration of SCHED_DEADLINE tasks

[Luca Abeni]

Hi Juri,

On Thu, 15 Oct 2015 17:40:19 +0100
Juri Lelli  wrote:
> On 15/10/15 12:09, Luca Abeni wrote:
> > Commit 9d5142624256 ("sched/deadline: Reduce rq lock contention by
> > eliminating locking of non-feasible target") broke
[...]
> > cpu_rq(target)->dl.earliest_dl.curr))
> > +   (dl_time_before(p->dl.deadline,
> > +
> > cpu_rq(target)->dl.earliest_dl.curr) ||
> > +
> > (cpu_rq(target)->dl.earliest_dl.curr == 0)))
> 
> Can't we actually use dl.dl_nr_running here and below, so
> that we won't incur any wraparound problem?
I copied the "earliest_dl.curr == 0" check from inc_dl_deadline():
if (dl_rq->earliest_dl.curr == 0 ||
dl_time_before(deadline, dl_rq->earliest_dl.curr)) {
/*
 * If the dl_rq had no -deadline tasks, or if the new
   task
 * has shorter deadline than the current one on dl_rq,
...

And init_dl_rq() has a comment saying "zero means no -deadline tasks"...
But now I see what you mean: actually, find_lock_later_rq() contains
the correct version of the check few lines below the wrong check (after
acquiring the rq lock).

Tomorrow I'll try the version of the check with 
later_rq->dl.dl_nr_running, and if it works I'll send an updated patch.



Thanks,
Luca
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Fix migration of SCHED_DEADLINE tasks

[Juri Lelli]

Hi Luca,

On 15/10/15 12:09, Luca Abeni wrote:
> Commit 9d5142624256 ("sched/deadline: Reduce rq lock contention by
> eliminating locking of non-feasible target") broke select_task_rq_dl()
> and find_lock_later_rq(), because it introduced a comparison between
> the local task's deadline and dl.earliest_dl.curr of the remote queue.
> However, if the remote runqueue does not contain any SCHED_DEADLINE
> task its earliest_dl.curr is 0 (always smaller than the deadline of
> the local task) and the remote runqueue is not selected for pushing.
> As a result, if an application creates multiple SCHED_DEADLINE threads,
> they will never be pushed to runqueues that do not already contain
> SCHED_DEADLINE tasks.
> This patches fixes the issue by checking if dl.earliest_dl.curr == 0.
> 
> Signed-off-by: Luca Abeni 
> ---
>  kernel/sched/deadline.c | 8 +---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index fc8f010..0d86d60 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -1066,8 +1066,9 @@ select_task_rq_dl(struct task_struct *p, int cpu, int 
> sd_flag, int flags)
>   int target = find_later_rq(p);
>  
>   if (target != -1 &&
> - dl_time_before(p->dl.deadline,
> - cpu_rq(target)->dl.earliest_dl.curr))
> + (dl_time_before(p->dl.deadline,
> + cpu_rq(target)->dl.earliest_dl.curr) ||
> + (cpu_rq(target)->dl.earliest_dl.curr == 0)))

Can't we actually use dl.dl_nr_running here and below, so
that we won't incur any wraparound problem?

Thanks,

- Juri

>   cpu = target;
>   }
>   rcu_read_unlock();
> @@ -1417,7 +1418,8 @@ static struct rq *find_lock_later_rq(struct task_struct 
> *task, struct rq *rq)
>  
>   later_rq = cpu_rq(cpu);
>  
> - if (!dl_time_before(task->dl.deadline,
> + if (later_rq->dl.earliest_dl.curr &&
> + !dl_time_before(task->dl.deadline,
>   later_rq->dl.earliest_dl.curr)) {
>   /*
>* Target rq has tasks of equal or earlier deadline,
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Fix migration of SCHED_DEADLINE tasks

[Luca Abeni]

Hi Juri,

On Thu, 15 Oct 2015 17:40:19 +0100
Juri Lelli  wrote:
> On 15/10/15 12:09, Luca Abeni wrote:
> > Commit 9d5142624256 ("sched/deadline: Reduce rq lock contention by
> > eliminating locking of non-feasible target") broke
[...]
> > cpu_rq(target)->dl.earliest_dl.curr))
> > +   (dl_time_before(p->dl.deadline,
> > +
> > cpu_rq(target)->dl.earliest_dl.curr) ||
> > +
> > (cpu_rq(target)->dl.earliest_dl.curr == 0)))
> 
> Can't we actually use dl.dl_nr_running here and below, so
> that we won't incur any wraparound problem?
I copied the "earliest_dl.curr == 0" check from inc_dl_deadline():
if (dl_rq->earliest_dl.curr == 0 ||
dl_time_before(deadline, dl_rq->earliest_dl.curr)) {
/*
 * If the dl_rq had no -deadline tasks, or if the new
   task
 * has shorter deadline than the current one on dl_rq,
...

And init_dl_rq() has a comment saying "zero means no -deadline tasks"...
But now I see what you mean: actually, find_lock_later_rq() contains
the correct version of the check few lines below the wrong check (after
acquiring the rq lock).

Tomorrow I'll try the version of the check with 
later_rq->dl.dl_nr_running, and if it works I'll send an updated patch.



Thanks,
Luca
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Fix migration of SCHED_DEADLINE tasks

[Juri Lelli]

Hi Luca,

On 15/10/15 12:09, Luca Abeni wrote:
> Commit 9d5142624256 ("sched/deadline: Reduce rq lock contention by
> eliminating locking of non-feasible target") broke select_task_rq_dl()
> and find_lock_later_rq(), because it introduced a comparison between
> the local task's deadline and dl.earliest_dl.curr of the remote queue.
> However, if the remote runqueue does not contain any SCHED_DEADLINE
> task its earliest_dl.curr is 0 (always smaller than the deadline of
> the local task) and the remote runqueue is not selected for pushing.
> As a result, if an application creates multiple SCHED_DEADLINE threads,
> they will never be pushed to runqueues that do not already contain
> SCHED_DEADLINE tasks.
> This patches fixes the issue by checking if dl.earliest_dl.curr == 0.
> 
> Signed-off-by: Luca Abeni 
> ---
>  kernel/sched/deadline.c | 8 +---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index fc8f010..0d86d60 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -1066,8 +1066,9 @@ select_task_rq_dl(struct task_struct *p, int cpu, int 
> sd_flag, int flags)
>   int target = find_later_rq(p);
>  
>   if (target != -1 &&
> - dl_time_before(p->dl.deadline,
> - cpu_rq(target)->dl.earliest_dl.curr))
> + (dl_time_before(p->dl.deadline,
> + cpu_rq(target)->dl.earliest_dl.curr) ||
> + (cpu_rq(target)->dl.earliest_dl.curr == 0)))

Can't we actually use dl.dl_nr_running here and below, so
that we won't incur any wraparound problem?

Thanks,

- Juri

>   cpu = target;
>   }
>   rcu_read_unlock();
> @@ -1417,7 +1418,8 @@ static struct rq *find_lock_later_rq(struct task_struct 
> *task, struct rq *rq)
>  
>   later_rq = cpu_rq(cpu);
>  
> - if (!dl_time_before(task->dl.deadline,
> + if (later_rq->dl.earliest_dl.curr &&
> + !dl_time_before(task->dl.deadline,
>   later_rq->dl.earliest_dl.curr)) {
>   /*
>* Target rq has tasks of equal or earlier deadline,
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/