Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On 2014/7/8 15:49, Peter Zijlstra wrote: On Tue, Jul 08, 2014 at 10:51:02AM +0800, xiaofeng.yan wrote: On 2014/7/8 10:40, Li Zefan wrote: On 2014/7/8 9:10, xiaofeng.yan wrote: On 2014/7/7 16:41, Peter Zijlstra wrote: On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: It could be wrong for the precision of runtime and deadline when the precision is within microsecond level. For example: Task runtime deadline period P1 200us 500us 500us This case need enbale HRTICK feature by the next command PC#echo "HRTICK" > /sys/kernel/debug/sched_features PC#./schedtool -E -t 20:50 -e ./test& PC#trace-cmd record -e sched_switch Are you actually using HRTICK ? yes, If HRTICK is close , then all of runtime and deadline will be wrong. I think what peter meant is, do you use HRTICK in products or just use it for testing/experiment? Thanks for your timely comments. In fact, We use HRTICK feature in product. We need microsecond level precision. Ah, thanks. Be advised that currently HRTICK is rather expensive. The cost is twofold: 1) doing all the kernel side hrtimer things and 2) programming clock hardware. Of course, if that's what you need, you're willing to pay the price. I'll see if I can put making it less expensive slightly higher on the (endless) todo list. another fold: 3) Frequent migration :) In fact, frequent migration lead to higher overload. In our product we design new migration solution. The simple description is as follow: 1 Set affinity in user space program at the beginning 2 Migrate happen per 100ms 3 Free task (runtime < dl_runtime)is migrated to free cpu (rt task bandwidth < 65%) . 4 Busy task will run more time in a CPU by dynamic quota. So the condition of migration depends on whether task is busy or idle or not instead of deadline at some point. This could not meet EDF requirement but meet our product :) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On Tue, Jul 08, 2014 at 10:51:02AM +0800, xiaofeng.yan wrote: > On 2014/7/8 10:40, Li Zefan wrote: > >On 2014/7/8 9:10, xiaofeng.yan wrote: > >>On 2014/7/7 16:41, Peter Zijlstra wrote: > >>>On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: > It could be wrong for the precision of runtime and deadline > when the precision is within microsecond level. For example: > Task runtime deadline period > P1 200us 500us 500us > > This case need enbale HRTICK feature by the next command > PC#echo "HRTICK" > /sys/kernel/debug/sched_features > PC#./schedtool -E -t 20:50 -e ./test& > PC#trace-cmd record -e sched_switch > >>>Are you actually using HRTICK ? > >>yes, If HRTICK is close , then all of runtime and deadline will be wrong. > >I think what peter meant is, do you use HRTICK in products or > >just use it for testing/experiment? > > > Thanks for your timely comments. In fact, We use HRTICK feature in product. > We need microsecond level > precision. Ah, thanks. Be advised that currently HRTICK is rather expensive. The cost is twofold: 1) doing all the kernel side hrtimer things and 2) programming clock hardware. Of course, if that's what you need, you're willing to pay the price. I'll see if I can put making it less expensive slightly higher on the (endless) todo list. pgpGe_JxT2vtS.pgp Description: PGP signature
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On Tue, Jul 08, 2014 at 10:51:02AM +0800, xiaofeng.yan wrote: On 2014/7/8 10:40, Li Zefan wrote: On 2014/7/8 9:10, xiaofeng.yan wrote: On 2014/7/7 16:41, Peter Zijlstra wrote: On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: It could be wrong for the precision of runtime and deadline when the precision is within microsecond level. For example: Task runtime deadline period P1 200us 500us 500us This case need enbale HRTICK feature by the next command PC#echo HRTICK /sys/kernel/debug/sched_features PC#./schedtool -E -t 20:50 -e ./test PC#trace-cmd record -e sched_switch Are you actually using HRTICK ? yes, If HRTICK is close , then all of runtime and deadline will be wrong. I think what peter meant is, do you use HRTICK in products or just use it for testing/experiment? Thanks for your timely comments. In fact, We use HRTICK feature in product. We need microsecond level precision. Ah, thanks. Be advised that currently HRTICK is rather expensive. The cost is twofold: 1) doing all the kernel side hrtimer things and 2) programming clock hardware. Of course, if that's what you need, you're willing to pay the price. I'll see if I can put making it less expensive slightly higher on the (endless) todo list. pgpGe_JxT2vtS.pgp Description: PGP signature
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On 2014/7/8 15:49, Peter Zijlstra wrote: On Tue, Jul 08, 2014 at 10:51:02AM +0800, xiaofeng.yan wrote: On 2014/7/8 10:40, Li Zefan wrote: On 2014/7/8 9:10, xiaofeng.yan wrote: On 2014/7/7 16:41, Peter Zijlstra wrote: On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: It could be wrong for the precision of runtime and deadline when the precision is within microsecond level. For example: Task runtime deadline period P1 200us 500us 500us This case need enbale HRTICK feature by the next command PC#echo HRTICK /sys/kernel/debug/sched_features PC#./schedtool -E -t 20:50 -e ./test PC#trace-cmd record -e sched_switch Are you actually using HRTICK ? yes, If HRTICK is close , then all of runtime and deadline will be wrong. I think what peter meant is, do you use HRTICK in products or just use it for testing/experiment? Thanks for your timely comments. In fact, We use HRTICK feature in product. We need microsecond level precision. Ah, thanks. Be advised that currently HRTICK is rather expensive. The cost is twofold: 1) doing all the kernel side hrtimer things and 2) programming clock hardware. Of course, if that's what you need, you're willing to pay the price. I'll see if I can put making it less expensive slightly higher on the (endless) todo list. another fold: 3) Frequent migration :) In fact, frequent migration lead to higher overload. In our product we design new migration solution. The simple description is as follow: 1 Set affinity in user space program at the beginning 2 Migrate happen per 100ms 3 Free task (runtime dl_runtime)is migrated to free cpu (rt task bandwidth 65%) . 4 Busy task will run more time in a CPU by dynamic quota. So the condition of migration depends on whether task is busy or idle or not instead of deadline at some point. This could not meet EDF requirement but meet our product :) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On 2014/7/8 10:40, Li Zefan wrote: On 2014/7/8 9:10, xiaofeng.yan wrote: On 2014/7/7 16:41, Peter Zijlstra wrote: On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: It could be wrong for the precision of runtime and deadline when the precision is within microsecond level. For example: Task runtime deadline period P1 200us 500us 500us This case need enbale HRTICK feature by the next command PC#echo "HRTICK" > /sys/kernel/debug/sched_features PC#./schedtool -E -t 20:50 -e ./test& PC#trace-cmd record -e sched_switch Are you actually using HRTICK ? yes, If HRTICK is close , then all of runtime and deadline will be wrong. I think what peter meant is, do you use HRTICK in products or just use it for testing/experiment? Thanks for your timely comments. In fact, We use HRTICK feature in product. We need microsecond level precision. Thanks Yan . -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On 2014/7/8 9:10, xiaofeng.yan wrote: > On 2014/7/7 16:41, Peter Zijlstra wrote: >> On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: >>> It could be wrong for the precision of runtime and deadline >>> when the precision is within microsecond level. For example: >>> Task runtime deadline period >>> P1 200us 500us 500us >>> >>> This case need enbale HRTICK feature by the next command >>> PC#echo "HRTICK" > /sys/kernel/debug/sched_features >>> PC#./schedtool -E -t 20:50 -e ./test& >>> PC#trace-cmd record -e sched_switch >> Are you actually using HRTICK ? > yes, If HRTICK is close , then all of runtime and deadline will be wrong. I think what peter meant is, do you use HRTICK in products or just use it for testing/experiment? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On 2014/7/7 16:41, Peter Zijlstra wrote: On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: It could be wrong for the precision of runtime and deadline when the precision is within microsecond level. For example: Task runtime deadline period P1 200us 500us 500us This case need enbale HRTICK feature by the next command PC#echo "HRTICK" > /sys/kernel/debug/sched_features PC#./schedtool -E -t 20:50 -e ./test& PC#trace-cmd record -e sched_switch Are you actually using HRTICK ? yes, If HRTICK is close , then all of runtime and deadline will be wrong. Some of runtime and deadline run with millisecond level by reading kernershark. The problem is caused by a conditional judgment "delta > 1". Because no hrtimer start up to control the runtime when runtime is less than 10us. So the process will continue to run until tick-period coming. For fixing this problem, Let delta is equal to 10us when it is less than 10us. So the hrtimer will start up to control the end of process. Signed-off-by: xiaofeng.yan Always when sending patches for deadline, also CC Juri. --- kernel/sched/deadline.c |6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index fc4f98b1..dfefa82 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -997,10 +997,8 @@ static void check_preempt_curr_dl(struct rq *rq, struct task_struct *p, #ifdef CONFIG_SCHED_HRTICK static void start_hrtick_dl(struct rq *rq, struct task_struct *p) { - s64 delta = p->dl.dl_runtime - p->dl.runtime; - - if (delta > 1) - hrtick_start(rq, p->dl.runtime); + s64 delta = p->dl.runtime > 1 ? p->dl.runtime : 1; + hrtick_start(rq, delta); Yeah, that looks funny. And seeing how the only other user does something similar: hrtick_start_fair() delta = max(1ULL, delta) hrtick_start(rq, delta) I will modify my code according to your suggest. Does it make sense to move this max() into hrtick_start()? Also; and I don't think you mentioned that but did fix, the argument to hrtick_start() is wrong, it should be the delta, not the absolute timeout. Perhaps , if the runtime is less than 10us, the context switch overhead for system could be closed to 10us. So it could loss more then you gain. Thanks for your reply. Thanks Yan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: > It could be wrong for the precision of runtime and deadline > when the precision is within microsecond level. For example: > Task runtime deadline period > P1 200us 500us 500us > > This case need enbale HRTICK feature by the next command > PC#echo "HRTICK" > /sys/kernel/debug/sched_features > PC#./schedtool -E -t 20:50 -e ./test& > PC#trace-cmd record -e sched_switch Are you actually using HRTICK ? > Some of runtime and deadline run with millisecond level by > reading kernershark. > The problem is caused by a conditional judgment "delta > 1". > Because no hrtimer start up to control the runtime when runtime is less than > 10us. > So the process will continue to run until tick-period coming. > For fixing this problem, Let delta is equal to 10us when it is less than 10us. > So the hrtimer will start up to control the end of process. > > Signed-off-by: xiaofeng.yan Always when sending patches for deadline, also CC Juri. > --- > kernel/sched/deadline.c |6 ++ > 1 file changed, 2 insertions(+), 4 deletions(-) > > diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c > index fc4f98b1..dfefa82 100644 > --- a/kernel/sched/deadline.c > +++ b/kernel/sched/deadline.c > @@ -997,10 +997,8 @@ static void check_preempt_curr_dl(struct rq *rq, struct > task_struct *p, > #ifdef CONFIG_SCHED_HRTICK > static void start_hrtick_dl(struct rq *rq, struct task_struct *p) > { > - s64 delta = p->dl.dl_runtime - p->dl.runtime; > - > - if (delta > 1) > - hrtick_start(rq, p->dl.runtime); > + s64 delta = p->dl.runtime > 1 ? p->dl.runtime : 1; > + hrtick_start(rq, delta); Yeah, that looks funny. And seeing how the only other user does something similar: hrtick_start_fair() delta = max(1ULL, delta) hrtick_start(rq, delta) Does it make sense to move this max() into hrtick_start()? Also; and I don't think you mentioned that but did fix, the argument to hrtick_start() is wrong, it should be the delta, not the absolute timeout. pgpV7gidRiP8O.pgp Description: PGP signature
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: It could be wrong for the precision of runtime and deadline when the precision is within microsecond level. For example: Task runtime deadline period P1 200us 500us 500us This case need enbale HRTICK feature by the next command PC#echo HRTICK /sys/kernel/debug/sched_features PC#./schedtool -E -t 20:50 -e ./test PC#trace-cmd record -e sched_switch Are you actually using HRTICK ? Some of runtime and deadline run with millisecond level by reading kernershark. The problem is caused by a conditional judgment delta 1. Because no hrtimer start up to control the runtime when runtime is less than 10us. So the process will continue to run until tick-period coming. For fixing this problem, Let delta is equal to 10us when it is less than 10us. So the hrtimer will start up to control the end of process. Signed-off-by: xiaofeng.yan xiaofeng@huawei.com Always when sending patches for deadline, also CC Juri. --- kernel/sched/deadline.c |6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index fc4f98b1..dfefa82 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -997,10 +997,8 @@ static void check_preempt_curr_dl(struct rq *rq, struct task_struct *p, #ifdef CONFIG_SCHED_HRTICK static void start_hrtick_dl(struct rq *rq, struct task_struct *p) { - s64 delta = p-dl.dl_runtime - p-dl.runtime; - - if (delta 1) - hrtick_start(rq, p-dl.runtime); + s64 delta = p-dl.runtime 1 ? p-dl.runtime : 1; + hrtick_start(rq, delta); Yeah, that looks funny. And seeing how the only other user does something similar: hrtick_start_fair() delta = max(1ULL, delta) hrtick_start(rq, delta) Does it make sense to move this max() into hrtick_start()? Also; and I don't think you mentioned that but did fix, the argument to hrtick_start() is wrong, it should be the delta, not the absolute timeout. pgpV7gidRiP8O.pgp Description: PGP signature
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On 2014/7/7 16:41, Peter Zijlstra wrote: On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: It could be wrong for the precision of runtime and deadline when the precision is within microsecond level. For example: Task runtime deadline period P1 200us 500us 500us This case need enbale HRTICK feature by the next command PC#echo HRTICK /sys/kernel/debug/sched_features PC#./schedtool -E -t 20:50 -e ./test PC#trace-cmd record -e sched_switch Are you actually using HRTICK ? yes, If HRTICK is close , then all of runtime and deadline will be wrong. Some of runtime and deadline run with millisecond level by reading kernershark. The problem is caused by a conditional judgment delta 1. Because no hrtimer start up to control the runtime when runtime is less than 10us. So the process will continue to run until tick-period coming. For fixing this problem, Let delta is equal to 10us when it is less than 10us. So the hrtimer will start up to control the end of process. Signed-off-by: xiaofeng.yan xiaofeng@huawei.com Always when sending patches for deadline, also CC Juri. --- kernel/sched/deadline.c |6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index fc4f98b1..dfefa82 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -997,10 +997,8 @@ static void check_preempt_curr_dl(struct rq *rq, struct task_struct *p, #ifdef CONFIG_SCHED_HRTICK static void start_hrtick_dl(struct rq *rq, struct task_struct *p) { - s64 delta = p-dl.dl_runtime - p-dl.runtime; - - if (delta 1) - hrtick_start(rq, p-dl.runtime); + s64 delta = p-dl.runtime 1 ? p-dl.runtime : 1; + hrtick_start(rq, delta); Yeah, that looks funny. And seeing how the only other user does something similar: hrtick_start_fair() delta = max(1ULL, delta) hrtick_start(rq, delta) I will modify my code according to your suggest. Does it make sense to move this max() into hrtick_start()? Also; and I don't think you mentioned that but did fix, the argument to hrtick_start() is wrong, it should be the delta, not the absolute timeout. Perhaps , if the runtime is less than 10us, the context switch overhead for system could be closed to 10us. So it could loss more then you gain. Thanks for your reply. Thanks Yan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On 2014/7/8 9:10, xiaofeng.yan wrote: On 2014/7/7 16:41, Peter Zijlstra wrote: On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: It could be wrong for the precision of runtime and deadline when the precision is within microsecond level. For example: Task runtime deadline period P1 200us 500us 500us This case need enbale HRTICK feature by the next command PC#echo HRTICK /sys/kernel/debug/sched_features PC#./schedtool -E -t 20:50 -e ./test PC#trace-cmd record -e sched_switch Are you actually using HRTICK ? yes, If HRTICK is close , then all of runtime and deadline will be wrong. I think what peter meant is, do you use HRTICK in products or just use it for testing/experiment? -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] sched/rt: overrun could happen in start_hrtick_dl
On 2014/7/8 10:40, Li Zefan wrote: On 2014/7/8 9:10, xiaofeng.yan wrote: On 2014/7/7 16:41, Peter Zijlstra wrote: On Fri, Jul 04, 2014 at 12:02:21PM +, xiaofeng.yan wrote: It could be wrong for the precision of runtime and deadline when the precision is within microsecond level. For example: Task runtime deadline period P1 200us 500us 500us This case need enbale HRTICK feature by the next command PC#echo HRTICK /sys/kernel/debug/sched_features PC#./schedtool -E -t 20:50 -e ./test PC#trace-cmd record -e sched_switch Are you actually using HRTICK ? yes, If HRTICK is close , then all of runtime and deadline will be wrong. I think what peter meant is, do you use HRTICK in products or just use it for testing/experiment? Thanks for your timely comments. In fact, We use HRTICK feature in product. We need microsecond level precision. Thanks Yan . -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/