Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-02-28 Thread Viresh Kumar
On 15 January 2014 17:04, Peter Zijlstra wrote: > On Wed, Jan 15, 2014 at 04:17:26PM +0530, Viresh Kumar wrote: >> On 15 January 2014 16:08, Peter Zijlstra wrote: >> > Nah, its just ugly and we should fix it. You need to be careful to not >> > place tasks in a cpuset you're going to unplug

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-02-28 Thread Viresh Kumar
On 15 January 2014 17:04, Peter Zijlstra pet...@infradead.org wrote: On Wed, Jan 15, 2014 at 04:17:26PM +0530, Viresh Kumar wrote: On 15 January 2014 16:08, Peter Zijlstra pet...@infradead.org wrote: Nah, its just ugly and we should fix it. You need to be careful to not place tasks in a

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-02-13 Thread Frederic Weisbecker
On Tue, Feb 11, 2014 at 02:22:43PM +0530, Viresh Kumar wrote: > On 28 January 2014 18:53, Frederic Weisbecker wrote: > > No, when a single task is running on a full dynticks CPU, the tick is > > supposed to run > > every seconds. I'm actually suprised it doesn't happen in your traces, did > >

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-02-13 Thread Frederic Weisbecker
On Tue, Feb 11, 2014 at 02:22:43PM +0530, Viresh Kumar wrote: On 28 January 2014 18:53, Frederic Weisbecker fweis...@gmail.com wrote: No, when a single task is running on a full dynticks CPU, the tick is supposed to run every seconds. I'm actually suprised it doesn't happen in your traces,

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-02-11 Thread Viresh Kumar
On 28 January 2014 18:53, Frederic Weisbecker wrote: > No, when a single task is running on a full dynticks CPU, the tick is > supposed to run > every seconds. I'm actually suprised it doesn't happen in your traces, did > you tweak > something specific? Why do we need this 1 second tick

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-02-11 Thread Viresh Kumar
On 28 January 2014 18:53, Frederic Weisbecker fweis...@gmail.com wrote: No, when a single task is running on a full dynticks CPU, the tick is supposed to run every seconds. I'm actually suprised it doesn't happen in your traces, did you tweak something specific? Why do we need this 1

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-02-03 Thread Viresh Kumar
On 28 January 2014 21:41, Kevin Hilman wrote: > I think Viresh is using my patch/hack to configure/disable the 1Hz > residual tick. Yeah. I am using sched_tick_max_deferment by setting it to -1. Why do we need a timer every second for NO_HZ_FULL currently? -- To unsubscribe from this list: send

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-02-03 Thread Viresh Kumar
On 28 January 2014 21:41, Kevin Hilman khil...@linaro.org wrote: I think Viresh is using my patch/hack to configure/disable the 1Hz residual tick. Yeah. I am using sched_tick_max_deferment by setting it to -1. Why do we need a timer every second for NO_HZ_FULL currently? -- To unsubscribe from

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-28 Thread Kevin Hilman
On Tue, Jan 28, 2014 at 5:23 AM, Frederic Weisbecker wrote: > On Fri, Jan 24, 2014 at 10:51:14AM +0530, Viresh Kumar wrote: >> On 23 January 2014 20:28, Frederic Weisbecker wrote: >> > On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote: >> >> >> So, the main problem in my case was

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-28 Thread Frederic Weisbecker
On Fri, Jan 24, 2014 at 10:51:14AM +0530, Viresh Kumar wrote: > On 23 January 2014 20:28, Frederic Weisbecker wrote: > > On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote: > > >> So, the main problem in my case was caused by this: > >> > >><...>-2147 [001] d..2

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-28 Thread Frederic Weisbecker
On Fri, Jan 24, 2014 at 10:51:14AM +0530, Viresh Kumar wrote: On 23 January 2014 20:28, Frederic Weisbecker fweis...@gmail.com wrote: On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote: So, the main problem in my case was caused by this: ...-2147 [001] d..2

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-28 Thread Kevin Hilman
On Tue, Jan 28, 2014 at 5:23 AM, Frederic Weisbecker fweis...@gmail.com wrote: On Fri, Jan 24, 2014 at 10:51:14AM +0530, Viresh Kumar wrote: On 23 January 2014 20:28, Frederic Weisbecker fweis...@gmail.com wrote: On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote: So, the main

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-24 Thread Viresh Kumar
On 23 January 2014 19:31, Frederic Weisbecker wrote: > Ok, so it is fine to migrate the latter kind I guess? Unless somebody has abused the API and used bound workqueues where he should have used unbound ones. > I haven't checked the details but then this quiesce option would involve > a

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-24 Thread Mike Galbraith
On Fri, 2014-01-24 at 10:51 +0530, Viresh Kumar wrote: > On 23 January 2014 20:28, Frederic Weisbecker wrote: > > On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote: > > >> So, the main problem in my case was caused by this: > >> > >><...>-2147 [001] d..2 302.573881:

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-24 Thread Mike Galbraith
On Fri, 2014-01-24 at 10:51 +0530, Viresh Kumar wrote: On 23 January 2014 20:28, Frederic Weisbecker fweis...@gmail.com wrote: On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote: So, the main problem in my case was caused by this: ...-2147 [001] d..2 302.573881:

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-24 Thread Viresh Kumar
On 23 January 2014 19:31, Frederic Weisbecker fweis...@gmail.com wrote: Ok, so it is fine to migrate the latter kind I guess? Unless somebody has abused the API and used bound workqueues where he should have used unbound ones. I haven't checked the details but then this quiesce option would

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-23 Thread Viresh Kumar
On 23 January 2014 20:28, Frederic Weisbecker wrote: > On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote: >> So, the main problem in my case was caused by this: >> >><...>-2147 [001] d..2 302.573881: hrtimer_start: >> hrtimer=c172aa50 function=tick_sched_timer

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-23 Thread Frederic Weisbecker
On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote: > On 20 January 2014 21:21, Frederic Weisbecker wrote: > > I fear you can't. If you schedule a timer in 4 seconds away and your > > clockdevice > > can only count up to 2 seconds, you can't help much the interrupt in the > > middle

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-23 Thread Viresh Kumar
On 23 January 2014 19:24, Frederic Weisbecker wrote: > On Tue, Jan 21, 2014 at 10:07:58AM +0800, Lei Wen wrote: >> I find the cpuset's cpus member becomes NULL even I hotplug the core3 >> back again. >> So is it a bug? > > Not sure, you may need to check cpuset internals. I think this is the

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-23 Thread Frederic Weisbecker
On Tue, Jan 21, 2014 at 03:19:36PM +0530, Viresh Kumar wrote: > On 20 January 2014 21:11, Frederic Weisbecker wrote: > > But for workqueues having a global affinity, I think they can be > > rescheduled later > > on the old CPUs. Although I'm not sure about that, I'm Cc'ing Tejun. > > Works

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-23 Thread Frederic Weisbecker
On Tue, Jan 21, 2014 at 10:07:58AM +0800, Lei Wen wrote: > On Mon, Jan 20, 2014 at 11:41 PM, Frederic Weisbecker > wrote: > > On Mon, Jan 20, 2014 at 08:30:10PM +0530, Viresh Kumar wrote: > >> On 20 January 2014 19:29, Lei Wen wrote: > >> > Hi Viresh, > >> > >> Hi Lei, > >> > >> > I have one

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-23 Thread Frederic Weisbecker
On Tue, Jan 21, 2014 at 10:07:58AM +0800, Lei Wen wrote: On Mon, Jan 20, 2014 at 11:41 PM, Frederic Weisbecker fweis...@gmail.com wrote: On Mon, Jan 20, 2014 at 08:30:10PM +0530, Viresh Kumar wrote: On 20 January 2014 19:29, Lei Wen adrian.w...@gmail.com wrote: Hi Viresh, Hi Lei,

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-23 Thread Frederic Weisbecker
On Tue, Jan 21, 2014 at 03:19:36PM +0530, Viresh Kumar wrote: On 20 January 2014 21:11, Frederic Weisbecker fweis...@gmail.com wrote: But for workqueues having a global affinity, I think they can be rescheduled later on the old CPUs. Although I'm not sure about that, I'm Cc'ing Tejun.

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-23 Thread Viresh Kumar
On 23 January 2014 19:24, Frederic Weisbecker fweis...@gmail.com wrote: On Tue, Jan 21, 2014 at 10:07:58AM +0800, Lei Wen wrote: I find the cpuset's cpus member becomes NULL even I hotplug the core3 back again. So is it a bug? Not sure, you may need to check cpuset internals. I think this

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-23 Thread Frederic Weisbecker
On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote: On 20 January 2014 21:21, Frederic Weisbecker fweis...@gmail.com wrote: I fear you can't. If you schedule a timer in 4 seconds away and your clockdevice can only count up to 2 seconds, you can't help much the interrupt in the

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-23 Thread Viresh Kumar
On 23 January 2014 20:28, Frederic Weisbecker fweis...@gmail.com wrote: On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote: So, the main problem in my case was caused by this: ...-2147 [001] d..2 302.573881: hrtimer_start: hrtimer=c172aa50 function=tick_sched_timer

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-21 Thread Viresh Kumar
On 20 January 2014 21:21, Frederic Weisbecker wrote: > I fear you can't. If you schedule a timer in 4 seconds away and your > clockdevice > can only count up to 2 seconds, you can't help much the interrupt in the > middle to > cope with the overflow. > > So you need to act on the source of the

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-21 Thread Viresh Kumar
On 21 January 2014 07:37, Lei Wen wrote: > What is more, I also did similiar test, and find when I set two such > cpuset group, > like core 0-2 to cpuset1, core 3 to cpuset2, while hotunplug the core3 > afterwise. > I find the cpuset's cpus member becomes NULL even I hotplug the core3 > back

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-21 Thread Viresh Kumar
On 20 January 2014 21:11, Frederic Weisbecker wrote: > But for workqueues having a global affinity, I think they can be rescheduled > later > on the old CPUs. Although I'm not sure about that, I'm Cc'ing Tejun. Works queued on workqueues with WQ_UNBOUND flag set are run on any cpu and is

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-21 Thread Viresh Kumar
On 20 January 2014 21:11, Frederic Weisbecker fweis...@gmail.com wrote: But for workqueues having a global affinity, I think they can be rescheduled later on the old CPUs. Although I'm not sure about that, I'm Cc'ing Tejun. Works queued on workqueues with WQ_UNBOUND flag set are run on any

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-21 Thread Viresh Kumar
On 21 January 2014 07:37, Lei Wen adrian.w...@gmail.com wrote: What is more, I also did similiar test, and find when I set two such cpuset group, like core 0-2 to cpuset1, core 3 to cpuset2, while hotunplug the core3 afterwise. I find the cpuset's cpus member becomes NULL even I hotplug the

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-21 Thread Viresh Kumar
On 20 January 2014 21:21, Frederic Weisbecker fweis...@gmail.com wrote: I fear you can't. If you schedule a timer in 4 seconds away and your clockdevice can only count up to 2 seconds, you can't help much the interrupt in the middle to cope with the overflow. So you need to act on the

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Lei Wen
On Mon, Jan 20, 2014 at 11:41 PM, Frederic Weisbecker wrote: > On Mon, Jan 20, 2014 at 08:30:10PM +0530, Viresh Kumar wrote: >> On 20 January 2014 19:29, Lei Wen wrote: >> > Hi Viresh, >> >> Hi Lei, >> >> > I have one question regarding unbounded workqueue migration in your case. >> > You use

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Frederic Weisbecker
On Mon, Jan 20, 2014 at 05:00:20PM +0530, Viresh Kumar wrote: > On 16 January 2014 15:16, Thomas Gleixner wrote: > > Just do the math. > > > > max reload value / timer freq = max time span > > Thanks. > > > So: > > > > 0x7fff / 24MHz = 89.478485 sec > > > > Nothing to do here

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Frederic Weisbecker
On Mon, Jan 20, 2014 at 08:30:10PM +0530, Viresh Kumar wrote: > On 20 January 2014 19:29, Lei Wen wrote: > > Hi Viresh, > > Hi Lei, > > > I have one question regarding unbounded workqueue migration in your case. > > You use hotplug to migrate the unbounded work to other cpus, but its cpu > >

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Viresh Kumar
On 20 January 2014 19:29, Lei Wen wrote: > Hi Viresh, Hi Lei, > I have one question regarding unbounded workqueue migration in your case. > You use hotplug to migrate the unbounded work to other cpus, but its cpu mask > would still be 0xf, since cannot be changed by cpuset. > > My question is

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Lei Wen
Hi Viresh, On Wed, Jan 15, 2014 at 5:27 PM, Viresh Kumar wrote: > Hi Again, > > I am now successful in isolating a CPU completely using CPUsets, > NO_HZ_FULL and CPU hotplug.. > > My setup and requirements for those who weren't following the > earlier mails: > > For networking machines it is

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Viresh Kumar
On 16 January 2014 15:16, Thomas Gleixner wrote: > Just do the math. > > max reload value / timer freq = max time span Thanks. > So: > > 0x7fff / 24MHz = 89.478485 sec > > Nothing to do here except to get rid of the requirement to arm the > timer at all. @Frederic: Any inputs on

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Lei Wen
Hi Viresh, On Wed, Jan 15, 2014 at 5:27 PM, Viresh Kumar viresh.ku...@linaro.org wrote: Hi Again, I am now successful in isolating a CPU completely using CPUsets, NO_HZ_FULL and CPU hotplug.. My setup and requirements for those who weren't following the earlier mails: For networking

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Viresh Kumar
On 20 January 2014 19:29, Lei Wen adrian.w...@gmail.com wrote: Hi Viresh, Hi Lei, I have one question regarding unbounded workqueue migration in your case. You use hotplug to migrate the unbounded work to other cpus, but its cpu mask would still be 0xf, since cannot be changed by cpuset.

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Frederic Weisbecker
On Mon, Jan 20, 2014 at 08:30:10PM +0530, Viresh Kumar wrote: On 20 January 2014 19:29, Lei Wen adrian.w...@gmail.com wrote: Hi Viresh, Hi Lei, I have one question regarding unbounded workqueue migration in your case. You use hotplug to migrate the unbounded work to other cpus, but its

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Frederic Weisbecker
On Mon, Jan 20, 2014 at 05:00:20PM +0530, Viresh Kumar wrote: On 16 January 2014 15:16, Thomas Gleixner t...@linutronix.de wrote: Just do the math. max reload value / timer freq = max time span Thanks. So: 0x7fff / 24MHz = 89.478485 sec Nothing to do here except

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Lei Wen
On Mon, Jan 20, 2014 at 11:41 PM, Frederic Weisbecker fweis...@gmail.com wrote: On Mon, Jan 20, 2014 at 08:30:10PM +0530, Viresh Kumar wrote: On 20 January 2014 19:29, Lei Wen adrian.w...@gmail.com wrote: Hi Viresh, Hi Lei, I have one question regarding unbounded workqueue migration in

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-20 Thread Viresh Kumar
On 16 January 2014 15:16, Thomas Gleixner t...@linutronix.de wrote: Just do the math. max reload value / timer freq = max time span Thanks. So: 0x7fff / 24MHz = 89.478485 sec Nothing to do here except to get rid of the requirement to arm the timer at all. @Frederic: Any

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-16 Thread Thomas Gleixner
On Thu, 16 Jan 2014, Viresh Kumar wrote: > On 15 January 2014 22:47, Frederic Weisbecker wrote: > > Are you sure about that? NO_HZ_FULL shouldn't touch much hrtimers. > > Those are independant from the tick. > > > > Although some of them seem to rely on the softirq, but that seem to > > concern

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-16 Thread Thomas Gleixner
On Thu, 16 Jan 2014, Viresh Kumar wrote: On 15 January 2014 22:47, Frederic Weisbecker fweis...@gmail.com wrote: Are you sure about that? NO_HZ_FULL shouldn't touch much hrtimers. Those are independant from the tick. Although some of them seem to rely on the softirq, but that seem to

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-15 Thread Frederic Weisbecker
On Wed, Jan 15, 2014 at 02:57:36PM +0530, Viresh Kumar wrote: > Hi Again, > > I am now successful in isolating a CPU completely using CPUsets, > NO_HZ_FULL and CPU hotplug.. > > My setup and requirements for those who weren't following the > earlier mails: > > For networking machines it is

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-15 Thread Peter Zijlstra
On Wed, Jan 15, 2014 at 04:17:26PM +0530, Viresh Kumar wrote: > On 15 January 2014 16:08, Peter Zijlstra wrote: > > Nah, its just ugly and we should fix it. You need to be careful to not > > place tasks in a cpuset you're going to unplug though, that'll give > > funny results. > > Okay. So how

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-15 Thread Viresh Kumar
On 15 January 2014 16:08, Peter Zijlstra wrote: > Nah, its just ugly and we should fix it. You need to be careful to not > place tasks in a cpuset you're going to unplug though, that'll give > funny results. Okay. So how do you suggest to get rid of cases like a work queued on CPU1 initially and

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-15 Thread Peter Zijlstra
On Wed, Jan 15, 2014 at 02:57:36PM +0530, Viresh Kumar wrote: > Hi Again, > > I am now successful in isolating a CPU completely using CPUsets, > NO_HZ_FULL and CPU hotplug.. > > My setup and requirements for those who weren't following the > earlier mails: > > For networking machines it is

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-15 Thread Peter Zijlstra
On Wed, Jan 15, 2014 at 02:57:36PM +0530, Viresh Kumar wrote: Hi Again, I am now successful in isolating a CPU completely using CPUsets, NO_HZ_FULL and CPU hotplug.. My setup and requirements for those who weren't following the earlier mails: For networking machines it is required to

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-15 Thread Viresh Kumar
On 15 January 2014 16:08, Peter Zijlstra pet...@infradead.org wrote: Nah, its just ugly and we should fix it. You need to be careful to not place tasks in a cpuset you're going to unplug though, that'll give funny results. Okay. So how do you suggest to get rid of cases like a work queued on

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-15 Thread Peter Zijlstra
On Wed, Jan 15, 2014 at 04:17:26PM +0530, Viresh Kumar wrote: On 15 January 2014 16:08, Peter Zijlstra pet...@infradead.org wrote: Nah, its just ugly and we should fix it. You need to be careful to not place tasks in a cpuset you're going to unplug though, that'll give funny results.

Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

2014-01-15 Thread Frederic Weisbecker
On Wed, Jan 15, 2014 at 02:57:36PM +0530, Viresh Kumar wrote: Hi Again, I am now successful in isolating a CPU completely using CPUsets, NO_HZ_FULL and CPU hotplug.. My setup and requirements for those who weren't following the earlier mails: For networking machines it is required to