Re: Scheduler improvements, take 1001, Patch 2/5

2012-10-15 Thread David Coppa
On Sun, Oct 14, 2012 at 4:53 PM, Gregor Best g...@ring0.de wrote: On Sun, Oct 14, 2012 at 11:05:36AM +0200, David Coppa wrote: On Tue, Oct 9, 2012 at 6:21 PM, Gregor Best g...@ring0.de wrote: This patch simply halves the timeslice processes get until they are preempted. This patch is

Re: Scheduler improvements, take 1001, Patch 2/5

2012-10-15 Thread Stuart Henderson
On 2012/10/15 16:18, David Coppa wrote: On Sun, Oct 14, 2012 at 4:53 PM, Gregor Best g...@ring0.de wrote: On Sun, Oct 14, 2012 at 11:05:36AM +0200, David Coppa wrote: On Tue, Oct 9, 2012 at 6:21 PM, Gregor Best g...@ring0.de wrote: This patch simply halves the timeslice processes get until

Re: Scheduler improvements, take 1001, Patch 2/5

2012-10-15 Thread Stefan Fritsch
On Monday 15 October 2012, Stuart Henderson wrote: Best to send the diff, with the accompanying text, in the same email, and make sure they still all apply, I tried testing some of these but didn't manage to get them to apply (either some conflicting change, or they were in the wrong order and

Re: Scheduler improvements, take 1001, Patch 2/5

2012-10-14 Thread David Coppa
On Tue, Oct 9, 2012 at 6:21 PM, Gregor Best g...@ring0.de wrote: This patch simply halves the timeslice processes get until they are preempted. This patch is standalone and the rest of the patches does not depend on it, but I figured I'd throw it in anyway. -- Gregor Best Didn't get

Re: Scheduler improvements, take 1001, Patch 2/5

2012-10-14 Thread Gregor Best
On Sun, Oct 14, 2012 at 11:05:36AM +0200, David Coppa wrote: On Tue, Oct 9, 2012 at 6:21 PM, Gregor Best g...@ring0.de wrote: This patch simply halves the timeslice processes get until they are preempted. This patch is standalone and the rest of the patches does not depend on it, but I

Re: Scheduler improvements, take 1001, Patch 2/5

2012-10-14 Thread Bob Beck
Gregor you would perhaps get better feedback if it were easier to discern where your patches are and what each one is doing. If you can't be inclined to keep the subjects matching the diffs and are sending stuff out with subjects like scheduler improvement diff X instead of something like reduce

Re: Scheduler improvements, take 1001, Patch 5/5

2012-10-13 Thread Philip Guenther
On Tue, Oct 9, 2012 at 9:27 AM, Gregor Best g...@ring0.de wrote: This patch moves struct schedstate_percpu to kernel land, which I think is cleaner than exposing structures for scheduler state to userland, especially since grepping for 'schedstate' in /usr/src yielded no results outside of

Re: Scheduler improvements, take 1001, Patch 1/5

2012-10-09 Thread Gregor Best
diff --git a/kern/sched_bsd.c b/kern/sched_bsd.c index 172bb8f..c7121dc 100644 --- a/kern/sched_bsd.c +++ b/kern/sched_bsd.c @@ -77,12 +77,12 @@ scheduler_start(void) timeout_set(schedcpu_to, schedcpu, schedcpu_to); - rrticks_init = hz / 10; + rrticks_init = hz / 20;

Re: Scheduler improvements, take 1001, Patch 1/5

2012-10-09 Thread Gregor Best
diff --git a/kern/kern_clock.c b/kern/kern_clock.c index 843965b..f598afc 100644 --- a/kern/kern_clock.c +++ b/kern/kern_clock.c @@ -233,7 +233,7 @@ hardclock(struct clockframe *frame) if (stathz == 0) statclock(frame); - if (--ci-ci_schedstate.spc_rrticks = 0) +

Re: Scheduler improvements, take 1001, Patch 5/5

2012-10-09 Thread Gregor Best
diff --git a/sys/sched.h b/sys/sched.h index fb01f21..1784ee2 100644 --- a/sys/sched.h +++ b/sys/sched.h @@ -69,8 +69,10 @@ #ifndef_SYS_SCHED_H_ #define_SYS_SCHED_H_ +#ifdef _KERNEL #include sys/queue.h #include sys/tree.h +#endif /* * Posix defines a sched.h which may

Re: Scheduler improvements, take 1001, Patch 4/5

2012-10-09 Thread Gregor Best
diff --git a/arch/amd64/include/cpu.h b/arch/amd64/include/cpu.h index 12e48d6..99501a1 100644 --- a/arch/amd64/include/cpu.h +++ b/arch/amd64/include/cpu.h @@ -102,9 +102,11 @@ struct cpu_info { u_int32_t ci_cflushsz; u_int64_t ci_tsc_freq; +#define

Re: Scheduler improvements, take 1001, Patch 3/5

2012-10-09 Thread Gregor Best
diff --git a/arch/amd64/amd64/identcpu.c b/arch/amd64/amd64/identcpu.c index c597bb0..982c2bb 100644 --- a/arch/amd64/amd64/identcpu.c +++ b/arch/amd64/amd64/identcpu.c @@ -210,6 +210,8 @@ void (*setperf_setup)(struct cpu_info *); void via_nano_setup(struct cpu_info *ci); +void

Re: Scheduler improvements, take 1001, Patch 2/5

2012-10-09 Thread Gregor Best
This patch simply halves the timeslice processes get until they are preempted. This patch is standalone and the rest of the patches does not depend on it, but I figured I'd throw it in anyway. -- Gregor Best

Re: Scheduler improvements, take 1001, Patch 3/5

2012-10-09 Thread Gregor Best
This patch simply imports Christiano's code for detecting CPU topology, as posted on tech@ a while (more than two months) ago. I took it verbatim and didn't change anything yet. -- Gregor Best

Re: Scheduler improvements, take 1001, Patch 4/5

2012-10-09 Thread Gregor Best
This patch uses the previous one to take CPU topology into account when calculating the cost of moving a process between CPUs. This is only done on amd64 at the moment, and the cost factors are guesses right now, but it's a start. -- Gregor Best

Re: Scheduler improvements, take 1001, Patch 5/5

2012-10-09 Thread Gregor Best
This patch moves struct schedstate_percpu to kernel land, which I think is cleaner than exposing structures for scheduler state to userland, especially since grepping for 'schedstate' in /usr/src yielded no results outside of /usr/src/sys. I have not seen negative impact from this, but I haven't

Re: Scheduler improvements

2012-10-08 Thread Ville Valkonen
Hi, Seems to be a bit snappier, though more thorough testing needed. Many web pages became more usable after the patch (chrome as browser) and some gains in desktop too. System is amd64 @ Intel(R) Core(TM)2 Duo CPU T5870 @ 2.00GHz and 4G RAM. -- cheers, Ville

Re: Scheduler improvements

2012-10-08 Thread Marc Espie
log2 should probably be scrapped, since you're reinventing ffs.

Re: Scheduler improvements

2012-10-08 Thread Christiano F. Haesbaert
On 8 October 2012 11:24, Marc Espie es...@nerim.net wrote: log2 should probably be scrapped, since you're reinventing ffs. That is actually my code, back then I failed at finding an ffs. I'll try to have a look at the code this week.

Re: Scheduler improvements

2012-10-08 Thread Alexandre Ratchov
On Sat, Oct 06, 2012 at 10:38:34PM +0200, Gregor Best wrote: Hi Alexandre, [...] This change is unclear for me; AFAIU, it removes the mechanism which makes processes wake up with a priority depending on what they are blocked on. [...] Where do you see that? The code I

Re: Scheduler improvements

2012-10-08 Thread Christiano F. Haesbaert
On Oct 8, 2012 6:33 PM, Alexandre Ratchov a...@caoua.org wrote: On Sat, Oct 06, 2012 at 10:38:34PM +0200, Gregor Best wrote: Hi Alexandre, [...] This change is unclear for me; AFAIU, it removes the mechanism which makes processes wake up with a priority depending on what they are

Re: Scheduler improvements

2012-10-08 Thread Gregor Best
On Mon, Oct 08, 2012 at 06:28:53PM +0200, Alexandre Ratchov wrote: [...] AFAIU, when a process waits for an event (with tsleep(9)), it provides the priority of the event it's waiting for (eg an audio interrupt). When the event occurs, the process is put in the runnable queue calculated from

Re: Scheduler improvements

2012-10-08 Thread Ted Unangst
On Mon, Oct 08, 2012 at 18:43, Christiano F. Haesbaert wrote: Ratchov is correct, when a process sleeps you specify the priority you should have when waking up. I think george was referring to the old version of his code. The 4.4 bsd scheduler relies heavily on these priority boosts, i

Re: Scheduler improvements

2012-10-08 Thread Marc Espie
On Sun, Oct 07, 2012 at 04:04:16PM +0200, Gregor Best wrote: [...] I don't think changing the idle loop like this is ok. You want to continue checking whether the runqueue is empty in between cpu_idle_enter() and cpu_idle_leave(). [...] Fair point. I'll change that :) Your diff

Re: Scheduler improvements

2012-10-08 Thread Gregor Best
On Mon, Oct 08, 2012 at 07:42:13PM +0200, Marc Espie wrote: [...] Your diff doesn't pass userland compiles. You're adding a dependency on tree.h in proc.h. In kernel land on amd64, that's okay, since machine/param.h will pull machine/cpu.h which pulls sys/sched.h which now pulls sys/tree.h,

Re: Scheduler improvements

2012-10-07 Thread Gregor Best
[...] I don't think changing the idle loop like this is ok. You want to continue checking whether the runqueue is empty in between cpu_idle_enter() and cpu_idle_leave(). [...] Fair point. I'll change that :) -- Gregor Best

Re: Scheduler improvements

2012-10-06 Thread Gregor Best
Hi Alexandre, [...] This change is unclear for me; AFAIU, it removes the mechanism which makes processes wake up with a priority depending on what they are blocked on. [...] Where do you see that? The code I removed/changed simply calulated the queue from which to remove `p` and removed

Re: Scheduler improvements

2012-10-05 Thread Antoine Jacoutot
On Thu, Oct 04, 2012 at 11:42:45PM +0200, Gregor Best wrote: As before, I'm looking forward to anything you have to comment, especially cool benchmark ideas or the like. I know my report is not a benchmark of any kind but I do see a slight improvements when running a full GNOME 3

Re: Scheduler improvements

2012-10-05 Thread Juan Francisco Cantero Hurtado
On Thu, Oct 04, 2012 at 11:42:45PM +0200, Gregor Best wrote: Hi people, after a (very) long time of silence on this, here's another go at it. This time, I basically started from scratch and used a bit of code by Christiano Haesberth which had been posted to tech@ a while ago to detect

Re: Scheduler improvements

2012-10-05 Thread Alexandre Ratchov
On Thu, Oct 04, 2012 at 11:42:45PM +0200, Gregor Best wrote: @@ -222,14 +230,13 @@ setrunqueue(struct proc *p) { struct schedstate_percpu *spc; - int queue = p-p_priority 2; SCHED_ASSERT_LOCKED(); spc = p-p_cpu-ci_schedstate; spc-spc_nrun++; -

Re: Scheduler improvements

2012-10-05 Thread Kevin Chadwick
It appears to have sped up porn. movies on the machine seem a bit better. I will try this in a few other places Just not at the mother in laws or in public places no matter how impressed you are at the difference? -- ___

Re: Scheduler improvements

2012-10-05 Thread Norman Golisz
On Fri Oct 5 2012 14:24, Antoine Jacoutot wrote: On Thu, Oct 04, 2012 at 11:42:45PM +0200, Gregor Best wrote: As before, I'm looking forward to anything you have to comment, especially cool benchmark ideas or the like. I know my report is not a benchmark of any kind but I do see a

Re: Scheduler improvements

2012-10-05 Thread Mark Kettenis
Date: Thu, 4 Oct 2012 23:42:45 +0200 From: Gregor Best g...@ring0.de Index: kern/kern_sched.c === RCS file: /cvs/src/sys/kern/kern_sched.c,v retrieving revision 1.27 diff -u -r1.27 kern_sched.c --- kern/kern_sched.c 10 Jul

Re: Scheduler improvements

2012-10-04 Thread Ted Unangst
On Thu, Oct 04, 2012 at 23:42, Gregor Best wrote: As before, I'm looking forward to anything you have to comment, especially cool benchmark ideas or the like. A short one is building a kernel. Try before and after with make, make -j 2, and make -j 4. (or ncpu and ncpu * 2).

Re: Scheduler improvements

2012-10-04 Thread Marc Espie
On Thu, Oct 04, 2012 at 06:28:11PM -0400, Ted Unangst wrote: On Thu, Oct 04, 2012 at 23:42, Gregor Best wrote: As before, I'm looking forward to anything you have to comment, especially cool benchmark ideas or the like. A short one is building a kernel. Try before and after with make,

Re: Scheduler improvements

2012-01-21 Thread Gregor Best
On Fri, Jan 20, 2012 at 06:06:08PM +0100, Ariane van der Steldt wrote: [...] * Timeslices are 20ms long instead of 10ms. This solves the issue of 0ms long timeslices on machines with hz 100. I'm not really happy having timeslices get larger. They're considerably large already. Can you

Re: Scheduler improvements

2012-01-20 Thread Ariane van der Steldt
On Wed, Jan 18, 2012 at 04:41:35PM +0100, Gregor Best wrote: after a long time of silence here's a second iteration of the patch. I've addressed a few concerns voiced here: * Process lookup and friends now have O(log n) runtime. I achieved that by abusing RB-trees as priority queues since

Re: Scheduler improvements

2012-01-20 Thread Marc Espie
On Fri, Jan 20, 2012 at 06:06:08PM +0100, Ariane van der Steldt wrote: * Timeslices are 20ms long instead of 10ms. This solves the issue of 0ms long timeslices on machines with hz 100. I'm not really happy having timeslices get larger. They're considerably large already. Can you think

Re: Scheduler improvements

2012-01-18 Thread Gregor Best
Hi people, after a long time of silence here's a second iteration of the patch. I've addressed a few concerns voiced here: * Process lookup and friends now have O(log n) runtime. I achieved that by abusing RB-trees as priority queues since they have runtime O(log n) in all relevant

Re: Scheduler improvements

2012-01-18 Thread Gregor Best
And it didn't take long for me to find a small bug... attached is a fixed version of the patch. Such things happen if one decides to regenerate a patch just in case and forgets to revert to a working version before doing that :D -- Gregor Best Index: sys/proc.h

Re: Scheduler improvements

2011-10-13 Thread Alexandre Ratchov
On Wed, Oct 12, 2011 at 02:55:39PM +0200, Gregor Best wrote: Hi people, attached is a patch that basically rewrites the CPU scheduler. It replaces the multilevel feedback queue currently employed with an earliest effective deadline first approach, similar to the BrainFuck Scheduler for

Re: Scheduler improvements

2011-10-12 Thread Miod Vallat
Hi people, [...] I'm not aiming for a yeah, nice, we'll merge it on this, but rather for suggestions whether it's worth anyones time to pursue this further. This is interesting, there might be a good outcome of this diff eventually. However as of now: - you have removed the bunch of code

Re: Scheduler improvements

2011-10-12 Thread Gregor Best
On Wed, Oct 12, 2011 at 05:51:33PM +, Miod Vallat wrote: Hi people, [...] I'm not aiming for a yeah, nice, we'll merge it on this, but rather for suggestions whether it's worth anyones time to pursue this further. This is interesting, there might be a good outcome of this diff

Re: Scheduler improvements

2011-10-12 Thread Tobias Weingartner
I like what I see. It could be the start for the means of managing a single socket's queue of processes. You do mention that this won't really scale beyond roughly 8 cores. I would love to see you extend this with a 2D array of weights that can be populated by various means (run-time testing,