Linux-5.10-ck1, MuQSS CPU scheduler v0.205

[Con Kolivas]

Just reminding people I'm still around and maintaining this patchset.

Announcing a new -ck release, 5.10-ck1  with the latest version of the
Multiple Queue Skiplist Scheduler, version 0.205 These are patches
designed to improve system responsiveness and interactivity with
specific emphasis on the desktop, but configurable for any workload.

http://ck-hack.blogspot.com/2020/12/linux-510-ck1-muqss-version-0205-for.html

http://ck.kolivas.org/patches/5.0/5.10/5.10-ck1/patch-5.10-ck1.xz
https://github.com/ckolivas/linux/tree/5.10-ck

http://ck.kolivas.org/patches/muqss/5.0/5.10/0001-MultiQueue-Skiplist-Scheduler-v0.205.patch
https://github.com/ckolivas/linux/tree/5.10-muqss

Enjoy!
お楽しみ下さい
-ck

linux-5.1-ck1, MuQSS version 0.192 for linux-5.1

[Con Kolivas]

Announcing a new -ck release, 5.1-ck1  with the latest version of the
Multiple Queue Skiplist Scheduler, version 0.192. These are patches
designed to improve system responsiveness and interactivity with
specific emphasis on the desktop, but configurable for any workload.

linux-5.1-ck1:
-ck1 patches:
http://ck.kolivas.org/patches/5.0/5.1/5.1-ck1/
Git tree:
https://github.com/ckolivas/linux/tree/5.1-ck

MuQSS only:
Download:
http://ck.kolivas.org/patches/muqss/5.0/5.1/0001-MultiQueue-Skiplist-Scheduler-version-0.192.patch
Git tree:
https://github.com/ckolivas/linux/tree/5.1-muqss

Blog:
http://ck-hack.blogspot.com/

Web:
http://kernel.kolivas.org


This is mostly a resync from 5.0-ck1 with some build and boot fixes
courtesy of Serge Belyshev (thanks!)

Enjoy!
お楽しみ下さい
-ck

Re: [PATCH V2] usbcore: Select only first configuration for non-UAC3 compliant devices

[Con Kolivas]

Hi Saranya

On Sun, 6 Jan 2019 at 04:41,  wrote:
>
> From: Saranya Gopal 
>
> In most of the UAC1 and UAC2 audio devices, the first
> configuration is most often the best configuration.
> However, with recent patch to support UAC3 configuration,
> second configuration was unintentionally chosen for
> some of the UAC1/2 devices that had more than one
> configuration. This was because of the existing check
> after the audio config check which selected any config
> which had a non-vendor class. This patch fixes this issue.
>
> Fixes: f13912d3f014 ("usbcore: Select UAC3 configuration for audio if 
> present")
> Reported-by: Con Kolivas 
> Signed-off-by: Saranya Gopal >

I can confirm the above patch fixes my problem.
Tested-by: Con Kolivas 

Thanks!
Con

Re: [alsa-devel] ALSA:usb audio Higher sample rates on usb audio no longer working.

[Con Kolivas]

Hi Saranya.

On Sat, 5 Jan 2019 at 03:52, Gopal, Saranya  wrote:

> And since I was not part of the initial mail thread, I might have missed some 
> information.
> Could someone give me lsusb -v output for this USB audio device.

These outputs are with the UAC3 patch backed out:

dmesg:
[50384.859492] usb 2-1.8.4: new high-speed USB device number 26 using ehci-pci
[50384.974496] usb 2-1.8.4: New USB device found, idVendor=19fb,
idProduct=2040, bcdDevice= 2.00
[50384.974500] usb 2-1.8.4: New USB device strings: Mfr=1, Product=2,
SerialNumber=0
[50384.974501] usb 2-1.8.4: Product: Anti-Mode X4
[50384.974503] usb 2-1.8.4: Manufacturer: DSPeaker

lsusb -v:
Bus 002 Device 026: ID 19fb:2040
Device Descriptor:
  bLength18
  bDescriptorType 1
  bcdUSB   2.00
  bDeviceClass  239 Miscellaneous Device
  bDeviceSubClass 2 ?
  bDeviceProtocol 1 Interface Association
  bMaxPacketSize064
  idVendor   0x19fb
  idProduct  0x2040
  bcdDevice2.00
  iManufacturer   1 DSPeaker
  iProduct2 Anti-Mode X4
  iSerial 0
  bNumConfigurations  2
  Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength  254
bNumInterfaces  2
bConfigurationValue 1
iConfiguration  0
bmAttributes 0xc0
  Self Powered
MaxPower  100mA
Interface Association:
  bLength 8
  bDescriptorType11
  bFirstInterface 0
  bInterfaceCount 2
  bFunctionClass  1 Audio
  bFunctionSubClass   0
  bFunctionProtocol  32
  iFunction   0
Interface Descriptor:
  bLength 9
  bDescriptorType 4
  bInterfaceNumber0
  bAlternateSetting   0
  bNumEndpoints   0
  bInterfaceClass 1 Audio
  bInterfaceSubClass  1 Control Device
  bInterfaceProtocol 32
  iInterface  0
  AudioControl Interface Descriptor:
bLength 9
bDescriptorType36
bDescriptorSubtype  1 (HEADER)
bcdADC   2.00
bCategory  10
wTotalLength   60
bmControl0x00
  AudioControl Interface Descriptor:
bLength 8
bDescriptorType36
bDescriptorSubtype 10 (CLOCK_SOURCE)
bClockID   41
bmAttributes 0x03 Internal programmable Clock
bmControls   0x07
  Clock Frequency Control (read/write)
  Clock Validity Control (read-only)
bAssocTerminal  0
iClockSource0
  AudioControl Interface Descriptor:
bLength 8
bDescriptorType36
bDescriptorSubtype 11 (CLOCK_SELECTOR)
bUnitID40
bNrInPins   1
baCSourceID( 0)41
bmControls   0x00
iClockSelector  0
  AudioControl Interface Descriptor:
bLength17
bDescriptorType36
bDescriptorSubtype  2 (INPUT_TERMINAL)
bTerminalID 1
wTerminalType  0x0101 USB Streaming
bAssocTerminal  0
bCSourceID 40
bNrChannels 2
bmChannelConfig   0x
bmControls0x
iChannelNames   4 Analog 1
iTerminal   0
  AudioControl Interface Descriptor:
bLength18
bDescriptorType36
bDescriptorSubtype  6 (FEATURE_UNIT)
bUnitID 3
bSourceID   1
bmaControls( 0)  0x000f
  Mute Control (read/write)
  Volume Control (read/write)
bmaControls( 1)  0x
bmaControls( 2)  0x
iFeature0
Interface Descriptor:
  bLength 9
  bDescriptorType 4
  bInterfaceNumber1
  bAlternateSetting   0
  bNumEndpoints   0
  bInterfaceClass 1 Audio
  bInterfaceSubClass  2 Streaming
  bInterfaceProtocol 32
  iInterface  0
Interface Descriptor:
  bLength 9
  bDescriptorType 4
  bInterfaceNumber1
  bAlternateSetting   1
  bNumEndpoints   2
  bInterfaceClass 1 Audio
  bInterfaceSubClass  2 Streaming
  bInterfaceProtocol 32
  iInterface  0
  AudioStreaming Interface Descriptor:
bLength16
bDescriptorType36
bDescriptorSubtype  1 (AS_GENERAL)
bTerminalLink   1
bmControls   0x00
bFormatType  

Re: ALSA:usb audio Higher sample rates on usb audio no longer working.

[Con Kolivas]

Hi Iwai-san.

Added some relevant CCs.

On Fri, 4 Jan 2019 at 00:23, Takashi Iwai  wrote:
>
> On Thu, 03 Jan 2019 12:43:54 +0100,
> Con Kolivas wrote:
> >
> > Upon switching from 4.19.0 to 4.20.0, pulseaudio started complaining
> > that sinks that previously worked are no longer supported.
> >
> > On 4.19.0 trying 24 bit 88200, 176400, and 192000 I get the following
> > output from pulse.
> > resampler.c: Forcing resampler 'copy', because of fixed, identical
> > sample rates.sink-input.c: Created input 15 "Playback Stream" on
> > alsa_output.usb-DSPeaker_Anti-Mode_X4-00.iec958-stereo with sample
> > spec float32le 2ch 176400Hz and channel map front-left,front-right
> >
> > Switching to 4.20 gives me:
> > alsa-sink.c: Sink does not support sample rate of 176400 Hz
> > and
> > alsa-sink.c: Sink does not support sample rate of 88200 Hz
> > and
> > alsa-sink.c: Sink does not support sample rate of 192000 Hz
> >
> > Sample rates of 44100, 48000, and 96000 work fine, but 88200, 176400,
> > and 192000 no longer work
> >
> > Switching back to 4.19 immediately fixes the issue.
> >
> >
> > I tried looking through the alsa changelogs but there were too many to
> > give an obvious culprit, and haven't had time to do a git bisect. If
> > there's an obvious choice patch to back out I'd be grateful for the
> > heads up.
>
> Hm, through a quick glance, there hasn't been any relevant changes in
> USB-audio part (sound/usb/*).  Also, the changes in sound/core/* are
> irrelevant with your problem.
>
> So I have no idea what went wrong.  The bisection, or at least,
> narrowing down the commits would be helpful.

I've done a git bisect and found the offending commit:

commit f13912d3f014a7f2fa5c35d25ee8c3f96bda6272 (refs/bisect/bad)
Author: Saranya Gopal 
Date:   Wed Sep 12 08:46:26 2018 +0530

usbcore: Select UAC3 configuration for audio if present

USB audio class 3.0 specification introduced many significant
changes like
 - new power domains, support for LPM/L1
 - new cluster descriptor
 - new high capability and class-specific string descriptors
 - BADD profiles
 - ... and many other things (check spec from link below:
http://www.usb.org/developers/docs/devclass_docs/USB_Audio_v3.0.zip)

Now that UAC3 is supported in linux, choose UAC3
configuration for audio if the device supports it.
Selecting this configuration will enable the system to
save power by leveraging the new power domains and LPM L1
capability and also support new codec types and data formats
for consumer audio applications.

Signed-off-by: Saranya Gopal 
Reviewed-by: Felipe Balbi 
Signed-off-by: Greg Kroah-Hartman 

Reverting this patch fixes the problem for me.

Hope this helps.

Thanks,
Con

ALSA:usb audio Higher sample rates on usb audio no longer working.

[Con Kolivas]

Upon switching from 4.19.0 to 4.20.0, pulseaudio started complaining
that sinks that previously worked are no longer supported.

On 4.19.0 trying 24 bit 88200, 176400, and 192000 I get the following
output from pulse.
resampler.c: Forcing resampler 'copy', because of fixed, identical
sample rates.sink-input.c: Created input 15 "Playback Stream" on
alsa_output.usb-DSPeaker_Anti-Mode_X4-00.iec958-stereo with sample
spec float32le 2ch 176400Hz and channel map front-left,front-right

Switching to 4.20 gives me:
alsa-sink.c: Sink does not support sample rate of 176400 Hz
and
alsa-sink.c: Sink does not support sample rate of 88200 Hz
and
alsa-sink.c: Sink does not support sample rate of 192000 Hz

Sample rates of 44100, 48000, and 96000 work fine, but 88200, 176400,
and 192000 no longer work

Switching back to 4.19 immediately fixes the issue.


I tried looking through the alsa changelogs but there were too many to
give an obvious culprit, and haven't had time to do a git bisect. If
there's an obvious choice patch to back out I'd be grateful for the
heads up.

Regards,
Con


Output from 4.19 dmesg grep snd:
[4.941164] snd_hda_intel :02:00.1: Disabling MSI
[4.957534] snd_hda_codec_realtek hdaudioC0D0: autoconfig for
ALC892: line_outs=4 (0x14/0x15/0x16/0x17/0x0) type:line
[4.957536] snd_hda_codec_realtek hdaudioC0D0:speaker_outs=0
(0x0/0x0/0x0/0x0/0x0)
[4.957537] snd_hda_codec_realtek hdaudioC0D0:hp_outs=1
(0x1b/0x0/0x0/0x0/0x0)
[4.957538] snd_hda_codec_realtek hdaudioC0D0:mono: mono_out=0x0
[4.957538] snd_hda_codec_realtek hdaudioC0D0:dig-out=0x11/0x1e
[4.957539] snd_hda_codec_realtek hdaudioC0D0:inputs:
[4.957540] snd_hda_codec_realtek hdaudioC0D0:  Front Mic=0x19
[4.957541] snd_hda_codec_realtek hdaudioC0D0:  Rear Mic=0x18
[4.957542] snd_hda_codec_realtek hdaudioC0D0:  Line=0x1a
[6.215818] usbcore: registered new interface driver snd-usb-audio

Output from 4.20 dmesg grep snd (identical):
[4.960455] snd_hda_intel :02:00.1: Disabling MSI
[4.986848] snd_hda_codec_realtek hdaudioC0D0: autoconfig for
ALC892: line_outs=4 (0x14/0x15/0x16/0x17/0x0) type:line
[4.986850] snd_hda_codec_realtek hdaudioC0D0:speaker_outs=0
(0x0/0x0/0x0/0x0/0x0)
[4.986852] snd_hda_codec_realtek hdaudioC0D0:hp_outs=1
(0x1b/0x0/0x0/0x0/0x0)
[4.986853] snd_hda_codec_realtek hdaudioC0D0:mono: mono_out=0x0
[4.986854] snd_hda_codec_realtek hdaudioC0D0:dig-out=0x11/0x1e
[4.986855] snd_hda_codec_realtek hdaudioC0D0:inputs:
[4.986856] snd_hda_codec_realtek hdaudioC0D0:  Front Mic=0x19
[4.986857] snd_hda_codec_realtek hdaudioC0D0:  Rear Mic=0x18
[4.986858] snd_hda_codec_realtek hdaudioC0D0:  Line=0x1a
[6.211079] usbcore: registered new interface driver snd-usb-audio

Output from usb connected audio device 4.19:
[  998.567015] usb 2-1.8.4: new high-speed USB device number 19 using ehci-pci
[  998.682497] usb 2-1.8.4: New USB device found, idVendor=19fb,
idProduct=2040, bcdDevice= 2.00
[  998.682502] usb 2-1.8.4: New USB device strings: Mfr=1, Product=2,
SerialNumber=0
[  998.682505] usb 2-1.8.4: Product: Anti-Mode X4
[  998.682507] usb 2-1.8.4: Manufacturer: DSPeaker

4.20 (identical):
[   98.428175] usb 2-1.8.4: new high-speed USB device number 19 using ehci-pci
[   98.543175] usb 2-1.8.4: New USB device found, idVendor=19fb,
idProduct=2040, bcdDevice= 2.00
[   98.543177] usb 2-1.8.4: New USB device strings: Mfr=1, Product=2,
SerialNumber=0
[   98.543178] usb 2-1.8.4: Product: Anti-Mode X4
[   98.543179] usb 2-1.8.4: Manufacturer: DSPeaker

lspci output:
00:00.0 Host bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7
DMI2 (rev 04)
00:01.0 PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7
PCI Express Root Port 1a (rev 04)
00:02.0 PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7
PCI Express Root Port 2a (rev 04)
00:03.0 PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7
PCI Express Root Port 3a (rev 04)
00:05.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5
v2/Core i7 VTd/Memory Map/Misc (rev 04)
00:05.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5
v2/Core i7 IIO RAS (rev 04)
00:05.4 PIC: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 IOAPIC (rev 04)
00:11.0 PCI bridge: Intel Corporation C600/X79 series chipset PCI
Express Virtual Root Port (rev 06)
00:16.0 Communication controller: Intel Corporation C600/X79 series
chipset MEI Controller #1 (rev 05)
00:1a.0 USB controller: Intel Corporation C600/X79 series chipset USB2
Enhanced Host Controller #2 (rev 06)
00:1b.0 Audio device: Intel Corporation C600/X79 series chipset High
Definition Audio Controller (rev 06)
00:1c.0 PCI bridge: Intel Corporation C600/X79 series chipset PCI
Express Root Port 1 (rev b6)
00:1c.2 PCI bridge: Intel Corporation C600/X79 series chipset PCI
Express Root Port 3 (rev b6)
00:1c.3 PCI bridge: Intel Corporation C600/X79 series chipset PCI
Express Root Port 4 (rev b6)

[ANNOUNCE] linux-4.18-ck1, Multiple Queue Skiplist Scheduler version 0.173 for linux 4.18

[Con Kolivas]

 Announcing a new -ck release, 4.18-ck1  with the latest version of the 
Multiple Queue Skiplist Scheduler, version 0.173. These are patches designed 
to improve system responsiveness and interactivity with specific emphasis on 
the desktop, but configurable for any workload.

linux-4.18-ck1:
-ck1 patches:
http://ck.kolivas.org/patches/4.0/4.18/4.18-ck1/
Git tree:
https://github.com/ckolivas/linux/tree/4.18-ck

MuQSS only:
Download:
http://ck.kolivas.org/patches/muqss/4.0/4.18/0001-MultiQueue-Skiplist-Scheduler-version-v0.173.patch
Git tree:
https://github.com/ckolivas/linux/tree/4.18-muqss


This is just a resync from 4.17 MuQSS and -ck patches.


Blog: ck-hack.blogspot.com

Enjoy!
お楽しみ下さい

-- 
-ck

[ANNOUNCE] linux-4.18-ck1, Multiple Queue Skiplist Scheduler version 0.173 for linux 4.18

[Con Kolivas]

 Announcing a new -ck release, 4.18-ck1  with the latest version of the 
Multiple Queue Skiplist Scheduler, version 0.173. These are patches designed 
to improve system responsiveness and interactivity with specific emphasis on 
the desktop, but configurable for any workload.

linux-4.18-ck1:
-ck1 patches:
http://ck.kolivas.org/patches/4.0/4.18/4.18-ck1/
Git tree:
https://github.com/ckolivas/linux/tree/4.18-ck

MuQSS only:
Download:
http://ck.kolivas.org/patches/muqss/4.0/4.18/0001-MultiQueue-Skiplist-Scheduler-version-v0.173.patch
Git tree:
https://github.com/ckolivas/linux/tree/4.18-muqss


This is just a resync from 4.17 MuQSS and -ck patches.


Blog: ck-hack.blogspot.com

Enjoy!
お楽しみ下さい

-- 
-ck

[ANNOUNCE] linux-4.15-ck1, Multiple Queue Skiplist Scheduler version 0.170 with variable runqueue sharing.

[Con Kolivas]

Announcing a new -ck release, 4.15-ck1  with the latest version of the
Multiple Queue Skiplist Scheduler, version 0.170. These are patches
designed to improve system responsiveness and interactivity with
specific emphasis on the desktop, but configurable for any workload.


linux-4.15-ck1:
http://ck.kolivas.org/patches/4.0/4.15/4.15-ck1/

Git tree:
https://github.com/ckolivas/linux/tree/4.15-ck

MuQSS only:

Download:
http://ck.kolivas.org/patches/muqss/4.0/4.15/0001-MultiQueue-Skiplist-Scheduler-version-0.170.patch

Git tree:
https://github.com/ckolivas/linux/tree/4.15-muqss


Web: http://kernel.kolivas.org


Alas Google has falsely flagged my site as having malware, even the
kernel patch pages and they're such a monstrosity there is no place to
meaningfully appeal this so I've given up trying so you'll have to
wade through moronic warnings to get to the patches, or just use git.


The major change in this release is the addition of a much more mature
version of the experimental runqueue sharing code I posted on my blog
earlier. After further experimenting and with lots of feedback from
users, I decided to make multicore based sharing default instead of
multithread. The numbers support better throughput and it should
definitely provide more consistent low latency compared to previous
versions of MuQSS. For those that found that interactivity on MuQSS
never quite matched that of BFS before it, you should find this
version now equals it.


In addition, the runqueue sharing code in this release also allows you
to share runqueues for SMP as well so you can share runqueues with all
physical CPUs if latency is your primary concern, even though it will
likely lead to worse throughput. I have not made it possible to share
between NUMA nodes because the cost of shifting tasks across nodes is
usually substantial and it may even have worse latency, and will
definitely have worse throughput.


I've also made the runqueue sharing possible to be configured at boot
time with the boot parameter rqshare. Setting it to one of none, smt,
mc, smp is done by appending the following to your kernel command
line:

 rqshare=mc


Documentation has been added for the runqueue sharing code above to
the MuQSS patch.


A number of minor bugs were discovered and have been fixed, which has
also made booting more robust.


The -ck tree is mostly just a resync of previous patches, but with the
addition of a patch to disable a -Werror CFLAG setting in the build
tools which has suddenly made it impossible to build the kernel with
newer GCCs on some distros.

Patchlist:

0001-MultiQueue-Skiplist-Scheduler-version-0.170.patch
0002-Make-preemptible-kernel-default.patch
0003-Expose-vmsplit-for-our-poor-32-bit-users.patch
0004-Create-highres-timeout-variants-of-schedule_timeout-.patch
0005-Special-case-calls-of-schedule_timeout-1-to-use-the-.patch
0006-Convert-msleep-to-use-hrtimers-when-active.patch
0007-Replace-all-schedule-timeout-1-with-schedule_min_hrt.patch
0008-Replace-all-calls-to-schedule_timeout_interruptible-.patch
0009-Replace-all-calls-to-schedule_timeout_uninterruptibl.patch
0010-Don-t-use-hrtimer-overlay-when-pm_freezing-since-som.patch
0011-Make-hrtimer-granularity-and-minimum-hrtimeout-confi.patch
0012-Reinstate-default-Hz-of-100-in-combination-with-MuQS.patch
0013-Make-threaded-IRQs-optionally-the-default-which-can-.patch
0014-Swap-sucks.patch
0015-Enable-BFQ-io-scheduler-by-default.patch
0016-Fix-Werror-build-failure-in-tools.patch
0017-Add-ck1-version.patch


Enjoy!

お楽しみ下さい

-ck

[ANNOUNCE] linux-4.15-ck1, Multiple Queue Skiplist Scheduler version 0.170 with variable runqueue sharing.

[Con Kolivas]

Announcing a new -ck release, 4.15-ck1  with the latest version of the
Multiple Queue Skiplist Scheduler, version 0.170. These are patches
designed to improve system responsiveness and interactivity with
specific emphasis on the desktop, but configurable for any workload.


linux-4.15-ck1:
http://ck.kolivas.org/patches/4.0/4.15/4.15-ck1/

Git tree:
https://github.com/ckolivas/linux/tree/4.15-ck

MuQSS only:

Download:
http://ck.kolivas.org/patches/muqss/4.0/4.15/0001-MultiQueue-Skiplist-Scheduler-version-0.170.patch

Git tree:
https://github.com/ckolivas/linux/tree/4.15-muqss


Web: http://kernel.kolivas.org


Alas Google has falsely flagged my site as having malware, even the
kernel patch pages and they're such a monstrosity there is no place to
meaningfully appeal this so I've given up trying so you'll have to
wade through moronic warnings to get to the patches, or just use git.


The major change in this release is the addition of a much more mature
version of the experimental runqueue sharing code I posted on my blog
earlier. After further experimenting and with lots of feedback from
users, I decided to make multicore based sharing default instead of
multithread. The numbers support better throughput and it should
definitely provide more consistent low latency compared to previous
versions of MuQSS. For those that found that interactivity on MuQSS
never quite matched that of BFS before it, you should find this
version now equals it.


In addition, the runqueue sharing code in this release also allows you
to share runqueues for SMP as well so you can share runqueues with all
physical CPUs if latency is your primary concern, even though it will
likely lead to worse throughput. I have not made it possible to share
between NUMA nodes because the cost of shifting tasks across nodes is
usually substantial and it may even have worse latency, and will
definitely have worse throughput.


I've also made the runqueue sharing possible to be configured at boot
time with the boot parameter rqshare. Setting it to one of none, smt,
mc, smp is done by appending the following to your kernel command
line:

 rqshare=mc


Documentation has been added for the runqueue sharing code above to
the MuQSS patch.


A number of minor bugs were discovered and have been fixed, which has
also made booting more robust.


The -ck tree is mostly just a resync of previous patches, but with the
addition of a patch to disable a -Werror CFLAG setting in the build
tools which has suddenly made it impossible to build the kernel with
newer GCCs on some distros.

Patchlist:

0001-MultiQueue-Skiplist-Scheduler-version-0.170.patch
0002-Make-preemptible-kernel-default.patch
0003-Expose-vmsplit-for-our-poor-32-bit-users.patch
0004-Create-highres-timeout-variants-of-schedule_timeout-.patch
0005-Special-case-calls-of-schedule_timeout-1-to-use-the-.patch
0006-Convert-msleep-to-use-hrtimers-when-active.patch
0007-Replace-all-schedule-timeout-1-with-schedule_min_hrt.patch
0008-Replace-all-calls-to-schedule_timeout_interruptible-.patch
0009-Replace-all-calls-to-schedule_timeout_uninterruptibl.patch
0010-Don-t-use-hrtimer-overlay-when-pm_freezing-since-som.patch
0011-Make-hrtimer-granularity-and-minimum-hrtimeout-confi.patch
0012-Reinstate-default-Hz-of-100-in-combination-with-MuQS.patch
0013-Make-threaded-IRQs-optionally-the-default-which-can-.patch
0014-Swap-sucks.patch
0015-Enable-BFQ-io-scheduler-by-default.patch
0016-Fix-Werror-build-failure-in-tools.patch
0017-Add-ck1-version.patch


Enjoy!

お楽しみ下さい

-ck

[ANNOUNCE] linux-4.11-ck1 / MuQSS CPU scheduler 0.155

[Con Kolivas]

These are patches designed to improve system responsiveness and interactivity 
with specific emphasis on the desktop, but configurable for any workload. The 
patchset is mainly centred around the Multiple Queue Skiplist Scheduler, 
MuQSS.


linux-4.11-ck1
-ck1 patches:
http://ck.kolivas.org/patches/4.0/4.11/4.11-ck1/

Git tree:
https://github.com/ckolivas/linux/tree/4.11-ck


MuQSS
Download:
http://ck.kolivas.org/patches/muqss/4.0/4.11/4.11-sched-MuQSS_155.patch

Git tree:
https://github.com/ckolivas/linux/tree/4.11-muqss


4.11-ck1 full patch list:
0001-Multiple-Queue-Skiplist-Scheduler-version-0.155-for-.patch
0002-Make-preemptible-kernel-default.patch
0003-Expose-vmsplit-for-our-pour-32-bit-users.patch
0004-Create-highres-timeout-variants-of-schedule_timeout-.patch
0005-Special-case-calls-of-schedule_timeout-1-to-use-the-.patch
0006-Convert-msleep-to-use-hrtimers-when-active.patch
0007-Replace-all-schedule-timeout-1-with-schedule_min_hrt.patch
0008-Replace-all-calls-to-schedule_timeout_interruptible-.patch
0009-Replace-all-calls-to-schedule_timeout_uninterruptibl.patch
0010-Make-hrtimer-granularity-and-minimum-hrtimeout-confi.patch
0011-Don-t-use-hrtimer-overlay-when-pm_freezing-since-som.patch
0012-Make-threaded-IRQs-optionally-the-default-which-can-.patch
0013-Reinstate-default-Hz-of-100-in-combination-with-MuQS.patch
0014-Make-writeback-throttling-default-enabled.patch
0015-Swap-sucks.patch
0016-BFQ-v8-for-linux-4.11.patch
0017-Make-BFQ-default-IO-scheduler.patch
0018-Add-ck1-version.patch

For a brief description of each patch without trawling the git tree, each 
patch can be found as a quilt series here:
http://ck.kolivas.org/patches/4.0/4.11/4.11-ck1/patches/


MuQSS 0.155 updates:
Fixed syscall convention in the code for sched_setscheduler.
Fixed copy to user error with max CPUs enabled.
Changed skip lists to 25% probability of increased level (from 50%) scaling 
each CPU up to loads of 64k as a result.
Minor preemption code tweak.


4.11-ck1 updates:
Cleaned up the patchset.


This was the most massive resync I think I can ever recall with the mainline 
kernel thanks to the complete restructuring of the scheduler code by Mingo. 
Fortunately there weren't that many actual changes to the scheduler itself 
that I needed to port but it was still a protracted effort. Probably the most 
amusing part of the resync was seeing my name mysteriously disappear from the 
credits of sched/core.c from mainline, although everyone's names were removed 
along with it. 

Given the size of this merge, it is possible there are build configurations 
that fail, so bear with me and post your config if that's the case.

Hacking blog:
http://ck-hack.blogspot.com/

Simple homepage:
http://kernel.kolivas.org

Enjoy!
お楽しみ下さい
-- 
-ck

[ANNOUNCE] linux-4.11-ck1 / MuQSS CPU scheduler 0.155

[Con Kolivas]

These are patches designed to improve system responsiveness and interactivity 
with specific emphasis on the desktop, but configurable for any workload. The 
patchset is mainly centred around the Multiple Queue Skiplist Scheduler, 
MuQSS.


linux-4.11-ck1
-ck1 patches:
http://ck.kolivas.org/patches/4.0/4.11/4.11-ck1/

Git tree:
https://github.com/ckolivas/linux/tree/4.11-ck


MuQSS
Download:
http://ck.kolivas.org/patches/muqss/4.0/4.11/4.11-sched-MuQSS_155.patch

Git tree:
https://github.com/ckolivas/linux/tree/4.11-muqss


4.11-ck1 full patch list:
0001-Multiple-Queue-Skiplist-Scheduler-version-0.155-for-.patch
0002-Make-preemptible-kernel-default.patch
0003-Expose-vmsplit-for-our-pour-32-bit-users.patch
0004-Create-highres-timeout-variants-of-schedule_timeout-.patch
0005-Special-case-calls-of-schedule_timeout-1-to-use-the-.patch
0006-Convert-msleep-to-use-hrtimers-when-active.patch
0007-Replace-all-schedule-timeout-1-with-schedule_min_hrt.patch
0008-Replace-all-calls-to-schedule_timeout_interruptible-.patch
0009-Replace-all-calls-to-schedule_timeout_uninterruptibl.patch
0010-Make-hrtimer-granularity-and-minimum-hrtimeout-confi.patch
0011-Don-t-use-hrtimer-overlay-when-pm_freezing-since-som.patch
0012-Make-threaded-IRQs-optionally-the-default-which-can-.patch
0013-Reinstate-default-Hz-of-100-in-combination-with-MuQS.patch
0014-Make-writeback-throttling-default-enabled.patch
0015-Swap-sucks.patch
0016-BFQ-v8-for-linux-4.11.patch
0017-Make-BFQ-default-IO-scheduler.patch
0018-Add-ck1-version.patch

For a brief description of each patch without trawling the git tree, each 
patch can be found as a quilt series here:
http://ck.kolivas.org/patches/4.0/4.11/4.11-ck1/patches/


MuQSS 0.155 updates:
Fixed syscall convention in the code for sched_setscheduler.
Fixed copy to user error with max CPUs enabled.
Changed skip lists to 25% probability of increased level (from 50%) scaling 
each CPU up to loads of 64k as a result.
Minor preemption code tweak.


4.11-ck1 updates:
Cleaned up the patchset.


This was the most massive resync I think I can ever recall with the mainline 
kernel thanks to the complete restructuring of the scheduler code by Mingo. 
Fortunately there weren't that many actual changes to the scheduler itself 
that I needed to port but it was still a protracted effort. Probably the most 
amusing part of the resync was seeing my name mysteriously disappear from the 
credits of sched/core.c from mainline, although everyone's names were removed 
along with it. 

Given the size of this merge, it is possible there are build configurations 
that fail, so bear with me and post your config if that's the case.

Hacking blog:
http://ck-hack.blogspot.com/

Simple homepage:
http://kernel.kolivas.org

Enjoy!
お楽しみ下さい
-- 
-ck

[ANNOUNCE] linux-4.10-ck1 / MuQSS CPU scheduler 0.152

[Con Kolivas]

These are patches designed to improve system responsiveness and interactivity 
with specific emphasis on the desktop, but configurable for any workload. The 
patchset is mainly centred around the Multiple Queue Skiplist Scheduler, 
MuQSS.


-ck1 patches:
http://ck.kolivas.org/patches/4.0/4.10/4.10-ck1/

Git tree:
https://github.com/ckolivas/linux/tree/4.10-ck

Ubuntu 16.10 packages:
http://ck.kolivas.org/patches/4.0/4.10/4.10-ck1/Ubuntu16.10/


Patch for those struggling with the nvidia driver and 4.10:
http://ck.kolivas.org/patches/4.0/4.10/nvidia-375.39-linux-4.10.patch


Full patch list:

0001-MuQSS-150-pre-merge.patch
0002-Remove-quick-ramp-up-which-may-have-been-keeping-CPU.patch
0003-Fix-iso-tick-calculation.patch
0004-Prevent-negative-value-causing-further-overflow-in-t.patch
0005-Demote-no-cpumask-message-to-info.patch
0006-Bump-muqss-version-to-0.152.patch
0007-Make-preemptible-kernel-default.patch
0008-Expose-vmsplit-option-for-our-poor-32bit-users.patch
0009-Implement-min-and-msec-hrtimeout-un-interruptible-sc.patch
0010-Special-case-calls-of-schedule_timeout-1-to-use-the-.patch
0011-Convert-msleep-to-use-hrtimers-when-active.patch
0012-Replace-all-schedule-timeout-1-with-schedule_min_hrt.patch
0013-Change-all-schedule_timeout-with-msecs_to_jiffies-po.patch
0014-Replace-all-calls-to-schedule_timeout_interruptible-.patch
0015-Replace-all-calls-to-schedule_timeout_uninterruptibl.patch
0016-Fix-build-for-disabled-highres-timers-with-hrtimeout.patch
0017-Make-hrtimer-granularity-and-minimum-hrtimeout-confi.patch
0018-Make-threaded-IRQs-optionally-the-default-which-can-.patch
0019-Reinstate-default-Hz-of-100-in-combination-with-MuQS.patch
0020-Don-t-use-hrtimer-overlay-when-pm_freezing-since-som.patch
0021-Make-writeback-throttling-default-enabled.patch
0022-Swap-sucks.patch
0023-BFQ-version-8.patch
0024-Make-BFQ-default-IO-scheduler.patch
0025-Add-ck1-version.patch

For a brief description of each patch without trawling the git tree, each 
patch can be found as a quilt series here:
http://ck.kolivas.org/patches/4.0/4.10/4.10-ck1/patches/


Hacking blog:
http://ck-hack.blogspot.com/

Simple homepage:
http://kernel.kolivas.org


Enjoy!
お楽しみ下さい
-- 
-ck

[ANNOUNCE] linux-4.10-ck1 / MuQSS CPU scheduler 0.152

[Con Kolivas]

These are patches designed to improve system responsiveness and interactivity 
with specific emphasis on the desktop, but configurable for any workload. The 
patchset is mainly centred around the Multiple Queue Skiplist Scheduler, 
MuQSS.


-ck1 patches:
http://ck.kolivas.org/patches/4.0/4.10/4.10-ck1/

Git tree:
https://github.com/ckolivas/linux/tree/4.10-ck

Ubuntu 16.10 packages:
http://ck.kolivas.org/patches/4.0/4.10/4.10-ck1/Ubuntu16.10/


Patch for those struggling with the nvidia driver and 4.10:
http://ck.kolivas.org/patches/4.0/4.10/nvidia-375.39-linux-4.10.patch


Full patch list:

0001-MuQSS-150-pre-merge.patch
0002-Remove-quick-ramp-up-which-may-have-been-keeping-CPU.patch
0003-Fix-iso-tick-calculation.patch
0004-Prevent-negative-value-causing-further-overflow-in-t.patch
0005-Demote-no-cpumask-message-to-info.patch
0006-Bump-muqss-version-to-0.152.patch
0007-Make-preemptible-kernel-default.patch
0008-Expose-vmsplit-option-for-our-poor-32bit-users.patch
0009-Implement-min-and-msec-hrtimeout-un-interruptible-sc.patch
0010-Special-case-calls-of-schedule_timeout-1-to-use-the-.patch
0011-Convert-msleep-to-use-hrtimers-when-active.patch
0012-Replace-all-schedule-timeout-1-with-schedule_min_hrt.patch
0013-Change-all-schedule_timeout-with-msecs_to_jiffies-po.patch
0014-Replace-all-calls-to-schedule_timeout_interruptible-.patch
0015-Replace-all-calls-to-schedule_timeout_uninterruptibl.patch
0016-Fix-build-for-disabled-highres-timers-with-hrtimeout.patch
0017-Make-hrtimer-granularity-and-minimum-hrtimeout-confi.patch
0018-Make-threaded-IRQs-optionally-the-default-which-can-.patch
0019-Reinstate-default-Hz-of-100-in-combination-with-MuQS.patch
0020-Don-t-use-hrtimer-overlay-when-pm_freezing-since-som.patch
0021-Make-writeback-throttling-default-enabled.patch
0022-Swap-sucks.patch
0023-BFQ-version-8.patch
0024-Make-BFQ-default-IO-scheduler.patch
0025-Add-ck1-version.patch

For a brief description of each patch without trawling the git tree, each 
patch can be found as a quilt series here:
http://ck.kolivas.org/patches/4.0/4.10/4.10-ck1/patches/


Hacking blog:
http://ck-hack.blogspot.com/

Simple homepage:
http://kernel.kolivas.org


Enjoy!
お楽しみ下さい
-- 
-ck

[ANNOUNCE] linux-4.9-ck1

[Con Kolivas]

These are patches designed to improve system responsiveness and interactivity 
with specific emphasis on the desktop, but configurable for any workload. The 
patchset is mainly centred around the Multiple Queue Skiplist Scheduler, 
MuQSS.


-ck1 patches:
http://ck.kolivas.org/patches/4.0/4.9/4.9-ck1/

Git tree:
https://github.com/ckolivas/linux/tree/4.9-ck

Ubuntu 16.04 LTS packages:
http://ck.kolivas.org/patches/4.0/4.9/4.9-ck1/Ubuntu16.04/


Full patch list:

0001-Multiple-Queue-Skiplist-Scheduler-version-0.15.patch
0002-Make-preemptible-kernel-default.patch
0003-Expose-vmsplit-option-for-our-poor-32bit-users.patch
0004-Implement-min-and-msec-hrtimeout-un-interruptible-sc.patch
0005-Special-case-calls-of-schedule_timeout-1-to-use-the-.patch
0006-Convert-msleep-to-use-hrtimers-when-active.patch
0007-Replace-all-schedule_timeout-1-with-schedule_min_hrt.patch
0008-Change-all-schedule_timeout-with-msecs_to_jiffies-po.patch
0009-Replace-all-calls-to-schedule_timeout_interruptible-.patch
0010-Replace-all-calls-to-schedule_timeout_uninterruptibl.patch
0011-Fix-build-for-disabled-highres-timers-with-hrtimeout.patch
0012-Make-hrtimer-granularity-and-minimum-hrtimeout-confi.patch
0013-Make-threaded-IRQs-optionally-the-default-which-can-.patch
0014-Reinstate-default-Hz-of-100-in-combination-with-MuQS.patch
0015-Don-t-use-hrtimer-overlay-when-pm_freezing-since-som.patch
0016-wb-buf-throttle-v7.patch
0017-4.9-bfq-v8.patch
0018-Make-writeback-throttling-default-enabled.patch
0019-Add-ck1-version.patch


For a brief description of each patch without trawling the git tree, each 
patch can be found as a quilt series here:
http://ck.kolivas.org/patches/4.0/4.9/4.9-ck1/patches/


Hacking blog:
http://ck-hack.blogspot.com/

Simple homepage:
http://kernel.kolivas.org


Enjoy!
お楽しみ下さい
-- 
-ck

[ANNOUNCE] linux-4.9-ck1

[Con Kolivas]

These are patches designed to improve system responsiveness and interactivity 
with specific emphasis on the desktop, but configurable for any workload. The 
patchset is mainly centred around the Multiple Queue Skiplist Scheduler, 
MuQSS.


-ck1 patches:
http://ck.kolivas.org/patches/4.0/4.9/4.9-ck1/

Git tree:
https://github.com/ckolivas/linux/tree/4.9-ck

Ubuntu 16.04 LTS packages:
http://ck.kolivas.org/patches/4.0/4.9/4.9-ck1/Ubuntu16.04/


Full patch list:

0001-Multiple-Queue-Skiplist-Scheduler-version-0.15.patch
0002-Make-preemptible-kernel-default.patch
0003-Expose-vmsplit-option-for-our-poor-32bit-users.patch
0004-Implement-min-and-msec-hrtimeout-un-interruptible-sc.patch
0005-Special-case-calls-of-schedule_timeout-1-to-use-the-.patch
0006-Convert-msleep-to-use-hrtimers-when-active.patch
0007-Replace-all-schedule_timeout-1-with-schedule_min_hrt.patch
0008-Change-all-schedule_timeout-with-msecs_to_jiffies-po.patch
0009-Replace-all-calls-to-schedule_timeout_interruptible-.patch
0010-Replace-all-calls-to-schedule_timeout_uninterruptibl.patch
0011-Fix-build-for-disabled-highres-timers-with-hrtimeout.patch
0012-Make-hrtimer-granularity-and-minimum-hrtimeout-confi.patch
0013-Make-threaded-IRQs-optionally-the-default-which-can-.patch
0014-Reinstate-default-Hz-of-100-in-combination-with-MuQS.patch
0015-Don-t-use-hrtimer-overlay-when-pm_freezing-since-som.patch
0016-wb-buf-throttle-v7.patch
0017-4.9-bfq-v8.patch
0018-Make-writeback-throttling-default-enabled.patch
0019-Add-ck1-version.patch


For a brief description of each patch without trawling the git tree, each 
patch can be found as a quilt series here:
http://ck.kolivas.org/patches/4.0/4.9/4.9-ck1/patches/


Hacking blog:
http://ck-hack.blogspot.com/

Simple homepage:
http://kernel.kolivas.org


Enjoy!
お楽しみ下さい
-- 
-ck

[ANNOUNCE] MuQSS CPU scheduler v0.15 for linux-4.9

[Con Kolivas]

Announcing an updated stable version of the Multiple Queue Skiplist Scheduler, 
the successor to BFS, version 0.150 for linux-4.9.

Download:
http://ck.kolivas.org/patches/muqss/4.0/4.9/4.9-sched-MuQSS_150.patch

Git tree:
https://github.com/ckolivas/linux/tree/4.9-muqss

--- 

Patch summary:

The MuQSS (Multiple Queue Skiplist Scheduler - pronounced mux) v0.150 by
Con Kolivas.

This is a multiple runqueue skiplist evolution of the Brain Fuck Scheduler,
designed to provide excellent latency, throughput and scalability to any
number of CPUs, with primary emphasis on latency for interactivity and
responsiveness.

A multiple runqueue strict fairness earliest effective virtual deadline first
design.

Runqueue insertion is O(log(n)), lookup is O(1), removal is amortised O(1).

Interactive mode is enabled by default but can be disabled for improved
throughput at the expense of deterministic low latency.

echo 0 > /proc/sys/kernel/interactive

Features SCHED_IDLEPRIO and SCHED_ISO scheduling policies as well.
You do NOT need to use these policies for good performance, they are purely
optional for even better performance in extreme conditions.

To run something idleprio, use schedtool like so:

schedtool -D -e make -j4

To run something isoprio, use schedtool like so:

schedtool -I -e amarok

Includes configurable SMT-nice support for better nice level and scheduling
policy support across SMT (aka hyperthread) sibling CPUs.

Includes accurate sub-tick accounting of tasks so userspace reported
cpu usage may be very different if you have very short lived tasks.

-ck

>From 0ce996c80e2c547e8bc9cfb23f028d80f34a5210 Mon Sep 17 00:00:00 2001
From: Con Kolivas <ker...@kolivas.org>
Date: Sat, 10 Dec 2016 13:37:55 +1100
Subject: [PATCH 01/19] Multiple Queue Skiplist Scheduler version 0.15

---
 Documentation/scheduler/sched-BFS.txt |  351 ++
 Documentation/scheduler/sched-MuQSS.txt   |  345 ++
 Documentation/sysctl/kernel.txt   |   37 +
 arch/powerpc/platforms/cell/spufs/sched.c |5 -
 arch/x86/Kconfig  |   18 +-
 fs/proc/base.c|2 +-
 include/linux/init_task.h |   76 +-
 include/linux/ioprio.h|2 +
 include/linux/sched.h |   69 +-
 include/linux/sched/prio.h|   12 +
 include/linux/skip_list.h |   33 +
 include/uapi/linux/sched.h|9 +-
 init/Kconfig  |   25 +-
 init/main.c   |3 +-
 kernel/Makefile   |2 +-
 kernel/delayacct.c|2 +-
 kernel/exit.c |2 +-
 kernel/kthread.c  |   30 +-
 kernel/sched/Makefile |   13 +-
 kernel/sched/MuQSS.c  | 8033 
+
 kernel/sched/MuQSS.h  |  348 ++
 kernel/sched/cpufreq.c|4 +
 kernel/sched/cpufreq_schedutil.c  |   16 +
 kernel/sched/cputime.c|   27 +-
 kernel/sched/idle.c   |   14 +-
 kernel/sched/sched.h  |   25 +
 kernel/sched/stats.c  |4 +
 kernel/skip_list.c|  148 +
 kernel/sysctl.c   |   52 +-
 kernel/time/clockevents.c |5 +
 kernel/time/posix-cpu-timers.c|   10 +-
 kernel/time/timer.c   |7 +-
 kernel/trace/trace_selftest.c |5 +
 33 files changed, 9670 insertions(+), 64 deletions(-)
 create mode 100644 Documentation/scheduler/sched-BFS.txt
 create mode 100644 Documentation/scheduler/sched-MuQSS.txt
 create mode 100644 include/linux/skip_list.h
 create mode 100644 kernel/sched/MuQSS.c
 create mode 100644 kernel/sched/MuQSS.h
 create mode 100644 kernel/skip_list.c

-- 
-ck

[ANNOUNCE] MuQSS CPU scheduler v0.15 for linux-4.9

[Con Kolivas]

Announcing an updated stable version of the Multiple Queue Skiplist Scheduler, 
the successor to BFS, version 0.150 for linux-4.9.

Download:
http://ck.kolivas.org/patches/muqss/4.0/4.9/4.9-sched-MuQSS_150.patch

Git tree:
https://github.com/ckolivas/linux/tree/4.9-muqss

--- 

Patch summary:

The MuQSS (Multiple Queue Skiplist Scheduler - pronounced mux) v0.150 by
Con Kolivas.

This is a multiple runqueue skiplist evolution of the Brain Fuck Scheduler,
designed to provide excellent latency, throughput and scalability to any
number of CPUs, with primary emphasis on latency for interactivity and
responsiveness.

A multiple runqueue strict fairness earliest effective virtual deadline first
design.

Runqueue insertion is O(log(n)), lookup is O(1), removal is amortised O(1).

Interactive mode is enabled by default but can be disabled for improved
throughput at the expense of deterministic low latency.

echo 0 > /proc/sys/kernel/interactive

Features SCHED_IDLEPRIO and SCHED_ISO scheduling policies as well.
You do NOT need to use these policies for good performance, they are purely
optional for even better performance in extreme conditions.

To run something idleprio, use schedtool like so:

schedtool -D -e make -j4

To run something isoprio, use schedtool like so:

schedtool -I -e amarok

Includes configurable SMT-nice support for better nice level and scheduling
policy support across SMT (aka hyperthread) sibling CPUs.

Includes accurate sub-tick accounting of tasks so userspace reported
cpu usage may be very different if you have very short lived tasks.

-ck

>From 0ce996c80e2c547e8bc9cfb23f028d80f34a5210 Mon Sep 17 00:00:00 2001
From: Con Kolivas 
Date: Sat, 10 Dec 2016 13:37:55 +1100
Subject: [PATCH 01/19] Multiple Queue Skiplist Scheduler version 0.15

---
 Documentation/scheduler/sched-BFS.txt |  351 ++
 Documentation/scheduler/sched-MuQSS.txt   |  345 ++
 Documentation/sysctl/kernel.txt   |   37 +
 arch/powerpc/platforms/cell/spufs/sched.c |5 -
 arch/x86/Kconfig  |   18 +-
 fs/proc/base.c|2 +-
 include/linux/init_task.h |   76 +-
 include/linux/ioprio.h|2 +
 include/linux/sched.h |   69 +-
 include/linux/sched/prio.h|   12 +
 include/linux/skip_list.h |   33 +
 include/uapi/linux/sched.h|9 +-
 init/Kconfig  |   25 +-
 init/main.c   |3 +-
 kernel/Makefile   |2 +-
 kernel/delayacct.c|2 +-
 kernel/exit.c |2 +-
 kernel/kthread.c  |   30 +-
 kernel/sched/Makefile |   13 +-
 kernel/sched/MuQSS.c  | 8033 
+
 kernel/sched/MuQSS.h  |  348 ++
 kernel/sched/cpufreq.c|4 +
 kernel/sched/cpufreq_schedutil.c  |   16 +
 kernel/sched/cputime.c|   27 +-
 kernel/sched/idle.c   |   14 +-
 kernel/sched/sched.h  |   25 +
 kernel/sched/stats.c  |4 +
 kernel/skip_list.c|  148 +
 kernel/sysctl.c   |   52 +-
 kernel/time/clockevents.c |5 +
 kernel/time/posix-cpu-timers.c|   10 +-
 kernel/time/timer.c   |7 +-
 kernel/trace/trace_selftest.c |5 +
 33 files changed, 9670 insertions(+), 64 deletions(-)
 create mode 100644 Documentation/scheduler/sched-BFS.txt
 create mode 100644 Documentation/scheduler/sched-MuQSS.txt
 create mode 100644 include/linux/skip_list.h
 create mode 100644 kernel/sched/MuQSS.c
 create mode 100644 kernel/sched/MuQSS.h
 create mode 100644 kernel/skip_list.c

-- 
-ck

Re: [PATCH SND/USB]: Add QuickCam Communicate Deluxe/S7500 to volume_control_quirks.

[Con Kolivas]

On Friday, 9 December 2016 11:22:15 AM AEDT Takashi Iwai wrote:
> On Fri, 09 Dec 2016 05:15:57 +0100,
> 
> Con Kolivas wrote:
> > The Logitech QuickCam Communicate Deluxe/S7500 microphone fails with the
> > following warning.
> > 
> > [6.778995] usb 2-1.2.2.2: Warning! Unlikely big volume range (=3072),
> > cval->res is probably wrong.
> > [6.778996] usb 2-1.2.2.2: [5] FU [Mic Capture Volume] ch = 1, val =
> > 4608/7680/1
> > 
> > Adding it to the list of devices in volume_control_quirks makes it work
> > properly, fixing related typo.
> > 
> > Signed-off-by: Con Kolivas <ker...@kolivas.org>
> 
> Applied (with Cc to stable), thanks.

(Apologies for google mail defaulting to html, resending with a proper mail 
client)

You're welcome. As there appear to be a lot of these devices from the same 
manufacturer with the same error, and there are likely a lot of devices out 
there that are missed from this whitelist, do you think it would be worthwhile 
changing the code to detect the combination of unlikely big volume range and 
usb vendor of 0x046d and automatically try using the volume_control_quirk on 
them instead of a discrete whitelist?


Thanks,

Con

> 
> 
> Takashi
> 
> > ---
> > 
> >  sound/usb/mixer.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/sound/usb/mixer.c b/sound/usb/mixer.c
> > index 2f8c388..4703cae 100644
> > --- a/sound/usb/mixer.c
> > +++ b/sound/usb/mixer.c
> > @@ -932,9 +932,10 @@ static void volume_control_quirks(struct
> > usb_mixer_elem_info *cval,> 
> > case USB_ID(0x046d, 0x0826): /* HD Webcam c525 */
> > case USB_ID(0x046d, 0x08ca): /* Logitech Quickcam Fusion */
> > 
> > case USB_ID(0x046d, 0x0991):
> > +   case USB_ID(0x046d, 0x09a2): /* QuickCam Communicate Deluxe/S7500 */
> > 
> > /* Most audio usb devices lie about volume resolution.
> > 
> >  * Most Logitech webcams have res = 384.
> > 
> > -* Proboly there is some logitech magic behind this number --fishor
> > +* Probably there is some logitech magic behind this number --fishor
> > 
> >  */
> >  
> > if (!strcmp(kctl->id.name, "Mic Capture Volume")) {
> > 
> > usb_audio_info(chip,


-- 
-ck

Re: [PATCH SND/USB]: Add QuickCam Communicate Deluxe/S7500 to volume_control_quirks.

[Con Kolivas]

On Friday, 9 December 2016 11:22:15 AM AEDT Takashi Iwai wrote:
> On Fri, 09 Dec 2016 05:15:57 +0100,
> 
> Con Kolivas wrote:
> > The Logitech QuickCam Communicate Deluxe/S7500 microphone fails with the
> > following warning.
> > 
> > [6.778995] usb 2-1.2.2.2: Warning! Unlikely big volume range (=3072),
> > cval->res is probably wrong.
> > [6.778996] usb 2-1.2.2.2: [5] FU [Mic Capture Volume] ch = 1, val =
> > 4608/7680/1
> > 
> > Adding it to the list of devices in volume_control_quirks makes it work
> > properly, fixing related typo.
> > 
> > Signed-off-by: Con Kolivas 
> 
> Applied (with Cc to stable), thanks.

(Apologies for google mail defaulting to html, resending with a proper mail 
client)

You're welcome. As there appear to be a lot of these devices from the same 
manufacturer with the same error, and there are likely a lot of devices out 
there that are missed from this whitelist, do you think it would be worthwhile 
changing the code to detect the combination of unlikely big volume range and 
usb vendor of 0x046d and automatically try using the volume_control_quirk on 
them instead of a discrete whitelist?


Thanks,

Con

> 
> 
> Takashi
> 
> > ---
> > 
> >  sound/usb/mixer.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/sound/usb/mixer.c b/sound/usb/mixer.c
> > index 2f8c388..4703cae 100644
> > --- a/sound/usb/mixer.c
> > +++ b/sound/usb/mixer.c
> > @@ -932,9 +932,10 @@ static void volume_control_quirks(struct
> > usb_mixer_elem_info *cval,> 
> > case USB_ID(0x046d, 0x0826): /* HD Webcam c525 */
> > case USB_ID(0x046d, 0x08ca): /* Logitech Quickcam Fusion */
> > 
> > case USB_ID(0x046d, 0x0991):
> > +   case USB_ID(0x046d, 0x09a2): /* QuickCam Communicate Deluxe/S7500 */
> > 
> > /* Most audio usb devices lie about volume resolution.
> > 
> >  * Most Logitech webcams have res = 384.
> > 
> > -* Proboly there is some logitech magic behind this number --fishor
> > +* Probably there is some logitech magic behind this number --fishor
> > 
> >  */
> >  
> > if (!strcmp(kctl->id.name, "Mic Capture Volume")) {
> > 
> > usb_audio_info(chip,


-- 
-ck

[PATCH SND/USB]: Add QuickCam Communicate Deluxe/S7500 to volume_control_quirks.

[Con Kolivas]

The Logitech QuickCam Communicate Deluxe/S7500 microphone fails with the
following warning.

[6.778995] usb 2-1.2.2.2: Warning! Unlikely big volume range (=3072),
cval->res is probably wrong.
[6.778996] usb 2-1.2.2.2: [5] FU [Mic Capture Volume] ch = 1, val =
4608/7680/1

Adding it to the list of devices in volume_control_quirks makes it work
properly, fixing related typo.

Signed-off-by: Con Kolivas <ker...@kolivas.org>

---
 sound/usb/mixer.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/sound/usb/mixer.c b/sound/usb/mixer.c
index 2f8c388..4703cae 100644
--- a/sound/usb/mixer.c
+++ b/sound/usb/mixer.c
@@ -932,9 +932,10 @@ static void volume_control_quirks(struct 
usb_mixer_elem_info *cval,
case USB_ID(0x046d, 0x0826): /* HD Webcam c525 */
case USB_ID(0x046d, 0x08ca): /* Logitech Quickcam Fusion */
case USB_ID(0x046d, 0x0991):
+   case USB_ID(0x046d, 0x09a2): /* QuickCam Communicate Deluxe/S7500 */
/* Most audio usb devices lie about volume resolution.
 * Most Logitech webcams have res = 384.
-* Proboly there is some logitech magic behind this number --fishor
+* Probably there is some logitech magic behind this number --fishor
 */
if (!strcmp(kctl->id.name, "Mic Capture Volume")) {
usb_audio_info(chip,
-- 
2.7.4

[PATCH SND/USB]: Add QuickCam Communicate Deluxe/S7500 to volume_control_quirks.

[Con Kolivas]

The Logitech QuickCam Communicate Deluxe/S7500 microphone fails with the
following warning.

[6.778995] usb 2-1.2.2.2: Warning! Unlikely big volume range (=3072),
cval->res is probably wrong.
[6.778996] usb 2-1.2.2.2: [5] FU [Mic Capture Volume] ch = 1, val =
4608/7680/1

Adding it to the list of devices in volume_control_quirks makes it work
properly, fixing related typo.

Signed-off-by: Con Kolivas 

---
 sound/usb/mixer.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/sound/usb/mixer.c b/sound/usb/mixer.c
index 2f8c388..4703cae 100644
--- a/sound/usb/mixer.c
+++ b/sound/usb/mixer.c
@@ -932,9 +932,10 @@ static void volume_control_quirks(struct 
usb_mixer_elem_info *cval,
case USB_ID(0x046d, 0x0826): /* HD Webcam c525 */
case USB_ID(0x046d, 0x08ca): /* Logitech Quickcam Fusion */
case USB_ID(0x046d, 0x0991):
+   case USB_ID(0x046d, 0x09a2): /* QuickCam Communicate Deluxe/S7500 */
/* Most audio usb devices lie about volume resolution.
 * Most Logitech webcams have res = 384.
-* Proboly there is some logitech magic behind this number --fishor
+* Probably there is some logitech magic behind this number --fishor
 */
if (!strcmp(kctl->id.name, "Mic Capture Volume")) {
usb_audio_info(chip,
-- 
2.7.4

[ANNOUNCE] Multiple Queue Skiplist Scheduler version 0.120

[Con Kolivas]

This is to announce the first major ~stable public release of MuQSS (pronounced 
mux), the Multiple Queue Skiplist Scheduler. 

MuQSS for linux 4.8:
http://ck.kolivas.org/patches/muqss/4.0/4.8/4.8-sched-MuQSS_120.patch

MuQSS for linux 4.7:
http://ck.kolivas.org/patches/muqss/4.0/4.7/4.7-sched-MuQSS_120.patch

Git tree:
https://github.com/ckolivas/linux


MuQSS is the evolution and drop-in replacement for BFS, the Brain Fuck 
Scheduler, both of which were designed with responsiveness and interactivity 
as their primary goal. MuQSS is my response for requests for a more scalable 
version of BFS for our ever-increasing multicore hardware. It is a massive 
rewrite of BFS designed to maintain the same interactivity and responsiveness 
using the same algorithm for scheduling decisions, along with a very simple 
overall design that is easy to understand, model, and hack on, yet able to 
scale to hardware of virtually any size.

It is meant as a replacement for BFS primarily, and NOT a comprehensive 
replacement for the mainline scheduler since it lacks some of the features of 
mainline still (specifically cgroups and sched deadline support.) However 
outside of these feature requirements it performs better in latency and 
responsiveness areas and, while it performs poorer in some throughput 
benchmarks, it also performs better in many others. Additionally MuQSS has a 
number of unique features missing from mainline described in the full 
documentation below. Results from early 12x benchmarks below.

Interbench interactivity comparison:
http://ck-hack.blogspot.com/2016/10/interbench-benchmarks-for-muqss-116.html

Throughput comparison:
http://ck.kolivas.org/patches/muqss/Benchmarks/20161018/

Very little effort has gone into optimising throughput in its existing form, 
being effectively a first stable working version of the scheduler, though 
clearly it does well in some areas (postgres particularly.)

To be clear what my intentions are, this will be maintained out of mainline 
development - as BFS was - for those that wish an alternative scheduler with a 
different focus and set of features. There is plenty of interest in this 
outside of mainline development and there are some perverse things I do in the 
code that would make most mainline developers' skin crawl, but application and 
utility are far more important than technology in my opinion. I do not have 
the energy and time to engage with the mainline development process alone to 
try and make this replace the existing scheduler though I'm always happy to 
accept outside help and patches for this project.

Below is the complete documentation as included in the patch.

Enjoy!
お楽しみください
-- 
-ck

MuQSS - The Multiple Queue Skiplist Scheduler by Con Kolivas.

MuQSS is a per-cpu runqueue variant of the original BFS scheduler with
one 8 level skiplist per runqueue, and fine grained locking for much more
scalability.


Goals.

The goal of the Multiple Queue Skiplist Scheduler, referred to as MuQSS from
here on (pronounced mux) is to completely do away with the complex designs of
the past for the cpu process scheduler and instead implement one that is very
simple in basic design. The main focus of MuQSS is to achieve excellent 
desktop
interactivity and responsiveness without heuristics and tuning knobs that are
difficult to understand, impossible to model and predict the effect of, and 
when
tuned to one workload cause massive detriment to another, while still being
scalable to many CPUs and processes.


Design summary.

MuQSS is best described as per-cpu multiple runqueue, O(log n) insertion, O(1)
lookup, earliest effective virtual deadline first tickless design, loosely 
based
on EEVDF (earliest eligible virtual deadline first) and my previous Staircase
Deadline scheduler, and evolved from the single runqueue O(n) BFS scheduler.
Each component shall be described in order to understand the significance of,
and reasoning for it.


Design reasoning.

In BFS, the use of a single runqueue across all CPUs meant that each CPU would
need to scan the entire runqueue looking for the process with the earliest
deadline and schedule that next, regardless of which CPU it originally came
from. This made BFS deterministic with respect to latency and provided
guaranteed latencies dependent on number of processes and CPUs. The single
runqueue, however, meant that all CPUs would compete for the single lock
protecting it, which would lead to increasing lock contention as the number of
CPUs rose and appeared to limit scalability of common workloads beyond 16
logical CPUs. Additionally, the O(n) lookup of the runqueue list obviously
increased overhead proportionate to the number of queued proecesses and led to
cache thrashing while iterating over the linked list.

MuQSS is an evolution of BFS, designed to maintain the same scheduling
decision mechanism and be virtually deterministic without relying on the
constrained design of the single runqueue by splitting out the single runqueue

[ANNOUNCE] Multiple Queue Skiplist Scheduler version 0.120

[Con Kolivas]

This is to announce the first major ~stable public release of MuQSS (pronounced 
mux), the Multiple Queue Skiplist Scheduler. 

MuQSS for linux 4.8:
http://ck.kolivas.org/patches/muqss/4.0/4.8/4.8-sched-MuQSS_120.patch

MuQSS for linux 4.7:
http://ck.kolivas.org/patches/muqss/4.0/4.7/4.7-sched-MuQSS_120.patch

Git tree:
https://github.com/ckolivas/linux


MuQSS is the evolution and drop-in replacement for BFS, the Brain Fuck 
Scheduler, both of which were designed with responsiveness and interactivity 
as their primary goal. MuQSS is my response for requests for a more scalable 
version of BFS for our ever-increasing multicore hardware. It is a massive 
rewrite of BFS designed to maintain the same interactivity and responsiveness 
using the same algorithm for scheduling decisions, along with a very simple 
overall design that is easy to understand, model, and hack on, yet able to 
scale to hardware of virtually any size.

It is meant as a replacement for BFS primarily, and NOT a comprehensive 
replacement for the mainline scheduler since it lacks some of the features of 
mainline still (specifically cgroups and sched deadline support.) However 
outside of these feature requirements it performs better in latency and 
responsiveness areas and, while it performs poorer in some throughput 
benchmarks, it also performs better in many others. Additionally MuQSS has a 
number of unique features missing from mainline described in the full 
documentation below. Results from early 12x benchmarks below.

Interbench interactivity comparison:
http://ck-hack.blogspot.com/2016/10/interbench-benchmarks-for-muqss-116.html

Throughput comparison:
http://ck.kolivas.org/patches/muqss/Benchmarks/20161018/

Very little effort has gone into optimising throughput in its existing form, 
being effectively a first stable working version of the scheduler, though 
clearly it does well in some areas (postgres particularly.)

To be clear what my intentions are, this will be maintained out of mainline 
development - as BFS was - for those that wish an alternative scheduler with a 
different focus and set of features. There is plenty of interest in this 
outside of mainline development and there are some perverse things I do in the 
code that would make most mainline developers' skin crawl, but application and 
utility are far more important than technology in my opinion. I do not have 
the energy and time to engage with the mainline development process alone to 
try and make this replace the existing scheduler though I'm always happy to 
accept outside help and patches for this project.

Below is the complete documentation as included in the patch.

Enjoy!
お楽しみください
-- 
-ck

MuQSS - The Multiple Queue Skiplist Scheduler by Con Kolivas.

MuQSS is a per-cpu runqueue variant of the original BFS scheduler with
one 8 level skiplist per runqueue, and fine grained locking for much more
scalability.


Goals.

The goal of the Multiple Queue Skiplist Scheduler, referred to as MuQSS from
here on (pronounced mux) is to completely do away with the complex designs of
the past for the cpu process scheduler and instead implement one that is very
simple in basic design. The main focus of MuQSS is to achieve excellent 
desktop
interactivity and responsiveness without heuristics and tuning knobs that are
difficult to understand, impossible to model and predict the effect of, and 
when
tuned to one workload cause massive detriment to another, while still being
scalable to many CPUs and processes.


Design summary.

MuQSS is best described as per-cpu multiple runqueue, O(log n) insertion, O(1)
lookup, earliest effective virtual deadline first tickless design, loosely 
based
on EEVDF (earliest eligible virtual deadline first) and my previous Staircase
Deadline scheduler, and evolved from the single runqueue O(n) BFS scheduler.
Each component shall be described in order to understand the significance of,
and reasoning for it.


Design reasoning.

In BFS, the use of a single runqueue across all CPUs meant that each CPU would
need to scan the entire runqueue looking for the process with the earliest
deadline and schedule that next, regardless of which CPU it originally came
from. This made BFS deterministic with respect to latency and provided
guaranteed latencies dependent on number of processes and CPUs. The single
runqueue, however, meant that all CPUs would compete for the single lock
protecting it, which would lead to increasing lock contention as the number of
CPUs rose and appeared to limit scalability of common workloads beyond 16
logical CPUs. Additionally, the O(n) lookup of the runqueue list obviously
increased overhead proportionate to the number of queued proecesses and led to
cache thrashing while iterating over the linked list.

MuQSS is an evolution of BFS, designed to maintain the same scheduling
decision mechanism and be virtually deterministic without relying on the
constrained design of the single runqueue by splitting out the single runqueue

[ANNOUNCE] BFS CPU scheduler v0.512 for linux-4.8, 4.8-ck1

[Con Kolivas]

This is to announce an updated stable version of the Brain Fuck Scheduler, 
version 0.512 for the current stable linux kernel for improved responsiveness 
and interactivity.

http://ck.kolivas.org/patches/bfs/4.0/4.8/4.8-sched-bfs-512.patch


A -ck branded release with minor tweaks and the addition of Jens' writeback 
throttling v7 (I'm watching the BFQ discussions with great interest at the 
moment) is also available as -ck1:

http://ck.kolivas.org/patches/4.0/4.8/4.8-ck1/patch-4.8-ck1.xz


Note that this may be the last BFS based -ck release as the Multiple Queue 
Skiplist Scheduler, MuQSS (pronounced mux) is designed to replace it, being 
the logical evolution of the same scheduler into a more scalable discrete 
runqueue design with an identical scheduling decision mechanism as BFS.

See introduction here:

http://ck-hack.blogspot.com/2016/10/muqss-multiple-queue-skiplist-scheduler.html


It is still immature and requires more testing and no doubt bug-fixes but is 
usable in its current form already - although lock debugging may be very 
unhappy for it does perverse things with locking. For those willing to try it 
in its current version, an incremental patch can be applied to BFS 512:

http://ck.kolivas.org/patches/muqss/4.0/4.8/bfs512-muqss104.patch


or there is a full patch against 4.8:

http://ck.kolivas.org/patches/muqss/4.0/4.8/4.8-sched-MuQSS_104.patch


Feedback and bug reports are most welcome, though bear in mind that it is 
(intentionally) extremely simple in its balancing design between CPUs in its 
current form and there may be workloads that benefit more from something like 
the complex balancing mechanism in mainline - although the opposite may also 
be true.


All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com


Enjoy!
お楽しみ下さい

-- 
-ck

[ANNOUNCE] BFS CPU scheduler v0.512 for linux-4.8, 4.8-ck1

[Con Kolivas]

This is to announce an updated stable version of the Brain Fuck Scheduler, 
version 0.512 for the current stable linux kernel for improved responsiveness 
and interactivity.

http://ck.kolivas.org/patches/bfs/4.0/4.8/4.8-sched-bfs-512.patch


A -ck branded release with minor tweaks and the addition of Jens' writeback 
throttling v7 (I'm watching the BFQ discussions with great interest at the 
moment) is also available as -ck1:

http://ck.kolivas.org/patches/4.0/4.8/4.8-ck1/patch-4.8-ck1.xz


Note that this may be the last BFS based -ck release as the Multiple Queue 
Skiplist Scheduler, MuQSS (pronounced mux) is designed to replace it, being 
the logical evolution of the same scheduler into a more scalable discrete 
runqueue design with an identical scheduling decision mechanism as BFS.

See introduction here:

http://ck-hack.blogspot.com/2016/10/muqss-multiple-queue-skiplist-scheduler.html


It is still immature and requires more testing and no doubt bug-fixes but is 
usable in its current form already - although lock debugging may be very 
unhappy for it does perverse things with locking. For those willing to try it 
in its current version, an incremental patch can be applied to BFS 512:

http://ck.kolivas.org/patches/muqss/4.0/4.8/bfs512-muqss104.patch


or there is a full patch against 4.8:

http://ck.kolivas.org/patches/muqss/4.0/4.8/4.8-sched-MuQSS_104.patch


Feedback and bug reports are most welcome, though bear in mind that it is 
(intentionally) extremely simple in its balancing design between CPUs in its 
current form and there may be workloads that benefit more from something like 
the complex balancing mechanism in mainline - although the opposite may also 
be true.


All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com


Enjoy!
お楽しみ下さい

-- 
-ck

[ANNOUNCE] linux-4.7-ck5

[Con Kolivas]

Announcing the latest release of the -ck patchset for improved responsiveness 
and interactivity.

http://ck.kolivas.org/patches/4.0/4.7/4.7-ck5/

This is normally just a branded version of BFS with some different default 
kernel options, however this version incorporates Jens Axboe's writeback 
throttling patch version 7 which in my testing made a dramatic improvement to 
behaviour under heavy write loads (no benchmarks.)

https://marc.info/?l=linux-block=147325975312628

A note about CPU scheduler cgroups - BFS does not support any of the cgroup 
features but does implement a basic stub for the primary "CPU controller" 
cgroup which creates the relevant cgroup filesystem but using it does nothing. 
The reason for implementing this was that some applications now refuse to work 
without it though they'll happily work with these stubs, even if they don't do 
anything.

Enjoy!
お楽しみ下さい
-- 
-ck

[ANNOUNCE] linux-4.7-ck5

[Con Kolivas]

Announcing the latest release of the -ck patchset for improved responsiveness 
and interactivity.

http://ck.kolivas.org/patches/4.0/4.7/4.7-ck5/

This is normally just a branded version of BFS with some different default 
kernel options, however this version incorporates Jens Axboe's writeback 
throttling patch version 7 which in my testing made a dramatic improvement to 
behaviour under heavy write loads (no benchmarks.)

https://marc.info/?l=linux-block=147325975312628

A note about CPU scheduler cgroups - BFS does not support any of the cgroup 
features but does implement a basic stub for the primary "CPU controller" 
cgroup which creates the relevant cgroup filesystem but using it does nothing. 
The reason for implementing this was that some applications now refuse to work 
without it though they'll happily work with these stubs, even if they don't do 
anything.

Enjoy!
お楽しみ下さい
-- 
-ck

[ANNOUNCE] BFS CPU scheduler v0.502 for linux-4.7 with skip list.

[Con Kolivas]

This is to announce an updated stable version of the Brain Fuck Scheduler, 
version 0.502 for the current stable linux kernel. 

http://ck.kolivas.org/patches/bfs/4.0/4.7/4.7-sched-bfs-502.patch

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

I was reluctant to announce previous versions as the cpu frequency load 
signalling changes mandated by mainline were initially poorly handled and then 
buggy in BFS until recently.

In 2011 I played with skip lists to replace the simple linked list that is 
used in BFS and after a brief period of testing did not find it advantageous 
though very little effort was putting into optimising the skip list 
implementation. Additionally I was unable to maintain the scheduling decisions 
entirely with the existing BFS implementation.
 
Recently I found renewed interest in improving the most obvious flaws in the 
design and trying again and found I could maintain the behaviour and 
performance at low load with improved performance at (very) high loads so I've 
incorporated them into the current version. No doubt there are other 
improvements that can be made to the skip list implementation but the low 
hanging fruit has been attended to and there seems to be little need given the 
target audience for BFS.

Patch introduction follows:

---

The Brain Fuck Scheduler v0.502 by Con Kolivas.

A single shared runqueue strict fairness earliest deadline first design.
Runqueue insertion is O(log(n)), lookup is O(1), removal is O(k) where k <= 16

Excellent throughput and latency for 1 to many CPUs on desktop and server
commodity hardware.
Not recommended for 4096 cpus.

Scalability is optimal when your workload is equal to the number of CPUs on
bfs. ie you should ONLY do make -j4 on quad core, -j2 on dual core and so on.

Interactive mode is enabled by default but can be disabled for improved
throughput

echo 0 > /proc/sys/kernel/interactive

Features SCHED_IDLEPRIO and SCHED_ISO scheduling policies as well.
You do NOT need to use these policies for good performance, they are purely
optional for even better performance in extreme conditions.

To run something idleprio, use schedtool like so:

schedtool -D -e make -j4

To run something isoprio, use schedtool like so:

schedtool -I -e amarok

Includes configurable SMT-nice support for better nice level and scheduling
policy support across SMT (aka hyperthread) sibling CPUs.

Includes accurate sub-tick accounting of tasks so userspace reported
cpu usage may be very different if you have very short lived tasks.

-- 
-ck

[ANNOUNCE] BFS CPU scheduler v0.502 for linux-4.7 with skip list.

[Con Kolivas]

This is to announce an updated stable version of the Brain Fuck Scheduler, 
version 0.502 for the current stable linux kernel. 

http://ck.kolivas.org/patches/bfs/4.0/4.7/4.7-sched-bfs-502.patch

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

I was reluctant to announce previous versions as the cpu frequency load 
signalling changes mandated by mainline were initially poorly handled and then 
buggy in BFS until recently.

In 2011 I played with skip lists to replace the simple linked list that is 
used in BFS and after a brief period of testing did not find it advantageous 
though very little effort was putting into optimising the skip list 
implementation. Additionally I was unable to maintain the scheduling decisions 
entirely with the existing BFS implementation.
 
Recently I found renewed interest in improving the most obvious flaws in the 
design and trying again and found I could maintain the behaviour and 
performance at low load with improved performance at (very) high loads so I've 
incorporated them into the current version. No doubt there are other 
improvements that can be made to the skip list implementation but the low 
hanging fruit has been attended to and there seems to be little need given the 
target audience for BFS.

Patch introduction follows:

---

The Brain Fuck Scheduler v0.502 by Con Kolivas.

A single shared runqueue strict fairness earliest deadline first design.
Runqueue insertion is O(log(n)), lookup is O(1), removal is O(k) where k <= 16

Excellent throughput and latency for 1 to many CPUs on desktop and server
commodity hardware.
Not recommended for 4096 cpus.

Scalability is optimal when your workload is equal to the number of CPUs on
bfs. ie you should ONLY do make -j4 on quad core, -j2 on dual core and so on.

Interactive mode is enabled by default but can be disabled for improved
throughput

echo 0 > /proc/sys/kernel/interactive

Features SCHED_IDLEPRIO and SCHED_ISO scheduling policies as well.
You do NOT need to use these policies for good performance, they are purely
optional for even better performance in extreme conditions.

To run something idleprio, use schedtool like so:

schedtool -D -e make -j4

To run something isoprio, use schedtool like so:

schedtool -I -e amarok

Includes configurable SMT-nice support for better nice level and scheduling
policy support across SMT (aka hyperthread) sibling CPUs.

Includes accurate sub-tick accounting of tasks so userspace reported
cpu usage may be very different if you have very short lived tasks.

-- 
-ck

[tip:sched/core] sched/core: Do not use smp_processor_id() with preempt enabled in smpboot_thread_fn()

[tip-bot for Con Kolivas]

Commit-ID:  4fa5cd5245b627db88c9ca08ae442373b02596b4
Gitweb: http://git.kernel.org/tip/4fa5cd5245b627db88c9ca08ae442373b02596b4
Author: Con Kolivas <ker...@kolivas.org>
AuthorDate: Tue, 13 Sep 2016 16:27:05 +1000
Committer:  Ingo Molnar <mi...@kernel.org>
CommitDate: Thu, 22 Sep 2016 12:28:00 +0200

sched/core: Do not use smp_processor_id() with preempt enabled in 
smpboot_thread_fn()

We should not be using smp_processor_id() with preempt enabled.

Bug identified and fix provided by Alfred Chen.

Reported-by: Alfred Chen <cchal...@gmail.com>
Signed-off-by: Con Kolivas <ker...@kolivas.org>
Cc: Alfred Chen <cchal...@gmail.com>
Cc: Peter Zijlstra <pet...@infradead.org>
Link: http://lkml.kernel.org/r/2042051.3vvUWIM0vs@hex
Signed-off-by: Ingo Molnar <mi...@kernel.org>
---
 kernel/smpboot.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/smpboot.c b/kernel/smpboot.c
index 13bc43d..fc0d8270 100644
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -122,12 +122,12 @@ static int smpboot_thread_fn(void *data)
 
if (kthread_should_park()) {
__set_current_state(TASK_RUNNING);
-   preempt_enable();
if (ht->park && td->status == HP_THREAD_ACTIVE) {
BUG_ON(td->cpu != smp_processor_id());
ht->park(td->cpu);
td->status = HP_THREAD_PARKED;
}
+   preempt_enable();
kthread_parkme();
/* We might have been woken for stop */
continue;

[tip:sched/core] sched/core: Do not use smp_processor_id() with preempt enabled in smpboot_thread_fn()

[tip-bot for Con Kolivas]

Commit-ID:  4fa5cd5245b627db88c9ca08ae442373b02596b4
Gitweb: http://git.kernel.org/tip/4fa5cd5245b627db88c9ca08ae442373b02596b4
Author: Con Kolivas 
AuthorDate: Tue, 13 Sep 2016 16:27:05 +1000
Committer:  Ingo Molnar 
CommitDate: Thu, 22 Sep 2016 12:28:00 +0200

sched/core: Do not use smp_processor_id() with preempt enabled in 
smpboot_thread_fn()

We should not be using smp_processor_id() with preempt enabled.

Bug identified and fix provided by Alfred Chen.

Reported-by: Alfred Chen 
Signed-off-by: Con Kolivas 
Cc: Alfred Chen 
Cc: Peter Zijlstra 
Link: http://lkml.kernel.org/r/2042051.3vvUWIM0vs@hex
Signed-off-by: Ingo Molnar 
---
 kernel/smpboot.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/smpboot.c b/kernel/smpboot.c
index 13bc43d..fc0d8270 100644
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -122,12 +122,12 @@ static int smpboot_thread_fn(void *data)
 
if (kthread_should_park()) {
__set_current_state(TASK_RUNNING);
-   preempt_enable();
if (ht->park && td->status == HP_THREAD_ACTIVE) {
BUG_ON(td->cpu != smp_processor_id());
ht->park(td->cpu);
td->status = HP_THREAD_PARKED;
}
+   preempt_enable();
kthread_parkme();
/* We might have been woken for stop */
continue;

[PATCH] sched: Do not use smp_processor_id() with preempt enabled in smpboot_thread_fn

[Con Kolivas]

We should not be using smp_processor_id() with preempt enabled.

Bug identified and fix provided by Alfred Chen.

Signed-off-by: Con Kolivas <ker...@kolivas.org>

---
 kernel/smpboot.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux-4.7.3-ck3/kernel/smpboot.c
===
--- linux-4.7.3-ck3.orig/kernel/smpboot.c   2016-05-16 08:43:13.0 
+1000
+++ linux-4.7.3-ck3/kernel/smpboot.c2016-09-13 16:17:33.535655129 +1000
@@ -122,12 +122,12 @@ static int smpboot_thread_fn(void *data)
 
if (kthread_should_park()) {
__set_current_state(TASK_RUNNING);
-   preempt_enable();
if (ht->park && td->status == HP_THREAD_ACTIVE) {
BUG_ON(td->cpu != smp_processor_id());
ht->park(td->cpu);
td->status = HP_THREAD_PARKED;
}
+   preempt_enable();
kthread_parkme();
/* We might have been woken for stop */
continue;

[PATCH] sched: Do not use smp_processor_id() with preempt enabled in smpboot_thread_fn

[Con Kolivas]

We should not be using smp_processor_id() with preempt enabled.

Bug identified and fix provided by Alfred Chen.

Signed-off-by: Con Kolivas 

---
 kernel/smpboot.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux-4.7.3-ck3/kernel/smpboot.c
===
--- linux-4.7.3-ck3.orig/kernel/smpboot.c   2016-05-16 08:43:13.0 
+1000
+++ linux-4.7.3-ck3/kernel/smpboot.c2016-09-13 16:17:33.535655129 +1000
@@ -122,12 +122,12 @@ static int smpboot_thread_fn(void *data)
 
if (kthread_should_park()) {
__set_current_state(TASK_RUNNING);
-   preempt_enable();
if (ht->park && td->status == HP_THREAD_ACTIVE) {
BUG_ON(td->cpu != smp_processor_id());
ht->park(td->cpu);
td->status = HP_THREAD_PARKED;
}
+   preempt_enable();
kthread_parkme();
/* We might have been woken for stop */
continue;

[ANNOUNCE] BFS CPU scheduler v0.469 for linux-4.5

[Con Kolivas]

This is to announce a resync and  update of the Brain Fuck Scheduler, 
version 0.469 for the latest stable linux kernel.

The patch against linux-4.5(.x) is available here:
http://ck.kolivas.org/patches/bfs/4.0/4.5/4.5-sched-bfs-469.patch

A -ck branded linux-4.5-ck1 patch is available here:
http://ck.kolivas.org/patches/4.0/4.5/4.5-ck1/

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

Enjoy!
お楽しみください

-- 
-ck

[ANNOUNCE] BFS CPU scheduler v0.469 for linux-4.5

[Con Kolivas]

This is to announce a resync and  update of the Brain Fuck Scheduler, 
version 0.469 for the latest stable linux kernel.

The patch against linux-4.5(.x) is available here:
http://ck.kolivas.org/patches/bfs/4.0/4.5/4.5-sched-bfs-469.patch

A -ck branded linux-4.5-ck1 patch is available here:
http://ck.kolivas.org/patches/4.0/4.5/4.5-ck1/

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

Enjoy!
お楽しみください

-- 
-ck

Commemorative linux-1.0-3.19 compressed tarball

[Con Kolivas]

As I did prior to the linux-3.0 release, I've created a commemorative tarball 
of all stable point releases from linux 1.0 to linux 3.19 to commemorate the 
upcoming 4.0 release, excluding minor point releases.

http://ck.kolivas.org/linux-1.0-3.19.tar.lrz

This was a 29GB tarball compressed to 355MB.

Of course this is mostly just for fun, and as a shameless plug for lrzip. Be 
aware that to create the tarball I scripted up downloading of patches and 
recreated tarballs so many of the tarballs will not be identical files to the 
official release ones. 


The command used to create the tarball was:

lrzip -U linux-1.0-3.19.tar

linux-1.0-3.19.tar - Compression Ratio: 77.885. Average Compression Speed: 
37.565MB/s.


md5sums:

b9a1e7e6762150028ec36b481df7f264  linux-1.0-3.19.tar
5f5dff66416346fdf9453825f8e4435c  linux-1.0-3.19.tar.lrz


tar -tvf linux-1.0-3.19.tar 

drwxrwxr-x con/con   0 2015-03-07 11:33 v1.0/
-rw-rw-r-- con/con 5171200 1994-03-13 10:00 v1.0/linux-1.0.tar
drwxrwxr-x con/con   0 2015-03-07 11:33 v1.2/
-rw-rw-r-- con/con 9308160 1995-04-29 10:00 v1.2/linux-1.2.7.tar
-rw-rw-r-- con/con 9379840 1995-08-02 10:00 v1.2/linux-1.2.13.tar
-rw-rw-r-- con/con 9195520 1995-03-17 11:00 v1.2/linux-1.2.1.tar
-rw-rw-r-- con/con 9359360 1995-05-03 10:00 v1.2/linux-1.2.8.tar
-rw-rw-r-- con/con 9267200 1995-04-02 10:00 v1.2/linux-1.2.3.tar
-rw-rw-r-- con/con 9369600 1995-06-12 10:00 v1.2/linux-1.2.10.tar
-rw-rw-r-- con/con 9369600 1995-06-01 10:00 v1.2/linux-1.2.9.tar
-rw-rw-r-- con/con 9267200 1995-03-27 10:00 v1.2/linux-1.2.2.tar
-rw-rw-r-- con/con 9369600 1995-06-26 10:00 v1.2/linux-1.2.11.tar
-rw-rw-r-- con/con 9277440 1995-04-12 10:00 v1.2/linux-1.2.5.tar
-rw-rw-r-- con/con 9277440 1995-04-06 10:00 v1.2/linux-1.2.4.tar
-rw-rw-r-- con/con 9379840 1995-07-25 10:00 v1.2/linux-1.2.12.tar
-rw-rw-r-- con/con 9297920 1995-04-23 10:00 v1.2/linux-1.2.6.tar
-rw-rw-r-- con/con 9185280 1995-03-07 11:00 v1.2/linux-1.2.0.tar
drwxrwxr-x con/con   0 2015-03-07 11:02 v2.0/
-rw-rw-r-- con/con26603520 1997-11-18 13:34 v2.0/linux-2.0.32.tar
-rw-rw-r-- con/con24709120 1996-12-02 05:18 v2.0/linux-2.0.27.tar
-rw-rw-r-- con/con23900160 1996-07-05 10:00 v2.0/linux-2.0.2.tar
-rw-rw-r-- con/con30228480 1998-11-16 16:50 v2.0/linux-2.0.36.tar
-rw-rw-r-- con/con28231680 1998-06-04 15:15 v2.0/linux-2.0.34.tar
-rw-rw-r-- con/con24688640 1996-10-30 14:14 v2.0/linux-2.0.24.tar
-rw-rw-r-- con/con24401920 1996-09-01 04:03 v2.0/linux-2.0.16.tar
-rw-rw-r-- con/con24166400 1996-07-12 10:00 v2.0/linux-2.0.6.tar
-rw-rw-r-- con/con24483840 1996-10-18 22:20 v2.0/linux-2.0.23.tar
-rw-rw-r-- con/con26562560 1997-10-18 08:25 v2.0/linux-2.0.31.tar
-rw-rw-r-- con/con24494080 1997-02-08 01:56 v2.0/linux-2.0.29.tar
-rw-rw-r-- con/con24176640 1996-07-15 10:00 v2.0/linux-2.0.7.tar
-rw-rw-r-- con/con24432640 1996-09-12 00:21 v2.0/linux-2.0.19.tar
-rw-rw-r-- con/con31324160 1999-08-26 08:11 v2.0/linux-2.0.38.tar
-rw-rw-r-- con/con24238080 1996-08-05 10:00 v2.0/linux-2.0.11.tar
-rw-rw-r-- con/con24238080 1996-08-09 10:00 v2.0/linux-2.0.12.tar
-rw-rw-r-- con/con24412160 1996-09-02 20:37 v2.0/linux-2.0.17.tar
-rw-rw-r-- con/con23879680 1996-06-09 10:00 v2.0/linux-2.0.tar
-rw-rw-r-- con/con31324160 1999-06-14 15:15 v2.0/linux-2.0.37.tar
-rw-rw-r-- con/con24422400 1996-09-06 00:38 v2.0/linux-2.0.18.tar
-rw-rw-r-- con/con24197120 1996-07-26 10:00 v2.0/linux-2.0.9.tar
-rw-rw-r-- con/con24186880 1996-07-25 10:00 v2.0/linux-2.0.8.tar
-rw-rw-r-- con/con29122560 1998-07-14 07:09 v2.0/linux-2.0.35.tar
-rw-rw-r-- con/con24166400 1996-07-06 10:00 v2.0/linux-2.0.3.tar
-rw-rw-r-- con/con24401920 1996-08-25 20:20 v2.0/linux-2.0.15.tar
-rw-rw-r-- con/con24729600 1996-11-23 00:17 v2.0/linux-2.0.26.tar
-rw-rw-r-- con/con31426560 2001-01-10 08:30 v2.0/linux-2.0.39.tar
-rw-rw-r-- con/con24494080 1997-01-14 23:33 v2.0/linux-2.0.28.tar
-rw-rw-r-- con/con24432640 1996-09-13 22:53 v2.0/linux-2.0.20.tar
-rw-rw-r-- con/con24473600 1996-10-09 03:02 v2.0/linux-2.0.22.tar
-rw-rw-r-- con/con24698880 1996-11-08 20:31 v2.0/linux-2.0.25.tar
-rw-rw-r-- con/con25108480 1997-04-09 02:34 v2.0/linux-2.0.30.tar
-rw-rw-r-- con/con24442880 1996-09-20 23:51 v2.0/linux-2.0.21.tar
-rw-rw-r-- con/con24145920 1996-07-08 10:00 v2.0/linux-2.0.4.tar
-rw-rw-r-- con/con23900160 1996-07-03 10:00 v2.0/linux-2.0.1.tar
-rw-rw-r-- con/con26624000 1997-12-17 09:55 v2.0/linux-2.0.33.tar
-rw-rw-r-- con/con24391680 1996-08-21 01:52 v2.0/linux-2.0.14.tar
-rw-rw-r-- con/con24371200 1996-08-16 20:19 v2.0/linux-2.0.13.tar
-rw-rw-r-- con/con31467520 2004-02-08 18:13 v2.0/linux-2.0.40.tar
-rw-rw-r-- con/con24156160 1996-07-10 10:00 v2.0/linux-2.0.5.tar
-rw-rw-r-- con/con24197120 1996-07-27 10:00 v2.0/linux-2.0.10.tar
drwxrwxr-x con/con   0 2015-03-07 11:06 v2.2/

Commemorative linux-1.0-3.19 compressed tarball

[Con Kolivas]

As I did prior to the linux-3.0 release, I've created a commemorative tarball 
of all stable point releases from linux 1.0 to linux 3.19 to commemorate the 
upcoming 4.0 release, excluding minor point releases.

http://ck.kolivas.org/linux-1.0-3.19.tar.lrz

This was a 29GB tarball compressed to 355MB.

Of course this is mostly just for fun, and as a shameless plug for lrzip. Be 
aware that to create the tarball I scripted up downloading of patches and 
recreated tarballs so many of the tarballs will not be identical files to the 
official release ones. 


The command used to create the tarball was:

lrzip -U linux-1.0-3.19.tar

linux-1.0-3.19.tar - Compression Ratio: 77.885. Average Compression Speed: 
37.565MB/s.


md5sums:

b9a1e7e6762150028ec36b481df7f264  linux-1.0-3.19.tar
5f5dff66416346fdf9453825f8e4435c  linux-1.0-3.19.tar.lrz


tar -tvf linux-1.0-3.19.tar 

drwxrwxr-x con/con   0 2015-03-07 11:33 v1.0/
-rw-rw-r-- con/con 5171200 1994-03-13 10:00 v1.0/linux-1.0.tar
drwxrwxr-x con/con   0 2015-03-07 11:33 v1.2/
-rw-rw-r-- con/con 9308160 1995-04-29 10:00 v1.2/linux-1.2.7.tar
-rw-rw-r-- con/con 9379840 1995-08-02 10:00 v1.2/linux-1.2.13.tar
-rw-rw-r-- con/con 9195520 1995-03-17 11:00 v1.2/linux-1.2.1.tar
-rw-rw-r-- con/con 9359360 1995-05-03 10:00 v1.2/linux-1.2.8.tar
-rw-rw-r-- con/con 9267200 1995-04-02 10:00 v1.2/linux-1.2.3.tar
-rw-rw-r-- con/con 9369600 1995-06-12 10:00 v1.2/linux-1.2.10.tar
-rw-rw-r-- con/con 9369600 1995-06-01 10:00 v1.2/linux-1.2.9.tar
-rw-rw-r-- con/con 9267200 1995-03-27 10:00 v1.2/linux-1.2.2.tar
-rw-rw-r-- con/con 9369600 1995-06-26 10:00 v1.2/linux-1.2.11.tar
-rw-rw-r-- con/con 9277440 1995-04-12 10:00 v1.2/linux-1.2.5.tar
-rw-rw-r-- con/con 9277440 1995-04-06 10:00 v1.2/linux-1.2.4.tar
-rw-rw-r-- con/con 9379840 1995-07-25 10:00 v1.2/linux-1.2.12.tar
-rw-rw-r-- con/con 9297920 1995-04-23 10:00 v1.2/linux-1.2.6.tar
-rw-rw-r-- con/con 9185280 1995-03-07 11:00 v1.2/linux-1.2.0.tar
drwxrwxr-x con/con   0 2015-03-07 11:02 v2.0/
-rw-rw-r-- con/con26603520 1997-11-18 13:34 v2.0/linux-2.0.32.tar
-rw-rw-r-- con/con24709120 1996-12-02 05:18 v2.0/linux-2.0.27.tar
-rw-rw-r-- con/con23900160 1996-07-05 10:00 v2.0/linux-2.0.2.tar
-rw-rw-r-- con/con30228480 1998-11-16 16:50 v2.0/linux-2.0.36.tar
-rw-rw-r-- con/con28231680 1998-06-04 15:15 v2.0/linux-2.0.34.tar
-rw-rw-r-- con/con24688640 1996-10-30 14:14 v2.0/linux-2.0.24.tar
-rw-rw-r-- con/con24401920 1996-09-01 04:03 v2.0/linux-2.0.16.tar
-rw-rw-r-- con/con24166400 1996-07-12 10:00 v2.0/linux-2.0.6.tar
-rw-rw-r-- con/con24483840 1996-10-18 22:20 v2.0/linux-2.0.23.tar
-rw-rw-r-- con/con26562560 1997-10-18 08:25 v2.0/linux-2.0.31.tar
-rw-rw-r-- con/con24494080 1997-02-08 01:56 v2.0/linux-2.0.29.tar
-rw-rw-r-- con/con24176640 1996-07-15 10:00 v2.0/linux-2.0.7.tar
-rw-rw-r-- con/con24432640 1996-09-12 00:21 v2.0/linux-2.0.19.tar
-rw-rw-r-- con/con31324160 1999-08-26 08:11 v2.0/linux-2.0.38.tar
-rw-rw-r-- con/con24238080 1996-08-05 10:00 v2.0/linux-2.0.11.tar
-rw-rw-r-- con/con24238080 1996-08-09 10:00 v2.0/linux-2.0.12.tar
-rw-rw-r-- con/con24412160 1996-09-02 20:37 v2.0/linux-2.0.17.tar
-rw-rw-r-- con/con23879680 1996-06-09 10:00 v2.0/linux-2.0.tar
-rw-rw-r-- con/con31324160 1999-06-14 15:15 v2.0/linux-2.0.37.tar
-rw-rw-r-- con/con24422400 1996-09-06 00:38 v2.0/linux-2.0.18.tar
-rw-rw-r-- con/con24197120 1996-07-26 10:00 v2.0/linux-2.0.9.tar
-rw-rw-r-- con/con24186880 1996-07-25 10:00 v2.0/linux-2.0.8.tar
-rw-rw-r-- con/con29122560 1998-07-14 07:09 v2.0/linux-2.0.35.tar
-rw-rw-r-- con/con24166400 1996-07-06 10:00 v2.0/linux-2.0.3.tar
-rw-rw-r-- con/con24401920 1996-08-25 20:20 v2.0/linux-2.0.15.tar
-rw-rw-r-- con/con24729600 1996-11-23 00:17 v2.0/linux-2.0.26.tar
-rw-rw-r-- con/con31426560 2001-01-10 08:30 v2.0/linux-2.0.39.tar
-rw-rw-r-- con/con24494080 1997-01-14 23:33 v2.0/linux-2.0.28.tar
-rw-rw-r-- con/con24432640 1996-09-13 22:53 v2.0/linux-2.0.20.tar
-rw-rw-r-- con/con24473600 1996-10-09 03:02 v2.0/linux-2.0.22.tar
-rw-rw-r-- con/con24698880 1996-11-08 20:31 v2.0/linux-2.0.25.tar
-rw-rw-r-- con/con25108480 1997-04-09 02:34 v2.0/linux-2.0.30.tar
-rw-rw-r-- con/con24442880 1996-09-20 23:51 v2.0/linux-2.0.21.tar
-rw-rw-r-- con/con24145920 1996-07-08 10:00 v2.0/linux-2.0.4.tar
-rw-rw-r-- con/con23900160 1996-07-03 10:00 v2.0/linux-2.0.1.tar
-rw-rw-r-- con/con26624000 1997-12-17 09:55 v2.0/linux-2.0.33.tar
-rw-rw-r-- con/con24391680 1996-08-21 01:52 v2.0/linux-2.0.14.tar
-rw-rw-r-- con/con24371200 1996-08-16 20:19 v2.0/linux-2.0.13.tar
-rw-rw-r-- con/con31467520 2004-02-08 18:13 v2.0/linux-2.0.40.tar
-rw-rw-r-- con/con24156160 1996-07-10 10:00 v2.0/linux-2.0.5.tar
-rw-rw-r-- con/con24197120 1996-07-27 10:00 v2.0/linux-2.0.10.tar
drwxrwxr-x con/con   0 2015-03-07 11:06 v2.2/

[ANNOUNCE] BFS CPU scheduler v0.460 for linux-3.18

[Con Kolivas]

This is to announce a resync and  update of the Brain Fuck Scheduler, 
version 0.460 for the latest stable linux kernel.

The patch against linux-3.18(.x) is available here:
http://ck.kolivas.org/patches/bfs/3.0/3.18/3.18-sched-bfs-460.patch

A -ck branded linux-3.18-ck1 patch is available here:
http://ck.kolivas.org/patches/3.0/3.18/3.18-ck1/

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

Enjoy!
お楽しみください

-- 
-ck

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] BFS CPU scheduler v0.460 for linux-3.18

[Con Kolivas]

This is to announce a resync and  update of the Brain Fuck Scheduler, 
version 0.460 for the latest stable linux kernel.

The patch against linux-3.18(.x) is available here:
http://ck.kolivas.org/patches/bfs/3.0/3.18/3.18-sched-bfs-460.patch

A -ck branded linux-3.18-ck1 patch is available here:
http://ck.kolivas.org/patches/3.0/3.18/3.18-ck1/

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

Enjoy!
お楽しみください

-- 
-ck

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ANNOUNCE] BFS CPU scheduler v0.450 with SMT nice support for linux-3.16

[Con Kolivas]

On 3 December 2014 at 19:12, Hillf Danton  wrote:
> Hey Con

Hi Hilf

>> This is to announce a resync and  update of the Brain Fuck Scheduler,
>> version 0.450 for the latest stable linux kernel.
>>
> [...]
>> The patch against linux-3.16(.x) is available here:
>> http://ck.kolivas.org/patches/bfs/3.0/3.16/3.16-sched-bfs-450.patch
>>
> $ grep -nr above_background_load  bfs/src
> bfs/src/3.16-sched-bfs-450.patch:689:+bool above_background_load(void);
> bfs/src/3.16-sched-bfs-450.patch:736:+static inline bool 
> above_background_load(void)
> bfs/src/3.16-sched-bfs-450.patch:1665:+bool above_background_load(void)
>
> Would you please pin-point what is that routine used for?

This function used to be used with the VM tweaks that the -ck patchset
used to come with. I have since deprecated those VM tweaks but the
function remains in the scheduler. You're right to point it out and
I'll remove it when I resync with 3.18 (it's still in the BFS 458
patch for 3.17).

Thanks,
Con

> Thanks
> Hillf
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ANNOUNCE] BFS CPU scheduler v0.450 with SMT nice support for linux-3.16

[Con Kolivas]

On 3 December 2014 at 19:12, Hillf Danton hillf...@alibaba-inc.com wrote:
 Hey Con

Hi Hilf

 This is to announce a resync and  update of the Brain Fuck Scheduler,
 version 0.450 for the latest stable linux kernel.

 [...]
 The patch against linux-3.16(.x) is available here:
 http://ck.kolivas.org/patches/bfs/3.0/3.16/3.16-sched-bfs-450.patch

 $ grep -nr above_background_load  bfs/src
 bfs/src/3.16-sched-bfs-450.patch:689:+bool above_background_load(void);
 bfs/src/3.16-sched-bfs-450.patch:736:+static inline bool 
 above_background_load(void)
 bfs/src/3.16-sched-bfs-450.patch:1665:+bool above_background_load(void)

 Would you please pin-point what is that routine used for?

This function used to be used with the VM tweaks that the -ck patchset
used to come with. I have since deprecated those VM tweaks but the
function remains in the scheduler. You're right to point it out and
I'll remove it when I resync with 3.18 (it's still in the BFS 458
patch for 3.17).

Thanks,
Con

 Thanks
 Hillf

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: i8k: Don't revert affinity in i8k_smm

[Con Kolivas]

On Mon, 18 Aug 2014 07:32:15 PM Guenter Roeck wrote:
> On Tue, Aug 19, 2014 at 09:19:55AM +1000, Con Kolivas wrote:
> > As a followup to this discussion:
> > 
> > On Tue, 15 Jul 2014 08:01:13 PM Sam Asadi wrote:
> > > Commit f36fdb9f0266 (i8k: Force SMM to run on CPU 0) adds support
> > > for multi-core CPUs to the driver. Unfortunately, that causes it
> > > to fail loading if compiled without SMP support, at least on
> > > 32 bit kernels. Kernel log shows "i8k: unable to get SMM Dell
> > > signature", and function i8k_smm is found to return -EINVAL.
> > > 
> > > Testing revealed that the culprit is the missing return value check
> > > of set_cpus_allowed_ptr.
> > 
> > It appears that the original commit f36fdb9f0266 changes the affinity for
> > the duration of i8k_smm function and then unconditionally reverts the
> > affinity to the old cpu mask regardless of whether the function succeeds
> > or fails. As this must run on CPU 0 at all times it does not make sense
> > to revert the affinity at the end of the function. Proposed patch
> > attached.
> 
> Sorry, I must have missed the rest of the discussion. What problem is this
> patch supposed to fix ? Or, in other words, is there a problem with the
> current code ? I also don't really understand the argument above. Why does
> it not make sense to revert to the original affinity ? After all, only the
> SMM call must run on CPU 0. Why does it not make sense to let the rest of
> the code run on another CPU ?

My mistake. If only the i8k_smm function needs to run on CPU 0 then it is 
appropriate to return affinity to the previous CPU mask. Please disregard and 
apologies for the noise.

Thanks,
Con
-- 
-ck

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: i8k: Don't revert affinity in i8k_smm

[Con Kolivas]

On Mon, 18 Aug 2014 07:32:15 PM Guenter Roeck wrote:
 On Tue, Aug 19, 2014 at 09:19:55AM +1000, Con Kolivas wrote:
  As a followup to this discussion:
  
  On Tue, 15 Jul 2014 08:01:13 PM Sam Asadi wrote:
   Commit f36fdb9f0266 (i8k: Force SMM to run on CPU 0) adds support
   for multi-core CPUs to the driver. Unfortunately, that causes it
   to fail loading if compiled without SMP support, at least on
   32 bit kernels. Kernel log shows i8k: unable to get SMM Dell
   signature, and function i8k_smm is found to return -EINVAL.
   
   Testing revealed that the culprit is the missing return value check
   of set_cpus_allowed_ptr.
  
  It appears that the original commit f36fdb9f0266 changes the affinity for
  the duration of i8k_smm function and then unconditionally reverts the
  affinity to the old cpu mask regardless of whether the function succeeds
  or fails. As this must run on CPU 0 at all times it does not make sense
  to revert the affinity at the end of the function. Proposed patch
  attached.
 
 Sorry, I must have missed the rest of the discussion. What problem is this
 patch supposed to fix ? Or, in other words, is there a problem with the
 current code ? I also don't really understand the argument above. Why does
 it not make sense to revert to the original affinity ? After all, only the
 SMM call must run on CPU 0. Why does it not make sense to let the rest of
 the code run on another CPU ?

My mistake. If only the i8k_smm function needs to run on CPU 0 then it is 
appropriate to return affinity to the previous CPU mask. Please disregard and 
apologies for the noise.

Thanks,
Con
-- 
-ck

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

i8k: Don't revert affinity in i8k_smm

[Con Kolivas]

As a followup to this discussion:

On Tue, 15 Jul 2014 08:01:13 PM Sam Asadi wrote:
> Commit f36fdb9f0266 (i8k: Force SMM to run on CPU 0) adds support
> for multi-core CPUs to the driver. Unfortunately, that causes it
> to fail loading if compiled without SMP support, at least on
> 32 bit kernels. Kernel log shows "i8k: unable to get SMM Dell
> signature", and function i8k_smm is found to return -EINVAL.
> 
> Testing revealed that the culprit is the missing return value check
> of set_cpus_allowed_ptr.

It appears that the original commit f36fdb9f0266 changes the affinity for the 
duration of i8k_smm function and then unconditionally reverts the affinity to 
the old cpu mask regardless of whether the function succeeds or fails. As this 
must run on CPU 0 at all times it does not make sense to revert the affinity at 
the end of the function. Proposed patch attached.

Signed-off-by: Con Kolivas 

---
 drivers/char/i8k.c |6 --
 1 file changed, 6 deletions(-)

Index: linux-3.16-ck1/drivers/char/i8k.c
===
--- linux-3.16-ck1.orig/drivers/char/i8k.c  2014-08-12 14:07:49.0 
+1000
+++ linux-3.16-ck1/drivers/char/i8k.c   2014-08-19 09:09:57.939056696 +1000
@@ -132,12 +132,8 @@ static int i8k_smm(struct smm_regs *regs
 {
int rc;
int eax = regs->eax;
-   cpumask_var_t old_mask;
 
/* SMM requires CPU 0 */
-   if (!alloc_cpumask_var(_mask, GFP_KERNEL))
-   return -ENOMEM;
-   cpumask_copy(old_mask, >cpus_allowed);
rc = set_cpus_allowed_ptr(current, cpumask_of(0));
if (rc)
goto out;
@@ -203,8 +199,6 @@ static int i8k_smm(struct smm_regs *regs
rc = -EINVAL;
 
 out:
-   set_cpus_allowed_ptr(current, old_mask);
-   free_cpumask_var(old_mask);
return rc;
 }
 

-- 
-ck
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

i8k: Don't revert affinity in i8k_smm

[Con Kolivas]

As a followup to this discussion:

On Tue, 15 Jul 2014 08:01:13 PM Sam Asadi wrote:
 Commit f36fdb9f0266 (i8k: Force SMM to run on CPU 0) adds support
 for multi-core CPUs to the driver. Unfortunately, that causes it
 to fail loading if compiled without SMP support, at least on
 32 bit kernels. Kernel log shows i8k: unable to get SMM Dell
 signature, and function i8k_smm is found to return -EINVAL.
 
 Testing revealed that the culprit is the missing return value check
 of set_cpus_allowed_ptr.

It appears that the original commit f36fdb9f0266 changes the affinity for the 
duration of i8k_smm function and then unconditionally reverts the affinity to 
the old cpu mask regardless of whether the function succeeds or fails. As this 
must run on CPU 0 at all times it does not make sense to revert the affinity at 
the end of the function. Proposed patch attached.

Signed-off-by: Con Kolivas ker...@kolivas.org

---
 drivers/char/i8k.c |6 --
 1 file changed, 6 deletions(-)

Index: linux-3.16-ck1/drivers/char/i8k.c
===
--- linux-3.16-ck1.orig/drivers/char/i8k.c  2014-08-12 14:07:49.0 
+1000
+++ linux-3.16-ck1/drivers/char/i8k.c   2014-08-19 09:09:57.939056696 +1000
@@ -132,12 +132,8 @@ static int i8k_smm(struct smm_regs *regs
 {
int rc;
int eax = regs-eax;
-   cpumask_var_t old_mask;
 
/* SMM requires CPU 0 */
-   if (!alloc_cpumask_var(old_mask, GFP_KERNEL))
-   return -ENOMEM;
-   cpumask_copy(old_mask, current-cpus_allowed);
rc = set_cpus_allowed_ptr(current, cpumask_of(0));
if (rc)
goto out;
@@ -203,8 +199,6 @@ static int i8k_smm(struct smm_regs *regs
rc = -EINVAL;
 
 out:
-   set_cpus_allowed_ptr(current, old_mask);
-   free_cpumask_var(old_mask);
return rc;
 }
 

-- 
-ck
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] BFS CPU scheduler v0.450 with SMT nice support for linux-3.16

[Con Kolivas]

This is to announce a resync and  update of the Brain Fuck Scheduler, 
version 0.450 for the latest stable linux kernel.

In addition to the usual round of fixes and changes to resync with mainline 
changes, this release brings a new feature of configurable SMT nice support.

The configuration help follows to explain:

Enabling Hyperthreading on Intel CPUs decreases the effectiveness   

of the use of 'nice' levels and different scheduling policies   

(e.g. realtime) due to sharing of CPU power between hyperthreads.   

SMT nice support makes each logical CPU aware of what is running on 

its hyperthread siblings, maintaining appropriate distribution of   

CPU according to nice levels and scheduling policies at the expense 

of slightly increased overhead.  

This code is based on my original idea from ten years ago discussed here:
http://lwn.net/Articles/73614/

A full discussion of the rationale and development follows here:
http://ck-hack.blogspot.com/2014/08/smthyperthreading-nice-and-scheduling.html

The patch against linux-3.16(.x) is available here:
http://ck.kolivas.org/patches/bfs/3.0/3.16/3.16-sched-bfs-450.patch

A -ck branded linux-3.16-ck1 patch is available here:
http://ck.kolivas.org/patches/3.0/3.16/3.16-ck1/

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

Enjoy!
お楽しみください

-- 
-ck

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] BFS CPU scheduler v0.450 with SMT nice support for linux-3.16

[Con Kolivas]

This is to announce a resync and  update of the Brain Fuck Scheduler, 
version 0.450 for the latest stable linux kernel.

In addition to the usual round of fixes and changes to resync with mainline 
changes, this release brings a new feature of configurable SMT nice support.

The configuration help follows to explain:

Enabling Hyperthreading on Intel CPUs decreases the effectiveness   

of the use of 'nice' levels and different scheduling policies   

(e.g. realtime) due to sharing of CPU power between hyperthreads.   

SMT nice support makes each logical CPU aware of what is running on 

its hyperthread siblings, maintaining appropriate distribution of   

CPU according to nice levels and scheduling policies at the expense 

of slightly increased overhead.  

This code is based on my original idea from ten years ago discussed here:
http://lwn.net/Articles/73614/

A full discussion of the rationale and development follows here:
http://ck-hack.blogspot.com/2014/08/smthyperthreading-nice-and-scheduling.html

The patch against linux-3.16(.x) is available here:
http://ck.kolivas.org/patches/bfs/3.0/3.16/3.16-sched-bfs-450.patch

A -ck branded linux-3.16-ck1 patch is available here:
http://ck.kolivas.org/patches/3.0/3.16/3.16-ck1/

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

Enjoy!
お楽しみください

-- 
-ck

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ANNOUNCE] BFS CPU scheduler v0.444 for linux-3.12

[Con Kolivas]

On 10 December 2013 09:30, David Rientjes  wrote:
> Any reason that BFS hardcodes CONFIG_SLUB as the only slab allocator
> allowed?  I've cc'd Pekka and Christoph and I'm sure they'd be interested
> in any reasons that CONFIG_SLAB doesn't work correctly with a different
> drop-in scheduler, or is it just that interactivity has tested better with
> slub?

Hi David.

Thanks, and an interesting question you have there.

To be honest it's been probably over 2 years since I hard coded SLUB
into the BFS patch and all I can recall is that it caused a crash that
was purely due to enabling SLAB that went away with SLUB when used
with BFS. Despite the possibility that BFS exposes an issue in the
kernel that may be possible with the mainline scheduler (due to BFS
being extremely good at exposing race conditions), if the problem is
never reported with mainline, it's probably of no significance to
mainline. Changes in scheduler initialisation sequence alone may be
the real reason. The same goes with the offline CPU code which had to
be drastically reworked to work properly with suspend/hibernate on
BFS.

There certainly have been cases that BFS has exposed races that have
only become apparent in mainline a long time later. Here's one I
recall reporting as a potential issue a long time ago here:
http://marc.info/?l=linux-kernel=130613435113919=2
It was instantly discounted as not a problem, yet about 6 months later
a real issue in this code showed up.

I have no idea if the CONFIG_SLAB problem falls into this category.

Regards,
Con
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ANNOUNCE] BFS CPU scheduler v0.444 for linux-3.12

[Con Kolivas]

On 10 December 2013 09:30, David Rientjes rient...@google.com wrote:
 Any reason that BFS hardcodes CONFIG_SLUB as the only slab allocator
 allowed?  I've cc'd Pekka and Christoph and I'm sure they'd be interested
 in any reasons that CONFIG_SLAB doesn't work correctly with a different
 drop-in scheduler, or is it just that interactivity has tested better with
 slub?

Hi David.

Thanks, and an interesting question you have there.

To be honest it's been probably over 2 years since I hard coded SLUB
into the BFS patch and all I can recall is that it caused a crash that
was purely due to enabling SLAB that went away with SLUB when used
with BFS. Despite the possibility that BFS exposes an issue in the
kernel that may be possible with the mainline scheduler (due to BFS
being extremely good at exposing race conditions), if the problem is
never reported with mainline, it's probably of no significance to
mainline. Changes in scheduler initialisation sequence alone may be
the real reason. The same goes with the offline CPU code which had to
be drastically reworked to work properly with suspend/hibernate on
BFS.

There certainly have been cases that BFS has exposed races that have
only become apparent in mainline a long time later. Here's one I
recall reporting as a potential issue a long time ago here:
http://marc.info/?l=linux-kernelm=130613435113919w=2
It was instantly discounted as not a problem, yet about 6 months later
a real issue in this code showed up.

I have no idea if the CONFIG_SLAB problem falls into this category.

Regards,
Con
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] BFS CPU scheduler v0.444 for linux-3.12

[Con Kolivas]

This is to announce a resync and minor update of the Brain Fuck Scheduler, 
version 0.444 for the latest stable linux kernel. The main changes include a 
resync against linux kernel version 3.12 and a rewritten mechanism for coping 
with suspend to ram/disk and resume issues present in previous versions.

The patch against linux-3.12(.x) is available here:
http://ck.kolivas.org/patches/bfs/3.0/3.12/3.12-sched-bfs-444.patch

A -ck branded linux-3.12-ck2 patch is available here:
http://ck.kolivas.org/patches/3.0/3.12/3.12-ck2/

Note that the other changes in the -ck patch are trivial only and the -ck 
patch is just the latest BFS patch with the cosmetic change of the -ck suffix.

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

Enjoy!
お楽しみください

-- 
-ck

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] BFS CPU scheduler v0.444 for linux-3.12

[Con Kolivas]

This is to announce a resync and minor update of the Brain Fuck Scheduler, 
version 0.444 for the latest stable linux kernel. The main changes include a 
resync against linux kernel version 3.12 and a rewritten mechanism for coping 
with suspend to ram/disk and resume issues present in previous versions.

The patch against linux-3.12(.x) is available here:
http://ck.kolivas.org/patches/bfs/3.0/3.12/3.12-sched-bfs-444.patch

A -ck branded linux-3.12-ck2 patch is available here:
http://ck.kolivas.org/patches/3.0/3.12/3.12-ck2/

Note that the other changes in the -ck patch are trivial only and the -ck 
patch is just the latest BFS patch with the cosmetic change of the -ck suffix.

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

Enjoy!
お楽しみください

-- 
-ck

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] BFS CPU scheduler v0.442 for linux-3.11

[Con Kolivas]

This is to announce a resync and minor update of the Brain Fuck Scheduler, 
version 0.442 for the latest stable linux kernel.

The patch against linux-3.11 is available here:
http://ck.kolivas.org/patches/bfs/3.0/3.11/3.11-sched-bfs-442.patch

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

Enjoy!
お楽しみください

-- 
-ck

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] BFS CPU scheduler v0.442 for linux-3.11

[Con Kolivas]

This is to announce a resync and minor update of the Brain Fuck Scheduler, 
version 0.442 for the latest stable linux kernel.

The patch against linux-3.11 is available here:
http://ck.kolivas.org/patches/bfs/3.0/3.11/3.11-sched-bfs-442.patch

All patches available here:
http://ck.kolivas.org/patches

Code blog:
http://ck-hack.blogspot.com

Enjoy!
お楽しみください

-- 
-ck

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ANNOUNCE] Multiple run-queues for BFS

[Con Kolivas]

Hi Matthias, et al.

On Sat, 15 Dec 2012 18:16:43 Matthias Kohler wrote:
> I'm doing a CPU-Scheduler based on BFS by Con Kolivas with support for
> multiple run-queues. 

Nice to see you doing interesting hacking on BFS and thanks for your bugfixes 
previously. Well done making it this far! Multiple runqueues is something 
that's been on and off the boil for BFS right from the start. I've also done 
various experiments and some ground work along similar lines (I'll explain 
later), but soon lost interest since my experiments suggested I would need a 
heck of a lot of time to implement it the way I envisioned.

> BFS in itself uses only one run-queue for all
> CPU's. This avoids the load-balancing overhead, but does not scale well.

Actually describing it like this is a little bit of a generalisation that 
unfortunately has the tendency to point to the obvious culprit for scaling 
without really defining what the limitations are to scaling and how removing it 
might help. The culprit people point to is the single runqueue because of the 
potential for lock contention rising exponentially as the number of CPUs 
rises. However while some very early testing showed that scalability of -some- 
workloads did not match mainline once we reached 16 CPUs, it was just an 
assumption that it was lock contention that caused this. The most CPUs I can 
find someone to test BFS on previously was 16, and lock debugging actually did 
NOT show there to be significant contention as registering any more than 
mainline. This is not that surprising really since the critical sections under 
the global runqueue lock are kept as short as possible.

Scalability seems only affected on workloads that rely greatly on cache 
warmth, and very low cache requirement workloads like simple networking based 
tasks do not suffer at all (and some benchmarks have shown it to be better 
than mainline at these). This implies to me that complex balancing mechanisms 
designed to keep tasks cache warm is the reason mainline outperforms with 
these workloads. The main thrust of BFS is to offer -any- free CPU to the task 
that has waited the longest, thereby minimising latency at all times, and 
bringing it down the more CPUs you have. This approach is pretty much 
orthogonal to trying to keep tasks bound to their old CPU or a cache shared 
sibling if it means occasionally leaving a cold cache CPU free for a period 
instead. BFS uses only the simplest possible cache warmth approach which 
proves to help, but overrides it in the interests of latency. Certainly in the 
- now overworked example - of the make kernel benchmarks, it performs very 
well up to 16x.

Which brings up the question of just how many CPUs it requires for lock 
contention to be a problem. I don't have the answer to that question, but I 
would not be surprised if it was a significant problem at 64 or more. Luckily, 
while the general trend is for more and more threads even on commodity 
hardware, I don't see this size happening any time soon (note I'm not saying 
never). That said, it is a valid endpoint to scale beyond that if possible in 
the design if one wished BFS to be used outside the commodity hardware 
spectrum.

> One run-queue per CPU does scale well, but then the scheduler has
> load-balancing overhead. The scheduler I'm developing supports every
> possible run-queues configuration. You can have one single run-queue
> like in BFS, or you can have one run-queue per CPU, or something
> completely different like one run-queue every two CPU's. This, in theory
> would  allow the scheduler to be fine-tuned to the hardware and the
> workload.

I agree with you this is by far the most interesting way of tackling this 
problem, while trying to maintain the benefits of the shared runqueue as much 
as possible. Given that lock contention is not a problem even up to 16 
threads/CPUs, it gives scope to have clusters of CPUs benefiting from the 
shared runqueue aspects BFS has while still allowing it to scale beyond that. 
I would hazard a guess that optimal would simply be one runqueue per shared 
cache.

> 
> What state is it in?
> Currently it is very unstable, CPU-Hotplug is broken, scheduling
> statistics are broken, support for real-time tasks is broken. Load
> balancing when having more than one run-queue is working, but is nothing
> more than keeping the load on all run-queues equal. Associating a CPU
> and a run-queue is currently done with a system call and there is no
> access right checking. The source is in a very bad state.
> Uni-processor build is broken.
> It lacks proper Documentation.
> 
> Why allow the user to change the run-queue layout?
> To optimize the scheduler to specific hardware and workloads.
> You could use one run-queue for all CPU's if you want low latency and
> low scheduling overhead.
> You could use one run-queue per CPU if you want high scalability.
> You could use on

Re: [ANNOUNCE] Multiple run-queues for BFS

[Con Kolivas]

Hi Matthias, et al.

On Sat, 15 Dec 2012 18:16:43 Matthias Kohler wrote:
 I'm doing a CPU-Scheduler based on BFS by Con Kolivas with support for
 multiple run-queues. 

Nice to see you doing interesting hacking on BFS and thanks for your bugfixes 
previously. Well done making it this far! Multiple runqueues is something 
that's been on and off the boil for BFS right from the start. I've also done 
various experiments and some ground work along similar lines (I'll explain 
later), but soon lost interest since my experiments suggested I would need a 
heck of a lot of time to implement it the way I envisioned.

 BFS in itself uses only one run-queue for all
 CPU's. This avoids the load-balancing overhead, but does not scale well.

Actually describing it like this is a little bit of a generalisation that 
unfortunately has the tendency to point to the obvious culprit for scaling 
without really defining what the limitations are to scaling and how removing it 
might help. The culprit people point to is the single runqueue because of the 
potential for lock contention rising exponentially as the number of CPUs 
rises. However while some very early testing showed that scalability of -some- 
workloads did not match mainline once we reached 16 CPUs, it was just an 
assumption that it was lock contention that caused this. The most CPUs I can 
find someone to test BFS on previously was 16, and lock debugging actually did 
NOT show there to be significant contention as registering any more than 
mainline. This is not that surprising really since the critical sections under 
the global runqueue lock are kept as short as possible.

Scalability seems only affected on workloads that rely greatly on cache 
warmth, and very low cache requirement workloads like simple networking based 
tasks do not suffer at all (and some benchmarks have shown it to be better 
than mainline at these). This implies to me that complex balancing mechanisms 
designed to keep tasks cache warm is the reason mainline outperforms with 
these workloads. The main thrust of BFS is to offer -any- free CPU to the task 
that has waited the longest, thereby minimising latency at all times, and 
bringing it down the more CPUs you have. This approach is pretty much 
orthogonal to trying to keep tasks bound to their old CPU or a cache shared 
sibling if it means occasionally leaving a cold cache CPU free for a period 
instead. BFS uses only the simplest possible cache warmth approach which 
proves to help, but overrides it in the interests of latency. Certainly in the 
- now overworked example - of the make kernel benchmarks, it performs very 
well up to 16x.

Which brings up the question of just how many CPUs it requires for lock 
contention to be a problem. I don't have the answer to that question, but I 
would not be surprised if it was a significant problem at 64 or more. Luckily, 
while the general trend is for more and more threads even on commodity 
hardware, I don't see this size happening any time soon (note I'm not saying 
never). That said, it is a valid endpoint to scale beyond that if possible in 
the design if one wished BFS to be used outside the commodity hardware 
spectrum.

 One run-queue per CPU does scale well, but then the scheduler has
 load-balancing overhead. The scheduler I'm developing supports every
 possible run-queues configuration. You can have one single run-queue
 like in BFS, or you can have one run-queue per CPU, or something
 completely different like one run-queue every two CPU's. This, in theory
 would  allow the scheduler to be fine-tuned to the hardware and the
 workload.

I agree with you this is by far the most interesting way of tackling this 
problem, while trying to maintain the benefits of the shared runqueue as much 
as possible. Given that lock contention is not a problem even up to 16 
threads/CPUs, it gives scope to have clusters of CPUs benefiting from the 
shared runqueue aspects BFS has while still allowing it to scale beyond that. 
I would hazard a guess that optimal would simply be one runqueue per shared 
cache.

 
 What state is it in?
 Currently it is very unstable, CPU-Hotplug is broken, scheduling
 statistics are broken, support for real-time tasks is broken. Load
 balancing when having more than one run-queue is working, but is nothing
 more than keeping the load on all run-queues equal. Associating a CPU
 and a run-queue is currently done with a system call and there is no
 access right checking. The source is in a very bad state.
 Uni-processor build is broken.
 It lacks proper Documentation.
 
 Why allow the user to change the run-queue layout?
 To optimize the scheduler to specific hardware and workloads.
 You could use one run-queue for all CPU's if you want low latency and
 low scheduling overhead.
 You could use one run-queue per CPU if you want high scalability.
 You could use one run-queue per n CPU's is these n CPU's share cache and
 there is not much benefit in load balancing between them

Re: [ANNOUNCE] 3.7-ck1, BFS 426 for linux-3.7

[Con Kolivas]

On Sun, 16 Dec 2012 18:33:14 Hillf Danton wrote:
> On Sun, Dec 16, 2012 at 5:54 PM, Con Kolivas  wrote:
> > On Sun, 16 Dec 2012 11:16:31 Con Kolivas wrote:
> >> These are patches designed to improve system responsiveness and
> >> interactivity with specific emphasis on the desktop, but suitable to
> >> any commodity hardware workload.
> >> 
> >> Apply to 3.7.x:
> >> -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.bz2
> >> or
> >> -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.lrz
> >> 
> >> Broken out tarball:
> >> -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.bz2
> >> or
> >> -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.lrz
> >> 
> >> Discrete patches:
> >> -ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patches/
> >> 
> >> Latest BFS by itself:
> >> http://ck.kolivas.org/patches/bfs/3.0/3.7/3.7-sched-bfs-426.patch
> >> 
> >> Web:
> >> http://kernel.kolivas.org
> >> 
> >> Code blog when I feel like it:
> >> http://ck-hack.blogspot.com/
> > 
> > Of course these links all got scrambled somehow, but I'm sure you can
> > figure out where the patches are :p
> 
> Better if changes after 425 are directly attached?
> 
> Hillf

That's of not much use without all the merging changes as well. The 425 patch 
for mainline 3.7 looks nothing like the 425 patch for 3.6. see my blog for 
details.

Con
-- 
-ck
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ANNOUNCE] 3.7-ck1, BFS 426 for linux-3.7

[Con Kolivas]

On Sun, 16 Dec 2012 11:16:31 Con Kolivas wrote:
> These are patches designed to improve system responsiveness and
> interactivity with specific emphasis on the desktop, but suitable to
> any commodity hardware workload.
> 
> Apply to 3.7.x:
> -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.bz2
> or
> -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.lrz
> 
> Broken out tarball:
> -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.bz2
> or
> -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.lrz
> 
> Discrete patches:
> -ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patches/
> 
> Latest BFS by itself:
> http://ck.kolivas.org/patches/bfs/3.0/3.7/3.7-sched-bfs-426.patch
> 
> Web:
> http://kernel.kolivas.org
> 
> Code blog when I feel like it:
> http://ck-hack.blogspot.com/

Of course these links all got scrambled somehow, but I'm sure you can figure 
out where the patches are :p

-- 
-ck
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ANNOUNCE] 3.7-ck1, BFS 426 for linux-3.7

[Con Kolivas]

On Sun, 16 Dec 2012 11:16:31 Con Kolivas wrote:
 These are patches designed to improve system responsiveness and
 interactivity with specific emphasis on the desktop, but suitable to
 any commodity hardware workload.
 
 Apply to 3.7.x:
 -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.bz2
 or
 -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.lrz
 
 Broken out tarball:
 -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.bz2
 or
 -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.lrz
 
 Discrete patches:
 -ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patches/
 
 Latest BFS by itself:
 http://ck.kolivas.org/patches/bfs/3.0/3.7/3.7-sched-bfs-426.patch
 
 Web:
 http://kernel.kolivas.org
 
 Code blog when I feel like it:
 http://ck-hack.blogspot.com/

Of course these links all got scrambled somehow, but I'm sure you can figure 
out where the patches are :p

-- 
-ck
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ANNOUNCE] 3.7-ck1, BFS 426 for linux-3.7

[Con Kolivas]

On Sun, 16 Dec 2012 18:33:14 Hillf Danton wrote:
 On Sun, Dec 16, 2012 at 5:54 PM, Con Kolivas ker...@kolivas.org wrote:
  On Sun, 16 Dec 2012 11:16:31 Con Kolivas wrote:
  These are patches designed to improve system responsiveness and
  interactivity with specific emphasis on the desktop, but suitable to
  any commodity hardware workload.
  
  Apply to 3.7.x:
  -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.bz2
  or
  -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.lrz
  
  Broken out tarball:
  -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.bz2
  or
  -ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.lrz
  
  Discrete patches:
  -ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patches/
  
  Latest BFS by itself:
  http://ck.kolivas.org/patches/bfs/3.0/3.7/3.7-sched-bfs-426.patch
  
  Web:
  http://kernel.kolivas.org
  
  Code blog when I feel like it:
  http://ck-hack.blogspot.com/
  
  Of course these links all got scrambled somehow, but I'm sure you can
  figure out where the patches are :p
 
 Better if changes after 425 are directly attached?
 
 Hillf

That's of not much use without all the merging changes as well. The 425 patch 
for mainline 3.7 looks nothing like the 425 patch for 3.6. see my blog for 
details.

Con
-- 
-ck
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] 3.7-ck1, BFS 426 for linux-3.7

[Con Kolivas]

These are patches designed to improve system responsiveness and
interactivity with specific emphasis on the desktop, but suitable to
any commodity hardware workload.

Apply to 3.7.x:
-ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.bz2
or
-ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.lrz

Broken out tarball:
-ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.bz2
or
-ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.lrz

Discrete patches:
-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patches/

Latest BFS by itself:
http://ck.kolivas.org/patches/bfs/3.0/3.7/3.7-sched-bfs-426.patch

Web:
http://kernel.kolivas.org

Code blog when I feel like it:
http://ck-hack.blogspot.com/

-- 
-ck
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] 3.7-ck1, BFS 426 for linux-3.7

[Con Kolivas]

These are patches designed to improve system responsiveness and
interactivity with specific emphasis on the desktop, but suitable to
any commodity hardware workload.

Apply to 3.7.x:
-ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.bz2
or
-ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patch-3.71.lrz

Broken out tarball:
-ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.bz2
or
-ck-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/3.71-broken-out.tar.lrz

Discrete patches:
-ckhttp://ck.kolivas.org/patches/3.0/3.7/3.71/patches/

Latest BFS by itself:
http://ck.kolivas.org/patches/bfs/3.0/3.7/3.7-sched-bfs-426.patch

Web:
http://kernel.kolivas.org

Code blog when I feel like it:
http://ck-hack.blogspot.com/

-- 
-ck
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE} 3.5-ck1, BFS scheduler v0.24 for linux-3.5

[Con Kolivas]

These are patches designed to improve system responsiveness and
interactivity with specific emphasis on the desktop, but suitable to
any commodity hardware workload.

Apply to 3.5.x:
http://ck.kolivas.org/patches/3.0/3.5/3.5-ck1/patch-3.5-ck1.bz2
or
http://ck.kolivas.org/patches/3.0/3.5/3.5-ck1/patch-3.5-ck1.lrz

Broken out tarball:
http://ck.kolivas.org/patches/3.0/3.5/3.5-ck1/3.5-ck1-broken-out.tar.bz2
or
http://ck.kolivas.org/patches/3.0/3.5/3.5-ck1/3.5-ck1-broken-out.tar.lrz

Discrete patches:
http://ck.kolivas.org/patches/3.0/3.5/3.5-ck1/patches/

Latest BFS by itself:
http://ck.kolivas.org/patches/bfs/3.5.0/3.5-sched-bfs-424.patch

Web:
http://kernel.kolivas.org

Code blog when I feel like it:
http://ck-hack.blogspot.com/


This is a resync from 3.4-ck3. However, the broken out tarballs above also 
include the upgradeable rwlocks patch, and a modification of the global 
runqueue in BFS to use the urwlocks. These are NOT applied in the -ck1 patch, 
but can be applied manually at the  end of the series as indicated by the 
series file. It is currently of no demonstrable performance advantage OR 
detriment in its current state, but is code for future development.

Enjoy!
お楽しみください

-- 
-ck
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE} 3.5-ck1, BFS scheduler v0.24 for linux-3.5

[Con Kolivas]

These are patches designed to improve system responsiveness and
interactivity with specific emphasis on the desktop, but suitable to
any commodity hardware workload.

Apply to 3.5.x:
http://ck.kolivas.org/patches/3.0/3.5/3.5-ck1/patch-3.5-ck1.bz2
or
http://ck.kolivas.org/patches/3.0/3.5/3.5-ck1/patch-3.5-ck1.lrz

Broken out tarball:
http://ck.kolivas.org/patches/3.0/3.5/3.5-ck1/3.5-ck1-broken-out.tar.bz2
or
http://ck.kolivas.org/patches/3.0/3.5/3.5-ck1/3.5-ck1-broken-out.tar.lrz

Discrete patches:
http://ck.kolivas.org/patches/3.0/3.5/3.5-ck1/patches/

Latest BFS by itself:
http://ck.kolivas.org/patches/bfs/3.5.0/3.5-sched-bfs-424.patch

Web:
http://kernel.kolivas.org

Code blog when I feel like it:
http://ck-hack.blogspot.com/


This is a resync from 3.4-ck3. However, the broken out tarballs above also 
include the upgradeable rwlocks patch, and a modification of the global 
runqueue in BFS to use the urwlocks. These are NOT applied in the -ck1 patch, 
but can be applied manually at the  end of the series as indicated by the 
series file. It is currently of no demonstrable performance advantage OR 
detriment in its current state, but is code for future development.

Enjoy!
お楽しみください

-- 
-ck
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ck] Re: Linus 2.6.23-rc1

[Con Kolivas]

Interesting... Trying to avoid reading email but with a flooded inbox it's 
quite hard to do.

A lot of useful discussion seems to have generated in response to people's 
_interpretation_ of my interview rather than what I actually said. For 
example, everyone seems to think I quit because CFS was chosen over SD (hint: 
it wasn't). Since it's generating good discussion I'll otherwise leave it as 
is.


As a parting gesture; a couple of hints for CFS. 

Any difference in behaviour between CFS and SD since they both aim for 
fairness would come down to the way they interpret fair. Since CFS accounts 
sleep time whereas SD does not, that would be the reason.

As for volanomark regressions, they're always the sched_yield implementation. 
SD addressed a similar regression a few months back.

Good luck.

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ck] Re: Linus 2.6.23-rc1

[Con Kolivas]

Interesting... Trying to avoid reading email but with a flooded inbox it's 
quite hard to do.

A lot of useful discussion seems to have generated in response to people's 
_interpretation_ of my interview rather than what I actually said. For 
example, everyone seems to think I quit because CFS was chosen over SD (hint: 
it wasn't). Since it's generating good discussion I'll otherwise leave it as 
is.


As a parting gesture; a couple of hints for CFS. 

Any difference in behaviour between CFS and SD since they both aim for 
fairness would come down to the way they interpret fair. Since CFS accounts 
sleep time whereas SD does not, that would be the reason.

As for volanomark regressions, they're always the sched_yield implementation. 
SD addressed a similar regression a few months back.

Good luck.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: -mm merge plans for 2.6.23

[Con Kolivas]

On Tuesday 10 July 2007 20:15, Con Kolivas wrote:
> On Tuesday 10 July 2007 18:31, Andrew Morton wrote:
> > When replying, please rewrite the subject suitably and try to Cc: the
> > appropriate developer(s).
>
> ~swap prefetch
>
> Nick's only remaining issue which I could remotely identify was to make it
> cpuset aware:
> http://marc.info/?l=linux-mm=117875557014098=2
> as discussed with Paul Jackson it was cpuset aware:
> http://marc.info/?l=linux-mm=117895463120843=2
>
> I fixed all bugs I could find and improved it as much as I could last
> kernel cycle.
>
> Put me and the users out of our misery and merge it now or delete it
> forever please. And if the meaningless handwaving that I 100% expect as a
> response begins again, then that's fine. I'll take that as a no and you can
> dump it.

The window for 2.6.23 has now closed and your position on this is clear. I've 
been supporting this code in -mm for 21 months since 16-Oct-2005 without any 
obvious decision for this code forwards or backwards.

I am no longer part of your operating system's kernel's world; thus I cannot 
support this code any longer. Unless someone takes over the code base for 
swap prefetch you have to assume it is now unmaintained and should delete it.

Please respect my request to not be contacted further regarding this or any 
other kernel code.

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: -mm merge plans for 2.6.23

[Con Kolivas]

On Tuesday 10 July 2007 20:15, Con Kolivas wrote:
 On Tuesday 10 July 2007 18:31, Andrew Morton wrote:
  When replying, please rewrite the subject suitably and try to Cc: the
  appropriate developer(s).

 ~swap prefetch

 Nick's only remaining issue which I could remotely identify was to make it
 cpuset aware:
 http://marc.info/?l=linux-mmm=117875557014098w=2
 as discussed with Paul Jackson it was cpuset aware:
 http://marc.info/?l=linux-mmm=117895463120843w=2

 I fixed all bugs I could find and improved it as much as I could last
 kernel cycle.

 Put me and the users out of our misery and merge it now or delete it
 forever please. And if the meaningless handwaving that I 100% expect as a
 response begins again, then that's fine. I'll take that as a no and you can
 dump it.

The window for 2.6.23 has now closed and your position on this is clear. I've 
been supporting this code in -mm for 21 months since 16-Oct-2005 without any 
obvious decision for this code forwards or backwards.

I am no longer part of your operating system's kernel's world; thus I cannot 
support this code any longer. Unless someone takes over the code base for 
swap prefetch you have to assume it is now unmaintained and should delete it.

Please respect my request to not be contacted further regarding this or any 
other kernel code.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: -mm merge plans for 2.6.23

[Con Kolivas]

On Tuesday 10 July 2007 18:31, Andrew Morton wrote:
> When replying, please rewrite the subject suitably and try to Cc: the
> appropriate developer(s).

~swap prefetch

Nick's only remaining issue which I could remotely identify was to make it 
cpuset aware:
http://marc.info/?l=linux-mm=117875557014098=2
as discussed with Paul Jackson it was cpuset aware:
http://marc.info/?l=linux-mm=117895463120843=2

I fixed all bugs I could find and improved it as much as I could last kernel 
cycle.

Put me and the users out of our misery and merge it now or delete it forever 
please. And if the meaningless handwaving that I 100% expect as a response 
begins again, then that's fine. I'll take that as a no and you can dump it.

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: -mm merge plans for 2.6.23

[Con Kolivas]

On Tuesday 10 July 2007 18:31, Andrew Morton wrote:
 When replying, please rewrite the subject suitably and try to Cc: the
 appropriate developer(s).

~swap prefetch

Nick's only remaining issue which I could remotely identify was to make it 
cpuset aware:
http://marc.info/?l=linux-mmm=117875557014098w=2
as discussed with Paul Jackson it was cpuset aware:
http://marc.info/?l=linux-mmm=117895463120843w=2

I fixed all bugs I could find and improved it as much as I could last kernel 
cycle.

Put me and the users out of our misery and merge it now or delete it forever 
please. And if the meaningless handwaving that I 100% expect as a response 
begins again, then that's fine. I'll take that as a no and you can dump it.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2.6.22-ck1

[Con Kolivas]

2.6.22-ck1 is the last ever -ck release.

So long, and thanks for all the fish


This patchset is designed to improve system responsiveness and interactivity. 
It is configurable to any workload but the default -ck patch is aimed at the 
desktop and -cks is available with more emphasis on serverspace.


Apply to 2.6.22
http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/2.6.22/2.6.22-ck1/patch-2.6.22-ck1.bz2

or server version
http://www.kernel.org/pub/linux/kernel/people/ck/patches/cks/patch-2.6.22-cks1.bz2

web:
http://kernel.kolivas.org

wiki:
http://ck.wikia.com

all patches:
http://www.kernel.org/pub/linux/kernel/people/ck/patches/


Full patch list:

sched-staircase-deadline-cpu-scheduler.patch
sched-range.patch
sched-iso-5.5.patch
track_mutexes-1.patch
sched-idleprio-2.4.patch
sched-limit_policy_changes.patch
sched-ck-add-above-background-load-function.patch
cfq-ioprio_inherit_rt_class.patch
cfq-iso_idleprio_ionice.patch
mm-swap_prefetch-41.patch 
mm-convert_swappiness_to_mapped.patch
mm-lots_watermark.diff
mm-kswapd_inherit_prio-1.patch 
mm-prio_dependant_scan-2.patch
mm-background_scan-2.patch
mm-filesize_dependant_lru_cache_add.patch
mm-idleprio_prio.patch
kconfig-expose_vmsplit_option.patch
hz-default_1000.patch 
hz-no_default_250.patch 
hz-raise_max-2.patch
ck-desktop-tune.patch
ck-final-version.patch


さようなら、いままで魚をありがとう

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2.6.22-ck1

[Con Kolivas]

2.6.22-ck1 is the last ever -ck release.

So long, and thanks for all the fish


This patchset is designed to improve system responsiveness and interactivity. 
It is configurable to any workload but the default -ck patch is aimed at the 
desktop and -cks is available with more emphasis on serverspace.


Apply to 2.6.22
http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/2.6.22/2.6.22-ck1/patch-2.6.22-ck1.bz2

or server version
http://www.kernel.org/pub/linux/kernel/people/ck/patches/cks/patch-2.6.22-cks1.bz2

web:
http://kernel.kolivas.org

wiki:
http://ck.wikia.com

all patches:
http://www.kernel.org/pub/linux/kernel/people/ck/patches/


Full patch list:

sched-staircase-deadline-cpu-scheduler.patch
sched-range.patch
sched-iso-5.5.patch
track_mutexes-1.patch
sched-idleprio-2.4.patch
sched-limit_policy_changes.patch
sched-ck-add-above-background-load-function.patch
cfq-ioprio_inherit_rt_class.patch
cfq-iso_idleprio_ionice.patch
mm-swap_prefetch-41.patch 
mm-convert_swappiness_to_mapped.patch
mm-lots_watermark.diff
mm-kswapd_inherit_prio-1.patch 
mm-prio_dependant_scan-2.patch
mm-background_scan-2.patch
mm-filesize_dependant_lru_cache_add.patch
mm-idleprio_prio.patch
kconfig-expose_vmsplit_option.patch
hz-default_1000.patch 
hz-no_default_250.patch 
hz-raise_max-2.patch
ck-desktop-tune.patch
ck-final-version.patch


さようなら、いままで魚をありがとう

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Wrong cache size reported on Q6600

[Con Kolivas]

On Friday 29 June 2007 09:33, Siddha, Suresh B wrote:
> On Fri, Jun 29, 2007 at 09:31:44AM +1000, Con Kolivas wrote:
> > This is a Q6600 which has cache size of 8 MB. Unless it's reporting each
> > half's effective L2, I think it should be reporting 8192 instead of 4096.
>
> There are two L2's, each of 4MB. Each L2 shared by two cores.

That was what I wasn't sure of as I said above; thanks for clarifying.

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Wrong cache size reported on Q6600

[Con Kolivas]

This is a Q6600 which has cache size of 8 MB. Unless it's reporting each 
half's effective L2, I think it should be reporting 8192 instead of 4096.

On 2.6.22-rc6:

cat /proc/cpuinfo
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model   : 15
model name  : Intel(R) Core(TM)2 Quad CPU   @ 2.40GHz
stepping: 7
cpu MHz : 2401.919
cache size  : 4096 KB
physical id : 0
siblings: 4
core id : 0
cpu cores   : 4
fpu : yes
fpu_exception   : yes
cpuid level : 10
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm 
constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips: 4806.38
clflush size: 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor   : 1
vendor_id   : GenuineIntel
cpu family  : 6
model   : 15
model name  : Intel(R) Core(TM)2 Quad CPU   @ 2.40GHz
stepping: 7
cpu MHz : 2401.919
cache size  : 4096 KB
physical id : 0
siblings: 4
core id : 1
cpu cores   : 4
fpu : yes
fpu_exception   : yes
cpuid level : 10
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm 
constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips: 4803.73
clflush size: 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor   : 2
vendor_id   : GenuineIntel
cpu family  : 6
model   : 15
model name  : Intel(R) Core(TM)2 Quad CPU   @ 2.40GHz
stepping: 7
cpu MHz : 2401.919
cache size  : 4096 KB
physical id : 0
siblings: 4
core id : 2
cpu cores   : 4
fpu : yes
fpu_exception   : yes
cpuid level : 10
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm 
constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips: 4803.86
clflush size: 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor   : 3
vendor_id   : GenuineIntel
cpu family  : 6
model   : 15
model name  : Intel(R) Core(TM)2 Quad CPU   @ 2.40GHz
stepping: 7
cpu MHz : 2401.919
cache size  : 4096 KB
physical id : 0
siblings: 4
core id : 3
cpu cores   : 4
fpu : yes
fpu_exception   : yes
cpuid level : 10
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm 
constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips: 4803.93
clflush size: 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Wrong cache size reported on Q6600

[Con Kolivas]

This is a Q6600 which has cache size of 8 MB. Unless it's reporting each 
half's effective L2, I think it should be reporting 8192 instead of 4096.

On 2.6.22-rc6:

cat /proc/cpuinfo
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model   : 15
model name  : Intel(R) Core(TM)2 Quad CPU   @ 2.40GHz
stepping: 7
cpu MHz : 2401.919
cache size  : 4096 KB
physical id : 0
siblings: 4
core id : 0
cpu cores   : 4
fpu : yes
fpu_exception   : yes
cpuid level : 10
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm 
constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips: 4806.38
clflush size: 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor   : 1
vendor_id   : GenuineIntel
cpu family  : 6
model   : 15
model name  : Intel(R) Core(TM)2 Quad CPU   @ 2.40GHz
stepping: 7
cpu MHz : 2401.919
cache size  : 4096 KB
physical id : 0
siblings: 4
core id : 1
cpu cores   : 4
fpu : yes
fpu_exception   : yes
cpuid level : 10
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm 
constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips: 4803.73
clflush size: 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor   : 2
vendor_id   : GenuineIntel
cpu family  : 6
model   : 15
model name  : Intel(R) Core(TM)2 Quad CPU   @ 2.40GHz
stepping: 7
cpu MHz : 2401.919
cache size  : 4096 KB
physical id : 0
siblings: 4
core id : 2
cpu cores   : 4
fpu : yes
fpu_exception   : yes
cpuid level : 10
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm 
constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips: 4803.86
clflush size: 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor   : 3
vendor_id   : GenuineIntel
cpu family  : 6
model   : 15
model name  : Intel(R) Core(TM)2 Quad CPU   @ 2.40GHz
stepping: 7
cpu MHz : 2401.919
cache size  : 4096 KB
physical id : 0
siblings: 4
core id : 3
cpu cores   : 4
fpu : yes
fpu_exception   : yes
cpuid level : 10
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm 
constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips: 4803.93
clflush size: 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Wrong cache size reported on Q6600

[Con Kolivas]

On Friday 29 June 2007 09:33, Siddha, Suresh B wrote:
 On Fri, Jun 29, 2007 at 09:31:44AM +1000, Con Kolivas wrote:
  This is a Q6600 which has cache size of 8 MB. Unless it's reporting each
  half's effective L2, I think it should be reporting 8192 instead of 4096.

 There are two L2's, each of 4MB. Each L2 shared by two cores.

That was what I wasn't sure of as I said above; thanks for clarifying.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [-mm patch] make mm/swap_prefetch.c:remove_from_swapped_list() static

[Con Kolivas]

On Tuesday 12 June 2007 21:07, Adrian Bunk wrote:
> remove_from_swapped_list() can become static.
>
> Signed-off-by: Adrian Bunk <[EMAIL PROTECTED]>

Thanks. Good pick up. It was a global function but now is only done locally 
from within swap_prefetch.c lazily since the improvements.

Signed-off-by: Con Kolivas <[EMAIL PROTECTED]>

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: call for more SD versus CFS comparisons (was: Re: [ck] Mainline plans)

[Con Kolivas]

On Tuesday 12 June 2007 18:57, Ingo Molnar wrote:
> * Ingo Molnar <[EMAIL PROTECTED]> wrote:
> > * Tobias Gerschner <[EMAIL PROTECTED]> wrote:
> > > I did run massive_intr.c for 60 secs with increasing nproc (
> > > 10,20,30,40,50,60) waiting for effects.
> > >
> > > Below a small table of the results

Nice results. Thanks for taking the time to post them!

> > >
> > > 2.6.21.1-cfs-v16
> > >
> > > nproc ,  usability result
> > >
> > > 10 , serious frame drops , Firefox  hardly recognizes clicked links,
> > > but still usable
> > > 20 - 30, usability loss ( somehow under cfs firefox never finished
> > > user requests like displaying web pages or opening new pages , no
> > > feedback anymore, sudden changes on the desktop )
> >
> > ouch! The expected load-testing result under CFS should be something
> > like this:
> >
> >http://bhhdoa.org.au/pipermail/ck/2007-June/007817.html
>
> i have just tried the same workload with cfs and with sd048 from -ck,
> and cannot reproduce this. To make sure it's not some other change in
> -ck, could you try the pure SD patches ontop of 2.6.21.1 too:

I'm pleased you think the rest of my patches may help there but only the SD 
patches affect scheduling unless you set a different scheduling policy or 
there is a vm issue.

List:
ck2-version.patch
2.6.21-sd-0.48.patch
sched-sd-0.48-interactive_tunable.patch
sched-range.patch
sched-iso-5.4.patch
track_mutexes-1.patch
sched-idleprio-2.3.patch
sched-limit_policy_changes.patch
sched-ck-add-above-background-load-function.patch
cfq-ioprio_inherit_rt_class.patch
cfq-iso_idleprio_ionice.patch
mm-swap_prefetch-35.patch 
mm-convert_swappiness_to_mapped.patch 
mm-lots_watermark.diff
mm-kswapd_inherit_prio-1.patch 
mm-prio_dependant_scan-2.patch
mm-background_scan-2.patch
mm-filesize_dependant_lru_cache_add.patch
mm-idleprio_prio.patch
kconfig-expose_vmsplit_option.patch
hz-default_1000.patch 
hz-no_default_250.patch 
hz-raise_max-2.patch
ck-desktop-tune.patch
mm-swap-prefetch-35-38.patch

apart from ck-desktop-tune.patch which is in total this:
-int rr_interval __read_mostly = 8;
+int rr_interval __read_mostly = 6;

which is already a tunable that's part of SD.

So unless there's a vm issue (which does not appear to be the case) I can't 
see how any of these will change Tobias' extensive testing results.

Thanks.

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: call for more SD versus CFS comparisons (was: Re: [ck] Mainline plans)

[Con Kolivas]

On Tuesday 12 June 2007 18:57, Ingo Molnar wrote:
 * Ingo Molnar [EMAIL PROTECTED] wrote:
  * Tobias Gerschner [EMAIL PROTECTED] wrote:
   I did run massive_intr.c for 60 secs with increasing nproc (
   10,20,30,40,50,60) waiting for effects.
  
   Below a small table of the results

Nice results. Thanks for taking the time to post them!

  
   2.6.21.1-cfs-v16
  
   nproc ,  usability result
  
   10 , serious frame drops , Firefox  hardly recognizes clicked links,
   but still usable
   20 - 30, usability loss ( somehow under cfs firefox never finished
   user requests like displaying web pages or opening new pages , no
   feedback anymore, sudden changes on the desktop )
 
  ouch! The expected load-testing result under CFS should be something
  like this:
 
 http://bhhdoa.org.au/pipermail/ck/2007-June/007817.html

 i have just tried the same workload with cfs and with sd048 from -ck,
 and cannot reproduce this. To make sure it's not some other change in
 -ck, could you try the pure SD patches ontop of 2.6.21.1 too:

I'm pleased you think the rest of my patches may help there but only the SD 
patches affect scheduling unless you set a different scheduling policy or 
there is a vm issue.

List:
ck2-version.patch
2.6.21-sd-0.48.patch
sched-sd-0.48-interactive_tunable.patch
sched-range.patch
sched-iso-5.4.patch
track_mutexes-1.patch
sched-idleprio-2.3.patch
sched-limit_policy_changes.patch
sched-ck-add-above-background-load-function.patch
cfq-ioprio_inherit_rt_class.patch
cfq-iso_idleprio_ionice.patch
mm-swap_prefetch-35.patch 
mm-convert_swappiness_to_mapped.patch 
mm-lots_watermark.diff
mm-kswapd_inherit_prio-1.patch 
mm-prio_dependant_scan-2.patch
mm-background_scan-2.patch
mm-filesize_dependant_lru_cache_add.patch
mm-idleprio_prio.patch
kconfig-expose_vmsplit_option.patch
hz-default_1000.patch 
hz-no_default_250.patch 
hz-raise_max-2.patch
ck-desktop-tune.patch
mm-swap-prefetch-35-38.patch

apart from ck-desktop-tune.patch which is in total this:
-int rr_interval __read_mostly = 8;
+int rr_interval __read_mostly = 6;

which is already a tunable that's part of SD.

So unless there's a vm issue (which does not appear to be the case) I can't 
see how any of these will change Tobias' extensive testing results.

Thanks.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [-mm patch] make mm/swap_prefetch.c:remove_from_swapped_list() static

[Con Kolivas]

On Tuesday 12 June 2007 21:07, Adrian Bunk wrote:
 remove_from_swapped_list() can become static.

 Signed-off-by: Adrian Bunk [EMAIL PROTECTED]

Thanks. Good pick up. It was a global function but now is only done locally 
from within swap_prefetch.c lazily since the improvements.

Signed-off-by: Con Kolivas [EMAIL PROTECTED]

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re:swap prefetch improvements

[Con Kolivas]

On Wednesday 30 May 2007 05:59, Antonino Ingargiola wrote:
> 2007/5/29, Antonino Ingargiola <[EMAIL PROTECTED]>:
> [cut]
>
> > Swap Prefetch OFF
> > # ./sp_tester
> > Ram 776388000  Swap 51404
> > Total ram to be malloced: 1033408000 bytes
> > Starting first malloc of 516704000 bytes
> > Starting 1st read of first malloc
> > Touching this much ram takes 1642 milliseconds
> > Starting second malloc of 516704000 bytes
> > Completed second malloc and free
> > Sleeping for 60 seconds
> > Important part - starting reread of first malloc
> > Completed read of first malloc
> > Timed portion 9089 milliseconds
> >
> >
> > Swap Prefetch OFF
> > # ./sp_tester
> > Ram 776388000  Swap 51404
> > Total ram to be malloced: 1033408000 bytes
> > Starting first malloc of 516704000 bytes
> > Starting 1st read of first malloc
> > Touching this much ram takes 1635 milliseconds
> > Starting second malloc of 516704000 bytes
> > Completed second malloc and free
> > Sleeping for 60 seconds
> > Important part - starting reread of first malloc
> > Completed read of first malloc
> > Timed portion 1783 milliseconds
>
> The second case is clearly with swap prefetch *ON*, sorry.

Thanks very much for testing! The patch has been taken up by Andrew for the 
next -mm.

どうも
-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re:swap prefetch improvements

[Con Kolivas]

On Wednesday 30 May 2007 05:59, Antonino Ingargiola wrote:
 2007/5/29, Antonino Ingargiola [EMAIL PROTECTED]:
 [cut]

  Swap Prefetch OFF
  # ./sp_tester
  Ram 776388000  Swap 51404
  Total ram to be malloced: 1033408000 bytes
  Starting first malloc of 516704000 bytes
  Starting 1st read of first malloc
  Touching this much ram takes 1642 milliseconds
  Starting second malloc of 516704000 bytes
  Completed second malloc and free
  Sleeping for 60 seconds
  Important part - starting reread of first malloc
  Completed read of first malloc
  Timed portion 9089 milliseconds
 
 
  Swap Prefetch OFF
  # ./sp_tester
  Ram 776388000  Swap 51404
  Total ram to be malloced: 1033408000 bytes
  Starting first malloc of 516704000 bytes
  Starting 1st read of first malloc
  Touching this much ram takes 1635 milliseconds
  Starting second malloc of 516704000 bytes
  Completed second malloc and free
  Sleeping for 60 seconds
  Important part - starting reread of first malloc
  Completed read of first malloc
  Timed portion 1783 milliseconds

 The second case is clearly with swap prefetch *ON*, sorry.

Thanks very much for testing! The patch has been taken up by Andrew for the 
next -mm.

どうも
-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH] mm: swap prefetch increase aggressiveness and tunability

[Con Kolivas]

Swap prefetch is currently too lax in prefetching with extended idle periods
unused. Increase its aggressiveness and tunability.

Make it possible for swap_prefetch to be set to a high value ignoring load
and prefetching regardless.

Add tunables to modify the swap prefetch delay and sleep period on the
fly, and decrease both periods to 1 and 5 seconds respectively.
Extended periods did not decrease the impact any further but greatly
diminished the rate ram was prefetched.

Remove the prefetch_watermark that left free ram unused. The impact of using
the free ram with prefetched pages being put on the tail end of the inactive
list would be minimal and potentially very beneficial, yet testing the
pagestate adds unnecessary expense.

Put kprefetchd to sleep if the low watermarks are hit instead of delaying it.

Increase the maxcount as the lazy removal of swapped entries means we can
easily have many stale entries and not enough entries for good swap prefetch.

Do not delay prefetch in cond_resched() returning positive. That was
pointless and frequently put kprefetchd to sleep for no reason.

Update comments and documentation.

Signed-off-by: Con Kolivas <[EMAIL PROTECTED]>

---
 Documentation/sysctl/vm.txt   |   33 +++-
 include/linux/swap-prefetch.h |3 
 kernel/sysctl.c   |   16 
 mm/swap_prefetch.c|  164 +-
 4 files changed, 117 insertions(+), 99 deletions(-)

Index: linux-2.6.22-rc2-mm1/include/linux/swap-prefetch.h
===
--- linux-2.6.22-rc2-mm1.orig/include/linux/swap-prefetch.h 2007-05-26 
18:52:52.0 +1000
+++ linux-2.6.22-rc2-mm1/include/linux/swap-prefetch.h  2007-05-26 
18:53:53.0 +1000
@@ -4,6 +4,9 @@
 #ifdef CONFIG_SWAP_PREFETCH
 /* mm/swap_prefetch.c */
 extern int swap_prefetch;
+extern int swap_prefetch_delay;
+extern int swap_prefetch_sleep;
+
 struct swapped_entry {
swp_entry_t swp_entry;  /* The actual swap entry */
struct list_headswapped_list;   /* Linked list of entries */
Index: linux-2.6.22-rc2-mm1/kernel/sysctl.c
===
--- linux-2.6.22-rc2-mm1.orig/kernel/sysctl.c   2007-05-26 18:52:52.0 
+1000
+++ linux-2.6.22-rc2-mm1/kernel/sysctl.c2007-05-26 18:53:53.0 
+1000
@@ -978,6 +978,22 @@ static ctl_table vm_table[] = {
.mode   = 0644,
.proc_handler   = _dointvec,
},
+   {
+   .ctl_name   = CTL_UNNUMBERED,
+   .procname   = "swap_prefetch_delay",
+   .data   = _prefetch_delay,
+   .maxlen = sizeof(swap_prefetch_delay),
+   .mode   = 0644,
+   .proc_handler   = _dointvec,
+   },
+   {
+   .ctl_name   = CTL_UNNUMBERED,
+   .procname   = "swap_prefetch_sleep",
+   .data   = _prefetch_sleep,
+   .maxlen = sizeof(swap_prefetch_sleep),
+   .mode   = 0644,
+   .proc_handler   = _dointvec,
+   },
 #endif
{ .ctl_name = 0 }
 };
Index: linux-2.6.22-rc2-mm1/mm/swap_prefetch.c
===
--- linux-2.6.22-rc2-mm1.orig/mm/swap_prefetch.c2007-05-26 
18:52:52.0 +1000
+++ linux-2.6.22-rc2-mm1/mm/swap_prefetch.c 2007-05-26 18:57:17.0 
+1000
@@ -1,7 +1,7 @@
 /*
  * linux/mm/swap_prefetch.c
  *
- * Copyright (C) 2005-2006 Con Kolivas
+ * Copyright (C) 2005-2007 Con Kolivas
  *
  * Written by Con Kolivas <[EMAIL PROTECTED]>
  *
@@ -23,15 +23,22 @@
 #include 
 
 /*
- * Time to delay prefetching if vm is busy or prefetching unsuccessful. There
- * needs to be at least this duration of idle time meaning in practice it can
- * be much longer
+ * sysctls:
+ * swap_prefetch:  0. Disable swap prefetching
+ * 1. Prefetch only when idle and not with laptop_mode
+ * 2. Prefetch when idle and with laptop_mode
+ * 3. Prefetch at all times.
+ * swap_prefetch_delay:Number of seconds to delay prefetching when 
system
+ * is not idle.
+ * swap_prefetch_sleep:Number of seconds to put kprefetchd to sleep 
when
+ * unable to prefetch.
  */
-#define PREFETCH_DELAY (HZ * 5)
-#define DISABLED_PREFETCH_DELAY(HZ * 60)
-
-/* sysctl - enable/disable swap prefetching */
 int swap_prefetch __read_mostly = 1;
+int swap_prefetch_delay __read_mostly = 1;
+int swap_prefetch_sleep __read_mostly = 5;
+
+#define PREFETCH_DELAY (HZ * swap_prefetch_delay)
+#define PREFETCH_SLEEP ((HZ * swap_prefetch_sleep) ? : 1)
 
 struct swapped_root {
unsigned long   busy;   /* vm busy */
@@ -73,7 +80,7 @@ static 

[PATCH] mm: swap prefetch increase aggressiveness and tunability

[Con Kolivas]

Swap prefetch is currently too lax in prefetching with extended idle periods
unused. Increase its aggressiveness and tunability.

Make it possible for swap_prefetch to be set to a high value ignoring load
and prefetching regardless.

Add tunables to modify the swap prefetch delay and sleep period on the
fly, and decrease both periods to 1 and 5 seconds respectively.
Extended periods did not decrease the impact any further but greatly
diminished the rate ram was prefetched.

Remove the prefetch_watermark that left free ram unused. The impact of using
the free ram with prefetched pages being put on the tail end of the inactive
list would be minimal and potentially very beneficial, yet testing the
pagestate adds unnecessary expense.

Put kprefetchd to sleep if the low watermarks are hit instead of delaying it.

Increase the maxcount as the lazy removal of swapped entries means we can
easily have many stale entries and not enough entries for good swap prefetch.

Do not delay prefetch in cond_resched() returning positive. That was
pointless and frequently put kprefetchd to sleep for no reason.

Update comments and documentation.

Signed-off-by: Con Kolivas [EMAIL PROTECTED]

---
 Documentation/sysctl/vm.txt   |   33 +++-
 include/linux/swap-prefetch.h |3 
 kernel/sysctl.c   |   16 
 mm/swap_prefetch.c|  164 +-
 4 files changed, 117 insertions(+), 99 deletions(-)

Index: linux-2.6.22-rc2-mm1/include/linux/swap-prefetch.h
===
--- linux-2.6.22-rc2-mm1.orig/include/linux/swap-prefetch.h 2007-05-26 
18:52:52.0 +1000
+++ linux-2.6.22-rc2-mm1/include/linux/swap-prefetch.h  2007-05-26 
18:53:53.0 +1000
@@ -4,6 +4,9 @@
 #ifdef CONFIG_SWAP_PREFETCH
 /* mm/swap_prefetch.c */
 extern int swap_prefetch;
+extern int swap_prefetch_delay;
+extern int swap_prefetch_sleep;
+
 struct swapped_entry {
swp_entry_t swp_entry;  /* The actual swap entry */
struct list_headswapped_list;   /* Linked list of entries */
Index: linux-2.6.22-rc2-mm1/kernel/sysctl.c
===
--- linux-2.6.22-rc2-mm1.orig/kernel/sysctl.c   2007-05-26 18:52:52.0 
+1000
+++ linux-2.6.22-rc2-mm1/kernel/sysctl.c2007-05-26 18:53:53.0 
+1000
@@ -978,6 +978,22 @@ static ctl_table vm_table[] = {
.mode   = 0644,
.proc_handler   = proc_dointvec,
},
+   {
+   .ctl_name   = CTL_UNNUMBERED,
+   .procname   = swap_prefetch_delay,
+   .data   = swap_prefetch_delay,
+   .maxlen = sizeof(swap_prefetch_delay),
+   .mode   = 0644,
+   .proc_handler   = proc_dointvec,
+   },
+   {
+   .ctl_name   = CTL_UNNUMBERED,
+   .procname   = swap_prefetch_sleep,
+   .data   = swap_prefetch_sleep,
+   .maxlen = sizeof(swap_prefetch_sleep),
+   .mode   = 0644,
+   .proc_handler   = proc_dointvec,
+   },
 #endif
{ .ctl_name = 0 }
 };
Index: linux-2.6.22-rc2-mm1/mm/swap_prefetch.c
===
--- linux-2.6.22-rc2-mm1.orig/mm/swap_prefetch.c2007-05-26 
18:52:52.0 +1000
+++ linux-2.6.22-rc2-mm1/mm/swap_prefetch.c 2007-05-26 18:57:17.0 
+1000
@@ -1,7 +1,7 @@
 /*
  * linux/mm/swap_prefetch.c
  *
- * Copyright (C) 2005-2006 Con Kolivas
+ * Copyright (C) 2005-2007 Con Kolivas
  *
  * Written by Con Kolivas [EMAIL PROTECTED]
  *
@@ -23,15 +23,22 @@
 #include linux/freezer.h
 
 /*
- * Time to delay prefetching if vm is busy or prefetching unsuccessful. There
- * needs to be at least this duration of idle time meaning in practice it can
- * be much longer
+ * sysctls:
+ * swap_prefetch:  0. Disable swap prefetching
+ * 1. Prefetch only when idle and not with laptop_mode
+ * 2. Prefetch when idle and with laptop_mode
+ * 3. Prefetch at all times.
+ * swap_prefetch_delay:Number of seconds to delay prefetching when 
system
+ * is not idle.
+ * swap_prefetch_sleep:Number of seconds to put kprefetchd to sleep 
when
+ * unable to prefetch.
  */
-#define PREFETCH_DELAY (HZ * 5)
-#define DISABLED_PREFETCH_DELAY(HZ * 60)
-
-/* sysctl - enable/disable swap prefetching */
 int swap_prefetch __read_mostly = 1;
+int swap_prefetch_delay __read_mostly = 1;
+int swap_prefetch_sleep __read_mostly = 5;
+
+#define PREFETCH_DELAY (HZ * swap_prefetch_delay)
+#define PREFETCH_SLEEP ((HZ * swap_prefetch_sleep) ? : 1)
 
 struct swapped_root {
unsigned long   busy;   /* vm busy */
@@ -73,7 +80,7 @@ static inline int

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Con Kolivas]

On Wednesday 23 May 2007 10:28, Bill Davidsen wrote:
> > kernel2.6.21-cfs-v132.6.21-ck2
> > a)194464254669
> > b)54159124
>
> Everyone seems to like ck2, this makes it look as if the video display
> would be really pretty unusable. While sd-0.48 does show an occasional
> video glitch when watching video under heavy load, it's annoying rather
> than unusable.

That's because the whole premise of your benchmark relies on a workload that 
yield()s itself to the eyeballs on most graphic card combinations when using 
glxgears. Your test remains a test of sched_yield in the presence of your 
workloads rather than anything else. If people like ck2 it's because in the 
real world with real workloads it is better, rather than on a yield() based 
benchmark. Repeatedly the reports are that 3d apps and games in normal usage 
under -ck are better than mainline and cfs.

--
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Wednesday 23 May 2007 06:42, Ash Milsted wrote:
> Hi. I just did some video encoding on my desktop and I was noticing
> (for the first time in a while) that running apps had to hit swap quite
> a lot when I switched to them (the encoding was going at full blast for
> most of the day, and most of the time other running apps were
> idle). Now, a good half of my RAM appeared to be free during all this,
> so I was thinking at the time that it would be nice if swap prefetch
> could be tunably more aggressive. I guess it would be ideal in this
> case if it could kick in during tunably low disk-IO periods, even if
> the CPU is rather busy. I'm sure you've considered this, so I only butt
> in here to cast a vote for it. :)

In this case nicing the video encode should be enough to make it prefetch even 
during heavy cpu usage. It detects the total nice level rather than the cpu 
usage.

> Of course, I could be completely wrong about the possibility.. and I
> seem to remember that the disk cache can take up about half the ram by
> default without this showing up in 'gnome-system-monitor'... which I
> guess might happen during heavy encoding.. but even if it did, I could
> have set the limit lower, and would then have still appreciated
> prefetching.

I plan to make it prefetch more aggressively by default soon and make it more 
tunable too.

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Tuesday 22 May 2007 20:57, Ingo Molnar wrote:
> * Con Kolivas <[EMAIL PROTECTED]> wrote:
> > On Tuesday 22 May 2007 20:46, Ingo Molnar wrote:
> > > It clearly should not consider 'itself' as IO activity. This
> > > suggests some bug in the 'detect activity' mechanism, agreed? I'm
> > > wondering whether you are seeing the same problem, or is all
> > > swap-prefetch IO on your system continuous until it's done [or some
> > > other IO comes inbetween]?
> >
> > When nothing else is happening anywhere on the system it reads in
> > bursts and goes to sleep during journal writeout.
>
> hm, what do you call 'journal writeout' here that would be happening on
> my system?

Not really sure what you have in terms of fs, but here even with nothing going 
on, ext3 writes to disk every 5 seconds with kjournald.

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Tuesday 22 May 2007 20:46, Ingo Molnar wrote:
> It clearly should not consider 'itself' as IO activity. This suggests
> some bug in the 'detect activity' mechanism, agreed? I'm wondering
> whether you are seeing the same problem, or is all swap-prefetch IO on
> your system continuous until it's done [or some other IO comes
> inbetween]?

When nothing else is happening anywhere on the system it reads in bursts and 
goes to sleep during journal writeout.
 
-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Tuesday 22 May 2007 20:25, Ingo Molnar wrote:
> * Con Kolivas <[EMAIL PROTECTED]> wrote:
> > > > there was nothing else running on the system - so i suspect the
> > > > swapin activity flagged 'itself' as some 'other' activity and
> > > > stopped? The swapins happened in 4 bursts, separated by 5 seconds
> > > > total idleness.
> > >
> > > I've noted burst swapins separated by some seconds of pause in my
> > > desktop system too (with sp_tester and an idle gnome).
> >
> > That really is expected, as just about anything, including journal
> > writeout, would be enough to put it back to sleep for 5 more seconds.
>
> note that nothing like that happened on my system - in the
> swap-prefetch-off case there was _zero_ IO activity during the sleep
> period.

Ok, granted it's _very_ conservative. I really don't want to risk its presence 
being a burden on anything, and the iowait it induces probably makes it turn 
itself off for another PREFETCH_DELAY (5s). I really don't want to cross the 
line to where it is detrimental in any way. Not dropping out on a 
cond_resched and perhaps making the delay tunable should be enough to make it 
a little less "sleepy".

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Tuesday 22 May 2007 20:15, Antonino Ingargiola wrote:
> 2007/5/21, Ingo Molnar <[EMAIL PROTECTED]>:
> > * Con Kolivas <[EMAIL PROTECTED]> wrote:
> > > > A suggestion for improvement: right now swap-prefetch does a small
> > > > bit of swapin every 5 seconds and stays idle inbetween. Could this
> > > > perhaps be made more agressive (optionally perhaps), if the system
> > > > is not swapping otherwise? If block-IO level instrumentation is
> > > > needed to determine idleness of block IO then that is justified too
> > > > i think.
> > >
> > > Hmm.. The timer waits 5 seconds before trying to prefetch, but then
> > > only stops if it detects any activity elsewhere. It doesn't actually
> > > try to go idle in between but it doesn't take much activity to put it
> > > back to sleep, hence detecting yet another "not quite idle" period and
> > > then it goes to sleep again. I guess the sleep interval can actually
> > > be changed as another tunable from 5 seconds to whatever the user
> > > wanted.
> >
> > there was nothing else running on the system - so i suspect the swapin
> > activity flagged 'itself' as some 'other' activity and stopped? The
> > swapins happened in 4 bursts, separated by 5 seconds total idleness.
>
> I've noted burst swapins separated by some seconds of pause in my
> desktop system too (with sp_tester and an idle gnome).

That really is expected, as just about anything, including journal writeout, 
would be enough to put it back to sleep for 5 more seconds.
-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Wednesday 23 May 2007 06:42, Ash Milsted wrote:
 Hi. I just did some video encoding on my desktop and I was noticing
 (for the first time in a while) that running apps had to hit swap quite
 a lot when I switched to them (the encoding was going at full blast for
 most of the day, and most of the time other running apps were
 idle). Now, a good half of my RAM appeared to be free during all this,
 so I was thinking at the time that it would be nice if swap prefetch
 could be tunably more aggressive. I guess it would be ideal in this
 case if it could kick in during tunably low disk-IO periods, even if
 the CPU is rather busy. I'm sure you've considered this, so I only butt
 in here to cast a vote for it. :)

In this case nicing the video encode should be enough to make it prefetch even 
during heavy cpu usage. It detects the total nice level rather than the cpu 
usage.

 Of course, I could be completely wrong about the possibility.. and I
 seem to remember that the disk cache can take up about half the ram by
 default without this showing up in 'gnome-system-monitor'... which I
 guess might happen during heavy encoding.. but even if it did, I could
 have set the limit lower, and would then have still appreciated
 prefetching.

I plan to make it prefetch more aggressively by default soon and make it more 
tunable too.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Con Kolivas]

On Wednesday 23 May 2007 10:28, Bill Davidsen wrote:
  kernel2.6.21-cfs-v132.6.21-ck2
  a)194464254669
  b)54159124

 Everyone seems to like ck2, this makes it look as if the video display
 would be really pretty unusable. While sd-0.48 does show an occasional
 video glitch when watching video under heavy load, it's annoying rather
 than unusable.

That's because the whole premise of your benchmark relies on a workload that 
yield()s itself to the eyeballs on most graphic card combinations when using 
glxgears. Your test remains a test of sched_yield in the presence of your 
workloads rather than anything else. If people like ck2 it's because in the 
real world with real workloads it is better, rather than on a yield() based 
benchmark. Repeatedly the reports are that 3d apps and games in normal usage 
under -ck are better than mainline and cfs.

--
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Tuesday 22 May 2007 20:15, Antonino Ingargiola wrote:
 2007/5/21, Ingo Molnar [EMAIL PROTECTED]:
  * Con Kolivas [EMAIL PROTECTED] wrote:
A suggestion for improvement: right now swap-prefetch does a small
bit of swapin every 5 seconds and stays idle inbetween. Could this
perhaps be made more agressive (optionally perhaps), if the system
is not swapping otherwise? If block-IO level instrumentation is
needed to determine idleness of block IO then that is justified too
i think.
  
   Hmm.. The timer waits 5 seconds before trying to prefetch, but then
   only stops if it detects any activity elsewhere. It doesn't actually
   try to go idle in between but it doesn't take much activity to put it
   back to sleep, hence detecting yet another not quite idle period and
   then it goes to sleep again. I guess the sleep interval can actually
   be changed as another tunable from 5 seconds to whatever the user
   wanted.
 
  there was nothing else running on the system - so i suspect the swapin
  activity flagged 'itself' as some 'other' activity and stopped? The
  swapins happened in 4 bursts, separated by 5 seconds total idleness.

 I've noted burst swapins separated by some seconds of pause in my
 desktop system too (with sp_tester and an idle gnome).

That really is expected, as just about anything, including journal writeout, 
would be enough to put it back to sleep for 5 more seconds.
-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Tuesday 22 May 2007 20:25, Ingo Molnar wrote:
 * Con Kolivas [EMAIL PROTECTED] wrote:
there was nothing else running on the system - so i suspect the
swapin activity flagged 'itself' as some 'other' activity and
stopped? The swapins happened in 4 bursts, separated by 5 seconds
total idleness.
  
   I've noted burst swapins separated by some seconds of pause in my
   desktop system too (with sp_tester and an idle gnome).
 
  That really is expected, as just about anything, including journal
  writeout, would be enough to put it back to sleep for 5 more seconds.

 note that nothing like that happened on my system - in the
 swap-prefetch-off case there was _zero_ IO activity during the sleep
 period.

Ok, granted it's _very_ conservative. I really don't want to risk its presence 
being a burden on anything, and the iowait it induces probably makes it turn 
itself off for another PREFETCH_DELAY (5s). I really don't want to cross the 
line to where it is detrimental in any way. Not dropping out on a 
cond_resched and perhaps making the delay tunable should be enough to make it 
a little less sleepy.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Tuesday 22 May 2007 20:46, Ingo Molnar wrote:
 It clearly should not consider 'itself' as IO activity. This suggests
 some bug in the 'detect activity' mechanism, agreed? I'm wondering
 whether you are seeing the same problem, or is all swap-prefetch IO on
 your system continuous until it's done [or some other IO comes
 inbetween]?

When nothing else is happening anywhere on the system it reads in bursts and 
goes to sleep during journal writeout.
 
-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Tuesday 22 May 2007 20:57, Ingo Molnar wrote:
 * Con Kolivas [EMAIL PROTECTED] wrote:
  On Tuesday 22 May 2007 20:46, Ingo Molnar wrote:
   It clearly should not consider 'itself' as IO activity. This
   suggests some bug in the 'detect activity' mechanism, agreed? I'm
   wondering whether you are seeing the same problem, or is all
   swap-prefetch IO on your system continuous until it's done [or some
   other IO comes inbetween]?
 
  When nothing else is happening anywhere on the system it reads in
  bursts and goes to sleep during journal writeout.

 hm, what do you call 'journal writeout' here that would be happening on
 my system?

Not really sure what you have in terms of fs, but here even with nothing going 
on, ext3 writes to disk every 5 seconds with kjournald.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Monday 21 May 2007 20:03, Ingo Molnar wrote:
> * Con Kolivas <[EMAIL PROTECTED]> wrote:
> > It turns out that fixing swap prefetch was not that hard to fix and
> > improve upon, and since Andrew hasn't dropped swap prefetch, instead
> > here are a swag of fixes and improvements, [...]
>
> it's a reliable win on my testbox too:
>
>  # echo 1 > /proc/sys/vm/swap_prefetch

>  Timed portion 30279 milliseconds
>
> versus:
>
>  # echo 0 > /proc/sys/vm/swap_prefetch
>  # ./sp_tester
>  [...]
>
>  Timed portion 36605 milliseconds
>
> i've repeated these tests to make sure it's a stable win and indeed it
> is:
>
># swap-prefetch-on:
>
>Timed portion 29704 milliseconds
>
># swap-prefetch-off:
>
>Timed portion 34863 milliseconds
>
> Nice work Con!

Thanks!

>
> A suggestion for improvement: right now swap-prefetch does a small bit
> of swapin every 5 seconds and stays idle inbetween. Could this perhaps
> be made more agressive (optionally perhaps), if the system is not
> swapping otherwise? If block-IO level instrumentation is needed to
> determine idleness of block IO then that is justified too i think.

Hmm.. The timer waits 5 seconds before trying to prefetch, but then only stops 
if it detects any activity elsewhere. It doesn't actually try to go idle in 
between but it doesn't take much activity to put it back to sleep, hence 
detecting yet another "not quite idle" period and then it goes to sleep 
again. I guess the sleep interval can actually be changed as another tunable 
from 5 seconds to whatever the user wanted.

> Another suggestion: swap-prefetch seems to be doing all the right
> decisions in the sp_test.c case - so would it be possible to add
> statistics so that it could be verified how much of the swapped-in pages
> were indeed a 'hit' - and how many were recycled without them being
> reused? That could give a reliable, objective metric about how efficient
> swap-prefetch is in any workload.

Well the advantage is twofold potentially; 1. the pages that have been 
prefecthed and become minor faults when they would have been major faults, 
and 2. those that become minor faults (via 1) and then become major faults 
again (since a copy is kept on backing store with swap prefetch). The 
sp_tester only tests for 1, although it would be easy enough to simply do 
another big malloc at the end and see how fast it swapped out again as a 
marker of 2. As for an in-kernel option, it could get kind of expensive 
tracking pages that have done one or both of these. I'll think about an 
affordable way to do this, perhaps it could be just done as a 
debugging/testing patch, but if would be nice to make it cheap enough to have 
there permanently as well. The pages end up in swap cache (in the reverse 
direction pages normally get to swap cache) so the accounting could be done 
somewhere around there.

>   Ingo

Thanks for comments!

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] mm: swap prefetch improvements

[Con Kolivas]

On Monday 21 May 2007 20:03, Ingo Molnar wrote:
 * Con Kolivas [EMAIL PROTECTED] wrote:
  It turns out that fixing swap prefetch was not that hard to fix and
  improve upon, and since Andrew hasn't dropped swap prefetch, instead
  here are a swag of fixes and improvements, [...]

 it's a reliable win on my testbox too:

  # echo 1  /proc/sys/vm/swap_prefetch

  Timed portion 30279 milliseconds

 versus:

  # echo 0  /proc/sys/vm/swap_prefetch
  # ./sp_tester
  [...]

  Timed portion 36605 milliseconds

 i've repeated these tests to make sure it's a stable win and indeed it
 is:

# swap-prefetch-on:

Timed portion 29704 milliseconds

# swap-prefetch-off:

Timed portion 34863 milliseconds

 Nice work Con!

Thanks!


 A suggestion for improvement: right now swap-prefetch does a small bit
 of swapin every 5 seconds and stays idle inbetween. Could this perhaps
 be made more agressive (optionally perhaps), if the system is not
 swapping otherwise? If block-IO level instrumentation is needed to
 determine idleness of block IO then that is justified too i think.

Hmm.. The timer waits 5 seconds before trying to prefetch, but then only stops 
if it detects any activity elsewhere. It doesn't actually try to go idle in 
between but it doesn't take much activity to put it back to sleep, hence 
detecting yet another not quite idle period and then it goes to sleep 
again. I guess the sleep interval can actually be changed as another tunable 
from 5 seconds to whatever the user wanted.

 Another suggestion: swap-prefetch seems to be doing all the right
 decisions in the sp_test.c case - so would it be possible to add
 statistics so that it could be verified how much of the swapped-in pages
 were indeed a 'hit' - and how many were recycled without them being
 reused? That could give a reliable, objective metric about how efficient
 swap-prefetch is in any workload.

Well the advantage is twofold potentially; 1. the pages that have been 
prefecthed and become minor faults when they would have been major faults, 
and 2. those that become minor faults (via 1) and then become major faults 
again (since a copy is kept on backing store with swap prefetch). The 
sp_tester only tests for 1, although it would be easy enough to simply do 
another big malloc at the end and see how fast it swapped out again as a 
marker of 2. As for an in-kernel option, it could get kind of expensive 
tracking pages that have done one or both of these. I'll think about an 
affordable way to do this, perhaps it could be just done as a 
debugging/testing patch, but if would be nice to make it cheap enough to have 
there permanently as well. The pages end up in swap cache (in the reverse 
direction pages normally get to swap cache) so the accounting could be done 
somewhere around there.

   Ingo

Thanks for comments!

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] Staircase Deadline v1.00 cpu scheduler

[Con Kolivas]

The staircase deadline cpu scheduler continues to be the reference with 
respect to interactive fairness for many workloads especially with 3d gaming. 
This version is only trivially different from the version included in the -ck 
patchset which has been very stable. The version number has been incremented 
to version 1.00 purely to reflect that stability. Those on -ck are already 
running the equivalent of this version.

http://www.kernel.org/pub/linux/kernel/people/ck/patches/staircase-deadline/2.6.22-rc2/2.6.22-rc2-sd-1.00.patch

In comparison with the standalone version 0.48 of sd the changes are as 
follows:

Default rr_interval was increased to 10ms on uniprocessor (it is scaled up on 
SMP) for throughput reasons. Note that -ck has 6ms and -cks has 10ms as 
defaults (on UP, double that for 2 cpus). This can be altered (as always) by 
modifying the value in /proc/sys/kernel/rr_interval.

The PRIO value exported to userspace as seen by 'top', 'ps' etc will now 
reflect the relative priority of tasks on the expired array by their value 
being greater than 39.

The interactive tunable patch was rolled into this patch and is enabled by 
default. Note the effect of this tunable is subtle except on 3d games. This 
can be altered by modifying /proc/sys/kernel/interactive. A summary of the 
interactive sysctl from Documentation/sysctl/kernel is:

The staircase-deadline cpu scheduler can be set in either purely
forward-looking mode for absolutely rigid fairness and cpu distribution
according to nice level, or it can allow a small per-process history
to smooth out cpu usage perturbations common in interactive tasks by
enabling this sysctl. While small fairness issues can arise with this
enabled, overall fairness is usually still strongly maintained and
starvation is never possible. Enabling this can significantly smooth
out 3d graphics and games.

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[ANNOUNCE] Staircase Deadline v1.00 cpu scheduler

[Con Kolivas]

The staircase deadline cpu scheduler continues to be the reference with 
respect to interactive fairness for many workloads especially with 3d gaming. 
This version is only trivially different from the version included in the -ck 
patchset which has been very stable. The version number has been incremented 
to version 1.00 purely to reflect that stability. Those on -ck are already 
running the equivalent of this version.

http://www.kernel.org/pub/linux/kernel/people/ck/patches/staircase-deadline/2.6.22-rc2/2.6.22-rc2-sd-1.00.patch

In comparison with the standalone version 0.48 of sd the changes are as 
follows:

Default rr_interval was increased to 10ms on uniprocessor (it is scaled up on 
SMP) for throughput reasons. Note that -ck has 6ms and -cks has 10ms as 
defaults (on UP, double that for 2 cpus). This can be altered (as always) by 
modifying the value in /proc/sys/kernel/rr_interval.

The PRIO value exported to userspace as seen by 'top', 'ps' etc will now 
reflect the relative priority of tasks on the expired array by their value 
being greater than 39.

The interactive tunable patch was rolled into this patch and is enabled by 
default. Note the effect of this tunable is subtle except on 3d games. This 
can be altered by modifying /proc/sys/kernel/interactive. A summary of the 
interactive sysctl from Documentation/sysctl/kernel is:

The staircase-deadline cpu scheduler can be set in either purely
forward-looking mode for absolutely rigid fairness and cpu distribution
according to nice level, or it can allow a small per-process history
to smooth out cpu usage perturbations common in interactive tasks by
enabling this sysctl. While small fairness issues can arise with this
enabled, overall fairness is usually still strongly maintained and
starvation is never possible. Enabling this can significantly smooth
out 3d graphics and games.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/