[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
Kernel EOL, closing this bug, please feel free to open a new one. ** Changed in: linux (Ubuntu) Status: Confirmed => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
@Todd are you sure they are not swappable? according to lwn they are: http://lwn.net/Articles/423584/ This scheme will increase the use of huge pages transparently, but it does not yet solve the whole problem. Huge pages must be swappable, lest the system run out of memory in a hurry. Rather than complicate the swapping code with an understanding of huge pages, Andrea simply splits a huge page back into its component small pages if that page needs to be reclaimed. Many other operations (mprotect(), mlock(), ...) will also result in the splitting of a page. also the performance regression might be fixed in 3.11; http://kernel.opensuse.org/cgit/kernel/commit/?id=348944504362417205107e63cfe4821aa87ec1bb mm: fix aio performance regression for database caused by THP -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
@Julian, That isn't what the article is saying. Huge pages are not swappable at the moment. However Andrea's work splits huge pages into base pages for a number of reasons, one of those is memory pressure that would force part of the page to be swapped out. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
I know non-anonymous huge pages, like those from hugetblfs, are currently not swappable. But if I understand the article correctly _anonymous_ huge pages can be swapped. You lose the benefits of huge pages in this case (as they are split) but the system does not regress in functionality. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
Based on what we've been seeing maintaining Oracle Linux, there are many workloads for which THP results in a ~9-10% performance hit overall on the system, and this includes kernels up through 3.8.13 (which we're using as the baseline for a new kernel in that distro). It has to do with the fact that the memory allocator uses the slowpath almost all the time, rather than a faster path when memory fragmentation is low. It's not that THP is _enabled_ so much that it is _compiled into the kernel at all_; as a result, we're probably going to remove it for the time being, and re-enable it when the allocator is revamped. However, there is also good reason for choosing madvise as the default. Transparent hugepages *cannot be swapped out*. Read that carefully: when you use them, the allocated memory is always in-core, as if it had been mlock()ed into place. While systems with plenty of memory shouldn't need to worry about this (hell, I run my systems without swap at all!), it's a consideration to keep in mind. Applications for which huge pages may provide proven benefit, but aren't required, should start to adopt MADV_HUGEPAGE as an advisory call; this might include databases, Web browsers, and other data-intensive applications. That way, you don't run the risk of hitting memory walls thanks to a bunch of smaller system processes that just happen to use hugepages because it's set to always. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
BTW, by we're probably going to remove it for the time being, I'm referring to the kernels in the distro at my day job (OL). I use Ubuntu at home. ;) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
According to Documentation/vm/transhuge.txt you can also set it as the boot parameter transparent_hugepage=always. The problem with setting it via sysfs is that none of the processes started before you made the change will use huge pages. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
I spent a day compiling a custom kernel with transparent huge pages enabled and set to always use mode (not madvise). I have been using the kernel for the last 24 hours without any trouble. I find my laptop more responsive now. An informal test is that Stellarium's CPU utilisation was 45% in the stock kernel, while it went down to 30% in the custom kernel. For a more formal test, we can use oprofile with these commands: sudo opcontrol --init sudo opcontrol --setup --event DTLB_MISSES:1000 --no-vmlinux sudo opcontrol --start [... run some benchmark here ...] sudo opcontrol --stop sudo opcontrol --dump sudo opreport I am able to run the above commands in both stock and custom kernels and see some numbers. But I need to properly define the benchmark before reporting the numbers. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
HRJ: per https://lists.ubuntu.com/archives/ubuntu-devel/2011-September/034060.html, Oneiric will ship with CONFIG_TRANSPARENT_HUGEPAGE=y # CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS is not set CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
Thanks Julian. Too bad they made it madvise and not always. There is sysfs entry at /sys/kernel/mm/transparent_hugepage/ which allows changing the policy at run time, but I am not sure if it can be turned on if the kernel config has madvise. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
$ grep TRANSPARENT /boot/config-2.6.38-11-generic # CONFIG_TRANSPARENT_HUGEPAGE is not set Is there any hope of this being enabled on 11.10? ** Changed in: linux (Ubuntu) Status: Incomplete = Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
so maybe i'm not understanding this correctly, but in regards to * [Config] disable CONFIG_TRANSPARENT_HUGEPAGE to fix i386 boot crashes ... why isn't it only disabled for the i386 ??? ... it appears to be disabled for everything i just installed linux-image-server-lts-backport-natty on Ubuntu 10.04.3 LTS x86_64 , and was looking forward to trying out TRANSPARENT HUGEPAGES -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
Come on, this is automated triage gone insane. Why does anyone have to collect any logs for a report where it takes anyone interested a second or two to confirm that yes, the kernel config is missing the option reported as missing?? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/743688/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
THP is disabled before rebasing to 2.6.38 release. Is this bug still remaining? If so, how about enabling THP on amd64 only? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
Transparent huge pages are enabled in the Fedora Core 15 and work well. I think THP helps common workloads (browser, office suite). If they are not enabled in natty, I may have to switch to Fedora - or at least run some benchmarks, because a 10-15% performance gain is material for me. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
Just noticed this myself. In the changelog for linux-image-2.6.38, I see: linux (2.6.38-1.27) natty; urgency=low ... * [Config] disable CONFIG_TRANSPARENT_HUGEPAGE to fix i386 boot crashes Pretty disappointing that this is just turned off instead of debugging the crash, since it is a major speedup especially for virtualization. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 743688] Re: Transparent HugePages not enabled in 11.04 kernel
** Package changed: ubuntu = linux (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/743688 Title: Transparent HugePages not enabled in 11.04 kernel -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
