Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-23 Thread Ingo Molnar
* Borislav Petkov wrote: > On Tue, Jul 16, 2013 at 05:55:02PM +0900, Joonsoo Kim wrote: > > How about executing a perf in usermodehelper and collecting output > > in tmpfs? Using this approach, we can start a perf after rootfs > > initialization, > > What for if we can start logging to buffers

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-23 Thread Ingo Molnar
* Borislav Petkov b...@alien8.de wrote: On Tue, Jul 16, 2013 at 05:55:02PM +0900, Joonsoo Kim wrote: How about executing a perf in usermodehelper and collecting output in tmpfs? Using this approach, we can start a perf after rootfs initialization, What for if we can start logging to

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-22 Thread Robin Holt
On Fri, Jul 19, 2013 at 04:51:49PM -0700, Yinghai Lu wrote: > On Wed, Jul 17, 2013 at 2:30 AM, Robin Holt wrote: > > On Wed, Jul 17, 2013 at 01:17:44PM +0800, Sam Ben wrote: > >> >With this patch, we did boot a 16TiB machine. Without the patches, > >> >the v3.10 kernel with the same

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-22 Thread Robin Holt
On Fri, Jul 19, 2013 at 04:51:49PM -0700, Yinghai Lu wrote: On Wed, Jul 17, 2013 at 2:30 AM, Robin Holt h...@sgi.com wrote: On Wed, Jul 17, 2013 at 01:17:44PM +0800, Sam Ben wrote: With this patch, we did boot a 16TiB machine. Without the patches, the v3.10 kernel with the same

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-19 Thread Yinghai Lu
On Wed, Jul 17, 2013 at 2:30 AM, Robin Holt wrote: > On Wed, Jul 17, 2013 at 01:17:44PM +0800, Sam Ben wrote: >> >With this patch, we did boot a 16TiB machine. Without the patches, >> >the v3.10 kernel with the same configuration took 407 seconds for >> >free_all_bootmem. With the patches and

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-19 Thread Yinghai Lu
On Wed, Jul 17, 2013 at 2:30 AM, Robin Holt h...@sgi.com wrote: On Wed, Jul 17, 2013 at 01:17:44PM +0800, Sam Ben wrote: With this patch, we did boot a 16TiB machine. Without the patches, the v3.10 kernel with the same configuration took 407 seconds for free_all_bootmem. With the patches and

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-17 Thread Robin Holt
On Wed, Jul 17, 2013 at 01:17:44PM +0800, Sam Ben wrote: > On 07/12/2013 10:03 AM, Robin Holt wrote: > >We have been working on this since we returned from shutdown and have > >something to discuss now. We restricted ourselves to 2MiB initialization > >to keep the patch set a little smaller and

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-17 Thread Robin Holt
On Wed, Jul 17, 2013 at 01:17:44PM +0800, Sam Ben wrote: On 07/12/2013 10:03 AM, Robin Holt wrote: We have been working on this since we returned from shutdown and have something to discuss now. We restricted ourselves to 2MiB initialization to keep the patch set a little smaller and more

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-16 Thread Sam Ben
On 07/12/2013 10:03 AM, Robin Holt wrote: We have been working on this since we returned from shutdown and have something to discuss now. We restricted ourselves to 2MiB initialization to keep the patch set a little smaller and more clear. First, I think I want to propose getting rid of the

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-16 Thread Borislav Petkov
On Tue, Jul 16, 2013 at 05:55:02PM +0900, Joonsoo Kim wrote: > How about executing a perf in usermodehelper and collecting output > in tmpfs? Using this approach, we can start a perf after rootfs > initialization, What for if we can start logging to buffers much earlier? *Reading* from those

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-16 Thread Joonsoo Kim
On Fri, Jul 12, 2013 at 10:27:56AM +0200, Ingo Molnar wrote: > > * Robin Holt wrote: > > > [...] > > > > With this patch, we did boot a 16TiB machine. Without the patches, the > > v3.10 kernel with the same configuration took 407 seconds for > > free_all_bootmem. With the patches and

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-16 Thread Joonsoo Kim
On Fri, Jul 12, 2013 at 10:27:56AM +0200, Ingo Molnar wrote: * Robin Holt h...@sgi.com wrote: [...] With this patch, we did boot a 16TiB machine. Without the patches, the v3.10 kernel with the same configuration took 407 seconds for free_all_bootmem. With the patches and

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-16 Thread Borislav Petkov
On Tue, Jul 16, 2013 at 05:55:02PM +0900, Joonsoo Kim wrote: How about executing a perf in usermodehelper and collecting output in tmpfs? Using this approach, we can start a perf after rootfs initialization, What for if we can start logging to buffers much earlier? *Reading* from those buffers

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-16 Thread Sam Ben
On 07/12/2013 10:03 AM, Robin Holt wrote: We have been working on this since we returned from shutdown and have something to discuss now. We restricted ourselves to 2MiB initialization to keep the patch set a little smaller and more clear. First, I think I want to propose getting rid of the

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-15 Thread Robin Holt
On Fri, Jul 12, 2013 at 10:27:56AM +0200, Ingo Molnar wrote: > > * Robin Holt wrote: > > > [...] > > > > With this patch, we did boot a 16TiB machine. Without the patches, the > > v3.10 kernel with the same configuration took 407 seconds for > > free_all_bootmem. With the patches and

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-15 Thread Robin Holt
On Thu, Jul 11, 2013 at 09:03:51PM -0500, Robin Holt wrote: > We have been working on this since we returned from shutdown and have > something to discuss now. We restricted ourselves to 2MiB initialization > to keep the patch set a little smaller and more clear. > > First, I think I want to

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-15 Thread Robin Holt
On Thu, Jul 11, 2013 at 09:03:51PM -0500, Robin Holt wrote: We have been working on this since we returned from shutdown and have something to discuss now. We restricted ourselves to 2MiB initialization to keep the patch set a little smaller and more clear. First, I think I want to propose

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-15 Thread Robin Holt
On Fri, Jul 12, 2013 at 10:27:56AM +0200, Ingo Molnar wrote: * Robin Holt h...@sgi.com wrote: [...] With this patch, we did boot a 16TiB machine. Without the patches, the v3.10 kernel with the same configuration took 407 seconds for free_all_bootmem. With the patches and

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-12 Thread Robert Richter
On 12.07.13 10:27:56, Ingo Molnar wrote: > > * Robin Holt wrote: > > > [...] > > > > With this patch, we did boot a 16TiB machine. Without the patches, the > > v3.10 kernel with the same configuration took 407 seconds for > > free_all_bootmem. With the patches and operating on 2MiB pages

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-12 Thread Ingo Molnar
* Robin Holt wrote: > [...] > > With this patch, we did boot a 16TiB machine. Without the patches, the > v3.10 kernel with the same configuration took 407 seconds for > free_all_bootmem. With the patches and operating on 2MiB pages instead > of 1GiB, it took 26 seconds so performance was

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-12 Thread Ingo Molnar
* Robin Holt h...@sgi.com wrote: [...] With this patch, we did boot a 16TiB machine. Without the patches, the v3.10 kernel with the same configuration took 407 seconds for free_all_bootmem. With the patches and operating on 2MiB pages instead of 1GiB, it took 26 seconds so

Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-12 Thread Robert Richter
On 12.07.13 10:27:56, Ingo Molnar wrote: * Robin Holt h...@sgi.com wrote: [...] With this patch, we did boot a 16TiB machine. Without the patches, the v3.10 kernel with the same configuration took 407 seconds for free_all_bootmem. With the patches and operating on 2MiB pages

[RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-11 Thread Robin Holt
We have been working on this since we returned from shutdown and have something to discuss now. We restricted ourselves to 2MiB initialization to keep the patch set a little smaller and more clear. First, I think I want to propose getting rid of the page flag. If I knew of a concrete way to

[RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

2013-07-11 Thread Robin Holt
We have been working on this since we returned from shutdown and have something to discuss now. We restricted ourselves to 2MiB initialization to keep the patch set a little smaller and more clear. First, I think I want to propose getting rid of the page flag. If I knew of a concrete way to