[PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-03-01 Thread Mel Gorman
Huge pages are not movable so are not allocated from ZONE_MOVABLE. However, as ZONE_MOVABLE will always have pages that can be migrated or reclaimed, it can be used to satisfy hugepage allocations even when the system has been running a long time. This allows an administrator to resize the

[PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-03-01 Thread Mel Gorman
Huge pages are not movable so are not allocated from ZONE_MOVABLE. However, as ZONE_MOVABLE will always have pages that can be migrated or reclaimed, it can be used to satisfy hugepage allocations even when the system has been running a long time. This allows an administrator to resize the

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Chris Friesen
Mel Gorman wrote: On Fri, 26 Jan 2007, Chris Friesen wrote: We currently see this issue on our systems, as we have older e1000 hardware with 9KB jumbo frames. After a while we just fail to allocate buffers and the system goes belly-up. Can you describe a reliable way of triggering this

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Chris Friesen wrote: Mel Gorman wrote: Worse, the problem is to have high order contiguous blocks free at the time of allocation without reclaim or migration. If the allocations were not atomic, anti-fragmentation as it is today would be enough. Has anyone looked at

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Christoph Lameter wrote: On Fri, 26 Jan 2007, Mel Gorman wrote: The zone-based approach does nothing to help jumbo frame allocations. It only helps hugepage allocations at runtime and potentially memory hot-remove. Sounds like the max order based approach is better in

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Chris Friesen
Mel Gorman wrote: Worse, the problem is to have high order contiguous blocks free at the time of allocation without reclaim or migration. If the allocations were not atomic, anti-fragmentation as it is today would be enough. Has anyone looked at marking the buffers as "needs refilling" then

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Christoph Lameter
On Fri, 26 Jan 2007, Mel Gorman wrote: > The zone-based approach does nothing to help jumbo frame allocations. It only > helps hugepage allocations at runtime and potentially memory hot-remove. Sounds like the max order based approach is better in many ways. Also avoids modifications to

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Christoph Lameter wrote: On Fri, 26 Jan 2007, Mel Gorman wrote: It's come up a few times and the converation is always fairly similar although the thread http://lkml.org/lkml/2006/9/22/44 has interesting information on the topic. There has been no serious discussion on

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Christoph Lameter
On Fri, 26 Jan 2007, Mel Gorman wrote: > It's come up a few times and the converation is always fairly similar although > the thread http://lkml.org/lkml/2006/9/22/44 has interesting information on > the topic. There has been no serious discussion on whether anti-fragmentation > would help it or

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Christoph Lameter wrote: On Fri, 26 Jan 2007, Mel Gorman wrote: What is the e1000 problem? Jumbo packet allocation via GFP_KERNEL? Yes. Potentially the anti-fragmentation patches could address this by clustering atomic allocations together as much as possible.

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Christoph Lameter
On Fri, 26 Jan 2007, Mel Gorman wrote: > > What is the e1000 problem? Jumbo packet allocation via GFP_KERNEL? > Yes. Potentially the anti-fragmentation patches could address this by > clustering atomic allocations together as much as possible. GFP_ATOMIC allocs? Do you have a reference to the

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Christoph Lameter wrote: On Fri, 26 Jan 2007, Mel Gorman wrote: Because Andrew has made it pretty clear he will not take those patches on the grounds of complexity - at least until it can be shown that they fix the e1000 problem. Any improvement on the behavior of those

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Christoph Lameter
On Fri, 26 Jan 2007, Mel Gorman wrote: > Because Andrew has made it pretty clear he will not take those patches on the > grounds of complexity - at least until it can be shown that they fix the e1000 > problem. Any improvement on the behavior of those patches such as address > biasing to allow

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Christoph Lameter wrote: Unmovable allocations in the movable zone. Yuck. I know, but my objective at this time is to allow the hugepage pool to be resized at runtime for situations where the number of required hugepages is not known in advance. Having a zone for

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Christoph Lameter
Unmovable allocations in the movable zone. Yuck. Why dont you abandon the whole concept of statically sized movable zone and go back to the nice earlier idea of dynamically assigning MAX_ORDER chunks to be movable or not? - To unsubscribe from this list: send the line "unsubscribe linux-kernel"

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Christoph Lameter
Unmovable allocations in the movable zone. Yuck. Why dont you abandon the whole concept of statically sized movable zone and go back to the nice earlier idea of dynamically assigning MAX_ORDER chunks to be movable or not? - To unsubscribe from this list: send the line unsubscribe linux-kernel in

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Christoph Lameter wrote: Unmovable allocations in the movable zone. Yuck. I know, but my objective at this time is to allow the hugepage pool to be resized at runtime for situations where the number of required hugepages is not known in advance. Having a zone for

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Christoph Lameter
On Fri, 26 Jan 2007, Mel Gorman wrote: Because Andrew has made it pretty clear he will not take those patches on the grounds of complexity - at least until it can be shown that they fix the e1000 problem. Any improvement on the behavior of those patches such as address biasing to allow memory

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Christoph Lameter wrote: On Fri, 26 Jan 2007, Mel Gorman wrote: Because Andrew has made it pretty clear he will not take those patches on the grounds of complexity - at least until it can be shown that they fix the e1000 problem. Any improvement on the behavior of those

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Christoph Lameter
On Fri, 26 Jan 2007, Mel Gorman wrote: What is the e1000 problem? Jumbo packet allocation via GFP_KERNEL? Yes. Potentially the anti-fragmentation patches could address this by clustering atomic allocations together as much as possible. GFP_ATOMIC allocs? Do you have a reference to the thread

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Christoph Lameter wrote: On Fri, 26 Jan 2007, Mel Gorman wrote: What is the e1000 problem? Jumbo packet allocation via GFP_KERNEL? Yes. Potentially the anti-fragmentation patches could address this by clustering atomic allocations together as much as possible.

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Christoph Lameter
On Fri, 26 Jan 2007, Mel Gorman wrote: It's come up a few times and the converation is always fairly similar although the thread http://lkml.org/lkml/2006/9/22/44 has interesting information on the topic. There has been no serious discussion on whether anti-fragmentation would help it or not.

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Christoph Lameter wrote: On Fri, 26 Jan 2007, Mel Gorman wrote: It's come up a few times and the converation is always fairly similar although the thread http://lkml.org/lkml/2006/9/22/44 has interesting information on the topic. There has been no serious discussion on

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Christoph Lameter
On Fri, 26 Jan 2007, Mel Gorman wrote: The zone-based approach does nothing to help jumbo frame allocations. It only helps hugepage allocations at runtime and potentially memory hot-remove. Sounds like the max order based approach is better in many ways. Also avoids modifications to

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Chris Friesen
Mel Gorman wrote: Worse, the problem is to have high order contiguous blocks free at the time of allocation without reclaim or migration. If the allocations were not atomic, anti-fragmentation as it is today would be enough. Has anyone looked at marking the buffers as needs refilling then

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Christoph Lameter wrote: On Fri, 26 Jan 2007, Mel Gorman wrote: The zone-based approach does nothing to help jumbo frame allocations. It only helps hugepage allocations at runtime and potentially memory hot-remove. Sounds like the max order based approach is better in

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Mel Gorman
On Fri, 26 Jan 2007, Chris Friesen wrote: Mel Gorman wrote: Worse, the problem is to have high order contiguous blocks free at the time of allocation without reclaim or migration. If the allocations were not atomic, anti-fragmentation as it is today would be enough. Has anyone looked at

Re: [PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-26 Thread Chris Friesen
Mel Gorman wrote: On Fri, 26 Jan 2007, Chris Friesen wrote: We currently see this issue on our systems, as we have older e1000 hardware with 9KB jumbo frames. After a while we just fail to allocate buffers and the system goes belly-up. Can you describe a reliable way of triggering this

[PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-25 Thread Mel Gorman
Huge pages are not movable so are not allocated from ZONE_MOVABLE. However, as ZONE_MOVABLE will always have pages that can be migrated or reclaimed, it can be used to satisfy hugepage allocations even when the system has been running a long time. This allows an administrator to resize the

[PATCH 3/8] Allow huge page allocations to use GFP_HIGH_MOVABLE

2007-01-25 Thread Mel Gorman
Huge pages are not movable so are not allocated from ZONE_MOVABLE. However, as ZONE_MOVABLE will always have pages that can be migrated or reclaimed, it can be used to satisfy hugepage allocations even when the system has been running a long time. This allows an administrator to resize the