Re: [RFC 0/8] Variable Order Page Cache

2007-04-21 Thread Andrew Morton
On Fri, 20 Apr 2007 17:48:18 +1000 David Chinner <[EMAIL PROTECTED]> wrote: > Agreed - I was talking about a quick way to hack a real filesystem > in to the VM to start exercising the new VM code without needing to > implement compound page support down the whole I/O stack. Yes. The whole

Re: [RFC 0/8] Variable Order Page Cache

2007-04-21 Thread Andrew Morton
On Fri, 20 Apr 2007 17:48:18 +1000 David Chinner [EMAIL PROTECTED] wrote: Agreed - I was talking about a quick way to hack a real filesystem in to the VM to start exercising the new VM code without needing to implement compound page support down the whole I/O stack. Yes. The whole point of

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread Christoph Lameter
On Fri, 20 Apr 2007, Mel Gorman wrote: > I believe there is an assumption in parts of reclaim that LRU pages are > order-0. An interesting bug or two is likely to rear its head there. Correct. We need to deal with reclaim etc. > > Note that this is proof-of-concept. Lots of functionality is

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread Mel Gorman
On (19/04/07 21:11), Andi Kleen didst pronounce: > > We likely need actual defragmentation support. > > To be honest it looks quite pointless before this is solved. So far it is > not even clear if it is feasible to solve it. > I've written a proposal in an OLS paper on how such a mechanism

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread Mel Gorman
On (19/04/07 09:35), Christoph Lameter didst pronounce: > Variable Order Page Cache Patchset > > This patchset modifies the core VM so that higher order page cache pages > become possible. The higher order page cache pages are compound pages > and can be handled in the same way as regular pages.

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread David Chinner
On Thu, Apr 19, 2007 at 09:47:18PM -0700, William Lee Irwin III wrote: > On Thu, Apr 19, 2007 at 09:35:04AM -0700, Christoph Lameter wrote: > > This patchset modifies the core VM so that higher order page cache pages > > become possible. The higher order page cache pages are compound pages > > and

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread David Chinner
On Fri, Apr 20, 2007 at 08:32:57AM +0200, Jens Axboe wrote: > On Fri, Apr 20 2007, David Chinner wrote: > > > The ramfs driver can be used to test higher order page cache functionality > > > (and may help troubleshoot the VM support until we get some real > > > filesystem > > > and real devices

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread Jens Axboe
On Fri, Apr 20 2007, David Chinner wrote: > > - Higher order pages in the block layer etc. > > It's more drivers that we have to worry about, I think. We don't need to > modify bios to explicitly support compound pages. From bio.h: > > /* > * was unsigned short, but we might as well be ready

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread William Lee Irwin III
On Thu, 19 Apr 2007, William Lee Irwin III wrote: >> Oh dear. Per-file pagesizes are foul. Better to fix up the pagecache's >> radix tree than to restrict it like this. There are other attacks on the >> multiple horizontal internal tree node allocation problem beyond >> outright B+ trees that

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread William Lee Irwin III
On Thu, 19 Apr 2007, William Lee Irwin III wrote: Oh dear. Per-file pagesizes are foul. Better to fix up the pagecache's radix tree than to restrict it like this. There are other attacks on the multiple horizontal internal tree node allocation problem beyond outright B+ trees that allow radix

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread Jens Axboe
On Fri, Apr 20 2007, David Chinner wrote: - Higher order pages in the block layer etc. It's more drivers that we have to worry about, I think. We don't need to modify bios to explicitly support compound pages. From bio.h: /* * was unsigned short, but we might as well be ready for 64kB

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread David Chinner
On Fri, Apr 20, 2007 at 08:32:57AM +0200, Jens Axboe wrote: On Fri, Apr 20 2007, David Chinner wrote: The ramfs driver can be used to test higher order page cache functionality (and may help troubleshoot the VM support until we get some real filesystem and real devices supporting

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread David Chinner
On Thu, Apr 19, 2007 at 09:47:18PM -0700, William Lee Irwin III wrote: On Thu, Apr 19, 2007 at 09:35:04AM -0700, Christoph Lameter wrote: This patchset modifies the core VM so that higher order page cache pages become possible. The higher order page cache pages are compound pages and can be

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread Mel Gorman
On (19/04/07 09:35), Christoph Lameter didst pronounce: Variable Order Page Cache Patchset This patchset modifies the core VM so that higher order page cache pages become possible. The higher order page cache pages are compound pages and can be handled in the same way as regular pages.

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread Mel Gorman
On (19/04/07 21:11), Andi Kleen didst pronounce: We likely need actual defragmentation support. To be honest it looks quite pointless before this is solved. So far it is not even clear if it is feasible to solve it. I've written a proposal in an OLS paper on how such a mechanism would

Re: [RFC 0/8] Variable Order Page Cache

2007-04-20 Thread Christoph Lameter
On Fri, 20 Apr 2007, Mel Gorman wrote: I believe there is an assumption in parts of reclaim that LRU pages are order-0. An interesting bug or two is likely to rear its head there. Correct. We need to deal with reclaim etc. Note that this is proof-of-concept. Lots of functionality is missing

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
On Thu, 19 Apr 2007, William Lee Irwin III wrote: > Oh dear. Per-file pagesizes are foul. Better to fix up the pagecache's > radix tree than to restrict it like this. There are other attacks on the > multiple horizontal internal tree node allocation problem beyond > outright B+ trees that allow

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread William Lee Irwin III
On Thu, Apr 19, 2007 at 09:35:04AM -0700, Christoph Lameter wrote: > This patchset modifies the core VM so that higher order page cache pages > become possible. The higher order page cache pages are compound pages > and can be handled in the same way as regular pages. > The order of the pages is

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
On Fri, 20 Apr 2007, Maxim Levitsky wrote: > First of all, today, packet writing on cd/dvd doesn't work well, it is very > slow because > now all file-systems are limited to 4k-barrier and cd/dvd can write only > 32k/64k packets. > This is why a pktcdvd was written and it emulates those 4k

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
On Fri, 20 Apr 2007, David Chinner wrote: > So looking at this the main thing for converting a filesystem is some extra > bits in the mount process and replacing PAGE_CACHE_* macros with > page_cache_*() wrapper functions. Right. > We can probably set all this up trivially with XFS by allowing

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Maxim Levitsky
On Thursday 19 April 2007 19:35:04 Christoph Lameter wrote: > Variable Order Page Cache Patchset > > This patchset modifies the core VM so that higher order page cache pages > become possible. The higher order page cache pages are compound pages > and can be handled in the same way as regular

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread David Chinner
On Thu, Apr 19, 2007 at 09:35:04AM -0700, Christoph Lameter wrote: > Variable Order Page Cache Patchset > > This patchset modifies the core VM so that higher order page cache pages > become possible. The higher order page cache pages are compound pages > and can be handled in the same way as

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
On Thu, 19 Apr 2007, Andi Kleen wrote: > > We likely need actual defragmentation support. > > To be honest it looks quite pointless before this is solved. So far it is > not even clear if it is feasible to solve it. We have done order 1 / 2 allocations for some limited purposes for some time

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Andi Kleen
> We likely need actual defragmentation support. To be honest it looks quite pointless before this is solved. So far it is not even clear if it is feasible to solve it. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED]

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
On Thu, 19 Apr 2007, Badari Pulavarty wrote: > On Thu, 2007-04-19 at 09:35 -0700, Christoph Lameter wrote: > > Variable Order Page Cache Patchset > > > > .. > mm/built-in.o: In function `__generic_file_aio_write_nolock': > filemap.c:(.text+0x295c): undefined reference to `page_cache_shift' >

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Badari Pulavarty
On Thu, 2007-04-19 at 09:35 -0700, Christoph Lameter wrote: > Variable Order Page Cache Patchset > .. mm/built-in.o: In function `__generic_file_aio_write_nolock': filemap.c:(.text+0x295c): undefined reference to `page_cache_shift' filemap.c:(.text+0x296c): undefined reference to

[RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
Variable Order Page Cache Patchset This patchset modifies the core VM so that higher order page cache pages become possible. The higher order page cache pages are compound pages and can be handled in the same way as regular pages. The order of the pages is determined by the order set up in the

[RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
Variable Order Page Cache Patchset This patchset modifies the core VM so that higher order page cache pages become possible. The higher order page cache pages are compound pages and can be handled in the same way as regular pages. The order of the pages is determined by the order set up in the

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Badari Pulavarty
On Thu, 2007-04-19 at 09:35 -0700, Christoph Lameter wrote: Variable Order Page Cache Patchset .. mm/built-in.o: In function `__generic_file_aio_write_nolock': filemap.c:(.text+0x295c): undefined reference to `page_cache_shift' filemap.c:(.text+0x296c): undefined reference to

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Andi Kleen
We likely need actual defragmentation support. To be honest it looks quite pointless before this is solved. So far it is not even clear if it is feasible to solve it. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
On Thu, 19 Apr 2007, Badari Pulavarty wrote: On Thu, 2007-04-19 at 09:35 -0700, Christoph Lameter wrote: Variable Order Page Cache Patchset .. mm/built-in.o: In function `__generic_file_aio_write_nolock': filemap.c:(.text+0x295c): undefined reference to `page_cache_shift'

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
On Thu, 19 Apr 2007, Andi Kleen wrote: We likely need actual defragmentation support. To be honest it looks quite pointless before this is solved. So far it is not even clear if it is feasible to solve it. We have done order 1 / 2 allocations for some limited purposes for some time (task

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread David Chinner
On Thu, Apr 19, 2007 at 09:35:04AM -0700, Christoph Lameter wrote: Variable Order Page Cache Patchset This patchset modifies the core VM so that higher order page cache pages become possible. The higher order page cache pages are compound pages and can be handled in the same way as regular

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Maxim Levitsky
On Thursday 19 April 2007 19:35:04 Christoph Lameter wrote: Variable Order Page Cache Patchset This patchset modifies the core VM so that higher order page cache pages become possible. The higher order page cache pages are compound pages and can be handled in the same way as regular pages.

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
On Fri, 20 Apr 2007, David Chinner wrote: So looking at this the main thing for converting a filesystem is some extra bits in the mount process and replacing PAGE_CACHE_* macros with page_cache_*() wrapper functions. Right. We can probably set all this up trivially with XFS by allowing

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
On Fri, 20 Apr 2007, Maxim Levitsky wrote: First of all, today, packet writing on cd/dvd doesn't work well, it is very slow because now all file-systems are limited to 4k-barrier and cd/dvd can write only 32k/64k packets. This is why a pktcdvd was written and it emulates those 4k sectors

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread William Lee Irwin III
On Thu, Apr 19, 2007 at 09:35:04AM -0700, Christoph Lameter wrote: This patchset modifies the core VM so that higher order page cache pages become possible. The higher order page cache pages are compound pages and can be handled in the same way as regular pages. The order of the pages is

Re: [RFC 0/8] Variable Order Page Cache

2007-04-19 Thread Christoph Lameter
On Thu, 19 Apr 2007, William Lee Irwin III wrote: Oh dear. Per-file pagesizes are foul. Better to fix up the pagecache's radix tree than to restrict it like this. There are other attacks on the multiple horizontal internal tree node allocation problem beyond outright B+ trees that allow radix