Re: [PATCH][RFC] fast file mapping for loop

2008-01-15 Thread Chris Mason
On Tue, 15 Jan 2008 11:07:40 +0100 Jens Axboe <[EMAIL PROTECTED]> wrote: > > > I split and merged the patch into five bits (added ext3 support), > > > so perhaps that would be easier for people to read/review. > > > Attached and also exist in the loop-extent_map branch here: Thanks! > > > > >

Re: [PATCH][RFC] fast file mapping for loop

2008-01-15 Thread Jens Axboe
On Tue, Jan 15 2008, Jens Axboe wrote: > On Tue, Jan 15 2008, Jens Axboe wrote: > > On Mon, Jan 14 2008, Jens Axboe wrote: > > > On Mon, Jan 14 2008, Chris Mason wrote: > > > > Hello everyone, > > > > > > > > Here is a modified version of Jens' patch. The basic idea is to push > > > > the

Re: [PATCH][RFC] fast file mapping for loop

2008-01-15 Thread Jens Axboe
On Tue, Jan 15 2008, Jens Axboe wrote: > On Mon, Jan 14 2008, Jens Axboe wrote: > > On Mon, Jan 14 2008, Chris Mason wrote: > > > Hello everyone, > > > > > > Here is a modified version of Jens' patch. The basic idea is to push > > > the mapping maintenance out of loop and down into the

Re: [PATCH][RFC] fast file mapping for loop

2008-01-15 Thread Jens Axboe
On Mon, Jan 14 2008, Jens Axboe wrote: > On Mon, Jan 14 2008, Chris Mason wrote: > > Hello everyone, > > > > Here is a modified version of Jens' patch. The basic idea is to push > > the mapping maintenance out of loop and down into the filesystem (ext2 > > in this case). > > > > Two new

Re: [PATCH][RFC] fast file mapping for loop

2008-01-15 Thread Jens Axboe
On Mon, Jan 14 2008, Jens Axboe wrote: On Mon, Jan 14 2008, Chris Mason wrote: Hello everyone, Here is a modified version of Jens' patch. The basic idea is to push the mapping maintenance out of loop and down into the filesystem (ext2 in this case). Two new address_space

Re: [PATCH][RFC] fast file mapping for loop

2008-01-15 Thread Jens Axboe
On Tue, Jan 15 2008, Jens Axboe wrote: On Mon, Jan 14 2008, Jens Axboe wrote: On Mon, Jan 14 2008, Chris Mason wrote: Hello everyone, Here is a modified version of Jens' patch. The basic idea is to push the mapping maintenance out of loop and down into the filesystem (ext2 in

Re: [PATCH][RFC] fast file mapping for loop

2008-01-15 Thread Jens Axboe
On Tue, Jan 15 2008, Jens Axboe wrote: On Tue, Jan 15 2008, Jens Axboe wrote: On Mon, Jan 14 2008, Jens Axboe wrote: On Mon, Jan 14 2008, Chris Mason wrote: Hello everyone, Here is a modified version of Jens' patch. The basic idea is to push the mapping maintenance out of

Re: [PATCH][RFC] fast file mapping for loop

2008-01-15 Thread Chris Mason
On Tue, 15 Jan 2008 11:07:40 +0100 Jens Axboe [EMAIL PROTECTED] wrote: I split and merged the patch into five bits (added ext3 support), so perhaps that would be easier for people to read/review. Attached and also exist in the loop-extent_map branch here: Thanks!

Re: [PATCH][RFC] fast file mapping for loop

2008-01-14 Thread Jens Axboe
On Mon, Jan 14 2008, Chris Mason wrote: > Hello everyone, > > Here is a modified version of Jens' patch. The basic idea is to push > the mapping maintenance out of loop and down into the filesystem (ext2 > in this case). > > Two new address_space operations are added, one to map > extents and

Re: [PATCH][RFC] fast file mapping for loop

2008-01-14 Thread Chris Mason
Hello everyone, Here is a modified version of Jens' patch. The basic idea is to push the mapping maintenance out of loop and down into the filesystem (ext2 in this case). Two new address_space operations are added, one to map extents and the other to provide call backs into the FS as io is

Re: [PATCH][RFC] fast file mapping for loop

2008-01-14 Thread Chris Mason
Hello everyone, Here is a modified version of Jens' patch. The basic idea is to push the mapping maintenance out of loop and down into the filesystem (ext2 in this case). Two new address_space operations are added, one to map extents and the other to provide call backs into the FS as io is

Re: [PATCH][RFC] fast file mapping for loop

2008-01-14 Thread Jens Axboe
On Mon, Jan 14 2008, Chris Mason wrote: Hello everyone, Here is a modified version of Jens' patch. The basic idea is to push the mapping maintenance out of loop and down into the filesystem (ext2 in this case). Two new address_space operations are added, one to map extents and the other

Re: [PATCH][RFC] fast file mapping for loop

2008-01-11 Thread Jens Axboe
On Fri, Jan 11 2008, Daniel Phillips wrote: > Hi Jens, > > This looks really useful. > > On Wednesday 09 January 2008 00:52, Jens Axboe wrote: > > Disadvantages: > > > > - The file block mappings must not change while loop is using the > > file. This means that we have to ensure exclusive access

Re: [PATCH][RFC] fast file mapping for loop

2008-01-11 Thread Daniel Phillips
Hi Jens, This looks really useful. On Wednesday 09 January 2008 00:52, Jens Axboe wrote: > Disadvantages: > > - The file block mappings must not change while loop is using the > file. This means that we have to ensure exclusive access to the file > and this is the bit that is currently missing

Re: [PATCH][RFC] fast file mapping for loop

2008-01-11 Thread Chris Mason
On Fri, 11 Jan 2008 10:01:18 +1100 Neil Brown <[EMAIL PROTECTED]> wrote: > On Thursday January 10, [EMAIL PROTECTED] wrote: > > On Thu, Jan 10 2008, Chris Mason wrote: > > > On Thu, 10 Jan 2008 09:31:31 +0100 > > > Jens Axboe <[EMAIL PROTECTED]> wrote: > > > > > > > On Wed, Jan 09 2008, Alasdair

Re: [PATCH][RFC] fast file mapping for loop

2008-01-11 Thread Mikulas Patocka
> So I looked at the code - it seems you build a full extent of the blocks > in the file, filling holes as you go along. I initally did that as well, > but that is to slow to be usable in real life. > > You also don't support sparse files, falling back to normal fs > read/write paths. Supporting

Re: [PATCH][RFC] fast file mapping for loop

2008-01-11 Thread Mikulas Patocka
So I looked at the code - it seems you build a full extent of the blocks in the file, filling holes as you go along. I initally did that as well, but that is to slow to be usable in real life. You also don't support sparse files, falling back to normal fs read/write paths. Supporting sparse

Re: [PATCH][RFC] fast file mapping for loop

2008-01-11 Thread Daniel Phillips
Hi Jens, This looks really useful. On Wednesday 09 January 2008 00:52, Jens Axboe wrote: Disadvantages: - The file block mappings must not change while loop is using the file. This means that we have to ensure exclusive access to the file and this is the bit that is currently missing in the

Re: [PATCH][RFC] fast file mapping for loop

2008-01-11 Thread Jens Axboe
On Fri, Jan 11 2008, Daniel Phillips wrote: Hi Jens, This looks really useful. On Wednesday 09 January 2008 00:52, Jens Axboe wrote: Disadvantages: - The file block mappings must not change while loop is using the file. This means that we have to ensure exclusive access to the file

Re: [PATCH][RFC] fast file mapping for loop

2008-01-11 Thread Chris Mason
On Fri, 11 Jan 2008 10:01:18 +1100 Neil Brown [EMAIL PROTECTED] wrote: On Thursday January 10, [EMAIL PROTECTED] wrote: On Thu, Jan 10 2008, Chris Mason wrote: On Thu, 10 Jan 2008 09:31:31 +0100 Jens Axboe [EMAIL PROTECTED] wrote: On Wed, Jan 09 2008, Alasdair G Kergon wrote:

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Fri, Jan 11 2008, Mikulas Patocka wrote: > > So I looked at the code - it seems you build a full extent of the blocks > > in the file, filling holes as you go along. I initally did that as well, > > but that is to slow to be usable in real life. > > > > You also don't support sparse files,

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Bill Davidsen
Jens Axboe wrote: Hi, loop.c currently uses the page cache interface to do IO to file backed devices. This works reasonably well for simple things, like mapping an iso9660 file for direct mount and other read-only workloads. Writing is somewhat problematic, as anyone who has really used this

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Neil Brown
On Thursday January 10, [EMAIL PROTECTED] wrote: > On Thu, Jan 10 2008, Chris Mason wrote: > > On Thu, 10 Jan 2008 09:31:31 +0100 > > Jens Axboe <[EMAIL PROTECTED]> wrote: > > > > > On Wed, Jan 09 2008, Alasdair G Kergon wrote: > > > > Here's the latest version of dm-loop, for comparison. > > > >

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Chris Mason
On Thu, 10 Jan 2008 14:03:24 +0100 Jens Axboe <[EMAIL PROTECTED]> wrote: > On Thu, Jan 10 2008, Chris Mason wrote: > > On Thu, 10 Jan 2008 08:54:59 + > > Christoph Hellwig <[EMAIL PROTECTED]> wrote: > > > > > On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: > > > > > IMHO this

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Chris Mason wrote: > On Thu, 10 Jan 2008 08:54:59 + > Christoph Hellwig <[EMAIL PROTECTED]> wrote: > > > On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: > > > > IMHO this shouldn't be done in the loop driver anyway. > > > > Filesystems have their own

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Chris Mason wrote: > On Thu, 10 Jan 2008 09:31:31 +0100 > Jens Axboe <[EMAIL PROTECTED]> wrote: > > > On Wed, Jan 09 2008, Alasdair G Kergon wrote: > > > Here's the latest version of dm-loop, for comparison. > > > > > > To try it out, > > > ln -s dmsetup dmlosetup > > >

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Chris Mason
On Thu, 10 Jan 2008 08:54:59 + Christoph Hellwig <[EMAIL PROTECTED]> wrote: > On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: > > > IMHO this shouldn't be done in the loop driver anyway. > > > Filesystems have their own effricient extent lookup trees (well, > > > at least xfs and

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Chris Mason
On Thu, 10 Jan 2008 09:31:31 +0100 Jens Axboe <[EMAIL PROTECTED]> wrote: > On Wed, Jan 09 2008, Alasdair G Kergon wrote: > > Here's the latest version of dm-loop, for comparison. > > > > To try it out, > > ln -s dmsetup dmlosetup > > and supply similar basic parameters to losetup. > > (using

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Peter Zijlstra
On Thu, 2008-01-10 at 11:02 +0100, Jens Axboe wrote: > On Thu, Jan 10 2008, Peter Zijlstra wrote: > > > > On Thu, 2008-01-10 at 10:49 +0100, Jens Axboe wrote: > > > On Thu, Jan 10 2008, Peter Zijlstra wrote: > > > > > > > > On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: > > > > >

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Peter Zijlstra wrote: > > On Thu, 2008-01-10 at 10:49 +0100, Jens Axboe wrote: > > On Thu, Jan 10 2008, Peter Zijlstra wrote: > > > > > > On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: > > > > > > > Peter, any chance you could chime in here? > > > > > > I have

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Peter Zijlstra
On Thu, 2008-01-10 at 10:49 +0100, Jens Axboe wrote: > On Thu, Jan 10 2008, Peter Zijlstra wrote: > > > > On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: > > > > > Peter, any chance you could chime in here? > > > > I have this patch to add swap_out/_in methods. I expect we can

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Peter Zijlstra wrote: > > On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: > > > Peter, any chance you could chime in here? > > I have this patch to add swap_out/_in methods. I expect we can loosen > the requirement for swapcache pages and change the name a

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Peter Zijlstra
On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: > Peter, any chance you could chime in here? I have this patch to add swap_out/_in methods. I expect we can loosen the requirement for swapcache pages and change the name a little. previously posted here:

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Christoph Hellwig wrote: > On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: > > > IMHO this shouldn't be done in the loop driver anyway. Filesystems have > > > their own effricient extent lookup trees (well, at least xfs and btrfs > > > do), and we should leverage

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Christoph Hellwig
On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: > > IMHO this shouldn't be done in the loop driver anyway. Filesystems have > > their own effricient extent lookup trees (well, at least xfs and btrfs > > do), and we should leverage that instead of reinventing it. > > Completely agree,

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Christoph Hellwig wrote: > > > loop maintains a prio tree of known > > > extents in the file (populated lazily on demand, as needed). > > > > Just a quick question (I haven't looked closely at the code): how come > > you are using a prio tree for extents? I don't think they

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Wed, Jan 09 2008, Andi Kleen wrote: > Jens Axboe <[EMAIL PROTECTED]> writes: > > > > So how does it work? Instead of punting IO to a thread and passing it > > through the page cache, we instead attempt to send the IO directly to the > > Great -- something like this was needed for a long time.

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Jens Axboe wrote: > On Wed, Jan 09 2008, Alasdair G Kergon wrote: > > Here's the latest version of dm-loop, for comparison. > > > > To try it out, > > ln -s dmsetup dmlosetup > > and supply similar basic parameters to losetup. > > (using dmsetup version 1.02.11 or higher)

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Christoph Hellwig
On Thu, Jan 10, 2008 at 12:42:25PM +1100, Nick Piggin wrote: > > So how does it work? Instead of punting IO to a thread and passing it > > through the page cache, we instead attempt to send the IO directly to the > > filesystem block that it maps to. > > You told Christoph that just using

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Nick Piggin wrote: > On Wednesday 09 January 2008 19:52, Jens Axboe wrote: > > > So how does it work? Instead of punting IO to a thread and passing it > > through the page cache, we instead attempt to send the IO directly to the > > filesystem block that it maps to. > > You

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Wed, Jan 09 2008, Alasdair G Kergon wrote: > Here's the latest version of dm-loop, for comparison. > > To try it out, > ln -s dmsetup dmlosetup > and supply similar basic parameters to losetup. > (using dmsetup version 1.02.11 or higher) Why oh why does dm always insist to reinvent

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Wed, Jan 09 2008, Alasdair G Kergon wrote: Here's the latest version of dm-loop, for comparison. To try it out, ln -s dmsetup dmlosetup and supply similar basic parameters to losetup. (using dmsetup version 1.02.11 or higher) Why oh why does dm always insist to reinvent everything?

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Nick Piggin wrote: On Wednesday 09 January 2008 19:52, Jens Axboe wrote: So how does it work? Instead of punting IO to a thread and passing it through the page cache, we instead attempt to send the IO directly to the filesystem block that it maps to. You told

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Christoph Hellwig
On Thu, Jan 10, 2008 at 12:42:25PM +1100, Nick Piggin wrote: So how does it work? Instead of punting IO to a thread and passing it through the page cache, we instead attempt to send the IO directly to the filesystem block that it maps to. You told Christoph that just using direct-IO from

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Jens Axboe wrote: On Wed, Jan 09 2008, Alasdair G Kergon wrote: Here's the latest version of dm-loop, for comparison. To try it out, ln -s dmsetup dmlosetup and supply similar basic parameters to losetup. (using dmsetup version 1.02.11 or higher) Why oh why

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Wed, Jan 09 2008, Andi Kleen wrote: Jens Axboe [EMAIL PROTECTED] writes: So how does it work? Instead of punting IO to a thread and passing it through the page cache, we instead attempt to send the IO directly to the Great -- something like this was needed for a long time. - The

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Christoph Hellwig wrote: loop maintains a prio tree of known extents in the file (populated lazily on demand, as needed). Just a quick question (I haven't looked closely at the code): how come you are using a prio tree for extents? I don't think they could be

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Christoph Hellwig
On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: IMHO this shouldn't be done in the loop driver anyway. Filesystems have their own effricient extent lookup trees (well, at least xfs and btrfs do), and we should leverage that instead of reinventing it. Completely agree, it's

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Christoph Hellwig wrote: On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: IMHO this shouldn't be done in the loop driver anyway. Filesystems have their own effricient extent lookup trees (well, at least xfs and btrfs do), and we should leverage that

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Peter Zijlstra
On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: Peter, any chance you could chime in here? I have this patch to add swap_out/_in methods. I expect we can loosen the requirement for swapcache pages and change the name a little. previously posted here:

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Peter Zijlstra wrote: On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: Peter, any chance you could chime in here? I have this patch to add swap_out/_in methods. I expect we can loosen the requirement for swapcache pages and change the name a little.

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Peter Zijlstra
On Thu, 2008-01-10 at 10:49 +0100, Jens Axboe wrote: On Thu, Jan 10 2008, Peter Zijlstra wrote: On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: Peter, any chance you could chime in here? I have this patch to add swap_out/_in methods. I expect we can loosen the

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Peter Zijlstra wrote: On Thu, 2008-01-10 at 10:49 +0100, Jens Axboe wrote: On Thu, Jan 10 2008, Peter Zijlstra wrote: On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: Peter, any chance you could chime in here? I have this patch to add

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Peter Zijlstra
On Thu, 2008-01-10 at 11:02 +0100, Jens Axboe wrote: On Thu, Jan 10 2008, Peter Zijlstra wrote: On Thu, 2008-01-10 at 10:49 +0100, Jens Axboe wrote: On Thu, Jan 10 2008, Peter Zijlstra wrote: On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: Peter, any chance

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Chris Mason wrote: On Thu, 10 Jan 2008 09:31:31 +0100 Jens Axboe [EMAIL PROTECTED] wrote: On Wed, Jan 09 2008, Alasdair G Kergon wrote: Here's the latest version of dm-loop, for comparison. To try it out, ln -s dmsetup dmlosetup and supply similar

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Chris Mason
On Thu, 10 Jan 2008 08:54:59 + Christoph Hellwig [EMAIL PROTECTED] wrote: On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: IMHO this shouldn't be done in the loop driver anyway. Filesystems have their own effricient extent lookup trees (well, at least xfs and btrfs do),

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Thu, Jan 10 2008, Chris Mason wrote: On Thu, 10 Jan 2008 08:54:59 + Christoph Hellwig [EMAIL PROTECTED] wrote: On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: IMHO this shouldn't be done in the loop driver anyway. Filesystems have their own effricient extent lookup

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Chris Mason
On Thu, 10 Jan 2008 14:03:24 +0100 Jens Axboe [EMAIL PROTECTED] wrote: On Thu, Jan 10 2008, Chris Mason wrote: On Thu, 10 Jan 2008 08:54:59 + Christoph Hellwig [EMAIL PROTECTED] wrote: On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: IMHO this shouldn't be done in

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Neil Brown
On Thursday January 10, [EMAIL PROTECTED] wrote: On Thu, Jan 10 2008, Chris Mason wrote: On Thu, 10 Jan 2008 09:31:31 +0100 Jens Axboe [EMAIL PROTECTED] wrote: On Wed, Jan 09 2008, Alasdair G Kergon wrote: Here's the latest version of dm-loop, for comparison. To try it out,

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Bill Davidsen
Jens Axboe wrote: Hi, loop.c currently uses the page cache interface to do IO to file backed devices. This works reasonably well for simple things, like mapping an iso9660 file for direct mount and other read-only workloads. Writing is somewhat problematic, as anyone who has really used this

Re: [PATCH][RFC] fast file mapping for loop

2008-01-10 Thread Jens Axboe
On Fri, Jan 11 2008, Mikulas Patocka wrote: So I looked at the code - it seems you build a full extent of the blocks in the file, filling holes as you go along. I initally did that as well, but that is to slow to be usable in real life. You also don't support sparse files, falling back

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Nick Piggin
On Wednesday 09 January 2008 19:52, Jens Axboe wrote: > So how does it work? Instead of punting IO to a thread and passing it > through the page cache, we instead attempt to send the IO directly to the > filesystem block that it maps to. You told Christoph that just using direct-IO from kernel

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Alasdair G Kergon
On Thu, Jan 10, 2008 at 12:43:19AM +0100, [EMAIL PROTECTED] wrote: > oh, nice to see that this is still alive. > at least i got no crashes and was able to mount and acess more than 300 > iso-images with that. > were there fixes/chances since then? Little has changed for some time - mostly code

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread devzero
oh, nice to see that this is still alive. i tried this around half a year ago because i needed more than 256 loop devices and iirc, this was working quite fine. at least i got no crashes and was able to mount and acess more than 300 iso-images with that. shortly after, loop device was extended

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Alasdair G Kergon
Here's the latest version of dm-loop, for comparison. To try it out, ln -s dmsetup dmlosetup and supply similar basic parameters to losetup. (using dmsetup version 1.02.11 or higher) Alasdair From: Bryn Reeves <[EMAIL PROTECTED]> This implements a loopback target for device mapper allowing

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Andi Kleen
Jens Axboe <[EMAIL PROTECTED]> writes: > > So how does it work? Instead of punting IO to a thread and passing it > through the page cache, we instead attempt to send the IO directly to the Great -- something like this was needed for a long time. > - The file block mappings must not change while

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Chris Mason
On Wed, 9 Jan 2008 10:43:21 +0100 Jens Axboe <[EMAIL PROTECTED]> wrote: > On Wed, Jan 09 2008, Christoph Hellwig wrote: > > On Wed, Jan 09, 2008 at 09:52:32AM +0100, Jens Axboe wrote: > > > - The file block mappings must not change while loop is using the > > > file. This means that we have to

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Jens Axboe
On Wed, Jan 09 2008, Christoph Hellwig wrote: > On Wed, Jan 09, 2008 at 09:52:32AM +0100, Jens Axboe wrote: > > - The file block mappings must not change while loop is using the file. > > This means that we have to ensure exclusive access to the file and > > this is the bit that is currently

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Christoph Hellwig
On Wed, Jan 09, 2008 at 09:52:32AM +0100, Jens Axboe wrote: > - The file block mappings must not change while loop is using the file. > This means that we have to ensure exclusive access to the file and > this is the bit that is currently missing in the implementation. It > would be nice if

[PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Jens Axboe
Hi, loop.c currently uses the page cache interface to do IO to file backed devices. This works reasonably well for simple things, like mapping an iso9660 file for direct mount and other read-only workloads. Writing is somewhat problematic, as anyone who has really used this feature can attest to

[PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Jens Axboe
Hi, loop.c currently uses the page cache interface to do IO to file backed devices. This works reasonably well for simple things, like mapping an iso9660 file for direct mount and other read-only workloads. Writing is somewhat problematic, as anyone who has really used this feature can attest to

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Jens Axboe
On Wed, Jan 09 2008, Christoph Hellwig wrote: On Wed, Jan 09, 2008 at 09:52:32AM +0100, Jens Axboe wrote: - The file block mappings must not change while loop is using the file. This means that we have to ensure exclusive access to the file and this is the bit that is currently missing

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Christoph Hellwig
On Wed, Jan 09, 2008 at 09:52:32AM +0100, Jens Axboe wrote: - The file block mappings must not change while loop is using the file. This means that we have to ensure exclusive access to the file and this is the bit that is currently missing in the implementation. It would be nice if we

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Chris Mason
On Wed, 9 Jan 2008 10:43:21 +0100 Jens Axboe [EMAIL PROTECTED] wrote: On Wed, Jan 09 2008, Christoph Hellwig wrote: On Wed, Jan 09, 2008 at 09:52:32AM +0100, Jens Axboe wrote: - The file block mappings must not change while loop is using the file. This means that we have to ensure

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Andi Kleen
Jens Axboe [EMAIL PROTECTED] writes: So how does it work? Instead of punting IO to a thread and passing it through the page cache, we instead attempt to send the IO directly to the Great -- something like this was needed for a long time. - The file block mappings must not change while loop

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Alasdair G Kergon
Here's the latest version of dm-loop, for comparison. To try it out, ln -s dmsetup dmlosetup and supply similar basic parameters to losetup. (using dmsetup version 1.02.11 or higher) Alasdair From: Bryn Reeves [EMAIL PROTECTED] This implements a loopback target for device mapper allowing a

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread devzero
oh, nice to see that this is still alive. i tried this around half a year ago because i needed more than 256 loop devices and iirc, this was working quite fine. at least i got no crashes and was able to mount and acess more than 300 iso-images with that. shortly after, loop device was extended

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Alasdair G Kergon
On Thu, Jan 10, 2008 at 12:43:19AM +0100, [EMAIL PROTECTED] wrote: oh, nice to see that this is still alive. at least i got no crashes and was able to mount and acess more than 300 iso-images with that. were there fixes/chances since then? Little has changed for some time - mostly code

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Nick Piggin
On Wednesday 09 January 2008 19:52, Jens Axboe wrote: So how does it work? Instead of punting IO to a thread and passing it through the page cache, we instead attempt to send the IO directly to the filesystem block that it maps to. You told Christoph that just using direct-IO from kernel