Re: [PATCH][RFC] fast file mapping for loop
Here's the latest version of dm-loop, for comparison.
To try it out,
ln -s dmsetup dmlosetup
and supply similar basic parameters to losetup.
(using dmsetup version 1.02.11 or higher)
Alasdair
From: Bryn Reeves [EMAIL PROTECTED]
This implements a loopback target for device mapper allowing a regular
file to be treated as a block device.
Signed-off-by: Bryn Reeves [EMAIL PROTECTED]
Signed-off-by: Alasdair G Kergon [EMAIL PROTECTED]
---
drivers/md/Kconfig |9
drivers/md/Makefile |1
drivers/md/dm-loop.c | 1036 +++
3 files changed, 1046 insertions(+)
Index: linux-2.6.24-rc6/drivers/md/Kconfig
===
--- linux-2.6.24-rc6.orig/drivers/md/Kconfig2008-01-07 18:32:05.0
+
+++ linux-2.6.24-rc6/drivers/md/Kconfig 2008-01-07 18:33:12.0 +
@@ -229,6 +229,15 @@ config DM_CRYPT
If unsure, say N.
+config DM_LOOP
+ tristate Loop target (EXPERIMENTAL)
+ depends on BLK_DEV_DM EXPERIMENTAL
+ ---help---
+ This device-mapper target allows you to treat a regular file as
+ a block device.
+
+ If unsure, say N.
+
config DM_SNAPSHOT
tristate Snapshot target (EXPERIMENTAL)
depends on BLK_DEV_DM EXPERIMENTAL
Index: linux-2.6.24-rc6/drivers/md/Makefile
===
--- linux-2.6.24-rc6.orig/drivers/md/Makefile 2008-01-07 18:32:05.0
+
+++ linux-2.6.24-rc6/drivers/md/Makefile2008-01-07 18:33:12.0
+
@@ -34,6 +34,7 @@ obj-$(CONFIG_BLK_DEV_MD) += md-mod.o
obj-$(CONFIG_BLK_DEV_DM) += dm-mod.o
obj-$(CONFIG_DM_CRYPT) += dm-crypt.o
obj-$(CONFIG_DM_DELAY) += dm-delay.o
+obj-$(CONFIG_DM_LOOP) += dm-loop.o
obj-$(CONFIG_DM_MULTIPATH) += dm-multipath.o dm-round-robin.o
obj-$(CONFIG_DM_MULTIPATH_EMC) += dm-emc.o
obj-$(CONFIG_DM_MULTIPATH_HP) += dm-hp-sw.o
Index: linux-2.6.24-rc6/drivers/md/dm-loop.c
===
--- /dev/null 1970-01-01 00:00:00.0 +
+++ linux-2.6.24-rc6/drivers/md/dm-loop.c 2008-01-07 18:33:12.0
+
@@ -0,0 +1,1036 @@
+/*
+ * Copyright (C) 2006-2008 Red Hat, Inc. All rights reserved.
+ *
+ * This file is part of device-mapper.
+ *
+ * Extent mapping implementation heavily influenced by mm/swapfile.c
+ * Bryn Reeves [EMAIL PROTECTED]
+ *
+ * File mapping and block lookup algorithms support by
+ * Heinz Mauelshagen [EMAIL PROTECTED].
+ *
+ * This file is released under the GPL.
+ */
+
+#include linux/kernel.h
+#include linux/slab.h
+#include linux/fs.h
+#include linux/module.h
+#include linux/vmalloc.h
+#include linux/syscalls.h
+#include linux/workqueue.h
+#include linux/file.h
+#include linux/bio.h
+
+#include dm.h
+#include dm-bio-list.h
+
+#define DM_LOOP_DAEMON kloopd
+#define DM_MSG_PREFIX loop
+
+enum flags { DM_LOOP_BMAP, DM_LOOP_FSIO };
+
+/*
+ * Loop context
+ **/
+
+struct loop_c {
+ unsigned long flags;
+
+ /* Backing store */
+
+ struct file *filp;
+ char *path;
+ loff_t offset;
+ struct block_device *bdev;
+ unsigned blkbits; /* file system block size shift bits */
+
+ loff_t size;/* size of entire file in bytes */
+ loff_t blocks; /* blocks allocated to loop file */
+ sector_t mapped_sectors;/* size of mapped area in sectors */
+
+ int (*map_fn)(struct dm_target *, struct bio *);
+ void *map_data;
+};
+
+/*
+ * Block map extent
+ */
+struct dm_loop_extent {
+ sector_t start; /* start sector in mapped device */
+ sector_t to;/* start sector on target device */
+ sector_t len; /* length in sectors */
+};
+
+/*
+ * Temporary extent list
+ */
+struct extent_list {
+ struct dm_loop_extent *extent;
+ struct list_head list;
+};
+
+static struct kmem_cache *dm_loop_extent_cache;
+
+/*
+ * Block map private context
+ */
+struct block_map_c {
+ int nr_extents; /* number of extents in map */
+ struct dm_loop_extent **map;/* linear map of extent pointers */
+ struct dm_loop_extent **mru;/* pointer to mru entry */
+ spinlock_t mru_lock;/* protects mru */
+};
+
+/*
+ * File map private context
+ */
+struct file_map_c {
+ spinlock_t lock;/* protects in */
+ struct bio_list in; /* new bios for processing */
+ struct bio_list work; /* bios queued for processing */
+ struct workqueue_struct *wq;/* workqueue */
+ struct work_struct ws; /* loop work */
+ struct loop_c *loop;/* for filp
Re: [PATCH][RFC] fast file mapping for loop
On Thu, Jan 10, 2008 at 12:43:19AM +0100, [EMAIL PROTECTED] wrote: oh, nice to see that this is still alive. at least i got no crashes and was able to mount and acess more than 300 iso-images with that. were there fixes/chances since then? Little has changed for some time - mostly code cleanups and the occasional bug fix. It's time to give it wider exposure, I think, and we'll find out how well it holds up. Alasdair -- [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [1/1] Block device throttling [Re: Distributed storage.]
On Thu, Aug 30, 2007 at 04:20:35PM -0700, Daniel Phillips wrote: Resubmitting a bio or submitting a dependent bio from inside a block driver does not need to be throttled because all resources required to guarantee completion must have been obtained _before_ the bio was allowed to proceed into the block layer. I'm toying with the idea of keeping track of the maximum device stack depth for each stacked device, and only permitting it to increase in controlled circumstances. Alasdair -- [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] block_device_operations prototype changes
On Mon, Aug 27, 2007 at 11:30:53AM +0100, Al Viro wrote: 3) -ioctl(). What a mess... Yup. See also: Subject: [PATCH] dm: support ioctls on mapped devices: fix with fake file http://uwsg.indiana.edu/hypermail/linux/kernel/0606.2/2979.html and related threads. First of all, we have 3 methods with different calling conventions: -ioctl(inode, file, cmd, arg) -unlocked_ioctl(inode, file, cmd, arg) When I last looked it was: long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long); with the lack of inode forcing dm to use -ioctl (because file can be NULL when only the block device is known) and immediately drop the pointless-for-us lock! Alasdair -- [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [dm-devel] Re: [RFD] BIO_RW_BARRIER - what it means for devices, filesystems, and dm/md.
On Wed, May 30, 2007 at 11:12:37AM +0200, Stefan Bader wrote: it might be better to indicate -EOPNOTSUPP right from device-mapper. Indeed we should. For support, on receipt of a barrier, dm core should send a zero-length barrier to all active underlying paths, and delay mapping any further I/O. Alasdair -- [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [dm-devel] Re: [RFD] BIO_RW_BARRIER - what it means for devices, filesystems, and dm/md.
On Thu, May 31, 2007 at 10:46:04AM +1000, Neil Brown wrote: What if the truth changes (as can happen with md or dm)? You get notified in endio() that the barrier had to be emulated? Alasdair -- [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [dm-devel] Re: [RFD] BIO_RW_BARRIER - what it means for devices, filesystems, and dm/md.
On Thu, May 31, 2007 at 10:46:04AM +1000, Neil Brown wrote: If a filesystem cares, it could 'ask' as suggested above. What would be a good interface for asking? XFS already tests: bd_disk-queue-ordered == QUEUE_ORDERED_NONE Alasdair -- [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [dm-devel] Re: [RFD] BIO_RW_BARRIER - what it means for devices, filesystems, and dm/md.
On Tue, May 29, 2007 at 11:25:42AM +0200, Stefan Bader wrote: doing a sort of suspend, issuing the barrier request, calling flush to all mapped devices and then wait for in-flight I/O to go to zero? Something like that is needed for some dm targets to support barriers. (We needn't always wait for *all* in-flight I/O.) When faced with -EOPNOTSUP, do all callers fall back to a sync in the places a barrier would have been used, or are there any more sophisticated strategies attempting to optimise code without barriers? I am not a hundred percent sure about that but I think that just passing the barrier flag on to mapped devices might in some (maybe they are rare) cases cause a layer above to think all data is on-disk while this isn't necessarily true (see my previous post). What do you think? An efficient I/O barrier implementation would not normally involve flushing AFAIK: dm surely wouldn't cause a higher layer to assume stronger semantics than are provided. Alasdair -- [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [dm-devel] Re: [RFD] BIO_RW_BARRIER - what it means for devices, filesystems, and dm/md.
On Mon, May 28, 2007 at 11:30:32AM +1000, Neil Brown wrote: 1/ A BIO_RW_BARRIER request should never fail with -EOPNOTSUP. The device-mapper position has always been that we require a zero-length BIO_RW_BARRIER (i.e. containing no data to read or write - or emulated, possibly device-specific) before we can provide full barrier support. (Consider multiple active paths - each must see barrier.) Until every device supports barriers -EOPNOTSUP support is required. (Consider reconfiguration of stacks of devices - barrier support is a dynamic block device property that can switch between available and unavailable at any time.) Alasdair -- [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
