Re: [Cluster-devel] [PATCH 08/27] iomap: add the new iomap_iter model
On Mon, Jul 19, 2021 at 09:56:00AM -0700, Darrick J. Wong wrote: > Linus previously complained to me about filesystem code (especially > iomap since it was "newer") (ab)using loff_t variables to store the > lengths of byte ranges. It was "loff_t length;" (or so willy > recollects) that tripped him up. > > ISTR he also said we should use size_t for all lengths because nobody > should do operations larger than ~2G, but I reject that because iomap > has users that iterate large ranges of data without generating any IO > (e.g. fiemap, seek, swapfile activation). > > So... rather than confusing things even more by mixing u64 and ssize_t > for lengths, can we introduce a new 64-bit length typedef for iomap? > Last summer, Dave suggested[1] something like: > > typedef long long lsize_t; > > That would enable cleanup of all the count/size/length parameters in > fs/remap_range.c and fs/xfs/xfs_reflink.c to use the new 64-bit length > type, since those operations have never been limited to 32-bit sizes. I'd rather avoid playing guinea pig for a somewhat odd new type. For now I've switched it to the loff_t as that matches the rest of iomap. If we switch away either to a new type or s64/u64 we should probably do it as a big sweep.
Re: [Cluster-devel] [PATCH 08/27] iomap: add the new iomap_iter model
On Tue, Jul 20, 2021 at 07:48:38AM +1000, Dave Chinner wrote: > We should avoid namespace conflicts where function names shadow > object types. iomap_iterate() is fine as the function name - there's > no need for abbreviation here because it's not an overly long name. > This will makes it clearly different to the struct iomap_iter that > is passed to it and it will also make grep, cscope and other > code searching tools much more precise... Well, there isn't really a conflict by definition. I actually like this choice of names (stolen from the original patch from willy) as it clearly indicates they go together. But I'm happy to collect a few more opinions.
Re: [Cluster-devel] [PATCH 08/27] iomap: add the new iomap_iter model
On Mon, Jul 19, 2021 at 12:35:01PM +0200, Christoph Hellwig wrote:
> The iomap_iter struct provides a convenient way to package up and
> maintain all the arguments to the various mapping and operation
> functions. It is operated on using the iomap_iter() function that
> is called in loop until the whole range has been processed. Compared
> to the existing iomap_apply() function this avoid an indirect call
> for each iteration.
>
> For now iomap_iter() calls back into the existing ->iomap_begin and
> ->iomap_end methods, but in the future this could be further optimized
> to avoid indirect calls entirely.
>
> Based on an earlier patch from Matthew Wilcox .
>
> Signed-off-by: Christoph Hellwig
> ---
> fs/iomap/Makefile | 1 +
> fs/iomap/iter.c | 74 +++
> fs/iomap/trace.h | 37 +-
> include/linux/iomap.h | 56
> 4 files changed, 167 insertions(+), 1 deletion(-)
> create mode 100644 fs/iomap/iter.c
>
> diff --git a/fs/iomap/Makefile b/fs/iomap/Makefile
> index eef2722d93a183..85034deb5a2f19 100644
> --- a/fs/iomap/Makefile
> +++ b/fs/iomap/Makefile
> @@ -10,6 +10,7 @@ obj-$(CONFIG_FS_IOMAP) += iomap.o
>
> iomap-y += trace.o \
> apply.o \
> +iter.o \
Can we break this cycle of creating new files and removing old files
when changing the iomap core code? It breaks the ability to troll
git history easily through git blame and other techniques that are
file based.
If we are going to create a new file, then the core iomap code that
every thing depends on should just be in a neutrally names file such
as "iomap.c" so that we don't need to play these games in future.
> +/**
> + * iomap_iter - iterate over a ranges in a file
> + * @iter: iteration structue
> + * @ops: iomap ops provided by the file system
> + *
> + * Iterate over file system provided contiguous ranges of blocks with the
> same
> + * state. Should be called in a loop that continues as long as this function
> + * returns a positive value. If 0 or a negative value is returned the caller
> + * should break out of the loop - a negative value is an error either from
> the
> + * file system or from the last iteration stored in @iter.copied.
> + */
> +int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)
> +{
We should avoid namespace conflicts where function names shadow
object types. iomap_iterate() is fine as the function name - there's
no need for abbreviation here because it's not an overly long name.
This will makes it clearly different to the struct iomap_iter that
is passed to it and it will also make grep, cscope and other
code searching tools much more precise...
Cheers,
Dave.
--
Dave Chinner
da...@fromorbit.com
Re: [Cluster-devel] [PATCH 08/27] iomap: add the new iomap_iter model
On Mon, Jul 19, 2021 at 12:35:01PM +0200, Christoph Hellwig wrote:
> The iomap_iter struct provides a convenient way to package up and
> maintain all the arguments to the various mapping and operation
> functions. It is operated on using the iomap_iter() function that
> is called in loop until the whole range has been processed. Compared
> to the existing iomap_apply() function this avoid an indirect call
> for each iteration.
>
> For now iomap_iter() calls back into the existing ->iomap_begin and
> ->iomap_end methods, but in the future this could be further optimized
> to avoid indirect calls entirely.
>
> Based on an earlier patch from Matthew Wilcox .
>
> Signed-off-by: Christoph Hellwig
> ---
> fs/iomap/Makefile | 1 +
> fs/iomap/iter.c | 74 +++
> fs/iomap/trace.h | 37 +-
> include/linux/iomap.h | 56
> 4 files changed, 167 insertions(+), 1 deletion(-)
> create mode 100644 fs/iomap/iter.c
>
> diff --git a/fs/iomap/Makefile b/fs/iomap/Makefile
> index eef2722d93a183..85034deb5a2f19 100644
> --- a/fs/iomap/Makefile
> +++ b/fs/iomap/Makefile
> @@ -10,6 +10,7 @@ obj-$(CONFIG_FS_IOMAP) += iomap.o
>
> iomap-y += trace.o \
> apply.o \
> +iter.o \
> buffered-io.o \
> direct-io.o \
> fiemap.o \
> diff --git a/fs/iomap/iter.c b/fs/iomap/iter.c
> new file mode 100644
> index 00..b21e2489700b7c
> --- /dev/null
> +++ b/fs/iomap/iter.c
> @@ -0,0 +1,74 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 Christoph Hellwig.
> + */
> +#include
> +#include
> +#include "trace.h"
> +
> +static inline int iomap_iter_advance(struct iomap_iter *iter)
> +{
> + /* handle the previous iteration (if any) */
> + if (iter->iomap.length) {
> + if (iter->processed <= 0)
> + return iter->processed;
Hmm, converting ssize_t to int here... I suppose that's fine since we're
merely returning "the usual negative errno code", but read on.
> + WARN_ON_ONCE(iter->processed > iomap_length(iter));
> + iter->pos += iter->processed;
> + iter->len -= iter->processed;
> + if (!iter->len)
> + return 0;
> + }
> +
> + /* clear the state for the next iteration */
> + iter->processed = 0;
> + memset(>iomap, 0, sizeof(iter->iomap));
> + memset(>srcmap, 0, sizeof(iter->srcmap));
> + return 1;
> +}
> +
> +static inline void iomap_iter_done(struct iomap_iter *iter)
> +{
> + WARN_ON_ONCE(iter->iomap.offset > iter->pos);
> + WARN_ON_ONCE(iter->iomap.length == 0);
> + WARN_ON_ONCE(iter->iomap.offset + iter->iomap.length <= iter->pos);
> +
> + trace_iomap_iter_dstmap(iter->inode, >iomap);
> + if (iter->srcmap.type != IOMAP_HOLE)
> + trace_iomap_iter_srcmap(iter->inode, >srcmap);
> +}
> +
> +/**
> + * iomap_iter - iterate over a ranges in a file
> + * @iter: iteration structue
> + * @ops: iomap ops provided by the file system
> + *
> + * Iterate over file system provided contiguous ranges of blocks with the
> same
> + * state. Should be called in a loop that continues as long as this function
> + * returns a positive value. If 0 or a negative value is returned the caller
> + * should break out of the loop - a negative value is an error either from
> the
> + * file system or from the last iteration stored in @iter.copied.
> + */
> +int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)
> +{
> + int ret;
> +
> + if (iter->iomap.length && ops->iomap_end) {
> + ret = ops->iomap_end(iter->inode, iter->pos, iomap_length(iter),
> + iter->processed > 0 ? iter->processed : 0,
> + iter->flags, >iomap);
> + if (ret < 0 && !iter->processed)
> + return ret;
> + }
> +
> + trace_iomap_iter(iter, ops, _RET_IP_);
> + ret = iomap_iter_advance(iter);
> + if (ret <= 0)
> + return ret;
> +
> + ret = ops->iomap_begin(iter->inode, iter->pos, iter->len, iter->flags,
> +>iomap, >srcmap);
> + if (ret < 0)
> + return ret;
> + iomap_iter_done(iter);
> + return 1;
> +}
> diff --git a/include/linux/iomap.h b/include/linux/iomap.h
> index f9c36df6a3061b..a9f3f736017989 100644
> --- a/include/linux/iomap.h
> +++ b/include/linux/iomap.h
> @@ -143,6 +143,62 @@ struct iomap_ops {
> ssize_t written, unsigned flags, struct iomap *iomap);
> };
>
> +/**
> + * struct iomap_iter - Iterate through a range of a file
> + * @inode: Set at the start of the iteration and should not change.
> + * @pos: The current file position we are operating on. It is
[Cluster-devel] [PATCH 08/27] iomap: add the new iomap_iter model
The iomap_iter struct provides a convenient way to package up and
maintain all the arguments to the various mapping and operation
functions. It is operated on using the iomap_iter() function that
is called in loop until the whole range has been processed. Compared
to the existing iomap_apply() function this avoid an indirect call
for each iteration.
For now iomap_iter() calls back into the existing ->iomap_begin and
->iomap_end methods, but in the future this could be further optimized
to avoid indirect calls entirely.
Based on an earlier patch from Matthew Wilcox .
Signed-off-by: Christoph Hellwig
---
fs/iomap/Makefile | 1 +
fs/iomap/iter.c | 74 +++
fs/iomap/trace.h | 37 +-
include/linux/iomap.h | 56
4 files changed, 167 insertions(+), 1 deletion(-)
create mode 100644 fs/iomap/iter.c
diff --git a/fs/iomap/Makefile b/fs/iomap/Makefile
index eef2722d93a183..85034deb5a2f19 100644
--- a/fs/iomap/Makefile
+++ b/fs/iomap/Makefile
@@ -10,6 +10,7 @@ obj-$(CONFIG_FS_IOMAP)+= iomap.o
iomap-y+= trace.o \
apply.o \
+ iter.o \
buffered-io.o \
direct-io.o \
fiemap.o \
diff --git a/fs/iomap/iter.c b/fs/iomap/iter.c
new file mode 100644
index 00..b21e2489700b7c
--- /dev/null
+++ b/fs/iomap/iter.c
@@ -0,0 +1,74 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 Christoph Hellwig.
+ */
+#include
+#include
+#include "trace.h"
+
+static inline int iomap_iter_advance(struct iomap_iter *iter)
+{
+ /* handle the previous iteration (if any) */
+ if (iter->iomap.length) {
+ if (iter->processed <= 0)
+ return iter->processed;
+ WARN_ON_ONCE(iter->processed > iomap_length(iter));
+ iter->pos += iter->processed;
+ iter->len -= iter->processed;
+ if (!iter->len)
+ return 0;
+ }
+
+ /* clear the state for the next iteration */
+ iter->processed = 0;
+ memset(>iomap, 0, sizeof(iter->iomap));
+ memset(>srcmap, 0, sizeof(iter->srcmap));
+ return 1;
+}
+
+static inline void iomap_iter_done(struct iomap_iter *iter)
+{
+ WARN_ON_ONCE(iter->iomap.offset > iter->pos);
+ WARN_ON_ONCE(iter->iomap.length == 0);
+ WARN_ON_ONCE(iter->iomap.offset + iter->iomap.length <= iter->pos);
+
+ trace_iomap_iter_dstmap(iter->inode, >iomap);
+ if (iter->srcmap.type != IOMAP_HOLE)
+ trace_iomap_iter_srcmap(iter->inode, >srcmap);
+}
+
+/**
+ * iomap_iter - iterate over a ranges in a file
+ * @iter: iteration structue
+ * @ops: iomap ops provided by the file system
+ *
+ * Iterate over file system provided contiguous ranges of blocks with the same
+ * state. Should be called in a loop that continues as long as this function
+ * returns a positive value. If 0 or a negative value is returned the caller
+ * should break out of the loop - a negative value is an error either from the
+ * file system or from the last iteration stored in @iter.copied.
+ */
+int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)
+{
+ int ret;
+
+ if (iter->iomap.length && ops->iomap_end) {
+ ret = ops->iomap_end(iter->inode, iter->pos, iomap_length(iter),
+ iter->processed > 0 ? iter->processed : 0,
+ iter->flags, >iomap);
+ if (ret < 0 && !iter->processed)
+ return ret;
+ }
+
+ trace_iomap_iter(iter, ops, _RET_IP_);
+ ret = iomap_iter_advance(iter);
+ if (ret <= 0)
+ return ret;
+
+ ret = ops->iomap_begin(iter->inode, iter->pos, iter->len, iter->flags,
+ >iomap, >srcmap);
+ if (ret < 0)
+ return ret;
+ iomap_iter_done(iter);
+ return 1;
+}
diff --git a/fs/iomap/trace.h b/fs/iomap/trace.h
index e9cd5cc0d6ba40..1012d7af6b689b 100644
--- a/fs/iomap/trace.h
+++ b/fs/iomap/trace.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
- * Copyright (c) 2009-2019 Christoph Hellwig
+ * Copyright (c) 2009-2021 Christoph Hellwig
*
* NOTE: none of these tracepoints shall be considered a stable kernel ABI
* as they can change at any time.
@@ -140,6 +140,8 @@ DEFINE_EVENT(iomap_class, name, \
TP_ARGS(inode, iomap))
DEFINE_IOMAP_EVENT(iomap_apply_dstmap);
DEFINE_IOMAP_EVENT(iomap_apply_srcmap);
+DEFINE_IOMAP_EVENT(iomap_iter_dstmap);
+DEFINE_IOMAP_EVENT(iomap_iter_srcmap);
TRACE_EVENT(iomap_apply,
TP_PROTO(struct inode *inode, loff_t pos, loff_t length,
@@ -179,6 +181,39 @@ TRACE_EVENT(iomap_apply,
__entry->actor)
);
