Re: [PATCH v7 02/28] mm: Introduce struct folio
On Fri, Apr 16, 2021 at 04:55:16PM +0100, Matthew Wilcox wrote:
> On Fri, Apr 09, 2021 at 07:50:39PM +0100, Matthew Wilcox (Oracle) wrote:
> > A struct folio is a new abstraction to replace the venerable struct page.
> > A function which takes a struct folio argument declares that it will
> > operate on the entire (possibly compound) page, not just PAGE_SIZE bytes.
> > In return, the caller guarantees that the pointer it is passing does
> > not point to a tail page.
> > +++ b/include/linux/mm_types.h
> [...]
> > +static inline struct folio *page_folio(struct page *page)
> > +{
> > + unsigned long head = READ_ONCE(page->compound_head);
> > +
> > + if (unlikely(head & 1))
> > + return (struct folio *)(head - 1);
> > + return (struct folio *)page;
> > +}
>
> I'm looking at changing this for the next revision, and basing it on
> my recent patch to make compound_head() const-preserving:
>
> +#define page_folio(page) _Generic((page),\
> + const struct page *:(const struct folio *)_compound_head(page), \
> + struct page *: (struct folio *)_compound_head(page))
>
> I've also noticed an awkward pattern occurring that I think this makes
> less awkward:
>
> +/**
> + * folio_page - Return a page from a folio.
> + * @folio: The folio.
> + * @n: The page number to return.
> + *
> + * @n is relative to the start of the folio. It should be between
> + * 0 and folio_nr_pages(@folio) - 1, but this is not checked for.
> + */
> +#define folio_page(folio, n) nth_page(&(folio)->page, n)
>
> That lets me simplify folio_next():
>
> +static inline struct folio *folio_next(struct folio *folio)
> +{
> + return (struct folio *)folio_page(folio, folio_nr_pages(folio));
> +}
>
> (it occurs to me this should also be const-preserving, but it's not clear
> that's needed yet)
Are we risking that we would need to replace inline functions with macros
all the way down? Not sure const-preserving worth it.
--
Kirill A. Shutemov
Re: [PATCH v7 02/28] mm: Introduce struct folio
On Fri, Apr 09, 2021 at 07:50:39PM +0100, Matthew Wilcox (Oracle) wrote:
> A struct folio is a new abstraction to replace the venerable struct page.
> A function which takes a struct folio argument declares that it will
> operate on the entire (possibly compound) page, not just PAGE_SIZE bytes.
> In return, the caller guarantees that the pointer it is passing does
> not point to a tail page.
> +++ b/include/linux/mm_types.h
[...]
> +static inline struct folio *page_folio(struct page *page)
> +{
> + unsigned long head = READ_ONCE(page->compound_head);
> +
> + if (unlikely(head & 1))
> + return (struct folio *)(head - 1);
> + return (struct folio *)page;
> +}
I'm looking at changing this for the next revision, and basing it on
my recent patch to make compound_head() const-preserving:
+#define page_folio(page) _Generic((page),\
+ const struct page *:(const struct folio *)_compound_head(page), \
+ struct page *: (struct folio *)_compound_head(page))
I've also noticed an awkward pattern occurring that I think this makes
less awkward:
+/**
+ * folio_page - Return a page from a folio.
+ * @folio: The folio.
+ * @n: The page number to return.
+ *
+ * @n is relative to the start of the folio. It should be between
+ * 0 and folio_nr_pages(@folio) - 1, but this is not checked for.
+ */
+#define folio_page(folio, n) nth_page(&(folio)->page, n)
That lets me simplify folio_next():
+static inline struct folio *folio_next(struct folio *folio)
+{
+ return (struct folio *)folio_page(folio, folio_nr_pages(folio));
+}
(it occurs to me this should also be const-preserving, but it's not clear
that's needed yet)
Re: [PATCH v7 02/28] mm: Introduce struct folio
Hi "Matthew, Thank you for the patch! Yet something to improve: [auto build test ERROR on next-20210409] [also build test ERROR on v5.12-rc6] [cannot apply to linux/master linus/master hnaz-linux-mm/master v5.12-rc6 v5.12-rc5 v5.12-rc4] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch] url: https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Memory-Folios/20210410-031353 base:e99d8a8495175df8cb8b739f8cf9b0fc9d0cd3b5 config: mips-gpr_defconfig (attached as .config) compiler: mipsel-linux-gcc (GCC) 9.3.0 reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # https://github.com/0day-ci/linux/commit/5658a201516d2ed74a34c328e3b55f552d4861d8 git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Memory-Folios/20210410-031353 git checkout 5658a201516d2ed74a34c328e3b55f552d4861d8 # save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=mips If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot All errors (new ones prefixed by >>): In file included from include/linux/bits.h:22, from include/linux/bitops.h:6, from include/linux/log2.h:12, from include/asm-generic/div64.h:53, from arch/mips/include/asm/div64.h:12, from include/linux/math64.h:7, from include/linux/time.h:6, from include/linux/compat.h:10, from arch/mips/kernel/asm-offsets.c:12: >> include/linux/build_bug.h:78:41: error: static assertion failed: >> "offsetof(struct page, lru) == offsetof(struct folio, lru)" 78 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg) | ^~ include/linux/build_bug.h:77:34: note: in expansion of macro '__static_assert' 77 | #define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr) | ^~~ include/linux/mm_types.h:272:2: note: in expansion of macro 'static_assert' 272 | static_assert(offsetof(struct page, pg) == offsetof(struct folio, fl)) | ^ include/linux/mm_types.h:274:1: note: in expansion of macro 'FOLIO_MATCH' 274 | FOLIO_MATCH(lru, lru); | ^~~ >> include/linux/build_bug.h:78:41: error: static assertion failed: >> "offsetof(struct page, compound_head) == offsetof(struct folio, lru)" 78 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg) | ^~ include/linux/build_bug.h:77:34: note: in expansion of macro '__static_assert' 77 | #define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr) | ^~~ include/linux/mm_types.h:272:2: note: in expansion of macro 'static_assert' 272 | static_assert(offsetof(struct page, pg) == offsetof(struct folio, fl)) | ^ include/linux/mm_types.h:275:1: note: in expansion of macro 'FOLIO_MATCH' 275 | FOLIO_MATCH(compound_head, lru); | ^~~ >> include/linux/build_bug.h:78:41: error: static assertion failed: >> "offsetof(struct page, index) == offsetof(struct folio, index)" 78 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg) | ^~ include/linux/build_bug.h:77:34: note: in expansion of macro '__static_assert' 77 | #define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr) | ^~~ include/linux/mm_types.h:272:2: note: in expansion of macro 'static_assert' 272 | static_assert(offsetof(struct page, pg) == offsetof(struct folio, fl)) | ^ include/linux/mm_types.h:276:1: note: in expansion of macro 'FOLIO_MATCH' 276 | FOLIO_MATCH(index, index); | ^~~ >> include/linux/build_bug.h:78:41: error: static assertion failed: >> "offsetof(struct page, private) == offsetof(struct folio, private)" 78 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg) | ^~ include/linux/build_bug.h:77:34: note: in expansion of macro '__static_assert' 77 | #define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr) | ^~~ include/linux/mm_types.h:272:2: note: in expansion
Re: [PATCH v7 02/28] mm: Introduce struct folio
Hi "Matthew, Thank you for the patch! Yet something to improve: [auto build test ERROR on next-20210409] [also build test ERROR on v5.12-rc6] [cannot apply to linux/master linus/master hnaz-linux-mm/master v5.12-rc6 v5.12-rc5 v5.12-rc4] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch] url: https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Memory-Folios/20210410-031353 base:e99d8a8495175df8cb8b739f8cf9b0fc9d0cd3b5 config: powerpc-randconfig-r032-20210409 (attached as .config) compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project dd453a1389b6a7e6d9214b449d3c54981b1a89b6) reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # install powerpc cross compiling tool for clang build # apt-get install binutils-powerpc-linux-gnu # https://github.com/0day-ci/linux/commit/5658a201516d2ed74a34c328e3b55f552d4861d8 git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Memory-Folios/20210410-031353 git checkout 5658a201516d2ed74a34c328e3b55f552d4861d8 # save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=powerpc If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot All errors (new ones prefixed by >>): In file included from arch/powerpc/kernel/asm-offsets.c:14: In file included from include/linux/compat.h:17: In file included from include/linux/fs.h:15: In file included from include/linux/radix-tree.h:19: In file included from include/linux/xarray.h:14: In file included from include/linux/gfp.h:6: In file included from include/linux/mmzone.h:21: >> include/linux/mm_types.h:274:1: error: static_assert failed due to >> requirement '__builtin_offsetof(struct page, lru) == >> __builtin_offsetof(struct folio, lru)' "offsetof(struct page, lru) == >> offsetof(struct folio, lru)" FOLIO_MATCH(lru, lru); ^ include/linux/mm_types.h:272:2: note: expanded from macro 'FOLIO_MATCH' static_assert(offsetof(struct page, pg) == offsetof(struct folio, fl)) ^~ include/linux/build_bug.h:77:34: note: expanded from macro 'static_assert' #define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr) ^~~ include/linux/build_bug.h:78:41: note: expanded from macro '__static_assert' #define __static_assert(expr, msg, ...) _Static_assert(expr, msg) ^ In file included from arch/powerpc/kernel/asm-offsets.c:14: In file included from include/linux/compat.h:17: In file included from include/linux/fs.h:15: In file included from include/linux/radix-tree.h:19: In file included from include/linux/xarray.h:14: In file included from include/linux/gfp.h:6: In file included from include/linux/mmzone.h:21: >> include/linux/mm_types.h:275:1: error: static_assert failed due to >> requirement '__builtin_offsetof(struct page, compound_head) == >> __builtin_offsetof(struct folio, lru)' "offsetof(struct page, compound_head) >> == offsetof(struct folio, lru)" FOLIO_MATCH(compound_head, lru); ^~~ include/linux/mm_types.h:272:2: note: expanded from macro 'FOLIO_MATCH' static_assert(offsetof(struct page, pg) == offsetof(struct folio, fl)) ^~ include/linux/build_bug.h:77:34: note: expanded from macro 'static_assert' #define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr) ^~~ include/linux/build_bug.h:78:41: note: expanded from macro '__static_assert' #define __static_assert(expr, msg, ...) _Static_assert(expr, msg) ^ In file included from arch/powerpc/kernel/asm-offsets.c:14: In file included from include/linux/compat.h:17: In file included from include/linux/fs.h:15: In file included from include/linux/radix-tree.h:19: In file included from include/linux/xarray.h:14: In file included from include/linux/gfp.h:6: In file included from include/linux/mmzone.h:21: >> include/linux/mm_types.h:276:1: error: static_assert failed due to >> requirement '__builtin_offsetof(struct page, index) == >> __builtin_offsetof(struct folio, index)' "offsetof(struct page, index) == >> offsetof(struct folio, index)"
[PATCH v7 02/28] mm: Introduce struct folio
A struct folio is a new abstraction to replace the venerable struct page.
A function which takes a struct folio argument declares that it will
operate on the entire (possibly compound) page, not just PAGE_SIZE bytes.
In return, the caller guarantees that the pointer it is passing does
not point to a tail page.
Signed-off-by: Matthew Wilcox (Oracle)
Acked-by: Jeff Layton
---
include/linux/mm.h | 74 +
include/linux/mm_types.h | 80
2 files changed, 154 insertions(+)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 036f63a44a5c..4ece80aa8d05 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -950,6 +950,20 @@ static inline unsigned int compound_order(struct page
*page)
return page[1].compound_order;
}
+/**
+ * folio_order - The allocation order of a folio.
+ * @folio: The folio.
+ *
+ * A folio is composed of 2^order pages. See get_order() for the definition
+ * of order.
+ *
+ * Return: The order of the folio.
+ */
+static inline unsigned int folio_order(struct folio *folio)
+{
+ return compound_order(>page);
+}
+
static inline bool hpage_pincount_available(struct page *page)
{
/*
@@ -1595,6 +1609,65 @@ static inline void set_page_links(struct page *page,
enum zone_type zone,
#endif
}
+/**
+ * folio_nr_pages - The number of pages in the folio.
+ * @folio: The folio.
+ *
+ * Return: A number which is a power of two.
+ */
+static inline unsigned long folio_nr_pages(struct folio *folio)
+{
+ return compound_nr(>page);
+}
+
+/**
+ * folio_next - Move to the next physical folio.
+ * @folio: The folio we're currently operating on.
+ *
+ * If you have physically contiguous memory which may span more than
+ * one folio (eg a bio_vec), use this function to move from one
+ * folio to the next. Do not use it if the memory is only virtually
+ * contiguous as the folios are almost certainly not adjacent to each
+ * other. This is the folio equivalent to writing ``page++``.
+ *
+ * Context: We assume that the folios are refcounted and/or locked at a
+ * higher level and do not adjust the reference counts.
+ * Return: The next struct folio.
+ */
+static inline struct folio *folio_next(struct folio *folio)
+{
+ return (struct folio *)nth_page(>page, folio_nr_pages(folio));
+}
+
+/**
+ * folio_shift - The number of bits covered by this folio.
+ * @folio: The folio.
+ *
+ * A folio contains a number of bytes which is a power-of-two in size.
+ * This function tells you which power-of-two the folio is.
+ *
+ * Context: The caller should have a reference on the folio to prevent
+ * it from being split. It is not necessary for the folio to be locked.
+ * Return: The base-2 logarithm of the size of this folio.
+ */
+static inline unsigned int folio_shift(struct folio *folio)
+{
+ return PAGE_SHIFT + folio_order(folio);
+}
+
+/**
+ * folio_size - The number of bytes in a folio.
+ * @folio: The folio.
+ *
+ * Context: The caller should have a reference on the folio to prevent
+ * it from being split. It is not necessary for the folio to be locked.
+ * Return: The number of bytes in this folio.
+ */
+static inline size_t folio_size(struct folio *folio)
+{
+ return PAGE_SIZE << folio_order(folio);
+}
+
/*
* Some inline functions in vmstat.h depend on page_zone()
*/
@@ -1699,6 +1772,7 @@ extern void pagefault_out_of_memory(void);
#define offset_in_page(p) ((unsigned long)(p) & ~PAGE_MASK)
#define offset_in_thp(page, p) ((unsigned long)(p) & (thp_size(page) - 1))
+#define offset_in_folio(folio, p) ((unsigned long)(p) & (folio_size(folio) -
1))
/*
* Flags passed to show_mem() and show_free_areas() to suppress output in
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 6613b26a8894..d65050851037 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -224,6 +224,86 @@ struct page {
#endif
} _struct_page_alignment;
+/**
+ * struct folio - Represents a contiguous set of bytes.
+ * @flags: Identical to the page flags.
+ * @lru: Least Recently Used list; tracks how recently this folio was used.
+ * @mapping: The file this page belongs to, or refers to the anon_vma for
+ *anonymous pages.
+ * @index: Offset within the file, in units of pages. For anonymous pages,
+ *this is the index from the beginning of the mmap.
+ * @private: Filesystem per-folio data (see attach_folio_private()).
+ *Used for swp_entry_t if FolioSwapCache().
+ * @_mapcount: How many times this folio is mapped to userspace. Use
+ *folio_mapcount() to access it.
+ * @_refcount: Number of references to this folio. Use folio_ref_count()
+ *to read it.
+ * @memcg_data: Memory Control Group data.
+ *
+ * A folio is a physically, virtually and logically contiguous set
+ * of bytes. It is a power-of-two in size, and it is aligned to that
+ * same power-of-two. It is at least as large as %PAGE_SIZE. If it is
+ * in the page
