[PATCH v6 69/99] brd: Convert to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Convert brd_pages from a radix tree to an XArray. Simpler and smaller code; in particular another user of radix_tree_preload is eliminated. Signed-off-by: Matthew Wilcox --- drivers/block/brd.c | 93 - 1 file changed

[PATCH v6 26/99] page cache: Convert page cache lookups to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Introduce page_cache_pin() to factor out the common logic between the various lookup routines: find_get_entry find_get_entries find_get_pages_range find_get_pages_contig find_get_pages_range_tag find_get_entries_tag filemap_map_pages By using the xa_state to control the

[PATCH v6 67/99] mm: Convert cgroup writeback to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This is a fairly naive conversion, leaving in place the GFP_ATOMIC allocation. By switching the locking around, we could use GFP_KERNEL and probably simplify the error handling. Signed-off-by: Matthew Wilcox --- include/linux/backing-dev-defs.h | 2 +- include/linux

[PATCH v6 66/99] page cache: Finish XArray conversion

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox With no more radix tree API users left, we can drop the GFP flags and use xa_init() instead of INIT_RADIX_TREE(). Signed-off-by: Matthew Wilcox --- fs/inode.c | 2 +- include/linux/fs.h | 2 +- mm/swap_state.c| 2 +- 3 files changed, 3 insertions(+), 3

[PATCH v6 64/99] dax: Convert grab_mapping_entry to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Signed-off-by: Matthew Wilcox --- fs/dax.c | 98 +--- 1 file changed, 26 insertions(+), 72 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index 494e8fb7a98f..3eb0cf176d69 100644 --- a/fs/dax.c +++ b/fs/dax.c

[PATCH v6 65/99] dax: Fix sparse warning

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox sparse doesn't know that follow_pte_pmd() conditionally acquires the ptl, because it's in a separate compilation unit. Move follow_pte_pmd() to mm.h where sparse can see it. Signed-off-by: Matthew Wilcox --- include/linux/mm.h | 15 ++- m

[PATCH v6 62/99] dax: Convert dax_insert_pfn_mkwrite to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Signed-off-by: Matthew Wilcox --- fs/dax.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index b66b8c896ed8..e6b25ef112f2 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1497,21 +1497,21 @@ static int dax_insert_pfn_mkwrite(struct

[PATCH v6 63/99] dax: Convert dax_insert_mapping_entry to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Signed-off-by: Matthew Wilcox --- fs/dax.c | 18 ++ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index e6b25ef112f2..494e8fb7a98f 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -498,9 +498,9 @@ static void

[PATCH v6 25/99] page cache: Convert page deletion to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox The code is slightly shorter and simpler. Signed-off-by: Matthew Wilcox --- mm/filemap.c | 30 ++ 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index e6371b551de1..ed30d5310e50 100644 --- a/mm

[PATCH v6 60/99] dax: Convert __dax_invalidate_mapping_entry to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Simple now that we already have an xa_state! Signed-off-by: Matthew Wilcox --- fs/dax.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index d3fe61b95216..9a30224da4d6 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -413,24

[PATCH v6 61/99] dax: Convert dax_writeback_one to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Likewise easy Signed-off-by: Matthew Wilcox --- fs/dax.c | 17 +++-- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index 9a30224da4d6..b66b8c896ed8 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -632,8 +632,7 @@ static int

[PATCH v6 57/99] dax: Convert dax_unlock_mapping_entry to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Replace slot_locked() with dax_locked() and inline unlock_slot() into its only caller. Signed-off-by: Matthew Wilcox --- fs/dax.c | 48 1 file changed, 16 insertions(+), 32 deletions(-) diff --git a/fs/dax.c b/fs/dax.c

[PATCH v6 58/99] dax: Convert lock_slot to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Signed-off-by: Matthew Wilcox --- fs/dax.c | 22 -- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index f3463d93a6ce..8eab0b56f7f9 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -188,12 +188,11 @@ static void

[PATCH v6 59/99] dax: More XArray conversion

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This time, we want to convert get_unlocked_mapping_entry() to use the XArray. That has a ripple effect, causing us to change the waitqueues to hash on the address of the xarray rather than the address of the mapping (functionally equivalent), and create a lot of on-the

[PATCH v6 56/99] lustre: Convert to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Signed-off-by: Matthew Wilcox --- drivers/staging/lustre/lustre/llite/glimpse.c | 12 +--- drivers/staging/lustre/lustre/mdc/mdc_request.c | 16 2 files changed, 13 insertions(+), 15 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite

[PATCH v6 52/99] fs: Convert buffer to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Mostly comment fixes, but one use of __xa_set_tag. Signed-off-by: Matthew Wilcox --- fs/buffer.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 1a6ae530156b..e1d18307d5c8 100644 --- a/fs/buffer.c +++ b/fs

[PATCH v6 53/99] fs: Convert writeback to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox A couple of short loops. Signed-off-by: Matthew Wilcox --- fs/fs-writeback.c | 25 + 1 file changed, 9 insertions(+), 16 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index e2c1ca667d9a..897a89489fe9 100644 --- a/fs/fs

[PATCH v6 54/99] nilfs2: Convert to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox I'm not 100% convinced that the rewrite of nilfs_copy_back_pages is correct, but it will at least have different bugs from the current version. Signed-off-by: Matthew Wilcox --- fs/nilfs2/btnode.c | 37 +++- fs/nilfs2/page.c

[PATCH v6 55/99] f2fs: Convert to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This is a straightforward conversion. Signed-off-by: Matthew Wilcox --- fs/f2fs/data.c | 3 +-- fs/f2fs/dir.c| 5 + fs/f2fs/inline.c | 6 +- fs/f2fs/node.c | 10 ++ 4 files changed, 5 insertions(+), 19 deletions(-) diff --git a/fs/f2fs/data.c b

[PATCH v6 51/99] btrfs: Convert page cache to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Signed-off-by: Matthew Wilcox --- fs/btrfs/compression.c | 4 +--- fs/btrfs/extent_io.c | 6 ++ 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index e687d06cd97c..4174b166e235 100644 --- a/fs/btrfs

[PATCH v6 50/99] shmem: Comment fixups

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Remove the last mentions of radix tree from various comments. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 4dbcfb436bd1..5110848885d4 100644 --- a/mm/shmem.c

[PATCH v6 46/99] shmem: Convert shmem_add_to_page_cache to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This removes the last caller of radix_tree_maybe_preload_order(). Simpler code, unless we run out of memory for new xa_nodes partway through inserting entries into the xarray. Hopefully we can support multi-index entries in the page cache soon and all the awful code goes

[PATCH v6 47/99] shmem: Convert shmem_alloc_hugepage to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox xa_find() is a slightly easier API to use than radix_tree_gang_lookup_slot() because it contains its own RCU locking. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 14 -- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c

[PATCH v6 48/99] shmem: Convert shmem_free_swap to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This is a perfect use for xa_cmpxchg(). Note the use of 0 for GFP flags; we won't be allocating memory. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index e8233cb

[PATCH v6 49/99] shmem: Convert shmem_partial_swap_usage to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Simpler code because the xarray takes care of things like the limit and dereferencing the slot. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 18 +++--- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 5a2226e06f8c

[PATCH v6 45/99] shmem: Convert shmem_wait_for_pins to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox As with shmem_tag_pins(), hold the lock around the entire loop instead of acquiring & dropping it for each entry we're going to untag. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 59 --- 1 file ch

[PATCH v6 44/99] shmem: Convert shmem_tag_pins to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Simplify the locking by taking the spinlock while we walk the tree on the assumption that many acquires and releases of the lock will be worse than holding the lock for a (potentially) long time. We could replicate the same locking behaviour with the xarray, but would have

[PATCH v6 41/99] shmem: Convert replace to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox shmem_radix_tree_replace() is renamed to shmem_xa_replace() and converted to use the XArray API. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 22 -- 1 file changed, 8 insertions(+), 14 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index

[PATCH v6 43/99] shmem: Convert find_swap_entry to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This is a 1:1 conversion. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 23 +++ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 654f367aca90..ce285ae635ea 100644 --- a/mm/shmem.c +++ b/mm/shmem.c

[PATCH v6 40/99] pagevec: Use xa_tag_t

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Removes sparse warnings. Signed-off-by: Matthew Wilcox --- fs/btrfs/extent_io.c| 4 ++-- fs/ext4/inode.c | 2 +- fs/f2fs/data.c | 2 +- fs/gfs2/aops.c | 2 +- include/linux/pagevec.h | 8 +--- mm/swap.c | 4 ++-- 6 files

[PATCH v6 42/99] shmem: Convert shmem_confirm_swap to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox xa_load has its own RCU locking, so we can eliminate it here. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index fad6c9e7402e..654f367aca90 100644 --- a/mm/shmem.c +++ b/mm

[PATCH v6 38/99] mm: Convert collapse_shmem to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox I found another victim of the radix tree being hard to use. Because there was no call to radix_tree_preload(), khugepaged was allocating radix_tree_nodes using GFP_ATOMIC. I also converted a local_irq_save()/restore() pair to disable()/enable(). Signed-off-by: Matthew

[PATCH v6 39/99] mm: Convert khugepaged_scan_shmem to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Slightly shorter and easier to read code. Signed-off-by: Matthew Wilcox --- mm/khugepaged.c | 17 + 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 9f49d0cd61c2..15f1b2d81a69 100644 --- a/mm

[PATCH v6 35/99] mm: Convert __do_page_cache_readahead to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This one is trivial. Signed-off-by: Matthew Wilcox --- mm/readahead.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index f64b31b3a84a..66bcaffd47f0 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -174,9 +174,7

[PATCH v6 36/99] mm: Convert page migration to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Signed-off-by: Matthew Wilcox --- mm/migrate.c | 41 - 1 file changed, 16 insertions(+), 25 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 75d19904dd9a..7122fec9b075 100644 --- a/mm/migrate.c +++ b/mm/migrate.c

[PATCH v6 37/99] mm: Convert huge_memory to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Quite a straightforward conversion. Signed-off-by: Matthew Wilcox --- mm/huge_memory.c | 19 --- 1 file changed, 8 insertions(+), 11 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f71dd3e7d8cd..5c275295bbd3 100644 --- a/mm

[PATCH v6 33/99] mm: Convert add_to_swap_cache to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Combine __add_to_swap_cache and add_to_swap_cache into one function since there is no more need to preload. Signed-off-by: Matthew Wilcox --- mm/swap_state.c | 93 ++--- 1 file changed, 29 insertions(+), 64 deletions

[PATCH v6 32/99] mm: Convert truncate to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This is essentially xa_cmpxchg() with the locking handled above us, and it doesn't have to handle replacing a NULL entry. Signed-off-by: Matthew Wilcox --- mm/truncate.c | 15 ++- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/mm/truncate.c

[PATCH v6 34/99] mm: Convert delete_from_swap_cache to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Both callers of __delete_from_swap_cache have the swp_entry_t already, so pass that in to make constructing the XA_STATE easier. Signed-off-by: Matthew Wilcox --- include/linux/swap.h | 5 +++-- mm/swap_state.c | 24 ++-- mm/vmscan.c

[PATCH v6 31/99] mm: Convert workingset to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox We construct a fake XA_STATE and use it to delete the node with xa_store() rather than adding a special function for this unique use case. Signed-off-by: Matthew Wilcox --- include/linux/swap.h | 9 - mm/workingset.c | 51

[PATCH v6 30/99] mm: Convert page-writeback to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Includes moving mapping_tagged() to fs.h as a static inline, and changing it to return bool. Signed-off-by: Matthew Wilcox --- include/linux/fs.h | 17 +-- mm/page-writeback.c | 62 +++-- 2 files changed, 32

[PATCH v6 27/99] page cache: Convert delete_batch to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Rename the function from page_cache_tree_delete_batch to just page_cache_delete_batch. Signed-off-by: Matthew Wilcox --- mm/filemap.c | 28 +--- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index

[PATCH v6 29/99] page cache: Convert filemap_range_has_page to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Instead of calling find_get_pages_range() and putting any reference, just use xa_find() to look for a page in the right range. Signed-off-by: Matthew Wilcox --- mm/filemap.c | 9 + 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/mm/filemap.c b/mm

[PATCH v6 22/99] page cache: Convert hole search to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox The page cache offers the ability to search for a miss in the previous or next N locations. Rather than teach the XArray about the page cache's definition of a miss, use xas_prev() and xas_next() to search the page array. This should be more efficient as it does not

[PATCH v6 23/99] page cache: Add page_cache_range_empty function

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox btrfs has its own custom function for determining whether the page cache has any pages in a particular range. Move this functionality to the page cache, and call it from btrfs. Signed-off-by: Matthew Wilcox --- fs/btrfs/btrfs_inode.h | 7 - fs/btrfs/inode.c

[PATCH v6 24/99] page cache: Add and replace pages using the XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Use the XArray APIs to add and replace pages in the page cache. This removes two uses of the radix tree preload API and is significantly shorter code. Signed-off-by: Matthew Wilcox --- include/linux/swap.h | 8 ++- mm/filemap.c | 143

[PATCH v6 20/99] ida: Convert to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Use the xarray infrstructure like we used the radix tree infrastructure. This lets us get rid of idr_get_free() from the radix tree code. Signed-off-by: Matthew Wilcox --- include/linux/idr.h| 8 +- include/linux/radix-tree.h | 4 - lib/idr.c

[PATCH v6 21/99] xarray: Add xa_reserve and xa_release

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This function simply creates a slot in the XArray for users which need to acquire multiple locks before storing their entry in the tree and so cannot use a plain xa_store(). Signed-off-by: Matthew Wilcox --- include/linux/xarray.h | 14 ++ lib

[PATCH v6 18/99] xarray: Add ability to store errno values

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox While the radix tree offers no ability to store IS_ERR pointers, documenting that the XArray does not led to some concern. Here is a sanctioned way to store errnos in the XArray. I'm concerned that it will confuse people who can't tell the difference between

[PATCH v6 19/99] idr: Convert to XArray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox The IDR distinguishes between unallocated entries (read as NULL) and entries where the user has chosen to store NULL. The radix tree was modified to consider NULL entries which had tag 0 _clear_ as being allocated, but it added a lot of complexity. Instead, the XArray has

[PATCH v6 17/99] xarray: Add MAINTAINERS entry

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Add myself as XArray and IDR maintainer. Signed-off-by: Matthew Wilcox --- MAINTAINERS | 12 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 18994806e441..55ae4c0b38d5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14893,6

[PATCH v6 16/99] xarray: Add xas_create_range

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This hopefully temporary function is useful for users who have not yet been converted to multi-index entries. Signed-off-by: Matthew Wilcox --- include/linux/xarray.h | 2 ++ lib/xarray.c | 22 ++ 2 files changed, 24 insertions(+) diff

[PATCH v6 14/99] xarray: Add xa_destroy

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This function frees all the internal memory allocated to the xarray and reinitialises it to be empty. Signed-off-by: Matthew Wilcox --- include/linux/xarray.h | 1 + lib/xarray.c | 26 ++ 2 files changed, 27 insertions(+) diff --git a

[PATCH v6 15/99] xarray: Add xas_next and xas_prev

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox These two functions move the xas index by one position, and adjust the rest of the iterator state to match it. This is more efficient than calling xas_set() as it keeps the iterator at the leaves of the tree instead of walking the iterator from the root each time. Signed

[PATCH v6 00/99] XArray version 6

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This version of the XArray has no known bugs. I have converted the radix tree test suite entirely over to the XArray and fixed all bugs that it has uncovered. There are additional tests in the test suite for the XArray, so I now claim the XArray has better test coverage

[PATCH v6 13/99] xarray: Add xa_extract

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This function combines the functionality of radix_tree_gang_lookup() and radix_tree_gang_lookup_tagged(). It extracts entries matching the specified filter into a normal array. Signed-off-by: Matthew Wilcox --- include/linux/xarray.h | 2 ++ lib/xarray.c | 80

[PATCH v6 12/99] xarray: Add xa_for_each

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This iterator allows the user to efficiently walk a range of the array, executing the loop body once for each entry in that range that matches the filter. This commit also includes xa_find() and xa_find_above() which are helper functions for xa_for_each() but may also be

[PATCH v6 11/99] xarray: Add xa_cmpxchg and xa_insert

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Like cmpxchg(), xa_cmpxchg will only store to the index if the current entry matches the old entry. It returns the current entry, which is usually more useful than the errno returned by radix_tree_insert(). For the users who really only want the errno, the xa_insert

[PATCH v6 08/99] xarray: Add xa_load

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This first function in the XArray API brings with it a lot of support infrastructure. The advanced API is based around the xa_state which is a more capable version of the radix_tree_iter. As the test-suite demonstrates, it is possible to use the xarray and radix tree APIs

[PATCH v6 09/99] xarray: Add xa_get_tag, xa_set_tag and xa_clear_tag

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox XArray tags are slightly more strongly typed than the radix tree tags, but occupy the same bits. This commit also adds the xas_ family of tag operations, for cases where the caller is already holding the lock, and xa_tagged() to ask whether any array member has a particular

[PATCH v6 10/99] xarray: Add xa_store

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox xa_store() differs from radix_tree_insert() in that it will overwrite an existing element in the array rather than returning an error. This is the behaviour which most users want, and those that want more complex behaviour generally want to use the xas family of routines

[PATCH v6 07/99] xarray: Add documentation

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This is documentation on how to use the XArray, not details about its internal implementation. Signed-off-by: Matthew Wilcox --- Documentation/core-api/index.rst | 1 + Documentation/core-api/xarray.rst | 361 ++ 2 files changed, 362

[PATCH v6 06/99] xarray: Define struct xa_node

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This is a direct replacement for struct radix_tree_node. A couple of struct members have changed name, so convert those. Use a #define so that radix tree users continue to work without change. Signed-off-by: Matthew Wilcox --- include/linux/radix-tree.h| 29

[PATCH v6 03/99] xarray: Replace exceptional entries

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Introduce xarray value entries to replace the radix tree exceptional entry code. This is a slight change in encoding to allow the use of an extra bit (we can now store BITS_PER_LONG - 1 bits in a value entry). It is also a change in emphasis; exceptional entries are

[PATCH v6 02/99] page cache: Use xa_lock

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Remove the address_space ->tree_lock and use the xa_lock newly added to the radix_tree_root. Rename the address_space ->page_tree to ->pages, since we don't really care that it's a tree. Take the opportunity to rearrange the elements of address_space to

[PATCH v6 04/99] xarray: Change definition of sibling entries

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox Instead of storing a pointer to the slot containing the canonical entry, store the offset of the slot. Produces slightly more efficient code (~300 bytes) and simplifies the implementation. Signed-off-by: Matthew Wilcox --- include/linux/xarray.h | 90

[PATCH v6 05/99] xarray: Add definition of struct xarray

2018-01-17 Thread Matthew Wilcox
From: Matthew Wilcox This is a direct replacement for struct radix_tree_root. Some of the struct members have changed name; convert those, and use a #define so that radix_tree users continue to work without change. Signed-off-by: Matthew Wilcox --- include/linux/radix-tree.h

Re: [PATCH] rcu: Use wrapper for lockdep asserts

2018-01-17 Thread Matthew Wilcox
On Wed, Jan 17, 2018 at 03:29:13PM -0800, Paul E. McKenney wrote: > On Wed, Jan 17, 2018 at 06:24:30AM -0800, Matthew Wilcox wrote: > > > > From: Matthew Wilcox > > > > Commits c0b334c5bfa9 and ea9b0c8a26a2 introduced new sparse warnings > > by accessing rcu_nod

Re: [PATCH 13/38] ext4: Define usercopy region in ext4_inode_cache slab cache

2018-01-14 Thread Matthew Wilcox
On Thu, Jan 11, 2018 at 03:05:14PM -0800, Kees Cook wrote: > On Thu, Jan 11, 2018 at 9:01 AM, Theodore Ts'o wrote: > > On Wed, Jan 10, 2018 at 06:02:45PM -0800, Kees Cook wrote: > >> The ext4 symlink pathnames, stored in struct ext4_inode_info.i_data > >> and therefore contained in the ext4_inode_

Re: [PATCH 04/36] usercopy: Prepare for usercopy whitelisting

2018-01-14 Thread Matthew Wilcox
On Wed, Jan 10, 2018 at 12:28:23PM -0600, Christopher Lameter wrote: > On Tue, 9 Jan 2018, Kees Cook wrote: > > +struct kmem_cache *kmem_cache_create_usercopy(const char *name, > > + size_t size, size_t align, slab_flags_t flags, > > + size_t useroffset, size_t u

Re: [PATCH v6 22/24] mm: Speculative page fault handler return VMA

2018-01-16 Thread Matthew Wilcox
On Tue, Jan 16, 2018 at 03:47:51PM +0100, Laurent Dufour wrote: > On 13/01/2018 05:23, Matthew Wilcox wrote: > > Of course, we don't need to change them all. Try this: > > That would be good candidate for a clean up but I'm not sure this should be > part of this alre

Re: kmem_cache_attr (was Re: [PATCH 04/36] usercopy: Prepare for usercopy whitelisting)

2018-01-16 Thread Matthew Wilcox
On Tue, Jan 16, 2018 at 09:21:30AM -0600, Christopher Lameter wrote: > > struct kmem_cache_attr { > > const char name[32]; > > Want to avoid the string reference mess that occurred in the past? > Is that really necessary? But it would limit the size of the name. I think that's a good thing!

Re: [PATCH 4/7] Protectable Memory

2018-03-12 Thread Matthew Wilcox
On Wed, Feb 28, 2018 at 10:06:17PM +0200, Igor Stoppa wrote: > struct gen_pool *pmalloc_create_pool(const char *name, >int min_alloc_order); > int is_pmalloc_object(const void *ptr, const unsigned long n); > bool pmalloc_prealloc(struct gen_pool *pool, size_t

Re: [RFC PATCH] Randomization of address chosen by mmap.

2018-03-05 Thread Matthew Wilcox
On Mon, Mar 05, 2018 at 04:09:31PM +0300, Ilya Smith wrote: > > On 4 Mar 2018, at 23:56, Matthew Wilcox wrote: > > Thinking about this more ... > > > > - When you call munmap, if you pass in the same (addr, length) that were > > used for mmap, then it should unmap

Re: [RFC, PATCH 00/22] Partial MKTME enabling

2018-03-05 Thread Matthew Wilcox
On Mon, Mar 05, 2018 at 10:30:50AM -0800, Christoph Hellwig wrote: > On Mon, Mar 05, 2018 at 07:25:48PM +0300, Kirill A. Shutemov wrote: > > Hi everybody, > > > > Here's updated version of my patchset that brings support of MKTME. > > It would really help if you'd explain what "MKTME" is.. You n

Re: [RFC PATCH] Randomization of address chosen by mmap.

2018-03-05 Thread Matthew Wilcox
On Mon, Mar 05, 2018 at 10:27:32PM +0300, Ilya Smith wrote: > > On 5 Mar 2018, at 19:23, Matthew Wilcox wrote: > > On Mon, Mar 05, 2018 at 04:09:31PM +0300, Ilya Smith wrote: > >> I’m analysing that approach and see much more problems: > >> - each time you call mmap

Re: [PATCH v4 3/3] mm/free_pcppages_bulk: prefetch buddy while not holding lock

2018-03-06 Thread Matthew Wilcox
On Tue, Mar 06, 2018 at 08:27:33PM +0800, Aaron Lu wrote: > On Tue, Mar 06, 2018 at 08:55:57AM +0100, Vlastimil Babka wrote: > > So the adjacent line prefetch might be disabled? Could you check bios or > > the MSR mentioned in > > https://software.intel.com/en-us/articles/disclosure-of-hw-prefetche

Re: [PATCH 1/7] genalloc: track beginning of allocations

2018-03-06 Thread Matthew Wilcox
On Wed, Feb 28, 2018 at 10:06:14PM +0200, Igor Stoppa wrote: > + * Encoding of the bitmap tracking the allocations > + * --- > + * > + * The bitmap is composed of units of allocations. > + * > + * Each unit of allocation is represented using 2 consecutive

Re: [PATCH 1/7] genalloc: track beginning of allocations

2018-03-06 Thread Matthew Wilcox
On Tue, Mar 06, 2018 at 02:19:03PM +0100, Mike Rapoport wrote: > > +/** > > + * gen_pool_create() - create a new special memory pool > > + * @min_alloc_order: log base 2 of number of bytes each bitmap entry > > + * represents > > + * @nid: node id of the node the pool structure should

[PATCH v8 13/63] xarray: Add definition of struct xarray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox This is a direct replacement for struct radix_tree_root. Some of the struct members have changed name; convert those, and use a #define so that radix_tree users continue to work without change. Signed-off-by: Matthew Wilcox --- include/linux/radix-tree.h

[PATCH v8 48/63] shmem: Convert shmem_add_to_page_cache to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox This removes the last caller of radix_tree_maybe_preload_order(). Simpler code, unless we run out of memory for new xa_nodes partway through inserting entries into the xarray. Hopefully we can support multi-index entries in the page cache soon and all the awful code goes

[PATCH v8 02/63] radix tree: Use GFP_ZONEMASK bits of gfp_t for flags

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox None of these bits may be used for slab allocations, so we can use them as radix tree flags as long as we mask them off before passing them to the slab allocator. Move the IDR flag from the high bits to the GFP_ZONEMASK bits. Signed-off-by: Matthew Wilcox Acked-by: Jeff

[PATCH v8 46/63] shmem: Convert shmem_confirm_swap to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox xa_load has its own RCU locking, so we can eliminate it here. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 5813808965cd..0af8a439dfad 100644 --- a/mm/shmem.c +++ b/mm

[PATCH v8 62/63] page cache: Finish XArray conversion

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox With no more radix tree API users left, we can drop the GFP flags and use xa_init() instead of INIT_RADIX_TREE(). Signed-off-by: Matthew Wilcox --- fs/inode.c | 2 +- mm/swap_state.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/inode.c b

[PATCH v8 63/63] radix tree: Remove unused functions

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox The following functions are (now) unused: - __radix_tree_delete_node - radix_tree_gang_lookup_slot - radix_tree_join - radix_tree_maybe_preload_order - radix_tree_split - radix_tree_split_preload Signed-off-by: Matthew Wilcox --- include/linux/radix-tree.h | 16

[PATCH v8 45/63] shmem: Convert replace to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox shmem_radix_tree_replace() is renamed to shmem_xa_replace() and converted to use the XArray API. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 22 -- 1 file changed, 8 insertions(+), 14 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index

[PATCH v8 60/63] lustre: Convert to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox Signed-off-by: Matthew Wilcox --- drivers/staging/lustre/lustre/llite/glimpse.c | 12 +--- drivers/staging/lustre/lustre/mdc/mdc_request.c | 16 2 files changed, 13 insertions(+), 15 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite

[PATCH v8 61/63] dax: Convert to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox The DAX code (by its nature) is deeply interwoven with the radix tree infrastructure, doing operations directly on the radix tree slots. Convert the whole file to use XArray concepts; mostly passing around xa_state instead of address_space, index or slot. Signed-off-by

[PATCH v8 56/63] fs: Convert buffer to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox Mostly comment fixes, but one use of __xa_set_tag. Signed-off-by: Matthew Wilcox --- fs/buffer.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 3ee82c056d85..70af8fbc64cf 100644 --- a/fs/buffer.c +++ b/fs

[PATCH v8 55/63] btrfs: Convert page cache to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox Signed-off-by: Matthew Wilcox --- fs/btrfs/compression.c | 4 +--- fs/btrfs/extent_io.c | 8 +++- 2 files changed, 4 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index ad330af89eef..c2286f436571 100644 --- a/fs/btrfs

[PATCH v8 58/63] nilfs2: Convert to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox I'm not 100% convinced that the rewrite of nilfs_copy_back_pages is correct, but it will at least have different bugs from the current version. Signed-off-by: Matthew Wilcox --- fs/nilfs2/btnode.c | 37 +++- fs/nilfs2/page.c

[PATCH v8 57/63] fs: Convert writeback to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox A couple of short loops. Signed-off-by: Matthew Wilcox --- fs/fs-writeback.c | 25 + 1 file changed, 9 insertions(+), 16 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index 091577edc497..98e5e08274a2 100644 --- a/fs/fs

[PATCH v8 59/63] f2fs: Convert to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox This is a straightforward conversion. Signed-off-by: Matthew Wilcox --- fs/f2fs/data.c | 3 +-- fs/f2fs/dir.c| 5 + fs/f2fs/inline.c | 6 +- fs/f2fs/node.c | 10 ++ 4 files changed, 5 insertions(+), 19 deletions(-) diff --git a/fs/f2fs/data.c b

[PATCH v8 54/63] shmem: Comment fixups

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox Remove the last mentions of radix tree from various comments. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 707430003ec7..6b044cb6c8b5 100644 --- a/mm/shmem.c

[PATCH v8 49/63] shmem: Convert shmem_alloc_hugepage to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox xa_find() is a slightly easier API to use than radix_tree_gang_lookup_slot() because it contains its own RCU locking. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 14 -- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c

[PATCH v8 52/63] memfd: Convert shmem_tag_pins to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox Simplify the locking by taking the spinlock while we walk the tree on the assumption that many acquires and releases of the lock will be worse than holding the lock for a (potentially) long time. We could replicate the same locking behaviour with the xarray, but would have

[PATCH v8 50/63] shmem: Convert shmem_free_swap to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox This is a perfect use for xa_cmpxchg(). Note the use of 0 for GFP flags; we won't be allocating memory. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index a0a354a

[PATCH v8 51/63] shmem: Convert shmem_partial_swap_usage to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox Simpler code because the xarray takes care of things like the limit and dereferencing the slot. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 18 +++--- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index cfbffb4b47a2

[PATCH v8 53/63] memfd: Convert shmem_wait_for_pins to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox As with shmem_tag_pins(), hold the lock around the entire loop instead of acquiring & dropping it for each entry we're going to untag. Signed-off-by: Matthew Wilcox --- mm/memfd.c | 61 + 1 file ch

[PATCH v8 47/63] shmem: Convert find_swap_entry to XArray

2018-03-06 Thread Matthew Wilcox
From: Matthew Wilcox This is a 1:1 conversion. Signed-off-by: Matthew Wilcox --- mm/shmem.c | 23 +++ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 0af8a439dfad..49f42dc9e1dc 100644 --- a/mm/shmem.c +++ b/mm/shmem.c

<    3   4   5   6   7   8   9   10   11   12   >