[PATCH 1/1] ndctl/namespace: Fix disable-namespace accounting relative to seed devices

2021-01-28 Thread redhairer
From: Redhairer Li Seed namespaces are included in "ndctl disable-namespace all". However since the user never "creates" them it is surprising to see "disable-namespace" report 1 more namespace relative to the number that have been created. Catch attempts to disable a zero-sized namespace:

Re: [PATCH v16 08/11] secretmem: add memcg accounting

2021-01-28 Thread Shakeel Butt
On Wed, Jan 27, 2021 at 11:59 PM Michal Hocko wrote: > > On Wed 27-01-21 10:42:13, Roman Gushchin wrote: > > On Tue, Jan 26, 2021 at 04:05:55PM +0100, Michal Hocko wrote: > > > On Tue 26-01-21 14:48:38, Matthew Wilcox wrote: > > > > On Mon, Jan 25, 2021 at 11:38:17PM +0200, Mike Rapoport wrote: >

RE: [PATCH 1/1] ndctl/namespace: Fix disable-namespace accounting relative to seed devices

2021-01-28 Thread Li, Redhairer
Resend it base on v71.1. -Original Message- From: Dan Williams Sent: Thursday, January 28, 2021 4:47 PM To: Li, Redhairer Cc: linux-nvdimm Subject: Re: [PATCH 1/1] ndctl/namespace: Fix disable-namespace accounting relative to seed devices On Fri, Jan 15, 2021 at 9:22 AM Li,

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Michal Hocko
On Thu 28-01-21 15:56:36, Cristopher Lameter wrote: > On Thu, 28 Jan 2021, Michal Hocko wrote: > > > > > If you kill the allocating process then yes, it would work, but your > > > > process might be the very last to be selected. > > > > > > OOMs are different if you have a "constrained

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread James Bottomley
On Thu, 2021-01-28 at 14:01 +0100, Michal Hocko wrote: > On Thu 28-01-21 11:22:59, Mike Rapoport wrote: [...] > > I like the idea to have a pool as an optimization rather than a > > hard requirement but I don't see why would it need a careful access > > control. As the direct map fragmentation is

RE: 10K LinkedIn Leads at 500

2021-01-28 Thread Emily Johnson
Are you interested to purchase 100% accurate 10,000 Targeted Leads from LinkedIn at $500(Any titles/industry/location/keywords)? If you have a very targeted data requirement and you need LinkedIn database, we will pull targeted databases for you with their LinkedIn profile link, name, title, email

Re: [RFC v5 0/7] PMEM device emulation without nfit depenency

2021-01-28 Thread Dan Williams
On Mon, Dec 14, 2020 at 2:39 AM Santosh Sivaraj wrote: > > The current test module cannot be used for testing platforms (make check) > that do not have support for NFIT. In order to get the ndctl tests working, > we need a module which can emulate NVDIMM devices without relying on > ACPI/NFIT. >

Re: [PATCH 1/1] ndctl/namespace: Fix disable-namespace accounting relative to seed devices

2021-01-28 Thread Dan Williams
On Fri, Jan 15, 2021 at 9:22 AM Li, Redhairer wrote: > > Hi Dan, > > Your comment is make sense. > ndctl_namespace_disable_safe will return 1 if namespace size is 0. > I send a new patch out for review. It looks ok but it does not apply to the current tip of tree now v71.1 can you resend? > >

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Mike Rapoport
On Tue, Jan 26, 2021 at 01:08:23PM +0100, Michal Hocko wrote: > On Tue 26-01-21 12:56:48, David Hildenbrand wrote: > > On 26.01.21 12:46, Michal Hocko wrote: > > > On Thu 21-01-21 14:27:19, Mike Rapoport wrote: > > > > From: Mike Rapoport > > > > > > > > Removing a PAGE_SIZE page from the direct

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Christoph Lameter
On Thu, 28 Jan 2021, Michal Hocko wrote: > > > If you kill the allocating process then yes, it would work, but your > > > process might be the very last to be selected. > > > > OOMs are different if you have a "constrained allocation". In that case it > > is the fault of the process who wanted

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Michal Hocko
On Thu 28-01-21 11:22:59, Mike Rapoport wrote: > On Tue, Jan 26, 2021 at 01:08:23PM +0100, Michal Hocko wrote: > > On Tue 26-01-21 12:56:48, David Hildenbrand wrote: > > > On 26.01.21 12:46, Michal Hocko wrote: > > > > On Thu 21-01-21 14:27:19, Mike Rapoport wrote: > > > > > From: Mike Rapoport >

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Michal Hocko
On Thu 28-01-21 13:28:10, Cristopher Lameter wrote: > On Thu, 28 Jan 2021, Michal Hocko wrote: > > > > So, if I understand your concerns correct this implementation has two > > > issues: > > > 1) allocation failure at page fault that causes unrecoverable OOM and > > > 2) a possibility for an

Re: [PATCH v16 08/11] secretmem: add memcg accounting

2021-01-28 Thread Michal Hocko
On Thu 28-01-21 06:05:11, Shakeel Butt wrote: > On Wed, Jan 27, 2021 at 11:59 PM Michal Hocko wrote: > > > > On Wed 27-01-21 10:42:13, Roman Gushchin wrote: > > > On Tue, Jan 26, 2021 at 04:05:55PM +0100, Michal Hocko wrote: > > > > On Tue 26-01-21 14:48:38, Matthew Wilcox wrote: > > > > > On

Re: [PATCH v16 08/11] secretmem: add memcg accounting

2021-01-28 Thread Shakeel Butt
On Thu, Jan 28, 2021 at 6:22 AM Michal Hocko wrote: > > On Thu 28-01-21 06:05:11, Shakeel Butt wrote: > > On Wed, Jan 27, 2021 at 11:59 PM Michal Hocko wrote: > > > > > > On Wed 27-01-21 10:42:13, Roman Gushchin wrote: > > > > On Tue, Jan 26, 2021 at 04:05:55PM +0100, Michal Hocko wrote: > > > >

Re: PATCH] fs/dax: fix compile problem on parisc and mips

2021-01-28 Thread Helge Deller
On 12/4/20 1:33 AM, James Bottomley wrote: > These platforms define PMD_ORDER in asm/pgtable.h > > This means that as soon as dax.c included asm/pgtable.h in commit > 11cf9d863dcb ("fs/dax: Deposit pagetable even when installing zero > page") we clash with PMD_ORDER introduced by cfc93c6c6c96

Re: [PATCH v16 08/11] secretmem: add memcg accounting

2021-01-28 Thread Shakeel Butt
On Mon, Jan 25, 2021 at 1:35 PM Mike Rapoport wrote: > > On Mon, Jan 25, 2021 at 09:18:04AM -0800, Shakeel Butt wrote: > > On Mon, Jan 25, 2021 at 8:20 AM Matthew Wilcox wrote: > > > > > > On Thu, Jan 21, 2021 at 02:27:20PM +0200, Mike Rapoport wrote: > > > > From: Mike Rapoport > > > > > > > >

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Mike Rapoport
On Thu, Jan 28, 2021 at 07:28:57AM -0800, James Bottomley wrote: > On Thu, 2021-01-28 at 14:01 +0100, Michal Hocko wrote: > > On Thu 28-01-21 11:22:59, Mike Rapoport wrote: > [...] > > > One of the major pushbacks on the first RFC [1] of the concept was > > > about the direct map fragmentation. I

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Mike Rapoport
On Thu, Jan 28, 2021 at 02:01:06PM +0100, Michal Hocko wrote: > On Thu 28-01-21 11:22:59, Mike Rapoport wrote: > > > And hugetlb pools may be also depleted by anybody by calling > > mmap(MAP_HUGETLB) and there is no any limiting knob for this, while > > secretmem has RLIMIT_MEMLOCK. > > Yes it

[PATCH RESEND v2 09/10] xfs: Implement ->corrupted_range() for XFS

2021-01-28 Thread Shiyang Ruan
This function is used to handle errors which may cause data lost in filesystem. Such as memory failure in fsdax mode. In XFS, it requires "rmapbt" feature in order to query for files or metadata which associated to the corrupted data. Then we could call fs recover functions to try to repair the

[PATCH RESEND v2 07/10] dm: Introduce ->rmap() to find bdev offset

2021-01-28 Thread Shiyang Ruan
Pmem device could be a target of mapped device. In order to obtain superblock on the mapped device, we introduce this to translate offset from target device to md device. Currently, we implement it on linear target, which is easy to do the translation. Other targets will be supported in the

[PATCH RESEND v2 00/10] fsdax: introduce fs query to support reflink

2021-01-28 Thread Shiyang Ruan
This patchset is aimed to support shared pages tracking for fsdax. Resend V2: - Cc dm-devel instead of linux-raid Change from V1: - Add the old memory-failure handler back for rolling back - Add callback in MD's ->rmap() to support multiple mapping of dm device - Add judgement for

[PATCH RESEND v2 10/10] fs/dax: Remove useless functions

2021-01-28 Thread Shiyang Ruan
Since owner tarcking is triggerred by pmem device, these functions are useless. So remove it. Signed-off-by: Shiyang Ruan --- fs/dax.c | 46 -- 1 file changed, 46 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index c64c3a0e76a6..e20a5df03eec 100644

[PATCH RESEND v2 08/10] md: Implement ->corrupted_range()

2021-01-28 Thread Shiyang Ruan
With the support of ->rmap(), it is possible to obtain the superblock on a mapped device. If a pmem device is used as one target of mapped device, we cannot obtain its superblock directly. With the help of SYSFS, the mapped device can be found on the target devices. So, we iterate the

[PATCH RESEND v2 03/10] fs: Introduce ->corrupted_range() for superblock

2021-01-28 Thread Shiyang Ruan
Memory failure occurs in fsdax mode will finally be handled in filesystem. We introduce this interface to find out files or metadata affected by the corrupted range, and try to recover the corrupted data if possiable. Signed-off-by: Shiyang Ruan --- include/linux/fs.h | 2 ++ 1 file changed, 2

[PATCH RESEND v2 05/10] mm, pmem: Implement ->memory_failure() in pmem driver

2021-01-28 Thread Shiyang Ruan
Call the ->memory_failure() which is implemented by pmem driver, in order to finally notify filesystem to handle the corrupted data. The handler which collects and kills processes are moved into mf_dax_mapping_kill_procs(), which will be called by filesystem. Keep the old handler in order to

[PATCH RESEND v2 04/10] mm, fsdax: Refactor memory-failure handler for dax mapping

2021-01-28 Thread Shiyang Ruan
The current memory_failure_dev_pagemap() can only handle single-mapped dax page for fsdax mode. The dax page could be mapped by multiple files and offsets if we let reflink feature & fsdax mode work together. So, we refactor current implementation to support handle memory failure on each file

[PATCH RESEND v2 06/10] pmem: Implement ->corrupted_range() for pmem driver

2021-01-28 Thread Shiyang Ruan
Obtain the superblock of a pmem disk, and call filesystem's ->corrupted_range() to handle the corrupted data. Signed-off-by: Shiyang Ruan --- block/genhd.c | 6 ++ drivers/nvdimm/pmem.c | 24 include/linux/genhd.h | 1 + 3 files changed, 31 insertions(+)

[PATCH RESEND v2 01/10] pagemap: Introduce ->memory_failure()

2021-01-28 Thread Shiyang Ruan
When memory-failure occurs, we call this function which is implemented by each kind of devices. For the fsdax case, pmem device driver implements it. Pmem device driver will find out the block device where the error page locates in, and try to get the filesystem on this block device. And

[PATCH RESEND v2 02/10] blk: Introduce ->corrupted_range() for block device

2021-01-28 Thread Shiyang Ruan
In fsdax mode, the memory failure happens on block device. So, it is needed to introduce an interface for block devices. Each kind of block device can handle the memory failure in ther own ways. Signed-off-by: Shiyang Ruan --- include/linux/blkdev.h | 2 ++ 1 file changed, 2 insertions(+)

Re: [ndctl 5/5] Use page size as alignment value

2021-01-28 Thread Santosh Sivaraj
Dan Williams writes: > On Mon, Dec 21, 2020 at 8:26 PM Santosh Sivaraj wrote: >> >> The alignment sizes passed to ndctl in the tests are all hardcoded to 4k, >> the default page size on x86. Change those to the default page size on that >> architecture (sysconf/getconf). No functional changes

Re: [PATCH v16 08/11] secretmem: add memcg accounting

2021-01-28 Thread Michal Hocko
On Wed 27-01-21 10:42:13, Roman Gushchin wrote: > On Tue, Jan 26, 2021 at 04:05:55PM +0100, Michal Hocko wrote: > > On Tue 26-01-21 14:48:38, Matthew Wilcox wrote: > > > On Mon, Jan 25, 2021 at 11:38:17PM +0200, Mike Rapoport wrote: > > > > I cannot use __GFP_ACCOUNT because cma_alloc() does not

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Christoph Lameter
On Thu, 28 Jan 2021, Michal Hocko wrote: > > So, if I understand your concerns correct this implementation has two > > issues: > > 1) allocation failure at page fault that causes unrecoverable OOM and > > 2) a possibility for an unprivileged user to deplete secretmem pool and > > cause (1) to

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread James Bottomley
On Thu, 2021-01-28 at 14:01 +0100, Michal Hocko wrote: > On Thu 28-01-21 11:22:59, Mike Rapoport wrote: [...] > > One of the major pushbacks on the first RFC [1] of the concept was > > about the direct map fragmentation. I tried really hard to find > > data that shows what is the performance