Re: [dm-devel] Snapshot target and DAX-capable devices
On Fri, Aug 31, 2018 at 11:42:55AM +0200, Jan Kara wrote: > On Fri 31-08-18 09:38:09, Dave Chinner wrote: > > On Thu, Aug 30, 2018 at 03:47:32PM -0400, Mikulas Patocka wrote: > > > You can't support dax on snapshot - if someone maps a block and the block > > > needs to be moved, then what? > > > > This is only a problem for access via mmap and page faults. > > It's a whole different ballgame for a dm-snapshot device - block > > devices are completely unaware of page faults to DAX file mappings. > > Actually, block devices are not completely unaware of DAX page faults - > they will get ->direct_access callback for the fault range. It does not > currently convey enough information - we also need to inform the block > device whether it is read or write. But that's about all that's needed to > add AFAICT. And by comparing returned PFN with the one we have stored in > the radix tree (which we have if that file offset is mapped by anybody), > the filesystem / DAX code can tell whether remapping happened and do the > unmapping. I forgot about the direct access call. But it seems like a hack to redefine the simple, fast sector-to-pfn translation into a slow and potentially resource hungry interface for physical storage reallocation. Doing storage layer COW operations inside direct_access takes us straight back to the bad ways of get_block() interfaces. We moved all the filesystem allocation to iomap so that the storage management is separated from the mm/physical address translation side of DAX - doing block device storage management operations inside ->direct_access effectively reverts that separation and so just seems like a hack to me. Oh, right, DAX. Silly me. :/ Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] dm bufio: Reduce dm_bufio_lock contention
On Tue 04-09-18 13:30:44, Mikulas Patocka wrote: > > > On Tue, 4 Sep 2018, Michal Hocko wrote: > > > Regarding other other workloads. AFAIR the problem was due to the > > wait_iff_congested in the direct reclaim. And I've been arguing that > > special casing __GFP_NORETRY is not a propoer way to handle that case. > > We have PF_LESS_THROTTLE to handle cases where the caller cannot be > > really throttled because it is a part of the congestion control. I dunno > > what happened in that regards since then though. > > -- > > Michal Hocko > > SUSE Labs > > So, could you take this patch https://patchwork.kernel.org/patch/10505523/ > and commit it to the kernel? > > You agreed with that patch, but you didn't commit it. Because I do not maintain any tree that Linus pulls from. You need to involve Andrew Morton to get this to the mm tree and then to Linus. -- Michal Hocko SUSE Labs -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] dm bufio: Reduce dm_bufio_lock contention
On Tue, 4 Sep 2018, Michal Hocko wrote: > Regarding other other workloads. AFAIR the problem was due to the > wait_iff_congested in the direct reclaim. And I've been arguing that > special casing __GFP_NORETRY is not a propoer way to handle that case. > We have PF_LESS_THROTTLE to handle cases where the caller cannot be > really throttled because it is a part of the congestion control. I dunno > what happened in that regards since then though. > -- > Michal Hocko > SUSE Labs So, could you take this patch https://patchwork.kernel.org/patch/10505523/ and commit it to the kernel? You agreed with that patch, but you didn't commit it. Mikulas -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] dm bufio: Reduce dm_bufio_lock contention
On Tue 04-09-18 11:18:44, Mike Snitzer wrote: > On Tue, Sep 04 2018 at 3:08am -0400, > Michal Hocko wrote: > > > On Mon 03-09-18 18:23:17, Mikulas Patocka wrote: > > > > > > > > > On Wed, 1 Aug 2018, jing xia wrote: > > > > > > > We reproduced this issue again and found out the root cause. > > > > dm_bufio_prefetch() with dm_bufio_lock enters the direct reclaim and > > > > takes a long time to do the soft_limit_reclaim, because of the huge > > > > number of memory excess of the memcg. > > > > Then, all the task who do shrink_slab() wait for dm_bufio_lock. > > > > > > > > Any suggestions for this?Thanks. > > > > > > There's hardly any solution because Michal Hocko refuses to change > > > __GFP_NORETRY behavior. > > > > > > The patches 41c73a49df31151f4ff868f28fe4f129f113fa2c and > > > d12067f428c037b4575aaeb2be00847fc214c24a could reduce the lock contention > > > on the dm-bufio lock - the patches don't fix the high CPU consumption > > > inside the memory allocation, but the kernel code should wait less on the > > > bufio lock. > > > > If you actually looked at the bottom line of the problem then you would > > quickly find out that dm-bufio lock is the least of the problem with the > > soft limit reclaim. This is a misfeature which has been merged and we > > have to live with it. All we can do is to discourage people from using > > it and use much more saner low limit instead. > > > > So please stop this stupid blaming, try to understand the reasoning > > behind my arguments. > > Yes, this bickering isn't productive. Michal, your responses are pretty > hard to follow. I'm just trying to follow along on what it is you're > saying should be done. It isn't clear to me. > > PLEASE, restate what we should be doing differently. Or what changes > need to happen outside of DM, etc. For this particular case I can only recommend to not use the memcg soft limit. This is guaranteed to stall and there is no way around it because this is the semantic of the soft limit. Sad, I know. Regarding other other workloads. AFAIR the problem was due to the wait_iff_congested in the direct reclaim. And I've been arguing that special casing __GFP_NORETRY is not a propoer way to handle that case. We have PF_LESS_THROTTLE to handle cases where the caller cannot be really throttled because it is a part of the congestion control. I dunno what happened in that regards since then though. -- Michal Hocko SUSE Labs -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] dm bufio: Reduce dm_bufio_lock contention
On Tue, Sep 04 2018 at 3:08am -0400, Michal Hocko wrote: > On Mon 03-09-18 18:23:17, Mikulas Patocka wrote: > > > > > > On Wed, 1 Aug 2018, jing xia wrote: > > > > > We reproduced this issue again and found out the root cause. > > > dm_bufio_prefetch() with dm_bufio_lock enters the direct reclaim and > > > takes a long time to do the soft_limit_reclaim, because of the huge > > > number of memory excess of the memcg. > > > Then, all the task who do shrink_slab() wait for dm_bufio_lock. > > > > > > Any suggestions for this?Thanks. > > > > There's hardly any solution because Michal Hocko refuses to change > > __GFP_NORETRY behavior. > > > > The patches 41c73a49df31151f4ff868f28fe4f129f113fa2c and > > d12067f428c037b4575aaeb2be00847fc214c24a could reduce the lock contention > > on the dm-bufio lock - the patches don't fix the high CPU consumption > > inside the memory allocation, but the kernel code should wait less on the > > bufio lock. > > If you actually looked at the bottom line of the problem then you would > quickly find out that dm-bufio lock is the least of the problem with the > soft limit reclaim. This is a misfeature which has been merged and we > have to live with it. All we can do is to discourage people from using > it and use much more saner low limit instead. > > So please stop this stupid blaming, try to understand the reasoning > behind my arguments. Yes, this bickering isn't productive. Michal, your responses are pretty hard to follow. I'm just trying to follow along on what it is you're saying should be done. It isn't clear to me. PLEASE, restate what we should be doing differently. Or what changes need to happen outside of DM, etc. Thanks, Mike -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
[dm-devel] [PATCH v2] crypto: xts - Drop use of auxiliary buffer
Since commit acb9b159c784 ("crypto: gf128mul - define gf128mul_x_* in
gf128mul.h"), the gf128mul_x_*() functions are very fast and therefore
caching the computed XTS tweaks has only negligible advantage over
computing them twice.
In fact, since the current caching implementation limits the size of
the calls to the child ecb(...) algorithm to PAGE_SIZE (usually 4096 B),
it is often actually slower than the simple recomputing implementation.
This patch simplifies the XTS template to recompute the XTS tweaks from
scratch in the second pass and thus also removes the need to allocate a
dynamic buffer using kmalloc().
As discussed at [1], the use of kmalloc causes deadlocks with dm-crypt.
PERFORMANCE RESULTS
I measured time to encrypt/decrypt a memory buffer of varying sizes with
xts(ecb-aes-aesni) using a tool I wrote ([2]) and the results suggest
that after this patch the performance is either better or comparable for
both small and large buffers. Note that there is a lot of noise in the
measurements, but the overall difference is easy to see.
Old code:
ALGORITHM KEY (b) DATA (B)TIME ENC (ns) TIME DEC (ns)
xts(aes) 256 64 331 328
xts(aes) 384 64 332 333
xts(aes) 512 64 338 348
xts(aes) 256 512 889 920
xts(aes) 384 5121019 993
xts(aes) 512 5121032 990
xts(aes) 256409621522292
xts(aes) 384409624532597
xts(aes) 512409630412641
xts(aes) 256 1638494438027
xts(aes) 384 1638485368925
xts(aes) 512 1638492329417
xts(aes) 256 32768 16383 14897
xts(aes) 384 32768 17527 16102
xts(aes) 512 32768 18483 17322
New code:
ALGORITHM KEY (b) DATA (B)TIME ENC (ns) TIME DEC (ns)
xts(aes) 256 64 328 324
xts(aes) 384 64 324 319
xts(aes) 512 64 320 322
xts(aes) 256 512 476 473
xts(aes) 384 512 509 492
xts(aes) 512 512 531 514
xts(aes) 256409621321829
xts(aes) 384409623572055
xts(aes) 512409621782027
xts(aes) 256 1638469206983
xts(aes) 384 1638485977505
xts(aes) 512 1638478418164
xts(aes) 256 32768 13468 12307
xts(aes) 384 32768 14808 13402
xts(aes) 512 32768 15753 14636
[1] https://lkml.org/lkml/2018/8/23/1315
[2] https://gitlab.com/omos/linux-crypto-bench
Cc: Mikulas Patocka
Signed-off-by: Ondrej Mosnacek
---
crypto/xts.c | 258 ++-
1 file changed, 30 insertions(+), 228 deletions(-)
v1: https://www.spinics.net/lists/linux-crypto/msg34776.html
Changes in v2:
- rebase to latest cryptodev tree
diff --git a/crypto/xts.c b/crypto/xts.c
index ccf55fbb8bc2..6c49e76df8ca 100644
--- a/crypto/xts.c
+++ b/crypto/xts.c
@@ -26,8 +26,6 @@
#include
#include
-#define XTS_BUFFER_SIZE 128u
-
struct priv {
struct crypto_skcipher *child;
struct crypto_cipher *tweak;
@@ -39,19 +37,7 @@ struct xts_instance_ctx {
};
struct rctx {
- le128 buf[XTS_BUFFER_SIZE / sizeof(le128)];
-
le128 t;
-
- le128 *ext;
-
- struct scatterlist srcbuf[2];
- struct scatterlist dstbuf[2];
- struct scatterlist *src;
- struct scatterlist *dst;
-
- unsigned int left;
-
struct skcipher_request subreq;
};
@@ -96,265 +82,81 @@ static int setkey(struct crypto_skcipher *parent, const u8
*key,
return err;
}
-static int post_crypt(struct skcipher_request *req)
+static int xor_tweak(struct rctx *rctx, struct skcipher_request *req)
{
- struct rctx *rctx = skcipher_request_ctx(req);
- le128 *buf = rctx->ext ?: rctx->buf;
- struct skcipher_request *subreq;
const int bs = XTS_BLOCK_SIZE;
struct skcipher_walk w;
- struct scatterlist *sg;
- unsigned offset;
+ le128 t = rctx->t;
Re: [dm-devel] dm bufio: Reduce dm_bufio_lock contention
On Mon 03-09-18 18:23:17, Mikulas Patocka wrote: > > > On Wed, 1 Aug 2018, jing xia wrote: > > > We reproduced this issue again and found out the root cause. > > dm_bufio_prefetch() with dm_bufio_lock enters the direct reclaim and > > takes a long time to do the soft_limit_reclaim, because of the huge > > number of memory excess of the memcg. > > Then, all the task who do shrink_slab() wait for dm_bufio_lock. > > > > Any suggestions for this?Thanks. > > There's hardly any solution because Michal Hocko refuses to change > __GFP_NORETRY behavior. > > The patches 41c73a49df31151f4ff868f28fe4f129f113fa2c and > d12067f428c037b4575aaeb2be00847fc214c24a could reduce the lock contention > on the dm-bufio lock - the patches don't fix the high CPU consumption > inside the memory allocation, but the kernel code should wait less on the > bufio lock. If you actually looked at the bottom line of the problem then you would quickly find out that dm-bufio lock is the least of the problem with the soft limit reclaim. This is a misfeature which has been merged and we have to live with it. All we can do is to discourage people from using it and use much more saner low limit instead. So please stop this stupid blaming, try to understand the reasoning behind my arguments. -- Michal Evil Hocko SUSE Labs -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
