The 04/09/2021 18:23, Mike Snitzer wrote:
> On Fri, Apr 09 2021 at 7:48am -0400,
> Sergei Shtepa wrote:
>
> > I think I'm ready to suggest the next version of block device interposer
> > (blk_interposer). It allows to redirect bio requests to other block
> > devic
Signed-off-by: Sergei Shtepa
---
drivers/md/dm-snap.c | 15 +++
1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/drivers/md/dm-snap.c b/drivers/md/dm-snap.c
index 11890db71f3f..81e8e3bb6d25 100644
--- a/drivers/md/dm-snap.c
+++ b/drivers/md/dm-snap.c
@@ -2685,11
() and __dm_resume(). The flag
DM_SUSPEND_DETACH_IP_FLAG was added for this purpose.
* dm_submit_bio() sets BIO_INTERPOSED for each bio from the interposer.
Signed-off-by: Sergei Shtepa
---
drivers/md/dm-core.h | 1 +
drivers/md/dm-ioctl.c | 95 ++---
drivers/md/dm
Signed-off-by: Sergei Shtepa
---
block/genhd.c | 51 +++
fs/block_dev.c| 3 +++
include/linux/blk_types.h | 6 +
include/linux/blkdev.h| 32
4 files changed, 92 insertions(+)
diff --git a/block
the bd_interposer_lock.
Signed-off-by: Sergei Shtepa
---
block/bio.c | 2 +
block/blk-core.c | 194 ++-
2 files changed, 108 insertions(+), 88 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 50e579088aca..6fc9e8f395a6 100644
--- a/block/bio.c
snapshots.
Thank you to everyone who was able to take the time to review
the previous versions.
I hope that this time I achieved the required quality.
Thanks,
Sergei.
Sergei Shtepa (4):
Adds blk_interposer. It allows to redirect bio requests to another
block device.
Adds the blk_interposers logic
The 03/24/2021 11:18, Christoph Hellwig wrote:
> On Tue, Mar 23, 2021 at 02:48:36PM +0300, Sergei Shtepa wrote:
> > When the blk_crypto_bio_prep() function returns false, the processing
> > of the bio request must end. Repeated access to blk_crypto_bio_prep()
> > for t
to access the freed data or access the null pointer.
At the same time, the implementation of the negative branch of the
blk_crypto_bio_prep() function implemented correctly in the
__submit_bio_noacct() and __submit_bio() functions.
Sergei Shtepa (1):
block: fix potential infinite loop
to leave the processing of the failed bio and
go to the next one from the bio_list.
The error can only occur when using inline encryption on
request-based blk-mq devices and something went wrong in the
__blk_crypto_bio_prep().
Signed-off-by: Sergei Shtepa
---
block/blk-core.c | 7 +++
1 file
The 03/17/2021 22:13, Mike Snitzer wrote:
> On Wed, Mar 17 2021 at 2:14pm -0400,
> Sergei Shtepa wrote:
>
> > The 03/17/2021 18:04, Mike Snitzer wrote:
> > > On Wed, Mar 17 2021 at 8:22am -0400,
> > > Sergei Shtepa wrote:
> > >
> > > >
The 03/17/2021 18:04, Mike Snitzer wrote:
> On Wed, Mar 17 2021 at 8:22am -0400,
> Sergei Shtepa wrote:
>
> > The 03/17/2021 06:03, Ming Lei wrote:
> > > On Tue, Mar 16, 2021 at 07:35:44PM +0300, Sergei Shtepa wrote:
> > > > The 03/16/2021 11:09, Ming Lei w
The 03/17/2021 06:03, Ming Lei wrote:
> On Tue, Mar 16, 2021 at 07:35:44PM +0300, Sergei Shtepa wrote:
> > The 03/16/2021 11:09, Ming Lei wrote:
> > > On Fri, Mar 12, 2021 at 06:44:54PM +0300, Sergei Shtepa wrote:
> > > > bdev_interposer allows to redirect b
The 03/16/2021 11:09, Ming Lei wrote:
> On Fri, Mar 12, 2021 at 06:44:54PM +0300, Sergei Shtepa wrote:
> > bdev_interposer allows to redirect bio requests to another devices.
> >
> > Signed-off-by: Sergei Shtepa
> > ---
> > block/bio.c | 2 ++
&g
Thanks!
I've already started doing something like that.
I'm glad we're thinking in the same direction.
The 03/14/2021 12:30, Christoph Hellwig wrote:
> On Fri, Mar 12, 2021 at 06:44:55PM +0300, Sergei Shtepa wrote:
> > DM_INTERPOSED_FLAG allow to create DM targets on "the fly".
> > Underlying block device opens without a flag FMODE_EXCL.
> > DM target receives b
The 03/14/2021 12:28, Christoph Hellwig wrote:
> On Fri, Mar 12, 2021 at 06:44:54PM +0300, Sergei Shtepa wrote:
> > bdev_interposer allows to redirect bio requests to another devices.
>
> I think this warrants a somewhat more detailed description.
>
> The code itself looks
The 03/12/2021 22:00, Mike Snitzer wrote:
> On Fri, Mar 12 2021 at 10:44am -0500,
> Sergei Shtepa wrote:
>
> > DM_INTERPOSED_FLAG allow to create DM targets on "the fly".
> > Underlying block device opens without a flag FMODE_EXCL.
> > DM target rece
e to the racey nature of this state once the mutex is dropped.
>
> The code only uses it for asserts in the form of WARN_ONs.
But perhaps it is possible to come up with a more elegant solution?
I'll think about it.
--
Sergei Shtepa
Veeam Software developer.
DM_INTERPOSED_FLAG allow to create DM targets on "the fly".
Underlying block device opens without a flag FMODE_EXCL.
DM target receives bio from the original device via
bdev_interposer.
Signed-off-by: Sergei Shtepa
---
drivers/md/dm-core.h | 3 ++
drivers/md/dm-ioctl.c
blk_mq_is_queue_frozen() allow to assert that the queue is frozen.
Signed-off-by: Sergei Shtepa
---
block/blk-mq.c | 13 +
include/linux/blk-mq.h | 1 +
2 files changed, 14 insertions(+)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index d4d7c1caa439..2f188a865024 100644
nks,
Sergei.
Sergei Shtepa (3):
block: add blk_mq_is_queue_frozen()
block: add bdev_interposer
dm: add DM_INTERPOSED_FLAG
block/bio.c | 2 ++
block/blk-core.c | 57
block/blk-mq.c| 13
block/genh
bdev_interposer allows to redirect bio requests to another devices.
Signed-off-by: Sergei Shtepa
---
block/bio.c | 2 ++
block/blk-core.c | 57 +++
block/genhd.c | 54 +
include/linux
The 03/10/2021 15:34, Christoph Hellwig wrote:
> On Wed, Mar 10, 2021 at 08:28:12AM +0300, Sergei Shtepa wrote:
> > > So instead of doing this shoudn't the interposer just always submit to the
> > > whole device? But if we keep it, the logic in this funtion should go
>
The 03/09/2021 20:35, Christoph Hellwig wrote:
> On Wed, Mar 03, 2021 at 03:30:18PM +0300, Sergei Shtepa wrote:
> > DM_INTERPOSED_FLAG allow to create dm targets on "the fly".
> > Underlying block device opens without a flag FMODE_EXCL.
> > Dm target receives b
p_submit_bio)
> > +{
>
> > + /* Check if it is really our interposer. */
> > + if (WARN_ON(bdev->bd_interposer->ip_submit_bio != ip_submit_bio))
> > + goto out;
>
> I don't really see any need to pass ip_submit_bio just for this check.
>
>
The four simplest IOCTL's allow to create new filters, remove them
and pass control commands specific to each target.
Signed-off-by: Sergei Shtepa
---
drivers/md/Makefile | 2 +-
drivers/md/dm-ioctl.c | 22 ++
drivers/md/flt-ctl.c | 25
y possible to create a DF independently of the DM.
However, I think that DF can use a significant part of the code already
existing in the DM without creating duplication.
I look forward to your feedback on device filter.
Sergei Shtepa (1):
dm: adds an IOCTL to work with device-filters
drivers/m
blk_interposer allows to intercept bio requests, remap bio
to another devices or add new bios.
Signed-off-by: Sergei Shtepa
---
block/bio.c | 2 +
block/blk-core.c | 36 +++
block/genhd.c | 93 +++
include/linux
DM_INTERPOSED_FLAG allow to create dm targets on "the fly".
Underlying block device opens without a flag FMODE_EXCL.
Dm target receives bio from the original device via
blk_interposer.
Signed-off-by: Sergei Shtepa
---
drivers/md/dm-core.h | 6 ++
drivers/md/dm-ioctl.c
dm-interposer.c/. h contains code for working with blk_interposer
and provides an API for interposer in device-mapper.
Signed-off-by: Sergei Shtepa
---
drivers/md/Makefile| 2 +-
drivers/md/dm-interposer.c | 258 +
drivers/md/dm-interposer.h | 40
the required quality.
Thanks,
Sergei.
Sergei Shtepa (4):
block: add blk_mq_is_queue_frozen()
block: add blk_interposer
dm: introduce dm-interposer
dm: add DM_INTERPOSED_FLAG
block/bio.c | 2 +
block/blk-core.c | 36 +
block/blk-mq.c| 12
blk_mq_is_queue_frozen() allow to assert that the queue is frozen.
Signed-off-by: Sergei Shtepa
---
block/blk-mq.c | 12
include/linux/blk-mq.h | 1 +
2 files changed, 13 insertions(+)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index d4d7c1caa439..d5e7122789fc 100644
The 02/15/2021 19:08, Mike Snitzer wrote:
> On Mon, Feb 15 2021 at 5:34am -0500,
> Sergei Shtepa wrote:
>
> > The 02/12/2021 19:06, Mike Snitzer wrote:
> > > On Fri, Feb 12 2021 at 6:34am -0500,
> > > Sergei Shtepa wrote:
> > >
> &
The 02/12/2021 19:06, Mike Snitzer wrote:
> On Fri, Feb 12 2021 at 6:34am -0500,
> Sergei Shtepa wrote:
>
> > The 02/11/2021 20:51, Mike Snitzer wrote:
> > > On Tue, Feb 09 2021 at 9:30am -0500,
> > > Sergei Shtepa wrote:
> > >
> > > &
The 02/11/2021 20:51, Mike Snitzer wrote:
> On Tue, Feb 09 2021 at 9:30am -0500,
> Sergei Shtepa wrote:
>
> > The 'noexcl' option allow to open underlying block-device
> > without FMODE_EXCL.
> >
> > Signed-off-by: Sergei Shtepa
> > ---
The 'noexcl' option allow to open underlying block-device
without FMODE_EXCL.
Signed-off-by: Sergei Shtepa
---
drivers/md/dm-linear.c| 14 +-
drivers/md/dm-table.c | 14 --
drivers/md/dm.c | 26 +++---
drivers/md/dm.h
New 'noexcl' option allow to open underlying block-device
without FMODE_EXCL flag.
Signed-off-by: Sergei Shtepa
---
.../admin-guide/device-mapper/linear.rst | 26 ---
1 file changed, 23 insertions(+), 3 deletions(-)
diff --git a/Documentation/admin-guide/device-mapper
New ioctl DM_DEV_REMAP_CMD allow to remap bio requests
from regular block device to dm device.
Signed-off-by: Sergei Shtepa
---
drivers/md/dm-core.h | 20 ++
drivers/md/dm-ioctl.c | 35
drivers/md/dm.c | 375 +-
include/uapi
blk_mq_is_queue_frozen() allow to assert that the queue is frozen.
Signed-off-by: Sergei Shtepa
---
block/blk-mq.c | 13 +
include/linux/blk-mq.h | 1 +
2 files changed, 14 insertions(+)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index f285a9123a8b..924ec26fae5f 100644
blk_interposer allows to intercept bio requests, remap bio
to another devices or add new bios.
Signed-off-by: Sergei Shtepa
---
block/bio.c | 2 +
block/blk-core.c | 35
block/genhd.c | 86 +++
include
remap_and_filter - describes the new features that
blk_interposer provides for device mapper.
Signed-off-by: Sergei Shtepa
---
.../admin-guide/device-mapper/index.rst | 1 +
.../device-mapper/remap_and_filter.rst| 132 ++
2 files changed, 133 insertions
...@veeam.com/
I tried to offer a rather cumbersome blk-filter and a monster-like
blk-snap module for creating snapshots.
Thank you to everyone who was able to take the time to review
the previous versions.
I hope that this time I achieved the required quality.
Thanks,
Sergei.
Sergei Shtepa (6
The 02/03/2021 11:18, Mike Snitzer wrote:
> On Wed, Feb 03 2021 at 10:53am -0500,
> Sergei Shtepa wrote:
>
> > blk_interposer allows to intercept bio requests, remap bio to another
> > devices or add new bios.
> >
> > Signed-off-by: Sergei Shtepa
> >
Thank you, I'll fix it all.
--
Sergei Shtepa
Veeam Software developer.
New 'noexcl' option allow to open underlying block-device
without FMODE_EXCL flag.
Signed-off-by: Sergei Shtepa
---
.../admin-guide/device-mapper/linear.rst | 26 ---
1 file changed, 23 insertions(+), 3 deletions(-)
diff --git a/Documentation/admin-guide/device-mapper
blk_mq_is_queue_frozen() allow to assert that the queue is frozen.
Signed-off-by: Sergei Shtepa
---
block/blk-mq.c | 13 +
include/linux/blk-mq.h | 1 +
2 files changed, 14 insertions(+)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index f285a9123a8b..924ec26fae5f 100644
for creating snapshots.
Thank you to everyone who was able to take the time to review
the previous versions.
I hope that this time I achieved the required quality.
Thanks,
Sergei.
Sergei Shtepa (6):
docs: device-mapper: add remap_and_filter
block: add blk_interposer
block: add
blk_interposer allows to intercept bio requests, remap bio to another devices
or add new bios.
Signed-off-by: Sergei Shtepa
---
block/bio.c | 2 +
block/blk-core.c | 33
block/genhd.c | 82 +++
include
New ioctl DM_DEV_REMAP_CMD allow to remap bio requests
from regular block device to dm device.
Signed-off-by: Sergei Shtepa
---
drivers/md/dm-core.h | 20 ++
drivers/md/dm-ioctl.c | 39
drivers/md/dm.c | 376 +-
include/uapi
remap_and_filter - describes the new features that
blk_interposer provides for device mapper.
Signed-off-by: Sergei Shtepa
---
.../admin-guide/device-mapper/index.rst | 1 +
.../device-mapper/remap_and_filter.rst| 132 ++
2 files changed, 133 insertions
The 'noexcl' option allow to open underlying block-device
without FMODE_EXCL.
Signed-off-by: Sergei Shtepa
---
drivers/md/dm-linear.c| 14 +-
drivers/md/dm-table.c | 12 +++-
drivers/md/dm.c | 24 +---
drivers/md/dm.h
`s
data.
blk_interposer allows to get full access to bio.
For use in the DM, we must also be able to add new bio's.
Summary:
For device-mapper purposes, bpf_io_filter is not suitable.
bpf_io_filter in this form is probably convenient to use for monitoring and
studying the block layer.
For the security task, I would suggest making a separate module and using
blk_interposer to intercept bio requests. This will give more flexible
functionality and better performance.
--
Sergei Shtepa
Veeam Software developer.
The 12/09/2020 17:36, Mike Snitzer wrote:
> On Wed, Dec 09 2020 at 8:01am -0500,
> Sergei Shtepa wrote:
>
> > This sample demonstrates how to use blk_interposer.
> > It show how to properly connect the interposer module to kernel,
> > and perform the simplest moni
The 12/09/2020 16:51, Mike Snitzer wrote:
> On Wed, Dec 09 2020 at 8:01am -0500,
> Sergei Shtepa wrote:
>
> > Hi all.
> >
> > I try to suggest the Block Layer Interposer (blk_interposer) again.
> > It`s allows to intercept bio requests, remap bio to an
as possible.
Sergei Shtepa (3):
block: blk_interposer - Block Layer Interposer
block: blk_interposer - sample
block: blk_interposer - sample config
block/blk-core.c| 32 +++
include/linux/blk_types.h | 6 +-
include/linux/genhd.h | 12
Attaches the blk_interposer sample to the kernel assembly.
Signed-off-by: Sergei Shtepa
---
samples/Kconfig | 7 +++
samples/Makefile | 1 +
2 files changed, 8 insertions(+)
diff --git a/samples/Kconfig b/samples/Kconfig
index 0ed6e4d71d87..72e2a9399e10 100644
--- a/samples/Kconfig
+++ b
already been interposed or cannot be interposed.
* The bi_flags member of the BIO structure is expanded
to a 32-bit variable to accommodate the new flag.
* New structure blk_interposer contains the interposer
callback function.
Signed-off-by: Sergei Shtepa
---
block/blk-core.c
This sample demonstrates how to use blk_interposer.
It show how to properly connect the interposer module to kernel,
and perform the simplest monitoring of the number of bio.
Signed-off-by: Sergei Shtepa
---
samples/blk_interposer/Makefile | 2 +
samples/blk_interposer/blk
The dm_get_live_table() function makes RCU read lock and
dm_put_live_table() should call even when dm_table map was not found.
Sergei Shtepa (1):
Fix bug with RCU locking in dm_blk_report_zones().
drivers/md/dm.c | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
--
2.20.1
The dm_get_live_table() function makes RCU read lock and
dm_put_live_table() should call even when dm_table map was not found.
Signed-off-by: Sergei Shtepa
---
drivers/md/dm.c | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index
more into the direction of hooking into ->submit_bio,
> seeing that it's a NULL pointer for most (all?) block drivers.
>
> But sure, I'll check how the interposer approach would turn out.
If anyone will do the patch blk-interposer, please add me to CC.
I will try to offer my module that will use blk-interposer.
--
Sergei Shtepa
Veeam Software developer.
The 10/22/2020 13:28, Damien Le Moal wrote:
> On 2020/10/22 18:43, Sergei Shtepa wrote:
> >
> > Maybe, but the problem is that I can't imagine how to implement
> > dm-intercept yet.
> > How to use dm to implement interception without changing the stack
> > of
The 10/22/2020 08:58, Hannes Reinecke wrote:
> On 10/21/20 4:10 PM, Sergei Shtepa wrote:
> > The 10/21/2020 16:31, Hannes Reinecke wrote:
> >> I do understand where you are coming from, but then we already have a
> >> dm-snap which does exactly what you want to achieve.
The 10/21/2020 16:07, Matthew Wilcox wrote:
> On Wed, Oct 21, 2020 at 03:55:55PM +0300, Sergei Shtepa wrote:
> > The 10/21/2020 14:44, Matthew Wilcox wrote:
> > > I don't understand why O_DIRECT gets to bypass the block filter. Nor do
> > > I understand why anybody
rcept',
which would use the re-route mechanism from the dm?
I think I may be able to implement it, if you describe your idea in more
detail.
--
Sergei Shtepa
Veeam Software developer.
the blk_qc_t type when
intercepting a request, because at that time it does not know which queue
the request will fall into.
If function submit_bio() will always return BLK_QC_T_NONE - I think the
algorithm of the __blk dev_direct_IO() will not work correctly.
If we need to intercept direct acce
tch.
Yes, it will be a lot of work. But I hope that the code architecture
and splitting entities into separate files will help.
If someone can advise how to divide a module into a chain of patches,
I will be happy. I do not dare to divide it without losing meaning.
--
Sergei Shtepa
Veeam Software developer.
the block layer where needed.
I'll think about it. Personally, it seems to me that this separation
does not make it easier to understand the code.
It is important for me to know immediately where the function is called,
and this determines its behavior.
--
Sergei Shtepa
Veeam Software developer.
The 10/21/2020 12:14, Johannes Thumshirn wrote:
> On 21/10/2020 11:04, Sergei Shtepa wrote:
> > + help
> > + Enabling this lets third-party kernel modules intercept
> > + bio requests for any block device. This allows them to implement
>
> The "third-p
#pragma once still banned? I think need to add a check for this
./scripts/checkpatch.pl.
Comment code style - ok, thank you.
--
Sergei Shtepa
Veeam Software developer.
Signed-off-by: Sergei Shtepa
---
block/Kconfig | 11 ++
block/Makefile | 1 +
block/blk-core.c| 52 +--
block/blk-filter-internal.h | 29
block/blk-filter.c | 286
block/partitions/core.c | 14
to be done:
* refactor the module interface for interaction with a user-space code -
it is already clear that the implementation of some calls can be
improved.
Your feedback would be greatly appreciated!
Sergei Shtepa (2):
Block layer filter - second version
blk-snap - snapshots
like to avoid impacting the IO scheduler operations.
--
Sergei Shtepa
Veeam Software developer.
the module
is to be created;
* autotests need to be created;
* regression testing has been conducted in a cursory manner, so there is
a chance for mistakes.
Sergei Shtepa (1):
blk-snap - Block snapshot module This module implements snapshot and
changed block tracking functionality
, but frankly speaking we were simply too shy to go
this route when we first started 4 years ago, with just a handful of users and
an unclear demand/future. But now in 2020, it’s a different story.
We’ll get started on the in-tree kernel mode right away, and will reconvene
once ready.
--
Sergei
The 08/28/2020 16:54, Jens Axboe wrote:
> On 8/27/20 1:13 PM, Sergei Shtepa wrote:
> > Hello everyone! Requesting for your comments and suggestions.
> >
> > We propose new kernel API that should be beneficial for out-of-tree
> > kernel modules of multiple backup vend
() multiple filters can be stacked at different
altitudes when bio request is intercepted, it can be passed to filter
at lower level or it can be sent for completion
Signed-off-by: Sergei Shtepa
---
block/Kconfig | 11 ++
block/Makefile | 1 +
block/blk-core.c
] can no longer
hook/replace it to intercept bio requests. And fops in struct gendisk
is declared as const and cannot be hooked as well.
We would appreciate your feedback!
[1] https://github.com/veeam/veeamsnap
Sergei Shtepa (1):
block io layer filters api
block/Kconfig | 11
78 matches
Mail list logo
