[dm-devel] Device mapper for temporarily buffering writes on fast device before pushing to slower device

2018-02-20 Thread Drew Hastings
I found stochastic multi-queue and writeboost to be insufficient for this purpose. I'm wondering if anything exists that fits this description: Device mapper creates a "cache" device on a fast device (SSD/NVME, etc)... and writes to the device *always* hit the fast device. Writes are later flushed

Re: [dm-devel] Device mapper for temporarily buffering writes on fast device before pushing to slower device

2018-02-20 Thread Heinz Mauelshagen
dm-writecache (see list archieve) is what you're looking for. On 02/19/2018 09:47 PM, Drew Hastings wrote: I found stochastic multi-queue and writeboost to be insufficient for this purpose. I'm wondering if anything exists that fits this description: Device mapper creates a "cache" device on

[dm-devel] [RFC PATCH 08/20] libmultipath: fix compiler warnings for -Wcast-qual

2018-02-20 Thread Martin Wilck
Fix the warnings that were caused by adding the -Wcast-qual compiler flag in the previous patch. Signed-off-by: Martin Wilck --- kpartx/devmapper.c | 3 ++- libmpathcmd/mpath_cmd.c | 2 +- libmultipath/checkers/rbd.c | 4 ++-- libmultipath/devmapper.c| 3 ++- libmultipath/di

[dm-devel] [RFC PATCH 14/20] libmultipath: print: use generic API for get_x_layout()

2018-02-20 Thread Martin Wilck
Introduce new functions _get_path_layout and _get_multipath_layout using the new "generic" API to determine field widths, and map the old API to them. Furthermore, replace the boolean "header" by an enum with 3 possible values. The new value LAYOUT_RESET_NOT allows calling the get_x_layout functio

[dm-devel] [RFC PATCH 15/20] libmultipath: API for foreign multipath handling

2018-02-20 Thread Martin Wilck
Add an API for "foreign" multipaths. Foreign libraries are loaded from ${multipath_dir}/libforeign-*.so, as we do for checkers. Refer to "foreign.h" for details about the API itself. Like we do for checkers, high-level multipath code isn't supposed to call the API directly, but rather the wrapper

[dm-devel] [RFC PATCH 03/20] libmultipath: get rid of selector "hack" in print.c

2018-02-20 Thread Martin Wilck
By properly linking the path groups with their parent multipath, we don't need this "hack" any more. Signed-off-by: Martin Wilck --- libmultipath/dmparser.c | 2 +- libmultipath/pgpolicies.c | 11 ++- libmultipath/print.c | 6 +++--- libmultipath/structs.c| 10 ++ li

[dm-devel] [RFC PATCH 00/20] "Foreign" NVMe support for multipath-tools

2018-02-20 Thread Martin Wilck
Hello Christophe, This patch series adds limited support for "foreign" multipath devices to multipath and multipathd, and implements the respective API for "native" NVMe multipath devices. The implementation is done using a shared library approach similar to checkers, so other non-dm multipath imp

[dm-devel] [RFC PATCH 06/20] libmultipath/print: use "const" where appropriate

2018-02-20 Thread Martin Wilck
Convert the print.h/print.c code to use "const" qualifiers properly. This is generally considered good programming practice, and the printing code shouldn't change any objects anyway. Signed-off-by: Martin Wilck --- libmultipath/configure.h | 1 - libmultipath/discovery.c | 6 +-- libmultipa

[dm-devel] [RFC PATCH 11/20] libmultipath: add vector_convert()

2018-02-20 Thread Martin Wilck
This is a handy helper for creating one vector from another, mapping each element of the origin vector to an element of the target vector with a given conversion function. It can also be used to "concatenate" vectors by passing in a non-NULL first argument. Signed-off-by: Martin Wilck --- libmul

[dm-devel] [RFC PATCH 17/20] libmultipath/foreign: nvme foreign library

2018-02-20 Thread Martin Wilck
This still contains stubs for path handling and checking, but it's functional for printing already. Signed-off-by: Martin Wilck --- Makefile | 1 + libmultipath/foreign/Makefile | 30 +++ libmultipath/foreign/nvme.c | 444 ++ 3 fi

[dm-devel] [RFC PATCH 04/20] libmultipath: parser: use call-by-value for "snprint" methods

2018-02-20 Thread Martin Wilck
Convert the snprint methods for all keywords to call-by-value, and use "const" qualifier for the "data" argument. This makes sure that "snprint" type functions don't modify the data they're print, helps compile-time correctness checking, and allows more proper "const" cleanups in the future. Signe

[dm-devel] [RFC PATCH 07/20] libmultipath: use "const" in devmapper code

2018-02-20 Thread Martin Wilck
Improve use of "const" qualifiers in libmultipath's devmapper code. Signed-off-by: Martin Wilck --- libmultipath/devmapper.c | 32 libmultipath/devmapper.h | 16 2 files changed, 24 insertions(+), 24 deletions(-) diff --git a/libmultipath/devmap

[dm-devel] [RFC PATCH 09/20] multipath-tools: Makefile.inc: use -Werror=cast-qual

2018-02-20 Thread Martin Wilck
Casting "const" away is often an indicator for wrong code. Add a compiler flag to warn about such possible breakage. Signed-off-by: Martin Wilck --- Makefile.inc | 1 + 1 file changed, 1 insertion(+) diff --git a/Makefile.inc b/Makefile.inc index eb99c36010c1..a5b9d4e3fa74 100644 --- a/Makefile

[dm-devel] [RFC PATCH 13/20] libmultipath: print: convert API to generic data type

2018-02-20 Thread Martin Wilck
Convert higher level API (snprint_multipath_topology() etc) to using the generic multipath API. This will allow "foreign" multipath objects that implement the generic API to be printed exactly like native multipathd objects. The previous API (using "struct multipath*" and "struct path" remains in

[dm-devel] [RFC PATCH 19/20] multipathd: use foreign API

2018-02-20 Thread Martin Wilck
Call into the foreign library code when paths are discovered, uevents are received, and in the checker loop. Furthermore, use the foreign code to print information in the "multipathd show paths", "multipathd show maps", and "multipathd show topology" client commands. We don't support foreign data

[dm-devel] [RFC PATCH 02/20] libmultipath: remove unused "stdout helpers"

2018-02-20 Thread Martin Wilck
Signed-off-by: Martin Wilck --- libmultipath/print.c | 26 -- libmultipath/print.h | 5 - 2 files changed, 31 deletions(-) diff --git a/libmultipath/print.c b/libmultipath/print.c index 65a98247a753..27636c35e5ff 100644 --- a/libmultipath/print.c +++ b/libmultipath/p

[dm-devel] [RFC PATCH 20/20] libmultipath: foreign/nvme: implement path display

2018-02-20 Thread Martin Wilck
implement display of path information for NVMe foreign paths and maps. With this patch, I get output like this for Linux NVMe soft targets: multipathd show topology sys0:NQN:subsysname (uuid.96926ba3-b207-437c-902c-4a4df6538c3f) [nvme] nvme0n1 NVMe,Linux,4.15.0-r size=2097152 features='n/a' hwhan

[dm-devel] [RFC PATCH 12/20] libmultipath: "generic multipath" interface

2018-02-20 Thread Martin Wilck
This patch adds a simplified abstract interface to the multipath data structures. The idea is to allow "foreign" data structures to be treated by libmultipath if they implement the same interface. Currently, the intention is to use this only to provide formatted output about from this interface.

[dm-devel] [RFC PATCH 01/20] multipath(d)/Makefile: add explicit dependency on libraries

2018-02-20 Thread Martin Wilck
Otherwise the binaries won't be re-linked if the libraries change. Signed-off-by: Martin Wilck --- multipath/Makefile | 2 +- multipathd/Makefile | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/multipath/Makefile b/multipath/Makefile index 654568af3576..0828a8f72db7 10064

[dm-devel] [RFC PATCH 18/20] multipath: use foreign API

2018-02-20 Thread Martin Wilck
Use the "foreign" code to print information about multipath maps owned by foreign libraries in print mode (multipath -ll, -l). Signed-off-by: Martin Wilck --- multipath/main.c | 13 + 1 file changed, 13 insertions(+) diff --git a/multipath/main.c b/multipath/main.c index a0c750e6f62

[dm-devel] [RFC PATCH 16/20] libmultipath/print: add "%G - foreign" wildcard

2018-02-20 Thread Martin Wilck
This adds a format field to identify foreign maps as such, and uses it in default-formatted topology output (generic_style()). Signed-off-by: Martin Wilck --- libmultipath/generic.c | 4 +++- libmultipath/print.c | 14 ++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --gi

[dm-devel] [RFC PATCH 05/20] libmultipath: don't update path groups when printing

2018-02-20 Thread Martin Wilck
Updating the prio values for printing makes no sense. The user wants to see the prio values multipath is actually using for path group selection, and updating the values here means actually lying to the user if the prio values have changed, but multipathd hasn't updated them internally. If we real

[dm-devel] [RFC PATCH 10/20] libmultipath: add vector_free_const()

2018-02-20 Thread Martin Wilck
... to dispose of constant vectors (const struct _vector*). Signed-off-by: Martin Wilck --- libmultipath/vector.h | 1 + 1 file changed, 1 insertion(+) diff --git a/libmultipath/vector.h b/libmultipath/vector.h index 5cfd4d060412..3f6e579ae19a 100644 --- a/libmultipath/vector.h +++ b/libmultipa