Re: [dm-devel] multipath-tools licenses (was Re: [PATCH] multipath-tools: replace FSF address with a www pointer)

2018-04-06 Thread Greg KH 
On Fri, Apr 06, 2018 at 06:10:48PM +0200, Xose Vazquez Perez wrote:
> On 03/28/2018 05:14 PM, Martin Wilck wrote:
> 
> > On Wed, 2018-03-28 at 00:24 +0200, Xose Vazquez Perez wrote:
> 
> > Multiple licenses are acceptable for multipath-tools, too. Yet we need
> > to understand, and clearly communicate, which license applies to which
> > source file, and what that means for the binaries and libraries that
> > are part of the package. And, needless to say, reducing the number of
> > licenses and getting rid of the obsolete LGPL-2.0 would simplify
> > matters significantly, both for us and other parties.
> 
> It would be nice to have the old cvs repo, from 2003-09-18 multipath-0.0.1 to
> 2005-05-23 multipath-tools-0.4.5, online. Or converted to git.
> 
> >> And the SPDX License Identifier is being used:
> >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tr
> >> ee/Documentation/process/license-rules.rst
> > 
> > Yeah, it's probably a good idea to do that. I'm not sure if it should
> > replace the boilerplate license header or just be added on top of it.
> > Either way, when we do this, we should make sure that we understand
> > which license covers the individual files, in particular those that
> > currently have no license header. We're assuming that these are covered
> > by COPYING, but is that actually true for all 130+ files?
> > 
> > This shouldn't be taken too lightly. Assume you add an "LGPL-2.1" SPDX
> > header to some file. Company X links to the file in it's proprietary
> > product. Later, company Y finds some of its own GPL-2.0 licensed code
> > in the same file and sues X over 100 million for GPL breakage. Now X
> > claims the money back from the person who inserted the misleading
> > license header in the file ...
> > 
> > That sounds paranoid and exaggerated, but I've heard exactly arguments
> > like this in discussions about proprietary software using FLOSS. It's
> > the kind of thing Black Duck and similar companies make money with.
> 
> 
> Kernel guys are replacing boiler plate text with a SPDX tag.
> I suppose, by advice and with assistance of the lawyers of The Linux 
> Foundation.
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b24413180f5600bcb3bb70fbed5cf186b60864bd
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a04c7278d3042cb30c8a66197d900209a4f2417c

Not just the LF lawyers, but the lawyers from almost all major Linux
copyright holders (Intel, Google, Red Hat, IBM, and so on...)

Here's the rules for how we structure the tags and why:
https://www.kernel.org/doc/html/latest/process/license-rules.html

If you are going to use SPDX for your tools (and you should!), you might
want to look at the REUSE Initiative:
https://reuse.software/

That provides a great framework for how you should probably tag things
in your codebase.

Hope this helps,

greg k-h

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] multipath-tools licenses (was Re: [PATCH] multipath-tools: replace FSF address with a www pointer)

2018-04-06 Thread Xose Vazquez Perez 
On 03/28/2018 05:14 PM, Martin Wilck wrote:

> On Wed, 2018-03-28 at 00:24 +0200, Xose Vazquez Perez wrote:

> Multiple licenses are acceptable for multipath-tools, too. Yet we need
> to understand, and clearly communicate, which license applies to which
> source file, and what that means for the binaries and libraries that
> are part of the package. And, needless to say, reducing the number of
> licenses and getting rid of the obsolete LGPL-2.0 would simplify
> matters significantly, both for us and other parties.

It would be nice to have the old cvs repo, from 2003-09-18 multipath-0.0.1 to
2005-05-23 multipath-tools-0.4.5, online. Or converted to git.

>> And the SPDX License Identifier is being used:
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tr
>> ee/Documentation/process/license-rules.rst
> 
> Yeah, it's probably a good idea to do that. I'm not sure if it should
> replace the boilerplate license header or just be added on top of it.
> Either way, when we do this, we should make sure that we understand
> which license covers the individual files, in particular those that
> currently have no license header. We're assuming that these are covered
> by COPYING, but is that actually true for all 130+ files?
> 
> This shouldn't be taken too lightly. Assume you add an "LGPL-2.1" SPDX
> header to some file. Company X links to the file in it's proprietary
> product. Later, company Y finds some of its own GPL-2.0 licensed code
> in the same file and sues X over 100 million for GPL breakage. Now X
> claims the money back from the person who inserted the misleading
> license header in the file ...
> 
> That sounds paranoid and exaggerated, but I've heard exactly arguments
> like this in discussions about proprietary software using FLOSS. It's
> the kind of thing Black Duck and similar companies make money with.


Kernel guys are replacing boiler plate text with a SPDX tag.
I suppose, by advice and with assistance of the lawyers of The Linux Foundation.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b24413180f5600bcb3bb70fbed5cf186b60864bd
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a04c7278d3042cb30c8a66197d900209a4f2417c


This template could be good enough and neat for multipath-tools:
// SPDX-License-Identifier: 
// File-Originally-From: >
//  , .
// Author(s): >
//...

e.g.

// SPDX-License-Identifier: GPL-2.0-or-later
// File-Originally-From: linux-tool 
// Copyright 1999, 2001-2018, Foo Corp.
// Author(s): Bar 

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [BUG] dm-thin metadata operation failed due to -ENOSPC returned by dm_pool_alloc_data_block() after processing DISCARD bios

2018-04-06 Thread Zdenek Kabelac 

Dne 3.4.2018 v 06:07 Dennis Yang napsal(a):

Hi,

Recently we have came across an issue that dm-thin pool will be
switched to READ_ONLY mode because dm_pool_alloc_data_block() returns
-ENOSPC. AFAIK, this should not happen since alloc_data_block() will
check if there is any free space (and commit metadata if it first
reports no free space) before it allocates pool block. In addition,
total virtual space of all thin volumes is smaller than the pool
physical space in my testing environment which makes pool impossible
to run out of space.

This issue could be easily reproduced by the following steps.

1) Create a thin pool and a slightly smaller thin volume

sudo dmsetup create meta --table "0 4000 linear /dev/sdf 0"
sudo dmsetup create data --table "0 1024 linear /dev/md125 0"
sudo dd if=/dev/zero of=/dev/mapper/meta bs=1M count=1
sudo dmsetup create pool --table "0 1024 thin-pool /dev/mapper/meta 
/dev/mapper/data 1024 0 2 skip_block_zeroing error_if_no_space"
sudo dmsetup message pool 0 "create_thin 0"
sudo dmsetup create thin --table "0 10238976 thin /dev/mapper/pool 0"


2) Make a filesystem and mount it

sudo mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0 /dev/mapper/thin
sudo mount /dev/mapper/thin /mnt


3) Write a file to mount point until it takes all the space

sudo dd if=/dev/zero of=/mnt/zero.img bs=1M


4) Remove this file and trim mount point

sudo rm /mnt/zero.img
sudo fstrim /mnt


Repeat step 3 and 4 multiple times and the pool will be switched to
READ_ONLY mode and need_checks flag will be set. Kernel message shows
the following messages.
[ 3952.723937] device-mapper: thin: 252:2: metadata operation
'dm_pool_alloc_data_block' failed: error = -28
[ 3952.723940] device-mapper: thin: 252:2: aborting current metadata transaction
[ 3952.725860] device-mapper: thin: 252:2: switching pool to read-only mode

This root cause of this issue is that dm-thin will first remove
mapping and increase corresponding blocks' reference count to prevent
them from being reused before DISCARD bios get processed by the
underlying layers. However. increasing blocks' reference count could
also increase the nr_allocated_this_transaction in struct sm_disk
which makes smd->old_ll.nr_allocated +
smd->nr_allocated_this_transaction bigger than smd->old_ll.nr_blocks.
In this case, alloc_data_block() will never commit metadata to reset
the begin pointer of struct sm_disk, because sm_disk_get_nr_free()
always return an underflow value.

If you need more information, please feel free to let me know.


Hi

Just forgotten to mention - tracked through this BZ:

https://bugzilla.redhat.com/1563697

Regards

Zdenek

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

[dm-devel] [PATCH] 11-dm-mpath.rules: dont't run "multipath -U" during coldplug

2018-04-06 Thread Martin Wilck 
When udevadm trigger is run after switching root, lots of simultaneous uevents
for both path and multipath devices arrive. In may happen that when "multipath
-U" is called on a dm device, the path's uevents haven't finished yet, thus
the paths aren't found in the udev db, and multipath -U erroneously concludes
that there are no usable paths. Avoid that by skipping "multipath -U" during
coldplug.

Fixes: "ce5ea6a 11-dm-mpath.rules: multipath -U for READY check"
Signed-off-by: Martin Wilck 
---
 multipath/11-dm-mpath.rules | 4 
 1 file changed, 4 insertions(+)

diff --git a/multipath/11-dm-mpath.rules b/multipath/11-dm-mpath.rules
index 03ac5da..07320a1 100644
--- a/multipath/11-dm-mpath.rules
+++ b/multipath/11-dm-mpath.rules
@@ -26,6 +26,10 @@ ENV{DM_NR_VALID_PATHS}=="0", ENV{MPATH_DEVICE_READY}="0", 
GOTO="mpath_action"
 ENV{MPATH_SBIN_PATH}="/sbin"
 TEST!="$env{MPATH_SBIN_PATH}/multipath", ENV{MPATH_SBIN_PATH}="/usr/sbin"
 
+# Don't run multipath -U during "coldplug" after switching root,
+# because paths are just being added to the udev db.
+ACTION=="add", ENV{.MPATH_DEVICE_READY_OLD}=="1", GOTO="paths_ok"
+
 # Check the map state directly with multipath -U.
 # This doesn't attempt I/O on the device.
 PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -U %k", GOTO="paths_ok"
-- 
2.16.1

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel