Re: [OE-core] [PATCH 4/4] license: Rework INCOMPATIBLE_LICENSE wildcard handling

2022-03-02 Thread Richard Purdie 
On Wed, 2022-03-02 at 15:09 +, Peter Kjellerstedt wrote:
> > -Original Message-
> > From: openembedded-core@lists.openembedded.org  > c...@lists.openembedded.org> On Behalf Of Richard Purdie
> > Sent: den 2 mars 2022 00:42
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [PATCH 4/4] license: Rework INCOMPATIBLE_LICENSE
> > wildcard handling
> > 
> > The current wildcard handling is badly documented and inconsistently
> > used and understood.
> > 
> > Forcing users to have to use "GPL-3.0-only GPL-3.0-or-later" whilst
> > explict is not very user friendly. Equally, using the current wildcards
> > is ambigious. This supports pre-defined expansions only and at least makes
> > it clear what GPL-3.0* means (it doesn't include the exception licenses).
> > 
> > This is hopefully an acceptable compromise between literal meaning and
> > having something usable.
> > 
> > Non-SPDX forms of license in this field have been dropped and errors are
> > shown for unsupported expansions and unsupported old style license terms.
> > 
> > Users need to carefully consider how to migrate to the new syntax but
> > the meaning should be well defined and clear from here forward.
> > 
> > Signed-off-by: Richard Purdie 
> > ---
> >  meta/classes/license.bbclass  | 41 +
> >  .../oeqa/selftest/cases/incompatible_lic.py   | 86 +++
> >  2 files changed, 70 insertions(+), 57 deletions(-)
> > 
> > diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
> > index 68c022248c8..cb1f46983ac 100644
> > --- a/meta/classes/license.bbclass
> > +++ b/meta/classes/license.bbclass
> > @@ -277,28 +277,27 @@ AVAILABLE_LICENSES := "${@'
> > '.join(available_licenses(d))}"
> > 
> >  def expand_wildcard_licenses(d, wildcard_licenses):
> >  """
> > -Return actual spdx format license names if wildcards are used. We
> > expand
> > -wildcards from SPDXLICENSEMAP flags and AVAILABLE_LICENSES.
> > +There are some common wildcard values users may want to use. Support
> > them
> > +here.
> >  """
> > -import fnmatch
> > -
> > -licenses = wildcard_licenses[:]
> > -spdxmapkeys = d.getVarFlags('SPDXLICENSEMAP').keys()
> > -for wld_lic in wildcard_licenses:
> > -spdxflags = fnmatch.filter(spdxmapkeys, wld_lic)
> > -licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in
> > spdxflags]
> > -# Assume that if we are passed "GPL-3.0" or "*GPL-3.0", then it
> > means
> > -# "-or-later" as well.
> > -if not wld_lic.endswith(("-or-later", "-only", "*", "+")):
> > -spdxflags = fnmatch.filter(spdxmapkeys, wld_lic + "+")
> > -licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in
> > spdxflags]
> > -
> > -spdx_lics = d.getVar('AVAILABLE_LICENSES').split()
> > -for wld_lic in wildcard_licenses:
> > -licenses += fnmatch.filter(spdx_lics, wld_lic)
> > -
> > -licenses = list(set(licenses))
> > -return licenses
> > +licenses = set(wildcard_licenses)
> > +mapping = {
> > +"GPL-3.0*" : ["GPL-3.0-only", "GPL-3.0-or-later"],
> > +"LGPL-3.0*" : ["LGPL-3.0-only", "LGPL-3.0-or-later"],
> 
> If you are adamant about using "*" in these shortcuts (which I 
> still think is a mistake), then you should add these two as well:
> 
> "AGPL-3.0*" : ["AGPL-3.0-only", "AGPL-3.0-or-later"],
> "*GPL-3.0*" : ["GPL-3.0-only", "GPL-3.0-or-later", "LGPL-3.0-only",
> "LGPL-3.0-or-later", "AGPL-3.0-only", "AGPL-3.0-or-later"],
> 
> I believe the last one is what most really want to do, i.e., 
> exclude all GPL-3.0 variants.

I don't think I've seen anything under the AGPL. I don't have a strong
preference for adding it here or not, I was tempted to wait for patches and see
if anyone needed it.

The idea is the wildcards added should remove the need for the latter as people
can say "GPL-3.0* LGPL-3.0*" which I think is manageable.

Cheers,

Richard




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162617): 
https://lists.openembedded.org/g/openembedded-core/message/162617
Mute This Topic: https://lists.openembedded.org/mt/89490747/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 4/4] license: Rework INCOMPATIBLE_LICENSE wildcard handling

2022-03-02 Thread Peter Kjellerstedt 
> -Original Message-
> From: openembedded-core@lists.openembedded.org  c...@lists.openembedded.org> On Behalf Of Richard Purdie
> Sent: den 2 mars 2022 00:42
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH 4/4] license: Rework INCOMPATIBLE_LICENSE
> wildcard handling
> 
> The current wildcard handling is badly documented and inconsistently
> used and understood.
> 
> Forcing users to have to use "GPL-3.0-only GPL-3.0-or-later" whilst
> explict is not very user friendly. Equally, using the current wildcards
> is ambigious. This supports pre-defined expansions only and at least makes
> it clear what GPL-3.0* means (it doesn't include the exception licenses).
> 
> This is hopefully an acceptable compromise between literal meaning and
> having something usable.
> 
> Non-SPDX forms of license in this field have been dropped and errors are
> shown for unsupported expansions and unsupported old style license terms.
> 
> Users need to carefully consider how to migrate to the new syntax but
> the meaning should be well defined and clear from here forward.
> 
> Signed-off-by: Richard Purdie 
> ---
>  meta/classes/license.bbclass  | 41 +
>  .../oeqa/selftest/cases/incompatible_lic.py   | 86 +++
>  2 files changed, 70 insertions(+), 57 deletions(-)
> 
> diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
> index 68c022248c8..cb1f46983ac 100644
> --- a/meta/classes/license.bbclass
> +++ b/meta/classes/license.bbclass
> @@ -277,28 +277,27 @@ AVAILABLE_LICENSES := "${@' 
> '.join(available_licenses(d))}"
> 
>  def expand_wildcard_licenses(d, wildcard_licenses):
>  """
> -Return actual spdx format license names if wildcards are used. We expand
> -wildcards from SPDXLICENSEMAP flags and AVAILABLE_LICENSES.
> +There are some common wildcard values users may want to use. Support them
> +here.
>  """
> -import fnmatch
> -
> -licenses = wildcard_licenses[:]
> -spdxmapkeys = d.getVarFlags('SPDXLICENSEMAP').keys()
> -for wld_lic in wildcard_licenses:
> -spdxflags = fnmatch.filter(spdxmapkeys, wld_lic)
> -licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in 
> spdxflags]
> -# Assume that if we are passed "GPL-3.0" or "*GPL-3.0", then it means
> -# "-or-later" as well.
> -if not wld_lic.endswith(("-or-later", "-only", "*", "+")):
> -spdxflags = fnmatch.filter(spdxmapkeys, wld_lic + "+")
> -licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in 
> spdxflags]
> -
> -spdx_lics = d.getVar('AVAILABLE_LICENSES').split()
> -for wld_lic in wildcard_licenses:
> -licenses += fnmatch.filter(spdx_lics, wld_lic)
> -
> -licenses = list(set(licenses))
> -return licenses
> +licenses = set(wildcard_licenses)
> +mapping = {
> +"GPL-3.0*" : ["GPL-3.0-only", "GPL-3.0-or-later"],
> +"LGPL-3.0*" : ["LGPL-3.0-only", "LGPL-3.0-or-later"],

If you are adamant about using "*" in these shortcuts (which I 
still think is a mistake), then you should add these two as well:

"AGPL-3.0*" : ["AGPL-3.0-only", "AGPL-3.0-or-later"],
"*GPL-3.0*" : ["GPL-3.0-only", "GPL-3.0-or-later", "LGPL-3.0-only", 
"LGPL-3.0-or-later", "AGPL-3.0-only", "AGPL-3.0-or-later"],

I believe the last one is what most really want to do, i.e., 
exclude all GPL-3.0 variants.

//Peter

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162614): 
https://lists.openembedded.org/g/openembedded-core/message/162614
Mute This Topic: https://lists.openembedded.org/mt/89490747/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 4/4] license: Rework INCOMPATIBLE_LICENSE wildcard handling

2022-03-01 Thread Richard Purdie 
The current wildcard handling is badly documented and inconsistently
used and understood.

Forcing users to have to use "GPL-3.0-only GPL-3.0-or-later" whilst
explict is not very user friendly. Equally, using the current wildcards
is ambigious. This supports pre-defined expansions only and at least makes
it clear what GPL-3.0* means (it doesn't include the exception licenses).

This is hopefully an acceptable compromise between literal meaning and
having something usable.

Non-SPDX forms of license in this field have been dropped and errors are
shown for unsupported expansions and unsupported old style license terms.

Users need to carefully consider how to migrate to the new syntax but
the meaning should be well defined and clear from here forward.

Signed-off-by: Richard Purdie 
---
 meta/classes/license.bbclass  | 41 +
 .../oeqa/selftest/cases/incompatible_lic.py   | 86 +++
 2 files changed, 70 insertions(+), 57 deletions(-)

diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index 68c022248c8..cb1f46983ac 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -277,28 +277,27 @@ AVAILABLE_LICENSES := "${@' 
'.join(available_licenses(d))}"
 
 def expand_wildcard_licenses(d, wildcard_licenses):
 """
-Return actual spdx format license names if wildcards are used. We expand
-wildcards from SPDXLICENSEMAP flags and AVAILABLE_LICENSES.
+There are some common wildcard values users may want to use. Support them
+here.
 """
-import fnmatch
-
-licenses = wildcard_licenses[:]
-spdxmapkeys = d.getVarFlags('SPDXLICENSEMAP').keys()
-for wld_lic in wildcard_licenses:
-spdxflags = fnmatch.filter(spdxmapkeys, wld_lic)
-licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in 
spdxflags]
-# Assume that if we are passed "GPL-3.0" or "*GPL-3.0", then it means
-# "-or-later" as well.
-if not wld_lic.endswith(("-or-later", "-only", "*", "+")):
-spdxflags = fnmatch.filter(spdxmapkeys, wld_lic + "+")
-licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in 
spdxflags]
-
-spdx_lics = d.getVar('AVAILABLE_LICENSES').split()
-for wld_lic in wildcard_licenses:
-licenses += fnmatch.filter(spdx_lics, wld_lic)
-
-licenses = list(set(licenses))
-return licenses
+licenses = set(wildcard_licenses)
+mapping = {
+"GPL-3.0*" : ["GPL-3.0-only", "GPL-3.0-or-later"],
+"LGPL-3.0*" : ["LGPL-3.0-only", "LGPL-3.0-or-later"],
+}
+for k in mapping:
+if k in wildcard_licenses:
+licenses.remove(k)
+for item in mapping[k]:
+licenses.add(item)
+
+for l in licenses:
+if l in oe.license.obsolete_license_list():
+bb.fatal("Error, %s is an obsolete license, please use an SPDX 
reference in INCOMPATIBLE_LICENSE" % l)
+if "*" in l:
+bb.fatal("Error, %s is an invalid license wildcard entry" % l)
+
+return list(licenses)
 
 def incompatible_license_contains(license, truevalue, falsevalue, d):
 license = canonical_license(d, license)
diff --git a/meta/lib/oeqa/selftest/cases/incompatible_lic.py 
b/meta/lib/oeqa/selftest/cases/incompatible_lic.py
index c68f920555f..0794d46e6dd 100644
--- a/meta/lib/oeqa/selftest/cases/incompatible_lic.py
+++ b/meta/lib/oeqa/selftest/cases/incompatible_lic.py
@@ -1,10 +1,11 @@
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.utils.commands import bitbake
 
-class IncompatibleLicenseTests(OESelftestTestCase):
+class IncompatibleLicenseTestObsolete(OESelftestTestCase):
 
-def lic_test(self, pn, pn_lic, lic):
-error_msg = 'ERROR: Nothing PROVIDES \'%s\'\n%s was skipped: it has 
incompatible license(s): %s' % (pn, pn, pn_lic)
+def lic_test(self, pn, pn_lic, lic, error_msg=None):
+if not error_msg:
+error_msg = 'ERROR: Nothing PROVIDES \'%s\'\n%s was skipped: it 
has incompatible license(s): %s' % (pn, pn, pn_lic)
 
 self.write_config("INCOMPATIBLE_LICENSE += \"%s\"" % (lic))
 
@@ -12,69 +13,82 @@ class IncompatibleLicenseTests(OESelftestTestCase):
 if error_msg not in result.output:
 raise AssertionError(result.output)
 
-# Verify that a package with an SPDX license (from AVAILABLE_LICENSES)
-# cannot be built when INCOMPATIBLE_LICENSE contains this SPDX license
-def test_incompatible_spdx_license(self):
-self.lic_test('incompatible-license', 'GPL-3.0-only', 'GPL-3.0-only')
-
 # Verify that a package with an SPDX license (from AVAILABLE_LICENSES)
 # cannot be built when INCOMPATIBLE_LICENSE contains an alias (in
 # SPDXLICENSEMAP) of this SPDX license
 def test_incompatible_alias_spdx_license(self):
-self.lic_test('incompatible-license', 'GPL-3.0-only', 'GPLv3')
-
-# Verify that a package with an SPDX license (from AVAILABLE_LICENSES)
-# cannot be built