Re: [edk2-devel] [PATCH 01/15] .pytool/Plugin: Add a plugin LicenseCheck

2020-07-29 Thread Liming Gao 
Shenglei:
  Readme mentions BSD-2-Clause Plus Patent License, but no BSD-3-Clause Plus 
Patent License. So, please check BSD-2-Clause Plus Patent License only. 

Thanks
Liming
-Original Message-
From: Zhang, Shenglei  
Sent: 2020年7月29日 14:07
To: Kinney, Michael D ; devel@edk2.groups.io
Cc: Sean Brogan ; Bret Barkelew 
; Gao, Liming 
Subject: RE: [PATCH 01/15] .pytool/Plugin: Add a plugin LicenseCheck

Five sorts of license are accepted in edk2(described in readme). They are BSD 
(2-clause), BSD (3-clause), MIT, Python-2.0 and zlib.
Minus non bsd plus patent license, they are BSD (2-clause) and BSD (3-clause).

Thanks,
Shenglei

> -Original Message-
> From: Kinney, Michael D 
> Sent: Wednesday, July 29, 2020 7:29 AM
> To: Zhang, Shenglei ; devel@edk2.groups.io; 
> Kinney, Michael D 
> Cc: Sean Brogan ; Bret Barkelew 
> ; Gao, Liming 
> Subject: RE: [PATCH 01/15] .pytool/Plugin: Add a plugin LicenseCheck
> 
> Where did the requirement for BSD-3-Clause-Patent come from?
> 
> Thanks,
> 
> Mike
> 
> > -Original Message-
> > From: Zhang, Shenglei 
> > Sent: Monday, July 20, 2020 1:37 AM
> > To: devel@edk2.groups.io
> > Cc: Sean Brogan ; Bret Barkelew 
> > ; Kinney, Michael D 
> > ; Gao, Liming 
> > Subject: [PATCH 01/15] .pytool/Plugin: Add a plugin LicenseCheck
> >
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2833
> > Add a plugin to check license conflict for new added files in a 
> > patch. It will report out errors when meeting files which are now 
> > contributed under BSD-2-Clause- Patent or BSD-3-Clause-Patent.
> >
> > Cc: Sean Brogan 
> > Cc: Bret Barkelew 
> > Cc: Michael D Kinney 
> > Cc: Liming Gao 
> > Signed-off-by: Shenglei Zhang 
> > ---
> >  .pytool/Plugin/LicenseCheck/LicenseCheck.py   | 118
> > ++
> >  .../LicenseCheck/LicenseCheck_plug_in.yaml|  11 ++
> >  .pytool/Plugin/LicenseCheck/Readme.md |  17 +++
> >  3 files changed, 146 insertions(+)
> >  create mode 100644
> > .pytool/Plugin/LicenseCheck/LicenseCheck.py
> >  create mode 100644
> > .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> >  create mode 100644
> > .pytool/Plugin/LicenseCheck/Readme.md
> >
> > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > new file mode 100644
> > index ..98941ddda758
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > @@ -0,0 +1,118 @@
> > +# @file LicenseCheck.py
> > +#
> > +# Copyright (c) 2020, Intel Corporation. All rights
> > reserved.
> > +# SPDX-License-Identifier: BSD-2-Clause-Patent ##
> > +
> > +import os
> > +import logging
> > +import re
> > +from io import StringIO
> > +from typing import List, Tuple
> > +from
> > edk2toolext.environment.plugintypes.ci_build_plugin
> > import ICiBuildPlugin
> > +from edk2toolext.environment.var_dict import VarDict from 
> > +edk2toollib.utility_functions import RunCmd
> > +
> > +
> > +class LicenseCheck(ICiBuildPlugin):
> > +
> > +"""
> > +A CiBuildPlugin to check the license for new added
> > files.
> > +
> > +Configuration options:
> > +"LicenseCheck": {
> > +"IgnoreFiles": []
> > +},
> > +"""
> > +
> > +license_format_preflix = 'SPDX-License-Identifier'
> > +
> > +bsd2_patent = 'BSD-2-Clause-Patent'
> > +
> > +bsd3_patent = 'BSD-3-Clause-Patent'
> > +
> > +Readdedfileformat = re.compile(r'\+\+\+ b\/(.*)')
> > +
> > +file_extension_list = [".c", ".h", ".inf", ".dsc",
> > ".dec", ".py", ".bat", ".sh", ".uni", ".yaml",
> > +   ".fdf", ".inc", "yml",
> > ".asm", ".asm16", ".asl", ".vfr", ".s", ".S", ".aslc",
> > +   ".nasm", ".nasmb", ".idf",
> > ".Vfr", ".H"]
> > +
> > +def GetTestName(self, packagename: str,
> > environment: VarDict) -> tuple:
> > +""" Provide the testcase name and classname for
> > use in reporting
> > +testclassname: a descriptive string for the
> > testcase can include whitespace
> > +classname: should be patterned
> > ..
> > +
> > +Args:
> > +  packagename: string containing name of
> > package to build
> > +  environment: The VarDict for the test to
> > run in
> > +Returns:
> > +a tuple containing the testcase name
> > and the classname
> > +(testcasename, classname)
> > +"""
> > +return ("Check for license for " + packagename,
> > packagename + ".LicenseCheck")
> > +
> > +##
> > +# External function of plugin.  This function is
> > used to perform the task of the ci_build_plugin Plugin
> > +#
> > +#   - package is the edk2 path to package.  This
> > means workspace/packagepath relative.
> > +#   - edk2path object configured with workspace and
> > packages path
> > +#   - PkgConfig Object (dict) for the pkg
> > +#   - EnvConfig Object
> > +#   - Plugin

Re: [edk2-devel] [PATCH 01/15] .pytool/Plugin: Add a plugin LicenseCheck

2020-07-29 Thread Zhang, Shenglei 
Five sorts of license are accepted in edk2(described in readme). They are BSD 
(2-clause), BSD (3-clause), MIT, Python-2.0 and zlib.
Minus non bsd plus patent license, they are BSD (2-clause) and BSD (3-clause).

Thanks,
Shenglei

> -Original Message-
> From: Kinney, Michael D 
> Sent: Wednesday, July 29, 2020 7:29 AM
> To: Zhang, Shenglei ; devel@edk2.groups.io;
> Kinney, Michael D 
> Cc: Sean Brogan ; Bret Barkelew
> ; Gao, Liming 
> Subject: RE: [PATCH 01/15] .pytool/Plugin: Add a plugin LicenseCheck
> 
> Where did the requirement for BSD-3-Clause-Patent come from?
> 
> Thanks,
> 
> Mike
> 
> > -Original Message-
> > From: Zhang, Shenglei 
> > Sent: Monday, July 20, 2020 1:37 AM
> > To: devel@edk2.groups.io
> > Cc: Sean Brogan ; Bret
> > Barkelew ; Kinney, Michael
> > D ; Gao, Liming
> > 
> > Subject: [PATCH 01/15] .pytool/Plugin: Add a plugin
> > LicenseCheck
> >
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2833
> > Add a plugin to check license conflict for new added
> > files in a patch. It will report out errors when meeting
> > files which are now contributed under BSD-2-Clause-
> > Patent
> > or BSD-3-Clause-Patent.
> >
> > Cc: Sean Brogan 
> > Cc: Bret Barkelew 
> > Cc: Michael D Kinney 
> > Cc: Liming Gao 
> > Signed-off-by: Shenglei Zhang 
> > ---
> >  .pytool/Plugin/LicenseCheck/LicenseCheck.py   | 118
> > ++
> >  .../LicenseCheck/LicenseCheck_plug_in.yaml|  11 ++
> >  .pytool/Plugin/LicenseCheck/Readme.md |  17 +++
> >  3 files changed, 146 insertions(+)
> >  create mode 100644
> > .pytool/Plugin/LicenseCheck/LicenseCheck.py
> >  create mode 100644
> > .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> >  create mode 100644
> > .pytool/Plugin/LicenseCheck/Readme.md
> >
> > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > new file mode 100644
> > index ..98941ddda758
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > @@ -0,0 +1,118 @@
> > +# @file LicenseCheck.py
> > +#
> > +# Copyright (c) 2020, Intel Corporation. All rights
> > reserved.
> > +# SPDX-License-Identifier: BSD-2-Clause-Patent
> > +##
> > +
> > +import os
> > +import logging
> > +import re
> > +from io import StringIO
> > +from typing import List, Tuple
> > +from
> > edk2toolext.environment.plugintypes.ci_build_plugin
> > import ICiBuildPlugin
> > +from edk2toolext.environment.var_dict import VarDict
> > +from edk2toollib.utility_functions import RunCmd
> > +
> > +
> > +class LicenseCheck(ICiBuildPlugin):
> > +
> > +"""
> > +A CiBuildPlugin to check the license for new added
> > files.
> > +
> > +Configuration options:
> > +"LicenseCheck": {
> > +"IgnoreFiles": []
> > +},
> > +"""
> > +
> > +license_format_preflix = 'SPDX-License-Identifier'
> > +
> > +bsd2_patent = 'BSD-2-Clause-Patent'
> > +
> > +bsd3_patent = 'BSD-3-Clause-Patent'
> > +
> > +Readdedfileformat = re.compile(r'\+\+\+ b\/(.*)')
> > +
> > +file_extension_list = [".c", ".h", ".inf", ".dsc",
> > ".dec", ".py", ".bat", ".sh", ".uni", ".yaml",
> > +   ".fdf", ".inc", "yml",
> > ".asm", ".asm16", ".asl", ".vfr", ".s", ".S", ".aslc",
> > +   ".nasm", ".nasmb", ".idf",
> > ".Vfr", ".H"]
> > +
> > +def GetTestName(self, packagename: str,
> > environment: VarDict) -> tuple:
> > +""" Provide the testcase name and classname for
> > use in reporting
> > +testclassname: a descriptive string for the
> > testcase can include whitespace
> > +classname: should be patterned
> > ..
> > +
> > +Args:
> > +  packagename: string containing name of
> > package to build
> > +  environment: The VarDict for the test to
> > run in
> > +Returns:
> > +a tuple containing the testcase name
> > and the classname
> > +(testcasename, classname)
> > +"""
> > +return ("Check for license for " + packagename,
> > packagename + ".LicenseCheck")
> > +
> > +##
> > +# External function of plugin.  This function is
> > used to perform the task of the ci_build_plugin Plugin
> > +#
> > +#   - package is the edk2 path to package.  This
> > means workspace/packagepath relative.
> > +#   - edk2path object configured with workspace and
> > packages path
> > +#   - PkgConfig Object (dict) for the pkg
> > +#   - EnvConfig Object
> > +#   - Plugin Manager Instance
> > +#   - Plugin Helper Obj Instance
> > +#   - Junit Logger
> > +#   - output_stream the StringIO output stream from
> > this plugin via logging
> > +def RunBuildPlugin(self, packagename, Edk2pathObj,
> > pkgconfig, environment, PLM, PLMHelper, tc,
> > output_stream=None):
> > +return_buffer = StringIO()
> > +params = "diff --unified=0

Re: [edk2-devel] [PATCH 01/15] .pytool/Plugin: Add a plugin LicenseCheck

2020-07-28 Thread Michael D Kinney 
Where did the requirement for BSD-3-Clause-Patent come from?

Thanks,

Mike

> -Original Message-
> From: Zhang, Shenglei 
> Sent: Monday, July 20, 2020 1:37 AM
> To: devel@edk2.groups.io
> Cc: Sean Brogan ; Bret
> Barkelew ; Kinney, Michael
> D ; Gao, Liming
> 
> Subject: [PATCH 01/15] .pytool/Plugin: Add a plugin
> LicenseCheck
> 
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2833
> Add a plugin to check license conflict for new added
> files in a patch. It will report out errors when meeting
> files which are now contributed under BSD-2-Clause-
> Patent
> or BSD-3-Clause-Patent.
> 
> Cc: Sean Brogan 
> Cc: Bret Barkelew 
> Cc: Michael D Kinney 
> Cc: Liming Gao 
> Signed-off-by: Shenglei Zhang 
> ---
>  .pytool/Plugin/LicenseCheck/LicenseCheck.py   | 118
> ++
>  .../LicenseCheck/LicenseCheck_plug_in.yaml|  11 ++
>  .pytool/Plugin/LicenseCheck/Readme.md |  17 +++
>  3 files changed, 146 insertions(+)
>  create mode 100644
> .pytool/Plugin/LicenseCheck/LicenseCheck.py
>  create mode 100644
> .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
>  create mode 100644
> .pytool/Plugin/LicenseCheck/Readme.md
> 
> diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> new file mode 100644
> index ..98941ddda758
> --- /dev/null
> +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> @@ -0,0 +1,118 @@
> +# @file LicenseCheck.py
> +#
> +# Copyright (c) 2020, Intel Corporation. All rights
> reserved.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +
> +import os
> +import logging
> +import re
> +from io import StringIO
> +from typing import List, Tuple
> +from
> edk2toolext.environment.plugintypes.ci_build_plugin
> import ICiBuildPlugin
> +from edk2toolext.environment.var_dict import VarDict
> +from edk2toollib.utility_functions import RunCmd
> +
> +
> +class LicenseCheck(ICiBuildPlugin):
> +
> +"""
> +A CiBuildPlugin to check the license for new added
> files.
> +
> +Configuration options:
> +"LicenseCheck": {
> +"IgnoreFiles": []
> +},
> +"""
> +
> +license_format_preflix = 'SPDX-License-Identifier'
> +
> +bsd2_patent = 'BSD-2-Clause-Patent'
> +
> +bsd3_patent = 'BSD-3-Clause-Patent'
> +
> +Readdedfileformat = re.compile(r'\+\+\+ b\/(.*)')
> +
> +file_extension_list = [".c", ".h", ".inf", ".dsc",
> ".dec", ".py", ".bat", ".sh", ".uni", ".yaml",
> +   ".fdf", ".inc", "yml",
> ".asm", ".asm16", ".asl", ".vfr", ".s", ".S", ".aslc",
> +   ".nasm", ".nasmb", ".idf",
> ".Vfr", ".H"]
> +
> +def GetTestName(self, packagename: str,
> environment: VarDict) -> tuple:
> +""" Provide the testcase name and classname for
> use in reporting
> +testclassname: a descriptive string for the
> testcase can include whitespace
> +classname: should be patterned
> ..
> +
> +Args:
> +  packagename: string containing name of
> package to build
> +  environment: The VarDict for the test to
> run in
> +Returns:
> +a tuple containing the testcase name
> and the classname
> +(testcasename, classname)
> +"""
> +return ("Check for license for " + packagename,
> packagename + ".LicenseCheck")
> +
> +##
> +# External function of plugin.  This function is
> used to perform the task of the ci_build_plugin Plugin
> +#
> +#   - package is the edk2 path to package.  This
> means workspace/packagepath relative.
> +#   - edk2path object configured with workspace and
> packages path
> +#   - PkgConfig Object (dict) for the pkg
> +#   - EnvConfig Object
> +#   - Plugin Manager Instance
> +#   - Plugin Helper Obj Instance
> +#   - Junit Logger
> +#   - output_stream the StringIO output stream from
> this plugin via logging
> +def RunBuildPlugin(self, packagename, Edk2pathObj,
> pkgconfig, environment, PLM, PLMHelper, tc,
> output_stream=None):
> +return_buffer = StringIO()
> +params = "diff --unified=0 origin/master HEAD"
> +RunCmd("git", params, outstream=return_buffer)
> +p = return_buffer.getvalue().strip()
> +patch = p.split("\n")
> +return_buffer.close()
> +
> +ignore_files = []
> +if "IgnoreFiles" in pkgconfig:
> +ignore_files = pkgconfig["IgnoreFiles"]
> +
> +self.ok = True
> +self.startcheck = False
> +self.license = True
> +self.all_file_pass = True
> +count = len(patch)
> +line_index = 0
> +for line in patch:
> +if line.startswith('--- /dev/null'):
> +nextline = patch[line_index + 1]
> +added_file =
> self.Readdedfileformat.search(nextline).group(1)
> +added_file_extension =
> os.path.splitext(added_file)[1]
>

[edk2-devel] [PATCH 01/15] .pytool/Plugin: Add a plugin LicenseCheck

2020-07-20 Thread Zhang, Shenglei 
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2833
Add a plugin to check license conflict for new added
files in a patch. It will report out errors when meeting
files which are now contributed under BSD-2-Clause-Patent
or BSD-3-Clause-Patent.

Cc: Sean Brogan 
Cc: Bret Barkelew 
Cc: Michael D Kinney 
Cc: Liming Gao 
Signed-off-by: Shenglei Zhang 
---
 .pytool/Plugin/LicenseCheck/LicenseCheck.py   | 118 ++
 .../LicenseCheck/LicenseCheck_plug_in.yaml|  11 ++
 .pytool/Plugin/LicenseCheck/Readme.md |  17 +++
 3 files changed, 146 insertions(+)
 create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py
 create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
 create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md

diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py 
b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
new file mode 100644
index ..98941ddda758
--- /dev/null
+++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
@@ -0,0 +1,118 @@
+# @file LicenseCheck.py
+#
+# Copyright (c) 2020, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+import os
+import logging
+import re
+from io import StringIO
+from typing import List, Tuple
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toolext.environment.var_dict import VarDict
+from edk2toollib.utility_functions import RunCmd
+
+
+class LicenseCheck(ICiBuildPlugin):
+
+"""
+A CiBuildPlugin to check the license for new added files.
+
+Configuration options:
+"LicenseCheck": {
+"IgnoreFiles": []
+},
+"""
+
+license_format_preflix = 'SPDX-License-Identifier'
+
+bsd2_patent = 'BSD-2-Clause-Patent'
+
+bsd3_patent = 'BSD-3-Clause-Patent'
+
+Readdedfileformat = re.compile(r'\+\+\+ b\/(.*)')
+
+file_extension_list = [".c", ".h", ".inf", ".dsc", ".dec", ".py", ".bat", 
".sh", ".uni", ".yaml",
+   ".fdf", ".inc", "yml", ".asm", ".asm16", ".asl", 
".vfr", ".s", ".S", ".aslc",
+   ".nasm", ".nasmb", ".idf", ".Vfr", ".H"]
+
+def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+""" Provide the testcase name and classname for use in reporting
+testclassname: a descriptive string for the testcase can include 
whitespace
+classname: should be patterned ..
+
+Args:
+  packagename: string containing name of package to build
+  environment: The VarDict for the test to run in
+Returns:
+a tuple containing the testcase name and the classname
+(testcasename, classname)
+"""
+return ("Check for license for " + packagename, packagename + 
".LicenseCheck")
+
+##
+# External function of plugin.  This function is used to perform the task 
of the ci_build_plugin Plugin
+#
+#   - package is the edk2 path to package.  This means 
workspace/packagepath relative.
+#   - edk2path object configured with workspace and packages path
+#   - PkgConfig Object (dict) for the pkg
+#   - EnvConfig Object
+#   - Plugin Manager Instance
+#   - Plugin Helper Obj Instance
+#   - Junit Logger
+#   - output_stream the StringIO output stream from this plugin via logging
+def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, 
PLM, PLMHelper, tc, output_stream=None):
+return_buffer = StringIO()
+params = "diff --unified=0 origin/master HEAD"
+RunCmd("git", params, outstream=return_buffer)
+p = return_buffer.getvalue().strip()
+patch = p.split("\n")
+return_buffer.close()
+
+ignore_files = []
+if "IgnoreFiles" in pkgconfig:
+ignore_files = pkgconfig["IgnoreFiles"]
+
+self.ok = True
+self.startcheck = False
+self.license = True
+self.all_file_pass = True
+count = len(patch)
+line_index = 0
+for line in patch:
+if line.startswith('--- /dev/null'):
+nextline = patch[line_index + 1]
+added_file = self.Readdedfileformat.search(nextline).group(1)
+added_file_extension = os.path.splitext(added_file)[1]
+if added_file_extension in self.file_extension_list and 
packagename in added_file:
+if (self.IsIgnoreFile(added_file, ignore_files)):
+line_index = line_index + 1
+continue
+self.startcheck = True
+self.license = False
+if self.startcheck and self.license_format_preflix in line:
+if self.bsd2_patent in line or self.bsd3_patent in line:
+self.license = True
+if line_index + 1 == count or patch[line_index + 
1].startswith('diff --')