Re: [edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment
Bob and Ray, My point isn't that there is anything wrong with script. This script is designed to work with edk2-platforms repo or other repos of similar layout. This is a "platform" thing that is needed on some platforms (not edk2 core platforms). To me this means it does't belong in edk2 repo. Anytime new functionality is proposed I think it is important to evaluate the stakeholders of the given repository that is targeted and make sure the functionality is aligned. In my opinion when edk2 get too prescriptive about how to do things outside of edk2 or tries to prematurely provide functionality for all consumers to use then we often see this overhead added to the tree and low adoption rate. I would propose to add this to edk2-platforms since that is where this is needed and build up a nice "platform" design pattern. If it provides great value then talk about moving up in the repository stack. Finally, I know this is a trivial and small script. The overhead is minimal but i see this as a barometer for how the community aligns to support all isn't consumers. Many of us have to maintain forks with significant changes because edk2 carries too many prescriptive ideas. I want to push Tianocore to have better alignment with different repositories, their functionality and focus. I want to see downstream consumers spend less time maintaining their fork and shipping code more closely aligned with the open source edk2. thanks Sean -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#57009): https://edk2.groups.io/g/devel/message/57009 Mute This Topic: https://groups.io/mt/72720370/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment
Hi Sean, That the Basetools works correctly depends on the PACAKGES_PATH is set correctly. I think this patch provides common function and provides assistance for the Basetools. Thanks, Bob From: Ni, Ray Sent: Tuesday, April 7, 2020 10:16 AM To: devel@edk2.groups.io; sean.bro...@microsoft.com; Luo, Heng Cc: Feng, Bob C Subject: RE: [edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment Sean, PyTools is good. This patch is to support those platforms that haven’t adopted PyTools for their platform build. Thanks, Ray From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> mailto:devel@edk2.groups.io>> On Behalf Of Sean via groups.io Sent: Thursday, April 2, 2020 11:59 PM To: Luo, Heng mailto:heng@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io> Subject: Re: [edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment I am not a fan of this. The community and design meetings have had a few discussions about tools (PyTools has been presented twice) and a few brief discussions about common patterns to build platforms but I don't think there is real alignment. Each "platform" has its own way of doing things. As for this patch, I don't really want to see a bunch of scripts added to edk2 basetools that are not aligned with a community agreed direction (or any clear direction). It has also been brought up that basetools as a python package/project is a challenge to work with and has some fundamental problems. Adding more in a similar design pattern is not how we start fixing it. As an example for edk2-pytools, an RFC was offered and even then it was added as its own repositories to avoid more directly to the edk2 repo. @Mike K - Any progress on getting a tools subtream setup? Any governance ideas to help align these efforts. Thanks Sean -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#57008): https://edk2.groups.io/g/devel/message/57008 Mute This Topic: https://groups.io/mt/72720370/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment
Sean, PyTools is good. This patch is to support those platforms that haven’t adopted PyTools for their platform build. Thanks, Ray From: devel@edk2.groups.io On Behalf Of Sean via groups.io Sent: Thursday, April 2, 2020 11:59 PM To: Luo, Heng ; devel@edk2.groups.io Subject: Re: [edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment I am not a fan of this. The community and design meetings have had a few discussions about tools (PyTools has been presented twice) and a few brief discussions about common patterns to build platforms but I don't think there is real alignment. Each "platform" has its own way of doing things. As for this patch, I don't really want to see a bunch of scripts added to edk2 basetools that are not aligned with a community agreed direction (or any clear direction). It has also been brought up that basetools as a python package/project is a challenge to work with and has some fundamental problems. Adding more in a similar design pattern is not how we start fixing it. As an example for edk2-pytools, an RFC was offered and even then it was added as its own repositories to avoid more directly to the edk2 repo. @Mike K - Any progress on getting a tools subtream setup? Any governance ideas to help align these efforts. Thanks Sean -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#57006): https://edk2.groups.io/g/devel/message/57006 Mute This Topic: https://groups.io/mt/72720370/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment
I am not a fan of this. The community and design meetings have had a few discussions about tools (PyTools has been presented twice) and a few brief discussions about common patterns to build platforms but I don't think there is real alignment. Each "platform" has its own way of doing things. As for this patch, I don't really want to see a bunch of scripts added to edk2 basetools that are not aligned with a community agreed direction (or any clear direction). It has also been brought up that basetools as a python package/project is a challenge to work with and has some fundamental problems. Adding more in a similar design pattern is not how we start fixing it. As an example for edk2-pytools, an RFC was offered and even then it was added as its own repositories to avoid more directly to the edk2 repo. @Mike K - Any progress on getting a tools subtream setup? Any governance ideas to help align these efforts. Thanks Sean -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56905): https://edk2.groups.io/g/devel/message/56905 Mute This Topic: https://groups.io/mt/72720370/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2656
1. Add GetPackagesPath.py, it will be used to get package pathes from
special directories. A sub directory is a qualified package path
when an EDKII Package can be found under it.
2. Add SetPackagesPath.bat and SetPackagesPath.sh, these scripts call
GetPackagesPath.py to collect all package paths under specified
directories and append them to PACKAGES_PATH environment variable.
Cc: Ray Ni
Cc: Eric Dong
Cc: Liming Gao
Cc: Bob Feng
Cc: Amy Chan
Signed-off-by: Heng Luo
---
BaseTools/Scripts/GetPackagesPath.py | 94
++
BaseTools/Scripts/SetPackagesPath.bat | 33 +
BaseTools/Scripts/SetPackagesPath.sh | 40
3 files changed, 167 insertions(+)
diff --git a/BaseTools/Scripts/GetPackagesPath.py
b/BaseTools/Scripts/GetPackagesPath.py
new file mode 100644
index 00..82d66a6106
--- /dev/null
+++ b/BaseTools/Scripts/GetPackagesPath.py
@@ -0,0 +1,94 @@
+## @file
+# Get all recursive package paths from special directories.
+#
+# Copyright (c) 2020, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+import os
+import glob
+import argparse
+
+#
+# Globals for help information
+#
+__prog__ = 'GetPackagesPath.py'
+__copyright__ = 'Copyright (c) 2020, Intel Corporation. All rights reserved.'
+__description__ = 'Gets all recursive package paths in specified directory.\n'
+
+def list_packages_path(root):
+""" Gets all recursive package paths in specified directory.
+A directory is a package path if it satisfies conditions below:
+1. it is a directory
+2. it is not an EDK II Package. An EDK II Package (directory) is
+ a directory that contains an EDK II package declaration (DEC) file.
+3. it contains at least one first level EDK II Package.
+Note: A directory is not package path but its subdirectory could be.
+Example: edk2-platforms/Features is not package path
+but edk2-platforms/Features/Intel is.
+
+:param root: The specified directory to find package paths in it
+:type root: String
+:returns: Return all recursive package paths
+:rtype: String list
+"""
+
+if (not os.path.exists(root)) or (not os.path.isdir(root)):
+ return []
+
+if glob.glob(os.path.join(root, '*.dec')):
+# it is an EDK II Package
+return []
+
+paths = []
+contain_package = False
+for filename in os.listdir(root):
+filepath = os.path.join(root, filename)
+if os.path.isdir(filepath):
+if glob.glob(os.path.join(filepath, '*.dec')):
+# it is an EDK II Package
+contain_package = True
+else:
+# gets package paths for subdirectory if it is not package
+paths = paths + list_packages_path(filepath)
+
+if contain_package:
+# root is a package path because it contains EDK II Package
+# in first level folder, inset it to head of list
+paths.insert(0, root)
+
+# return package paths
+return paths
+
+def get_packages_path(directories):
+""" For each direcory in directories, gets all recursive package paths
+in this directory and joins them into one string.
+
+:param directories: the list of directory
+:type directories: String list
+:returns: Return string of package paths
+:rtype: String
+"""
+
+packages_path = ''
+for directory in directories:
+directory = os.path.abspath(directory)
+paths = list_packages_path(directory)
+for path in paths:
+if packages_path == '':
+packages_path = path
+else:
+packages_path += os.pathsep + path
+return packages_path
+
+if __name__ == '__main__':
+# Create command line argument parser object
+parser = argparse.ArgumentParser(
+prog=__prog__,
+description=__description__ + __copyright__,
+conflict_handler='resolve'
+)
+parser.add_argument('directory', nargs='+',
+help='Specified directory where package packages are got from')
+args = parser.parse_args()
+print(get_packages_path(args.directory))
diff --git a/BaseTools/Scripts/SetPackagesPath.bat
b/BaseTools/Scripts/SetPackagesPath.bat
new file mode 100644
index 00..a8c45612c9
--- /dev/null
+++ b/BaseTools/Scripts/SetPackagesPath.bat
@@ -0,0 +1,33 @@
+@REM @file
+@REM Windows batch file to set PACKAGES_PATH environment
+@REM
+@REM Copyright (c) 2020, Intel Corporation. All rights reserved.
+@REM SPDX-License-Identifier: BSD-2-Clause-Patent
+@REM
+@REM This script calls GetPackagesPath.py to collect all package paths under
+@REM specified directories and append them to PACKAGES_PATH environment
