v2 changes:
- Rebase to the current git HEAD (821807bcefb9a36e598d71a8004fae5aab2052a0)
- Apply "futurize -f libfuturize.fixes.fix_absolute_import" and
refactor some python scripts to break the circular imports.
This patch series is also available in
https://github.com/lcp/edk2/tree/python3-futurize-v2
Since python2 will be EOL in 2020, we start to evaluate the impact of
the python2 removal. As expected, OMVF building failed the test. It's
actually a task noted in the wiki page:
https://github.com/tianocore/tianocore.github.io/wiki/Tasks-BaseTools-Python3-Support
Maybe it's time to convert the python scripts gradully.
This patchset doesn't make the python scripts in BaseTools compatible
with python3 immediately. It aims to do the trivial and safe conversion
and replacement to make some statements compatible with both python2 and
python3, so we can deal with the difficult cases later.
With the help of "futurize" from python-future, it's easier to refactor
the statements. This patchset is basically equivalent to "futurize -1"
plus "StringIO.StringIO => io.BytesIO".
For the "io.BytesIO" change, it MIGHT introduce slow down to the build
time since io.BytesIO is slower than StringIO.StringIO in python2(*).
For a quick test, I built OVMF with the following command based on
8ab0bd2397c9d3922e0c7dbb1aa6f7e08799079f:
$ rm -rf Build && make -C BaseTools/ clean
$ time ./OvmfPkg/build.sh -D SECURE_BOOT_ENABLE \
-D NETWORK_IP6_ENABLE \
-D HTTP_BOOT_ENABLE \
-D TLS_ENABLE
Before io.BytesIO:
Build total time: 00:03:56
real 4m22.991s
user 3m55.874s
sys 0m27.250s
After io.BytesIO:
Build total time: 00:03:57
real 4m23.953s
user 3m57.526s
sys 0m27.192s
The difference is only 1 second, and I would say the impact is subtle.
The next step will be fixing relative import and maybe applying more
futurize fixes. We won't get there soon but at least we are moving...
(*)
https://stackoverflow.com/questions/37462075/confusing-about-stringio-cstringio-and-byteio
Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <[email protected]>
Cc: Liming Gao <[email protected]>
Signed-off-by: Gary Lin <[email protected]>
Gary Lin (20):
BaseTools: Refactor python except statements
BaseTools: Refactor python print statements
BaseTools: Remove the old python "not-equal"
BaseTools: Use the python3-range functions
BaseTools: Remove tuple parameter in python scripts
BaseTools: Remove the deprecated hash_key()
BaseTools: Import reduce() from functools
BaseTools: Replace StandardError with Expression
BaseTools: Remove types.TypeType
BaseTools: Refactor python raise statement
BaseTools: Adjust the spaces around commas and colons
BaseTools: Migrate to the new octal literal
BaseTools: Unify long int and int in python scripts
BaseTools: Adjust old python2 idioms
BaseTools: Replace StringIO.StringIO with io.BytesIO
BaseTools: Treat GenFds.py and build.py as python modules
BaseTools: Adopt absolute import for python scripts
BaseTools: Move OverrideAttribs to OptRomInfStatement.py
BaseTools: Move FindExtendTool to GenFdsGlobalVariable.py
BaseTools: Move ImageBinDict to GenFdsGlobalVariable.py
BaseTools/Bin/CYGWIN_NT-5.1-i686/armcc_wrapper.py | 5 +-
BaseTools/BinWrappers/PosixLike/GenFds | 2 +-
BaseTools/BinWrappers/PosixLike/build | 2 +-
BaseTools/Scripts/BinToPcd.py | 46
+++--
BaseTools/Scripts/ConvertMasmToNasm.py | 1 +
BaseTools/Scripts/ConvertUni.py | 5 -
BaseTools/Scripts/MemoryProfileSymbolGen.py | 22 +-
BaseTools/Scripts/PatchCheck.py | 7 +-
BaseTools/Scripts/RunMakefile.py | 2 +-
BaseTools/Scripts/SmiHandlerProfileSymbolGen.py | 20 +-
BaseTools/Scripts/UpdateBuildVersions.py | 18 +-
BaseTools/Source/Python/AutoGen/AutoGen.py | 98
++++-----
BaseTools/Source/Python/AutoGen/BuildEngine.py | 38
++--
BaseTools/Source/Python/AutoGen/GenC.py | 12 +-
BaseTools/Source/Python/AutoGen/GenDepex.py | 8 +-
BaseTools/Source/Python/AutoGen/GenMake.py | 11 +-
BaseTools/Source/Python/AutoGen/GenPcdDb.py | 149
+++++++-------
BaseTools/Source/Python/AutoGen/GenVar.py | 166
+++++++--------
BaseTools/Source/Python/AutoGen/IdfClassObject.py | 1 -
BaseTools/Source/Python/AutoGen/InfSectionParser.py | 1 +
BaseTools/Source/Python/AutoGen/StrGather.py | 8 +-
BaseTools/Source/Python/AutoGen/UniClassObject.py | 18 +-
BaseTools/Source/Python/AutoGen/ValidCheckingInfoObject.py | 10 +-
BaseTools/Source/Python/BPDG/BPDG.py | 8 +-
BaseTools/Source/Python/BPDG/GenVpd.py | 28
+--
BaseTools/Source/Python/Common/DataType.py | 4 +-
BaseTools/Source/Python/Common/Database.py | 8 +-
BaseTools/Source/Python/Common/DecClassObject.py | 56
++---
BaseTools/Source/Python/Common/Dictionary.py | 14 +-
BaseTools/Source/Python/Common/DscClassObject.py | 91
+++++----
BaseTools/Source/Python/Common/EdkIIWorkspace.py | 28
+--
BaseTools/Source/Python/Common/EdkIIWorkspaceBuild.py | 152
+++++++-------
BaseTools/Source/Python/Common/EdkLogger.py | 3 +-
BaseTools/Source/Python/Common/Expression.py | 86
++++----
BaseTools/Source/Python/Common/FdfClassObject.py | 6 +-
BaseTools/Source/Python/Common/FdfParserLite.py | 47
++---
BaseTools/Source/Python/Common/InfClassObject.py | 134
++++++------
BaseTools/Source/Python/Common/LongFilePathOs.py | 5 +-
BaseTools/Source/Python/Common/MigrationUtilities.py | 4 +-
BaseTools/Source/Python/Common/Misc.py | 79
++++----
BaseTools/Source/Python/Common/Parsing.py | 6 +-
BaseTools/Source/Python/Common/RangeExpression.py | 32
+--
BaseTools/Source/Python/Common/String.py | 16 +-
BaseTools/Source/Python/Common/TargetTxtClassObject.py | 24
++-
BaseTools/Source/Python/Common/ToolDefClassObject.py | 12 +-
BaseTools/Source/Python/Common/VpdInfoFile.py | 23
++-
BaseTools/Source/Python/CommonDataClass/ModuleClass.py | 3 +-
BaseTools/Source/Python/CommonDataClass/PackageClass.py | 3 +-
BaseTools/Source/Python/CommonDataClass/PlatformClass.py | 3 +-
BaseTools/Source/Python/Ecc/CParser.py | 178
++++++++--------
BaseTools/Source/Python/Ecc/Check.py | 10 +-
BaseTools/Source/Python/Ecc/CodeFragmentCollector.py | 82
++++----
BaseTools/Source/Python/Ecc/Configuration.py | 5 +-
BaseTools/Source/Python/Ecc/Database.py | 7 +-
BaseTools/Source/Python/Ecc/Ecc.py | 25
+--
BaseTools/Source/Python/Ecc/Exception.py | 6 +-
BaseTools/Source/Python/Ecc/FileProfile.py | 5 +-
BaseTools/Source/Python/Ecc/MetaDataParser.py | 8 +-
BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaDataTable.py | 5 +-
BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py | 44
++--
BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileTable.py | 5 +-
BaseTools/Source/Python/Ecc/Xml/XmlRoutines.py | 9 +-
BaseTools/Source/Python/Ecc/c.py | 28
+--
BaseTools/Source/Python/Eot/CParser.py | 178
++++++++--------
BaseTools/Source/Python/Eot/CodeFragmentCollector.py | 72
+++----
BaseTools/Source/Python/Eot/Eot.py | 15 +-
BaseTools/Source/Python/Eot/FileProfile.py | 3 +-
BaseTools/Source/Python/Eot/FvImage.py | 28
+--
BaseTools/Source/Python/Eot/InfParserLite.py | 13 +-
BaseTools/Source/Python/Eot/Parser.py | 5 +-
BaseTools/Source/Python/Eot/Report.py | 3 +-
BaseTools/Source/Python/Eot/c.py | 32
+--
BaseTools/Source/Python/GenFds/AprioriSection.py | 12 +-
BaseTools/Source/Python/GenFds/Capsule.py | 22 +-
BaseTools/Source/Python/GenFds/CapsuleData.py | 11 +-
BaseTools/Source/Python/GenFds/CompressSection.py | 7 +-
BaseTools/Source/Python/GenFds/DataSection.py | 7 +-
BaseTools/Source/Python/GenFds/DepexSection.py | 7 +-
BaseTools/Source/Python/GenFds/EfiSection.py | 13 +-
BaseTools/Source/Python/GenFds/Fd.py | 32
+--
BaseTools/Source/Python/GenFds/FdfParser.py | 100
++++-----
BaseTools/Source/Python/GenFds/FfsFileStatement.py | 16 +-
BaseTools/Source/Python/GenFds/FfsInfStatement.py | 35
++--
BaseTools/Source/Python/GenFds/Fv.py | 34
++--
BaseTools/Source/Python/GenFds/FvImageSection.py | 15 +-
BaseTools/Source/Python/GenFds/GenFds.py | 126
++----------
BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py | 108
+++++++++-
BaseTools/Source/Python/GenFds/GuidSection.py | 11 +-
BaseTools/Source/Python/GenFds/OptRomFileStatement.py | 3 +-
BaseTools/Source/Python/GenFds/OptRomInfStatement.py | 30
++-
BaseTools/Source/Python/GenFds/OptionRom.py | 23
+--
BaseTools/Source/Python/GenFds/Region.py | 17 +-
BaseTools/Source/Python/GenFds/RuleComplexFile.py | 3 +-
BaseTools/Source/Python/GenFds/RuleSimpleFile.py | 3 +-
BaseTools/Source/Python/GenFds/Section.py | 3 +-
BaseTools/Source/Python/GenFds/UiSection.py | 7 +-
BaseTools/Source/Python/GenFds/VerSection.py | 7 +-
BaseTools/Source/Python/GenFds/Vtf.py | 3 +-
BaseTools/Source/Python/GenPatchPcdTable/GenPatchPcdTable.py | 9 +-
BaseTools/Source/Python/PatchPcdValue/PatchPcdValue.py | 1 +
BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py | 32
+--
BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py | 30
+--
BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256Sign.py | 36
++--
BaseTools/Source/Python/Table/TableDataModel.py | 3 +-
BaseTools/Source/Python/Table/TableDec.py | 3 +-
BaseTools/Source/Python/Table/TableDsc.py | 3 +-
BaseTools/Source/Python/Table/TableEotReport.py | 5 +-
BaseTools/Source/Python/Table/TableFdf.py | 3 +-
BaseTools/Source/Python/Table/TableFile.py | 3 +-
BaseTools/Source/Python/Table/TableFunction.py | 3 +-
BaseTools/Source/Python/Table/TableIdentifier.py | 5 +-
BaseTools/Source/Python/Table/TableInf.py | 3 +-
BaseTools/Source/Python/Table/TablePcd.py | 5 +-
BaseTools/Source/Python/Table/TableQuery.py | 3 +-
BaseTools/Source/Python/Table/TableReport.py | 3 +-
BaseTools/Source/Python/TargetTool/TargetTool.py | 39
++--
BaseTools/Source/Python/Trim/Trim.py | 25
+--
BaseTools/Source/Python/UPT/Core/DependencyRules.py | 12 +-
BaseTools/Source/Python/UPT/Core/DistributionPackageClass.py | 4 +-
BaseTools/Source/Python/UPT/Core/FileHook.py | 2 +-
BaseTools/Source/Python/UPT/Core/IpiDb.py | 6 +-
BaseTools/Source/Python/UPT/Core/PackageFile.py | 12 +-
BaseTools/Source/Python/UPT/GenMetaFile/GenDecFile.py | 15 +-
BaseTools/Source/Python/UPT/GenMetaFile/GenInfFile.py | 42
++--
BaseTools/Source/Python/UPT/InstallPkg.py | 2 +-
BaseTools/Source/Python/UPT/InventoryWs.py | 2 +-
BaseTools/Source/Python/UPT/Library/CommentParsing.py | 5 +-
BaseTools/Source/Python/UPT/Library/ExpressionValidate.py | 11 +-
BaseTools/Source/Python/UPT/Library/Misc.py | 11 +-
BaseTools/Source/Python/UPT/Library/ParserValidate.py | 2 +-
BaseTools/Source/Python/UPT/Library/Parsing.py | 6 +-
BaseTools/Source/Python/UPT/Library/String.py | 5 +-
BaseTools/Source/Python/UPT/Library/UniClassObject.py | 20 +-
BaseTools/Source/Python/UPT/Library/Xml/XmlRoutines.py | 4 +-
BaseTools/Source/Python/UPT/MkPkg.py | 2 +-
BaseTools/Source/Python/UPT/Object/Parser/InfBinaryObject.py | 6 +-
BaseTools/Source/Python/UPT/Object/Parser/InfDefineObject.py | 2 +-
BaseTools/Source/Python/UPT/Object/Parser/InfGuidObject.py | 4 +-
BaseTools/Source/Python/UPT/Object/Parser/InfLibraryClassesObject.py | 2 +-
BaseTools/Source/Python/UPT/Object/Parser/InfMisc.py | 4 +-
BaseTools/Source/Python/UPT/Object/Parser/InfPackagesObject.py | 4 +-
BaseTools/Source/Python/UPT/Object/Parser/InfPcdObject.py | 4 +-
BaseTools/Source/Python/UPT/Object/Parser/InfPpiObject.py | 4 +-
BaseTools/Source/Python/UPT/Object/Parser/InfProtocolObject.py | 2 +-
BaseTools/Source/Python/UPT/Object/Parser/InfSoucesObject.py | 3 +-
BaseTools/Source/Python/UPT/Object/Parser/InfUserExtensionObject.py | 4 +-
BaseTools/Source/Python/UPT/Parser/DecParserMisc.py | 1 +
BaseTools/Source/Python/UPT/Parser/InfSectionParser.py | 3 +-
BaseTools/Source/Python/UPT/PomAdapter/DecPomAlignment.py | 57
+++---
BaseTools/Source/Python/UPT/PomAdapter/InfPomAlignment.py | 3 +-
BaseTools/Source/Python/UPT/PomAdapter/InfPomAlignmentMisc.py | 3 +-
BaseTools/Source/Python/UPT/ReplacePkg.py | 2 +-
BaseTools/Source/Python/UPT/RmPkg.py | 2 +-
BaseTools/Source/Python/UPT/TestInstall.py | 4 +-
BaseTools/Source/Python/UPT/UPT.py | 9 +-
BaseTools/Source/Python/UPT/UnitTest/DecParserTest.py | 5 +-
BaseTools/Source/Python/UPT/UnitTest/InfBinarySectionTest.py | 10 +-
BaseTools/Source/Python/UPT/Xml/CommonXml.py | 2 +-
BaseTools/Source/Python/UPT/Xml/IniToXml.py | 1 +
BaseTools/Source/Python/UPT/Xml/XmlParser.py | 25
+--
BaseTools/Source/Python/UPT/Xml/XmlParserMisc.py | 3 +-
BaseTools/Source/Python/Workspace/BuildClassObject.py | 2 +-
BaseTools/Source/Python/Workspace/DecBuildData.py | 14 +-
BaseTools/Source/Python/Workspace/DscBuildData.py | 213
++++++++++----------
BaseTools/Source/Python/Workspace/InfBuildData.py | 6 +-
BaseTools/Source/Python/Workspace/MetaFileParser.py | 75
+++----
BaseTools/Source/Python/Workspace/MetaFileTable.py | 15 +-
BaseTools/Source/Python/Workspace/WorkspaceCommon.py | 5 +-
BaseTools/Source/Python/Workspace/WorkspaceDatabase.py | 7 +-
BaseTools/Source/Python/build/BuildReport.py | 19 +-
BaseTools/Source/Python/build/build.py | 38
++--
BaseTools/Tests/CheckPythonSyntax.py | 2 +-
BaseTools/Tests/TestTools.py | 13 +-
BaseTools/Tests/TianoCompress.py | 6 +-
BaseTools/gcc/mingw-gcc-build.py | 112
+++++-----
175 files changed, 2092 insertions(+), 1927 deletions(-)
--
2.16.1
_______________________________________________
edk2-devel mailing list
[email protected]
https://lists.01.org/mailman/listinfo/edk2-devel
