On 07/16/20 03:41, Andrew Fish via groups.io wrote: > I think I mentioned before on the list that I think a smart git grep that > only scopes to a give target would be useful. For our internal repo we have > lots of targets and lots of generations of platforms so lots of code that > hits in a git grep that is not in the platform. > > In thinking about this it looks to me like the Build/**/AutoGenTimeStamp > (module build files) files and the Build/**/AutoGen (global build files) > contain a list of all the files for a project. > > I wrote some python that takes the build target (OvmfX64 for example) parses > all the AutoGenTimeStamp and AutoGen files merges then together and removes > the workspace path. All the other arguments are passed to `git grep` and only > lines that match the file set are displayed. > > My questions are: > 1) Did I extract all the files?
I don't know enough of BaseTools to answer the question :) > 2) Are people interested in this feature? It seems useful. :) > 3) Any ideas on how to hook it in? I just dropped in the root of the git tree > for development. We usually place such utilities under BaseTools/Scripts/, I think (e.g. PatchCheck.py, SetupGit.py, GetMaintainer.py). > 4) Any other cool ideas of what we could do with a file list of the project? Perhaps introduce a more focused utility that only produces the file list (on stdout) so people could pass the list to whatever commands they desire? Thanks! Laszlo > > Example (to give people a feel for how it works): > $ time ./ggrep.py OvmfX64 CC_FLAGS | wc -l > 42 > > real 0m0.318s > user 0m0.243s > sys 0m0.576s > $ time git grep CC_FLAGS | wc -l > 479 > > real 0m0.110s > user 0m0.148s > sys 0m0.525s > $ ./ggrep.py OvmfX64 CC_FLAGS > OvmfX64 only: git grep CC_FLAGS > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:96: MSFT:*_*_*_CC_FLAGS = > /wd4090 > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:100: RVCT:*_*_ARM_CC_FLAGS = > -JCryptoPkg/Include --diag_remark=1 > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:102: > GCC:*_CLANG35_*_CC_FLAGS = -std=c99 > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:103: > GCC:*_CLANG38_*_CC_FLAGS = -std=c99 > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:104: > GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=incompatible-pointer-types > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:106: XCODE:*_*_*_CC_FLAGS = > -std=c99 > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:55: > MSFT:DEBUG_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D > UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:56: > MSFT:RELEASE_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D > UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:57: > MSFT:DEBUG_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D > UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:58: > MSFT:RELEASE_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D > UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:59: > MSFT:DEBUG_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D > UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:60: > MSFT:RELEASE_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D > UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:61: > MSFT:DEBUG_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D > UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:62: > MSFT:RELEASE_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D > UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:63: > MSFT:DEBUG_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 /D > UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:64: > MSFT:RELEASE_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 /D > UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:65: > MSFT:DEBUG_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 /Gs32768 > /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:66: > MSFT:RELEASE_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 /Gs32768 > /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:67: INTEL:*_*_*_CC_FLAGS > = /Oi- > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:569: > MSFT:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) > /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4310 /wd4389 /wd4700 /wd4702 > /wd4706 /wd4819 > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:570: MSFT:*_*_X64_CC_FLAGS > = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) /wd4090 /wd4132 /wd4244 > /wd4245 /wd4267 /wd4306 /wd4310 /wd4700 /wd4389 /wd4702 /wd4706 /wd4819 > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:572: > INTEL:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC > $(OPENSSL_FLAGS) /w > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:573: > INTEL:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC > $(OPENSSL_FLAGS) /w > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:582: GCC:*_*_IA32_CC_FLAGS > = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized > -Wno-error=unused-but-set-variable > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:583: GCC:*_*_X64_CC_FLAGS > = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized > -Wno-error=format -Wno-format -Wno-error=unused-but-set-variable > -DNO_MSABI_VA_FUNCS > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:584: GCC:*_*_ARM_CC_FLAGS > = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized > -Wno-error=unused-but-set-variable > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:585: > GCC:*_*_AARCH64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized > -Wno-format -Wno-error=unused-but-set-variable > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:586: > GCC:*_*_RISCV64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized > -Wno-format -Wno-error=unused-but-set-variable > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:587: > GCC:*_CLANG35_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:588: > GCC:*_CLANG38_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:589: > GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized > -Wno-error=incompatible-pointer-types -Wno-error=pointer-sign > -Wno-error=implicit-function-declaration -Wno-error=ignored-pragma-optimize > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:610: RVCT:*_*_ARM_CC_FLAGS > = $(OPENSSL_FLAGS) --library_interface=aeabi_clib99 > --diag_suppress=1296,1295,550,1293,111,68,177,223,144,513,188,128,546,1,3017 > -JCryptoPkg/Include > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:611: > XCODE:*_*_IA32_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -w > -std=c99 -Wno-error=uninitialized > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:612: > XCODE:*_*_X64_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -w > -std=c99 -Wno-error=uninitialized > OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = > -DMDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D > MDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D > MDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx > -mno-sse > OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D > DISABLE_NEW_DEPRECATED_INTERFACES > OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D > DISABLE_NEW_DEPRECATED_INTERFACES > OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D > DISABLE_NEW_DEPRECATED_INTERFACES > $./ggrep.py OvmfX64 CC_FLAG -- *.dsc > OvmfX64 only: git grep CC_FLAG -- *.dsc > OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = > -DMDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D > MDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D > MDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx > -mno-sse > OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D > DISABLE_NEW_DEPRECATED_INTERFACES > OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D > DISABLE_NEW_DEPRECATED_INTERFACES > OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D > DISABLE_NEW_DEPRECATED_INTERFACES > $ > > Note if we cache the combined AutoGenTimeStamp and AutoGen files into a > single file in the Build direction and use that file on the next run the time > goes down to: > real 0m0.177s > user 0m0.185s > sys 0m0.522s > > If you don’t pass any `git grep` commands you get the file list back: > $./ggrep.py OvmfX64 > Conf/build_rule.txt > Conf/tools_def.txt > CryptoPkg/CryptoPkg.dec > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf > CryptoPkg/Library/BaseCryptLib/Cipher/CryptAes.c > … > > > My prototype is around 135 lines of Python. > > Thanks, > > Andrew Fish > > > > > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#62687): https://edk2.groups.io/g/devel/message/62687 Mute This Topic: https://groups.io/mt/75533926/21656 Group Owner: [email protected] Unsubscribe: https://edk2.groups.io/g/devel/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
