> On Jul 16, 2020, at 6:18 AM, Laszlo Ersek <[email protected]> wrote: > > On 07/16/20 03:41, Andrew Fish via groups.io <http://groups.io/> wrote: >> I think I mentioned before on the list that I think a smart git grep that >> only scopes to a give target would be useful. For our internal repo we have >> lots of targets and lots of generations of platforms so lots of code that >> hits in a git grep that is not in the platform. >> >> In thinking about this it looks to me like the Build/**/AutoGenTimeStamp >> (module build files) files and the Build/**/AutoGen (global build files) >> contain a list of all the files for a project. >> >> I wrote some python that takes the build target (OvmfX64 for example) parses >> all the AutoGenTimeStamp and AutoGen files merges then together and removes >> the workspace path. All the other arguments are passed to `git grep` and >> only lines that match the file set are displayed. >> >> My questions are: >> 1) Did I extract all the files? > > I don't know enough of BaseTools to answer the question :) > >> 2) Are people interested in this feature? > > It seems useful. :) > >> 3) Any ideas on how to hook it in? I just dropped in the root of the git >> tree for development. > > We usually place such utilities under BaseTools/Scripts/, I think (e.g. > PatchCheck.py, SetupGit.py, GetMaintainer.py).
I figured out if I rename my script to git-pgrep and move it to BaseTools/BinWrappers/PosixLike it becomes a new git command. $ git pgrep OvmfX64 CC_FLAGS -- *.dsc OvmfX64 only: git grep CC_FLAGS -- *.dsc OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES I picked pgrep for platform grep. I guess we could add $EDK_TOOLS_PATH/Scripts to the $PATH and then custom git commands could get added in $EDK_TOOLS_PATH/Scripts > >> 4) Any other cool ideas of what we could do with a file list of the project? > > Perhaps introduce a more focused utility that only produces the file > list (on stdout) so people could pass the list to whatever commands they > desire? > I added that feature. If you don’t pass any git-grep arguments you get the file list for the passed in platform. $ git pgrep OvmfX64 Conf/build_rule.txt Conf/tools_def.txt CryptoPkg/CryptoPkg.dec CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf … Thanks, Andrew Fish > Thanks! > Laszlo > >> >> Example (to give people a feel for how it works): >> $ time ./ggrep.py OvmfX64 CC_FLAGS | wc -l >> 42 >> >> real 0m0.318s >> user 0m0.243s >> sys 0m0.576s >> $ time git grep CC_FLAGS | wc -l >> 479 >> >> real 0m0.110s >> user 0m0.148s >> sys 0m0.525s >> $ ./ggrep.py OvmfX64 CC_FLAGS >> OvmfX64 only: git grep CC_FLAGS >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:96: MSFT:*_*_*_CC_FLAGS = >> /wd4090 >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:100: RVCT:*_*_ARM_CC_FLAGS >> = -JCryptoPkg/Include --diag_remark=1 >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:102: >> GCC:*_CLANG35_*_CC_FLAGS = -std=c99 >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:103: >> GCC:*_CLANG38_*_CC_FLAGS = -std=c99 >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:104: >> GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=incompatible-pointer-types >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:106: XCODE:*_*_*_CC_FLAGS = >> -std=c99 >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:55: >> MSFT:DEBUG_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D >> UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:56: >> MSFT:RELEASE_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D >> UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:57: >> MSFT:DEBUG_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D >> UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:58: >> MSFT:RELEASE_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D >> UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:59: >> MSFT:DEBUG_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D >> UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:60: >> MSFT:RELEASE_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D >> UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:61: >> MSFT:DEBUG_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D >> UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:62: >> MSFT:RELEASE_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D >> UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:63: >> MSFT:DEBUG_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 >> /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:64: >> MSFT:RELEASE_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 >> /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:65: >> MSFT:DEBUG_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 >> /Gs32768 /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:66: >> MSFT:RELEASE_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 >> /Gs32768 /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:67: INTEL:*_*_*_CC_FLAGS >> = /Oi- >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:569: >> MSFT:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) >> /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4310 /wd4389 /wd4700 /wd4702 >> /wd4706 /wd4819 >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:570: >> MSFT:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) >> /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4306 /wd4310 /wd4700 /wd4389 >> /wd4702 /wd4706 /wd4819 >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:572: >> INTEL:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC >> $(OPENSSL_FLAGS) /w >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:573: >> INTEL:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC >> $(OPENSSL_FLAGS) /w >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:582: >> GCC:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) >> -Wno-error=maybe-uninitialized -Wno-error=unused-but-set-variable >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:583: GCC:*_*_X64_CC_FLAGS >> = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized >> -Wno-error=format -Wno-format -Wno-error=unused-but-set-variable >> -DNO_MSABI_VA_FUNCS >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:584: GCC:*_*_ARM_CC_FLAGS >> = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized >> -Wno-error=unused-but-set-variable >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:585: >> GCC:*_*_AARCH64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized >> -Wno-format -Wno-error=unused-but-set-variable >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:586: >> GCC:*_*_RISCV64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized >> -Wno-format -Wno-error=unused-but-set-variable >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:587: >> GCC:*_CLANG35_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:588: >> GCC:*_CLANG38_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:589: >> GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized >> -Wno-error=incompatible-pointer-types -Wno-error=pointer-sign >> -Wno-error=implicit-function-declaration -Wno-error=ignored-pragma-optimize >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:610: >> RVCT:*_*_ARM_CC_FLAGS = $(OPENSSL_FLAGS) >> --library_interface=aeabi_clib99 >> --diag_suppress=1296,1295,550,1293,111,68,177,223,144,513,188,128,546,1,3017 >> -JCryptoPkg/Include >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:611: >> XCODE:*_*_IA32_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) >> -w -std=c99 -Wno-error=uninitialized >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:612: >> XCODE:*_*_X64_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) >> -w -std=c99 -Wno-error=uninitialized >> OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = >> -DMDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D >> MDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D >> MDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx >> -mno-sse >> OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D >> DISABLE_NEW_DEPRECATED_INTERFACES >> OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D >> DISABLE_NEW_DEPRECATED_INTERFACES >> OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D >> DISABLE_NEW_DEPRECATED_INTERFACES >> $./ggrep.py OvmfX64 CC_FLAG -- *.dsc >> OvmfX64 only: git grep CC_FLAG -- *.dsc >> OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = >> -DMDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D >> MDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D >> MDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx >> -mno-sse >> OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D >> DISABLE_NEW_DEPRECATED_INTERFACES >> OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D >> DISABLE_NEW_DEPRECATED_INTERFACES >> OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D >> DISABLE_NEW_DEPRECATED_INTERFACES >> $ >> >> Note if we cache the combined AutoGenTimeStamp and AutoGen files into a >> single file in the Build direction and use that file on the next run the >> time goes down to: >> real 0m0.177s >> user 0m0.185s >> sys 0m0.522s >> >> If you don’t pass any `git grep` commands you get the file list back: >> $./ggrep.py OvmfX64 >> Conf/build_rule.txt >> Conf/tools_def.txt >> CryptoPkg/CryptoPkg.dec >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf >> CryptoPkg/Library/BaseCryptLib/Cipher/CryptAes.c >> … >> >> >> My prototype is around 135 lines of Python. >> >> Thanks, >> >> Andrew Fish >> >> >> -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#62703): https://edk2.groups.io/g/devel/message/62703 Mute This Topic: https://groups.io/mt/75533926/21656 Group Owner: [email protected] Unsubscribe: https://edk2.groups.io/g/devel/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
