llunak created this revision. llunak added reviewers: clayborg, labath. llunak added a project: LLDB. Herald added subscribers: usaxena95, JDevlieghere, kadircet. Herald added a project: All. llunak requested review of this revision. Herald added subscribers: lldb-commits, ilya-biryukov.
If LLDB index cache is enabled and everything is cached, then loading of debug info is essentially single-threaded. This patch parallelizes module loading in DynamicLoaderPOSIXDYLD::LoadAllCurrentModules(), which may greatly reduce the load time if the debugged program uses a large number of binaries (as opposed to monolithic programs where this presumably doesn't make a difference). In my specific case of LibreOffice Calc this reduces startup time from 6s to 2s. One problem with this change is that it creates a threadpool-within-threadpool situation in ManualDWARFIndex::Index(), and so the number of threads running at the same time may not be properly limited. I think the threadpool in ManualDWARFIndex::Index() is still useful even with this change because of the case of a monolithic program that's not cached. If not limiting thread count properly is considered a problem, I think a solution to that is using a global semaphore to limit tasks from both places. There's a simple Semaphore class in clangd (with clangd-specific trace code, so I guess it'd need to be copy&pasted into lldb) and the semaphore object would need to be stored somewhere (and I don't know where). I have not checked if all the relevant code is thread-safe, but since all of it is already running in another thread I assume that to be the case. Presumably a similar change could be done for other platforms (I have no plans to do so). Repository: rG LLVM Github Monorepo https://reviews.llvm.org/D122975 Files: lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp Index: lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp =================================================================== --- lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp +++ lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp @@ -24,6 +24,7 @@ #include "lldb/Utility/LLDBLog.h" #include "lldb/Utility/Log.h" #include "lldb/Utility/ProcessInfo.h" +#include "llvm/Support/ThreadPool.h" #include <memory> @@ -602,9 +603,24 @@ m_process->PrefetchModuleSpecs( module_names, m_process->GetTarget().GetArchitecture().GetTriple()); - for (I = m_rendezvous.begin(), E = m_rendezvous.end(); I != E; ++I) { - ModuleSP module_sp = + std::vector<DYLDRendezvous::iterator> infos; + for (I = m_rendezvous.begin(), E = m_rendezvous.end(); I != E; ++I) + infos.push_back(I); + std::vector<ModuleSP> loaded_modules; + loaded_modules.resize(infos.size()); + llvm::ThreadPool pool(llvm::optimal_concurrency(infos.size())); + auto load_module_fn = [&](size_t idx) { + DYLDRendezvous::iterator I = infos[idx]; + loaded_modules[idx] = LoadModuleAtAddress(I->file_spec, I->link_addr, I->base_addr, true); + }; + for (size_t i = 0; i < infos.size(); ++i) + pool.async(load_module_fn, i); + pool.wait(); + + for (size_t i = 0; i < infos.size(); ++i) { + DYLDRendezvous::iterator I = infos[i]; + ModuleSP module_sp = loaded_modules[i]; if (module_sp.get()) { LLDB_LOG(log, "LoadAllCurrentModules loading module: {0}", I->file_spec.GetFilename());
Index: lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp =================================================================== --- lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp +++ lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp @@ -24,6 +24,7 @@ #include "lldb/Utility/LLDBLog.h" #include "lldb/Utility/Log.h" #include "lldb/Utility/ProcessInfo.h" +#include "llvm/Support/ThreadPool.h" #include <memory> @@ -602,9 +603,24 @@ m_process->PrefetchModuleSpecs( module_names, m_process->GetTarget().GetArchitecture().GetTriple()); - for (I = m_rendezvous.begin(), E = m_rendezvous.end(); I != E; ++I) { - ModuleSP module_sp = + std::vector<DYLDRendezvous::iterator> infos; + for (I = m_rendezvous.begin(), E = m_rendezvous.end(); I != E; ++I) + infos.push_back(I); + std::vector<ModuleSP> loaded_modules; + loaded_modules.resize(infos.size()); + llvm::ThreadPool pool(llvm::optimal_concurrency(infos.size())); + auto load_module_fn = [&](size_t idx) { + DYLDRendezvous::iterator I = infos[idx]; + loaded_modules[idx] = LoadModuleAtAddress(I->file_spec, I->link_addr, I->base_addr, true); + }; + for (size_t i = 0; i < infos.size(); ++i) + pool.async(load_module_fn, i); + pool.wait(); + + for (size_t i = 0; i < infos.size(); ++i) { + DYLDRendezvous::iterator I = infos[i]; + ModuleSP module_sp = loaded_modules[i]; if (module_sp.get()) { LLDB_LOG(log, "LoadAllCurrentModules loading module: {0}", I->file_spec.GetFilename());
_______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits