lilongyue opened a new issue #18343: URL: https://github.com/apache/incubator-mxnet/issues/18343
I find that pytorch community has two easy optimization tools to accelerate the c++ compiling. I really saved a lot of time when change code-> debug .... [a lot of time] We can put these two easy methods to the compiling document of mxnet. 1.Use CCache "Even when dependencies are tracked with file modification, there are many situations where files get rebuilt when a previous compilation was exactly the same. Using ccache in a situation like this is a real time-saver. The ccache manual describes two ways to use ccache. In the PyTorch project, currently only the latter method of masquerading as the compiler via symlinks works for CUDA compilation." sudo apt-get update sudo apt-get install -y automake autoconf sudo apt-get install -y asciidoc mkdir ~/ccachesource cd ~/ccachesource git clone https://github.com/ccache/ccache.git ./autogen.sh ./configure make install prefix=~/ccache mkdir ~/ccache/lib mkdir -p ~/ccache/cuda ln -s ~/ccache/bin/ccache ~/ccache/lib/cc ln -s ~/ccache/bin/ccache ~/ccache/lib/c++ ln -s ~/ccache/bin/ccache ~/ccache/lib/gcc ln -s ~/ccache/bin/ccache ~/ccache/lib/g++ ln -s ~/ccache/bin/ccache ~/ccache/cuda/nvcc ~/ccache/bin/ccache -M 25Gi export PATH=~/ccache/lib:$PATH export CUDA_NVCC_EXECUTABLE=~/ccache/cuda/nvcc "Note that the original nvcc binary (typically at /usr/local/cuda/bin) must be on your PATH, otherwise ccache will emit the following error: ccache: error: Could not find compiler "nvcc" in PATH" 2. Use a faster linker If you are editing a single file and rebuilding in a tight loop, the time spent linking will dominate. The system linker available in most Linux distributions (GNU ld) is quite slow. Use a faster linker, like lld. official website: https://lld.llvm.org/ $ git clone https://github.com/llvm/llvm-project llvm-project $ mkdir build $ cd build $ cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=lld -DCMAKE_INSTALL_PREFIX=/usr/local ../llvm-project/llvm $ make install The easiest way to use lld this is download the latest LLVM binaries and run: ln -s /path/to/downloaded/ld.lld /usr/local/bin/ld ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
