apeforest commented on a change in pull request #15045: [Dependency Update] Dependency update doc URL: https://github.com/apache/incubator-mxnet/pull/15045#discussion_r306451864
########## File path: tools/dependencies/README.md ########## @@ -40,3 +40,349 @@ This issue appeared in the OSX build with XCode version 8.0 above (reproduced on ``` --without-libidn2 ``` + +*** + +# Dependency Update Runbook + +MXNet is built on top of many dependencies. Managing those dependencies could be a big headache. This goal of this document is to give a overview of those dependencies and how to upgrade when new version of those are rolled out. + +## Overview + +The dependencies could be categorized by several groups: BLAS libraries, CPU-based performance boost library, i.e. MKLDNN and GPU-based performance boosting library including CUDA, cuDNN, NCCL. and others including OpenCV, Numpy, S3-related, PS-lite dependencies. The list below shows all the dependencies and their version. Except for CUDA, cuDNN, NCCL which the user is required to install on their environments, we statically link those dependencies into libmxnet.so when we build PyPi package. By doing this, the user can take advantage of these dependencies without being worry about it. + +| Dependencies | MXNet Version | +| :------------: |:-------------:| +|OpenBLAS| 0.3.3 | +|MKLDNN| 0.19 | +|CUDA| 10.1 | +|cuDNN| 7.5.1 | +|NCCL| 2.4.2 | +|numpy| >1.16.0,<2.0.0 | +|request| >=2.20.0,< 3.0.0 | +|graphviz| <0.9.0,>=0.8.1 | +|OpenCV| 3.4.2 | +|zlib| 1.2.6 | +|libjpeg-turbo| 2.0.2 | +|libpng| 1.6.35 | +|libtiff| 4-0-10 | +|eigen| 3.3.4 | +|libcurl| 7.61.0 | +|libssl-dev| 1.1.1b | +|zmq| 4.2.2 | +|protobuf| 3.5.1 | +|lz4| r130 | +|cityhash| 1.1.1 | + +## How to update them? + +### MKL, MKLDNN + +@pengzhao-intel (https://github.com/apache/incubator-mxnet/commits?author=pengzhao-intel) and his team are tracking and updating these versions. Kudos to them! + +### CUDA, cuDNN, NCCL +#### 1. Environment Setup +``` +# Take Ubuntu 16.04 for example +sudo apt update +sudo apt-get install -y git \ + cmake \ + libcurl4-openssl-dev \ + unzip \ + gcc-4.8 \ + g++-4.8 \ + gfortran \ + gfortran-4.8 \ + binutils \ + nasm \ + libtool \ + curl \ + wget \ + sudo \ + gnupg \ + gnupg2 \ + gnupg-agent \ + pandoc \ + python3-pip \ + automake \ + pkg-config \ + openjdk-8-jdk + +# CUDA installation +# Take CUDA 10 for example +wget https://developer.nvidia.com/compute/cuda/10.0/Prod/local_installers/cuda_10.0.130_410.48_linux +chmod +x cuda_10.0.130_410.48_linux && sudo ./cuda_10.0.130_410.48_linux +# Installation except: +# Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 410.48? Review comment: I guess you can just point to the NVIDIA CUDA driver installation page directly. I also found installing from deb (network) is very simple. Just 4 lines according to: https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=debnetwork ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
