On 02.07.21 14:22, Janardhan wrote: > Hi all, >
Hi! > I am planning to add Native backend guide. The document should > answer the following questions. Please, feel free to add new points > or contribute to discussion. thanks. > > User guide > > 1. BLAS Installation instructions > > 1a. Intel MKL > Download Intel MKL from [1]. > Version of MKL (2017 and higher), do we use MKL DNN primitives? > Highest supported version is 2019.5. Everything above has API changes which are not supported yet by our native dnn ops. > 1b. OpenBLAS > Compiling OpenBLAS with OpenMP is recommended instead of apt-get/yum. > instructions are at [2]. > At least in Ubuntu 20.04 you don't need to compile OpenBLAS manually for OpenMP support. Just remove the libopenblas0-pthread package and install libopenblas0-openmp instead. > 2. How to provide the location of native libraries? > > via commandline > - Java: -Djava.library.path=/path/to/blas-n-other-dependencies > - Spark: --conf > spark.executorEnv.LD_LIBRARY_PATH=/path/to/blas-n-other-dependencies > > This can also be set with `sysds.native.blas.directory` > For Intel MKL the preferred method is to source their env setup script (path adjustments to that script according to your installation): source /opt/intel/bin/compilervars.sh intel64 > 3. How to enable native BLAS? > > set sysds.native.blas[3] to either mkl, openblas > > Troubleshooting > - 1. Unable to load gomp > - 2. Unable to load mkl_rt > - 3. Unable to load openblas > - 4. Using OpenBLAS without OpenMP can lead to performance issues > - 5. Using MKL can lead to slow performance for convolution instruction > On very old Linux distributions (we have CentOS 7.x around on an older cluster) there can be issues with loading libs because the gcc ABI changed from gcc 4.x to 5 and above. In that case you'd have to either fiddle with the patchelf utility or just quickly (?) compile yourself a recent gcc and adjust your env vars (PATH, LD_LIBRARY_PATH). > --- > [1] > https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl.html > [2] https://github.com/xianyi/OpenBLAS#installation-from-source > [3] > https://github.com/apache/systemds/blob/master/conf/SystemDS-config.xml.template#L61 > > Thank you, > Janardhan > hth, Mark
smime.p7s
Description: S/MIME Cryptographic Signature
