corepointer commented on a change in pull request #1338:
URL: https://github.com/apache/systemds/pull/1338#discussion_r664475436
##########
File path: docs/site/native-backend.md
##########
@@ -0,0 +1,123 @@
+---
+layout: site
+title: Native Backend guide for SystemDS
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+This guide covers the Native BLAS optimizations and software setup for using
SystemDS `native` mode.
+
+- [Requirements](#requirements)
+ - [Hardware](#hardware)
+ - [Software](#software)
+ - [INTEL MKL](#intel-mkl)
+ - [OpenBLAS](#openblas)
+- [Native BLAS setup](#native-blas-setup)
+ - [Add library to the System path](#add-library-to-the-system-path)
+ - [Enable Native BLAS in SystemDS](#enable-native-blas-in-systemds)
+ - [Troubleshooting](#troubleshooting)
+
+
+# Native BLAS mode
+
+SystemDS implements all the matrix operations in Java. This simplifies
deployment especially in
+a distributed environment.
+
+In Some cases (such as Deep Neural Networks), to take advantage of native BLAS
instead of SystemDS
+internal Java library for performing single node operations such as matrix
multiplication, convolution etc.
+
+By default, SystemDS will first attempt to use Intel MKL (if installed), and
then OpenBLAS (if installed).
+If none of the libraries are available, SystemDS falls back to its internal
java library.
+
+> Note: Current SystemDS version only supported on **Linux** platform.
Review comment:
There is Windows and Linux support on the x86-64 platform for OpenBLAS,
MKL and CUDA.
##########
File path: docs/site/native-backend.md
##########
@@ -0,0 +1,123 @@
+---
+layout: site
+title: Native Backend guide for SystemDS
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+This guide covers the Native BLAS optimizations and software setup for using
SystemDS `native` mode.
+
+- [Requirements](#requirements)
+ - [Hardware](#hardware)
+ - [Software](#software)
+ - [INTEL MKL](#intel-mkl)
+ - [OpenBLAS](#openblas)
+- [Native BLAS setup](#native-blas-setup)
+ - [Add library to the System path](#add-library-to-the-system-path)
+ - [Enable Native BLAS in SystemDS](#enable-native-blas-in-systemds)
+ - [Troubleshooting](#troubleshooting)
+
+
+# Native BLAS mode
+
+SystemDS implements all the matrix operations in Java. This simplifies
deployment especially in
+a distributed environment.
+
+In Some cases (such as Deep Neural Networks), to take advantage of native BLAS
instead of SystemDS
+internal Java library for performing single node operations such as matrix
multiplication, convolution etc.
+
+By default, SystemDS will first attempt to use Intel MKL (if installed), and
then OpenBLAS (if installed).
+If none of the libraries are available, SystemDS falls back to its internal
java library.
+
+> Note: Current SystemDS version only supported on **Linux** platform.
+
+## Requirements
+
+### Hardware
+
+To know Intel MKL system requirements, see
+[Intel® oneAPI Math Kernel Library System
Requirements](https://software.intel.com/content/www/us/en/develop/articles/oneapi-math-kernel-library-system-requirements.html)
+
+
+### Software
+
+Either of the following software is required to be installed in your system:
+
+#### INTEL MKL
+
+ Download [Intel
MKL](https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl.html).
+ Supported MKL version is `2017` to `2019.5`.
+
+#### OpenBLAS
+
+ Install [OpenBLAS](https://www.openblas.net/). Installation instructions on
[GitHub](https://github.com/xianyi/OpenBLAS#installation-from-source).
+
+ Note: In Ubuntu 20.04, remove `libopenblas0-pthread` package and install
`libopenblas0-openmp`
+ instead. So that OpenBLAS will be installed with
[OpenMP](https://www.openmp.org/) support.
+
+## Native BLAS setup
+
+### Add library to the System path
+
+#### Intel MKL
+
+[Scripts to set environmental
variables](https://software.intel.com/content/www/us/en/develop/documentation/onemkl-linux-developer-guide/top/getting-started/setting-environment-variables/scripts-to-set-environment-variables.html)
+
+For example, to set Intel MKL libraries in the Path:
+
+```sh
+# path adjustments to that script according to your installation
+source /opt/intel/bin/compilervars.sh intel64
Review comment:
There is an equivalent script for Windows cmd.exe. I'm not on Windows at
the moment so I can't give exact directions but there is a bin folder somewhere
in your installed Windows MKL and there is a script like that. If you now
launch a bash.exe from that Windows command prompt window, you can make use of
MKL when starting SystemDS through our run script.
##########
File path: docs/site/native-backend.md
##########
@@ -0,0 +1,123 @@
+---
+layout: site
+title: Native Backend guide for SystemDS
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+This guide covers the Native BLAS optimizations and software setup for using
SystemDS `native` mode.
+
+- [Requirements](#requirements)
+ - [Hardware](#hardware)
+ - [Software](#software)
+ - [INTEL MKL](#intel-mkl)
+ - [OpenBLAS](#openblas)
+- [Native BLAS setup](#native-blas-setup)
+ - [Add library to the System path](#add-library-to-the-system-path)
+ - [Enable Native BLAS in SystemDS](#enable-native-blas-in-systemds)
+ - [Troubleshooting](#troubleshooting)
+
+
+# Native BLAS mode
+
+SystemDS implements all the matrix operations in Java. This simplifies
deployment especially in
+a distributed environment.
+
+In Some cases (such as Deep Neural Networks), to take advantage of native BLAS
instead of SystemDS
+internal Java library for performing single node operations such as matrix
multiplication, convolution etc.
+
+By default, SystemDS will first attempt to use Intel MKL (if installed), and
then OpenBLAS (if installed).
+If none of the libraries are available, SystemDS falls back to its internal
java library.
+
+> Note: Current SystemDS version only supported on **Linux** platform.
+
+## Requirements
+
+### Hardware
+
+To know Intel MKL system requirements, see
+[Intel® oneAPI Math Kernel Library System
Requirements](https://software.intel.com/content/www/us/en/develop/articles/oneapi-math-kernel-library-system-requirements.html)
+
+
+### Software
+
+Either of the following software is required to be installed in your system:
+
+#### INTEL MKL
+
+ Download [Intel
MKL](https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl.html).
+ Supported MKL version is `2017` to `2019.5`.
+
+#### OpenBLAS
+
+ Install [OpenBLAS](https://www.openblas.net/). Installation instructions on
[GitHub](https://github.com/xianyi/OpenBLAS#installation-from-source).
+
+ Note: In Ubuntu 20.04, remove `libopenblas0-pthread` package and install
`libopenblas0-openmp`
+ instead. So that OpenBLAS will be installed with
[OpenMP](https://www.openmp.org/) support.
+
+## Native BLAS setup
+
+### Add library to the System path
+
+#### Intel MKL
+
+[Scripts to set environmental
variables](https://software.intel.com/content/www/us/en/develop/documentation/onemkl-linux-developer-guide/top/getting-started/setting-environment-variables/scripts-to-set-environment-variables.html)
+
+For example, to set Intel MKL libraries in the Path:
+
+```sh
+# path adjustments to that script according to your installation
+source /opt/intel/bin/compilervars.sh intel64
+```
+
+#### OpenBLAS
+
+Via commandline:
+
+Java:
+
+```sh
+-Djava.library.path=/path/to/blas-n-other-dependencies
+```
+
+Spark:
+
+```sh
+spark.executorEnv.LD_LIBRARY_PATH=/path/to/blas-n-other-dependencies
Review comment:
Did someone actually test that? I don't run Spark jobs frequently so I'm
not absolutely sure. If that is working fine, then we can close that ticket
that is still floating around somewhere about native blas on Spark iirc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
