Hi Mark, I have written down a GPU guide, with sufficient instructions[1]
In this doc, the python and spark-submit instructions are not yet provided. We will mark this for the future. As for the Developer guide is concerned, we need to document internals of the GPU design too[2]. If the internals are not documented explicitly, the feature might get dormant and difficult for other contributors. 1. GPU prototyping 2. Codegeneration 3. Lineage feature 4. Privacy feature if applicable. This process should not be treated as a counter productive one, but a necessity. We could come to this discussion in the future. :) [1] https://apache.github.io/systemds/site/gpu [2] https://github.com/apache/systemds/blob/branch-1.2.0/docs/devdocs/gpu-backend.md#initial-prototype-for-gpu-backend Thank you, Janardhan On Fri, May 7, 2021 at 3:42 AM Mark Dokter <[email protected]> wrote: > On 05/05/2021 02:00, Janardhan wrote: > > Hi all, > > > > Hi! Thank you Janardhan, to take care of this. > > > I am planning to add GPU docs like [1]. Essentially the document should > > answer > > the following questions. Please, feel free to add new points or > contribute > > to > > the discussion. thanks. > > > > 1. What are the ways in which to run SystemDS with gpu? > > For example, > > with -gpu flag > >> spark-submit --jars ./systemds-2.1.0-SNAPSHOT-extra.jar ./SystemDS.jar > -f > > test.dml -gpu > > > > Python is on the todo list now (see Sebastian Baunsgaard's response), > JMLC should work if you set the config but I don't know if we have an > example of that somewhere and for script execution from shell it's the > -gpu flag. There's also a force option. About that spark-submit: Is > running gpu instructions through spark supported (never tried it myself)? > > > 2. What are the CUDNN and CUDA versions supported? > > For example, which SDK[2] version to download > > > > At the moment we're tied to CUDA 10.2 and CUDNN 7.x due to API changes > in CUDNN 8.x. > > > > 3. About JCUDA, do we need to download anything or pom.xml will take care > > of it? > > > > From source should be handled by mvn. If the binary release is used, > everything that is needed should be in the extra jar. > > > 4. How do the python users invoke gpu? > > > > See answer to 1. and Sebastians answer. > > > 5. What is the default precision behaviour (single or double)? > > > > See Arnab's answer. The defaults can be viewed in the > SystemDS-config.xml.template and in DMLConfig.java. > > > 6. Default memory allocation - Unified memory[3]? > > For example, we use `cuda`[4] > > > > See Arnab's response. > > > -- > > [1] http://systemds.apache.org/docs/1.2.0/gpu > > [2] https://developer.nvidia.com/cuda-downloads > > [3] > > > https://developer.nvidia.com/blog/unified-memory-cuda-beginners/#:~:text=What%20is%20Unified%20Memory > > ? > > [4] > > > https://github.com/apache/systemds/blob/5af203a001229a6f4608040ed7e558f59ffe937c/conf/SystemDS-config.xml.template#L95 > > [SYSTEMDS-2970] https://issues.apache.org/jira/browse/SYSTEMDS-2970 > > > > Thank you, > > Janardhan > > > > hth, Mark > >
