WDRshadow opened a new pull request, #2050: URL: https://github.com/apache/systemds/pull/2050
## Main Updated: 1. Fixed a bug that while using multi-GPU with CUDA caused a "CurrentModificationException" error in `ParForProgramBlock`, where the GPU memory is modified during the process of freeing. Multi-GPU can now be used to accerate the `parfor` function and other functions using multiple workers and threads. 2. The mismatch between `_numThreads` and the actual number of threads is resolved when using multiple GPUs device but only allowing a single GPU (`sysds.gpu.availableGPUs=1`) to run `parfor` functions with multiple workers and multiple threads. ## Other bugs in Multi-GPU process: 1. In a multi-GPU environment, when initializing the first instance of `GPUContext` for `cudnnHandle`, `cublasHandle`, and `cusparseHandle` in Jcuda version `10.2.0`, the Native code freezes when executing into Jcuda code. The program cannot continue. This error can easily happen in the test environment of "4070+1080" dual graphics card environment, but sometimes it works fine. This bug is not present in Jcuda version `11.8.0`. This is presumed to be a Jcuda issue and cannot be fixed by SystemDS. ## Other bugs outside Multi-GPU process we found: 1. Example script `scripts/nn/examples/AttentionExample.dml` can not be runned with even one GPU. Error message is `RuntimeException -- Unsupported operator:MAP`. We found that a function (with a high probability that it is the `map` function) passes an operator `_map` to `TernaryOp` class where it is not categorized by a GPU operation. ## TODO List: 1. We'll write a test class to detect whether multi-gpu in `parfor` is actually being implemented. 2. We'll try to fix or work around the multi-GPU run Handle initialization freeze issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@systemds.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
