[
https://issues.apache.org/jira/browse/SINGA-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059899#comment-15059899
]
ASF subversion and git services commented on SINGA-100:
-------------------------------------------------------
Commit f3b47c706e09315f2eb8eb1d599597093c980962 in incubator-singa's branch
refs/heads/master from [~flytosky]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=f3b47c7 ]
SINGA-100 Implement layers using CUDNN for GPU training
Replace CopyFrom with Copy in softmax loss layers. CopyFrom copies both
cpu and gpu memory, while Copy copies either cpu or gpu memory.
Rename CudnnLayer to CudnnBase, which was changed but missed during rebase..
Fix bugs from getting stuck using GPU (CUDA) operations.
Possible causes include,
1. different threads access the same GPU memory addr. The non-owner thread
would get stuck.
2. cudnnCreate gets stuck if the GPU device is used by other programs.
3. CudnnSoftmaxLossLayer::Setup forgets to call LossLayer::Setup which leaves
layer_conf_ un-initialized.
TODO
1. move cudnn headers and libs before cuda in compiling, otherwise there would
be
link conflicts if there are multiple versions of cudnn in the system.
2. replace malloc with cudaMalloc in blob.cc(h) which pins the CPU
memory for efficient and stable memory transferring between CPU and GPU.
Tested with multiple GPUs and single GPU.
> Implement layers using CUDNN for GPU training
> ---------------------------------------------
>
> Key: SINGA-100
> URL: https://issues.apache.org/jira/browse/SINGA-100
> Project: Singa
> Issue Type: New Feature
> Reporter: wangwei
>
> NVIDIA has released the cudnn library optimized for CNN operations like
> convolution, pooling, etc. It has achieved overall good performance. Hence,
> it is essential to add cudnn supported layers in SINGA for efficient GPU
> training (SINGA-41).
> We will use the cudnn library to implement CNN layers, namely,
> cudnnConvolutionLayer, cudnnPoolingLayer, cudnnLRNLayer, cudnnSoftmaxLayer,
> cudnnReLULayer, cudnnSigmoidLayer, cudnnTanhLayer, cudnnDivNormLayer.
> Data type float-16 will not be consider in this ticket.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
