[
https://issues.apache.org/jira/browse/SYSTEMML-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Glenn Weidner updated SYSTEMML-1567:
------------------------------------
Fix Version/s: (was: SystemML 1.0)
SystemML 0.15
> Remove conditionals from nn layers
> ----------------------------------
>
> Key: SYSTEMML-1567
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1567
> Project: SystemML
> Issue Type: Improvement
> Components: APIs
> Affects Versions: SystemML 1.0
> Reporter: Niketan Pansare
> Fix For: SystemML 0.15
>
>
> Conditionals in nn layers introduce transient read/write variables that
> disables fused operators such as CP relu_maxpooling_backward and hence
> redundant execute sparsity-introducing sel+ operator. This operator causes
> unnecessary dense-to-sparse-to-dense conversion and becomes the heavy hitter
> after native BLAS change. Note: some fused operators such as CP
> relu_maxpooling are still applied because there is no conditional in between
> those layers.
> Without conditionals in dropout layer:
> https://github.com/apache/incubator-systemml/blob/master/scripts/nn/layers/dropout.dml#L49-L53
>
> {code}
> Iter:2000.0, training loss:0.003149394810197065, training accuracy:100.0
> Iter:2000.0, validation loss:191.9888157354513, validation accuracy:96.875
> SystemML Statistics:
> Total elapsed time: 416.609 sec.
> Total compilation time: 0.000 sec.
> Total execution time: 416.609 sec.
> Number of compiled Spark inst: 69.
> Number of executed Spark inst: 2.
> Native mkl calls (LibMatrixMult/LibMatrixDNN): 4270/10553.
> Cache hits (Mem, WB, FS, HDFS): 277973/0/0/0.
> Cache writes (WB, FS, HDFS): 143616/0/0.
> Cache times (ACQr/m, RLS, EXP): 0.101/0.080/1.988/0.000 sec.
> HOP DAGs recompiled (PRED, SB): 0/2277.
> HOP DAGs recompile time: 6.146 sec.
> Spark ctx create time (lazy): 0.027 sec.
> Spark trans counts (par,bc,col):0/0/0.
> Spark trans times (par,bc,col): 0.000/0.000/0.000 secs.
> Total JIT compile time: 37.746 sec.
> Total JVM GC count: 3949.
> Total JVM GC time: 56.609 sec.
> Heavy hitter instructions (name, time, count):
> -- 1) conv2d_bias_add 48.984 sec 4514
> -- 2) conv2d_backward_filter 47.780 sec 4026
> -- 3) -* 38.246 sec 16104
> -- 4) +* 35.902 sec 8052
> -- 5) + 34.227 sec 30566
> -- 6) ba+* 30.643 sec 12566
> -- 7) relu_maxpooling_backward 29.678 sec 4026
> -- 8) conv2d_backward_data 28.520 sec 2013
> -- 9) * 26.825 sec 35275
> -- 10) relu_backward 24.842 sec 6039
> {code}
> With conditional, we add sel+ to the heavy hitter:
> {code}
> -- 1) sel+ 55.054 sec 6283
> {code}
> [[email protected]] Since you created the layers, I think you should
> decide how best to restructure the DML. My recommendation would be to create
> two layers in case of conditionals.
> [~mboehm7] [~reinwald]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
