[ https://issues.apache.org/jira/browse/SYSTEMML-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Glenn Weidner updated SYSTEMML-1567: ------------------------------------ Fix Version/s: (was: SystemML 1.0) SystemML 0.15 > Remove conditionals from nn layers > ---------------------------------- > > Key: SYSTEMML-1567 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1567 > Project: SystemML > Issue Type: Improvement > Components: APIs > Affects Versions: SystemML 1.0 > Reporter: Niketan Pansare > Fix For: SystemML 0.15 > > > Conditionals in nn layers introduce transient read/write variables that > disables fused operators such as CP relu_maxpooling_backward and hence > redundant execute sparsity-introducing sel+ operator. This operator causes > unnecessary dense-to-sparse-to-dense conversion and becomes the heavy hitter > after native BLAS change. Note: some fused operators such as CP > relu_maxpooling are still applied because there is no conditional in between > those layers. > Without conditionals in dropout layer: > https://github.com/apache/incubator-systemml/blob/master/scripts/nn/layers/dropout.dml#L49-L53 > > {code} > Iter:2000.0, training loss:0.003149394810197065, training accuracy:100.0 > Iter:2000.0, validation loss:191.9888157354513, validation accuracy:96.875 > SystemML Statistics: > Total elapsed time: 416.609 sec. > Total compilation time: 0.000 sec. > Total execution time: 416.609 sec. > Number of compiled Spark inst: 69. > Number of executed Spark inst: 2. > Native mkl calls (LibMatrixMult/LibMatrixDNN): 4270/10553. > Cache hits (Mem, WB, FS, HDFS): 277973/0/0/0. > Cache writes (WB, FS, HDFS): 143616/0/0. > Cache times (ACQr/m, RLS, EXP): 0.101/0.080/1.988/0.000 sec. > HOP DAGs recompiled (PRED, SB): 0/2277. > HOP DAGs recompile time: 6.146 sec. > Spark ctx create time (lazy): 0.027 sec. > Spark trans counts (par,bc,col):0/0/0. > Spark trans times (par,bc,col): 0.000/0.000/0.000 secs. > Total JIT compile time: 37.746 sec. > Total JVM GC count: 3949. > Total JVM GC time: 56.609 sec. > Heavy hitter instructions (name, time, count): > -- 1) conv2d_bias_add 48.984 sec 4514 > -- 2) conv2d_backward_filter 47.780 sec 4026 > -- 3) -* 38.246 sec 16104 > -- 4) +* 35.902 sec 8052 > -- 5) + 34.227 sec 30566 > -- 6) ba+* 30.643 sec 12566 > -- 7) relu_maxpooling_backward 29.678 sec 4026 > -- 8) conv2d_backward_data 28.520 sec 2013 > -- 9) * 26.825 sec 35275 > -- 10) relu_backward 24.842 sec 6039 > {code} > With conditional, we add sel+ to the heavy hitter: > {code} > -- 1) sel+ 55.054 sec 6283 > {code} > [~mwdus...@us.ibm.com] Since you created the layers, I think you should > decide how best to restructure the DML. My recommendation would be to create > two layers in case of conditionals. > [~mboehm7] [~reinwald] -- This message was sent by Atlassian JIRA (v6.4.14#64029)