[jira] [Comment Edited] (SYSTEMML-1774) Improve Parfor parallelism for deep learning

[Fei Hu (JIRA)]

    [ 
https://issues.apache.org/jira/browse/SYSTEMML-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090325#comment-16090325
 ] 

Fei Hu edited comment on SYSTEMML-1774 at 7/17/17 10:22 PM:
------------------------------------------------------------

The errors when running the [distributed MNIST LeNet example |  
https://github.com/apache/systemml/blob/master/scripts/nn/examples/mnist_lenet_distrib_sgd.dml]
 using the mode {{SPARK}} are as following. One example of the errors is that 
at the convolutional layer, we need to randomly generate some matrixes, but 
SystemML choose {{RandSPInstruction}} instead of {{DataGenCPInstruction}}, 
which may be because SystemML could not determine the row number of the matrix. 
For this distributed MNIST LeNet  example, using CPInstruction may achieve 
better performance. 

{code:java}
Exception in thread "main" org.apache.sysml.api.mlcontext.MLContextException: 
Exception when executing script
        at org.apache.sysml.api.mlcontext.MLContext.execute(MLContext.java:320)
        at org.apache.sysml.api.mlcontext.MLContext.execute(MLContext.java:293)
        at org.apache.sysml.api.mlcontext.Script.execute(Script.java:717)
        at 
org.apache.sysml.scripts.nn.examples.Mnist_lenet_distrib_sgd.train(Mnist_lenet_distrib_sgd.java)
        at 
org.apache.sysml.examples.MNIST_Distrib_Sgd$.main(MNIST_Distrib_Sgd.scala:41)
        at 
org.apache.sysml.examples.MNIST_Distrib_Sgd.main(MNIST_Distrib_Sgd.scala)
Caused by: org.apache.sysml.api.mlcontext.MLContextException: Exception 
occurred while executing runtime program
        at 
org.apache.sysml.api.mlcontext.ScriptExecutor.executeRuntimeProgram(ScriptExecutor.java:397)
        at 
org.apache.sysml.api.mlcontext.ScriptExecutor.execute(ScriptExecutor.java:311)
        at org.apache.sysml.api.mlcontext.MLContext.execute(MLContext.java:316)
        ... 5 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: 
org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in program 
block generated from statement block between lines 11 and 11 -- Error 
evaluating instruction: 
CP°extfunct°./scripts/nn/examples/mnist_lenet_distrib_sgd.dml°train°10°8°X·MATRIX·DOUBLE°Y·MATRIX·DOUBLE°X_val·MATRIX·DOUBLE°Y_val·MATRIX·DOUBLE°C·SCALAR·INT·false°Hin·SCALAR·INT·false°Win·SCALAR·INT·false°batch_size·SCALAR·INT·false°parallel_batches·SCALAR·INT·false°epochs·SCALAR·INT·false°W1°b1°W2°b2°W3°b3°W4°b4
        at 
org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:130)
        at 
org.apache.sysml.api.ScriptExecutorUtils.executeRuntimeProgram(ScriptExecutorUtils.java:95)
        at 
org.apache.sysml.api.ScriptExecutorUtils.executeRuntimeProgram(ScriptExecutorUtils.java:54)
        at 
org.apache.sysml.api.mlcontext.ScriptExecutor.executeRuntimeProgram(ScriptExecutor.java:395)
        ... 7 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error 
in program block generated from statement block between lines 11 and 11 -- 
Error evaluating instruction: 
CP°extfunct°./scripts/nn/examples/mnist_lenet_distrib_sgd.dml°train°10°8°X·MATRIX·DOUBLE°Y·MATRIX·DOUBLE°X_val·MATRIX·DOUBLE°Y_val·MATRIX·DOUBLE°C·SCALAR·INT·false°Hin·SCALAR·INT·false°Win·SCALAR·INT·false°batch_size·SCALAR·INT·false°parallel_batches·SCALAR·INT·false°epochs·SCALAR·INT·false°W1°b1°W2°b2°W3°b3°W4°b4
        at 
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:316)
        at 
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:217)
        at 
org.apache.sysml.runtime.controlprogram.ProgramBlock.execute(ProgramBlock.java:163)
        at 
org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:123)
        ... 10 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: error executing 
function ./scripts/nn/examples/mnist_lenet_distrib_sgd.dml::train
        at 
org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:187)
        at 
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:286)
        ... 13 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error 
in function program block generated from function statement block between lines 
36 and 271 -- Error evaluating function program block
        at 
org.apache.sysml.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:122)
        at 
org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:180)
        ... 14 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error 
in for program block generated from for statement block between lines 115 and 
270 -- Error evaluating for program block
        at 
org.apache.sysml.runtime.controlprogram.ForProgramBlock.execute(ForProgramBlock.java:157)
        at 
org.apache.sysml.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:115)
        ... 15 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error 
in for program block generated from for statement block between lines 117 and 
265 -- Error evaluating for program block
        at 
org.apache.sysml.runtime.controlprogram.ForProgramBlock.execute(ForProgramBlock.java:157)
        at 
org.apache.sysml.runtime.controlprogram.ForProgramBlock.execute(ForProgramBlock.java:145)
        ... 16 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: PARFOR: Failed to 
execute loop in parallel.
        at 
org.apache.sysml.runtime.controlprogram.ParForProgramBlock.execute(ParForProgramBlock.java:660)
        at 
org.apache.sysml.runtime.controlprogram.ForProgramBlock.execute(ForProgramBlock.java:145)
        ... 17 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: Not supported: 
Instructions of type other than CP instructions 
org.apache.sysml.runtime.instructions.spark.RandSPInstruction
SPARK°rand°¶_Var3092¶°512°1000°1000°1°1°0.5°-1°scratch_space/_p10640_9.108.91.12//_t0/°uniform°1.0°_mVar3093·MATRIX·DOUBLE
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.serializeInstructions(ProgramConverter.java:937)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.rSerializeProgramBlock(ProgramConverter.java:1297)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.rSerializeProgramBlocks(ProgramConverter.java:1125)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.rSerializeProgramBlock(ProgramConverter.java:1264)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.rSerializeFunctionProgramBlocks(ProgramConverter.java:1106)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.serializeProgram(ProgramConverter.java:805)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.serializeParForBody(ProgramConverter.java:763)
        at 
org.apache.sysml.runtime.controlprogram.ParForProgramBlock.executeRemoteSparkParFor(ParForProgramBlock.java:1034)
        at 
org.apache.sysml.runtime.controlprogram.ParForProgramBlock.execute(ParForProgramBlock.java:647)
        ... 18 more
{code}

The explain information is as following:

{code:java}
# EXPLAIN (RUNTIME):
# Memory Budget local/remote = 2549MB/362MB/543MB/163MB
# Degree of Parallelism (vcores) local/remote = 8/4
PROGRAM ( size CP/SP = 16/6 )
--MAIN PROGRAM
----GENERIC (lines 1-8) [recompile=true]
------CP * C.SCALAR.INT.false Hin.SCALAR.INT.false _Var9.SCALAR.INT
------CP createvar _mVar10 scratch_space//_p16480_9.108.91.12//_t0/temp0 true 
MATRIX binaryblock -1 1 1000 1000 -1 copy
------SPARK seq -1 1 1000 1000 1 ¶N¶ 1 _mVar10.MATRIX.DOUBLE
------CP createvar _mVar11 scratch_space//_p16480_9.108.91.12//_t0/temp1 true 
MATRIX binaryblock -1 1 1000 1000 -1 copy
------SPARK rand ¶N¶ 1 1000 1000 1 ¶K¶ 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar11.MATRIX.DOUBLE
------CP * _Var9.SCALAR.INT.false Win.SCALAR.INT.false _Var12.SCALAR.INT
------CP rmvar _Var9
------CP createvar _mVar13 scratch_space//_p16480_9.108.91.12//_t0/temp2 true 
MATRIX binaryblock -1 1 1000 1000 -1 copy
------SPARK round _mVar11.MATRIX.DOUBLE _mVar13.MATRIX.DOUBLE
------CP rmvar _mVar11
------CP createvar _mVar14 scratch_space//_p16480_9.108.91.12//_t0/temp3 true 
MATRIX binaryblock -1 -1 1000 1000 -1 copy
------SPARK rand ¶N¶ ¶_Var12¶ 1000 1000 0.0 1.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ normal 1.0 _mVar14.MATRIX.DOUBLE
------CP rmvar _Var12
------CP createvar _mVar15 scratch_space//_p16480_9.108.91.12//_t0/temp4 true 
MATRIX binarycell -1 -1 -1 -1 -1 copy
------SPARK ctable _mVar10.MATRIX.DOUBLE _mVar13.MATRIX.DOUBLE 
1.0.SCALAR.DOUBLE.true N.false K.false _mVar15.MATRIX.DOUBLE false
------CP rmvar _mVar10 _mVar13
------CP createvar _mVar16 scratch_space//_p16480_9.108.91.12//_t0/temp5 true 
MATRIX binaryblock -1 -1 1000 1000 -1 copy
------SPARK rblk _mVar15.MATRIX.DOUBLE _mVar16.MATRIX.DOUBLE 1000 1000 true
------CP rmvar _mVar15
------CP cpvar _mVar14 X
------CP cpvar _mVar16 Y
------CP rmvar _mVar14 _mVar16


# EXPLAIN (RUNTIME):
# Memory Budget local/remote = 2549MB/362MB/543MB/163MB
# Degree of Parallelism (vcores) local/remote = 8/4
PROGRAM ( size CP/SP = 16/6 )
--MAIN PROGRAM
----GENERIC (lines 1-8) [recompile=true]
------CP * C.SCALAR.INT.false Hin.SCALAR.INT.false _Var30.SCALAR.INT
------CP createvar _mVar31 scratch_space//_p16480_9.108.91.12//_t0/temp10 true 
MATRIX binaryblock -1 1 1000 1000 -1 copy
------SPARK seq -1 1 1000 1000 1 ¶N¶ 1 _mVar31.MATRIX.DOUBLE
------CP createvar _mVar32 scratch_space//_p16480_9.108.91.12//_t0/temp11 true 
MATRIX binaryblock -1 1 1000 1000 -1 copy
------SPARK rand ¶N¶ 1 1000 1000 1 ¶K¶ 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar32.MATRIX.DOUBLE
------CP * _Var30.SCALAR.INT.false Win.SCALAR.INT.false _Var33.SCALAR.INT
------CP rmvar _Var30
------CP createvar _mVar34 scratch_space//_p16480_9.108.91.12//_t0/temp12 true 
MATRIX binaryblock -1 1 1000 1000 -1 copy
------SPARK round _mVar32.MATRIX.DOUBLE _mVar34.MATRIX.DOUBLE
------CP rmvar _mVar32
------CP createvar _mVar35 scratch_space//_p16480_9.108.91.12//_t0/temp13 true 
MATRIX binaryblock -1 -1 1000 1000 -1 copy
------SPARK rand ¶N¶ ¶_Var33¶ 1000 1000 0.0 1.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ normal 1.0 _mVar35.MATRIX.DOUBLE
------CP rmvar _Var33
------CP createvar _mVar36 scratch_space//_p16480_9.108.91.12//_t0/temp14 true 
MATRIX binarycell -1 -1 -1 -1 -1 copy
------SPARK ctable _mVar31.MATRIX.DOUBLE _mVar34.MATRIX.DOUBLE 
1.0.SCALAR.DOUBLE.true N.false K.false _mVar36.MATRIX.DOUBLE false
------CP rmvar _mVar31 _mVar34
------CP createvar _mVar37 scratch_space//_p16480_9.108.91.12//_t0/temp15 true 
MATRIX binaryblock -1 -1 1000 1000 -1 copy
------SPARK rblk _mVar36.MATRIX.DOUBLE _mVar37.MATRIX.DOUBLE 1000 1000 true
------CP rmvar _mVar36
------CP cpvar _mVar35 X
------CP cpvar _mVar37 Y
------CP rmvar _mVar35 _mVar37



# EXPLAIN (RUNTIME):
# Memory Budget local/remote = 2549MB/362MB/543MB/163MB
# Degree of Parallelism (vcores) local/remote = 8/4
PROGRAM ( size CP/SP = 16/6 )
--MAIN PROGRAM
----GENERIC (lines 1-8) [recompile=true]
------CP * C.SCALAR.INT.false Hin.SCALAR.INT.false _Var51.SCALAR.INT
------CP createvar _mVar52 scratch_space//_p16480_9.108.91.12//_t0/temp20 true 
MATRIX binaryblock -1 1 1000 1000 -1 copy
------SPARK seq -1 1 1000 1000 1 ¶N¶ 1 _mVar52.MATRIX.DOUBLE
------CP createvar _mVar53 scratch_space//_p16480_9.108.91.12//_t0/temp21 true 
MATRIX binaryblock -1 1 1000 1000 -1 copy
------SPARK rand ¶N¶ 1 1000 1000 1 ¶K¶ 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar53.MATRIX.DOUBLE
------CP * _Var51.SCALAR.INT.false Win.SCALAR.INT.false _Var54.SCALAR.INT
------CP rmvar _Var51
------CP createvar _mVar55 scratch_space//_p16480_9.108.91.12//_t0/temp22 true 
MATRIX binaryblock -1 1 1000 1000 -1 copy
------SPARK round _mVar53.MATRIX.DOUBLE _mVar55.MATRIX.DOUBLE
------CP rmvar _mVar53
------CP createvar _mVar56 scratch_space//_p16480_9.108.91.12//_t0/temp23 true 
MATRIX binaryblock -1 -1 1000 1000 -1 copy
------SPARK rand ¶N¶ ¶_Var54¶ 1000 1000 0.0 1.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ normal 1.0 _mVar56.MATRIX.DOUBLE
------CP rmvar _Var54
------CP createvar _mVar57 scratch_space//_p16480_9.108.91.12//_t0/temp24 true 
MATRIX binarycell -1 -1 -1 -1 -1 copy
------SPARK ctable _mVar52.MATRIX.DOUBLE _mVar55.MATRIX.DOUBLE 
1.0.SCALAR.DOUBLE.true N.false K.false _mVar57.MATRIX.DOUBLE false
------CP rmvar _mVar52 _mVar55
------CP createvar _mVar58 scratch_space//_p16480_9.108.91.12//_t0/temp25 true 
MATRIX binaryblock -1 -1 1000 1000 -1 copy
------SPARK rblk _mVar57.MATRIX.DOUBLE _mVar58.MATRIX.DOUBLE 1000 1000 true
------CP rmvar _mVar57
------CP cpvar _mVar56 X
------CP cpvar _mVar58 Y
------CP rmvar _mVar56 _mVar58



# EXPLAIN (RUNTIME):
# Memory Budget local/remote = 2549MB/362MB/543MB/163MB
# Degree of Parallelism (vcores) local/remote = 8/4
PROGRAM ( size CP/SP = 1036/272 )
--FUNCTIONS
----FUNCTION CALL GRAPH
------MAIN PROGRAM
--------./scripts/nn/examples/mnist_lenet_distrib_sgd.dml::train
----------./scripts/nn/examples/mnist_lenet_distrib_sgd.dml::predict
----------./nn/layers/dropout.dml::forward
----FUNCTION ./scripts/nn/examples/mnist_lenet_distrib_sgd.dml::train 
[recompile=true]
------GENERIC (lines 74-113) [recompile=true]
--------CP * C.SCALAR.INT.false 5.SCALAR.INT.true _Var178.SCALAR.INT
--------CP createvar _mVar179 scratch_space//_p16480_9.108.91.12//_t0/temp74 
true MATRIX binaryblock 32 1 1000 1000 0 copy
--------SPARK rand 32 1 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar179.MATRIX.DOUBLE
--------CP createvar _mVar180 scratch_space//_p16480_9.108.91.12//_t0/temp75 
true MATRIX binaryblock 64 800 1000 1000 -1 copy
--------SPARK rand 64 800 1000 1000 0.0 1.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ normal 1.0 _mVar180.MATRIX.DOUBLE
--------CP createvar _mVar181 scratch_space//_p16480_9.108.91.12//_t0/temp76 
true MATRIX binaryblock 64 1 1000 1000 0 copy
--------SPARK rand 64 1 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar181.MATRIX.DOUBLE
--------CP / Hin.SCALAR.INT.false 2.SCALAR.INT.true _Var182.SCALAR.DOUBLE
--------CP / Win.SCALAR.INT.false 2.SCALAR.INT.true _Var183.SCALAR.DOUBLE
--------CP createvar _mVar184 scratch_space//_p16480_9.108.91.12//_t0/temp77 
true MATRIX binaryblock 1 512 1000 1000 0 copy
--------SPARK rand 1 512 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar184.MATRIX.DOUBLE
--------CP createvar _mVar185 scratch_space//_p16480_9.108.91.12//_t0/temp78 
true MATRIX binaryblock 1 10 1000 1000 0 copy
--------SPARK rand 1 10 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar185.MATRIX.DOUBLE
--------CP createvar _mVar186 scratch_space//_p16480_9.108.91.12//_t0/temp79 
true MATRIX binaryblock 512 10 1000 1000 -1 copy
--------SPARK rand 512 10 1000 1000 0.0 1.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ normal 1.0 _mVar186.MATRIX.DOUBLE
--------CP createvar _mVar187 scratch_space//_p16480_9.108.91.12//_t0/temp80 
true MATRIX binaryblock 64 800 1000 1000 0 copy
--------SPARK rand 64 800 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar187.MATRIX.DOUBLE
--------CP createvar _mVar188 scratch_space//_p16480_9.108.91.12//_t0/temp81 
true MATRIX binaryblock 512 10 1000 1000 0 copy
--------SPARK rand 512 10 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar188.MATRIX.DOUBLE
--------CP print Starting optimization.SCALAR.STRING.true _Var189.SCALAR.STRING
--------CP * parallel_batches.SCALAR.INT.false batch_size.SCALAR.INT.false 
_Var190.SCALAR.INT
--------CP + Total Epochs: .SCALAR.STRING.true epochs.SCALAR.INT.false 
_Var191.SCALAR.STRING
--------CP * _Var178.SCALAR.INT.false 5.SCALAR.INT.true _Var192.SCALAR.INT
--------CP rmvar _Var178
--------CP createvar _mVar193 scratch_space//_p16480_9.108.91.12//_t0/temp82 
true MATRIX binaryblock 64 800 1000 1000 -1 copy
--------SPARK * _mVar180.MATRIX.DOUBLE 0.05.SCALAR.DOUBLE.true 
_mVar193.MATRIX.DOUBLE
--------CP rmvar _mVar180
--------CP / _Var182.SCALAR.DOUBLE.false 2.SCALAR.INT.true _Var194.SCALAR.DOUBLE
--------CP rmvar _Var182
--------CP / _Var183.SCALAR.DOUBLE.false 2.SCALAR.INT.true _Var195.SCALAR.DOUBLE
--------CP rmvar _Var183
--------CP createvar _mVar196 scratch_space//_p16480_9.108.91.12//_t0/temp83 
true MATRIX binaryblock 512 10 1000 1000 -1 copy
--------SPARK * _mVar186.MATRIX.DOUBLE 0.0625.SCALAR.DOUBLE.true 
_mVar196.MATRIX.DOUBLE
--------CP rmvar _mVar186
--------CP / 32.SCALAR.INT.true _Var190.SCALAR.INT.false _Var197.SCALAR.DOUBLE
--------CP + _Var191.SCALAR.STRING.false , Batch size: .SCALAR.STRING.true 
_Var198.SCALAR.STRING
--------CP rmvar _Var191
--------CP createvar _mVar199 scratch_space//_p16480_9.108.91.12//_t0/temp84 
true MATRIX binaryblock 32 -1 1000 1000 -1 copy
--------SPARK rand 32 ¶_Var192¶ 1000 1000 0.0 1.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ normal 1.0 _mVar199.MATRIX.DOUBLE
--------CP / 2.0.SCALAR.DOUBLE.true _Var192.SCALAR.INT.false 
_Var200.SCALAR.DOUBLE
--------CP rmvar _Var192
--------CP * 64.SCALAR.INT.true _Var194.SCALAR.DOUBLE.false 
_Var201.SCALAR.DOUBLE
--------CP rmvar _Var194
--------CP createvar _mVar202 scratch_space//_p16480_9.108.91.12//_t0/temp85 
true MATRIX binaryblock 512 10 1000 1000 -1 copy
--------SPARK / _mVar196.MATRIX.DOUBLE 1.4142135623730951.SCALAR.DOUBLE.true 
_mVar202.MATRIX.DOUBLE
--------CP rmvar _mVar196
--------CP ceil _Var197.SCALAR.DOUBLE.false _Var203.SCALAR.DOUBLE
--------CP rmvar _Var197
--------CP + _Var198.SCALAR.STRING.false batch_size.SCALAR.INT.false 
_Var204.SCALAR.STRING
--------CP rmvar _Var198
--------CP sqrt _Var200.SCALAR.DOUBLE.false _Var205.SCALAR.DOUBLE
--------CP rmvar _Var200
--------CP * _Var201.SCALAR.DOUBLE.false _Var195.SCALAR.DOUBLE.false 
_Var206.SCALAR.DOUBLE
--------CP rmvar _Var201 _Var195
--------CP castvti _Var203.SCALAR.DOUBLE.false _Var207.SCALAR.INT
--------CP rmvar _Var203
--------CP + _Var204.SCALAR.STRING.false , Degree of parallelism: 
.SCALAR.STRING.true _Var208.SCALAR.STRING
--------CP rmvar _Var204
--------CP createvar _mVar209 scratch_space//_p16480_9.108.91.12//_t0/temp86 
true MATRIX binaryblock 32 -1 1000 1000 -1 copy
--------SPARK * _mVar199.MATRIX.DOUBLE _Var205.SCALAR.DOUBLE.false 
_mVar209.MATRIX.DOUBLE
--------CP rmvar _mVar199 _Var205
--------CP createvar _mVar210 scratch_space//_p16480_9.108.91.12//_t0/temp87 
true MATRIX binaryblock -1 512 1000 1000 -1 copy
--------SPARK rand ¶_Var206¶ 512 1000 1000 0.0 1.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ normal 1.0 _mVar210.MATRIX.DOUBLE
--------CP / 2.0.SCALAR.DOUBLE.true _Var206.SCALAR.DOUBLE.false 
_Var211.SCALAR.DOUBLE
--------CP rmvar _Var206
--------CP + _Var208.SCALAR.STRING.false parallel_batches.SCALAR.INT.false 
_Var212.SCALAR.STRING
--------CP rmvar _Var208
--------CP sqrt _Var211.SCALAR.DOUBLE.false _Var213.SCALAR.DOUBLE
--------CP rmvar _Var211
--------CP ncol _mVar209.MATRIX.DOUBLE.false _Var214.SCALAR.INT
--------CP + _Var212.SCALAR.STRING.false , Group batch size: 
.SCALAR.STRING.true _Var215.SCALAR.STRING
--------CP rmvar _Var212
--------CP createvar _mVar216 scratch_space//_p16480_9.108.91.12//_t0/temp88 
true MATRIX binaryblock -1 512 1000 1000 -1 copy
--------SPARK * _mVar210.MATRIX.DOUBLE _Var213.SCALAR.DOUBLE.false 
_mVar216.MATRIX.DOUBLE
--------CP rmvar _mVar210 _Var213
--------CP createvar _mVar217 scratch_space//_p16480_9.108.91.12//_t0/temp89 
true MATRIX binaryblock 32 -1 1000 1000 0 copy
--------SPARK rand 32 ¶_Var214¶ 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar217.MATRIX.DOUBLE
--------CP rmvar _Var214
--------CP + _Var215.SCALAR.STRING.false _Var190.SCALAR.INT.false 
_Var218.SCALAR.STRING
--------CP rmvar _Var215
--------CP nrow _mVar216.MATRIX.DOUBLE.false _Var219.SCALAR.INT
--------CP + _Var218.SCALAR.STRING.false , Num groups: .SCALAR.STRING.true 
_Var220.SCALAR.STRING
--------CP rmvar _Var218
--------CP createvar _mVar221 scratch_space//_p16480_9.108.91.12//_t0/temp90 
true MATRIX binaryblock -1 512 1000 1000 0 copy
--------SPARK rand ¶_Var219¶ 512 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar221.MATRIX.DOUBLE
--------CP rmvar _Var219
--------CP + _Var220.SCALAR.STRING.false _Var207.SCALAR.INT.false 
_Var222.SCALAR.STRING
--------CP rmvar _Var220
--------CP print _Var222.SCALAR.STRING.false _Var223.SCALAR.STRING
--------CP rmvar _Var222
--------CP assignvar 32.SCALAR.INT.true N.SCALAR.INT
--------CP assignvar 5.SCALAR.INT.true Hf.SCALAR.INT
--------CP assignvar 5.SCALAR.INT.true Wf.SCALAR.INT
--------CP assignvar 1.SCALAR.INT.true stride.SCALAR.INT
--------CP assignvar 2.SCALAR.INT.true pad.SCALAR.INT
--------CP assignvar 32.SCALAR.INT.true F1.SCALAR.INT
--------CP assignvar 64.SCALAR.INT.true F2.SCALAR.INT
--------CP assignvar 0.01.SCALAR.DOUBLE.true lr.SCALAR.DOUBLE
--------CP assignvar 0.9.SCALAR.DOUBLE.true mu.SCALAR.DOUBLE
--------CP assignvar 0.95.SCALAR.DOUBLE.true decay.SCALAR.DOUBLE
--------CP assignvar 5.0E-4.SCALAR.DOUBLE.true lambda.SCALAR.DOUBLE
--------CP assignvar _Var190.SCALAR.INT.false group_batch_size.SCALAR.INT
--------CP assignvar _Var207.SCALAR.INT.false groups.SCALAR.INT
--------CP rmvar _Var189 _Var190
--------CP cpvar _mVar179 b1
--------CP cpvar _mVar181 b2
--------CP cpvar _mVar184 b3
--------CP cpvar _mVar185 b4
--------CP cpvar _mVar179 vb1
--------CP cpvar _mVar187 vW2
--------CP cpvar _mVar181 vb2
--------CP cpvar _mVar184 vb3
--------CP cpvar _mVar188 vW4
--------CP cpvar _mVar185 vb4
--------CP cpvar _mVar193 W2
--------CP cpvar _mVar202 W4
--------CP rmvar _Var207
--------CP cpvar _mVar209 W1
--------CP cpvar _mVar216 W3
--------CP cpvar _mVar217 vW1
--------CP cpvar _mVar221 vW3
--------CP rmvar _Var223 _mVar179 _mVar181 _mVar184 _mVar185 _mVar187 _mVar188 
_mVar193 _mVar202 _mVar209 _mVar216 _mVar217 _mVar221
------GENERIC (lines 115-270) [recompile=true]
--------CP createvar _mVar224 scratch_space//_p16480_9.108.91.12//_t0/temp91 
true MATRIX binaryblock 32 10 1000 1000 -1 copy
--------SPARK chkpoint Y_val.MATRIX.DOUBLE _mVar224.MATRIX.DOUBLE 
MEMORY_AND_DISK
--------CP createvar _mVar225 scratch_space//_p16480_9.108.91.12//_t0/temp92 
true MATRIX binaryblock 32 37632 1000 1000 1204224 copy
--------SPARK chkpoint X.MATRIX.DOUBLE _mVar225.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar226 scratch_space//_p16480_9.108.91.12//_t0/temp93 
true MATRIX binaryblock 32 10 1000 1000 -1 copy
--------SPARK chkpoint Y.MATRIX.DOUBLE _mVar226.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar227 scratch_space//_p16480_9.108.91.12//_t0/temp94 
true MATRIX binaryblock 32 37632 1000 1000 1204224 copy
--------SPARK chkpoint X_val.MATRIX.DOUBLE _mVar227.MATRIX.DOUBLE 
MEMORY_AND_DISK
--------CP rmvar Y_val X Y X_val
--------CP cpvar _mVar224 Y_val
--------CP cpvar _mVar225 X
--------CP cpvar _mVar226 Y
--------CP cpvar _mVar227 X_val
--------CP rmvar _mVar224 _mVar225 _mVar226 _mVar227
------GENERIC (lines 115-270) [recompile=true]
--------CP createvar _mVar228 scratch_space//_p16480_9.108.91.12//_t0/temp95 
true MATRIX binaryblock 32 10 1000 1000 -1 copy
--------SPARK chkpoint Y_val.MATRIX.DOUBLE _mVar228.MATRIX.DOUBLE 
MEMORY_AND_DISK
--------CP createvar _mVar229 scratch_space//_p16480_9.108.91.12//_t0/temp96 
true MATRIX binaryblock 32 37632 1000 1000 1204224 copy
--------SPARK chkpoint X.MATRIX.DOUBLE _mVar229.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar230 scratch_space//_p16480_9.108.91.12//_t0/temp97 
true MATRIX binaryblock 32 10 1000 1000 -1 copy
--------SPARK chkpoint Y.MATRIX.DOUBLE _mVar230.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar231 scratch_space//_p16480_9.108.91.12//_t0/temp98 
true MATRIX binaryblock 32 37632 1000 1000 1204224 copy
--------SPARK chkpoint X_val.MATRIX.DOUBLE _mVar231.MATRIX.DOUBLE 
MEMORY_AND_DISK
--------CP rmvar Y_val X Y X_val
--------CP cpvar _mVar228 Y_val
--------CP cpvar _mVar229 X
--------CP cpvar _mVar230 Y
--------CP cpvar _mVar231 X_val
--------CP rmvar _mVar228 _mVar229 _mVar230 _mVar231
------GENERIC (lines 115-270) [recompile=true]
--------CP createvar _mVar232 scratch_space//_p16480_9.108.91.12//_t0/temp99 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint Y_val.MATRIX.DOUBLE _mVar232.MATRIX.DOUBLE 
MEMORY_AND_DISK
--------CP createvar _mVar233 scratch_space//_p16480_9.108.91.12//_t0/temp100 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint X.MATRIX.DOUBLE _mVar233.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar234 scratch_space//_p16480_9.108.91.12//_t0/temp101 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint Y.MATRIX.DOUBLE _mVar234.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar235 scratch_space//_p16480_9.108.91.12//_t0/temp102 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint X_val.MATRIX.DOUBLE _mVar235.MATRIX.DOUBLE 
MEMORY_AND_DISK
--------CP rmvar Y_val X Y X_val
--------CP cpvar _mVar232 Y_val
--------CP cpvar _mVar233 X
--------CP cpvar _mVar234 Y
--------CP cpvar _mVar235 X_val
--------CP rmvar _mVar232 _mVar233 _mVar234 _mVar235
------FOR (lines 115-270)
--------GENERIC (lines 117-265) [recompile=true]
----------CP createvar _mVar236 scratch_space//_p16480_9.108.91.12//_t0/temp103 
true MATRIX binaryblock 32 10 1000 1000 -1 copy
----------SPARK chkpoint Y_val.MATRIX.DOUBLE _mVar236.MATRIX.DOUBLE 
MEMORY_AND_DISK
----------CP createvar _mVar237 scratch_space//_p16480_9.108.91.12//_t0/temp104 
true MATRIX binaryblock 32 37632 1000 1000 1204224 copy
----------SPARK chkpoint X.MATRIX.DOUBLE _mVar237.MATRIX.DOUBLE MEMORY_AND_DISK
----------CP createvar _mVar238 scratch_space//_p16480_9.108.91.12//_t0/temp105 
true MATRIX binaryblock 32 10 1000 1000 -1 copy
----------SPARK chkpoint Y.MATRIX.DOUBLE _mVar238.MATRIX.DOUBLE MEMORY_AND_DISK
----------CP createvar _mVar239 scratch_space//_p16480_9.108.91.12//_t0/temp106 
true MATRIX binaryblock 32 37632 1000 1000 1204224 copy
----------SPARK chkpoint X_val.MATRIX.DOUBLE _mVar239.MATRIX.DOUBLE 
MEMORY_AND_DISK
----------CP rmvar Y_val X Y X_val
----------CP cpvar _mVar236 Y_val
----------CP cpvar _mVar237 X
----------CP cpvar _mVar238 Y
----------CP cpvar _mVar239 X_val
----------CP rmvar _mVar236 _mVar237 _mVar238 _mVar239
--------GENERIC (lines 117-265) [recompile=true]
----------CP createvar _mVar240 scratch_space//_p16480_9.108.91.12//_t0/temp107 
true MATRIX binaryblock 32 10 1000 1000 -1 copy
----------SPARK chkpoint Y_val.MATRIX.DOUBLE _mVar240.MATRIX.DOUBLE 
MEMORY_AND_DISK
----------CP createvar _mVar241 scratch_space//_p16480_9.108.91.12//_t0/temp108 
true MATRIX binaryblock 32 37632 1000 1000 1204224 copy
----------SPARK chkpoint X.MATRIX.DOUBLE _mVar241.MATRIX.DOUBLE MEMORY_AND_DISK
----------CP createvar _mVar242 scratch_space//_p16480_9.108.91.12//_t0/temp109 
true MATRIX binaryblock 32 10 1000 1000 -1 copy
----------SPARK chkpoint Y.MATRIX.DOUBLE _mVar242.MATRIX.DOUBLE MEMORY_AND_DISK
----------CP createvar _mVar243 scratch_space//_p16480_9.108.91.12//_t0/temp110 
true MATRIX binaryblock 32 37632 1000 1000 1204224 copy
----------SPARK chkpoint X_val.MATRIX.DOUBLE _mVar243.MATRIX.DOUBLE 
MEMORY_AND_DISK
----------CP rmvar Y_val X Y X_val
----------CP cpvar _mVar240 Y_val
----------CP cpvar _mVar241 X
----------CP cpvar _mVar242 Y
----------CP cpvar _mVar243 X_val
----------CP rmvar _mVar240 _mVar241 _mVar242 _mVar243
--------GENERIC (lines 117-265) [recompile=true]
----------CP createvar _mVar244 scratch_space//_p16480_9.108.91.12//_t0/temp111 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK chkpoint Y_val.MATRIX.DOUBLE _mVar244.MATRIX.DOUBLE 
MEMORY_AND_DISK
----------CP createvar _mVar245 scratch_space//_p16480_9.108.91.12//_t0/temp112 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK chkpoint X.MATRIX.DOUBLE _mVar245.MATRIX.DOUBLE MEMORY_AND_DISK
----------CP createvar _mVar246 scratch_space//_p16480_9.108.91.12//_t0/temp113 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK chkpoint Y.MATRIX.DOUBLE _mVar246.MATRIX.DOUBLE MEMORY_AND_DISK
----------CP createvar _mVar247 scratch_space//_p16480_9.108.91.12//_t0/temp114 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK chkpoint X_val.MATRIX.DOUBLE _mVar247.MATRIX.DOUBLE 
MEMORY_AND_DISK
----------CP rmvar Y_val X Y X_val
----------CP cpvar _mVar244 Y_val
----------CP cpvar _mVar245 X
----------CP cpvar _mVar246 Y
----------CP cpvar _mVar247 X_val
----------CP rmvar _mVar244 _mVar245 _mVar246 _mVar247
--------FOR (lines 117-265)
----------GENERIC (lines 121-134) [recompile=true]
------------CP - g.SCALAR.INT.false 1.SCALAR.INT.true _Var248.SCALAR.INT
------------CP ncol W1.MATRIX.DOUBLE.false _Var249.SCALAR.INT
------------CP createvar _mVar250 
scratch_space//_p16480_9.108.91.12//_t0/temp115 true MATRIX binaryblock -1 
51200 1000 1000 0 copy
------------SPARK rand ¶parallel_batches¶ 51200 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar250.MATRIX.DOUBLE
------------CP nrow W3.MATRIX.DOUBLE.false _Var251.SCALAR.INT
------------CP createvar _mVar252 
scratch_space//_p16480_9.108.91.12//_t0/temp116 true MATRIX binaryblock -1 5120 
1000 1000 0 copy
------------SPARK rand ¶parallel_batches¶ 5120 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar252.MATRIX.DOUBLE
------------CP createvar _mVar253 
scratch_space//_p16480_9.108.91.12//_t0/temp117 true MATRIX binaryblock -1 32 
1000 1000 0 copy
------------SPARK rand ¶parallel_batches¶ 32 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar253.MATRIX.DOUBLE
------------CP createvar _mVar254 
scratch_space//_p16480_9.108.91.12//_t0/temp118 true MATRIX binaryblock -1 64 
1000 1000 0 copy
------------SPARK rand ¶parallel_batches¶ 64 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar254.MATRIX.DOUBLE
------------CP createvar _mVar255 
scratch_space//_p16480_9.108.91.12//_t0/temp119 true MATRIX binaryblock -1 512 
1000 1000 0 copy
------------SPARK rand ¶parallel_batches¶ 512 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar255.MATRIX.DOUBLE
------------CP createvar _mVar256 
scratch_space//_p16480_9.108.91.12//_t0/temp120 true MATRIX binaryblock -1 10 
1000 1000 0 copy
------------SPARK rand ¶parallel_batches¶ 10 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar256.MATRIX.DOUBLE
------------CP * _Var248.SCALAR.INT.false group_batch_size.SCALAR.INT.false 
_Var257.SCALAR.INT
------------CP rmvar _Var248
------------CP * 32.SCALAR.INT.true _Var249.SCALAR.INT.false _Var258.SCALAR.INT
------------CP rmvar _Var249
------------CP * _Var251.SCALAR.INT.false 512.SCALAR.INT.true _Var259.SCALAR.INT
------------CP rmvar _Var251
------------CP %% _Var257.SCALAR.INT.false 32.SCALAR.INT.true _Var260.SCALAR.INT
------------CP rmvar _Var257
------------CP createvar _mVar261 
scratch_space//_p16480_9.108.91.12//_t0/temp121 true MATRIX binaryblock -1 -1 
1000 1000 0 copy
------------SPARK rand ¶parallel_batches¶ ¶_Var258¶ 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar261.MATRIX.DOUBLE
------------CP rmvar _Var258
------------CP createvar _mVar262 
scratch_space//_p16480_9.108.91.12//_t0/temp122 true MATRIX binaryblock -1 -1 
1000 1000 0 copy
------------SPARK rand ¶parallel_batches¶ ¶_Var259¶ 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar262.MATRIX.DOUBLE
------------CP rmvar _Var259
------------CP + _Var260.SCALAR.INT.false 1.SCALAR.INT.true _Var263.SCALAR.INT
------------CP rmvar _Var260
------------CP + _Var263.SCALAR.INT.false group_batch_size.SCALAR.INT.false 
_Var264.SCALAR.INT
------------CP - _Var264.SCALAR.INT.false 1.SCALAR.INT.true _Var265.SCALAR.INT
------------CP rmvar _Var264
------------CP min 32.SCALAR.INT.true _Var265.SCALAR.INT.false 
_Var266.SCALAR.INT
------------CP rmvar _Var265
------------CP createvar _mVar267 
scratch_space//_p16480_9.108.91.12//_t0/temp123 true MATRIX binaryblock -1 
37632 1000 1000 -1 copy
------------SPARK rangeReIndex X.MATRIX.DOUBLE _Var263.SCALAR.INT.false 
_Var266.SCALAR.INT.false 1.SCALAR.INT.true 37632.SCALAR.INT.true 
_mVar267.MATRIX.DOUBLE MULTI_BLOCK
------------CP createvar _mVar268 
scratch_space//_p16480_9.108.91.12//_t0/temp124 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
------------SPARK rangeReIndex Y.MATRIX.DOUBLE _Var263.SCALAR.INT.false 
_Var266.SCALAR.INT.false 1.SCALAR.INT.true 10.SCALAR.INT.true 
_mVar268.MATRIX.DOUBLE MULTI_BLOCK
------------CP rmvar _Var263 _Var266
------------CP cpvar _mVar250 dW2_agg
------------CP cpvar _mVar252 dW4_agg
------------CP cpvar _mVar253 db1_agg
------------CP cpvar _mVar254 db2_agg
------------CP cpvar _mVar255 db3_agg
------------CP cpvar _mVar256 db4_agg
------------CP cpvar _mVar261 dW1_agg
------------CP cpvar _mVar262 dW3_agg
------------CP cpvar _mVar267 X_group_batch
------------CP cpvar _mVar268 y_group_batch
------------CP rmvar _mVar250 _mVar252 _mVar253 _mVar254 _mVar255 _mVar256 
_mVar261 _mVar262 _mVar267 _mVar268
----------GENERIC (lines 137-213) [recompile=true]
------------CP createvar _mVar269 
scratch_space//_p16480_9.108.91.12//_t0/temp125 true MATRIX binaryblock 32 1 
1000 1000 -1 copy
------------SPARK chkpoint b1.MATRIX.DOUBLE _mVar269.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar270 
scratch_space//_p16480_9.108.91.12//_t0/temp126 true MATRIX binaryblock 64 1 
1000 1000 -1 copy
------------SPARK chkpoint b2.MATRIX.DOUBLE _mVar270.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar271 
scratch_space//_p16480_9.108.91.12//_t0/temp127 true MATRIX binaryblock 1 512 
1000 1000 -1 copy
------------SPARK chkpoint b3.MATRIX.DOUBLE _mVar271.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar272 
scratch_space//_p16480_9.108.91.12//_t0/temp128 true MATRIX binaryblock 1 10 
1000 1000 -1 copy
------------SPARK chkpoint b4.MATRIX.DOUBLE _mVar272.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar273 
scratch_space//_p16480_9.108.91.12//_t0/temp129 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
------------SPARK chkpoint y_group_batch.MATRIX.DOUBLE _mVar273.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar274 
scratch_space//_p16480_9.108.91.12//_t0/temp130 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
------------SPARK chkpoint W1.MATRIX.DOUBLE _mVar274.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar275 
scratch_space//_p16480_9.108.91.12//_t0/temp131 true MATRIX binaryblock 64 800 
1000 1000 -1 copy
------------SPARK chkpoint W2.MATRIX.DOUBLE _mVar275.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar276 
scratch_space//_p16480_9.108.91.12//_t0/temp132 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
------------SPARK chkpoint W3.MATRIX.DOUBLE _mVar276.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar277 
scratch_space//_p16480_9.108.91.12//_t0/temp133 true MATRIX binaryblock 512 10 
1000 1000 -1 copy
------------SPARK chkpoint W4.MATRIX.DOUBLE _mVar277.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar278 
scratch_space//_p16480_9.108.91.12//_t0/temp134 true MATRIX binaryblock -1 
37632 1000 1000 -1 copy
------------SPARK chkpoint X_group_batch.MATRIX.DOUBLE _mVar278.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP rmvar b2 b3 b4 y_group_batch W1 W2 W3 W4 X_group_batch b1
------------CP cpvar _mVar269 b1
------------CP cpvar _mVar270 b2
------------CP cpvar _mVar271 b3
------------CP cpvar _mVar272 b4
------------CP cpvar _mVar273 y_group_batch
------------CP cpvar _mVar274 W1
------------CP cpvar _mVar275 W2
------------CP cpvar _mVar276 W3
------------CP cpvar _mVar277 W4
------------CP cpvar _mVar278 X_group_batch
------------CP rmvar _mVar269 _mVar270 _mVar271 _mVar272 _mVar273 _mVar274 
_mVar275 _mVar276 _mVar277 _mVar278
----------GENERIC (lines 137-213) [recompile=true]
------------CP createvar _mVar279 
scratch_space//_p16480_9.108.91.12//_t0/temp135 true MATRIX binaryblock 32 1 
1000 1000 -1 copy
------------SPARK chkpoint b1.MATRIX.DOUBLE _mVar279.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar280 
scratch_space//_p16480_9.108.91.12//_t0/temp136 true MATRIX binaryblock 64 1 
1000 1000 -1 copy
------------SPARK chkpoint b2.MATRIX.DOUBLE _mVar280.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar281 
scratch_space//_p16480_9.108.91.12//_t0/temp137 true MATRIX binaryblock 1 512 
1000 1000 -1 copy
------------SPARK chkpoint b3.MATRIX.DOUBLE _mVar281.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar282 
scratch_space//_p16480_9.108.91.12//_t0/temp138 true MATRIX binaryblock 1 10 
1000 1000 -1 copy
------------SPARK chkpoint b4.MATRIX.DOUBLE _mVar282.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar283 
scratch_space//_p16480_9.108.91.12//_t0/temp139 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
------------SPARK chkpoint y_group_batch.MATRIX.DOUBLE _mVar283.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar284 
scratch_space//_p16480_9.108.91.12//_t0/temp140 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
------------SPARK chkpoint W1.MATRIX.DOUBLE _mVar284.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar285 
scratch_space//_p16480_9.108.91.12//_t0/temp141 true MATRIX binaryblock 64 800 
1000 1000 -1 copy
------------SPARK chkpoint W2.MATRIX.DOUBLE _mVar285.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar286 
scratch_space//_p16480_9.108.91.12//_t0/temp142 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
------------SPARK chkpoint W3.MATRIX.DOUBLE _mVar286.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar287 
scratch_space//_p16480_9.108.91.12//_t0/temp143 true MATRIX binaryblock 512 10 
1000 1000 -1 copy
------------SPARK chkpoint W4.MATRIX.DOUBLE _mVar287.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar288 
scratch_space//_p16480_9.108.91.12//_t0/temp144 true MATRIX binaryblock -1 
37632 1000 1000 -1 copy
------------SPARK chkpoint X_group_batch.MATRIX.DOUBLE _mVar288.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP rmvar b2 b3 b4 y_group_batch W1 W2 W3 W4 X_group_batch b1
------------CP cpvar _mVar279 b1
------------CP cpvar _mVar280 b2
------------CP cpvar _mVar281 b3
------------CP cpvar _mVar282 b4
------------CP cpvar _mVar283 y_group_batch
------------CP cpvar _mVar284 W1
------------CP cpvar _mVar285 W2
------------CP cpvar _mVar286 W3
------------CP cpvar _mVar287 W4
------------CP cpvar _mVar288 X_group_batch
------------CP rmvar _mVar279 _mVar280 _mVar281 _mVar282 _mVar283 _mVar284 
_mVar285 _mVar286 _mVar287 _mVar288
----------GENERIC (lines 137-213) [recompile=true]
------------CP createvar _mVar289 
scratch_space//_p16480_9.108.91.12//_t0/temp145 true MATRIX binaryblock -1 1 
1000 1000 -1 copy
------------SPARK chkpoint b1.MATRIX.DOUBLE _mVar289.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar290 
scratch_space//_p16480_9.108.91.12//_t0/temp146 true MATRIX binaryblock -1 1 
1000 1000 -1 copy
------------SPARK chkpoint b2.MATRIX.DOUBLE _mVar290.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar291 
scratch_space//_p16480_9.108.91.12//_t0/temp147 true MATRIX binaryblock 1 -1 
1000 1000 -1 copy
------------SPARK chkpoint b3.MATRIX.DOUBLE _mVar291.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar292 
scratch_space//_p16480_9.108.91.12//_t0/temp148 true MATRIX binaryblock 1 -1 
1000 1000 -1 copy
------------SPARK chkpoint b4.MATRIX.DOUBLE _mVar292.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar293 
scratch_space//_p16480_9.108.91.12//_t0/temp149 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
------------SPARK chkpoint y_group_batch.MATRIX.DOUBLE _mVar293.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar294 
scratch_space//_p16480_9.108.91.12//_t0/temp150 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
------------SPARK chkpoint W1.MATRIX.DOUBLE _mVar294.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar295 
scratch_space//_p16480_9.108.91.12//_t0/temp151 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
------------SPARK chkpoint W2.MATRIX.DOUBLE _mVar295.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar296 
scratch_space//_p16480_9.108.91.12//_t0/temp152 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
------------SPARK chkpoint W3.MATRIX.DOUBLE _mVar296.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar297 
scratch_space//_p16480_9.108.91.12//_t0/temp153 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
------------SPARK chkpoint W4.MATRIX.DOUBLE _mVar297.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP createvar _mVar298 
scratch_space//_p16480_9.108.91.12//_t0/temp154 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
------------SPARK chkpoint X_group_batch.MATRIX.DOUBLE _mVar298.MATRIX.DOUBLE 
MEMORY_AND_DISK
------------CP rmvar b2 b3 b4 y_group_batch W1 W2 W3 W4 X_group_batch b1
------------CP cpvar _mVar289 b1
------------CP cpvar _mVar290 b2
------------CP cpvar _mVar291 b3
------------CP cpvar _mVar292 b4
------------CP cpvar _mVar293 y_group_batch
------------CP cpvar _mVar294 W1
------------CP cpvar _mVar295 W2
------------CP cpvar _mVar296 W3
------------CP cpvar _mVar297 W4
------------CP cpvar _mVar298 X_group_batch
------------CP rmvar _mVar289 _mVar290 _mVar291 _mVar292 _mVar293 _mVar294 
_mVar295 _mVar296 _mVar297 _mVar298
----------PARFOR (lines 137-213)
------------GENERIC (lines 139-162) [recompile=true]
--------------CP - j.SCALAR.INT.false 1.SCALAR.INT.true _Var299.SCALAR.INT
--------------CP nrow X_group_batch.MATRIX.DOUBLE.false _Var300.SCALAR.INT
--------------CP + Hin.SCALAR.INT.false 4.SCALAR.INT.true _Var301.SCALAR.INT
--------------CP + Win.SCALAR.INT.false 4.SCALAR.INT.true _Var302.SCALAR.INT
--------------CP * _Var299.SCALAR.INT.false batch_size.SCALAR.INT.false 
_Var303.SCALAR.INT
--------------CP rmvar _Var299
--------------CP - _Var301.SCALAR.INT.false 5.SCALAR.INT.true _Var304.SCALAR.INT
--------------CP rmvar _Var301
--------------CP - _Var302.SCALAR.INT.false 5.SCALAR.INT.true _Var305.SCALAR.INT
--------------CP rmvar _Var302
--------------CP %% _Var303.SCALAR.INT.false _Var300.SCALAR.INT.false 
_Var306.SCALAR.INT
--------------CP rmvar _Var303
--------------CP / _Var304.SCALAR.INT.false 1.SCALAR.INT.true 
_Var307.SCALAR.DOUBLE
--------------CP rmvar _Var304
--------------CP / _Var305.SCALAR.INT.false 1.SCALAR.INT.true 
_Var308.SCALAR.DOUBLE
--------------CP rmvar _Var305
--------------CP + _Var306.SCALAR.INT.false 1.SCALAR.INT.true _Var309.SCALAR.INT
--------------CP rmvar _Var306
--------------CP + _Var307.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var310.SCALAR.DOUBLE
--------------CP rmvar _Var307
--------------CP + _Var308.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var311.SCALAR.DOUBLE
--------------CP rmvar _Var308
--------------CP + _Var309.SCALAR.INT.false batch_size.SCALAR.INT.false 
_Var312.SCALAR.INT
--------------CP floor _Var310.SCALAR.DOUBLE.false _Var313.SCALAR.DOUBLE
--------------CP rmvar _Var310
--------------CP floor _Var311.SCALAR.DOUBLE.false _Var314.SCALAR.DOUBLE
--------------CP rmvar _Var311
--------------CP - _Var312.SCALAR.INT.false 1.SCALAR.INT.true _Var315.SCALAR.INT
--------------CP rmvar _Var312
--------------CP castvti _Var313.SCALAR.DOUBLE.false _Var316.SCALAR.INT
--------------CP rmvar _Var313
--------------CP castvti _Var314.SCALAR.DOUBLE.false _Var317.SCALAR.INT
--------------CP rmvar _Var314
--------------CP min _Var300.SCALAR.INT.false _Var315.SCALAR.INT.false 
_Var318.SCALAR.INT
--------------CP rmvar _Var300 _Var315
--------------CP + _Var316.SCALAR.INT.false 0.SCALAR.INT.true _Var319.SCALAR.INT
--------------CP + _Var317.SCALAR.INT.false 0.SCALAR.INT.true _Var320.SCALAR.INT
--------------CP createvar _mVar321 
scratch_space//_p16480_9.108.91.12//_t0/temp155 true MATRIX binaryblock -1 
37632 1000 1000 -1 copy
--------------SPARK rangeReIndex X_group_batch.MATRIX.DOUBLE 
_Var309.SCALAR.INT.false _Var318.SCALAR.INT.false 1.SCALAR.INT.true 
37632.SCALAR.INT.true _mVar321.MATRIX.DOUBLE MULTI_BLOCK
--------------CP createvar _mVar322 
scratch_space//_p16480_9.108.91.12//_t0/temp156 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK rangeReIndex y_group_batch.MATRIX.DOUBLE 
_Var309.SCALAR.INT.false _Var318.SCALAR.INT.false 1.SCALAR.INT.true 
10.SCALAR.INT.true _mVar322.MATRIX.DOUBLE MULTI_BLOCK
--------------CP rmvar _Var309 _Var318
--------------CP - _Var319.SCALAR.INT.false 2.SCALAR.INT.true _Var323.SCALAR.INT
--------------CP rmvar _Var319
--------------CP - _Var320.SCALAR.INT.false 2.SCALAR.INT.true _Var324.SCALAR.INT
--------------CP rmvar _Var320
--------------CP nrow _mVar321.MATRIX.DOUBLE.false _Var325.SCALAR.INT
--------------CP / _Var323.SCALAR.INT.false 2.SCALAR.INT.true 
_Var326.SCALAR.DOUBLE
--------------CP rmvar _Var323
--------------CP / _Var324.SCALAR.INT.false 2.SCALAR.INT.true 
_Var327.SCALAR.DOUBLE
--------------CP rmvar _Var324
--------------CP createvar _mVar328 
scratch_space//_p16480_9.108.91.12//_t0/temp157 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------CP conv2d_bias_add _mVar321.MATRIX.DOUBLE b1.MATRIX.DOUBLE 
W1.MATRIX.DOUBLE 1.SCALAR.INT.true 1.SCALAR.INT.true 2.SCALAR.INT.true 
2.SCALAR.INT.true _Var325.SCALAR.INT.false C.SCALAR.INT.false 
Hin.SCALAR.INT.false Win.SCALAR.INT.false 32.SCALAR.INT.true C.SCALAR.INT.false 
5.SCALAR.INT.true 5.SCALAR.INT.true _mVar328.MATRIX.DOUBLE 8
--------------CP rmvar _Var325
--------------CP + _Var326.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var329.SCALAR.DOUBLE
--------------CP rmvar _Var326
--------------CP + _Var327.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var330.SCALAR.DOUBLE
--------------CP rmvar _Var327
--------------CP createvar _mVar331 
scratch_space//_p16480_9.108.91.12//_t0/temp158 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK sel+ _mVar328.MATRIX.DOUBLE _mVar331.MATRIX.DOUBLE
--------------CP nrow _mVar328.MATRIX.DOUBLE.false _Var332.SCALAR.INT
--------------CP floor _Var329.SCALAR.DOUBLE.false _Var333.SCALAR.DOUBLE
--------------CP rmvar _Var329
--------------CP floor _Var330.SCALAR.DOUBLE.false _Var334.SCALAR.DOUBLE
--------------CP rmvar _Var330
--------------CP createvar _mVar335 
scratch_space//_p16480_9.108.91.12//_t0/temp159 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------CP relu_maxpooling _mVar328.MATRIX.DOUBLE 2.SCALAR.INT.true 
2.SCALAR.INT.true 0.SCALAR.INT.true 0.SCALAR.INT.true _Var332.SCALAR.INT.false 
32.SCALAR.INT.true _Var316.SCALAR.INT.false _Var317.SCALAR.INT.false 
1.SCALAR.INT.true 32.SCALAR.INT.true 2.SCALAR.INT.true 2.SCALAR.INT.true 
_mVar335.MATRIX.DOUBLE 8
--------------CP rmvar _Var332
--------------CP castvti _Var333.SCALAR.DOUBLE.false _Var336.SCALAR.INT
--------------CP rmvar _Var333
--------------CP castvti _Var334.SCALAR.DOUBLE.false _Var337.SCALAR.INT
--------------CP rmvar _Var334
--------------CP nrow _mVar335.MATRIX.DOUBLE.false _Var338.SCALAR.INT
--------------CP + _Var336.SCALAR.INT.false 4.SCALAR.INT.true _Var339.SCALAR.INT
--------------CP + _Var337.SCALAR.INT.false 4.SCALAR.INT.true _Var340.SCALAR.INT
--------------CP createvar _mVar341 
scratch_space//_p16480_9.108.91.12//_t0/temp160 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------CP conv2d_bias_add _mVar335.MATRIX.DOUBLE b2.MATRIX.DOUBLE 
W2.MATRIX.DOUBLE 1.SCALAR.INT.true 1.SCALAR.INT.true 2.SCALAR.INT.true 
2.SCALAR.INT.true _Var338.SCALAR.INT.false 32.SCALAR.INT.true 
_Var336.SCALAR.INT.false _Var337.SCALAR.INT.false 64.SCALAR.INT.true 
32.SCALAR.INT.true 5.SCALAR.INT.true 5.SCALAR.INT.true _mVar341.MATRIX.DOUBLE 8
--------------CP rmvar _Var338
--------------CP - _Var339.SCALAR.INT.false 5.SCALAR.INT.true _Var342.SCALAR.INT
--------------CP rmvar _Var339
--------------CP - _Var340.SCALAR.INT.false 5.SCALAR.INT.true _Var343.SCALAR.INT
--------------CP rmvar _Var340
--------------CP / _Var342.SCALAR.INT.false 1.SCALAR.INT.true 
_Var344.SCALAR.DOUBLE
--------------CP rmvar _Var342
--------------CP / _Var343.SCALAR.INT.false 1.SCALAR.INT.true 
_Var345.SCALAR.DOUBLE
--------------CP rmvar _Var343
--------------CP createvar _mVar346 
scratch_space//_p16480_9.108.91.12//_t0/temp161 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK sel+ _mVar341.MATRIX.DOUBLE _mVar346.MATRIX.DOUBLE
--------------CP nrow _mVar341.MATRIX.DOUBLE.false _Var347.SCALAR.INT
--------------CP + _Var344.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var348.SCALAR.DOUBLE
--------------CP rmvar _Var344
--------------CP + _Var345.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var349.SCALAR.DOUBLE
--------------CP rmvar _Var345
--------------CP floor _Var348.SCALAR.DOUBLE.false _Var350.SCALAR.DOUBLE
--------------CP rmvar _Var348
--------------CP floor _Var349.SCALAR.DOUBLE.false _Var351.SCALAR.DOUBLE
--------------CP rmvar _Var349
--------------CP castvti _Var350.SCALAR.DOUBLE.false _Var352.SCALAR.INT
--------------CP rmvar _Var350
--------------CP castvti _Var351.SCALAR.DOUBLE.false _Var353.SCALAR.INT
--------------CP rmvar _Var351
--------------CP createvar _mVar354 
scratch_space//_p16480_9.108.91.12//_t0/temp162 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------CP relu_maxpooling _mVar341.MATRIX.DOUBLE 2.SCALAR.INT.true 
2.SCALAR.INT.true 0.SCALAR.INT.true 0.SCALAR.INT.true _Var347.SCALAR.INT.false 
64.SCALAR.INT.true _Var352.SCALAR.INT.false _Var353.SCALAR.INT.false 
1.SCALAR.INT.true 64.SCALAR.INT.true 2.SCALAR.INT.true 2.SCALAR.INT.true 
_mVar354.MATRIX.DOUBLE 8
--------------CP rmvar _Var347
--------------CP + _Var352.SCALAR.INT.false 0.SCALAR.INT.true _Var355.SCALAR.INT
--------------CP + _Var353.SCALAR.INT.false 0.SCALAR.INT.true _Var356.SCALAR.INT
--------------CP - _Var355.SCALAR.INT.false 2.SCALAR.INT.true _Var357.SCALAR.INT
--------------CP rmvar _Var355
--------------CP - _Var356.SCALAR.INT.false 2.SCALAR.INT.true _Var358.SCALAR.INT
--------------CP rmvar _Var356
--------------CP createvar _mVar359 
scratch_space//_p16480_9.108.91.12//_t0/temp163 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK cpmm _mVar354.MATRIX.DOUBLE W3.MATRIX.DOUBLE 
_mVar359.MATRIX.DOUBLE MULTI_BLOCK
--------------CP / _Var357.SCALAR.INT.false 2.SCALAR.INT.true 
_Var360.SCALAR.DOUBLE
--------------CP rmvar _Var357
--------------CP / _Var358.SCALAR.INT.false 2.SCALAR.INT.true 
_Var361.SCALAR.DOUBLE
--------------CP rmvar _Var358
--------------CP createvar _mVar362 
scratch_space//_p16480_9.108.91.12//_t0/temp164 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK + _mVar359.MATRIX.DOUBLE b3.MATRIX.DOUBLE 
_mVar362.MATRIX.DOUBLE
--------------CP rmvar _mVar359
--------------CP + _Var360.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var363.SCALAR.DOUBLE
--------------CP rmvar _Var360
--------------CP + _Var361.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var364.SCALAR.DOUBLE
--------------CP rmvar _Var361
--------------CP createvar _mVar365 
scratch_space//_p16480_9.108.91.12//_t0/temp165 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK sel+ _mVar362.MATRIX.DOUBLE _mVar365.MATRIX.DOUBLE
--------------CP floor _Var363.SCALAR.DOUBLE.false _Var366.SCALAR.DOUBLE
--------------CP rmvar _Var363
--------------CP floor _Var364.SCALAR.DOUBLE.false _Var367.SCALAR.DOUBLE
--------------CP rmvar _Var364
--------------CP castvti _Var366.SCALAR.DOUBLE.false _Var368.SCALAR.INT
--------------CP rmvar _Var366
--------------CP castvti _Var367.SCALAR.DOUBLE.false _Var369.SCALAR.INT
--------------CP rmvar _Var367
--------------CP assignvar _Var316.SCALAR.INT.false Houtc1.SCALAR.INT
--------------CP assignvar _Var317.SCALAR.INT.false Woutc1.SCALAR.INT
--------------CP assignvar _Var336.SCALAR.INT.false Houtp1.SCALAR.INT
--------------CP assignvar _Var337.SCALAR.INT.false Woutp1.SCALAR.INT
--------------CP assignvar _Var352.SCALAR.INT.false Houtc2.SCALAR.INT
--------------CP assignvar _Var353.SCALAR.INT.false Woutc2.SCALAR.INT
--------------CP assignvar _Var368.SCALAR.INT.false Houtp2.SCALAR.INT
--------------CP assignvar _Var369.SCALAR.INT.false Woutp2.SCALAR.INT
--------------CP rmvar _Var316 _Var317
--------------CP cpvar _mVar321 X_batch
--------------CP cpvar _mVar322 y_batch
--------------CP cpvar _mVar328 outc1
--------------CP cpvar _mVar331 outr1
--------------CP rmvar _Var336 _Var337
--------------CP cpvar _mVar335 outp1
--------------CP cpvar _mVar341 outc2
--------------CP cpvar _mVar346 outr2
--------------CP rmvar _Var352 _Var353
--------------CP cpvar _mVar354 outp2
--------------CP cpvar _mVar362 outa3
--------------CP cpvar _mVar365 outr3
--------------CP rmvar _Var368 _Var369 _mVar321 _mVar322 _mVar328 _mVar331 
_mVar335 _mVar341 _mVar346 _mVar354 _mVar362 _mVar365
------------GENERIC (lines 163-163) [recompile=false]
--------------CP extfunct ./nn/layers/dropout.dml forward 3 2 
outr3.MATRIX.DOUBLE 0.5.SCALAR.DOUBLE.true -1.SCALAR.INT.true outd3 maskd3
------------GENERIC (lines 165-181) [recompile=true]
--------------CP createvar _mVar370 
scratch_space//_p16480_9.108.91.12//_t0/temp166 true MATRIX binaryblock 512 -1 
1000 1000 -1 copy
--------------SPARK r' outr3.MATRIX.DOUBLE _mVar370.MATRIX.DOUBLE
--------------CP nrow y_batch.MATRIX.DOUBLE.false _Var371.SCALAR.INT
--------------CP createvar _mVar372 
scratch_space//_p16480_9.108.91.12//_t0/temp167 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK - 0.SCALAR.INT.true y_batch.MATRIX.DOUBLE 
_mVar372.MATRIX.DOUBLE
--------------CP createvar _mVar373 
scratch_space//_p16480_9.108.91.12//_t0/temp168 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK mapmm outd3.MATRIX.DOUBLE W4.MATRIX.DOUBLE 
_mVar373.MATRIX.DOUBLE RIGHT true NONE
--------------CP createvar _mVar374 
scratch_space//_p16480_9.108.91.12//_t0/temp169 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK r' outp2.MATRIX.DOUBLE _mVar374.MATRIX.DOUBLE
--------------CP createvar _mVar375 
scratch_space//_p16480_9.108.91.12//_t0/temp170 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK > outa3.MATRIX.DOUBLE 0.SCALAR.INT.true 
_mVar375.MATRIX.DOUBLE
--------------CP createvar _mVar376 
scratch_space//_p16480_9.108.91.12//_t0/temp171 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK / maskd3.MATRIX.DOUBLE 0.5.SCALAR.DOUBLE.true 
_mVar376.MATRIX.DOUBLE
--------------CP createvar _mVar377 
scratch_space//_p16480_9.108.91.12//_t0/temp172 true MATRIX binaryblock 10 512 
1000 1000 -1 copy
--------------SPARK r' W4.MATRIX.DOUBLE _mVar377.MATRIX.DOUBLE
--------------CP createvar _mVar378 
scratch_space//_p16480_9.108.91.12//_t0/temp173 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK > outc2.MATRIX.DOUBLE 0.SCALAR.INT.true 
_mVar378.MATRIX.DOUBLE
--------------CP createvar _mVar379 
scratch_space//_p16480_9.108.91.12//_t0/temp174 true MATRIX binaryblock 512 -1 
1000 1000 -1 copy
--------------SPARK r' W3.MATRIX.DOUBLE _mVar379.MATRIX.DOUBLE
--------------CP nrow outr2.MATRIX.DOUBLE.false _Var380.SCALAR.INT
--------------CP / 1.SCALAR.INT.true _Var371.SCALAR.INT.false 
_Var381.SCALAR.DOUBLE
--------------CP rmvar _Var371
--------------CP createvar _mVar382 
scratch_space//_p16480_9.108.91.12//_t0/temp175 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK + _mVar373.MATRIX.DOUBLE b4.MATRIX.DOUBLE 
_mVar382.MATRIX.DOUBLE
--------------CP rmvar _mVar373
--------------CP createvar _mVar383 
scratch_space//_p16480_9.108.91.12//_t0/temp176 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK * _mVar375.MATRIX.DOUBLE _mVar376.MATRIX.DOUBLE 
_mVar383.MATRIX.DOUBLE
--------------CP createvar _mVar384 
scratch_space//_p16480_9.108.91.12//_t0/temp177 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK * _Var381.SCALAR.DOUBLE.false _mVar372.MATRIX.DOUBLE 
_mVar384.MATRIX.DOUBLE
--------------CP rmvar _Var381 _mVar372
--------------CP createvar _mVar385 
scratch_space//_p16480_9.108.91.12//_t0/temp178 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK binuaggchain - uarmax _mVar382.MATRIX.DOUBLE 
_mVar385.MATRIX.DOUBLE
--------------CP rmvar _mVar382
--------------CP createvar _mVar386 
scratch_space//_p16480_9.108.91.12//_t0/temp179 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK exp _mVar385.MATRIX.DOUBLE _mVar386.MATRIX.DOUBLE
--------------CP rmvar _mVar385
--------------CP createvar _mVar387 
scratch_space//_p16480_9.108.91.12//_t0/temp180 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK binuaggchain / uark+ _mVar386.MATRIX.DOUBLE 
_mVar387.MATRIX.DOUBLE
--------------CP rmvar _mVar386
--------------CP createvar _mVar388 
scratch_space//_p16480_9.108.91.12//_t0/temp181 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK + _mVar387.MATRIX.DOUBLE 1.0E-10.SCALAR.DOUBLE.true 
_mVar388.MATRIX.DOUBLE
--------------CP createvar _mVar389 
scratch_space//_p16480_9.108.91.12//_t0/temp182 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK / 1.SCALAR.INT.true _mVar388.MATRIX.DOUBLE 
_mVar389.MATRIX.DOUBLE
--------------CP rmvar _mVar388
--------------CP createvar _mVar390 
scratch_space//_p16480_9.108.91.12//_t0/temp183 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK * _mVar384.MATRIX.DOUBLE _mVar389.MATRIX.DOUBLE 
_mVar390.MATRIX.DOUBLE
--------------CP rmvar _mVar384 _mVar389
--------------CP createvar _mVar391 
scratch_space//_p16480_9.108.91.12//_t0/temp184 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK * _mVar390.MATRIX.DOUBLE _mVar387.MATRIX.DOUBLE 
_mVar391.MATRIX.DOUBLE
--------------CP rmvar _mVar390
--------------CP createvar _mVar392 
scratch_space//_p16480_9.108.91.12//_t0/temp185 true MATRIX binaryblock -1 1 
1000 1000 -1 copy
--------------SPARK uark+ _mVar391.MATRIX.DOUBLE _mVar392.MATRIX.DOUBLE NONE
--------------CP createvar _mVar393 
scratch_space//_p16480_9.108.91.12//_t0/temp186 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK * _mVar387.MATRIX.DOUBLE _mVar392.MATRIX.DOUBLE 
_mVar393.MATRIX.DOUBLE
--------------CP rmvar _mVar387 _mVar392
--------------CP createvar _mVar394 
scratch_space//_p16480_9.108.91.12//_t0/temp187 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK - _mVar391.MATRIX.DOUBLE _mVar393.MATRIX.DOUBLE 
_mVar394.MATRIX.DOUBLE
--------------CP rmvar _mVar391 _mVar393
--------------CP createvar _mVar395 
scratch_space//_p16480_9.108.91.12//_t0/temp188 true MATRIX binaryblock 512 10 
1000 1000 -1 copy
--------------SPARK cpmm _mVar370.MATRIX.DOUBLE _mVar394.MATRIX.DOUBLE 
_mVar395.MATRIX.DOUBLE SINGLE_BLOCK
--------------CP rmvar _mVar370
--------------CP createvar _mVar396 
scratch_space//_p16480_9.108.91.12//_t0/temp189 true MATRIX binaryblock 1 10 
1000 1000 -1 copy
--------------SPARK uack+ _mVar394.MATRIX.DOUBLE _mVar396.MATRIX.DOUBLE 
SINGLE_BLOCK
--------------CP createvar _mVar397 
scratch_space//_p16480_9.108.91.12//_t0/temp190 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK mapmm _mVar394.MATRIX.DOUBLE _mVar377.MATRIX.DOUBLE 
_mVar397.MATRIX.DOUBLE RIGHT true NONE
--------------CP rmvar _mVar394 _mVar377
--------------CP createvar _mVar398 
scratch_space//_p16480_9.108.91.12//_t0/temp191 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK * _mVar383.MATRIX.DOUBLE _mVar397.MATRIX.DOUBLE 
_mVar398.MATRIX.DOUBLE
--------------CP rmvar _mVar383
--------------CP createvar _mVar399 
scratch_space//_p16480_9.108.91.12//_t0/temp192 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK * _mVar376.MATRIX.DOUBLE _mVar397.MATRIX.DOUBLE 
_mVar399.MATRIX.DOUBLE
--------------CP rmvar _mVar376 _mVar397
--------------CP createvar _mVar400 
scratch_space//_p16480_9.108.91.12//_t0/temp193 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK cpmm _mVar374.MATRIX.DOUBLE _mVar398.MATRIX.DOUBLE 
_mVar400.MATRIX.DOUBLE MULTI_BLOCK
--------------CP rmvar _mVar374
--------------CP createvar _mVar401 
scratch_space//_p16480_9.108.91.12//_t0/temp194 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK * _mVar375.MATRIX.DOUBLE _mVar399.MATRIX.DOUBLE 
_mVar401.MATRIX.DOUBLE
--------------CP rmvar _mVar375 _mVar399
--------------CP createvar _mVar402 
scratch_space//_p16480_9.108.91.12//_t0/temp195 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK cpmm _mVar398.MATRIX.DOUBLE _mVar379.MATRIX.DOUBLE 
_mVar402.MATRIX.DOUBLE MULTI_BLOCK
--------------CP rmvar _mVar398 _mVar379
--------------CP createvar _mVar403 
scratch_space//_p16480_9.108.91.12//_t0/temp196 true MATRIX binaryblock 1 512 
1000 1000 -1 copy
--------------SPARK uack+ _mVar401.MATRIX.DOUBLE _mVar403.MATRIX.DOUBLE 
SINGLE_BLOCK
--------------CP rmvar _mVar401
--------------CP createvar _mVar404 
scratch_space//_p16480_9.108.91.12//_t0/temp197 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------CP maxpooling_backward outr2.MATRIX.DOUBLE _mVar402.MATRIX.DOUBLE 
2.SCALAR.INT.true 2.SCALAR.INT.true 0.SCALAR.INT.true 0.SCALAR.INT.true 
_Var380.SCALAR.INT.false 64.SCALAR.INT.true Houtc2.SCALAR.INT.false 
Woutc2.SCALAR.INT.false 1.SCALAR.INT.true 64.SCALAR.INT.true 2.SCALAR.INT.true 
2.SCALAR.INT.true _mVar404.MATRIX.DOUBLE 8
--------------CP rmvar _mVar402 _Var380
--------------CP createvar _mVar405 
scratch_space//_p16480_9.108.91.12//_t0/temp198 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK * _mVar378.MATRIX.DOUBLE _mVar404.MATRIX.DOUBLE 
_mVar405.MATRIX.DOUBLE
--------------CP rmvar _mVar378 _mVar404
--------------CP cpvar _mVar395 dW4
--------------CP cpvar _mVar396 db4
--------------CP cpvar _mVar400 dW3
--------------CP cpvar _mVar403 db3
--------------CP cpvar _mVar405 doutc2
--------------CP rmvar _mVar395 _mVar396 _mVar400 _mVar403 _mVar405 outa3 outr3 
outr2 outc2 maskd3 outp2 outd3 Woutp2 y_batch Houtp2
------------GENERIC (lines 182-183) [recompile=true]
--------------CP nrow outp1.MATRIX.DOUBLE.false _Var406.SCALAR.INT
--------------CP createvar _mVar407 
scratch_space//_p16480_9.108.91.12//_t0/temp199 true MATRIX binaryblock 1 -1 
1000 1000 -1 copy
--------------SPARK uack+ doutc2.MATRIX.DOUBLE _mVar407.MATRIX.DOUBLE 
MULTI_BLOCK
--------------CP * Houtc2.SCALAR.INT.false Woutc2.SCALAR.INT.false 
_Var408.SCALAR.INT
--------------CP createvar _mVar409 
scratch_space//_p16480_9.108.91.12//_t0/temp200 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------CP conv2d_backward_data W2.MATRIX.DOUBLE doutc2.MATRIX.DOUBLE 
1.SCALAR.INT.true 1.SCALAR.INT.true 2.SCALAR.INT.true 2.SCALAR.INT.true 
_Var406.SCALAR.INT.false 32.SCALAR.INT.true Houtp1.SCALAR.INT.false 
Woutp1.SCALAR.INT.false 64.SCALAR.INT.true 32.SCALAR.INT.true 5.SCALAR.INT.true 
5.SCALAR.INT.true _mVar409.MATRIX.DOUBLE 8
--------------CP createvar _mVar410 
scratch_space//_p16480_9.108.91.12//_t0/temp201 true MATRIX binaryblock 64 800 
1000 1000 -1 copy
--------------CP conv2d_backward_filter outp1.MATRIX.DOUBLE 
doutc2.MATRIX.DOUBLE 1.SCALAR.INT.true 1.SCALAR.INT.true 2.SCALAR.INT.true 
2.SCALAR.INT.true _Var406.SCALAR.INT.false 32.SCALAR.INT.true 
Houtp1.SCALAR.INT.false Woutp1.SCALAR.INT.false 64.SCALAR.INT.true 
32.SCALAR.INT.true 5.SCALAR.INT.true 5.SCALAR.INT.true _mVar410.MATRIX.DOUBLE 8
--------------CP rmvar _Var406
--------------CP createvar _mVar411 
scratch_space//_p16480_9.108.91.12//_t0/temp202 true MATRIX binaryblock 64 -1 
1000 1000 -1 copy
--------------SPARK rshape _mVar407.MATRIX.DOUBLE 64.SCALAR.INT.true 
_Var408.SCALAR.INT.false true.SCALAR.BOOLEAN.true _mVar411.MATRIX.DOUBLE
--------------CP rmvar _mVar407 _Var408
--------------CP createvar _mVar412 
scratch_space//_p16480_9.108.91.12//_t0/temp203 true MATRIX binaryblock 64 1 
1000 1000 -1 copy
--------------SPARK uark+ _mVar411.MATRIX.DOUBLE _mVar412.MATRIX.DOUBLE 
SINGLE_BLOCK
--------------CP rmvar _mVar411
--------------CP cpvar _mVar409 doutp1
--------------CP cpvar _mVar410 dW2
--------------CP cpvar _mVar412 db2
--------------CP rmvar _mVar409 _mVar410 _mVar412 Houtc2 Woutc2 outp1 doutc2
------------GENERIC (lines 185-187) [recompile=true]
--------------CP createvar _mVar413 
scratch_space//_p16480_9.108.91.12//_t0/temp204 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK > outc1.MATRIX.DOUBLE 0.SCALAR.INT.true 
_mVar413.MATRIX.DOUBLE
--------------CP nrow outr1.MATRIX.DOUBLE.false _Var414.SCALAR.INT
--------------CP createvar _mVar415 
scratch_space//_p16480_9.108.91.12//_t0/temp205 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------CP maxpooling_backward outr1.MATRIX.DOUBLE doutp1.MATRIX.DOUBLE 
2.SCALAR.INT.true 2.SCALAR.INT.true 0.SCALAR.INT.true 0.SCALAR.INT.true 
_Var414.SCALAR.INT.false 32.SCALAR.INT.true Houtc1.SCALAR.INT.false 
Woutc1.SCALAR.INT.false 1.SCALAR.INT.true 32.SCALAR.INT.true 2.SCALAR.INT.true 
2.SCALAR.INT.true _mVar415.MATRIX.DOUBLE 8
--------------CP rmvar _Var414
--------------CP createvar _mVar416 
scratch_space//_p16480_9.108.91.12//_t0/temp206 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK * _mVar413.MATRIX.DOUBLE _mVar415.MATRIX.DOUBLE 
_mVar416.MATRIX.DOUBLE
--------------CP rmvar _mVar413 _mVar415
--------------CP cpvar _mVar416 doutc1
--------------CP rmvar _mVar416 outr1 outc1 Woutp1 doutp1 Houtp1
------------GENERIC (lines 188-189) [recompile=true]
--------------CP nrow X_batch.MATRIX.DOUBLE.false _Var417.SCALAR.INT
--------------CP createvar _mVar418 
scratch_space//_p16480_9.108.91.12//_t0/temp207 true MATRIX binaryblock 1 -1 
1000 1000 -1 copy
--------------SPARK uack+ doutc1.MATRIX.DOUBLE _mVar418.MATRIX.DOUBLE 
MULTI_BLOCK
--------------CP * Houtc1.SCALAR.INT.false Woutc1.SCALAR.INT.false 
_Var419.SCALAR.INT
--------------CP createvar _mVar420 
scratch_space//_p16480_9.108.91.12//_t0/temp208 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
--------------CP conv2d_backward_filter X_batch.MATRIX.DOUBLE 
doutc1.MATRIX.DOUBLE 1.SCALAR.INT.true 1.SCALAR.INT.true 2.SCALAR.INT.true 
2.SCALAR.INT.true _Var417.SCALAR.INT.false C.SCALAR.INT.false 
Hin.SCALAR.INT.false Win.SCALAR.INT.false 32.SCALAR.INT.true C.SCALAR.INT.false 
5.SCALAR.INT.true 5.SCALAR.INT.true _mVar420.MATRIX.DOUBLE 8
--------------CP rmvar _Var417
--------------CP createvar _mVar421 
scratch_space//_p16480_9.108.91.12//_t0/temp209 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
--------------SPARK rshape _mVar418.MATRIX.DOUBLE 32.SCALAR.INT.true 
_Var419.SCALAR.INT.false true.SCALAR.BOOLEAN.true _mVar421.MATRIX.DOUBLE
--------------CP rmvar _mVar418 _Var419
--------------CP createvar _mVar422 
scratch_space//_p16480_9.108.91.12//_t0/temp210 true MATRIX binaryblock 32 1 
1000 1000 -1 copy
--------------SPARK uark+ _mVar421.MATRIX.DOUBLE _mVar422.MATRIX.DOUBLE 
SINGLE_BLOCK
--------------CP rmvar _mVar421
--------------CP cpvar _mVar420 dW1
--------------CP cpvar _mVar422 db1
--------------CP rmvar _mVar420 _mVar422 Houtc1 Woutc1 doutc1
------------GENERIC (lines 192-212) [recompile=true]
--------------CP createvar _mVar423 
scratch_space//_p16480_9.108.91.12//_t0/temp211 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
--------------SPARK +* dW1.MATRIX.DOUBLE 5.0E-4.SCALAR.DOUBLE.true 
W1.MATRIX.DOUBLE _mVar423.MATRIX.DOUBLE
--------------CP ncol W1.MATRIX.DOUBLE.false _Var424.SCALAR.INT
--------------CP nrow X_batch.MATRIX.DOUBLE.false _Var425.SCALAR.INT
--------------CP nrow X_group_batch.MATRIX.DOUBLE.false _Var426.SCALAR.INT
--------------CP ncol dW1_agg.MATRIX.DOUBLE.false _Var427.SCALAR.INT
--------------CP createvar _mVar428 
scratch_space//_p16480_9.108.91.12//_t0/temp212 true MATRIX binaryblock 64 800 
1000 1000 -1 copy
--------------SPARK +* dW2.MATRIX.DOUBLE 5.0E-4.SCALAR.DOUBLE.true 
W2.MATRIX.DOUBLE _mVar428.MATRIX.DOUBLE
--------------CP * 64.SCALAR.INT.true 800.SCALAR.INT.true _Var429.SCALAR.INT
--------------CP createvar _mVar430 
scratch_space//_p16480_9.108.91.12//_t0/temp213 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK +* dW3.MATRIX.DOUBLE 5.0E-4.SCALAR.DOUBLE.true 
W3.MATRIX.DOUBLE _mVar430.MATRIX.DOUBLE
--------------CP nrow W3.MATRIX.DOUBLE.false _Var431.SCALAR.INT
--------------CP ncol dW3_agg.MATRIX.DOUBLE.false _Var432.SCALAR.INT
--------------CP createvar _mVar433 
scratch_space//_p16480_9.108.91.12//_t0/temp214 true MATRIX binaryblock 512 10 
1000 1000 -1 copy
--------------SPARK +* dW4.MATRIX.DOUBLE 5.0E-4.SCALAR.DOUBLE.true 
W4.MATRIX.DOUBLE _mVar433.MATRIX.DOUBLE
--------------CP * 512.SCALAR.INT.true 10.SCALAR.INT.true _Var434.SCALAR.INT
--------------CP * 32.SCALAR.INT.true 1.SCALAR.INT.true _Var435.SCALAR.INT
--------------CP * 64.SCALAR.INT.true 1.SCALAR.INT.true _Var436.SCALAR.INT
--------------CP * 32.SCALAR.INT.true _Var424.SCALAR.INT.false 
_Var437.SCALAR.INT
--------------CP rmvar _Var424
--------------CP / _Var425.SCALAR.INT.false _Var426.SCALAR.INT.false 
_Var438.SCALAR.DOUBLE
--------------CP rmvar _Var425 _Var426
--------------CP createvar _mVar439 
scratch_space//_p16480_9.108.91.12//_t0/temp215 true MATRIX binaryblock 1 51200 
1000 1000 -1 copy
--------------SPARK rshape _mVar428.MATRIX.DOUBLE 1.SCALAR.INT.true 
_Var429.SCALAR.INT.false true.SCALAR.BOOLEAN.true _mVar439.MATRIX.DOUBLE
--------------CP rmvar _mVar428 _Var429
--------------CP * _Var431.SCALAR.INT.false 512.SCALAR.INT.true 
_Var440.SCALAR.INT
--------------CP rmvar _Var431
--------------CP createvar _mVar441 
scratch_space//_p16480_9.108.91.12//_t0/temp216 true MATRIX binaryblock 1 5120 
1000 1000 -1 copy
--------------SPARK rshape _mVar433.MATRIX.DOUBLE 1.SCALAR.INT.true 
_Var434.SCALAR.INT.false true.SCALAR.BOOLEAN.true _mVar441.MATRIX.DOUBLE
--------------CP rmvar _mVar433 _Var434
--------------CP createvar _mVar442 
scratch_space//_p16480_9.108.91.12//_t0/temp217 true MATRIX binaryblock 1 32 
1000 1000 -1 copy
--------------SPARK rshape db1.MATRIX.DOUBLE 1.SCALAR.INT.true 
_Var435.SCALAR.INT.false true.SCALAR.BOOLEAN.true _mVar442.MATRIX.DOUBLE
--------------CP rmvar _Var435
--------------CP createvar _mVar443 
scratch_space//_p16480_9.108.91.12//_t0/temp218 true MATRIX binaryblock 1 64 
1000 1000 -1 copy
--------------SPARK rshape db2.MATRIX.DOUBLE 1.SCALAR.INT.true 
_Var436.SCALAR.INT.false true.SCALAR.BOOLEAN.true _mVar443.MATRIX.DOUBLE
--------------CP rmvar _Var436
--------------CP createvar _mVar444 
scratch_space//_p16480_9.108.91.12//_t0/temp219 true MATRIX binaryblock 1 -1 
1000 1000 -1 copy
--------------SPARK rshape _mVar423.MATRIX.DOUBLE 1.SCALAR.INT.true 
_Var437.SCALAR.INT.false true.SCALAR.BOOLEAN.true _mVar444.MATRIX.DOUBLE
--------------CP rmvar _mVar423 _Var437
--------------CP createvar _mVar445 
scratch_space//_p16480_9.108.91.12//_t0/temp220 true MATRIX binaryblock 1 51200 
1000 1000 -1 copy
--------------SPARK * _mVar439.MATRIX.DOUBLE _Var438.SCALAR.DOUBLE.false 
_mVar445.MATRIX.DOUBLE
--------------CP rmvar _mVar439
--------------CP createvar _mVar446 
scratch_space//_p16480_9.108.91.12//_t0/temp221 true MATRIX binaryblock 1 -1 
1000 1000 -1 copy
--------------SPARK rshape _mVar430.MATRIX.DOUBLE 1.SCALAR.INT.true 
_Var440.SCALAR.INT.false true.SCALAR.BOOLEAN.true _mVar446.MATRIX.DOUBLE
--------------CP rmvar _mVar430 _Var440
--------------CP createvar _mVar447 
scratch_space//_p16480_9.108.91.12//_t0/temp222 true MATRIX binaryblock 1 5120 
1000 1000 -1 copy
--------------SPARK * _mVar441.MATRIX.DOUBLE _Var438.SCALAR.DOUBLE.false 
_mVar447.MATRIX.DOUBLE
--------------CP rmvar _mVar441
--------------CP createvar _mVar448 
scratch_space//_p16480_9.108.91.12//_t0/temp223 true MATRIX binaryblock 1 32 
1000 1000 -1 copy
--------------SPARK * _mVar442.MATRIX.DOUBLE _Var438.SCALAR.DOUBLE.false 
_mVar448.MATRIX.DOUBLE
--------------CP rmvar _mVar442
--------------CP createvar _mVar449 
scratch_space//_p16480_9.108.91.12//_t0/temp224 true MATRIX binaryblock 1 64 
1000 1000 -1 copy
--------------SPARK * _mVar443.MATRIX.DOUBLE _Var438.SCALAR.DOUBLE.false 
_mVar449.MATRIX.DOUBLE
--------------CP rmvar _mVar443
--------------CP createvar _mVar450 
scratch_space//_p16480_9.108.91.12//_t0/temp225 true MATRIX binaryblock 1 512 
1000 1000 -1 copy
--------------SPARK * db3.MATRIX.DOUBLE _Var438.SCALAR.DOUBLE.false 
_mVar450.MATRIX.DOUBLE
--------------CP createvar _mVar451 
scratch_space//_p16480_9.108.91.12//_t0/temp226 true MATRIX binaryblock 1 10 
1000 1000 -1 copy
--------------SPARK * db4.MATRIX.DOUBLE _Var438.SCALAR.DOUBLE.false 
_mVar451.MATRIX.DOUBLE
--------------CP createvar _mVar452 
scratch_space//_p16480_9.108.91.12//_t0/temp227 true MATRIX binaryblock 1 -1 
1000 1000 -1 copy
--------------SPARK * _mVar444.MATRIX.DOUBLE _Var438.SCALAR.DOUBLE.false 
_mVar452.MATRIX.DOUBLE
--------------CP rmvar _mVar444
--------------CP createvar _mVar453 
scratch_space//_p16480_9.108.91.12//_t0/temp228 true MATRIX binaryblock -1 
51200 1000 1000 -1 copy
--------------SPARK mapLeftIndex dW2_agg.MATRIX.DOUBLE _mVar445.MATRIX.DOUBLE 
j.SCALAR.INT.false j.SCALAR.INT.false 1.SCALAR.INT.true 51200.SCALAR.INT.true 
_mVar453.MATRIX.DOUBLE RIGHT
--------------CP rmvar _mVar445
--------------CP createvar _mVar454 
scratch_space//_p16480_9.108.91.12//_t0/temp229 true MATRIX binaryblock 1 -1 
1000 1000 -1 copy
--------------SPARK * _mVar446.MATRIX.DOUBLE _Var438.SCALAR.DOUBLE.false 
_mVar454.MATRIX.DOUBLE
--------------CP rmvar _mVar446 _Var438
--------------CP createvar _mVar455 
scratch_space//_p16480_9.108.91.12//_t0/temp230 true MATRIX binaryblock -1 5120 
1000 1000 -1 copy
--------------SPARK mapLeftIndex dW4_agg.MATRIX.DOUBLE _mVar447.MATRIX.DOUBLE 
j.SCALAR.INT.false j.SCALAR.INT.false 1.SCALAR.INT.true 5120.SCALAR.INT.true 
_mVar455.MATRIX.DOUBLE RIGHT
--------------CP rmvar _mVar447
--------------CP createvar _mVar456 
scratch_space//_p16480_9.108.91.12//_t0/temp231 true MATRIX binaryblock -1 32 
1000 1000 -1 copy
--------------SPARK mapLeftIndex db1_agg.MATRIX.DOUBLE _mVar448.MATRIX.DOUBLE 
j.SCALAR.INT.false j.SCALAR.INT.false 1.SCALAR.INT.true 32.SCALAR.INT.true 
_mVar456.MATRIX.DOUBLE RIGHT
--------------CP rmvar _mVar448
--------------CP createvar _mVar457 
scratch_space//_p16480_9.108.91.12//_t0/temp232 true MATRIX binaryblock -1 64 
1000 1000 -1 copy
--------------SPARK mapLeftIndex db2_agg.MATRIX.DOUBLE _mVar449.MATRIX.DOUBLE 
j.SCALAR.INT.false j.SCALAR.INT.false 1.SCALAR.INT.true 64.SCALAR.INT.true 
_mVar457.MATRIX.DOUBLE RIGHT
--------------CP rmvar _mVar449
--------------CP createvar _mVar458 
scratch_space//_p16480_9.108.91.12//_t0/temp233 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
--------------SPARK mapLeftIndex db3_agg.MATRIX.DOUBLE _mVar450.MATRIX.DOUBLE 
j.SCALAR.INT.false j.SCALAR.INT.false 1.SCALAR.INT.true 512.SCALAR.INT.true 
_mVar458.MATRIX.DOUBLE RIGHT
--------------CP rmvar _mVar450
--------------CP createvar _mVar459 
scratch_space//_p16480_9.108.91.12//_t0/temp234 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK mapLeftIndex db4_agg.MATRIX.DOUBLE _mVar451.MATRIX.DOUBLE 
j.SCALAR.INT.false j.SCALAR.INT.false 1.SCALAR.INT.true 10.SCALAR.INT.true 
_mVar459.MATRIX.DOUBLE RIGHT
--------------CP rmvar _mVar451
--------------CP createvar _mVar460 
scratch_space//_p16480_9.108.91.12//_t0/temp235 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK leftIndex dW1_agg.MATRIX.DOUBLE _mVar452.MATRIX.DOUBLE 
j.SCALAR.INT.false j.SCALAR.INT.false 1.SCALAR.INT.true 
_Var427.SCALAR.INT.false _mVar460.MATRIX.DOUBLE NONE
--------------CP rmvar _mVar452 _Var427
--------------CP createvar _mVar461 
scratch_space//_p16480_9.108.91.12//_t0/temp236 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK leftIndex dW3_agg.MATRIX.DOUBLE _mVar454.MATRIX.DOUBLE 
j.SCALAR.INT.false j.SCALAR.INT.false 1.SCALAR.INT.true 
_Var432.SCALAR.INT.false _mVar461.MATRIX.DOUBLE NONE
--------------CP rmvar _mVar454 _Var432 db2_agg db3_agg dW3_agg dW2_agg dW1_agg 
db4_agg db1_agg dW4_agg
--------------CP cpvar _mVar453 dW2_agg
--------------CP cpvar _mVar455 dW4_agg
--------------CP cpvar _mVar456 db1_agg
--------------CP cpvar _mVar457 db2_agg
--------------CP cpvar _mVar458 db3_agg
--------------CP cpvar _mVar459 db4_agg
--------------CP cpvar _mVar460 dW1_agg
--------------CP cpvar _mVar461 dW3_agg
--------------CP rmvar _mVar453 _mVar455 _mVar456 _mVar457 _mVar458 _mVar459 
_mVar460 _mVar461 dW2 dW1 dW4 dW3 X_batch db1 db3 db2 db4
----------GENERIC (lines 219-236) [recompile=true]
------------CP createvar _mVar462 
scratch_space//_p16480_9.108.91.12//_t0/temp237 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
------------SPARK * 0.9.SCALAR.DOUBLE.true vW1.MATRIX.DOUBLE 
_mVar462.MATRIX.DOUBLE
------------CP createvar _mVar463 
scratch_space//_p16480_9.108.91.12//_t0/temp238 true MATRIX binaryblock 1 -1 
1000 1000 -1 copy
------------SPARK uack+ dW1_agg.MATRIX.DOUBLE _mVar463.MATRIX.DOUBLE MULTI_BLOCK
------------CP ncol W1.MATRIX.DOUBLE.false _Var464.SCALAR.INT
------------CP createvar _mVar465 
scratch_space//_p16480_9.108.91.12//_t0/temp239 true MATRIX binaryblock 64 800 
1000 1000 -1 copy
------------SPARK * 0.9.SCALAR.DOUBLE.true vW2.MATRIX.DOUBLE 
_mVar465.MATRIX.DOUBLE
------------CP createvar _mVar466 
scratch_space//_p16480_9.108.91.12//_t0/temp240 true MATRIX binaryblock 1 51200 
1000 1000 -1 copy
------------SPARK uack+ dW2_agg.MATRIX.DOUBLE _mVar466.MATRIX.DOUBLE MULTI_BLOCK
------------CP createvar _mVar467 
scratch_space//_p16480_9.108.91.12//_t0/temp241 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
------------SPARK * 0.9.SCALAR.DOUBLE.true vW3.MATRIX.DOUBLE 
_mVar467.MATRIX.DOUBLE
------------CP createvar _mVar468 
scratch_space//_p16480_9.108.91.12//_t0/temp242 true MATRIX binaryblock 1 -1 
1000 1000 -1 copy
------------SPARK uack+ dW3_agg.MATRIX.DOUBLE _mVar468.MATRIX.DOUBLE MULTI_BLOCK
------------CP nrow W3.MATRIX.DOUBLE.false _Var469.SCALAR.INT
------------CP createvar _mVar470 
scratch_space//_p16480_9.108.91.12//_t0/temp243 true MATRIX binaryblock 512 10 
1000 1000 -1 copy
------------SPARK * 0.9.SCALAR.DOUBLE.true vW4.MATRIX.DOUBLE 
_mVar470.MATRIX.DOUBLE
------------CP createvar _mVar471 
scratch_space//_p16480_9.108.91.12//_t0/temp244 true MATRIX binaryblock 1 5120 
1000 1000 -1 copy
------------SPARK uack+ dW4_agg.MATRIX.DOUBLE _mVar471.MATRIX.DOUBLE MULTI_BLOCK
------------CP createvar _mVar472 
scratch_space//_p16480_9.108.91.12//_t0/temp245 true MATRIX binaryblock 32 1 
1000 1000 -1 copy
------------SPARK * 0.9.SCALAR.DOUBLE.true vb1.MATRIX.DOUBLE 
_mVar472.MATRIX.DOUBLE
------------CP createvar _mVar473 
scratch_space//_p16480_9.108.91.12//_t0/temp246 true MATRIX binaryblock 1 32 
1000 1000 -1 copy
------------SPARK uack+ db1_agg.MATRIX.DOUBLE _mVar473.MATRIX.DOUBLE 
SINGLE_BLOCK
------------CP createvar _mVar474 
scratch_space//_p16480_9.108.91.12//_t0/temp247 true MATRIX binaryblock 64 1 
1000 1000 -1 copy
------------SPARK * 0.9.SCALAR.DOUBLE.true vb2.MATRIX.DOUBLE 
_mVar474.MATRIX.DOUBLE
------------CP createvar _mVar475 
scratch_space//_p16480_9.108.91.12//_t0/temp248 true MATRIX binaryblock 1 64 
1000 1000 -1 copy
------------SPARK uack+ db2_agg.MATRIX.DOUBLE _mVar475.MATRIX.DOUBLE 
SINGLE_BLOCK
------------CP createvar _mVar476 
scratch_space//_p16480_9.108.91.12//_t0/temp249 true MATRIX binaryblock 1 512 
1000 1000 -1 copy
------------SPARK * 0.9.SCALAR.DOUBLE.true vb3.MATRIX.DOUBLE 
_mVar476.MATRIX.DOUBLE
------------CP createvar _mVar477 
scratch_space//_p16480_9.108.91.12//_t0/temp250 true MATRIX binaryblock 1 512 
1000 1000 -1 copy
------------SPARK uack+ db3_agg.MATRIX.DOUBLE _mVar477.MATRIX.DOUBLE 
SINGLE_BLOCK
------------CP createvar _mVar478 
scratch_space//_p16480_9.108.91.12//_t0/temp251 true MATRIX binaryblock 1 10 
1000 1000 -1 copy
------------SPARK * 0.9.SCALAR.DOUBLE.true vb4.MATRIX.DOUBLE 
_mVar478.MATRIX.DOUBLE
------------CP createvar _mVar479 
scratch_space//_p16480_9.108.91.12//_t0/temp252 true MATRIX binaryblock 1 10 
1000 1000 -1 copy
------------SPARK uack+ db4_agg.MATRIX.DOUBLE _mVar479.MATRIX.DOUBLE 
SINGLE_BLOCK
------------CP createvar _mVar480 
scratch_space//_p16480_9.108.91.12//_t0/temp253 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
------------SPARK - W1.MATRIX.DOUBLE _mVar462.MATRIX.DOUBLE 
_mVar480.MATRIX.DOUBLE
------------CP createvar _mVar481 
scratch_space//_p16480_9.108.91.12//_t0/temp254 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
------------SPARK rshape _mVar463.MATRIX.DOUBLE 32.SCALAR.INT.true 
_Var464.SCALAR.INT.false true.SCALAR.BOOLEAN.true _mVar481.MATRIX.DOUBLE
------------CP rmvar _mVar463 _Var464
------------CP createvar _mVar482 
scratch_space//_p16480_9.108.91.12//_t0/temp255 true MATRIX binaryblock 64 800 
1000 1000 -1 copy
------------SPARK - W2.MATRIX.DOUBLE _mVar465.MATRIX.DOUBLE 
_mVar482.MATRIX.DOUBLE
------------CP createvar _mVar483 
scratch_space//_p16480_9.108.91.12//_t0/temp256 true MATRIX binaryblock 64 800 
1000 1000 -1 copy
------------SPARK rshape _mVar466.MATRIX.DOUBLE 64.SCALAR.INT.true 
800.SCALAR.INT.true true.SCALAR.BOOLEAN.true _mVar483.MATRIX.DOUBLE
------------CP rmvar _mVar466
------------CP createvar _mVar484 
scratch_space//_p16480_9.108.91.12//_t0/temp257 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
------------SPARK - W3.MATRIX.DOUBLE _mVar467.MATRIX.DOUBLE 
_mVar484.MATRIX.DOUBLE
------------CP createvar _mVar485 
scratch_space//_p16480_9.108.91.12//_t0/temp258 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
------------SPARK rshape _mVar468.MATRIX.DOUBLE _Var469.SCALAR.INT.false 
512.SCALAR.INT.true true.SCALAR.BOOLEAN.true _mVar485.MATRIX.DOUBLE
------------CP rmvar _mVar468 _Var469
------------CP createvar _mVar486 
scratch_space//_p16480_9.108.91.12//_t0/temp259 true MATRIX binaryblock 512 10 
1000 1000 -1 copy
------------SPARK - W4.MATRIX.DOUBLE _mVar470.MATRIX.DOUBLE 
_mVar486.MATRIX.DOUBLE
------------CP createvar _mVar487 
scratch_space//_p16480_9.108.91.12//_t0/temp260 true MATRIX binaryblock 512 10 
1000 1000 -1 copy
------------SPARK rshape _mVar471.MATRIX.DOUBLE 512.SCALAR.INT.true 
10.SCALAR.INT.true true.SCALAR.BOOLEAN.true _mVar487.MATRIX.DOUBLE
------------CP rmvar _mVar471
------------CP createvar _mVar488 
scratch_space//_p16480_9.108.91.12//_t0/temp261 true MATRIX binaryblock 32 1 
1000 1000 -1 copy
------------SPARK map- b1.MATRIX.DOUBLE _mVar472.MATRIX.DOUBLE 
_mVar488.MATRIX.DOUBLE RIGHT COL_VECTOR
------------CP createvar _mVar489 
scratch_space//_p16480_9.108.91.12//_t0/temp262 true MATRIX binaryblock 32 1 
1000 1000 -1 copy
------------SPARK rshape _mVar473.MATRIX.DOUBLE 32.SCALAR.INT.true 
1.SCALAR.INT.true true.SCALAR.BOOLEAN.true _mVar489.MATRIX.DOUBLE
------------CP rmvar _mVar473
------------CP createvar _mVar490 
scratch_space//_p16480_9.108.91.12//_t0/temp263 true MATRIX binaryblock 64 1 
1000 1000 -1 copy
------------SPARK map- b2.MATRIX.DOUBLE _mVar474.MATRIX.DOUBLE 
_mVar490.MATRIX.DOUBLE RIGHT COL_VECTOR
------------CP createvar _mVar491 
scratch_space//_p16480_9.108.91.12//_t0/temp264 true MATRIX binaryblock 64 1 
1000 1000 -1 copy
------------SPARK rshape _mVar475.MATRIX.DOUBLE 64.SCALAR.INT.true 
1.SCALAR.INT.true true.SCALAR.BOOLEAN.true _mVar491.MATRIX.DOUBLE
------------CP rmvar _mVar475
------------CP createvar _mVar492 
scratch_space//_p16480_9.108.91.12//_t0/temp265 true MATRIX binaryblock 1 512 
1000 1000 -1 copy
------------SPARK map- b3.MATRIX.DOUBLE _mVar476.MATRIX.DOUBLE 
_mVar492.MATRIX.DOUBLE RIGHT ROW_VECTOR
------------CP createvar _mVar493 
scratch_space//_p16480_9.108.91.12//_t0/temp266 true MATRIX binaryblock 1 512 
1000 1000 -1 copy
------------SPARK -* _mVar476.MATRIX.DOUBLE lr.SCALAR.DOUBLE.false 
_mVar477.MATRIX.DOUBLE _mVar493.MATRIX.DOUBLE
------------CP rmvar _mVar476 _mVar477
------------CP createvar _mVar494 
scratch_space//_p16480_9.108.91.12//_t0/temp267 true MATRIX binaryblock 1 10 
1000 1000 -1 copy
------------SPARK map- b4.MATRIX.DOUBLE _mVar478.MATRIX.DOUBLE 
_mVar494.MATRIX.DOUBLE RIGHT ROW_VECTOR
------------CP createvar _mVar495 
scratch_space//_p16480_9.108.91.12//_t0/temp268 true MATRIX binaryblock 1 10 
1000 1000 -1 copy
------------SPARK -* _mVar478.MATRIX.DOUBLE lr.SCALAR.DOUBLE.false 
_mVar479.MATRIX.DOUBLE _mVar495.MATRIX.DOUBLE
------------CP rmvar _mVar478 _mVar479
------------CP createvar _mVar496 
scratch_space//_p16480_9.108.91.12//_t0/temp269 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
------------SPARK -* _mVar462.MATRIX.DOUBLE lr.SCALAR.DOUBLE.false 
_mVar481.MATRIX.DOUBLE _mVar496.MATRIX.DOUBLE
------------CP rmvar _mVar462 _mVar481
------------CP createvar _mVar497 
scratch_space//_p16480_9.108.91.12//_t0/temp270 true MATRIX binaryblock 64 800 
1000 1000 -1 copy
------------SPARK -* _mVar465.MATRIX.DOUBLE lr.SCALAR.DOUBLE.false 
_mVar483.MATRIX.DOUBLE _mVar497.MATRIX.DOUBLE
------------CP rmvar _mVar465 _mVar483
------------CP createvar _mVar498 
scratch_space//_p16480_9.108.91.12//_t0/temp271 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
------------SPARK -* _mVar467.MATRIX.DOUBLE lr.SCALAR.DOUBLE.false 
_mVar485.MATRIX.DOUBLE _mVar498.MATRIX.DOUBLE
------------CP rmvar _mVar467 _mVar485
------------CP createvar _mVar499 
scratch_space//_p16480_9.108.91.12//_t0/temp272 true MATRIX binaryblock 512 10 
1000 1000 -1 copy
------------SPARK -* _mVar470.MATRIX.DOUBLE lr.SCALAR.DOUBLE.false 
_mVar487.MATRIX.DOUBLE _mVar499.MATRIX.DOUBLE
------------CP rmvar _mVar470 _mVar487
------------CP createvar _mVar500 
scratch_space//_p16480_9.108.91.12//_t0/temp273 true MATRIX binaryblock 32 1 
1000 1000 -1 copy
------------SPARK -* _mVar472.MATRIX.DOUBLE lr.SCALAR.DOUBLE.false 
_mVar489.MATRIX.DOUBLE _mVar500.MATRIX.DOUBLE
------------CP rmvar _mVar472 _mVar489
------------CP createvar _mVar501 
scratch_space//_p16480_9.108.91.12//_t0/temp274 true MATRIX binaryblock 64 1 
1000 1000 -1 copy
------------SPARK -* _mVar474.MATRIX.DOUBLE lr.SCALAR.DOUBLE.false 
_mVar491.MATRIX.DOUBLE _mVar501.MATRIX.DOUBLE
------------CP rmvar _mVar474 _mVar491
------------CP createvar _mVar502 
scratch_space//_p16480_9.108.91.12//_t0/temp275 true MATRIX binaryblock 1 512 
1000 1000 -1 copy
------------SPARK +* _mVar492.MATRIX.DOUBLE 1.9.SCALAR.DOUBLE.true 
_mVar493.MATRIX.DOUBLE _mVar502.MATRIX.DOUBLE
------------CP rmvar _mVar492
------------CP createvar _mVar503 
scratch_space//_p16480_9.108.91.12//_t0/temp276 true MATRIX binaryblock 1 10 
1000 1000 -1 copy
------------SPARK +* _mVar494.MATRIX.DOUBLE 1.9.SCALAR.DOUBLE.true 
_mVar495.MATRIX.DOUBLE _mVar503.MATRIX.DOUBLE
------------CP rmvar _mVar494
------------CP createvar _mVar504 
scratch_space//_p16480_9.108.91.12//_t0/temp277 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
------------SPARK +* _mVar480.MATRIX.DOUBLE 1.9.SCALAR.DOUBLE.true 
_mVar496.MATRIX.DOUBLE _mVar504.MATRIX.DOUBLE
------------CP rmvar _mVar480
------------CP createvar _mVar505 
scratch_space//_p16480_9.108.91.12//_t0/temp278 true MATRIX binaryblock 64 800 
1000 1000 -1 copy
------------SPARK +* _mVar482.MATRIX.DOUBLE 1.9.SCALAR.DOUBLE.true 
_mVar497.MATRIX.DOUBLE _mVar505.MATRIX.DOUBLE
------------CP rmvar _mVar482
------------CP createvar _mVar506 
scratch_space//_p16480_9.108.91.12//_t0/temp279 true MATRIX binaryblock -1 512 
1000 1000 -1 copy
------------SPARK +* _mVar484.MATRIX.DOUBLE 1.9.SCALAR.DOUBLE.true 
_mVar498.MATRIX.DOUBLE _mVar506.MATRIX.DOUBLE
------------CP rmvar _mVar484
------------CP createvar _mVar507 
scratch_space//_p16480_9.108.91.12//_t0/temp280 true MATRIX binaryblock 512 10 
1000 1000 -1 copy
------------SPARK +* _mVar486.MATRIX.DOUBLE 1.9.SCALAR.DOUBLE.true 
_mVar499.MATRIX.DOUBLE _mVar507.MATRIX.DOUBLE
------------CP rmvar _mVar486
------------CP createvar _mVar508 
scratch_space//_p16480_9.108.91.12//_t0/temp281 true MATRIX binaryblock 32 1 
1000 1000 -1 copy
------------SPARK +* _mVar488.MATRIX.DOUBLE 1.9.SCALAR.DOUBLE.true 
_mVar500.MATRIX.DOUBLE _mVar508.MATRIX.DOUBLE
------------CP rmvar _mVar488
------------CP createvar _mVar509 
scratch_space//_p16480_9.108.91.12//_t0/temp282 true MATRIX binaryblock 64 1 
1000 1000 -1 copy
------------SPARK +* _mVar490.MATRIX.DOUBLE 1.9.SCALAR.DOUBLE.true 
_mVar501.MATRIX.DOUBLE _mVar509.MATRIX.DOUBLE
------------CP rmvar _mVar490 vb1 vb3 vb2 vb4 b1 vW2 b2 vW1 b3 vW4 b4 vW3 W1 W2 
W3 W4
------------CP cpvar _mVar493 vb3
------------CP cpvar _mVar495 vb4
------------CP cpvar _mVar496 vW1
------------CP cpvar _mVar497 vW2
------------CP cpvar _mVar498 vW3
------------CP cpvar _mVar499 vW4
------------CP cpvar _mVar500 vb1
------------CP cpvar _mVar501 vb2
------------CP cpvar _mVar502 b3
------------CP cpvar _mVar503 b4
------------CP cpvar _mVar504 W1
------------CP cpvar _mVar505 W2
------------CP cpvar _mVar506 W3
------------CP cpvar _mVar507 W4
------------CP cpvar _mVar508 b1
------------CP cpvar _mVar509 b2
------------CP rmvar _mVar493 _mVar495 _mVar496 _mVar497 _mVar498 _mVar499 
_mVar500 _mVar501 _mVar502 _mVar503 _mVar504 _mVar505 _mVar506 _mVar507 
_mVar508 _mVar509 dW3_agg db4_agg dW4_agg db3_agg db2_agg dW1_agg dW2_agg 
db1_agg
----------IF (lines 239-264)
------------CP %% g.SCALAR.INT.false 10.SCALAR.INT.true _Var510.SCALAR.INT
------------CP == _Var510.SCALAR.INT.false 0.SCALAR.INT.true 
_Var511.SCALAR.BOOLEAN
------------CP rmvar _Var510 _Var511
------------GENERIC (lines 241-244) [recompile=true]
--------------CP - j.SCALAR.INT.false 1.SCALAR.INT.true _Var512.SCALAR.INT
--------------CP nrow X_group_batch.MATRIX.DOUBLE.false _Var513.SCALAR.INT
--------------CP * _Var512.SCALAR.INT.false batch_size.SCALAR.INT.false 
_Var514.SCALAR.INT
--------------CP rmvar _Var512
--------------CP %% _Var514.SCALAR.INT.false _Var513.SCALAR.INT.false 
_Var515.SCALAR.INT
--------------CP rmvar _Var514
--------------CP + _Var515.SCALAR.INT.false 1.SCALAR.INT.true _Var516.SCALAR.INT
--------------CP rmvar _Var515
--------------CP + _Var516.SCALAR.INT.false batch_size.SCALAR.INT.false 
_Var517.SCALAR.INT
--------------CP - _Var517.SCALAR.INT.false 1.SCALAR.INT.true _Var518.SCALAR.INT
--------------CP rmvar _Var517
--------------CP min _Var513.SCALAR.INT.false _Var518.SCALAR.INT.false 
_Var519.SCALAR.INT
--------------CP rmvar _Var513 _Var518
--------------CP createvar _mVar520 
scratch_space//_p16480_9.108.91.12//_t0/temp283 true MATRIX binaryblock -1 
37632 1000 1000 -1 copy
--------------SPARK rangeReIndex X_group_batch.MATRIX.DOUBLE 
_Var516.SCALAR.INT.false _Var519.SCALAR.INT.false 1.SCALAR.INT.true 
37632.SCALAR.INT.true _mVar520.MATRIX.DOUBLE MULTI_BLOCK
--------------CP createvar _mVar521 
scratch_space//_p16480_9.108.91.12//_t0/temp284 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK rangeReIndex y_group_batch.MATRIX.DOUBLE 
_Var516.SCALAR.INT.false _Var519.SCALAR.INT.false 1.SCALAR.INT.true 
10.SCALAR.INT.true _mVar521.MATRIX.DOUBLE MULTI_BLOCK
--------------CP rmvar _Var516 _Var519
--------------CP cpvar _mVar520 X_batch
--------------CP cpvar _mVar521 y_batch
--------------CP rmvar _mVar520 _mVar521 y_group_batch X_group_batch
------------GENERIC (lines 247-247) [recompile=false]
--------------CP extfunct ./scripts/nn/examples/mnist_lenet_distrib_sgd.dml 
predict 12 1 X_batch.MATRIX.DOUBLE C.SCALAR.INT.false Hin.SCALAR.INT.false 
Win.SCALAR.INT.false W1.MATRIX.DOUBLE b1.MATRIX.DOUBLE W2.MATRIX.DOUBLE 
b2.MATRIX.DOUBLE W3.MATRIX.DOUBLE b3.MATRIX.DOUBLE W4.MATRIX.DOUBLE 
b4.MATRIX.DOUBLE probs
--------------CP rmvar X_batch
------------GENERIC (lines 248-254) [recompile=true]
--------------CP createvar _mVar522 
scratch_space//_p16480_9.108.91.12//_t0/temp285 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK - 0.SCALAR.INT.true y_batch.MATRIX.DOUBLE 
_mVar522.MATRIX.DOUBLE
--------------CP createvar _mVar523 
scratch_space//_p16480_9.108.91.12//_t0/temp286 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK + probs.MATRIX.DOUBLE 1.0E-10.SCALAR.DOUBLE.true 
_mVar523.MATRIX.DOUBLE
--------------CP nrow y_batch.MATRIX.DOUBLE.false _Var524.SCALAR.INT
--------------CP createvar _mVar525 
scratch_space//_p16480_9.108.91.12//_t0/temp287 true MATRIX binaryblock 32 -1 
1000 1000 -1 copy
--------------SPARK ^2 W1.MATRIX.DOUBLE 2.SCALAR.INT.true _mVar525.MATRIX.DOUBLE
--------------CP createvar _mVar526 
scratch_space//_p16480_9.108.91.12//_t0/temp288 true MATRIX binaryblock 1 1 
1000 1000 -1 copy
--------------SPARK uasqk+ W2.MATRIX.DOUBLE _mVar526.MATRIX.DOUBLE SINGLE_BLOCK
--------------CP createvar _mVar527 
scratch_space//_p16480_9.108.91.12//_t0/temp289 true MATRIX binaryblock 1 1 
1000 1000 -1 copy
--------------SPARK uasqk+ W3.MATRIX.DOUBLE _mVar527.MATRIX.DOUBLE SINGLE_BLOCK
--------------CP createvar _mVar528 
scratch_space//_p16480_9.108.91.12//_t0/temp290 true MATRIX binaryblock 1 1 
1000 1000 -1 copy
--------------SPARK uasqk+ W4.MATRIX.DOUBLE _mVar528.MATRIX.DOUBLE SINGLE_BLOCK
--------------CP createvar _mVar529 
scratch_space//_p16480_9.108.91.12//_t0/temp291 true MATRIX binaryblock -1 1 
1000 1000 -1 copy
--------------SPARK uarimax probs.MATRIX.DOUBLE _mVar529.MATRIX.DOUBLE 
MULTI_BLOCK
--------------CP createvar _mVar530 
scratch_space//_p16480_9.108.91.12//_t0/temp292 true MATRIX binaryblock -1 1 
1000 1000 -1 copy
--------------SPARK uarimax y_batch.MATRIX.DOUBLE _mVar530.MATRIX.DOUBLE NONE
--------------CP createvar _mVar531 
scratch_space//_p16480_9.108.91.12//_t0/temp293 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK log _mVar523.MATRIX.DOUBLE _mVar531.MATRIX.DOUBLE
--------------CP rmvar _mVar523
--------------CP createvar _mVar532 
scratch_space//_p16480_9.108.91.12//_t0/temp294 true MATRIX binaryblock 1 1 
1000 1000 -1 copy
--------------SPARK uak+ _mVar525.MATRIX.DOUBLE _mVar532.MATRIX.DOUBLE 
SINGLE_BLOCK
--------------CP rmvar _mVar525
--------------CP castdts _mVar526.MATRIX.DOUBLE.false _Var533.SCALAR.DOUBLE
--------------CP rmvar _mVar526
--------------CP castdts _mVar527.MATRIX.DOUBLE.false _Var534.SCALAR.DOUBLE
--------------CP rmvar _mVar527
--------------CP castdts _mVar528.MATRIX.DOUBLE.false _Var535.SCALAR.DOUBLE
--------------CP rmvar _mVar528
--------------CP createvar _mVar536 
scratch_space//_p16480_9.108.91.12//_t0/temp295 true MATRIX binaryblock -1 1 
1000 1000 -1 copy
--------------SPARK == _mVar529.MATRIX.DOUBLE _mVar530.MATRIX.DOUBLE 
_mVar536.MATRIX.DOUBLE
--------------CP rmvar _mVar529 _mVar530
--------------CP createvar _mVar537 
scratch_space//_p16480_9.108.91.12//_t0/temp296 true MATRIX binaryblock -1 10 
1000 1000 -1 copy
--------------SPARK * _mVar522.MATRIX.DOUBLE _mVar531.MATRIX.DOUBLE 
_mVar537.MATRIX.DOUBLE
--------------CP rmvar _mVar522 _mVar531
--------------CP castdts _mVar532.MATRIX.DOUBLE.false _Var538.SCALAR.DOUBLE
--------------CP rmvar _mVar532
--------------CP * 2.5E-4.SCALAR.DOUBLE.true _Var533.SCALAR.DOUBLE.false 
_Var539.SCALAR.DOUBLE
--------------CP rmvar _Var533
--------------CP * 2.5E-4.SCALAR.DOUBLE.true _Var534.SCALAR.DOUBLE.false 
_Var540.SCALAR.DOUBLE
--------------CP rmvar _Var534
--------------CP * 2.5E-4.SCALAR.DOUBLE.true _Var535.SCALAR.DOUBLE.false 
_Var541.SCALAR.DOUBLE
--------------CP rmvar _Var535
--------------CP createvar _mVar542 
scratch_space//_p16480_9.108.91.12//_t0/temp297 true MATRIX binaryblock 1 1 
1000 1000 -1 copy
--------------SPARK uamean _mVar536.MATRIX.DOUBLE _mVar542.MATRIX.DOUBLE 
SINGLE_BLOCK
--------------CP rmvar _mVar536
--------------CP createvar _mVar543 
scratch_space//_p16480_9.108.91.12//_t0/temp298 true MATRIX binaryblock 1 1 
1000 1000 -1 copy
--------------SPARK uak+ _mVar537.MATRIX.DOUBLE _mVar543.MATRIX.DOUBLE 
SINGLE_BLOCK
--------------CP rmvar _mVar537
--------------CP * 2.5E-4.SCALAR.DOUBLE.true _Var538.SCALAR.DOUBLE.false 
_Var544.SCALAR.DOUBLE
--------------CP rmvar _Var538
--------------CP castdts _mVar542.MATRIX.DOUBLE.false _Var545.SCALAR.DOUBLE
--------------CP rmvar _mVar542
--------------CP castdts _mVar543.MATRIX.DOUBLE.false _Var546.SCALAR.DOUBLE
--------------CP rmvar _mVar543
--------------CP / _Var546.SCALAR.DOUBLE.false _Var524.SCALAR.INT.false 
_Var547.SCALAR.DOUBLE
--------------CP rmvar _Var546 _Var524
--------------CP + _Var547.SCALAR.DOUBLE.false _Var544.SCALAR.DOUBLE.false 
_Var548.SCALAR.DOUBLE
--------------CP rmvar _Var547 _Var544
--------------CP + _Var548.SCALAR.DOUBLE.false _Var539.SCALAR.DOUBLE.false 
_Var549.SCALAR.DOUBLE
--------------CP rmvar _Var548 _Var539
--------------CP + _Var549.SCALAR.DOUBLE.false _Var540.SCALAR.DOUBLE.false 
_Var550.SCALAR.DOUBLE
--------------CP rmvar _Var549 _Var540
--------------CP + _Var550.SCALAR.DOUBLE.false _Var541.SCALAR.DOUBLE.false 
_Var551.SCALAR.DOUBLE
--------------CP rmvar _Var550 _Var541
--------------CP assignvar _Var545.SCALAR.DOUBLE.false accuracy.SCALAR.DOUBLE
--------------CP assignvar _Var551.SCALAR.DOUBLE.false loss.SCALAR.DOUBLE
--------------CP rmvar _Var545 _Var551 probs y_batch
------------GENERIC (lines 257-257) [recompile=false]
--------------CP extfunct ./scripts/nn/examples/mnist_lenet_distrib_sgd.dml 
predict 12 1 X_val.MATRIX.DOUBLE C.SCALAR.INT.false Hin.SCALAR.INT.false 
Win.SCALAR.INT.false W1.MATRIX.DOUBLE b1.MATRIX.DOUBLE W2.MATRIX.DOUBLE 
b2.MATRIX.DOUBLE W3.MATRIX.DOUBLE b3.MATRIX.DOUBLE W4.MATRIX.DOUBLE 
b4.MATRIX.DOUBLE probs_val
------------GENERIC (lines 258-263) [recompile=true]
--------------CP + Epoch: .SCALAR.STRING.true e.SCALAR.INT.false 
_Var552.SCALAR.STRING
--------------CP createvar _mVar553 
scratch_space//_p16480_9.108.91.12//_t0/temp299 true MATRIX binaryblock 32 10 
1000 1000 -1 copy
--------------SPARK - 0.SCALAR.INT.true Y_val.MATRIX.DOUBLE 
_mVar553.MATRIX.DOUBLE
--------------CP createvar _mVar554 
scratch_space//_p16480_9.108.91.12//_t0/temp300 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK + probs_val.MATRIX.DOUBLE 1.0E-10.SCALAR.DOUBLE.true 
_mVar554.MATRIX.DOUBLE
--------------CP createvar _mVar555 
scratch_space//_p16480_9.108.91.12//_t0/temp301 true MATRIX binaryblock -1 1 
1000 1000 -1 copy
--------------SPARK uarimax probs_val.MATRIX.DOUBLE _mVar555.MATRIX.DOUBLE 
MULTI_BLOCK
--------------CP createvar _mVar556 
scratch_space//_p16480_9.108.91.12//_t0/temp302 true MATRIX binaryblock 32 1 
1000 1000 -1 copy
--------------SPARK uarimax Y_val.MATRIX.DOUBLE _mVar556.MATRIX.DOUBLE NONE
--------------CP + _Var552.SCALAR.STRING.false , Group: .SCALAR.STRING.true 
_Var557.SCALAR.STRING
--------------CP rmvar _Var552
--------------CP createvar _mVar558 
scratch_space//_p16480_9.108.91.12//_t0/temp303 true MATRIX binaryblock -1 -1 
1000 1000 -1 copy
--------------SPARK log _mVar554.MATRIX.DOUBLE _mVar558.MATRIX.DOUBLE
--------------CP rmvar _mVar554
--------------CP createvar _mVar559 
scratch_space//_p16480_9.108.91.12//_t0/temp304 true MATRIX binaryblock 32 1 
1000 1000 -1 copy
--------------SPARK map== _mVar555.MATRIX.DOUBLE _mVar556.MATRIX.DOUBLE 
_mVar559.MATRIX.DOUBLE RIGHT ROW_VECTOR
--------------CP rmvar _mVar555 _mVar556
--------------CP + _Var557.SCALAR.STRING.false g.SCALAR.INT.false 
_Var560.SCALAR.STRING
--------------CP rmvar _Var557
--------------CP createvar _mVar561 
scratch_space//_p16480_9.108.91.12//_t0/temp305 true MATRIX binaryblock 32 10 
1000 1000 -1 copy
--------------SPARK * _mVar553.MATRIX.DOUBLE _mVar558.MATRIX.DOUBLE 
_mVar561.MATRIX.DOUBLE
--------------CP rmvar _mVar553 _mVar558
--------------CP createvar _mVar562 
scratch_space//_p16480_9.108.91.12//_t0/temp306 true MATRIX binaryblock 1 1 
1000 1000 -1 copy
--------------SPARK uamean _mVar559.MATRIX.DOUBLE _mVar562.MATRIX.DOUBLE 
SINGLE_BLOCK
--------------CP rmvar _mVar559
--------------CP + _Var560.SCALAR.STRING.false , Train Loss: 
.SCALAR.STRING.true _Var563.SCALAR.STRING
--------------CP rmvar _Var560
--------------CP createvar _mVar564 
scratch_space//_p16480_9.108.91.12//_t0/temp307 true MATRIX binaryblock 1 1 
1000 1000 -1 copy
--------------SPARK uak+ _mVar561.MATRIX.DOUBLE _mVar564.MATRIX.DOUBLE 
SINGLE_BLOCK
--------------CP rmvar _mVar561
--------------CP castdts _mVar562.MATRIX.DOUBLE.false _Var565.SCALAR.DOUBLE
--------------CP rmvar _mVar562
--------------CP + _Var563.SCALAR.STRING.false loss.SCALAR.DOUBLE.false 
_Var566.SCALAR.STRING
--------------CP rmvar _Var563
--------------CP castdts _mVar564.MATRIX.DOUBLE.false _Var567.SCALAR.DOUBLE
--------------CP rmvar _mVar564
--------------CP + _Var566.SCALAR.STRING.false , Train Accuracy: 
.SCALAR.STRING.true _Var568.SCALAR.STRING
--------------CP rmvar _Var566
--------------CP / _Var567.SCALAR.DOUBLE.false 32.SCALAR.INT.true 
_Var569.SCALAR.DOUBLE
--------------CP rmvar _Var567
--------------CP + _Var568.SCALAR.STRING.false accuracy.SCALAR.DOUBLE.false 
_Var570.SCALAR.STRING
--------------CP rmvar _Var568
--------------CP + _Var570.SCALAR.STRING.false , Val Loss: .SCALAR.STRING.true 
_Var571.SCALAR.STRING
--------------CP rmvar _Var570
--------------CP + _Var571.SCALAR.STRING.false _Var569.SCALAR.DOUBLE.false 
_Var572.SCALAR.STRING
--------------CP rmvar _Var571 _Var569
--------------CP + _Var572.SCALAR.STRING.false , Val Accuracy: 
.SCALAR.STRING.true _Var573.SCALAR.STRING
--------------CP rmvar _Var572
--------------CP + _Var573.SCALAR.STRING.false _Var565.SCALAR.DOUBLE.false 
_Var574.SCALAR.STRING
--------------CP rmvar _Var573 _Var565
--------------CP print _Var574.SCALAR.STRING.false _Var575.SCALAR.STRING
--------------CP rmvar _Var574 _Var575 accuracy loss probs_val
--------GENERIC (lines 269-269) [recompile=false]
----------CP * lr.SCALAR.DOUBLE.false 0.95.SCALAR.DOUBLE.true 
_Var576.SCALAR.DOUBLE
----------CP assignvar _Var576.SCALAR.DOUBLE.false lr.SCALAR.DOUBLE
----------CP rmvar _Var576
----FUNCTION ./scripts/nn/examples/mnist_lenet_distrib_sgd.dml::predict 
[recompile=true]
------GENERIC (lines 303-320) [recompile=true]
--------CP nrow X.MATRIX.DOUBLE.false _Var76.SCALAR.INT
--------CP nrow W1.MATRIX.DOUBLE.false _Var77.SCALAR.INT
--------CP nrow W2.MATRIX.DOUBLE.false _Var78.SCALAR.INT
--------CP ncol W4.MATRIX.DOUBLE.false _Var79.SCALAR.INT
--------CP createvar _mVar80 scratch_space//_p16480_9.108.91.12//_t0/temp30 
true MATRIX binaryblock -1 -1 1000 1000 0 copy
--------SPARK rand ¶_Var76¶ ¶_Var79¶ 1000 1000 0.0 0.0 1.0 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar80.MATRIX.DOUBLE
--------CP rmvar _Var79
--------CP / _Var76.SCALAR.INT.false 64.SCALAR.INT.true _Var81.SCALAR.DOUBLE
--------CP ceil _Var81.SCALAR.DOUBLE.false _Var82.SCALAR.DOUBLE
--------CP rmvar _Var81
--------CP assignvar 5.SCALAR.INT.true Hf.SCALAR.INT
--------CP assignvar 5.SCALAR.INT.true Wf.SCALAR.INT
--------CP assignvar 1.SCALAR.INT.true stride.SCALAR.INT
--------CP assignvar 2.SCALAR.INT.true pad.SCALAR.INT
--------CP assignvar 64.SCALAR.INT.true batch_size.SCALAR.INT
--------CP assignvar _Var76.SCALAR.INT.false N.SCALAR.INT
--------CP assignvar _Var77.SCALAR.INT.false F1.SCALAR.INT
--------CP assignvar _Var78.SCALAR.INT.false F2.SCALAR.INT
--------CP assignvar _Var82.SCALAR.DOUBLE.false iters.SCALAR.DOUBLE
--------CP rmvar _Var76 _Var77 _Var78
--------CP cpvar _mVar80 probs
--------CP rmvar _Var82 _mVar80
------GENERIC (lines 321-349) [recompile=true]
--------CP createvar _mVar83 scratch_space//_p16480_9.108.91.12//_t0/temp31 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b1.MATRIX.DOUBLE _mVar83.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar84 scratch_space//_p16480_9.108.91.12//_t0/temp32 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b2.MATRIX.DOUBLE _mVar84.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar85 scratch_space//_p16480_9.108.91.12//_t0/temp33 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b3.MATRIX.DOUBLE _mVar85.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar86 scratch_space//_p16480_9.108.91.12//_t0/temp34 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b4.MATRIX.DOUBLE _mVar86.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar87 scratch_space//_p16480_9.108.91.12//_t0/temp35 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint X.MATRIX.DOUBLE _mVar87.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar88 scratch_space//_p16480_9.108.91.12//_t0/temp36 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W1.MATRIX.DOUBLE _mVar88.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar89 scratch_space//_p16480_9.108.91.12//_t0/temp37 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W2.MATRIX.DOUBLE _mVar89.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar90 scratch_space//_p16480_9.108.91.12//_t0/temp38 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W3.MATRIX.DOUBLE _mVar90.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar91 scratch_space//_p16480_9.108.91.12//_t0/temp39 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W4.MATRIX.DOUBLE _mVar91.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP rmvar b2 b3 b4 X W1 W2 W3 W4 b1
--------CP cpvar _mVar83 b1
--------CP cpvar _mVar84 b2
--------CP cpvar _mVar85 b3
--------CP cpvar _mVar86 b4
--------CP cpvar _mVar87 X
--------CP cpvar _mVar88 W1
--------CP cpvar _mVar89 W2
--------CP cpvar _mVar90 W3
--------CP cpvar _mVar91 W4
--------CP rmvar _mVar83 _mVar84 _mVar85 _mVar86 _mVar87 _mVar88 _mVar89 
_mVar90 _mVar91
------GENERIC (lines 321-349) [recompile=true]
--------CP createvar _mVar92 scratch_space//_p16480_9.108.91.12//_t0/temp40 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b1.MATRIX.DOUBLE _mVar92.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar93 scratch_space//_p16480_9.108.91.12//_t0/temp41 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b2.MATRIX.DOUBLE _mVar93.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar94 scratch_space//_p16480_9.108.91.12//_t0/temp42 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b3.MATRIX.DOUBLE _mVar94.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar95 scratch_space//_p16480_9.108.91.12//_t0/temp43 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b4.MATRIX.DOUBLE _mVar95.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar96 scratch_space//_p16480_9.108.91.12//_t0/temp44 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint X.MATRIX.DOUBLE _mVar96.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar97 scratch_space//_p16480_9.108.91.12//_t0/temp45 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W1.MATRIX.DOUBLE _mVar97.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar98 scratch_space//_p16480_9.108.91.12//_t0/temp46 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W2.MATRIX.DOUBLE _mVar98.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar99 scratch_space//_p16480_9.108.91.12//_t0/temp47 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W3.MATRIX.DOUBLE _mVar99.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar100 scratch_space//_p16480_9.108.91.12//_t0/temp48 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W4.MATRIX.DOUBLE _mVar100.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP rmvar b2 b3 b4 X W1 W2 W3 W4 b1
--------CP cpvar _mVar92 b1
--------CP cpvar _mVar93 b2
--------CP cpvar _mVar94 b3
--------CP cpvar _mVar95 b4
--------CP cpvar _mVar96 X
--------CP cpvar _mVar97 W1
--------CP cpvar _mVar98 W2
--------CP cpvar _mVar99 W3
--------CP cpvar _mVar100 W4
--------CP rmvar _mVar92 _mVar93 _mVar94 _mVar95 _mVar96 _mVar97 _mVar98 
_mVar99 _mVar100
------GENERIC (lines 321-349) [recompile=true]
--------CP createvar _mVar101 scratch_space//_p16480_9.108.91.12//_t0/temp49 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b1.MATRIX.DOUBLE _mVar101.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar102 scratch_space//_p16480_9.108.91.12//_t0/temp50 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b2.MATRIX.DOUBLE _mVar102.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar103 scratch_space//_p16480_9.108.91.12//_t0/temp51 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b3.MATRIX.DOUBLE _mVar103.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar104 scratch_space//_p16480_9.108.91.12//_t0/temp52 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint b4.MATRIX.DOUBLE _mVar104.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar105 scratch_space//_p16480_9.108.91.12//_t0/temp53 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint X.MATRIX.DOUBLE _mVar105.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar106 scratch_space//_p16480_9.108.91.12//_t0/temp54 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W1.MATRIX.DOUBLE _mVar106.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar107 scratch_space//_p16480_9.108.91.12//_t0/temp55 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W2.MATRIX.DOUBLE _mVar107.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar108 scratch_space//_p16480_9.108.91.12//_t0/temp56 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W3.MATRIX.DOUBLE _mVar108.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP createvar _mVar109 scratch_space//_p16480_9.108.91.12//_t0/temp57 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
--------SPARK chkpoint W4.MATRIX.DOUBLE _mVar109.MATRIX.DOUBLE MEMORY_AND_DISK
--------CP rmvar b2 b3 b4 X W1 W2 W3 W4 b1
--------CP cpvar _mVar101 b1
--------CP cpvar _mVar102 b2
--------CP cpvar _mVar103 b3
--------CP cpvar _mVar104 b4
--------CP cpvar _mVar105 X
--------CP cpvar _mVar106 W1
--------CP cpvar _mVar107 W2
--------CP cpvar _mVar108 W3
--------CP cpvar _mVar109 W4
--------CP rmvar _mVar101 _mVar102 _mVar103 _mVar104 _mVar105 _mVar106 _mVar107 
_mVar108 _mVar109
------PARFOR (lines 321-349)
--------GENERIC (lines 323-348) [recompile=true]
----------CP - i.SCALAR.INT.false 1.SCALAR.INT.true _Var110.SCALAR.INT
----------CP ncol X.MATRIX.DOUBLE.false _Var111.SCALAR.INT
----------CP nrow W1.MATRIX.DOUBLE.false _Var112.SCALAR.INT
----------CP * 2.SCALAR.INT.true pad.SCALAR.INT.false _Var113.SCALAR.INT
----------CP nrow W2.MATRIX.DOUBLE.false _Var114.SCALAR.INT
----------CP ncol probs.MATRIX.DOUBLE.false _Var115.SCALAR.INT
----------CP * _Var110.SCALAR.INT.false 64.SCALAR.INT.true _Var116.SCALAR.INT
----------CP rmvar _Var110
----------CP + Hin.SCALAR.INT.false _Var113.SCALAR.INT.false _Var117.SCALAR.INT
----------CP + Win.SCALAR.INT.false _Var113.SCALAR.INT.false _Var118.SCALAR.INT
----------CP %% _Var116.SCALAR.INT.false N.SCALAR.INT.false _Var119.SCALAR.INT
----------CP rmvar _Var116
----------CP - _Var117.SCALAR.INT.false Hf.SCALAR.INT.false _Var120.SCALAR.INT
----------CP rmvar _Var117
----------CP - _Var118.SCALAR.INT.false Wf.SCALAR.INT.false _Var121.SCALAR.INT
----------CP rmvar _Var118
----------CP + _Var119.SCALAR.INT.false 1.SCALAR.INT.true _Var122.SCALAR.INT
----------CP rmvar _Var119
----------CP / _Var120.SCALAR.INT.false stride.SCALAR.INT.false 
_Var123.SCALAR.DOUBLE
----------CP rmvar _Var120
----------CP / _Var121.SCALAR.INT.false stride.SCALAR.INT.false 
_Var124.SCALAR.DOUBLE
----------CP rmvar _Var121
----------CP + _Var122.SCALAR.INT.false 64.SCALAR.INT.true _Var125.SCALAR.INT
----------CP + _Var123.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var126.SCALAR.DOUBLE
----------CP rmvar _Var123
----------CP + _Var124.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var127.SCALAR.DOUBLE
----------CP rmvar _Var124
----------CP - _Var125.SCALAR.INT.false 1.SCALAR.INT.true _Var128.SCALAR.INT
----------CP rmvar _Var125
----------CP floor _Var126.SCALAR.DOUBLE.false _Var129.SCALAR.DOUBLE
----------CP rmvar _Var126
----------CP floor _Var127.SCALAR.DOUBLE.false _Var130.SCALAR.DOUBLE
----------CP rmvar _Var127
----------CP min N.SCALAR.INT.false _Var128.SCALAR.INT.false _Var131.SCALAR.INT
----------CP rmvar _Var128
----------CP castvti _Var129.SCALAR.DOUBLE.false _Var132.SCALAR.INT
----------CP rmvar _Var129
----------CP castvti _Var130.SCALAR.DOUBLE.false _Var133.SCALAR.INT
----------CP rmvar _Var130
----------CP createvar _mVar134 scratch_space//_p16480_9.108.91.12//_t0/temp58 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK rangeReIndex X.MATRIX.DOUBLE _Var122.SCALAR.INT.false 
_Var131.SCALAR.INT.false 1.SCALAR.INT.true _Var111.SCALAR.INT.false 
_mVar134.MATRIX.DOUBLE MULTI_BLOCK
----------CP rmvar _Var111
----------CP + _Var132.SCALAR.INT.false 0.SCALAR.INT.true _Var135.SCALAR.INT
----------CP + _Var133.SCALAR.INT.false 0.SCALAR.INT.true _Var136.SCALAR.INT
----------CP nrow _mVar134.MATRIX.DOUBLE.false _Var137.SCALAR.INT
----------CP - _Var135.SCALAR.INT.false 2.SCALAR.INT.true _Var138.SCALAR.INT
----------CP rmvar _Var135
----------CP - _Var136.SCALAR.INT.false 2.SCALAR.INT.true _Var139.SCALAR.INT
----------CP rmvar _Var136
----------CP createvar _mVar140 scratch_space//_p16480_9.108.91.12//_t0/temp59 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------CP conv2d_bias_add _mVar134.MATRIX.DOUBLE b1.MATRIX.DOUBLE 
W1.MATRIX.DOUBLE stride.SCALAR.INT.false stride.SCALAR.INT.false 
pad.SCALAR.INT.false pad.SCALAR.INT.false _Var137.SCALAR.INT.false 
C.SCALAR.INT.false Hin.SCALAR.INT.false Win.SCALAR.INT.false 
_Var112.SCALAR.INT.false C.SCALAR.INT.false Hf.SCALAR.INT.false 
Wf.SCALAR.INT.false _mVar140.MATRIX.DOUBLE 8
----------CP rmvar _Var137 _Var112
----------CP / _Var138.SCALAR.INT.false 2.SCALAR.INT.true _Var141.SCALAR.DOUBLE
----------CP rmvar _Var138
----------CP / _Var139.SCALAR.INT.false 2.SCALAR.INT.true _Var142.SCALAR.DOUBLE
----------CP rmvar _Var139
----------CP nrow _mVar140.MATRIX.DOUBLE.false _Var143.SCALAR.INT
----------CP + _Var141.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var144.SCALAR.DOUBLE
----------CP rmvar _Var141
----------CP + _Var142.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var145.SCALAR.DOUBLE
----------CP rmvar _Var142
----------CP createvar _mVar146 scratch_space//_p16480_9.108.91.12//_t0/temp60 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------CP relu_maxpooling _mVar140.MATRIX.DOUBLE 2.SCALAR.INT.true 
2.SCALAR.INT.true 0.SCALAR.INT.true 0.SCALAR.INT.true _Var143.SCALAR.INT.false 
F1.SCALAR.INT.false _Var132.SCALAR.INT.false _Var133.SCALAR.INT.false 
1.SCALAR.INT.true F1.SCALAR.INT.false 2.SCALAR.INT.true 2.SCALAR.INT.true 
_mVar146.MATRIX.DOUBLE 8
----------CP rmvar _Var143 _Var132 _Var133
----------CP floor _Var144.SCALAR.DOUBLE.false _Var147.SCALAR.DOUBLE
----------CP rmvar _Var144
----------CP floor _Var145.SCALAR.DOUBLE.false _Var148.SCALAR.DOUBLE
----------CP rmvar _Var145
----------CP nrow _mVar146.MATRIX.DOUBLE.false _Var149.SCALAR.INT
----------CP castvti _Var147.SCALAR.DOUBLE.false _Var150.SCALAR.INT
----------CP rmvar _Var147
----------CP castvti _Var148.SCALAR.DOUBLE.false _Var151.SCALAR.INT
----------CP rmvar _Var148
----------CP createvar _mVar152 scratch_space//_p16480_9.108.91.12//_t0/temp61 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------CP conv2d_bias_add _mVar146.MATRIX.DOUBLE b2.MATRIX.DOUBLE 
W2.MATRIX.DOUBLE stride.SCALAR.INT.false stride.SCALAR.INT.false 
pad.SCALAR.INT.false pad.SCALAR.INT.false _Var149.SCALAR.INT.false 
F1.SCALAR.INT.false _Var150.SCALAR.INT.false _Var151.SCALAR.INT.false 
_Var114.SCALAR.INT.false F1.SCALAR.INT.false Hf.SCALAR.INT.false 
Wf.SCALAR.INT.false _mVar152.MATRIX.DOUBLE 8
----------CP rmvar _Var149 _Var114
----------CP + _Var150.SCALAR.INT.false _Var113.SCALAR.INT.false 
_Var153.SCALAR.INT
----------CP rmvar _Var150
----------CP + _Var151.SCALAR.INT.false _Var113.SCALAR.INT.false 
_Var154.SCALAR.INT
----------CP rmvar _Var151 _Var113
----------CP nrow _mVar152.MATRIX.DOUBLE.false _Var155.SCALAR.INT
----------CP - _Var153.SCALAR.INT.false Hf.SCALAR.INT.false _Var156.SCALAR.INT
----------CP rmvar _Var153
----------CP - _Var154.SCALAR.INT.false Wf.SCALAR.INT.false _Var157.SCALAR.INT
----------CP rmvar _Var154
----------CP / _Var156.SCALAR.INT.false stride.SCALAR.INT.false 
_Var158.SCALAR.DOUBLE
----------CP rmvar _Var156
----------CP / _Var157.SCALAR.INT.false stride.SCALAR.INT.false 
_Var159.SCALAR.DOUBLE
----------CP rmvar _Var157
----------CP + _Var158.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var160.SCALAR.DOUBLE
----------CP rmvar _Var158
----------CP + _Var159.SCALAR.DOUBLE.false 1.SCALAR.INT.true 
_Var161.SCALAR.DOUBLE
----------CP rmvar _Var159
----------CP floor _Var160.SCALAR.DOUBLE.false _Var162.SCALAR.DOUBLE
----------CP rmvar _Var160
----------CP floor _Var161.SCALAR.DOUBLE.false _Var163.SCALAR.DOUBLE
----------CP rmvar _Var161
----------CP castvti _Var162.SCALAR.DOUBLE.false _Var164.SCALAR.INT
----------CP rmvar _Var162
----------CP castvti _Var163.SCALAR.DOUBLE.false _Var165.SCALAR.INT
----------CP rmvar _Var163
----------CP createvar _mVar166 scratch_space//_p16480_9.108.91.12//_t0/temp62 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------CP relu_maxpooling _mVar152.MATRIX.DOUBLE 2.SCALAR.INT.true 
2.SCALAR.INT.true 0.SCALAR.INT.true 0.SCALAR.INT.true _Var155.SCALAR.INT.false 
F2.SCALAR.INT.false _Var164.SCALAR.INT.false _Var165.SCALAR.INT.false 
1.SCALAR.INT.true F2.SCALAR.INT.false 2.SCALAR.INT.true 2.SCALAR.INT.true 
_mVar166.MATRIX.DOUBLE 8
----------CP rmvar _Var155 _Var164 _Var165
----------CP createvar _mVar167 scratch_space//_p16480_9.108.91.12//_t0/temp63 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK cpmm _mVar166.MATRIX.DOUBLE W3.MATRIX.DOUBLE 
_mVar167.MATRIX.DOUBLE MULTI_BLOCK
----------CP rmvar _mVar166
----------CP createvar _mVar168 scratch_space//_p16480_9.108.91.12//_t0/temp64 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK + _mVar167.MATRIX.DOUBLE b3.MATRIX.DOUBLE _mVar168.MATRIX.DOUBLE
----------CP rmvar _mVar167
----------CP createvar _mVar169 scratch_space//_p16480_9.108.91.12//_t0/temp65 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK sel+ _mVar168.MATRIX.DOUBLE _mVar169.MATRIX.DOUBLE
----------CP rmvar _mVar168
----------CP createvar _mVar170 scratch_space//_p16480_9.108.91.12//_t0/temp66 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK cpmm _mVar169.MATRIX.DOUBLE W4.MATRIX.DOUBLE 
_mVar170.MATRIX.DOUBLE MULTI_BLOCK
----------CP rmvar _mVar169
----------CP createvar _mVar171 scratch_space//_p16480_9.108.91.12//_t0/temp67 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK + _mVar170.MATRIX.DOUBLE b4.MATRIX.DOUBLE _mVar171.MATRIX.DOUBLE
----------CP rmvar _mVar170
----------CP createvar _mVar172 scratch_space//_p16480_9.108.91.12//_t0/temp68 
true MATRIX binaryblock -1 1 1000 1000 -1 copy
----------SPARK uarmax _mVar171.MATRIX.DOUBLE _mVar172.MATRIX.DOUBLE MULTI_BLOCK
----------CP createvar _mVar173 scratch_space//_p16480_9.108.91.12//_t0/temp69 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK - _mVar171.MATRIX.DOUBLE _mVar172.MATRIX.DOUBLE 
_mVar173.MATRIX.DOUBLE
----------CP rmvar _mVar171 _mVar172
----------CP createvar _mVar174 scratch_space//_p16480_9.108.91.12//_t0/temp70 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK exp _mVar173.MATRIX.DOUBLE _mVar174.MATRIX.DOUBLE
----------CP rmvar _mVar173
----------CP createvar _mVar175 scratch_space//_p16480_9.108.91.12//_t0/temp71 
true MATRIX binaryblock -1 1 1000 1000 -1 copy
----------SPARK uark+ _mVar174.MATRIX.DOUBLE _mVar175.MATRIX.DOUBLE MULTI_BLOCK
----------CP createvar _mVar176 scratch_space//_p16480_9.108.91.12//_t0/temp72 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK / _mVar174.MATRIX.DOUBLE _mVar175.MATRIX.DOUBLE 
_mVar176.MATRIX.DOUBLE
----------CP rmvar _mVar174 _mVar175
----------CP createvar _mVar177 scratch_space//_p16480_9.108.91.12//_t0/temp73 
true MATRIX binaryblock -1 -1 1000 1000 -1 copy
----------SPARK leftIndex probs.MATRIX.DOUBLE _mVar176.MATRIX.DOUBLE 
_Var122.SCALAR.INT.false _Var131.SCALAR.INT.false 1.SCALAR.INT.true 
_Var115.SCALAR.INT.false _mVar177.MATRIX.DOUBLE NONE
----------CP rmvar _mVar176 _Var122 _Var131 _Var115 probs _mVar134 _mVar140 
_mVar146 _mVar152
----------CP cpvar _mVar177 probs
----------CP rmvar _mVar177
----FUNCTION ./nn/layers/dropout.dml::forward [recompile=false]
------GENERIC (lines 50-50) [recompile=true]
--------CP nrow X.MATRIX.DOUBLE.false _Var577.SCALAR.INT
--------CP createvar _mVar578 scratch_space//_p16480_9.108.91.12//_t0/temp308 
true MATRIX binaryblock -1 512 1000 1000 -1 copy
--------SPARK rand ¶_Var577¶ 512 1000 1000 1 1 0.5 -1 
scratch_space/_p16480_9.108.91.12//_t0/ uniform 1.0 _mVar578.MATRIX.DOUBLE
--------CP rmvar _Var577
--------CP cpvar _mVar578 mask
--------CP rmvar _mVar578
------GENERIC (lines 54-54) [recompile=true]
--------CP createvar _mVar579 scratch_space//_p16480_9.108.91.12//_t0/temp309 
true MATRIX binaryblock -1 512 1000 1000 -1 copy
--------SPARK * X.MATRIX.DOUBLE mask.MATRIX.DOUBLE _mVar579.MATRIX.DOUBLE
--------CP createvar _mVar580 scratch_space//_p16480_9.108.91.12//_t0/temp310 
true MATRIX binaryblock -1 512 1000 1000 -1 copy
--------SPARK / _mVar579.MATRIX.DOUBLE 0.5.SCALAR.DOUBLE.true 
_mVar580.MATRIX.DOUBLE
--------CP rmvar _mVar579
--------CP cpvar _mVar580 out
--------CP rmvar _mVar580 p X
--MAIN PROGRAM
----GENERIC (lines 1-0) [recompile=true]
------CP createvar _mVar581 scratch_space//_p16480_9.108.91.12//_t0/temp311 
true MATRIX binaryblock 32 37632 1000 1000 1204224 copy
------SPARK chkpoint X.MATRIX.DOUBLE _mVar581.MATRIX.DOUBLE MEMORY_AND_DISK
------CP createvar _mVar582 scratch_space//_p16480_9.108.91.12//_t0/temp312 
true MATRIX binaryblock 32 10 1000 1000 -1 copy
------SPARK chkpoint Y.MATRIX.DOUBLE _mVar582.MATRIX.DOUBLE MEMORY_AND_DISK
------CP createvar _mVar583 scratch_space//_p16480_9.108.91.12//_t0/temp313 
true MATRIX binaryblock 32 37632 1000 1000 1204224 copy
------SPARK chkpoint X_val.MATRIX.DOUBLE _mVar583.MATRIX.DOUBLE MEMORY_AND_DISK
------CP createvar _mVar584 scratch_space//_p16480_9.108.91.12//_t0/temp314 
true MATRIX binaryblock 32 10 1000 1000 -1 copy
------SPARK chkpoint Y_val.MATRIX.DOUBLE _mVar584.MATRIX.DOUBLE MEMORY_AND_DISK
------CP rmvar X Y_val Y X_val
------CP cpvar _mVar581 X
------CP cpvar _mVar582 Y
------CP cpvar _mVar583 X_val
------CP cpvar _mVar584 Y_val
------CP rmvar _mVar581 _mVar582 _mVar583 _mVar584
----GENERIC (lines 11-11) [recompile=false]
------CP extfunct ./scripts/nn/examples/mnist_lenet_distrib_sgd.dml train 10 8 
X.MATRIX.DOUBLE Y.MATRIX.DOUBLE X_val.MATRIX.DOUBLE Y_val.MATRIX.DOUBLE 
C.SCALAR.INT.false Hin.SCALAR.INT.false Win.SCALAR.INT.false 
batch_size.SCALAR.INT.false parallel_batches.SCALAR.INT.false 
epochs.SCALAR.INT.false W1 b1 W2 b2 W3 b3 W4 b4
------CP rmvar C batch_size X Y_val Y Hin parallel_batches Win epochs X_val
----GENERIC (lines 12-19) [recompile=true]
------CP rmvar b2 b3 b4 W1 W2 W3 W4 b1
{code}







was (Author: tenma):
The errors when running the [distributed MNIST LeNet example |  
https://github.com/apache/systemml/blob/master/scripts/nn/examples/mnist_lenet_distrib_sgd.dml]
 using the mode {{SPARK}} are as following. One example of the errors is that 
at the convolutional layer, we need to randomly generate some matrixes, but 
SystemML choose {{RandSPInstruction}} instead of {{DataGenCPInstruction}}, 
which may be because SystemML could not determine the row number of the matrix. 
For this distributed MNIST LeNet  example, using CPInstruction may achieve 
better performance. 

{code:java}
Exception in thread "main" org.apache.sysml.api.mlcontext.MLContextException: 
Exception when executing script
        at org.apache.sysml.api.mlcontext.MLContext.execute(MLContext.java:320)
        at org.apache.sysml.api.mlcontext.MLContext.execute(MLContext.java:293)
        at org.apache.sysml.api.mlcontext.Script.execute(Script.java:717)
        at 
org.apache.sysml.scripts.nn.examples.Mnist_lenet_distrib_sgd.train(Mnist_lenet_distrib_sgd.java)
        at 
org.apache.sysml.examples.MNIST_Distrib_Sgd$.main(MNIST_Distrib_Sgd.scala:41)
        at 
org.apache.sysml.examples.MNIST_Distrib_Sgd.main(MNIST_Distrib_Sgd.scala)
Caused by: org.apache.sysml.api.mlcontext.MLContextException: Exception 
occurred while executing runtime program
        at 
org.apache.sysml.api.mlcontext.ScriptExecutor.executeRuntimeProgram(ScriptExecutor.java:397)
        at 
org.apache.sysml.api.mlcontext.ScriptExecutor.execute(ScriptExecutor.java:311)
        at org.apache.sysml.api.mlcontext.MLContext.execute(MLContext.java:316)
        ... 5 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: 
org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in program 
block generated from statement block between lines 11 and 11 -- Error 
evaluating instruction: 
CP°extfunct°./scripts/nn/examples/mnist_lenet_distrib_sgd.dml°train°10°8°X·MATRIX·DOUBLE°Y·MATRIX·DOUBLE°X_val·MATRIX·DOUBLE°Y_val·MATRIX·DOUBLE°C·SCALAR·INT·false°Hin·SCALAR·INT·false°Win·SCALAR·INT·false°batch_size·SCALAR·INT·false°parallel_batches·SCALAR·INT·false°epochs·SCALAR·INT·false°W1°b1°W2°b2°W3°b3°W4°b4
        at 
org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:130)
        at 
org.apache.sysml.api.ScriptExecutorUtils.executeRuntimeProgram(ScriptExecutorUtils.java:95)
        at 
org.apache.sysml.api.ScriptExecutorUtils.executeRuntimeProgram(ScriptExecutorUtils.java:54)
        at 
org.apache.sysml.api.mlcontext.ScriptExecutor.executeRuntimeProgram(ScriptExecutor.java:395)
        ... 7 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error 
in program block generated from statement block between lines 11 and 11 -- 
Error evaluating instruction: 
CP°extfunct°./scripts/nn/examples/mnist_lenet_distrib_sgd.dml°train°10°8°X·MATRIX·DOUBLE°Y·MATRIX·DOUBLE°X_val·MATRIX·DOUBLE°Y_val·MATRIX·DOUBLE°C·SCALAR·INT·false°Hin·SCALAR·INT·false°Win·SCALAR·INT·false°batch_size·SCALAR·INT·false°parallel_batches·SCALAR·INT·false°epochs·SCALAR·INT·false°W1°b1°W2°b2°W3°b3°W4°b4
        at 
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:316)
        at 
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:217)
        at 
org.apache.sysml.runtime.controlprogram.ProgramBlock.execute(ProgramBlock.java:163)
        at 
org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:123)
        ... 10 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: error executing 
function ./scripts/nn/examples/mnist_lenet_distrib_sgd.dml::train
        at 
org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:187)
        at 
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:286)
        ... 13 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error 
in function program block generated from function statement block between lines 
36 and 271 -- Error evaluating function program block
        at 
org.apache.sysml.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:122)
        at 
org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:180)
        ... 14 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error 
in for program block generated from for statement block between lines 115 and 
270 -- Error evaluating for program block
        at 
org.apache.sysml.runtime.controlprogram.ForProgramBlock.execute(ForProgramBlock.java:157)
        at 
org.apache.sysml.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:115)
        ... 15 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error 
in for program block generated from for statement block between lines 117 and 
265 -- Error evaluating for program block
        at 
org.apache.sysml.runtime.controlprogram.ForProgramBlock.execute(ForProgramBlock.java:157)
        at 
org.apache.sysml.runtime.controlprogram.ForProgramBlock.execute(ForProgramBlock.java:145)
        ... 16 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: PARFOR: Failed to 
execute loop in parallel.
        at 
org.apache.sysml.runtime.controlprogram.ParForProgramBlock.execute(ParForProgramBlock.java:660)
        at 
org.apache.sysml.runtime.controlprogram.ForProgramBlock.execute(ForProgramBlock.java:145)
        ... 17 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: Not supported: 
Instructions of type other than CP instructions 
org.apache.sysml.runtime.instructions.spark.RandSPInstruction
SPARK°rand°¶_Var3092¶°512°1000°1000°1°1°0.5°-1°scratch_space/_p10640_9.108.91.12//_t0/°uniform°1.0°_mVar3093·MATRIX·DOUBLE
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.serializeInstructions(ProgramConverter.java:937)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.rSerializeProgramBlock(ProgramConverter.java:1297)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.rSerializeProgramBlocks(ProgramConverter.java:1125)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.rSerializeProgramBlock(ProgramConverter.java:1264)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.rSerializeFunctionProgramBlocks(ProgramConverter.java:1106)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.serializeProgram(ProgramConverter.java:805)
        at 
org.apache.sysml.runtime.controlprogram.parfor.ProgramConverter.serializeParForBody(ProgramConverter.java:763)
        at 
org.apache.sysml.runtime.controlprogram.ParForProgramBlock.executeRemoteSparkParFor(ParForProgramBlock.java:1034)
        at 
org.apache.sysml.runtime.controlprogram.ParForProgramBlock.execute(ParForProgramBlock.java:647)
        ... 18 more
{code}


> Improve Parfor parallelism for deep learning
> --------------------------------------------
>
>                 Key: SYSTEMML-1774
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1774
>             Project: SystemML
>          Issue Type: Improvement
>          Components: Algorithms, Compiler, ParFor
>    Affects Versions: SystemML 1.0
>            Reporter: Fei Hu
>              Labels: deeplearning
>         Attachments: mnist_lenet_distrib_sgd.dml
>
>
> When running the  [distributed MNIST LeNet example | 
> https://github.com/apache/systemml/blob/master/scripts/nn/examples/mnist_lenet_distrib_sgd.dml],
>  each mini-batch could ideally run in parallel without interaction. We try to 
> force {{parfor (j in 1:parallel_batches)}} at line 137 of 
> {{nn/examples/mnist_lenet_distrib_sgd.dml}} to be {{parfor (j in 
> 1:parallel_batches, mode=REMOTE_SPARK, opt=CONSTRAINED)}} use 
> {{REMOTE_SPARK}} mode, but got some errors about 
> {{org.apache.sysml.runtime.DMLRuntimeException: Not supported: Instructions 
> of type other than CP instructions}} using the mode {{SPARK}}, and the error 
> {{java.lang.NullPointerException}} using the mode {{HYBRID_SPARK}}. More log 
> information can be found at the following comments. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)