HubertKrawczyk opened a new pull request, #2349:
URL: https://github.com/apache/systemds/pull/2349
Removed old optimization for LibSpoofPrimitives.vectOuterMultAdd as
LibMatrixMult.vectMultiplyAdd with its vectorized implementation is faster.
Benchmark:
I modified test src/test/scripts/functions/codegen/rowAggPattern26.dml to
run multiple times on bigger data:
``` rowAggPattern26.dml
X = matrix(seq(1,3000000), 60000, 50);
P = matrix(seq(1,600000), 60000, 10)
while(FALSE){}
for (i in 1:500) {
R = t(P) %*% X;
}
write(R, $1)
```
Old version statistics:
```
SystemDS Statistics:
Total elapsed time: 10,088 sec.
Total compilation time: 0,692 sec.
Total execution time: 9,396 sec.
Number of compiled Spark inst: 8.
Number of executed Spark inst: 507.
Cache hits (Mem/Li/WB/FS/HDFS): 1/0/0/0/0.
Cache writes (Li/WB/FS/HDFS): 0/1/0/0.
Cache times (ACQr/m, RLS, EXP): 0,302/0,003/0,004/0,000 sec.
HOP DAGs recompiled (PRED, SB): 0/503.
HOP DAGs recompile time: 0,277 sec.
Functions recompiled: 2.
Functions recompile time: 0,002 sec.
Codegen compile (DAG,CP,JC): 516/502/1.
Codegen enum (ALLt/p,EVALt/p): 516/0/0/0.
Codegen compile times (DAG,JC): 0,333/0,153 sec.
Codegen enum plan cache hits: 0/0.
Codegen op plan cache hits: 501/502.
Spark ctx create time (lazy): 0,672 sec.
Spark trans counts (par,bc,col):1/500/1.
Spark trans times (par,bc,col): 0,000/0,348/0,302 secs.
Total JIT compile time: 11.256 sec.
Total JVM GC count: 13.
Total JVM GC time: 0.047 sec.
Heavy hitter instructions:
# Instruction Time(s) Count
1 sp_spoofRATMP1 7,636 500
2 sp_chkpoint 0,847 2
3 sp_write 0,383 1
4 sp_seq 0,184 2
5 sp_rshape 0,035 2
6 createvar 0,015 506
7 mvvar 0,007 500
8 rmvar 0,004 7
9 assignvar 0,000 4
10 cpvar 0,000 4
```
After this change:
```
SystemDS Statistics:
Total elapsed time: 8,618 sec.
Total compilation time: 0,685 sec.
Total execution time: 7,933 sec.
Number of compiled Spark inst: 8.
Number of executed Spark inst: 507.
Cache hits (Mem/Li/WB/FS/HDFS): 1/0/0/0/0.
Cache writes (Li/WB/FS/HDFS): 0/1/0/0.
Cache times (ACQr/m, RLS, EXP): 0,349/0,003/0,004/0,000 sec.
HOP DAGs recompiled (PRED, SB): 0/503.
HOP DAGs recompile time: 0,269 sec.
Functions recompiled: 2.
Functions recompile time: 0,002 sec.
Codegen compile (DAG,CP,JC): 516/502/1.
Codegen enum (ALLt/p,EVALt/p): 516/0/0/0.
Codegen compile times (DAG,JC): 0,332/0,150 sec.
Codegen enum plan cache hits: 0/0.
Codegen op plan cache hits: 501/502.
Spark ctx create time (lazy): 0,686 sec.
Spark trans counts (par,bc,col):1/500/1.
Spark trans times (par,bc,col): 0,000/0,394/0,349 secs.
Total JIT compile time: 11.18 sec.
Total JVM GC count: 13.
Total JVM GC time: 0.047 sec.
Heavy hitter instructions:
# Instruction Time(s) Count
1 sp_spoofRATMP1 6,216 500
2 sp_chkpoint 0,849 2
3 sp_write 0,362 1
4 sp_seq 0,172 2
5 sp_rshape 0,031 2
6 createvar 0,014 506
7 mvvar 0,007 500
8 rmvar 0,004 7
9 assignvar 0,000 4
10 cpvar 0,000 4
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
