[
https://issues.apache.org/jira/browse/SYSTEMML-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509523#comment-16509523
]
LI Guobao commented on SYSTEMML-2382:
-------------------------------------
Well,
1) input data: it is the data generated from the infinite MNIST dataset. And my
data size is 60k.
2) script: src/test/scripts/functions/paramserv/paramserv-nn-asp-batch.dml
3) the parameters are: mode="LOCAL", utype="ASP", freq="BATCH", epochs=10,
batchsize=16, k=2, scheme="DISJOINT_CONTIGUOUS", hyperparams=params,
checkpointing="NONE"
> Concurrent problem about LibMatrixMult
> --------------------------------------
>
> Key: SYSTEMML-2382
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2382
> Project: SystemML
> Issue Type: Sub-task
> Reporter: LI Guobao
> Assignee: LI Guobao
> Priority: Major
> Fix For: SystemML 1.2
>
>
> When the worker number is set to 2 and each worker has 2 threads to launch
> its instruction, in the case of small batchsize, the program fails with a
> concurrent error. But when I set only 1 thread to each worker, everything
> works fine. By the way, the number of vcore is 4 on my machine.
> Here is the stack:
> {code:java}
> Caused by: org.apache.sysml.runtime.DMLRuntimeException:
> java.util.concurrent.ExecutionException:
> java.lang.ArrayIndexOutOfBoundsException: 3
> at
> org.apache.sysml.runtime.matrix.data.LibMatrixMult.matrixMult(LibMatrixMult.java:221)
> at
> org.apache.sysml.runtime.matrix.data.MatrixBlock.aggregateBinaryOperations(MatrixBlock.java:4796)
> at
> org.apache.sysml.runtime.instructions.cp.AggregateBinaryCPInstruction.processInstruction(AggregateBinaryCPInstruction.java:73)
> at
> org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:252)
> ... 13 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.ArrayIndexOutOfBoundsException: 3
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.sysml.runtime.matrix.data.LibMatrixMult.matrixMult(LibMatrixMult.java:213)
> ... 16 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 3
> at
> org.apache.sysml.runtime.matrix.data.LibMatrixMult.matrixMultSparseDenseMMShortLHS(LibMatrixMult.java:1305)
> at
> org.apache.sysml.runtime.matrix.data.LibMatrixMult.matrixMultSparseDense(LibMatrixMult.java:1194)
> at
> org.apache.sysml.runtime.matrix.data.LibMatrixMult.access$300(LibMatrixMult.java:57)
> at
> org.apache.sysml.runtime.matrix.data.LibMatrixMult$MatrixMultTask.call(LibMatrixMult.java:3921)
> ... 5 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
