Matthias Boehm created SYSTEMML-2029:
----------------------------------------
Summary: Perftest L2SVM icpt 1 sparse 800GB w/ codegen failing
Key: SYSTEMML-2029
URL: https://issues.apache.org/jira/browse/SYSTEMML-2029
Project: SystemML
Issue Type: Bug
Reporter: Matthias Boehm
{code}
Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error
in program block generated from statement block between lines 141 and 161 --
Error evaluating instruction:
CP°ba+*°_mVar169·MATRIX·DOUBLE°_mVar170·MATRIX·DOUBLE°_mVar176·MATRIX·DOUBLE°16
at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:294)
at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:218)
at
org.apache.sysml.runtime.controlprogram.ProgramBlock.execute(ProgramBlock.java:163)
at
org.apache.sysml.runtime.controlprogram.WhileProgramBlock.execute(WhileProgramBlock.java:115)
... 14 more
Caused by: org.apache.sysml.runtime.controlprogram.caching.CacheException:
Reading of scratch_space//_p2037254_9.1.44.28//_t0/temp66_56 (656476731) failed.
at
org.apache.sysml.runtime.controlprogram.caching.CacheableData.acquireRead(CacheableData.java:436)
at
org.apache.sysml.runtime.controlprogram.context.ExecutionContext.getMatrixInput(ExecutionContext.java:274)
at
org.apache.sysml.runtime.controlprogram.context.ExecutionContext.getMatrixInput(ExecutionContext.java:252)
at
org.apache.sysml.runtime.instructions.cp.AggregateBinaryCPInstruction.processInstruction(AggregateBinaryCPInstruction.java:71)
at
org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:264)
... 17 more
Caused by: java.io.IOException: Failed parallel read of binary block input.
at
org.apache.sysml.runtime.io.ReaderBinaryBlockParallel.readBinaryBlockMatrixFromHDFS(ReaderBinaryBlockParallel.java:113)
at
org.apache.sysml.runtime.io.ReaderBinaryBlockParallel.readMatrixFromHDFS(ReaderBinaryBlockParallel.java:70)
at
org.apache.sysml.runtime.util.DataConverter.readMatrixFromHDFS(DataConverter.java:204)
at
org.apache.sysml.runtime.util.DataConverter.readMatrixFromHDFS(DataConverter.java:168)
at
org.apache.sysml.runtime.controlprogram.caching.MatrixObject.readBlobFromHDFS(MatrixObject.java:434)
at
org.apache.sysml.runtime.controlprogram.caching.MatrixObject.readBlobFromHDFS(MatrixObject.java:59)
at
org.apache.sysml.runtime.controlprogram.caching.CacheableData.readBlobFromHDFS(CacheableData.java:977)
at
org.apache.sysml.runtime.controlprogram.caching.MatrixObject.readBlobFromRDD(MatrixObject.java:491)
at
org.apache.sysml.runtime.controlprogram.caching.MatrixObject.readBlobFromRDD(MatrixObject.java:59)
at
org.apache.sysml.runtime.controlprogram.caching.CacheableData.acquireRead(CacheableData.java:426)
... 21 more
Caused by: java.util.concurrent.ExecutionException:
java.lang.ArrayIndexOutOfBoundsException: 46
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.sysml.runtime.io.ReaderBinaryBlockParallel.readBinaryBlockMatrixFromHDFS(ReaderBinaryBlockParallel.java:103)
... 30 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 46
at
org.apache.sysml.runtime.matrix.data.SparseRowVector.append(SparseRowVector.java:197)
at
org.apache.sysml.runtime.matrix.data.SparseBlockMCSR.append(SparseBlockMCSR.java:264)
at
org.apache.sysml.runtime.matrix.data.MatrixBlock.appendToSparse(MatrixBlock.java:758)
at
org.apache.sysml.runtime.matrix.data.MatrixBlock.appendToSparse(MatrixBlock.java:721)
at
org.apache.sysml.runtime.io.ReaderBinaryBlockParallel$ReadFileTask.call(ReaderBinaryBlockParallel.java:183)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
