[
https://issues.apache.org/jira/browse/SYSTEMML-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Boehm closed SYSTEMML-2152.
------------------------------------
Resolution: Fixed
Assignee: Matthias Boehm
Fix Version/s: SystemML 1.1
> Spark mapmm fails for broadcast with larger than integer dims
> -------------------------------------------------------------
>
> Key: SYSTEMML-2152
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2152
> Project: SystemML
> Issue Type: Bug
> Reporter: Matthias Boehm
> Assignee: Matthias Boehm
> Priority: Major
> Fix For: SystemML 1.1
>
>
> {code}
> at
> org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext.getBroadcastForVariable(SparkExecutionContext.java:545)
> at
> org.apache.sysml.runtime.instructions.spark.MapmmSPInstruction.processInstruction(MapmmSPInstruction.java:128)
> at
> org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:264)
> ... 17 more
> Caused by: java.io.IOException: org.apache.sysml.runtime.DMLRuntimeException:
> Matrix dimensions too large for CP runtime: 514 x 5129281161
> at
> org.apache.sysml.runtime.util.DataConverter.readMatrixFromHDFS(DataConverter.java:211)
> at
> org.apache.sysml.runtime.util.DataConverter.readMatrixFromHDFS(DataConverter.java:171)
> at
> org.apache.sysml.runtime.controlprogram.caching.MatrixObject.readBlobFromHDFS(MatrixObject.java:434)
> at
> org.apache.sysml.runtime.controlprogram.caching.MatrixObject.readBlobFromHDFS(MatrixObject.java:59)
> at
> org.apache.sysml.runtime.controlprogram.caching.CacheableData.readBlobFromHDFS(CacheableData.java:981)
> at
> org.apache.sysml.runtime.controlprogram.caching.CacheableData.acquireRead(CacheableData.java:417)
> ... 20 more
> Caused by: org.apache.sysml.runtime.DMLRuntimeException: Matrix dimensions
> too large for CP runtime: 514 x 5129281161
> at
> org.apache.sysml.runtime.io.MatrixReader.createOutputMatrixBlock(MatrixReader.java:80)
> at
> org.apache.sysml.runtime.io.ReaderBinaryBlockParallel.readMatrixFromHDFS(ReaderBinaryBlockParallel.java:59)
> at
> org.apache.sysml.runtime.util.DataConverter.readMatrixFromHDFS(DataConverter.java:207)
> ... 25 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
