(systemds) branch main updated: [SYSTEMDS-3723] Fix matrix-market reader for large symmetric matrices

[mboehm7]

This is an automated email from the ASF dual-hosted git repository.

mboehm7 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/systemds.git


The following commit(s) were added to refs/heads/main by this push:
     new 33f0c9c7ce [SYSTEMDS-3723] Fix matrix-market reader for large 
symmetric matrices
33f0c9c7ce is described below

commit 33f0c9c7ce7d6d52f31dc01fd86d0e6e341decb7
Author: Matthias Boehm <mboe...@gmail.com>
AuthorDate: Wed Aug 21 13:23:48 2024 +0200

    [SYSTEMDS-3723] Fix matrix-market reader for large symmetric matrices
    
    On a symmetric sparse matrix in matrix-market format from the sparse
    suite collection, SystemDS ran into index-out-of-bounds exceptions on
    the cell-buffer before flushing to the target matrix. The check for
    full buffer only checked if one more element fit in, but for symmetric
    matrices two elements are added for every read entry. In the unit tests
    this issue did not show up, because every thread reads fewer values that
    the relatively large buffers.
---
 src/main/java/org/apache/sysds/runtime/io/ReaderTextCellParallel.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git 
a/src/main/java/org/apache/sysds/runtime/io/ReaderTextCellParallel.java 
b/src/main/java/org/apache/sysds/runtime/io/ReaderTextCellParallel.java
index 1888d21a15..b9cfd852fd 100644
--- a/src/main/java/org/apache/sysds/runtime/io/ReaderTextCellParallel.java
+++ b/src/main/java/org/apache/sysds/runtime/io/ReaderTextCellParallel.java
@@ -217,7 +217,8 @@ public class ReaderTextCellParallel extends ReaderTextCell
                                                buff.addCell(cell.getI(), 
cell.getJ(), cell.getV());
                                                if( _mmProps != null && 
_mmProps.isSymmetric() && !cell.onDiag() )
                                                        
buff.addCell(cell.getJ(), cell.getI(), cell.getV());
-                                               if( 
buff.size()>=CellBuffer.CAPACITY ) 
+                                               //flush if needed (<=n-1 to 
allow symmetric mm, where 2 values are added)
+                                               if( 
buff.size()>=CellBuffer.CAPACITY-1 ) 
                                                        synchronized( _dest ){ 
//sparse requires lock
                                                                lnnz += 
buff.size();
                                                                
buff.flushCellBufferToSparseBlock(sblock);