Thank you Fred! We are actually doing what you suggested. We were concerned
about the large "null" matrix we are generating at the beginning of the
process. But if large zero matrix is not a performance issue, then I guess we
are on the right track.
Thanks a lot!
Arijit
________________________________
From: Frederick R Reiss <frre...@us.ibm.com>
Sent: Wednesday, May 3, 2017 5:14 AM
To: dev@systemml.incubator.apache.org
Subject: Re: Combining Multiple Matrix
Hi Arijit,
Have you considered using parfor and left indexing? Here's an example from the
docs that generates a large matrix out of smaller chunks:
ms = matrix(0, rows=2, cols=3*10)
parfor (v in 1:10, check=0) {
mv = matrix(v, rows=2, cols=3)
ms[,(v-1)*3+1:v*3] = mv
}
SystemML has a special data structure to represent a matrix block that is all
zeros, so creating a very large matrix of zeros (first line in the above
listing) is an inexpensive operation.
Fred
[Inactive hide details for arijit chakraborty ---05/02/2017 09:48:13 AM---Hi,
Our process is generating multiple matrix, of same]arijit chakraborty
---05/02/2017 09:48:13 AM---Hi, Our process is generating multiple matrix, of
same size, but number of matrix is random. Finally
From: arijit chakraborty <ak...@hotmail.com>
To: "dev@systemml.incubator.apache.org" <dev@systemml.incubator.apache.org>
Date: 05/02/2017 09:48 AM
Subject: Combining Multiple Matrix
________________________________
Hi,
Our process is generating multiple matrix, of same size, but number of matrix
is random. Finally, we combine all the smaller matrix to get a final large
matrix.
The way we are creating the large matrix is, we are creating a blank matrix and
then update the matrix once each smaller matrix is created. This is inefficient
because we are initially creating a large matrix, which sometimes take more
than required space.
So is there any method where we can combined all the final smaller matrix
together into a larger matrix? For rbind and cbind, we have to use loops to
merge the matrix.
Thank you!
Arijit
