[
https://issues.apache.org/jira/browse/SYSTEMML-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483199#comment-16483199
]
Matthias Boehm commented on SYSTEMML-2336:
------------------------------------------
sure - here we go:
* TransientRead: Reads an existing data object (e.g., matrix or scalar) by name
from the symbol table
* TransientWrite: Puts a data object by name into the symbol table.
* PersistentRead: Reads a data object from persistent storage (e.g., hdfs, s3,
or local file system)
* PersistentWrite: Writes a data object to persistent storage
So, to implement your data partitioning, you could create an empty symbol table
({{LocalVariableMap}}), put the matrix object X into it, and create a
{{TransientRead}}, the necessary operations, and a {{TransientWrite}} to a
temporary name. After generating and executing the result instructions, you can
finally obtain the partition from the symbol table.
> Data partition
> --------------
>
> Key: SYSTEMML-2336
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2336
> Project: SystemML
> Issue Type: Sub-task
> Reporter: LI Guobao
> Assignee: LI Guobao
> Priority: Major
>
> In the initial version of paramserv function, a simple scheme
> disjoint_contiguous will be implemented.
> * Disjoint_Contiguous: for each worker, use a right indexing operation
> {{X[beg:end,]}} to obtain contiguous, non-overlapping partitions of rows.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
