[ 
https://issues.apache.org/jira/browse/SYSTEMML-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150232#comment-16150232
 ] 

Matthias Boehm commented on SYSTEMML-1879:
------------------------------------------

FYI [~Tenma] - this will also help for the deep learning scenarios (weights are 
just read once per process not once per core, and probably less GC overhead).

> Parfor remote spark w/ reuse of shared inputs
> ---------------------------------------------
>
>                 Key: SYSTEMML-1879
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1879
>             Project: SystemML
>          Issue Type: Sub-task
>          Components: APIs, Runtime
>            Reporter: Matthias Boehm
>             Fix For: SystemML 1.0
>
>
> Currently, we read shared inputs redundantly in each parfor worker. This 
> causes redundant read and is unnecessarily memory-inefficient.
> This task aims to read shared inputs once per process and reuse them across 
> threads. The most elegant way of handling this is to reuse initially parsed 
> symbol table entries (instances of matrix objects), except for result 
> variables. Then the result happens automatically over the shared per-process 
> buffer pool. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to