[ https://issues.apache.org/jira/browse/SYSTEMML-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Boehm closed SYSTEMML-1879. ------------------------------------ > Parfor remote spark w/ reuse of shared inputs > --------------------------------------------- > > Key: SYSTEMML-1879 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1879 > Project: SystemML > Issue Type: Sub-task > Components: APIs, Runtime > Reporter: Matthias Boehm > Assignee: Matthias Boehm > Fix For: SystemML 1.0 > > > Currently, we read shared inputs redundantly in each parfor worker. This > causes redundant read and is unnecessarily memory-inefficient. > This task aims to read shared inputs once per process and reuse them across > threads. The most elegant way of handling this is to reuse initially parsed > symbol table entries (instances of matrix objects), except for result > variables. Then the sharing happens automatically (similar to local parfor) > over the shared per-process buffer pool. -- This message was sent by Atlassian JIRA (v6.4.14#64029)