[
https://issues.apache.org/jira/browse/SYSTEMML-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542835#comment-16542835
]
LI Guobao edited comment on SYSTEMML-2419 at 7/13/18 10:21 AM:
---------------------------------------------------------------
[~mboehm7], I have some questions. The first is about the setup of remote
parfor worker. In fact, I saw that this block of code is synchronized and so I
wonder if it means that in one executor, the parfor task will be launched in
multi-threaded way? The second is about the codegen class. Aiming to avoid the
concurrently and redundantly loading of the class, what will the codegen class
concretely be? Because when parsing the parfor body, it will generate a codegen
class map.
was (Author: guobao):
[~mboehm7], I have some questions. The first is about the setup of remote
parfor worker. In fact, I saw that this block of code is synchronized and so I
wonder if it means that in one executor, the parfor task will be launched in
multi-threaded way? The second is about the codegen class. Aiming to avoid the
concurrently and redundantly reloading of the class, what will the codegen
class concretely be? Because when parsing the parfor body, it will generate a
codegen class map.
> Setup and cleanup of remote workers
> -----------------------------------
>
> Key: SYSTEMML-2419
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2419
> Project: SystemML
> Issue Type: Sub-task
> Reporter: LI Guobao
> Assignee: LI Guobao
> Priority: Major
>
> In the context of distributed spark env, we need to firstly ship the
> necessary functions and variables to the remote workers and then to
> initialize and register the cleanup of buffer pool for each remote worker.
> All these are inspired by the parfor implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
