[
https://issues.apache.org/jira/browse/SYSTEMML-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518195#comment-16518195
]
LI Guobao edited comment on SYSTEMML-2413 at 6/20/18 2:26 PM:
--------------------------------------------------------------
[~mboehm7] , in fact, I found that maybe this expensive dag recompilation is
introduced by a coding fault in workers. It is that I wrongly used always the
same original function block to generate the instruction. But we should use the
recompiled new function block. So to be more clear, I remove the function
blocks which are not related to update function or agg function. Additionnally,
I could observe in the code that the dags have been already deep copied when
copying the function because they need the recompilation.
was (Author: guobao):
[~mboehm7] , in fact, I found that maybe this expensive dag recompilation is
introduced by a code fault in workers. It is that I wrongly used always the
same original function block to generate the instruction. But we should use the
recompiled new function block. So to be more clear, I remove the function
blocks which are not related to update function or agg function. Additionnally,
I could observe in the code that the dags have been already deep copied when
copying the function because they need the recompilation.
> Paramserv performance bottleneck on the DAG recompilation
> ---------------------------------------------------------
>
> Key: SYSTEMML-2413
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2413
> Project: SystemML
> Issue Type: Bug
> Reporter: LI Guobao
> Assignee: LI Guobao
> Priority: Major
>
> We could observe that the DAG recompilation of each experiment is expensive.
> Hence, we could improve this bottleneck by deep copying the dags into each
> workers' execution context.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
