[
https://issues.apache.org/jira/browse/SYSTEMML-2349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495685#comment-16495685
]
Matthias Boehm commented on SYSTEMML-2349:
------------------------------------------
Well, since both workers and aggregator execute user-defined functions we need
to make the threads robust enough the handle errors and ensure termination. One
approach to do that is to use a callback as follows: (1) keep the threads as
members in the invoking instruction, (2) pass the instruction as an argument
into the workers and aggregator, (3) wrap the invocation of user-defined
functions into try-catch, and (4) call a proper shutdown method of the
instruction (with access to the thread members) from the respective catch
clauses.
Also, once you reworked the queuing logic, we might be able to avoid running
the aggregator as a thread. I could imagine a design where we simply call
execute on the parameter server, and this parameter server internally spawns
its workers and simply returns whenever all workers are done or an error
occurred.
> Local worker error handling
> ---------------------------
>
> Key: SYSTEMML-2349
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2349
> Project: SystemML
> Issue Type: Sub-task
> Reporter: LI Guobao
> Assignee: LI Guobao
> Priority: Major
>
> While playing around with the locking scheme of the parameter server, I
> encountered unrelated errors that led to the parameter server hanging. We
> need to make sure all worker errors are correctly propagated so that we can
> guarantee termination.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
