Xiangrui Meng created SPARK-4590:
------------------------------------
Summary: Early investigation of parameter server
Key: SPARK-4590
URL: https://issues.apache.org/jira/browse/SPARK-4590
Project: Spark
Issue Type: Brainstorming
Components: ML, MLlib
Reporter: Xiangrui Meng
Assignee: Reza Zadeh
In the currently implementation of GLM solvers, we save intermediate models on
the driver node and update it through broadcast and aggregation. Even with
torrent broadcast and tree aggregation added in 1.1, it is hard to go beyond
~10 million features. This JIRA is for investigating the parameter server
approach, including algorithm, infrastructure, and dependencies.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
