Yi Liu created SPARK-10041:
------------------------------
Summary: Proposal of Parameter Server Interface for Spark
Key: SPARK-10041
URL: https://issues.apache.org/jira/browse/SPARK-10041
Project: Spark
Issue Type: New Feature
Components: ML
Reporter: Yi Liu
Many large-scale machine learning algorithms (logistic regression, LDA, neural
network, etc.) have been built on top of Apache Spark. As discussed in
SPARK-4590, a Parameter Server (PS) architecture can greatly improve the
scalability and efficiency for these large-scale machine learning. There are
some previous discussions on possible Parameter Server implementations inside
Spark (e.g., SPARK-6932). However, at this stage we believe it is more
important for the community to first define the proper interface of Parameter
Server, which can be decoupled from the actual PS implementations;
consequently, it is possible to support different implementations of Parameter
Servers in Spark later. The attached document contains our initial proposal of
Parameter Server interface for ML algorithms on Spark, including data model,
supported operations, epoch support and possible Spark integrations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
