[
https://issues.apache.org/jira/browse/SPARK-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699903#comment-14699903
]
Jason Dai commented on SPARK-10041:
-----------------------------------
I think this one is different from SPARK-6932 as it tries to decouple the
interface design from the actual implementation; a well defined
interface/abstraction that allows different underlying implementations can be
very useful.
> Proposal of Parameter Server Interface for Spark
> ------------------------------------------------
>
> Key: SPARK-10041
> URL: https://issues.apache.org/jira/browse/SPARK-10041
> Project: Spark
> Issue Type: New Feature
> Components: ML, MLlib
> Reporter: Yi Liu
> Attachments: Proposal of Parameter Server Interface for Spark - v1.pdf
>
>
> Many large-scale machine learning algorithms (logistic regression, LDA,
> neural network, etc.) have been built on top of Apache Spark. As discussed in
> SPARK-4590, a Parameter Server (PS) architecture can greatly improve the
> scalability and efficiency for these large-scale machine learning. There are
> some previous discussions on possible Parameter Server implementations inside
> Spark (e.g., SPARK-6932). However, at this stage we believe it is more
> important for the community to first define the proper interface of Parameter
> Server, which can be decoupled from the actual PS implementations;
> consequently, it is possible to support different implementations of
> Parameter Servers in Spark later. The attached document contains our initial
> proposal of Parameter Server interface for ML algorithms on Spark, including
> data model, supported operations, epoch support and possible Spark
> integrations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
