[
https://issues.apache.org/jira/browse/SPARK-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179311#comment-16179311
]
Seth Hendrickson commented on SPARK-17136:
------------------------------------------
Ping [~yanboliang]. This is relevant since we have recently been making
attempts to provide added features/optimizers/algorithms around linear/logistic
regression. This would be a good step toward building interfaces that can be
extended in Spark ML. Could you elaborate on mimicking Spark SQL?
One concern I have is that, under the current proposal, we'd have a parameter
`setMinimizer` that uses a generic Scala class that can't be easily serialized
to Python, etc... It wouldn't be compatible. Maybe we could use reflection like
Spark SQL does, but you'd still have to implement custom optimizers in Scala.
Anyway, I think this, and work related to this, would be really beneficial to
Spark ML.
> Design optimizer interface for ML algorithms
> --------------------------------------------
>
> Key: SPARK-17136
> URL: https://issues.apache.org/jira/browse/SPARK-17136
> Project: Spark
> Issue Type: Sub-task
> Components: ML
> Reporter: Seth Hendrickson
>
> We should consider designing an interface that allows users to use their own
> optimizers in some of the ML algorithms, similar to MLlib.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
