[
https://issues.apache.org/jira/browse/FLINK-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193130#comment-15193130
]
Theodore Vasiloudis commented on FLINK-1932:
--------------------------------------------
[~spkavulya] The main reason we didn't create a PR for this was the fact that
we I am using a call to .collect() to get the updated weight vector at each
iteration,
which can be very slow with the the way Flink works currently.
There might be a way to implement this without the collect, but I wasn't able
to figure something out.
In any case, one thing I would like us to do is compare this implementation
with the Spark one in terms of performance in a cluster.
If we are much slower than Spark, then I wouldn't recommend merging this
algorithm, until we get the performance right.
> Add L-BFGS to the optimization framework
> ----------------------------------------
>
> Key: FLINK-1932
> URL: https://issues.apache.org/jira/browse/FLINK-1932
> Project: Flink
> Issue Type: New Feature
> Components: Machine Learning Library
> Reporter: Till Rohrmann
> Assignee: Theodore Vasiloudis
> Labels: ML
>
> A good candidate to add to the new optimization framework could be L-BFGS [1,
> 2].
> Resources:
> [1] http://papers.nips.cc/paper/5333-large-scale-l-bfgs-using-mapreduce.pdf
> [2] http://en.wikipedia.org/wiki/Limited-memory_BFGS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
