GitHub user thomaspeel opened a pull request:
https://github.com/apache/flink/pull/1102
[ML] : Lasso regression with distributed Frank-Wolfe under SSP with
parameter server.
Implementation of the Distributed Frank-Wolfe optimization algorithm to
solve a Lasso regresssion problem under SSP with a parameter server.
This pull request is based on #967 for the SSP implementation. It uses
Apache Ignite as backend for the parameter server.
Distributed Frank-Wolfe is discussed here : http://arxiv.org/abs/1404.2644
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/thomaspeel/flink DFWSSP
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1102.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1102
----
commit 0435bb9114f48aeefb330e7cf92610be84f79c81
Author: Nam-Luc Tran <[email protected]>
Date: 2015-07-31T15:21:08Z
* Added the model for parameter server and parameter element
* Added parameter server implementation based on Apache Ignite
* Started instance of parameter server in TaskManager
* Added Apache Ignite as dependency in pom.xml
commit decc66db856d3d640e7e29128e2696c0941091dd
Author: Nam-Luc Tran <[email protected]>
Date: 2015-07-31T15:24:32Z
* Extended RichMapFunction with methods to access parameter server
commit 55398da553ce697af9bea881ea7131818edb82d2
Author: Nam-Luc Tran <[email protected]>
Date: 2015-07-31T15:25:08Z
* Extended DataSet API to enable SSP and configuration
commit 6ef80ceac5e6ef897769e1883823d152d0c04070
Author: Nam-Luc Tran <[email protected]>
Date: 2015-07-31T15:25:55Z
* Extended ExecutionEnvironment and ExecutionConfig to enable SSP and SPP
configuration
commit 368ca1c101034aefa8b6ac0ce4791133976831e0
Author: Nam-Luc Tran <[email protected]>
Date: 2015-07-31T15:29:29Z
* Added drop-in control structures for Stale Synchronous Parallel iterations
commit 23fb6518fe7042ad859bb9474266131f2bdb669c
Author: Nam-Luc Tran <[email protected]>
Date: 2015-07-31T15:30:31Z
* Added the events used by the control structures for Stale Synchronous
Parallel iterations
commit 48810bde7783610112751f2221126b69a1ac9b56
Author: Nam-Luc Tran <[email protected]>
Date: 2015-07-31T15:32:02Z
* Extended the job translation to take into account the control structures
related to Stale Synchronous Parallel Iterations
commit 772bfb62bf537fb4c6aa67a2c6e9899278b01450
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-03T08:35:14Z
* Fixed wrong reference in logger
* in reference to travis build error
commit 2caaebf805d1f1d020297eb9057dadbeb44d1b33
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-05T08:49:41Z
* Isolated parameter server from the core code
* Removed core dependency to Apache Ignite
commit a21e95c7902105931190544913d9622909d544f4
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-05T13:36:36Z
* RichMapFunctionWithParameterServer now start their own
instance at the first superstep
* Improved parameter server instantiation and cleanup code
* Remaining issue: how to cleanly invoke the parameter server
shutdown code at the end of the iterations or job
commit 2eebccf36bc9588e148d8d765a22871919abf353
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-05T13:55:57Z
* Removed parameter server startup and shutdown from TaskManager
commit a340d1aad6f1523b036949f21e111f5f02a9f595
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-05T14:13:32Z
* cleanup merge conflicts leftovers
* cleaned imports
* cleaner everything
commit 9b5c677b0b0ef900796a87c64562c3bcdd6f5b07
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-05T14:20:35Z
Merge branch 'master' of https://github.com/apache/flink into SSP
Conflicts:
flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala
commit 351614dd21e3d03c25bc01f65b29a5b5bb031463
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-11T15:40:09Z
Removed explicit copyright in header
commit 6e483e6e639aebf6dde47ccacecf06571c5fc18e
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-11T15:40:43Z
Fixed merge issues
commit df3f0c32e53641f455aa3193d22d6681747443fc
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-11T15:41:01Z
Merge leftovers
commit 8f15b81e0d224b5b4c05ab78c9175b17438890a2
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-11T15:53:22Z
Copyright leftover
commit d5c87244c38402a7ee8c166ed5629ecb73637b02
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-12T08:09:14Z
Fixed license
commit f8b870ec06726866f2fe57a27d897b34f58977c2
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-12T08:09:30Z
Fixed scala style
commit c85bf72f060bfb9b1a5dd2bf2139185c10008eb1
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-12T11:26:38Z
Set default iteration strategy to BulkIterationBase in the it is
created directly and not translated
* Should fix test in IterativeKMeansTest
commit e56465e65c5cfc9ec417fb17339a6f19d9edd6d4
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-13T09:29:22Z
Put slack parameter in TaskConfig instead of job-wide config
* Removed slack from ExecutionConfig
* Added slack to TaskConfig and head config
commit a5c49e702c8a3da8b9b9730caaa8bdbbba74c349
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-13T10:01:57Z
Changed default slack to 0 instead of -1 for coherence
commit ec23273aa3481faa867ef0f990d6873ecef27558
Author: Nam-Luc Tran <[email protected]>
Date: 2015-08-13T11:10:10Z
Fixed scala style
commit 2ce574f890184de9d9dfa1b57f2487683f37194c
Author: Thomas Peel <[email protected]>
Date: 2015-09-07T15:15:47Z
Implementation of the Distributed Frank-Wolfe optimization algorithm to
solve a Lasso regresssion problem under SSP with a parameter server.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
