GitHub user mengxr opened a pull request:
https://github.com/apache/spark/pull/1014
[SPARK-1672] Separate user and product partitioning in ALS
Some clean up work following #593.
1. Allow to set different number user blocks and number product blocks in
`ALS`.
2. Update `MovieLensALS` to reflect the change.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mengxr/spark SPARK-1672
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1014.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1014
----
commit c774d7d4bff91c9387d059d1189799fa0ff1f4b0
Author: Tor Myklebust <[email protected]>
Date: 2014-04-14T22:01:18Z
Make the partitioner a member variable and use it instead of modding
directly.
commit c90b6d8e91f86cf89adf28de6f9185647c87e5c8
Author: Tor Myklebust <[email protected]>
Date: 2014-04-14T22:10:30Z
Scramble user and product ids before bucketing.
commit df27697649de50d364c42c76aebaebb34cbe87e2
Author: Tor Myklebust <[email protected]>
Date: 2014-04-15T19:47:17Z
Support custom partitioners. Currently we use the same partitioner for
users and products.
commit d872b098d41c4fc088e579e8fe199aca149bca64
Author: Tor Myklebust <[email protected]>
Date: 2014-04-16T12:19:48Z
Add negative id ALS test.
commit 36a0f43519a1e8ea800b960157f8c8b050139105
Author: Tor Myklebust <[email protected]>
Date: 2014-04-16T16:42:31Z
Make the partitioner private.
commit 5ec9e6cd237c4ac7c1b597614c880ae75bacceee
Author: Tor Myklebust <[email protected]>
Date: 2014-04-16T17:00:39Z
Clean a couple of things up using 'map'.
commit f8413451c807282100a9be506ca2c992abb81918
Author: Tor Myklebust <[email protected]>
Date: 2014-04-16T17:12:47Z
Fix daft bug creating 'pairs', also for -> foreach.
commit 40edc235e59aab56d6f65c73ffe98859c78a889b
Author: Tor Myklebust <[email protected]>
Date: 2014-04-16T18:14:38Z
Fix missing space.
commit 674933abb7a373dc1c913467d668bad9045e560f
Author: Tor Myklebust <[email protected]>
Date: 2014-04-19T23:36:52Z
Fix style.
commit 495784f2a172957ab490e0f77ea504c0179ab798
Author: Tor Myklebust <[email protected]>
Date: 2014-04-19T23:41:23Z
Merge branch 'master' of https://github.com/apache/spark
commit 23d6f91b52c88b7006ec78496f777b72b1881bb4
Author: Tor Myklebust <[email protected]>
Date: 2014-04-21T00:06:19Z
Stop making the partitioner configurable.
commit dcf583ac4001c6da8d6b85e45e88043861a351d8
Author: Tor Myklebust <[email protected]>
Date: 2014-04-21T19:56:48Z
Remove the partitioner member variable; instead, thread that needle
everywhere it needs to go.
commit 021f54b5d3d5177a5185442aaf3ddcde51a354a2
Author: Tor Myklebust <[email protected]>
Date: 2014-04-29T05:47:41Z
Separate user and product blocks.
commit bd8a75c53e1b84b4dfb493581bc1945eb93d624b
Author: Tor Myklebust <[email protected]>
Date: 2014-04-29T23:21:26Z
Merge branch 'master' of github.com:apache/spark into alsseppar
Conflicts:
mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala
commit a4925fdd2bb9b648829120c834bea9e8f05803a3
Author: Tor Myklebust <[email protected]>
Date: 2014-04-30T00:36:21Z
Style.
commit d17a8bff323bb69dcc5050ee1623063193d12084
Author: Xiangrui Meng <[email protected]>
Date: 2014-06-04T19:20:02Z
merge master
commit 84c8e8cef1a55afe0e63cd6c10f0f71dbce32529
Author: Xiangrui Meng <[email protected]>
Date: 2014-06-08T18:13:41Z
Merge branch 'master' into SPARK-1672
commit 9bab77b7336792c84adfeb1813206c7d8e7bc765
Author: Xiangrui Meng <[email protected]>
Date: 2014-06-09T03:50:05Z
clean up
add numUserBlocks and numProductBlocks to MovieLensALS
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
