[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-06-05 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/17094 Sorry for not replying - yeah agree let's create a set of follow ups to this for potential improvements, and of course the migration of other models to the framework Thanks @sethah!

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-06-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17094 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-06-03 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17094 @srowen Speaking for myself, I think the other concerns can be issued as follow ups, yes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-06-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17094 @sethah @MLnick am I reading right that this can be merged as a step forward? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-31 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17094 Ok, yes all good points. I think since these are all private apis it gives us room for future changes. For now, I think we can get rid of a lot of code duplication and fill in some testing gaps with

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-31 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/17094 But even for the standard-scaling - it seems that could be expressed generically too with respect to scaling the coeff and gradient during the computation. Again, something perhaps for later. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-31 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/17094 Sure, makes sense. We can always consider it later. Or even an alternate version of it to have `L2` and a subclass `StandardizedL2` or whatever (that's more if we were to start thinking about

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-30 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17094 @MLnick I completely agree about the leaky regularization abstraction. In fact, I think the function composition feature would make it easy to get rid of that problem. Consider: In the

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-30 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/17094 Overall looks good to me. I think it's a good step to clean up the codebase and reduce the duplicated code. I think the impl is pretty well thought through. A few comments (that probably

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-30 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17094 Merging tomorrow if there are no objections. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77376/ Test PASSed. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #77376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77376/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #77376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77376/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77360/ Test PASSed. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #77360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77360/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17094 cc @srowen also --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #77360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77360/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-18 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17094 Thanks @MLnick! I am happy to discuss splitting this into smaller bits as well, if it can make things easier. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-18 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/17094 In terms of the high level intention of this, agree we definitely need it and it should clean things up substantially. I will start taking a look through ASAP. Thanks! --- If your project is set

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77034/ Test PASSed. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #77034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77034/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #77034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77034/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77033/ Test FAILed. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #77033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77033/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #77033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77033/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-17 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17094 ping! @MLnick @jkbradley @yanboliang @hhbyyh Is there any interest in this? I actually think this cleanup will be a precursor to several different improvements (adding more optimized

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73914/ Test PASSed. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #73914 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73914/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #73914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73914/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-04 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17094 Jenkins test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73819/ Test FAILed. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73821/ Test FAILed. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73823/ Test FAILed. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73820/ Test FAILed. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #73823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73823/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-02 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17094 Removed WIP, think it's ready now :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #73821 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73821/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #73820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73820/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #73819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73819/testReport)** for PR 17094 at commit