[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-13 Thread hhbyyh
Github user hhbyyh commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-171499664 @thunterdb Thanks a lot for the recommendation. I'll try with it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10146 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-13 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-171498026 LGTM Merging with master and branch-1.6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-12 Thread thunterdb
Github user thunterdb commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-170949938 @hhbyyh yes, option 3 sounds good. A caveat, though, about the numbers you posted: micro benchmarks on the JVM are very hard to get right, and a simple loop i

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-17010 **[Test build #2365 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2365/consoleFull)** for PR 10146 at commit [`a709f49`](https://

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-170770224 **[Test build #2365 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2365/consoleFull)** for PR 10146 at commit [`a709f49`](https://g

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-11 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-170770236 @hhbyyh Thanks for doing that test. Let's go with option 3 as you suggested. --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-11 Thread hhbyyh
Github user hhbyyh commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-170765819 Failed to fetch from https://github.com/apache/spark.git. need a retest --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-170762436 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-170762432 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-11 Thread hhbyyh
Github user hhbyyh commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-170760561 @jkbradley I changed it to invoking toBreeze directly. I ran some test on local. val sv = new SparseVector(10, Array(1), Array(2.5)) var t = 0.0

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/10146#discussion_r49408306 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/test/ChiSqTest.scala --- @@ -109,7 +109,9 @@ private[stat] object ChiSqTest extends Logging {

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/10146#discussion_r49392781 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/test/ChiSqTest.scala --- @@ -109,7 +109,9 @@ private[stat] object ChiSqTest extends Logging {

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/10146#discussion_r49298652 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/test/ChiSqTest.scala --- @@ -109,7 +109,9 @@ private[stat] object ChiSqTest extends Logging {

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-08 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/10146#discussion_r49235733 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/test/ChiSqTest.scala --- @@ -109,7 +109,9 @@ private[stat] object ChiSqTest extends Logging {

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-04 Thread thunterdb
Github user thunterdb commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-168867750 @hhbyyh thanks for the fix; I just have one small comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2016-01-04 Thread thunterdb
Github user thunterdb commented on a diff in the pull request: https://github.com/apache/spark/pull/10146#discussion_r48805286 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/test/ChiSqTest.scala --- @@ -109,7 +109,9 @@ private[stat] object ChiSqTest extends Logging {

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-07 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-162569175 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-162476627 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-162476626 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-162476502 **[Test build #47263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47263/consoleFull)** for PR 10146 at commit [`8d8327d`](https://g

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-162465080 **[Test build #47263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47263/consoleFull)** for PR 10146 at commit [`8d8327d`](https://gi

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-04 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/10146#discussion_r46748950 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/test/ChiSqTest.scala --- @@ -109,9 +109,10 @@ private[stat] object ChiSqTest extends Logging {

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-04 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/10146#discussion_r46708243 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/test/ChiSqTest.scala --- @@ -109,9 +109,10 @@ private[stat] object ChiSqTest extends Logging {

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-162000277 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-162000278 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-16285 **[Test build #47199 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47199/consoleFull)** for PR 10146 at commit [`1b1a0c6`](https://g

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10146#issuecomment-161987769 **[Test build #47199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47199/consoleFull)** for PR 10146 at commit [`1b1a0c6`](https://gi

[GitHub] spark pull request: [SPARK-12026] [MLlib] ChiSqTest gets slower an...

2015-12-04 Thread hhbyyh
GitHub user hhbyyh opened a pull request: https://github.com/apache/spark/pull/10146 [SPARK-12026] [MLlib] ChiSqTest gets slower and slower over time when number of features is large jira: https://issues.apache.org/jira/browse/SPARK-12026 The issue is valid as features.toAr