Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Thanks a lot @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22784
Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98178/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98178 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98178/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98178 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98178/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98164/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98164 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98164/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98161/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98161 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98161/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98164 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98164/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98161 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98161/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98145 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98145/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98145/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98144/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98144 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98144/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98145 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98145/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98144 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98144/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98141/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98141 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98141/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98140 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98140/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98140/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98141 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98141/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98140 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98140/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98135/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98135 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98135/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98134/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98134 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98134/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98135 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98135/testReport)**
for PR 22784 at commit
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Thank you @srowen for the review. I have addressed the comments.
> I wonder if the SVD should be used at even smaller scales? as you point
out, it's pretty hard to compute a gramian on
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #98134 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98134/testReport)**
for PR 22784 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Sorry for my mistake. My keyboard '4' sometimes has a trouble.
> I think, INT_MAX is 2147483647, so n ~= sqrt(2*2147483647) = 65536.
---
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Hi @srowen , Thanks for the comment. As per my knowledge, PCA/SVD is not
limited on row size.
1) Currently row size is not a constraint. Ultimately we need to compute
graminan
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22784
Hm, as a general comment, is this going to scale? This is making a
potentially huge sparse data set dense, and computing a PCA via SVD. I get the
idea that it's better to have some option than none,
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Hi @kiszk Maximum it can go upto the following limit.
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
One question: After this PR, what is the maximum column that we can accept?
---
-
To unsubscribe, e-mail:
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Hi @kiszk ,
I think, INT_MAX is 2147483647, so n ~= sqrt(2*2147483647) = 65536.
Thanks
---
-
To unsubscribe, e-mail:
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
Can I clarify the description?
> Because we are passing an array of size n*(n+1)/2 to the breeze library
and the size cannot be more than INT_MAX. so, the maximum column size we can
give is
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97683/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97683 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97683/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97683 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97683/testReport)**
for PR 22784 at commit
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
All the UTs are passing locally. Seems random error.
retest this please.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97682/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97682 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97682/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97682 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97682/testReport)**
for PR 22784 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22784
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97680/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97680 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97680/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97679/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97679 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97679/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97680 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97680/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97677/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97677 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97677/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97679 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97679/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97678/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97678 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97678/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97678 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97678/testReport)**
for PR 22784 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97677 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97677/testReport)**
for PR 22784 at commit
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97673/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97673 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97673/testReport)**
for PR 22784 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22784
**[Test build #97673 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97673/testReport)**
for PR 22784 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22784
ok to test.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
cc @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Test results with existing PCA and using SVD without computing covariance
matrix
val data = Array(
Vectors.sparse(5, Seq((1, 1.0), (3, 7.0))),
Vectors.dense(2.0, 0.0, 3.0,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22784
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
95 matches
Mail list logo
