Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/11553
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208503247
Merged into master. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208490664
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208490669
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208490419
**[Test build #55531 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55531/consoleFull)**
for PR 11553 at commit
[`a5ccc0e`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208478598
**[Test build #55531 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55531/consoleFull)**
for PR 11553 at commit
[`a5ccc0e`](https://gi
Github user oliverpierson commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208477524
@mengxr I've rebased. Looks like it's merging cleanly now. Let me know if
there's anything else. Thanks
---
If your project is set up for it, you can reply to
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208438387
Build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208438391
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208438376
**[Test build #55526 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55526/consoleFull)**
for PR 11553 at commit
[`c5e9c2c`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208436807
**[Test build #55526 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55526/consoleFull)**
for PR 11553 at commit
[`c5e9c2c`](https://gi
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208436301
@oliverpierson The changes LGTM2 but #12274 caused merge conflicts. Could
you rebase your PR? Thanks!
---
If your project is set up for it, you can reply to this email
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208435676
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this featu
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208406270
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208406266
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208406031
**[Test build #55525 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55525/consoleFull)**
for PR 11553 at commit
[`c5e9c2c`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208388834
**[Test build #55525 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55525/consoleFull)**
for PR 11553 at commit
[`c5e9c2c`](https://gi
Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208237568
Some minor comments, pending those LGTM. @mengxr @thunterdb could you take
a final pass?
---
If your project is set up for it, you can reply to this email and have your
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r59172573
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala
---
@@ -17,78 +17,59 @@
package org.apache.spark.ml.feat
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r59171165
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala
---
@@ -17,78 +17,59 @@
package org.apache.spark.ml.feat
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r59171013
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala
---
@@ -17,78 +17,59 @@
package org.apache.spark.ml.feat
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r59169981
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,15 +49,28 @@ private[feature] trait QuantileDiscretizerBase ex
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r59169441
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -51,19 +51,18 @@ private[feature] trait QuantileDiscretizerBase ex
Github user oliverpierson commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208130852
scalastyle check is failing because the `@see` link on [line
54](https://github.com/apache/spark/pull/11553/commits/c365da039f239e03f21fa927f8fa1ccaa9ca9ba8#diff-b
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208130405
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208130400
**[Test build #55500 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55500/consoleFull)**
for PR 11553 at commit
[`c365da0`](https://g
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208130403
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-208129954
**[Test build #55500 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55500/consoleFull)**
for PR 11553 at commit
[`c365da0`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-207704973
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-207704960
**[Test build #55418 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55418/consoleFull)**
for PR 11553 at commit
[`5196c2d`](https://g
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-207704971
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-207704521
**[Test build #55418 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55418/consoleFull)**
for PR 11553 at commit
[`5196c2d`](https://gi
Github user oliverpierson commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-206380690
@MLnick No worries. Glad you guys are so active in your PRs. Just letting
everyone know that I haven't forgotten about this. Should get to it later this
week/e
Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-204118698
Ok I'm happy with that. @oliverpierson sorry for the run around, but in the
end it is much simpler (though we end up where you started :)
---
If your project is set up
Github user oliverpierson commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-204092809
@mengxr @MLnick I'll go with a default relative error of `0.001` and log a
warning in the case that the number of buckets is `> 1/relErr`, if no one has
objection
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-204018629
@oliverpierson @MLnick There are multiple ways to retrieve a param value.
For example, users can call `explainParams` to see all param docs and values.
It would confuse
Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-202819300
My only issue with leaving the getters unchanged is, how do you actually
know what the computed relative error is (as the getters throw
`NoSuchElementException` since th
Github user oliverpierson commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-201321707
@mengxr Thanks for the review. Concerning `getRelativeError`, we had
decided to not call `$(relativeError)` directly because we wanted the default
value to vary
Github user oliverpierson commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r57450251
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala
---
@@ -91,6 +44,46 @@ class QuantileDiscretizerSuite
Github user oliverpierson commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r57450030
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,6 +49,21 @@ private[feature] trait QuantileDiscretizerB
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r57414495
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala
---
@@ -91,6 +44,46 @@ class QuantileDiscretizerSuite
"O
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r57414476
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -89,12 +106,13 @@ final class QuantileDiscretizer(override val uid
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r57414488
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala
---
@@ -17,72 +17,25 @@
package org.apache.spark.ml.feat
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r57414491
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala
---
@@ -91,6 +44,46 @@ class QuantileDiscretizerSuite
"O
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r57414473
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,6 +49,21 @@ private[feature] trait QuantileDiscretizerBase ext
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r57414469
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,6 +49,21 @@ private[feature] trait QuantileDiscretizerBase ext
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r57414489
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala
---
@@ -91,6 +44,46 @@ class QuantileDiscretizerSuite
"O
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r57414462
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,6 +49,21 @@ private[feature] trait QuantileDiscretizerBase ext
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-200145515
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-200145511
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-200145389
**[Test build #53870 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53870/consoleFull)**
for PR 11553 at commit
[`81102aa`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-200134267
**[Test build #53870 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53870/consoleFull)**
for PR 11553 at commit
[`81102aa`](https://gi
Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-199714877
I made a couple minor comments. LGTM otherwise (and those are not strictly
required).
@thunterdb @jkbradley care to take a final pass?
---
If your project is s
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r56956545
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala
---
@@ -91,6 +44,46 @@ class QuantileDiscretizerSuite
"O
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r56956374
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,6 +49,21 @@ private[feature] trait QuantileDiscretizerBase ext
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-199098826
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-199098825
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-199098777
**[Test build #53650 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53650/consoleFull)**
for PR 11553 at commit
[`77c6129`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-199090610
**[Test build #53650 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53650/consoleFull)**
for PR 11553 at commit
[`77c6129`](https://gi
Github user oliverpierson commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-199090378
@MLnick My bad, thanks for catching that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r56467230
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,6 +49,19 @@ private[feature] trait QuantileDiscretizerBase ext
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-197140007
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-197140010
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-197139782
**[Test build #53267 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53267/consoleFull)**
for PR 11553 at commit
[`18a3ec6`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-197128917
**[Test build #53267 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53267/consoleFull)**
for PR 11553 at commit
[`18a3ec6`](https://gi
Github user oliverpierson commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-197128539
@MLnick Thanks for the review. I've made the changes and added a few
tests. Let me know if you there's anything else.
---
If your project is set up for it, you
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r55966447
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,6 +49,20 @@ private[feature] trait QuantileDiscretizerBase ext
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r55964780
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,6 +49,20 @@ private[feature] trait QuantileDiscretizerBase ext
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r55964697
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,6 +49,20 @@ private[feature] trait QuantileDiscretizerBase ext
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/11553#discussion_r55963977
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala ---
@@ -49,6 +49,20 @@ private[feature] trait QuantileDiscretizerBase ext
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-195156634
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-195156635
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-195156464
**[Test build #52884 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52884/consoleFull)**
for PR 11553 at commit
[`bcb62dc`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-195147872
**[Test build #52884 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52884/consoleFull)**
for PR 11553 at commit
[`bcb62dc`](https://gi
Github user oliverpierson commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-195146742
So... I ended up exposing `relativeError`. Figured it wasn't that much
work and may be useful to someone. Let me know if there's any
questions/concerns/changes.
Github user oliverpierson commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-195067163
@MLnick thanks for the advice. @thunterdb I thought people may want the
ability to adjust it a la `ParamGridBuilder`. After thinking about it, I
believe that in
Github user thunterdb commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-195054258
@oliverpierson I was thinking that `relativeError` should be automatically
selected (and not exposed as a param). However, I am fine with exposing it for
the sake of
Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/11553#issuecomment-194231520
@oliverpierson `$(relativeError)` is just an alias for `getOrDefault`, so
you could remove `setDefault(relativeError -> 0.01)` and do something like:
```scala
78 matches
Mail list logo
