Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/15149
Thanks all for handling this edge case! Coming late to the
discussion...but I like the decisions made here.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15149
Merged to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65776/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65776 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65776/consoleFull)**
for PR 15149 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65776 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65776/consoleFull)**
for PR 15149 at commit
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/15149
Right now, if a multinomial family is used in LOR, it silently ignores
`threshold` regardless. I don't really like that behavior, but perhaps we can
focus on it in (and add some tests) in
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65768/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65768 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65768/consoleFull)**
for PR 15149 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65768 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65768/consoleFull)**
for PR 15149 at commit
Github user MLnick commented on the issue:
https://github.com/apache/spark/pull/15149
Sure - though actually I think it is perhaps simpler to just allow one 0 in
validation for `thresholds` - because we definitely don't want to throw an
error only at prediction time once the user has
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/15149
I think it would also be ok to explicitly fail if we don't want to support
that - but fail intentionally.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user MLnick commented on the issue:
https://github.com/apache/spark/pull/15149
Hmm, yes that would be an inconsistent scenario cos `thresholds` would be
used in that case rather than `threshold`. And `threshold` could have been set
to `0` => `thresholds = [1, 0]` (or set to
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/15149
Yah I guess we can consider the case where the user explicitly states
multinomial as the family but then only has two classes and uses `setThreshold`
rather than `setThresholds` an error state
Github user MLnick commented on the issue:
https://github.com/apache/spark/pull/15149
It does introduce a slight inconsistency because setting `thresholds` to
[0, 1] for binary is now not allowed, but setting `threshold` to 0 is fine.
Still, I think it's valid to set `threshold` to
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65724/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65724 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65724/consoleFull)**
for PR 15149 at commit
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15149
Ah, do we need to update that? it looks like `threshold` is separate, and
overrides `thresholds`. It's just used as a cutoff for the positive class, so
it doesn't have same problem when it's 0. You
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/15149
Sorry for my late review - was giving a talk yesterday so was focused on
that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65720/
Test PASSed.
---
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/15149
This looks really reasonable, the only catch is that the thresholds can be
effectively set through `setThreshold` as well as `setThresholds`.
So we probably also want to update the range
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65720 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65720/consoleFull)**
for PR 15149 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65724 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65724/consoleFull)**
for PR 15149 at commit
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/15149
One small comment, otherwise LGTM. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65720 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65720/consoleFull)**
for PR 15149 at commit
Github user MLnick commented on the issue:
https://github.com/apache/spark/pull/15149
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65714 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65714/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65714/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65714 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65714/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65713 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65713/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65713/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65713 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65713/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65660/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65660 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65660/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65659/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65659 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65659/consoleFull)**
for PR 15149 at commit
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15149
Right now, that limit is only for parity with the randomForest package that
this is apparently based on. I agree that it's not clear why these couldn't sum
to something more than 1. If they were to
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/15149
+1 for not changing the sum requirement. I agree that we need to restrict
them to sum to something non-zero and all positive. Thanks for the
clarification.
---
If your project is set up for it,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65660 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65660/consoleFull)**
for PR 15149 at commit
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/15149
Requiring these thresholds to sum <= 1 seems entirely arbitrary. I don't
know why thresholds that sum to `0.347` are any more valid than thresholds that
sum to `347`. If these are not meant to
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65659 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65659/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65650/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65650 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65650/consoleFull)**
for PR 15149 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65650 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65650/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65648 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65648/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65648/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65648 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65648/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65647/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65647 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65647/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65647 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65647/consoleFull)**
for PR 15149 at commit
Github user MLnick commented on the issue:
https://github.com/apache/spark/pull/15149
@sethah that is the case for R's randomForest:
https://github.com/cran/randomForest/blob/master/R/predict.randomForest.R#L47
---
If your project is set up for it, you can reply to this email and
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/15149
Why does the sum need to be less than one? That is not the case for R's
randomForest "cutoff" parameter.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65597/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65597 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65597/consoleFull)**
for PR 15149 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65597 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65597/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65593/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65593 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65593/consoleFull)**
for PR 15149 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15149
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15149
**[Test build #65593 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65593/consoleFull)**
for PR 15149 at commit
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15149
CC @MLnick @zhengruifeng
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
71 matches
Mail list logo
