Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/15763
LGTM. Merging to master/2.1/2.0. Thanks!
@nsyca could you open a JIRA discuss a generic subquery operator.
---
If your project is set up for it, you can reply to this email and have
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68623/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68623 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68623/consoleFull)**
for PR 15763 at commit
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
@hvanhovell could you please review the PR again? Hopefully this is the
last one.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68621/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68621 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68621/consoleFull)**
for PR 15763 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68623 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68623/consoleFull)**
for PR 15763 at commit
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
@hvanhovell
I removed the redundant checking in `ScalarSubquery` as pointed out in your
earlier comment (copied below).
"I was also wondering if we still need the following
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68621 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68621/consoleFull)**
for PR 15763 at commit
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
@hvanhovell could you please review the latest PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68551/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68551 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68551/consoleFull)**
for PR 15763 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68551 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68551/consoleFull)**
for PR 15763 at commit
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
Thanks for the tip on the code. I will work on it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/15763
I think it is also better that we start whitelisting operators instead
blacklisting them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/15763
@nsyca it should be relatively straighforward to implement this here:
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
@hvanhovell Then we will need to walk from the top of the operator hosting
the outer reference to the operator hosting the correlation to ensure there is
no Aggregate or Window operator.
If
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/15763
@nsyca I feel that the current approach is to restrictive. I would prefer
to just close the gap for Window and Aggregate.
---
If your project is set up for it, you can reply to this email and
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
@hvanhovell ping... Are there anything I need to do to close this incorrect
result problem?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
@srinathshankar It is intentional. It is impossible to do an analysis on
what in-between operations we can allow and what we cannot. Correlated
predicates can be placed in any arbitrary level of
Github user srinathshankar commented on the issue:
https://github.com/apache/spark/pull/15763
As it stands the check seems overly aggressive. The following works:
```sql
select t1.c1
from t1
where t1.c1 in (
select c1 + 1 as col
from t2
where t1.c2 <
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
I think I have addressed all the pending comments on this PR. Would be
there anything left to do?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68170/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68170 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68170/consoleFull)**
for PR 15763 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68170 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68170/consoleFull)**
for PR 15763 at commit
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68147/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68147 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68147/consoleFull)**
for PR 15763 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68142/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68142 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68142/consoleFull)**
for PR 15763 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68147 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68147/consoleFull)**
for PR 15763 at commit
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
@hvanhovell thank you for the trick.
Exploring your idea further, I looked at CheckAnalysis. This is a trait
that Analyzer extends from. The checks in def CheckAnalysis of trait
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/15763
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/15763
@nsyca you can retrigger builds by issuing `retest this please` this works
as soon as an `ok to test` has been issued by an admin.
It might be a good idea to extend the logic in
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
I'd like to a note that a piece of existing code that closes to address
this problem but only done for the scalar subquery context can be found by
searching this pattern in CheckAnalysis.scala (line
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68142 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68142/consoleFull)**
for PR 15763 at commit
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/15763
Hmm. don't know what's wrong with the test failures reported in hive/test
here. I will address @rxin's comment and submit a new version. That should
trigger a new build.
---
If your project is set
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68122 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68122/consoleFull)**
for PR 15763 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68122/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15763
**[Test build #68122 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68122/consoleFull)**
for PR 15763 at commit
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/15763
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15763
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
49 matches
Mail list logo
