Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4524#discussion_r24531964
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -128,6 +128,29 @@ abstract class LogicalPlan extends
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4524
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4524#issuecomment-73963498
I merged this since we're close to the deadline. Thanks!
Please address @rxin comments in a follow up.
---
If your project is set up for it, you can reply to
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/4524#discussion_r24533628
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -128,6 +128,29 @@ abstract class LogicalPlan
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/4524#issuecomment-73975973
FYI I added more inline comment https://github.com/apache/spark/pull/4539
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4524#issuecomment-73847372
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4524#issuecomment-73847367
[Test build #27278 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27278/consoleFull)
for PR 4524 at commit
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4524#issuecomment-73841717
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4524#issuecomment-73841997
[Test build #27278 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27278/consoleFull)
for PR 4524 at commit
GitHub user tianyi opened a pull request:
https://github.com/apache/spark/pull/4524
[SPARK-3688][SQL]LogicalPlan can't resolve column correctlly
This PR fixed the resolving problem described in
https://issues.apache.org/jira/browse/SPARK-3688
```
CREATE TABLE t1(x INT);
Github user tianyi commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-73837719
It's hard to rebase this branch due to so much changes in the latest master
branch.
So I had opened a new PR #4524 for this Issue.
---
If your project is set up for
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4524#issuecomment-73837743
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user tianyi closed the pull request at:
https://github.com/apache/spark/pull/2542
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-67802647
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-67802692
[Test build #24696 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24696/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-67805585
[Test build #24696 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24696/consoleFull)
for PR 2542 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-67805590
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-67398969
[Test build #548 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/548/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-67400995
[Test build #548 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/548/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-67407944
[Test build #549 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/549/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-67421071
[Test build #549 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/549/consoleFull)
for PR 2542 at commit
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-63147249
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-63147391
[Test build #23398 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23398/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-63152142
[Test build #23398 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23398/consoleFull)
for PR 2542 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-63152146
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-62629348
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-61043355
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-61043457
[Test build #22517 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22517/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-61043719
[Test build #22519 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22519/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-61045960
[Test build #22517 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22517/consoleFull)
for PR 2542 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-61045963
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-61046262
[Test build #22519 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22519/consoleFull)
for PR 2542 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-61046263
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user tianyi commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-61046528
Hi, @marmbrus @liancheng
I had rebased this PR after
[#2762](https://github.com/apache/spark/pull/2762).
Any more comments on this?
---
If your project is set
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-60865427
[Test build #22417 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22417/consoleFull)
for PR 2542 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-60871231
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-60871225
[Test build #22417 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22417/consoleFull)
for PR 2542 at commit
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-58941699
Hmm, thats unfortunate. You can use `.saveAsTable(...)`, but that is hard
to use with `createQueryTest` since that always calls `reset()`. We should
probably open a
Github user tianyi commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-58976342
https://www.github.com/apache/spark/pull/2762
This is a PR created by @chenghao-intel
I had tested my PR with it, it works fine.
---
If your project is set up
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-58338302
I tried this on hive:
CREATE TABLE t1(x INT);
CREATE TABLE t2(a STRUCTx: INT, k INT);
SELECT a.x FROM t1 a JOIN t2 b;
And hive can
Github user tianyi commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-58338634
@cloud-fan I had create a test case named test ambiguousReferences
resolved as hive , the case is almost the same as yours.
---
If your project is set up for it, you can
Github user tianyi commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-58339330
I think we should follow hive syntax as much as possible, which could save
lots of work when people move their application form hive to spark, @marmbrus
what's your
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-58412015
Yeah, it would be good to stick with hive semantics for exactly that
reason. I assume you can still access the other `x` as `b.a.x`?
---
If your project is set up for
Github user tianyi commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-58451999
@marmbrus , could u help me to review these codes? Any suggestion will be
appreciated.
---
If your project is set up for it, you can reply to this email and have your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-58310146
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21445/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-58313595
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21445/consoleFull)
for PR 2542 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-58313597
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57130258
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/191/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57135255
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/191/consoleFull)
for PR 2542 at commit
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18189290
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -110,8 +110,20 @@ abstract class LogicalPlan
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18189363
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -110,8 +110,20 @@ abstract class LogicalPlan
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18189391
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -110,8 +110,20 @@ abstract class LogicalPlan
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18189432
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
---
@@ -57,6 +57,12 @@ class SQLQuerySuite extends QueryTest {
Github user tianyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18191704
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -110,8 +110,20 @@ abstract class LogicalPlan
Github user tianyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18192244
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -110,8 +110,20 @@ abstract class LogicalPlan
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18196493
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -110,8 +110,20 @@ abstract class LogicalPlan
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57076212
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20929/consoleFull)
for PR 2542 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57078355
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57078351
**[Tests timed
out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20929/consoleFull)**
after a configured wait of `120m`.
---
If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57079256
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/174/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57081837
**[Tests timed
out](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/174/consoleFull)**
after a configured wait of `120m`.
---
If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57083428
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/175/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57086391
**[Tests timed
out](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/175/consoleFull)**
after a configured wait of `120m`.
---
If your project
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18123371
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -110,9 +110,18 @@ abstract class LogicalPlan
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18123559
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -110,9 +110,18 @@ abstract class LogicalPlan
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18123585
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -110,9 +110,18 @@ abstract class LogicalPlan
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57056092
@cloud-fan Sorry, made a mistake in the snippet I used, it should be:
```sql
CREATE TABLE t1(x INT);
CREATE TABLE t2(a STRUCTx: INT, k INT);
SELECT
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57062370
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57062530
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20921/consoleFull)
for PR 2542 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57065879
**[Tests timed
out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20921/consoleFull)**
after a configured wait of `120m`.
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57065881
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user tianyi commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57071901
Hi, @marmbrus . The current codes still have some bugs to fix, I talked
@liancheng yesterday, I will push a update later.
---
If your project is set up for it, you can
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57072994
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20926/consoleFull)
for PR 2542 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57073055
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57073139
@liancheng Hmm..I don't have a hive environment for test...
CREATE TABLE t1(x INT);
CREATE TABLE t2(a STRUCTx: INT, k INT);
SELECT a.x FROM
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57073173
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/172/consoleFull)
for PR 2542 at commit
Github user tianyi commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57074276
@cloud-fan, I think it is reasonable for return ambiguous references in
the case you mentioned, because we can't make sure whether 'a' is a table alias
or column name. In
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57074476
@tianyi
CREATE TABLE t1(x INT);
CREATE TABLE t2(a STRUCTx: INT, k INT);
SELECT a.x FROM t1 a JOIN t2 b ON a.x = b.k;
But hive can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57074970
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57074968
**[Tests timed
out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20926/consoleFull)**
after a configured wait of `120m`.
---
If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57075143
**[Tests timed
out](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/172/consoleFull)**
after a configured wait of `120m`.
---
If your project
GitHub user tianyi opened a pull request:
https://github.com/apache/spark/pull/2542
[SPARK-3688][SQL]LogicalPlan can't resolve column correctlly
we should add some judgements base on the datatype to handle some
particular sql descript in
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-56926422
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18080938
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
---
@@ -57,6 +57,12 @@ class SQLQuerySuite extends QueryTest {
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2542#discussion_r18080957
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -110,9 +110,18 @@ abstract class LogicalPlan
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-56940392
Nice catch! I think the problem is: for a table(a string, b string), when
we run select a.b from test a join test b, we have 2 options to resolve
`a.b`. One is table
Github user tianyi commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-56942258
@cloud-fan
I think we should follow some orders to resolve a.b
First, if 'a' match a table name or alias, then we recognize a as a table.
if 'a' did not match
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-56952680
@tianyi Let me raise an example. For `table: { a: { c: String }, b: String
}`, if we run `select a.b from test a join test b`, your PR will still give 2
options to
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/2542#issuecomment-57039692
@cloud-fan Tried the following snippet with ambiguous references in Hive:
```sql
CREATE TABLE t1(a STRUCTx: INT, k INT);
CREATE TABLE t2(x INT);
89 matches
Mail list logo
