Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-189155044
@viirya I really don't see any reason to keep ANTLR3 around after we
migrate the parser.
---
If your project is set up for it, you can reply to this email and have
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-189136538
@hvanhovell Great to see your initial work. It looks promising. I think
this can wait until the new parser. Besides, are we going to deprecate ANTLR3
used now?
---
If
Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-189131737
@viirya I am currently ANTLR4 based version of the parsers (see my repo for
a few initial commits). It is basically a port of the presto parsers. I need
another week
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188696475
ok. I will continue to see if we can improve the performance of parsing
nested union.
---
If your project is set up for it, you can reply to this email and have your
re
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188675431
Any other thing we can do for this perf problem? It's only 25 levels of
nesting. It seems strange to me that the parser would take mins to parse this
...
cc @hvan
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188665321
@rxin I don't think we should convert union sql query back to nested one. I
would like to remove the `()` from SQLBuilder for union. What do you think?
---
If your proj
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188664991
@rxin ok. I got why it takes so long to finish the test.
The original query:
SELECT count(1) FROM (
SELECT key, value FROM src UNION ALL
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188629681
Can you take a look why would 2 queries take 13 mins?
```
EXPLAIN
SELECT count(1) FROM (
SELECT key, value FROM src UNION ALL
SELECT key, value
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188627434
@rxin Looks like HiveCompatibilitySuite.union16 doesn't hang from this. But
it actually takes long time to finish that test (`[info] - union16 (13 minutes,
21 seconds)`)
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188626798
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188626796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188626175
**[Test build #51928 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51928/consoleFull)**
for PR 11361 at commit
[`5ff5ac2`](https://g
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188592169
cc @rxin
I am not sure if HiveCompatibilitySuite.union16 was hanging from this.
Because I copy the same queries from union16 to HiveQuerySuite and they are
wor
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11361#issuecomment-188592087
**[Test build #51928 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51928/consoleFull)**
for PR 11361 at commit
[`5ff5ac2`](https://gi
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/11361
[SPARK-13321][SQL] Support nested UNION in parser
JIRA: https://issues.apache.org/jira/browse/SPARK-13321
## What changes were proposed in this pull request?
The following SQL can n
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-188535960
@rxin ok, I will look into this today.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project do
Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-188460661
Just submit a PR: https://github.com/apache/spark/pull/11352. Thanks! @rxin
---
If your project is set up for it, you can reply to this email and have your
reply ap
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-188459220
@viirya when you fixed this issue, please make sure you add a unit test
aside from Hive compat test.
---
If your project is set up for it, you can reply to this email an
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-188459106
Please do.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enab
Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-188458568
@rxin Do you need me to submit a PR to disable the following test case?
This test case requires the nested Union support. Thanks!
https://github.com/apache/
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-188440444
Note: I reverted this patch because HiveCompatibilitySuite.union16 was
hanging from this.
Somehow we weren't running test cases in HiveCompatibilitySuite anymore f
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/11204
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-186980183
Thanks - merging in master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-186974334
cc @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enab
Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-186835931
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabl
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-186834983
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-186834982
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-186834887
**[Test build #51630 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51630/consoleFull)**
for PR 11204 at commit
[`d024fff`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-186822914
**[Test build #51630 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51630/consoleFull)**
for PR 11204 at commit
[`d024fff`](https://gi
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-186820360
@hvanhovell Do you have other comments on this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If you
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/11204#discussion_r52968390
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/CatalystQlSuite.scala
---
@@ -201,4 +201,68 @@ class CatalystQlSuite extends PlanTest {
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/11204#discussion_r52968367
--- Diff:
sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/SparkSqlParser.g
---
@@ -2320,6 +2320,19 @@ regularBody[boolean topLevel]
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/11204#discussion_r52968331
--- Diff:
sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/SparkSqlParser.g
---
@@ -2320,6 +2320,19 @@ regularBody[boolean topLevel]
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184297761
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184297752
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184296823
**[Test build #51314 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51314/consoleFull)**
for PR 11204 at commit
[`190b326`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184255054
**[Test build #51314 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51314/consoleFull)**
for PR 11204 at commit
[`190b326`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184250287
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184250290
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184250011
**[Test build #51310 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51310/consoleFull)**
for PR 11204 at commit
[`f462997`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184217585
**[Test build #51310 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51310/consoleFull)**
for PR 11204 at commit
[`f462997`](https://gi
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/11204#discussion_r52899001
--- Diff:
sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/SparkSqlParser.g
---
@@ -2320,6 +2320,19 @@ regularBody[boolean topLevel]
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/11204#discussion_r52896850
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/CatalystQlSuite.scala
---
@@ -201,4 +201,13 @@ class CatalystQlSuite extends PlanTest {
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/11204#discussion_r52896589
--- Diff:
sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/SparkSqlParser.g
---
@@ -2320,6 +2320,19 @@ regularBody[boolean topLevel]
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/11204#discussion_r52896251
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/CatalystQlSuite.scala
---
@@ -201,4 +201,13 @@ class CatalystQlSuite extends PlanTest
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184102321
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184102320
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184102156
**[Test build #51295 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51295/consoleFull)**
for PR 11204 at commit
[`93bd51e`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11204#issuecomment-184080252
**[Test build #51295 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51295/consoleFull)**
for PR 11204 at commit
[`93bd51e`](https://gi
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/11204#discussion_r52862728
--- Diff:
sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/SparkSqlParser.g
---
@@ -2320,6 +2320,19 @@ regularBody[boolean topLevel]
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/11204
[SPARK-13321][SQL] Support nested UNION in parser
JIRA: https://issues.apache.org/jira/browse/SPARK-13321
The following SQL can not be parsed with current parser:
SELECT `u_1`.
51 matches
Mail list logo
