Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-158787861
oh, i see. Yeah, we need to use `Integer` to get null. `Int` is not
nullable.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-158787727
Sorry, I think it is a test case issue.
```
testData = Seq((1, 2), (2, 2), (3, 4), (null.asInstanceOf[Int],
5)).toDF("a", "b")
```
Scala automati
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-158787457
@gatorsmile Can you create a jira (with repro in the description) and ping
me from that jira?
---
If your project is set up for it, you can reply to this email and have y
Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-158781733
This might not be related to rollup logics. It is a bug of Dataframe. I
will try to fix it soon.
Thanks!
---
If your project is set up for it, you can rep
Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-158781004
Thank you @aray @yhuai ! The code changes look great!
Based on my test cases, the rollup and cube still return incorrect results
when the table contains null
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/9815
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-158229469
Thank you for the fix! I am merging it to master and branch 1.6.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45412192
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -211,45 +211,35 @@ class Analyzer(
GroupingSets
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45411219
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -211,45 +211,35 @@ class Analyzer(
GroupingSets
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-158113009
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-158112777
**[Test build #46334 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46334/consoleFull)**
for PR 9815 at commit
[`2162b6c`](https://git
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-158113007
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-158076872
**[Test build #46334 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46334/consoleFull)**
for PR 9815 at commit
[`2162b6c`](https://gith
Github user aray commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45346085
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -211,45 +211,31 @@ class Analyzer(
GroupingSets(
Github user aray commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45345199
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala ---
@@ -60,6 +60,68 @@ class DataFrameAggregateSuite extends QueryTest with
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45306081
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -211,45 +211,31 @@ class Analyzer(
GroupingSets
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45306028
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -211,45 +211,31 @@ class Analyzer(
GroupingSets
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45306056
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala ---
@@ -60,6 +60,68 @@ class DataFrameAggregateSuite extends QueryTest with
Github user aray commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45300236
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -211,45 +211,31 @@ class Analyzer(
GroupingSets(
Github user aray commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157950224
@yhuai I do think this is the minimal fix. However like I stated in the
summary we are simplifying instead of making more exceptions that might
themselves have bugs. Let me
Github user aray commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45298904
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -211,45 +211,31 @@ class Analyzer(
GroupingSets(
Github user aray commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45298806
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala
---
@@ -323,6 +323,10 @@ trait GroupingAnalytics extends
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157944344
@aray Thank you for the PR! Since we are in the QA period for 1.6 release,
it will be great if we just fix the problem without any other changes. Is this
the minimal fix f
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45297166
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -211,45 +211,31 @@ class Analyzer(
GroupingSets
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/9815#discussion_r45297161
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala
---
@@ -323,6 +323,10 @@ trait GroupingAnalytics extends
Github user aray commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157922956
@yhuai can you take a look at this pr?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157906544
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157906547
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157906395
**[Test build #46256 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46256/consoleFull)**
for PR 9815 at commit
[`12914fa`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157878491
**[Test build #46256 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46256/consoleFull)**
for PR 9815 at commit
[`12914fa`](https://gith
Github user aray commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157876634
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this featur
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157866014
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157866017
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user aray commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157862146
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this featur
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157859791
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157859789
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157859705
**[Test build #46232 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46232/consoleFull)**
for PR 9815 at commit
[`12914fa`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9815#issuecomment-157831135
**[Test build #46232 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46232/consoleFull)**
for PR 9815 at commit
[`12914fa`](https://gith
GitHub user aray opened a pull request:
https://github.com/apache/spark/pull/9815
[SPARK-11275] [SQL] Incorrect results when using rollup/cube
Fixes bug with grouping sets (including cube/rollup) where aggregates that
included grouping expressions would return the wrong (null) resul
39 matches
Mail list logo
