[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/11371#discussion_r64522917
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1443,6 +1445,32 @@ class Analyzer(
}
/**
+ * Corrects attribute references in an expression tree of some operators
(e.g., filters and
+ * projects) if these operators have a join as a child and the
references point to columns on the
+ * input relation of the join. This is because some joins change the
nullability of input columns
+ * and this could cause illegal optimization (e.g., NULL propagation)
and wrong answers.
+ * See SPARK-13484 and SPARK-13801 for the concrete queries of this case.
+ */
+ object SolveIllegalReferences extends Rule[LogicalPlan] {
+
+private def replaceReferences(e: Expression, attrMap:
AttributeMap[Attribute]) = e.transform {
+ case a: AttributeReference => attrMap.get(a).getOrElse(a)
+}
+
+def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
+ case q: LogicalPlan =>
+q.transform {
+ case f @ Filter(filterCondition,
ExtractJoinOutputAttributes(join, joinOutputMap)) =>
+f.copy(condition = replaceReferences(filterCondition,
joinOutputMap))
--- End diff --
okay, I'll check it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/11371#discussion_r64521117
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1443,6 +1445,32 @@ class Analyzer(
}
/**
+ * Corrects attribute references in an expression tree of some operators
(e.g., filters and
+ * projects) if these operators have a join as a child and the
references point to columns on the
+ * input relation of the join. This is because some joins change the
nullability of input columns
+ * and this could cause illegal optimization (e.g., NULL propagation)
and wrong answers.
+ * See SPARK-13484 and SPARK-13801 for the concrete queries of this case.
+ */
+ object SolveIllegalReferences extends Rule[LogicalPlan] {
+
+private def replaceReferences(e: Expression, attrMap:
AttributeMap[Attribute]) = e.transform {
+ case a: AttributeReference => attrMap.get(a).getOrElse(a)
+}
+
+def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
+ case q: LogicalPlan =>
+q.transform {
+ case f @ Filter(filterCondition,
ExtractJoinOutputAttributes(join, joinOutputMap)) =>
+f.copy(condition = replaceReferences(filterCondition,
joinOutputMap))
--- End diff --
https://github.com/apache/spark/pull/13290/files
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/11371#discussion_r64507505
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1443,6 +1445,32 @@ class Analyzer(
}
/**
+ * Corrects attribute references in an expression tree of some operators
(e.g., filters and
+ * projects) if these operators have a join as a child and the
references point to columns on the
+ * input relation of the join. This is because some joins change the
nullability of input columns
+ * and this could cause illegal optimization (e.g., NULL propagation)
and wrong answers.
+ * See SPARK-13484 and SPARK-13801 for the concrete queries of this case.
+ */
+ object SolveIllegalReferences extends Rule[LogicalPlan] {
+
+private def replaceReferences(e: Expression, attrMap:
AttributeMap[Attribute]) = e.transform {
+ case a: AttributeReference => attrMap.get(a).getOrElse(a)
+}
+
+def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
+ case q: LogicalPlan =>
+q.transform {
+ case f @ Filter(filterCondition,
ExtractJoinOutputAttributes(join, joinOutputMap)) =>
+f.copy(condition = replaceReferences(filterCondition,
joinOutputMap))
--- End diff --
okay, I wait your ping.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/11371#discussion_r64506496
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1443,6 +1445,32 @@ class Analyzer(
}
/**
+ * Corrects attribute references in an expression tree of some operators
(e.g., filters and
+ * projects) if these operators have a join as a child and the
references point to columns on the
+ * input relation of the join. This is because some joins change the
nullability of input columns
+ * and this could cause illegal optimization (e.g., NULL propagation)
and wrong answers.
+ * See SPARK-13484 and SPARK-13801 for the concrete queries of this case.
+ */
+ object SolveIllegalReferences extends Rule[LogicalPlan] {
+
+private def replaceReferences(e: Expression, attrMap:
AttributeMap[Attribute]) = e.transform {
+ case a: AttributeReference => attrMap.get(a).getOrElse(a)
+}
+
+def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
+ case q: LogicalPlan =>
+q.transform {
+ case f @ Filter(filterCondition,
ExtractJoinOutputAttributes(join, joinOutputMap)) =>
+f.copy(condition = replaceReferences(filterCondition,
joinOutputMap))
--- End diff --
How about we use a `q.transformUp` to fix the nullability in a bottom-up
way? For every node, we create an `AttributeMap` using the output of its child.
Then, we use `transformExpressions` to fix the nullability if necessary. Let
me try it out and ping you when I have a version.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-219005700 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-219005701 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58552/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-219005500 **[Test build #58552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58552/consoleFull)** for PR 11371 at commit [`bd13652`](https://github.com/apache/spark/commit/bd13652961de5dd7ae0968b3b8c60bb0a191b87f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-218987330 **[Test build #58552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58552/consoleFull)** for PR 11371 at commit [`bd13652`](https://github.com/apache/spark/commit/bd13652961de5dd7ae0968b3b8c60bb0a191b87f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-218986651 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-218809716 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-218809726 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58491/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210402778 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55911/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210402776 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210402524 **[Test build #55911 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55911/consoleFull)** for PR 11371 at commit [`441d9a5`](https://github.com/apache/spark/commit/441d9a5c9a8aa5f7f008f92fcb29fdced79f9266). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210367560 **[Test build #55911 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55911/consoleFull)** for PR 11371 at commit [`441d9a5`](https://github.com/apache/spark/commit/441d9a5c9a8aa5f7f008f92fcb29fdced79f9266). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210297311 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210297313 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55899/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210297103 **[Test build #55899 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55899/consoleFull)** for PR 11371 at commit [`1e45943`](https://github.com/apache/spark/commit/1e45943fd8ac8db0292f9a042a62a3807e809869). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210296251 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210296164 **[Test build #55897 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55897/consoleFull)** for PR 11371 at commit [`0549f88`](https://github.com/apache/spark/commit/0549f88c8e11127ddc849a5b06f2363a37ce85a5). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210296253 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55897/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210285679 **[Test build #55899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55899/consoleFull)** for PR 11371 at commit [`1e45943`](https://github.com/apache/spark/commit/1e45943fd8ac8db0292f9a042a62a3807e809869). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-210282634 **[Test build #55897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55897/consoleFull)** for PR 11371 at commit [`0549f88`](https://github.com/apache/spark/commit/0549f88c8e11127ddc849a5b06f2363a37ce85a5). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-200239184 @cloud-fan If your pull request (#11632) merged, I think the query in the top throws analysis exception, right? `SPARK-13484` essentially indicates that the kinds of queries should be correctly resolved in terms of user's usability. Anyway, I agree with your idea in #11632, so I'd like to discuss this based on #11632. What do you think? cc: @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/11371#issuecomment-196710904
I think the fundamental problem is, we give users the resolved attribute
but it may not be the real column when using it. For example, `b("count")`
actually is not the real column of the join. Instead of adding some special
handling, how about my proposal at https://github.com/apache/spark/pull/11632?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-196707867 cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-192926998 @yhuai ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-191122157 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52300/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-191122153 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-191121248 **[Test build #52300 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52300/consoleFull)** for PR 11371 at commit [`9c981fe`](https://github.com/apache/spark/commit/9c981fe1479473d58ee7f5257438a5fb84b2a6e7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-191084940 **[Test build #52300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52300/consoleFull)** for PR 11371 at commit [`9c981fe`](https://github.com/apache/spark/commit/9c981fe1479473d58ee7f5257438a5fb84b2a6e7). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-191038935 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52278/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-191038926 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-191038012 **[Test build #52278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52278/consoleFull)** for PR 11371 at commit [`c17c2b2`](https://github.com/apache/spark/commit/c17c2b269410f5befda9cd88fd603a15004e6a19). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-191012390 **[Test build #52278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52278/consoleFull)** for PR 11371 at commit [`c17c2b2`](https://github.com/apache/spark/commit/c17c2b269410f5befda9cd88fd603a15004e6a19). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-191010575 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-190669888 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-190669891 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52232/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-190669431 **[Test build #52232 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52232/consoleFull)** for PR 11371 at commit [`c17c2b2`](https://github.com/apache/spark/commit/c17c2b269410f5befda9cd88fd603a15004e6a19). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-190639197 **[Test build #52232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52232/consoleFull)** for PR 11371 at commit [`c17c2b2`](https://github.com/apache/spark/commit/c17c2b269410f5befda9cd88fd603a15004e6a19). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-190040111 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-190040114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52159/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-190040034 **[Test build #52159 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52159/consoleFull)** for PR 11371 at commit [`f1718d6`](https://github.com/apache/spark/commit/f1718d6932122b9020ba9b86254f34b953a83e99). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-190026957 **[Test build #52159 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52159/consoleFull)** for PR 11371 at commit [`f1718d6`](https://github.com/apache/spark/commit/f1718d6932122b9020ba9b86254f34b953a83e99). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-190025850 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189680686 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52128/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189680684 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189680630 **[Test build #52128 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52128/consoleFull)** for PR 11371 at commit [`f1718d6`](https://github.com/apache/spark/commit/f1718d6932122b9020ba9b86254f34b953a83e99). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189662024 **[Test build #52128 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52128/consoleFull)** for PR 11371 at commit [`f1718d6`](https://github.com/apache/spark/commit/f1718d6932122b9020ba9b86254f34b953a83e99). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189660570 @yhuai I added a new role 'SolveIllegalReference` to solve these kinds of illegal references. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189551961 @yhuai okay. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189447700 @maropu Thank you for the PR. My thought is that we may need to have a place to correct those nullable fields in the analyzer. Let me also think about it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189193165 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52035/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189193161 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189192642 **[Test build #52035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52035/consoleFull)** for PR 11371 at commit [`d3733ba`](https://github.com/apache/spark/commit/d3733ba794a732c797706942a9bfdb44af274b6f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189191588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52034/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189191586 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189191263 **[Test build #52034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52034/consoleFull)** for PR 11371 at commit [`c305776`](https://github.com/apache/spark/commit/c30577607c96614f0dec0ab8230859109ba29190). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189153005 **[Test build #52035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52035/consoleFull)** for PR 11371 at commit [`d3733ba`](https://github.com/apache/spark/commit/d3733ba794a732c797706942a9bfdb44af274b6f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189151917 **[Test build #52034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52034/consoleFull)** for PR 11371 at commit [`c305776`](https://github.com/apache/spark/commit/c30577607c96614f0dec0ab8230859109ba29190). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189093305 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189093314 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52013/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189092387 **[Test build #52013 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52013/consoleFull)** for PR 11371 at commit [`568afee`](https://github.com/apache/spark/commit/568afee9ad58a276ce0ef8a998ba61d7843f5554). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189085864 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189073173 **[Test build #52013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52013/consoleFull)** for PR 11371 at commit [`568afee`](https://github.com/apache/spark/commit/568afee9ad58a276ce0ef8a998ba61d7843f5554). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-189072694 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-188905353 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51976/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-188905351 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-188905098 **[Test build #51976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51976/consoleFull)** for PR 11371 at commit [`568afee`](https://github.com/apache/spark/commit/568afee9ad58a276ce0ef8a998ba61d7843f5554). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-188902690 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-188902691 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51972/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-188902402 **[Test build #51972 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51972/consoleFull)** for PR 11371 at commit [`9f8ff3d`](https://github.com/apache/spark/commit/9f8ff3d5c487e838f5a634dfb04d106ed4d41aa3). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-188861591 **[Test build #51976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51976/consoleFull)** for PR 11371 at commit [`568afee`](https://github.com/apache/spark/commit/568afee9ad58a276ce0ef8a998ba61d7843f5554). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11371#issuecomment-188861433 **[Test build #51972 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51972/consoleFull)** for PR 11371 at commit [`9f8ff3d`](https://github.com/apache/spark/commit/9f8ff3d5c487e838f5a634dfb04d106ed4d41aa3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-13484][SQL] Prevent illegal NULL propag...
GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/11371
[SPARK-13484][SQL] Prevent illegal NULL propagation when filtering
outer-join results
## What changes were proposed in this pull request?
This pr is to prevent illegal NULL propagation in the query below;
```
val a = sqlContext.range(10).select(col("id"), lit(0).as("count"))
val b = sqlContext.range(10).select((col("id") %
3).as("id")).groupBy("id").count()
a.join(b, a("id") === b("id"), "left_outer").filter(b("count").isNull)
```
It returns nothing because `b("count")` is not nullable and the filter
condition is always false by `Optimizer`.
## How was this patch tested?
Added a test for the query above in `DataFrameJoinSuite`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/maropu/spark spark13484
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11371.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11371
commit 9f8ff3d5c487e838f5a634dfb04d106ed4d41aa3
Author: Takeshi YAMAMURO
Date: 2016-02-25T15:56:29Z
Avoid irregal NULL propagation
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
