[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3386 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67792894 [Test build #24690 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24690/consoleFull) for PR 3386 at commit [`38c78cc`](https://github.com/apache/spark/commit/38c78cc72f913b143c52f5fba691d14385a1781d). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67795733 [Test build #24690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24690/consoleFull) for PR 3386 at commit [`38c78cc`](https://github.com/apache/spark/commit/38c78cc72f913b143c52f5fba691d14385a1781d). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Sort(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67795738 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24690/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67643989 [Test build #24646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24646/consoleFull) for PR 3386 at commit [`b229fe9`](https://github.com/apache/spark/commit/b229fe97626ed2ba245d6e4c82ff10244a37dedb). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67653363 [Test build #24646 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24646/consoleFull) for PR 3386 at commit [`b229fe9`](https://github.com/apache/spark/commit/b229fe97626ed2ba245d6e4c82ff10244a37dedb). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Sort(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67653370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24646/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/3386#discussion_r22121768
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDD.scala ---
@@ -238,7 +238,7 @@ class SchemaRDD(
* {{{
* schemaRDD.limit(10)
* }}}
- *
+ * o
--- End diff --
Typo?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3386#discussion_r22138882
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDD.scala ---
@@ -238,7 +238,7 @@ class SchemaRDD(
* {{{
* schemaRDD.limit(10)
* }}}
- *
+ * o
--- End diff --
Oh, yes, it's a typo, revert it. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67720979 [Test build #24661 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24661/consoleFull) for PR 3386 at commit [`aae675f`](https://github.com/apache/spark/commit/aae675f15009cf6376832b0d3fdff4cdb2dfedb1). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67722899 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24661/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67722895 [Test build #24661 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24661/consoleFull) for PR 3386 at commit [`aae675f`](https://github.com/apache/spark/commit/aae675f15009cf6376832b0d3fdff4cdb2dfedb1). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Sort(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/3386#discussion_r21999209
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -180,13 +181,14 @@ class Analyzer(catalog: Catalog, registry:
FunctionRegistry, caseSensitive: Bool
if (missingInProject.nonEmpty) {
// Add missing attributes and then project them away after the
sort.
Project(projectList.map(_.toAttribute),
-Sort(ordering,
+Sort(ordering, global,
Project(projectList ++ missingInProject, child)))
} else {
logDebug(sFailed to find $missingInProject in
${p.output.mkString(, )})
s // Nothing we can do here. Return original plan.
}
- case s @ Sort(ordering, a @ Aggregate(grouping, aggs, child)) if
!s.resolved a.resolved =
+ case s @ Sort(ordering, global, a @ Aggregate(grouping, aggs, child))
+if !s.resolved a.resolved =
--- End diff --
Indent 4 spaces.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/3386#discussion_r21999252
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala
---
@@ -130,7 +130,10 @@ case class WriteToFile(
override def output = child.output
}
-case class Sort(order: Seq[SortOrder], child: LogicalPlan) extends
UnaryNode {
+case class Sort(
--- End diff --
Can you add scala doc including `@param` to say what `global` means.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67383569 Sorry for the delay! This looks great. Two minor comments then LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-64325757 [Test build #23832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23832/consoleFull) for PR 3386 at commit [`e0047a0`](https://github.com/apache/spark/commit/e0047a02183c52fa637e2808d94fc4b98fbe18c8). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-64329427 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23832/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-64329419 [Test build #23832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23832/consoleFull) for PR 3386 at commit [`e0047a0`](https://github.com/apache/spark/commit/e0047a02183c52fa637e2808d94fc4b98fbe18c8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-64407794 [Test build #23841 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23841/consoleFull) for PR 3386 at commit [`6e720af`](https://github.com/apache/spark/commit/6e720af49428817c6f48d5e161b34e182a31b872). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-64419786 [Test build #23841 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23841/consoleFull) for PR 3386 at commit [`6e720af`](https://github.com/apache/spark/commit/6e720af49428817c6f48d5e161b34e182a31b872). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Sort(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-64419796 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23841/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/3386 [Spark-4512] [SQL] Unresolved Attribute Exception in Sort By It will cause exception while do query like: SELECT key+key FROM src sort by value; This fix is inspired by #3363 , hope it goes in after #3363 merged. And I've removed the `logical.SortPartitions` and added a new attribute `global` for `logical.Sort`, the reason we do that is for sharing `ResolveSortReferences` for both `ORDER BY` and `SORT BY`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/chenghao-intel/spark sort Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3386.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3386 commit 503b263408924df42b26689a6f8b913a5185c10d Author: Cheng Hao hao.ch...@intel.com Date: 2014-11-20T14:40:42Z Remove the logical.SortPartitions and Add global sort flag for logical.Sort --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-63820723 [Test build #23673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23673/consoleFull) for PR 3386 at commit [`503b263`](https://github.com/apache/spark/commit/503b263408924df42b26689a6f8b913a5185c10d). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-63822505 [Test build #23673 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23673/consoleFull) for PR 3386 at commit [`503b263`](https://github.com/apache/spark/commit/503b263408924df42b26689a6f8b913a5185c10d). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Sort(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-63822515 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23673/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
