[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62715/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62715/consoleFull)** for PR 13373 at commit [`bf74b0e`](https://github.com/apache/spark/commit/bf74b0e7c76eb2462183e76c3e3c4f8405ff82f1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHE...
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14116#discussion_r71834268
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/systemcatalog/InformationSchema.scala
---
@@ -0,0 +1,312 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.systemcatalog
+
+import scala.collection.mutable.ArrayBuffer
+
+import org.apache.spark.rdd.RDD
+import org.apache.spark.sql._
+import
org.apache.spark.sql.catalyst.catalog.SessionCatalog.{DEFAULT_DATABASE,
INFORMATION_SCHEMA_DATABASE}
+import org.apache.spark.sql.catalyst.dsl.plans._
+import org.apache.spark.sql.catalyst.expressions._
+import org.apache.spark.sql.catalyst.plans.logical.Project
+import org.apache.spark.sql.execution.datasources._
+import org.apache.spark.sql.internal.CatalogImpl
+import org.apache.spark.sql.sources._
+import org.apache.spark.sql.types._
+
+/**
+ * INFORMATION_SCHEMA is a database consisting views which provide
information about all of the
+ * tables, views, columns in a database.
+ *
+ * These views are designed to be populated by this package in order to be
independent from
+ * Spark catalog. To keep minimal dependency, currently INFORMATION_SCHEMA
views are implemented as
+ * Spark temporary views with a database prefix:
`SessionCatalog.INFORMATION_SCHEMA_DATABASE`.
+ *
+ * The following is the class hierarchy in this package rooted at
InformationSchemaRelationProvider.
+ *
+ * InformationSchemaRelationProvider
+ * -> DatabasesRelationProvider
+ * -> TablesRelationProvider
+ * -> ViewsRelationProvider
+ * -> ColumnsRelationProvider
+ * -> SessionVariablesRelationProvider
+ */
+
+/**
+ * InformationSchema object provides bootstrap and utility functions.
+ */
+object InformationSchema {
+
+ /**
+ * Register INFORMATION_SCHEMA database. SessionCatalog.catalog invokes
this function.
+ */
+ def registerInformationSchema(sparkSession: SparkSession): Unit = {
+sparkSession.sql(s"CREATE DATABASE IF NOT EXISTS
$INFORMATION_SCHEMA_DATABASE")
+registerView(sparkSession, new DatabasesRelationProvider,
Seq("schemata", "databases"))
+registerView(sparkSession, new TablesRelationProvider, Seq("tables"))
+registerView(sparkSession, new ViewsRelationProvider, Seq("views"))
+registerView(sparkSession, new ColumnsRelationProvider, Seq("columns"))
+registerView(sparkSession, new SessionVariablesRelationProvider,
Seq("session_variables"))
+ }
+
+ /**
+ * Register an INFORMATION_SCHEMA relation provider as a temporary view
of Spark Catalog.
+ */
+ private def registerView(
+ sparkSession: SparkSession,
+ relationProvider: SchemaRelationProvider,
+ names: Seq[String]) {
+val plan =
+
LogicalRelation(relationProvider.createRelation(sparkSession.sqlContext, null,
null)).analyze
+val projectList = plan.output.zip(plan.schema).map {
+ case (attr, col) => Alias(attr, col.name)()
+}
+sparkSession.sessionState.executePlan(Project(projectList, plan))
+
+for (name <- names) {
+
sparkSession.sessionState.catalog.createTempView(s"$INFORMATION_SCHEMA_DATABASE.$name",
+plan, overrideIfExists = true)
+}
+ }
+
+ /**
+ * Only EqualTo filters are handled in INFORMATION_SCHEMA data sources.
+ */
+ def unhandledFilters(filters: Array[Filter], columnName: String):
Array[Filter] = {
+import org.apache.spark.sql.sources.EqualTo
+filters.filter {
+ case EqualTo(attribute, _) if attribute.equalsIgnoreCase(columnName)
=> false
+ case _ => true
+}
+ }
+
+ /**
+ * Return `EqualTo` filtered DataFrame.
+ */
+ def getFilteredTables(sparkSession: SparkSession, filters:
Seq[Expression], columnName: String)
+ : DataFrame = {
+import org.apache
[GitHub] spark pull request #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHE...
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14116#discussion_r71834208
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/systemcatalog/InformationSchema.scala
---
@@ -0,0 +1,312 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.systemcatalog
+
+import scala.collection.mutable.ArrayBuffer
+
+import org.apache.spark.rdd.RDD
+import org.apache.spark.sql._
+import
org.apache.spark.sql.catalyst.catalog.SessionCatalog.{DEFAULT_DATABASE,
INFORMATION_SCHEMA_DATABASE}
+import org.apache.spark.sql.catalyst.dsl.plans._
+import org.apache.spark.sql.catalyst.expressions._
+import org.apache.spark.sql.catalyst.plans.logical.Project
+import org.apache.spark.sql.execution.datasources._
+import org.apache.spark.sql.internal.CatalogImpl
+import org.apache.spark.sql.sources._
+import org.apache.spark.sql.types._
+
+/**
+ * INFORMATION_SCHEMA is a database consisting views which provide
information about all of the
+ * tables, views, columns in a database.
+ *
+ * These views are designed to be populated by this package in order to be
independent from
+ * Spark catalog. To keep minimal dependency, currently INFORMATION_SCHEMA
views are implemented as
+ * Spark temporary views with a database prefix:
`SessionCatalog.INFORMATION_SCHEMA_DATABASE`.
+ *
+ * The following is the class hierarchy in this package rooted at
InformationSchemaRelationProvider.
+ *
+ * InformationSchemaRelationProvider
+ * -> DatabasesRelationProvider
+ * -> TablesRelationProvider
+ * -> ViewsRelationProvider
+ * -> ColumnsRelationProvider
+ * -> SessionVariablesRelationProvider
+ */
+
+/**
+ * InformationSchema object provides bootstrap and utility functions.
+ */
+object InformationSchema {
+
+ /**
+ * Register INFORMATION_SCHEMA database. SessionCatalog.catalog invokes
this function.
+ */
+ def registerInformationSchema(sparkSession: SparkSession): Unit = {
+sparkSession.sql(s"CREATE DATABASE IF NOT EXISTS
$INFORMATION_SCHEMA_DATABASE")
+registerView(sparkSession, new DatabasesRelationProvider,
Seq("schemata", "databases"))
+registerView(sparkSession, new TablesRelationProvider, Seq("tables"))
+registerView(sparkSession, new ViewsRelationProvider, Seq("views"))
+registerView(sparkSession, new ColumnsRelationProvider, Seq("columns"))
+registerView(sparkSession, new SessionVariablesRelationProvider,
Seq("session_variables"))
+ }
+
+ /**
+ * Register an INFORMATION_SCHEMA relation provider as a temporary view
of Spark Catalog.
+ */
+ private def registerView(
+ sparkSession: SparkSession,
+ relationProvider: SchemaRelationProvider,
+ names: Seq[String]) {
+val plan =
+
LogicalRelation(relationProvider.createRelation(sparkSession.sqlContext, null,
null)).analyze
--- End diff --
Yep.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14315: [HOTFIX][BUILD][SPARK-16287][SQL] Fix annotation argumen...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14315 I remember that I did the same mistake and @jaceklaskowski helped at that time. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62713/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62713/consoleFull)** for PR 13373 at commit [`b573919`](https://github.com/apache/spark/commit/b5739193ecb4c50aacc9d450f37e7be0d49939c7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14296: [SPARK-16639][SQL] The query with having conditio...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/14296#discussion_r71832244
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1202,11 +1203,16 @@ class Analyzer(
if (resolvedOperator.resolved) {
// Try to replace all aggregate expressions in the filter by
an alias.
val aggregateExpressions = ArrayBuffer.empty[NamedExpression]
-val transformedAggregateFilter =
resolvedAggregateFilter.transform {
+val transformedAggregateFilter =
resolvedAggregateFilter.transformDown {
case ae: AggregateExpression =>
val alias = Alias(ae, ae.toString)()
aggregateExpressions += alias
alias.toAttribute
+ case ne: NamedExpression => ne
+ case e: Expression =>
+val alias = Alias(e, e.toString)()
+aggregateExpressions += alias
+alias.toAttribute
--- End diff --
Looks like pushing the whole filter condition brings more problem.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14259: [SPARK-16622][SQL] Fix NullPointerException when the ret...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14259 **[Test build #62720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62720/consoleFull)** for PR 14259 at commit [`57c7fb6`](https://github.com/apache/spark/commit/57c7fb6f5781184c6f636fda03cba440229de26b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14315: [HOTFIX][BUILD][SPARK-16287][SQL] Fix annotation argumen...
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/14315 I'm on Java 8. I bet you are not. True? Also, I remembered how to fix it since this kind of error happened in the past few times. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14314: [SPARK-16678] [SPARK-16677] [SQL] Fix two View-related b...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14314 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62712/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14314: [SPARK-16678] [SPARK-16677] [SQL] Fix two View-related b...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14314 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14314: [SPARK-16678] [SPARK-16677] [SQL] Fix two View-related b...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14314 **[Test build #62712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62712/consoleFull)** for PR 14314 at commit [`5c9be14`](https://github.com/apache/spark/commit/5c9be1438e57722dbcea7a77993df636394cd9bf). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14204: [SPARK-16520] [WEBUI] Link executors to correspon...
Github user nblintao commented on a diff in the pull request:
https://github.com/apache/spark/pull/14204#discussion_r71830047
--- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala
---
@@ -73,13 +73,15 @@ class JsonProtocolSuite extends SparkFunSuite {
BlockManagerId("Scarce", "to be counted...", 100))
val unpersistRdd = SparkListenerUnpersistRDD(12345)
val logUrlMap = Map("stderr" -> "mystderr", "stdout" ->
"mystdout").toMap
+val workerUrlMap = Map("url" -> "spark://Worker@192.168.1.104:32790",
+ "ui_url" -> "http://192.168.1.104:46445";).toMap
--- End diff --
Good idea. I've updated referring to the test cases in `ClientSuite.scala`.
Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14204: [SPARK-16520] [WEBUI] Link executors to corresponding wo...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14204 **[Test build #62719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62719/consoleFull)** for PR 14204 at commit [`48ae86f`](https://github.com/apache/spark/commit/48ae86f630465ff134da792eeba2007966b5cecf). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14315: [HOTFIX][BUILD][SPARK-16287][SQL] Fix annotation argumen...
Github user techaddict commented on the issue: https://github.com/apache/spark/pull/14315 @jaceklaskowski thanks for finding this out. Its weird it passed locally too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14298: [SPARK-16283][SQL] Implement `percentile_approx` SQL fun...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14298 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62706/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14298: [SPARK-16283][SQL] Implement `percentile_approx` SQL fun...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14298 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14298: [SPARK-16283][SQL] Implement `percentile_approx` SQL fun...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14298 **[Test build #62706 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62706/consoleFull)** for PR 14298 at commit [`da87bfd`](https://github.com/apache/spark/commit/da87bfdb0ab91a265f74df01ec87221651583919). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class StringToMap(text: Expression, pairDelim: Expression, keyValueDelim: Expression)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14302: [SPARK-16663][SQL] desc table should be consistent betwe...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14302 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62709/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14302: [SPARK-16663][SQL] desc table should be consistent betwe...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14302 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14302: [SPARK-16663][SQL] desc table should be consistent betwe...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14302 **[Test build #62709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62709/consoleFull)** for PR 14302 at commit [`1ffa49d`](https://github.com/apache/spark/commit/1ffa49d6f2b7e7ac4114e215b56002feaf4fde09). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14313: [SPARK-16674][SQL] Avoid per-record type dispatch in JDB...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14313 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62708/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14313: [SPARK-16674][SQL] Avoid per-record type dispatch in JDB...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14313 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14313: [SPARK-16674][SQL] Avoid per-record type dispatch in JDB...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14313 **[Test build #62708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62708/consoleFull)** for PR 14313 at commit [`5eae0e6`](https://github.com/apache/spark/commit/5eae0e6b7d68a7781b1849441a000cf3ff7fe804). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14315: [HOTFIX][BUILD][SPARK-16287][SQL] Fix annotation argumen...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14315 I'm wondering why that PR passed all tests... LGTM, pending jenkins, thanks for finding this out! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14210: [SPARK-16556] [SPARK-16559] [SQL] Fix Two Bugs in...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14210 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14315: [HOTFIX][BUILD][SPARK-16287][SQL] Fix annotation argumen...
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/14315 /cc @cloud-fan @techaddict --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14210: [SPARK-16556] [SPARK-16559] [SQL] Fix Two Bugs in Bucket...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14210 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14315: [HOTFIX][BUILD][SPARK-16287][SQL] Fix annotation argumen...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14315 **[Test build #62718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62718/consoleFull)** for PR 14315 at commit [`ee34380`](https://github.com/apache/spark/commit/ee34380a532ca8b5df5f938a0eba51c94a973dff). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62704/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62704/consoleFull)** for PR 13373 at commit [`4a3e72e`](https://github.com/apache/spark/commit/4a3e72e6321fcbc449eb2fad3c903bd790cea2cb). * This patch **fails Spark unit tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14315: [HOTFIX] Fix annotation argument needs to be a co...
GitHub user jaceklaskowski opened a pull request:
https://github.com/apache/spark/pull/14315
[HOTFIX] Fix annotation argument needs to be a constant
## What changes were proposed in this pull request?
Fix for compilation error:
```
/Users/jacek/dev/oss/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala:402:
error: annotation argument needs to be a constant; found: "_FUNC_(text[,
pairDelim, keyValueDelim]) - Creates a map after splitting the text ".+("into
key/value pairs using delimiters. ").+("Default delimiters are \',\' for
pairDelim and \':\' for keyValueDelim.")
"into key/value pairs using delimiters. " +
^
```
## How was this patch tested?
Local build
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jaceklaskowski/spark
build-fix-complexTypeCreator
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14315.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14315
commit ee34380a532ca8b5df5f938a0eba51c94a973dff
Author: Jacek Laskowski
Date: 2016-07-22T05:24:08Z
[HOTFIX] Fix annotation argument needs to be a constant
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14210: [SPARK-16556] [SPARK-16559] [SQL] Fix Two Bugs in...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14210#discussion_r71828372
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/sources/CreateTableAsSelectSuite.scala
---
@@ -212,7 +213,23 @@ class CreateTableAsSelectSuite extends DataSourceTest
with SharedSQLContext with
)
val table = catalog.getTableMetadata(TableIdentifier("t"))
assert(DDLUtils.getBucketSpecFromTableProperties(table) ==
-Some(BucketSpec(5, Seq("a"), Seq("b"
+Option(BucketSpec(5, Seq("a"), Seq("b"
+}
+ }
+
+ test("create table using as select - with zero buckets") {
+withTable("t") {
+ val e = intercept[AnalysisException] {
+sql(
+ s"""
+ |CREATE TABLE t USING PARQUET
+ |OPTIONS (PATH '${path.toString}')
+ |CLUSTERED BY (a) SORTED BY (b) INTO 0 BUCKETS
--- End diff --
hm? so we allow `CLUSTERED BY` when schema is not specified for data source
table?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14313: [SPARK-16674][SQL] Avoid per-record type dispatch in JDB...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14313 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62705/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14313: [SPARK-16674][SQL] Avoid per-record type dispatch in JDB...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14313 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14314: [SPARK-16678] [SPARK-16677] [SQL] Fix two View-related b...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14314 **[Test build #62717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62717/consoleFull)** for PR 14314 at commit [`c64092c`](https://github.com/apache/spark/commit/c64092c6aaec42663278343a27467c1c8c165b92). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14313: [SPARK-16674][SQL] Avoid per-record type dispatch in JDB...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14313 **[Test build #62705 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62705/consoleFull)** for PR 14313 at commit [`ad64483`](https://github.com/apache/spark/commit/ad64483e2263485f843a469310fb8d252824d09e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14285: [SPARK-16649][SQL] Push partition predicates down into m...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14285 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62703/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14285: [SPARK-16649][SQL] Push partition predicates down into m...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14285 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14285: [SPARK-16649][SQL] Push partition predicates down into m...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14285 **[Test build #62703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62703/consoleFull)** for PR 14285 at commit [`de3ca68`](https://github.com/apache/spark/commit/de3ca686ce54109dc3564de75b6cb6411ead7d5e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14314: [SPARK-16678] [SPARK-16677] [SQL] Fix two View-related b...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14314 **[Test build #62716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62716/consoleFull)** for PR 14314 at commit [`c698a4a`](https://github.com/apache/spark/commit/c698a4af68907f8f9eb874ae6bd0b69d7e9fee59). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62715/consoleFull)** for PR 13373 at commit [`bf74b0e`](https://github.com/apache/spark/commit/bf74b0e7c76eb2462183e76c3e3c4f8405ff82f1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14132: [SPARK-16475][SQL] Broadcast Hint for SQL Queries
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14132 **[Test build #62714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62714/consoleFull)** for PR 14132 at commit [`2ee7d21`](https://github.com/apache/spark/commit/2ee7d21889490f05b591a276fa33df96693caf05). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14314: [SPARK-16678] [SPARK-16677] [SQL] Fix two View-re...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14314#discussion_r71827097
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ---
@@ -105,6 +105,12 @@ case class CreateViewCommand(
val tableIdentifier = tableDesc.identifier.copy(database =
Option(database))
if (sessionState.catalog.tableExists(tableIdentifier)) {
+val tableMetadata =
sessionState.catalog.getTableMetadata(tableIdentifier)
+if (tableMetadata.tableType != CatalogTableType.VIEW) {
+ throw new AnalysisException(
--- End diff --
Yeah, a good question. Just did a check using Hive. It is allowed by Hive.
```
hive> CREATE VIEW IF NOT EXISTS tab1 AS SELECT * FROM t1;
OK
Time taken: 0.678 seconds
```
Let me move this a little bit down.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62713/consoleFull)** for PR 13373 at commit [`b573919`](https://github.com/apache/spark/commit/b5739193ecb4c50aacc9d450f37e7be0d49939c7). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user lianhuiwang commented on the issue: https://github.com/apache/spark/pull/13373 cc @cloud-fan @rxin @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14285: [SPARK-16649][SQL] Push partition predicates down into m...
Github user lianhuiwang commented on the issue: https://github.com/apache/spark/pull/14285 cc @cloud-fan @rxin @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14210: [SPARK-16556] [SPARK-16559] [SQL] Fix Two Bugs in...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14210#discussion_r71826808
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -344,6 +344,11 @@ class SparkSqlAstBuilder(conf: SQLConf) extends
AstBuilder {
table, provider, partitionColumnNames, bucketSpec, mode, options,
query)
} else {
val struct = Option(ctx.colTypeList()).map(createStructType)
+ if (struct.isEmpty && bucketSpec.nonEmpty) {
+throw new ParseException(
+ "Expected explicit specification of table schema when using
CLUSTERED BY clause.", ctx)
--- End diff --
`DataFrameWriter` does not allow it.
- First, `save()` API has an
[`assertNotBucketed("save")`](https://github.com/apache/spark/blob/864b764eafa57a1418b683ccf6899b01bab28fba/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala#L203)
- Second, `insertInto(tableName: String)` API also has an
[`assertNotBucketed("save")`](https://github.com/apache/spark/blob/864b764eafa57a1418b683ccf6899b01bab28fba/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala#L243)
- Third, `saveAsTable(tableName: String)` API relies on
`CreateTableUsingAsSelect`. This will be converted to
`CreateDataSourceTableAsSelectCommand`. It also specifies the schema
[here](https://github.com/apache/spark/blob/ce3b98bae28af72299722f56e4e4ef831f471ec0/sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala#L259)
I think we are pretty safe now. Please let me know if we still should move
it? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14132: [SPARK-16475][SQL] Broadcast Hint for SQL Queries
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14132#discussion_r71826514
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/catalyst/SQLBuilder.scala ---
@@ -193,6 +196,9 @@ class SQLBuilder(logicalPlan: LogicalPlan) extends
Logging {
case OneRowRelation =>
""
+case Hint(_, _, child) =>
--- End diff --
I remember the reason. Originally, I designed `Hint` is removed by some
rules of `Analyzer` or `Optimizer`. But, now, it's changed. Only rules of
`Analyzer` handles that and removes all unknown hints. `CheckAnalysis` is the
correct position.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14314: [SPARK-16678] [SPARK-16677] [SQL] Fix two View-re...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/14314#discussion_r71825974
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ---
@@ -105,6 +105,12 @@ case class CreateViewCommand(
val tableIdentifier = tableDesc.identifier.copy(database =
Option(database))
if (sessionState.catalog.tableExists(tableIdentifier)) {
+val tableMetadata =
sessionState.catalog.getTableMetadata(tableIdentifier)
+if (tableMetadata.tableType != CatalogTableType.VIEW) {
+ throw new AnalysisException(
--- End diff --
If allowing existing, do we still need to throw this excepion?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14132: [SPARK-16475][SQL] Broadcast Hint for SQL Queries
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14132#discussion_r71825927
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/catalyst/SQLBuilder.scala ---
@@ -193,6 +196,9 @@ class SQLBuilder(logicalPlan: LogicalPlan) extends
Logging {
case OneRowRelation =>
""
+case Hint(_, _, child) =>
--- End diff --
Ur, it's added by previous advice comments.
No problem. I'll move that into `CheckAnalysis`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14259: [SPARK-16622][SQL] Fix NullPointerException when ...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14259#discussion_r71825821
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -314,4 +343,14 @@ class DatasetAggregatorSuite extends QueryTest with
SharedSQLContext {
val ds3 = sql("SELECT 'Some String' AS b, 1279869254 AS a").as[AggData]
assert(ds3.select(NameAgg.toColumn).schema.head.nullable === true)
}
+
+ test("Aggregator on empty dataset") {
--- End diff --
we can create a `ObjectExpressionsSuite` under
org.apache.spark.sql.catalyst.expressions
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14132: [SPARK-16475][SQL] Broadcast Hint for SQL Queries
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14132#discussion_r71825810
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -339,8 +339,11 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef]
with Logging {
case SqlBaseParser.SELECT =>
// Regular select
+// Add hints.
+val withHint = relation.optionalMap(ctx.hint)(withHints)
--- End diff --
Yep. I grasp your point. First, the given `relation` is just a logical
plan, not a table. In the following case, the `relation` is `Join`.
```
SELECT /*+ MAPJOIN(parquet_t0) */ * FROM parquet_t0, parquet_t1
```
This means just a context.
Second, I agree with your point. Generally, `hint` is about tables like
join order or access path. We can provides some hints on filter conditions, I'm
not sure about the cases so far. But, even for that cases, we can adapt that in
`Analyzer` easily. In other words, we can identify `Hint("FILTERHINT")` in the
same SPJ.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14290: [SPARK-16657] [SQL] Replace children by innerChildren in...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14290 The analyzer resolution requires them as children. Although this is inconsistent with the other similar nodes, we still can tolerate it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14290: [SPARK-16657] [SQL] Replace children by innerChil...
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/14290 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14314: [SPARK-16678] [SPARK-16677] Fix two View-related bugs
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14314 **[Test build #62712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62712/consoleFull)** for PR 14314 at commit [`5c9be14`](https://github.com/apache/spark/commit/5c9be1438e57722dbcea7a77993df636394cd9bf). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14259: [SPARK-16622][SQL] Fix NullPointerException when ...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/14259#discussion_r71825205
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -314,4 +343,14 @@ class DatasetAggregatorSuite extends QueryTest with
SharedSQLContext {
val ds3 = sql("SELECT 'Some String' AS b, 1279869254 AS a").as[AggData]
assert(ds3.select(NameAgg.toColumn).schema.head.nullable === true)
}
+
+ test("Aggregator on empty dataset") {
--- End diff --
Another problem is this test should not be in `DatasetAggregatorSuite`. I
don't find a proper file for this test. Which one you would suggest?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14314: [SPARK-16678] [SPARK-16677] Fix two View-related ...
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/14314 [SPARK-16678] [SPARK-16677] Fix two View-related bugs ### What changes were proposed in this pull request? **Issue 1: Disallow Creating a View when the same-name Table Exists** When we create a view, we check whether the view already exists. In the current implementation, if a table with the same name exists, we treat it as a view. However, this is not the right behavior. We should follow what Hive does. For example, ``` hive> CREATE TABLE tab1 (id int); OK Time taken: 0.196 seconds hive> CREATE OR REPLACE VIEW tab1 AS SELECT * FROM t1; FAILED: SemanticException [Error 10218]: Existing table is not a view The following is an existing table, not a view: default.tab1 ``` **Issue 2: Strange Error when Issuing Load Table Against A View** Users should not be allowed to issue LOAD DATA against a view. Currently, when users doing it, we got a very strange runtime error. For example, ```SQL LOAD DATA LOCAL INPATH "$testData" INTO TABLE $viewName ``` ``` java.lang.reflect.InvocationTargetException was thrown. java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.sql.hive.client.Shim_v0_14.loadTable(HiveShim.scala:680) ``` ### How was this patch tested? Added test cases You can merge this pull request into a Git repository by running: $ git pull https://github.com/gatorsmile/spark tableDDLAgainstView Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14314.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14314 commit d92a95cee8572f631813f259620cc6fe2644ed36 Author: gatorsmile Date: 2016-07-22T03:48:50Z fix. commit 5c9be1438e57722dbcea7a77993df636394cd9bf Author: gatorsmile Date: 2016-07-22T04:13:28Z changed the test case names. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14292: [SPARK-14131][SQL[STREAMING] Improved fix for avoiding p...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14292 **[Test build #3189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3189/consoleFull)** for PR 14292 at commit [`0e67e26`](https://github.com/apache/spark/commit/0e67e26c0690dfecb011c872f58baf87e1bde3c0). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62711/consoleFull)** for PR 13373 at commit [`1e0a6f2`](https://github.com/apache/spark/commit/1e0a6f296a73822b250ab4d4e9a35d4bec129765). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62711/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62711/consoleFull)** for PR 13373 at commit [`1e0a6f2`](https://github.com/apache/spark/commit/1e0a6f296a73822b250ab4d4e9a35d4bec129765). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHE...
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14116#discussion_r71823904
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -484,8 +521,11 @@ class SessionCatalog(
val dbTables =
externalCatalog.listTables(dbName, pattern).map { t =>
TableIdentifier(t, Some(dbName)) }
synchronized {
- val _tempTables = StringUtils.filterPattern(tempTables.keys.toSeq,
pattern)
+ var _tempTables = StringUtils.filterPattern(tempTables.keys.toSeq,
pattern)
.map { t => TableIdentifier(t) }
+ if (db != INFORMATION_SCHEMA_DATABASE) {
--- End diff --
This is the current result.
```
scala> sql("use information_schema")
res11: org.apache.spark.sql.DataFrame = []
scala> sql("show tables").show(false)
++---+
|tableName |isTemporary|
++---+
|information_schema.columns |true |
|information_schema.databases|true |
|information_schema.schemata |true |
|information_schema.session_variables|true |
|information_schema.tables |true |
|information_schema.views|true |
++---+
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHE...
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14116#discussion_r71823733
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -471,6 +480,34 @@ class SessionCatalog(
}
/**
+ * Normalize TableIdentifier by consistently ensuring the following two
rules.
+ * 1. System-side temporary tables should have None as database.
+ * 2. System-side temporary tables should have prefixed table names.
+ * Currently, only INFORMATION_SCHEMA tables are system-side temporary
tables, and this function
+ * returns TableIdentifier("information_schema.databases", None).
+ */
+ protected def normalizeTableIdentifier(name: TableIdentifier):
TableIdentifier = synchronized {
+if (name.database.isDefined) {
+ if (name.database.contains(INFORMATION_SCHEMA_DATABASE)) {
+TableIdentifier(s"$INFORMATION_SCHEMA_DATABASE.${name.table}",
None)
+ } else {
+name
+ }
+} else {
+ val tableName = formatTableName(name.table)
+ if (tableName.startsWith(INFORMATION_SCHEMA_DATABASE + ".")) {
--- End diff --
Backtick-quoted one will not reach here.
```
scala> sql("create table `aaa.bbb`(a int)")
org.apache.spark.sql.AnalysisException:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException: [aaa.bbb]: is not a valid
table name;
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62710/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62710/consoleFull)** for PR 13373 at commit [`2d9e321`](https://github.com/apache/spark/commit/2d9e32142748ebb92d27f23609607d6395b512b1). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14297: [SPARK-16660][SQL] CreateViewCommand should not take Cat...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14297 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62701/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14297: [SPARK-16660][SQL] CreateViewCommand should not take Cat...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14297 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14132: [SPARK-16475][SQL] Broadcast Hint for SQL Queries
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14132#discussion_r71823645
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/catalyst/SQLBuilder.scala ---
@@ -193,6 +196,9 @@ class SQLBuilder(logicalPlan: LogicalPlan) extends
Logging {
case OneRowRelation =>
""
+case Hint(_, _, child) =>
--- End diff --
so why do we have this case? `SQLBuilder` will be applied on analyzed plan
right?
And I think it's better to push this check in `CheckAnalysis`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14132: [SPARK-16475][SQL] Broadcast Hint for SQL Queries
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14132#discussion_r71823613
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -339,8 +339,11 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef]
with Logging {
case SqlBaseParser.SELECT =>
// Regular select
+// Add hints.
+val withHint = relation.optionalMap(ctx.hint)(withHints)
--- End diff --
for now the hint can only affect the relations(broadcast hint), so it's ok
to just put the `Hint` on top of relation. But generally hint can affect
anything in the select clause, e.g. maybe the filter condition.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14302: [SPARK-16663][SQL] desc table should be consistent betwe...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14302 **[Test build #62709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62709/consoleFull)** for PR 14302 at commit [`1ffa49d`](https://github.com/apache/spark/commit/1ffa49d6f2b7e7ac4114e215b56002feaf4fde09). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62710/consoleFull)** for PR 13373 at commit [`2d9e321`](https://github.com/apache/spark/commit/2d9e32142748ebb92d27f23609607d6395b512b1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14297: [SPARK-16660][SQL] CreateViewCommand should not take Cat...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14297 **[Test build #62701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62701/consoleFull)** for PR 14297 at commit [`5072959`](https://github.com/apache/spark/commit/50729594c1956d185bde4c2b41891ce1567cfd5a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62707/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62707/consoleFull)** for PR 13373 at commit [`f3da998`](https://github.com/apache/spark/commit/f3da9983d79c6bf070cba3ccd863890f1b4c2233). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13373 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14313: [SPARK-16674][SQL] Avoid per-record type dispatch in JDB...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14313 **[Test build #62708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62708/consoleFull)** for PR 14313 at commit [`5eae0e6`](https://github.com/apache/spark/commit/5eae0e6b7d68a7781b1849441a000cf3ff7fe804). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14307: [SPARK-16672][SQL] SQLBuilder should not raise exception...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14307 Hi, @hvanhovell . Could you review this PR about `SQLBuilder`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14204: [SPARK-16520] [WEBUI] Link executors to corresponding wo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14204 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14132: [SPARK-16475][SQL] Broadcast Hint for SQL Queries
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14132#discussion_r71823152
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/catalyst/SQLBuilder.scala ---
@@ -193,6 +196,9 @@ class SQLBuilder(logicalPlan: LogicalPlan) extends
Logging {
case OneRowRelation =>
""
+case Hint(_, _, child) =>
--- End diff --
Also, `SparkStrategies` ensures that here
https://github.com/apache/spark/pull/14132/files#diff-7253a38df7e111ecf6b1ef71feba383bR347
, too.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13373: [SPARK-15616] [SQL] Metastore relation should fallback t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13373 **[Test build #62707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62707/consoleFull)** for PR 13373 at commit [`f3da998`](https://github.com/apache/spark/commit/f3da9983d79c6bf070cba3ccd863890f1b4c2233). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14204: [SPARK-16520] [WEBUI] Link executors to corresponding wo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62700/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14204: [SPARK-16520] [WEBUI] Link executors to corresponding wo...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14204 **[Test build #62700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62700/consoleFull)** for PR 14204 at commit [`519c329`](https://github.com/apache/spark/commit/519c32974d3239d5780fb94064126b4a0737c656). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14132: [SPARK-16475][SQL] Broadcast Hint for SQL Queries
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14132#discussion_r71823079
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/catalyst/SQLBuilder.scala ---
@@ -193,6 +196,9 @@ class SQLBuilder(logicalPlan: LogicalPlan) extends
Logging {
case OneRowRelation =>
""
+case Hint(_, _, child) =>
--- End diff --
After applying the rules of `Analyzer`, there will be no more `Hint` case
class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14132: [SPARK-16475][SQL] Broadcast Hint for SQL Queries
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14132#discussion_r71822992
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -339,8 +339,11 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef]
with Logging {
case SqlBaseParser.SELECT =>
// Regular select
+// Add hints.
+val withHint = relation.optionalMap(ctx.hint)(withHints)
--- End diff --
Ur, could you give some example what you concern?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14298: [SPARK-16283][SQL] Implement `percentile_approx` SQL fun...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14298 **[Test build #62706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62706/consoleFull)** for PR 14298 at commit [`da87bfd`](https://github.com/apache/spark/commit/da87bfdb0ab91a265f74df01ec87221651583919). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14132: [SPARK-16475][SQL] Broadcast Hint for SQL Queries
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14132#discussion_r71822903
--- Diff:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -347,6 +347,16 @@ querySpecification
windows?)
;
+hint
+: '/*+' hintStatement '*/'
+;
+
+hintStatement
+: hintName=identifier
+| hintName=identifier '(' parameters+=identifier
parameters+=identifier ')'
--- End diff --
Thank you for review, @cloud-fan .
The first goal of this PR provides a general syntax for hints, not only
broadcast hints. The `(` and `)` syntax is for `INDEX(t idx_emp)` style. You
can see the testcase for this in the testcase, too.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14313: [SPARK-16674][SQL] Avoid per-record type dispatch in JDB...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14313 **[Test build #62705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62705/consoleFull)** for PR 14313 at commit [`ad64483`](https://github.com/apache/spark/commit/ad64483e2263485f843a469310fb8d252824d09e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14313: [SPARK-16674][SQL] Avoid per-record type dispatch in JDB...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14313 Could you please take a look here @cloud-fan and @yhuai ? This is happening for writing too. I will open new one for writing as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14313: [SPARK-16674][SQL] Avoid per-record type dispatch...
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/14313 [SPARK-16674][SQL] Avoid per-record type dispatch in JDBC when reading ## What changes were proposed in this pull request? Currently, `JDBCRDD.compute` is doing type dispatch for each row to read appropriate values. It might not have to be done like this because the schema is already kept in `JDBCRDD`. So, appropriate converters can be created first according to the schema, and then apply them to each row. ## How was this patch tested? Existing tests should cover this. You can merge this pull request into a Git repository by running: $ git pull https://github.com/HyukjinKwon/spark SPARK-16674 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14313.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14313 commit ad64483e2263485f843a469310fb8d252824d09e Author: hyukjinkwon Date: 2016-07-22T03:32:44Z Avoid per-record type dispatch in JDBC when reading --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14298: [SPARK-16283][SQL] Implement `percentile_approx` SQL fun...
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/14298 @cloud-fan could you also help review this? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14210: [SPARK-16556] [SPARK-16559] [SQL] Fix Two Bugs in...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14210#discussion_r71822711
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -344,6 +344,11 @@ class SparkSqlAstBuilder(conf: SQLConf) extends
AstBuilder {
table, provider, partitionColumnNames, bucketSpec, mode, options,
query)
} else {
val struct = Option(ctx.colTypeList()).map(createStructType)
+ if (struct.isEmpty && bucketSpec.nonEmpty) {
+throw new ParseException(
+ "Expected explicit specification of table schema when using
CLUSTERED BY clause.", ctx)
--- End diff --
Yeah, let me find a place. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14297: [SPARK-16660][SQL] CreateViewCommand should not t...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14297#discussion_r71822672
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ---
@@ -88,23 +96,23 @@ case class CreateViewCommand(
qe.assertAnalyzed()
val analyzedPlan = qe.analyzed
-if (tableDesc.schema != Nil && tableDesc.schema.length !=
analyzedPlan.output.length) {
+if (userSpecifiedColumns.nonEmpty &&
--- End diff --
: ) Yeah, we should check it!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14296: [SPARK-16639][SQL] The query with having condition that ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14296 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
