[GitHub] [spark] AmplabJenkins commented on issue #24123: [SPARK-27184][CORE] Replace 'spark.jars' & 'spark.files' with the variables of this config object
AmplabJenkins commented on issue #24123: [SPARK-27184][CORE] Replace 'spark.jars' & 'spark.files' with the variables of this config object URL: https://github.com/apache/spark/pull/24123#issuecomment-475120643 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103749/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24123: [SPARK-27184][CORE] Replace 'spark.jars' & 'spark.files' with the variables of this config object
AmplabJenkins commented on issue #24123: [SPARK-27184][CORE] Replace 'spark.jars' & 'spark.files' with the variables of this config object URL: https://github.com/apache/spark/pull/24123#issuecomment-475120642 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24123: [SPARK-27184][CORE] Replace 'spark.jars' & 'spark.files' with the variables of this config object
SparkQA removed a comment on issue #24123: [SPARK-27184][CORE] Replace 'spark.jars' & 'spark.files' with the variables of this config object URL: https://github.com/apache/spark/pull/24123#issuecomment-475087603 **[Test build #103749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103749/testReport)** for PR 24123 at commit [`7a88f79`](https://github.com/apache/spark/commit/7a88f7982bb388913c5cf93a9f47a2bf1a96e745). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24123: [SPARK-27184][CORE] Replace 'spark.jars' & 'spark.files' with the variables of this config object
SparkQA commented on issue #24123: [SPARK-27184][CORE] Replace 'spark.jars' & 'spark.files' with the variables of this config object URL: https://github.com/apache/spark/pull/24123#issuecomment-475120444 **[Test build #103749 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103749/testReport)** for PR 24123 at commit [`7a88f79`](https://github.com/apache/spark/commit/7a88f7982bb388913c5cf93a9f47a2bf1a96e745). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted
AmplabJenkins commented on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted URL: https://github.com/apache/spark/pull/24154#issuecomment-475119556 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103751/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted
AmplabJenkins removed a comment on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted URL: https://github.com/apache/spark/pull/24154#issuecomment-475119556 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103751/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted
AmplabJenkins commented on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted URL: https://github.com/apache/spark/pull/24154#issuecomment-475119555 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted
AmplabJenkins removed a comment on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted URL: https://github.com/apache/spark/pull/24154#issuecomment-475119555 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted
SparkQA removed a comment on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted URL: https://github.com/apache/spark/pull/24154#issuecomment-475089697 **[Test build #103751 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103751/testReport)** for PR 24154 at commit [`8e12846`](https://github.com/apache/spark/commit/8e1284613244fc4d248b5189a5886d2304ae2457). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted
SparkQA commented on issue #24154: [SPARK-27210][SS] Cleanup incomplete output files in ManifestFileCommitProtocol if task is aborted URL: https://github.com/apache/spark/pull/24154#issuecomment-475119308 **[Test build #103751 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103751/testReport)** for PR 24154 at commit [`8e12846`](https://github.com/apache/spark/commit/8e1284613244fc4d248b5189a5886d2304ae2457). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support
AmplabJenkins removed a comment on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support URL: https://github.com/apache/spark/pull/24158#issuecomment-475118887 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103746/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support
AmplabJenkins commented on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support URL: https://github.com/apache/spark/pull/24158#issuecomment-475118887 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103746/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support
AmplabJenkins removed a comment on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support URL: https://github.com/apache/spark/pull/24158#issuecomment-475118884 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support
AmplabJenkins commented on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support URL: https://github.com/apache/spark/pull/24158#issuecomment-475118884 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support
viirya commented on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support URL: https://github.com/apache/spark/pull/24158#issuecomment-475118671 cc @cloud-fan @dongjoon-hyun @dbtsai This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support
SparkQA removed a comment on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support URL: https://github.com/apache/spark/pull/24158#issuecomment-475084764 **[Test build #103746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103746/testReport)** for PR 24158 at commit [`d388952`](https://github.com/apache/spark/commit/d3889529c306a2e386ee8ab875b3814594cb4a9b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support
SparkQA commented on issue #24158: [SPARK-26847][SQL] Pruning nested serializers from object serializers: MapType support URL: https://github.com/apache/spark/pull/24158#issuecomment-475118448 **[Test build #103746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103746/testReport)** for PR 24158 at commit [`d388952`](https://github.com/apache/spark/commit/d3889529c306a2e386ee8ab875b3814594cb4a9b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog
AmplabJenkins commented on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog URL: https://github.com/apache/spark/pull/23848#issuecomment-475117458 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog
AmplabJenkins removed a comment on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog URL: https://github.com/apache/spark/pull/23848#issuecomment-475117458 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog
AmplabJenkins commented on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog URL: https://github.com/apache/spark/pull/23848#issuecomment-475117459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103750/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog
AmplabJenkins removed a comment on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog URL: https://github.com/apache/spark/pull/23848#issuecomment-475117459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103750/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog
SparkQA commented on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog URL: https://github.com/apache/spark/pull/23848#issuecomment-475117213 **[Test build #103750 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103750/testReport)** for PR 23848 at commit [`3bb4485`](https://github.com/apache/spark/commit/3bb448503f35e9e035480401261702bcc3114e7a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog
SparkQA removed a comment on issue #23848: [SPARK-26946][SQL] Identifiers for multi-catalog URL: https://github.com/apache/spark/pull/23848#issuecomment-475087599 **[Test build #103750 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103750/testReport)** for PR 23848 at commit [`3bb4485`](https://github.com/apache/spark/commit/3bb448503f35e9e035480401261702bcc3114e7a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LantaoJin commented on a change in pull request #23951: [SPARK-13704][CORE][YARN] Re-implement RackResolver to reduce resolving time
LantaoJin commented on a change in pull request #23951:
[SPARK-13704][CORE][YARN] Re-implement RackResolver to reduce resolving time
URL: https://github.com/apache/spark/pull/23951#discussion_r267624353
##
File path: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala
##
@@ -192,17 +192,17 @@ private[spark] class TaskSetManager(
val (_, duration) = Utils.timeTakenMs {
val hostToIndices = new HashMap[String, ArrayBuffer[Int]]()
for (i <- (0 until numTasks).reverse) {
-addPendingTask(i, Option(hostToIndices))
+addPendingTask(i, Some(hostToIndices))
}
- // Convert preferred locations to racks in one invocation and zip with
the origin indices.
- // We de-duping the hosts to reduce this invocation further.
-
sched.getRacksForHosts(hostToIndices.keySet.toList).zip(hostToIndices.values)
foreach {
-case (Some(rack), indices) =>
- pendingTasksForRack.getOrElseUpdate(rack, new ArrayBuffer) ++=
indices
-case _ =>
+ // Resolve the rack for each host. This can be somehow slow, so de-dupe
the list of hosts,
Review comment:
will delete it
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LantaoJin commented on a change in pull request #23951: [SPARK-13704][CORE][YARN] Re-implement RackResolver to reduce resolving time
LantaoJin commented on a change in pull request #23951:
[SPARK-13704][CORE][YARN] Re-implement RackResolver to reduce resolving time
URL: https://github.com/apache/spark/pull/23951#discussion_r267624324
##
File path:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
##
@@ -375,9 +375,10 @@ private[spark] class TaskSchedulerImpl(
executorIdToRunningTaskIds(o.executorId) = HashSet[Long]()
newExecAvail = true
}
- for (rack <- getRackForHost(o.host)) {
-hostsByRack.getOrElseUpdate(rack, new HashSet[String]()) += o.host
- }
+}
+val hosts = offers.map(_.host)
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mccheah commented on a change in pull request #24029: [SPARK-27108][SQL] Add parsed SQL plans for create, CTAS.
mccheah commented on a change in pull request #24029: [SPARK-27108][SQL] Add parsed SQL plans for create, CTAS. URL: https://github.com/apache/spark/pull/24029#discussion_r267623814 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/sql/CreateTable.scala ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + *http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.spark.sql.catalyst.plans.logical.sql + +import org.apache.spark.sql.catalyst.TableIdentifier +import org.apache.spark.sql.catalyst.catalog.BucketSpec +import org.apache.spark.sql.catalyst.expressions.Attribute +import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan +import org.apache.spark.sql.types.StructType + +/** + * A CREATE TABLE command, as parsed from SQL. + * + * This is a metadata-only command and is not used to write data to the created table. + */ +case class CreateTable( Review comment: Just so I understand correctly, `DDLResolution` returns an instance of `org.apache.spark.sql.execution.datasources.CreateTable`, which is still a subtype of `LogicalPlan`. It's a little confusing at least to me to have two `LogicalPlan` types be named the same thing, but they represent different steps of the query planning, and are in different packages. What if we called this variant of `CreateTable`, say, making this `ParsedCreateTable` or perhaps `CreateTableStatement`? Also, when we start partitioning between V1 sources and V2 sources in the Analyzer, are we going to return the same type of `LogicalPlan` back - a `org.apache.spark.sql.execution.datasources.CreateTable` regardless of it's loading V1 or V2? Or should we instead be naming `org.apache.spark.sql.execution.datasources.CreateTableV1`, anticipating the Analyzer to return instances of `CreateTableV2` plans down the road? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LantaoJin edited a comment on issue #24157: [WIP][SPARK-27216][CORE] Kryo serialization with RoaringBitmap
LantaoJin edited a comment on issue #24157: [WIP][SPARK-27216][CORE] Kryo
serialization with RoaringBitmap
URL: https://github.com/apache/spark/pull/24157#issuecomment-475107799
@srowen @squito I've added another UT which is the minimized dataset from
our product issue.
In this UT, I roughly comment one line in `ShuffleBlockFetcherIterator` to
avoid job fail.
```scala
if (buf.size == 0) {
// throwFetchFailedException(blockId, address, new IOException(msg))
}
```
After that, the testing `fail zero-size blocks` in
`ShuffleBlockFetcherIteratorSuite` will fail. This was introduced by #21219. So
in Spark2.3.x, this UT doesn't need this hard code commenting.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mccheah commented on a change in pull request #24029: [SPARK-27108][SQL] Add parsed SQL plans for create, CTAS.
mccheah commented on a change in pull request #24029: [SPARK-27108][SQL] Add
parsed SQL plans for create, CTAS.
URL: https://github.com/apache/spark/pull/24029#discussion_r267621858
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/sql/ParsedLogicalPlan.scala
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.plans.logical.sql
+
+import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
+
+/**
+ * A logical plan node that contains exactly what was parsed from SQL.
+ *
+ * This is used to hold information parsed from SQL when there are multiple
implementations of a
+ * query or command. For example, CREATE TABLE may be implemented by different
nodes for v1 and v2.
+ * Instead of parsing directly to a v1 CreateTable that keeps metadata in
CatalogTable, and then
+ * converting that v1 metadata to the v2 equivalent, the sql [[CreateTable]]
plan is produced by
+ * the parser and converted once into both implementations.
+ *
+ * Parsed logical plans are not resolved because they must be converted to
concrete logical plans.
+ *
+ * Parsed logical plans are located in Catalyst so that as much SQL parsing
logic as possible is be
+ * kept in a [[org.apache.spark.sql.catalyst.parser.AbstractSqlParser]].
+ */
+private[sql] abstract class ParsedLogicalPlan extends LogicalPlan {
Review comment:
+1 - @rdblue these should only be inputs to the analyzer, not outputs. Would
be helpful to write specific JavaDoc on this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LantaoJin commented on issue #24157: [WIP][SPARK-27216][CORE] Kryo serialization with RoaringBitmap
LantaoJin commented on issue #24157: [WIP][SPARK-27216][CORE] Kryo serialization with RoaringBitmap URL: https://github.com/apache/spark/pull/24157#issuecomment-475107799 @srowen @squito I've added another UT which is the minimized dataset from our product issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24164: [SPARK-27225][SQL] Implement join strategy hints
AmplabJenkins removed a comment on issue #24164: [SPARK-27225][SQL] Implement join strategy hints URL: https://github.com/apache/spark/pull/24164#issuecomment-475104056 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24164: [SPARK-27225][SQL] Implement join strategy hints
AmplabJenkins removed a comment on issue #24164: [SPARK-27225][SQL] Implement join strategy hints URL: https://github.com/apache/spark/pull/24164#issuecomment-475104060 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9132/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24164: [SPARK-27225][SQL] Implement join strategy hints
SparkQA commented on issue #24164: [SPARK-27225][SQL] Implement join strategy hints URL: https://github.com/apache/spark/pull/24164#issuecomment-475104295 **[Test build #103754 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103754/testReport)** for PR 24164 at commit [`1426294`](https://github.com/apache/spark/commit/14262941d80f5f5b0075ee294769af3e81c2d7b6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] BryanCutler commented on issue #24095: [SPARK-27163][PYTHON] Cleanup and consolidate Pandas UDF functionality
BryanCutler commented on issue #24095: [SPARK-27163][PYTHON] Cleanup and consolidate Pandas UDF functionality URL: https://github.com/apache/spark/pull/24095#issuecomment-475104137 Apologies, I moved things around again for item (2) because I didn't really like having an option in `ArrowStreamPandasSerializer` to send the `START_ARROW_STREAM` either. Now, I have `_create_batch(...)` as a method in `ArrowStreamPandasSerializer` (where it belongs I think), and then have a subclass used for Pandas UDFs that overrides `dump_stream` that can send `START_ARROW_STREAM`. I think it's clearer this way because it's easier to see what serializer is used where, and I also tried to improve the docs. Let me know what you think when you get the chance to take another look @HyukjinKwon @ueshin . Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24164: [SPARK-27225][SQL] Implement join strategy hints
AmplabJenkins commented on issue #24164: [SPARK-27225][SQL] Implement join strategy hints URL: https://github.com/apache/spark/pull/24164#issuecomment-475104060 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9132/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24164: [SPARK-27225][SQL] Implement join strategy hints
AmplabJenkins commented on issue #24164: [SPARK-27225][SQL] Implement join strategy hints URL: https://github.com/apache/spark/pull/24164#issuecomment-475104056 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24153: [MINOR][EXAMPLES] Add missing return keyword streaming word count example
HyukjinKwon commented on issue #24153: [MINOR][EXAMPLES] Add missing return keyword streaming word count example URL: https://github.com/apache/spark/pull/24153#issuecomment-475104007 Yea, the change looked good. What I wanted to make sure was to keep the PR description format to make PR well-formed as guided in https://spark.apache.org/contributing.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maryannxue opened a new pull request #24164: [SPARK-27225][SQL] Implement join strategy hints
maryannxue opened a new pull request #24164: [SPARK-27225][SQL] Implement join
strategy hints
URL: https://github.com/apache/spark/pull/24164
## What changes were proposed in this pull request?
This PR extends the existing BROADCAST join hint (for both broadcast-hash
join and broadcast-nested-loop join) by implementing other join strategy hints
corresponding to the rest of Spark's existing join strategies: shuffle-hash,
sort-merge, cartesian-product. The hint names: SHUFFLE_MERGE, SHUFFLE_HASH,
SHUFFLE_REPLICATE_NL are partly different from the code names in order to make
them clearer to users and reflect the actual algorithms better.
The hinted strategy will be used for the join with which it is associated if
it is applicable/doable.
Conflict resolving rules in case of multiple hints:
1. Conflicts within either side of the join: take the first strategy hint
specified in the query, or the top hint node in Dataset. For example, in
"select /*+ merge(t1) */ /*+ broadcast(t1) */ k1, v2 from t1 join t2 on t1.k1 =
t2.k2", take "merge(t1)"; in
```df1.hint("merge").hint("shuffle_hash").join(df2)```, take "shuffle_hash".
This is a general hint conflict resolving strategy, not specific to join
strategy hint.
2. Conflicts between two sides of the join:
a) In case of different strategy hints, hints are prioritized as
```BROADCAST``` over ```SHUFFLE_MERGE``` over ```SHUFFLE_HASH``` over
```SHUFFLE_REPLICATE_NL```.
b) In case of same strategy hints but conflicts in build side, choose the
build side based on join type and size.
## How was this patch tested?
Added new UTs.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24047: [SPARK-25196][SQL] Extends the analyze column command for cached tables
dongjoon-hyun commented on a change in pull request #24047: [SPARK-25196][SQL]
Extends the analyze column command for cached tables
URL: https://github.com/apache/spark/pull/24047#discussion_r267616522
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala
##
@@ -39,32 +40,39 @@ case class AnalyzeColumnCommand(
require(columnNames.isDefined ^ allColumns, "Parameter `columnNames` or
`allColumns` are " +
"mutually exclusive. Only one of them should be specified.")
val sessionState = sparkSession.sessionState
-val db =
tableIdent.database.getOrElse(sessionState.catalog.getCurrentDatabase)
-val tableIdentWithDB = TableIdentifier(tableIdent.table, Some(db))
-val tableMeta = sessionState.catalog.getTableMetadata(tableIdentWithDB)
-if (tableMeta.tableType == CatalogTableType.VIEW) {
- throw new AnalysisException("ANALYZE TABLE is not supported on views.")
-}
-val sizeInBytes = CommandUtils.calculateTotalSize(sparkSession, tableMeta)
-val relation = sparkSession.table(tableIdent).logicalPlan
-val columnsToAnalyze = getColumnsToAnalyze(tableIdent, relation,
columnNames, allColumns)
-
-// Compute stats for the computed list of columns.
-val (rowCount, newColStats) =
- CommandUtils.computeColumnStats(sparkSession, relation, columnsToAnalyze)
-// We also update table-level stats in order to keep them consistent with
column-level stats.
-val statistics = CatalogStatistics(
- sizeInBytes = sizeInBytes,
- rowCount = Some(rowCount),
- // Newly computed column stats should override the existing ones.
- colStats = tableMeta.stats.map(_.colStats).getOrElse(Map.empty) ++
newColStats)
-
-sessionState.catalog.alterTableStats(tableIdentWithDB, Some(statistics))
+tableIdent.database match {
+ case Some(db) if db ==
sparkSession.sharedState.globalTempViewManager.database =>
+val plan =
sessionState.catalog.getGlobalTempView(tableIdent.identifier).getOrElse {
+ throw new NoSuchTableException(db = db, table =
tableIdent.identifier)
+}
+analyzeColumnInTempView(plan, sparkSession)
+ case Some(_) =>
+analyzeColumnInCatalog(sparkSession)
+ case None =>
+sessionState.catalog.getTempView(tableIdent.identifier) match {
+ case Some(tempView) => analyzeColumnInTempView(tempView,
sparkSession)
+ case _ => analyzeColumnInCatalog(sparkSession)
+}
+}
Seq.empty[Row]
}
+ private def analyzeColumnInTempView(plan: LogicalPlan, sparkSession:
SparkSession): Unit = {
+val cacheManager = sparkSession.sharedState.cacheManager
+cacheManager.lookupCachedData(plan) match {
+ case Some(cachedData) =>
+val columnsToAnalyze = getColumnsToAnalyze(
+ tableIdent, cachedData.plan, columnNames, allColumns)
+cacheManager.analyzeColumnCacheQuery(sparkSession, cachedData,
columnsToAnalyze)
+ case _ =>
+val catalog = sparkSession.sessionState.catalog
+val db = tableIdent.database.getOrElse(catalog.getCurrentDatabase)
+throw new NoSuchTableException(db = db, table = tableIdent.identifier)
Review comment:
Got it~
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats
dongjoon-hyun edited a comment on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats URL: https://github.com/apache/spark/pull/23803#issuecomment-475101858 Sure, @maropu . For the first contributor, you need to add him into `Apache Spark Contributor` group. 1. Visit Project Configuration Role Page - https://issues.apache.org/jira/plugins/servlet/project-config/SPARK/roles 2. Click `Add users to a role` button (at the right-top of the page). 3. Fill the the pop-up box. - The JIRA user ID or email - Choose `Contributor`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats
dongjoon-hyun edited a comment on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats URL: https://github.com/apache/spark/pull/23803#issuecomment-475101858 Sure, @maropu . For the first contributor, you need to add him into `Apache Spark Contributor` group. 1. Visit Project Configuration Role Page - https://issues.apache.org/jira/plugins/servlet/project-config/SPARK/roles 2. Click `Add users to a role` button (right-top of the page). 3. Fill the the pop-up box. - The JIRA user ID or email - Choose `Contributor`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats
dongjoon-hyun commented on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats URL: https://github.com/apache/spark/pull/23803#issuecomment-475101858 Sure, @maropu . For the first contributor, you need to him into `Apache Spark Contributor` group. 1. Visit Project Configuration Role Page - https://issues.apache.org/jira/plugins/servlet/project-config/SPARK/roles 2. Click `Add users to a role` button. 3. Fill the the pop-up box. - The JIRA user ID or email - Choose `Contributor`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats
dongjoon-hyun edited a comment on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats URL: https://github.com/apache/spark/pull/23803#issuecomment-475101858 Sure, @maropu . For the first contributor, you need to add him into `Apache Spark Contributor` group. 1. Visit Project Configuration Role Page - https://issues.apache.org/jira/plugins/servlet/project-config/SPARK/roles 2. Click `Add users to a role` button. 3. Fill the the pop-up box. - The JIRA user ID or email - Choose `Contributor`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer edited a comment on issue #23841: [SPARK-26936][SQL] Fix bug of insert overwrite local dir can not create temporary path in local staging directory
beliefer edited a comment on issue #23841: [SPARK-26936][SQL] Fix bug of insert overwrite local dir can not create temporary path in local staging directory URL: https://github.com/apache/spark/pull/23841#issuecomment-475097724 > Why did you remove the tests? Sorry, I can't find the reason why the feature 'insert overwrite local directory' has an inconsistent behavior in different environment. I have moved the conversation of create no exists local path with `insert overwrite local dir` to PR https://github.com/apache/spark/pull/23950. The current PR only used for `Mkdirs failed to create file` in yarn-client mode. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LantaoJin commented on a change in pull request #24157: [WIP][SPARK-27216][CORE] Kryo serialization with RoaringBitmap
LantaoJin commented on a change in pull request #24157:
[WIP][SPARK-27216][CORE] Kryo serialization with RoaringBitmap
URL: https://github.com/apache/spark/pull/24157#discussion_r267614390
##
File path:
core/src/test/scala/org/apache/spark/serializer/KryoSerializerSuite.scala
##
@@ -565,6 +565,20 @@ class KryoSerializerAutoResetDisabledSuite extends
SparkFunSuite with SharedSpar
assert(serInstance.deserialize[Any](serObj) === (obj))
assert(serInstance.deserialize[Any](byteBuffer) === (obj))
}
+
+ test("SPARK-27216: kryo serialization with RoaringBitmap") {
+val bitmap = new RoaringBitmap
+bitmap.add(1787)
+
+val safeSer = new KryoSerializer(conf).newInstance()
+val bitmap2 : RoaringBitmap =
safeSer.deserialize(safeSer.serialize(bitmap))
+assert(bitmap2.equals(bitmap))
+
+conf.set("spark.kryo.unsafe", "true")
Review comment:
It can be move to a totally new Suite. I will update it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #23841: [SPARK-26936][SQL] Fix bug of insert overwrite local dir can not create temporary path in local staging directory
beliefer commented on issue #23841: [SPARK-26936][SQL] Fix bug of insert overwrite local dir can not create temporary path in local staging directory URL: https://github.com/apache/spark/pull/23841#issuecomment-475097724 > Why did you remove the tests? Sorry, I can't find the reason why the feature 'insert overwrite local directory' has an inconsistent behavior in different environment. I have moved the conversation of create no exists local path with `insert overwrite local dir` to PR https://github.com/apache/spark/pull/23950. The current PR only used for `Mkdirs failed to create file` in yarn-client mode. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
SparkQA commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475097032 **[Test build #103753 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103753/testReport)** for PR 24161 at commit [`c51cd91`](https://github.com/apache/spark/commit/c51cd91551e4c547eebc148d486e83f7f86ac58c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
AmplabJenkins removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475096866 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9131/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
AmplabJenkins removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475096863 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
AmplabJenkins commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475096866 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9131/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
AmplabJenkins commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475096863 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path.
AmplabJenkins removed a comment on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path. URL: https://github.com/apache/spark/pull/24160#issuecomment-475096644 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103740/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path.
AmplabJenkins commented on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path. URL: https://github.com/apache/spark/pull/24160#issuecomment-475096644 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103740/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path.
AmplabJenkins commented on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path. URL: https://github.com/apache/spark/pull/24160#issuecomment-475096635 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path.
AmplabJenkins removed a comment on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path. URL: https://github.com/apache/spark/pull/24160#issuecomment-475096635 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path.
SparkQA removed a comment on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path. URL: https://github.com/apache/spark/pull/24160#issuecomment-475045854 **[Test build #103740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103740/testReport)** for PR 24160 at commit [`102e08b`](https://github.com/apache/spark/commit/102e08b620caa8582c140d5efe1744859f4c33e6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path.
SparkQA commented on issue #24160: [SPARK-27219][core] Treat timeouts as fatal in SASL fallback path. URL: https://github.com/apache/spark/pull/24160#issuecomment-475096294 **[Test build #103740 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103740/testReport)** for PR 24160 at commit [`102e08b`](https://github.com/apache/spark/commit/102e08b620caa8582c140d5efe1744859f4c33e6). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped
AmplabJenkins removed a comment on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped URL: https://github.com/apache/spark/pull/23912#issuecomment-475096017 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103744/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped
AmplabJenkins removed a comment on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped URL: https://github.com/apache/spark/pull/23912#issuecomment-475096012 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped
AmplabJenkins commented on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped URL: https://github.com/apache/spark/pull/23912#issuecomment-475096017 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103744/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped
AmplabJenkins commented on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped URL: https://github.com/apache/spark/pull/23912#issuecomment-475096012 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped
SparkQA removed a comment on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped URL: https://github.com/apache/spark/pull/23912#issuecomment-475045932 **[Test build #103744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103744/testReport)** for PR 23912 at commit [`60c3d24`](https://github.com/apache/spark/commit/60c3d24dce6c96432007e6f312136864f8c481b2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped
SparkQA commented on issue #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped URL: https://github.com/apache/spark/pull/23912#issuecomment-475095765 **[Test build #103744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103744/testReport)** for PR 23912 at commit [`60c3d24`](https://github.com/apache/spark/commit/60c3d24dce6c96432007e6f312136864f8c481b2). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file.
AmplabJenkins removed a comment on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file. URL: https://github.com/apache/spark/pull/24150#issuecomment-475095602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103742/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file.
AmplabJenkins removed a comment on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file. URL: https://github.com/apache/spark/pull/24150#issuecomment-475095600 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file.
AmplabJenkins commented on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file. URL: https://github.com/apache/spark/pull/24150#issuecomment-475095600 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file.
AmplabJenkins commented on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file. URL: https://github.com/apache/spark/pull/24150#issuecomment-475095602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103742/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file.
SparkQA removed a comment on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file. URL: https://github.com/apache/spark/pull/24150#issuecomment-475045862 **[Test build #103742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103742/testReport)** for PR 24150 at commit [`e651e14`](https://github.com/apache/spark/commit/e651e148fec330c0aaef1005128f8b497286f570). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file.
SparkQA commented on issue #24150: [SPARK-27209][SQL] Split parsing of SELECT and INSERT into two top-level rules in the grammar file. URL: https://github.com/apache/spark/pull/24150#issuecomment-475095265 **[Test build #103742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103742/testReport)** for PR 24150 at commit [`e651e14`](https://github.com/apache/spark/commit/e651e148fec330c0aaef1005128f8b497286f570). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats
maropu commented on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats URL: https://github.com/apache/spark/pull/23803#issuecomment-475094662 @dongjoon-hyun I couldn't assign @venkata91 in the jira and I'm not sure how to add a new account in contributor lists there... any info? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangruocheng edited a comment on issue #24153: [MINOR][EXAMPLES] Add missing return keyword streaming word count example
jiangruocheng edited a comment on issue #24153: [MINOR][EXAMPLES] Add missing return keyword streaming word count example URL: https://github.com/apache/spark/pull/24153#issuecomment-475094300 > Please keep the PR description format. How did you test? I got it. I just ran this example code and found the filter couldn't work. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangruocheng commented on issue #24153: [MINOR][EXAMPLES] Add missing return keyword streaming word count example
jiangruocheng commented on issue #24153: [MINOR][EXAMPLES] Add missing return keyword streaming word count example URL: https://github.com/apache/spark/pull/24153#issuecomment-475094300 > Please keep the PR description format. How did you test? I got it. I just runned this example code and found the filter couldn't work. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats
maropu commented on issue #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats URL: https://github.com/apache/spark/pull/23803#issuecomment-475094090 Thanks! Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen commented on issue #24127: [SPARK-27186][CORE] optimize sortshuffle write
srowen commented on issue #24127: [SPARK-27186][CORE] optimize sortshuffle write URL: https://github.com/apache/spark/pull/24127#issuecomment-475094168 I see, it's not a special Iterator implementation. Is it possible to refactor this code block, which looks like appears elsewhere, into any reasonable common function? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu closed pull request #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats
maropu closed pull request #23803: [SPARK-26894][SQL] Handle Alias as well in AggregateEstimation to propagate child stats URL: https://github.com/apache/spark/pull/23803 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24138: [MINOR][SQL]Put the grammar of database together, because this is good for maintenance and readability.
SparkQA commented on issue #24138: [MINOR][SQL]Put the grammar of database together, because this is good for maintenance and readability. URL: https://github.com/apache/spark/pull/24138#issuecomment-475093686 **[Test build #4649 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4649/testReport)** for PR 24138 at commit [`1a784a5`](https://github.com/apache/spark/commit/1a784a52a4a23d8be0664cd1c5ecfc7b3f6a0520). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
AmplabJenkins removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475093302 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103745/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
AmplabJenkins removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475093300 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
AmplabJenkins commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475093302 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103745/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson
HyukjinKwon closed pull request #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson URL: https://github.com/apache/spark/pull/24159 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
SparkQA commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475093198 **[Test build #103745 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103745/testReport)** for PR 24161 at commit [`80b2e30`](https://github.com/apache/spark/commit/80b2e309dbfcf796a09f7b08d5e68862014fa266). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
AmplabJenkins removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475074846 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103739/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #23841: [SPARK-26936][SQL] Fix bug of insert overwrite local dir can not create temporary path in local staging directory
maropu commented on issue #23841: [SPARK-26936][SQL] Fix bug of insert overwrite local dir can not create temporary path in local staging directory URL: https://github.com/apache/spark/pull/23841#issuecomment-475093193 Why did you remove the tests? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
AmplabJenkins commented on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475093300 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply
SparkQA removed a comment on issue #24161: [SPARK-27222][SQL] Support Instant and LocalDate in Literal.apply URL: https://github.com/apache/spark/pull/24161#issuecomment-475074766 **[Test build #103745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103745/testReport)** for PR 24161 at commit [`80b2e30`](https://github.com/apache/spark/commit/80b2e309dbfcf796a09f7b08d5e68862014fa266). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #24162: [SPARK-27223][SQL] Remove private methods that skip conversion when passing user schemas for constructing a DataFrame
HyukjinKwon closed pull request #24162: [SPARK-27223][SQL] Remove private methods that skip conversion when passing user schemas for constructing a DataFrame URL: https://github.com/apache/spark/pull/24162 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson
HyukjinKwon commented on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson URL: https://github.com/apache/spark/pull/24159#issuecomment-475092946 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson
AmplabJenkins commented on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson URL: https://github.com/apache/spark/pull/24159#issuecomment-475092826 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103741/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson
AmplabJenkins removed a comment on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson URL: https://github.com/apache/spark/pull/24159#issuecomment-475092826 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103741/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson
AmplabJenkins removed a comment on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson URL: https://github.com/apache/spark/pull/24159#issuecomment-475092823 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson
AmplabJenkins commented on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson URL: https://github.com/apache/spark/pull/24159#issuecomment-475092823 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #24047: [SPARK-25196][SQL] Extends the analyze column command for cached tables
maropu commented on issue #24047: [SPARK-25196][SQL] Extends the analyze column command for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-475092746 Thanks for your active reviews, @dongjoon-hyun ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24162: [SPARK-27223][SQL] Remove private methods that skip conversion when passing user schemas for constructing a DataFrame
HyukjinKwon commented on issue #24162: [SPARK-27223][SQL] Remove private methods that skip conversion when passing user schemas for constructing a DataFrame URL: https://github.com/apache/spark/pull/24162#issuecomment-475092658 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson
SparkQA removed a comment on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson URL: https://github.com/apache/spark/pull/24159#issuecomment-475045867 **[Test build #103741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103741/testReport)** for PR 24159 at commit [`ed3eea4`](https://github.com/apache/spark/commit/ed3eea4672b809da91904d2ffa4f48da5981dbe4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #24047: [SPARK-25196][SQL] Extends the analyze column command for cached tables
maropu commented on a change in pull request #24047: [SPARK-25196][SQL] Extends
the analyze column command for cached tables
URL: https://github.com/apache/spark/pull/24047#discussion_r267608769
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala
##
@@ -39,32 +40,39 @@ case class AnalyzeColumnCommand(
require(columnNames.isDefined ^ allColumns, "Parameter `columnNames` or
`allColumns` are " +
"mutually exclusive. Only one of them should be specified.")
val sessionState = sparkSession.sessionState
-val db =
tableIdent.database.getOrElse(sessionState.catalog.getCurrentDatabase)
-val tableIdentWithDB = TableIdentifier(tableIdent.table, Some(db))
-val tableMeta = sessionState.catalog.getTableMetadata(tableIdentWithDB)
-if (tableMeta.tableType == CatalogTableType.VIEW) {
- throw new AnalysisException("ANALYZE TABLE is not supported on views.")
-}
-val sizeInBytes = CommandUtils.calculateTotalSize(sparkSession, tableMeta)
-val relation = sparkSession.table(tableIdent).logicalPlan
-val columnsToAnalyze = getColumnsToAnalyze(tableIdent, relation,
columnNames, allColumns)
-
-// Compute stats for the computed list of columns.
-val (rowCount, newColStats) =
- CommandUtils.computeColumnStats(sparkSession, relation, columnsToAnalyze)
-// We also update table-level stats in order to keep them consistent with
column-level stats.
-val statistics = CatalogStatistics(
- sizeInBytes = sizeInBytes,
- rowCount = Some(rowCount),
- // Newly computed column stats should override the existing ones.
- colStats = tableMeta.stats.map(_.colStats).getOrElse(Map.empty) ++
newColStats)
-
-sessionState.catalog.alterTableStats(tableIdentWithDB, Some(statistics))
+tableIdent.database match {
+ case Some(db) if db ==
sparkSession.sharedState.globalTempViewManager.database =>
+val plan =
sessionState.catalog.getGlobalTempView(tableIdent.identifier).getOrElse {
+ throw new NoSuchTableException(db = db, table =
tableIdent.identifier)
+}
+analyzeColumnInTempView(plan, sparkSession)
+ case Some(_) =>
+analyzeColumnInCatalog(sparkSession)
+ case None =>
+sessionState.catalog.getTempView(tableIdent.identifier) match {
+ case Some(tempView) => analyzeColumnInTempView(tempView,
sparkSession)
+ case _ => analyzeColumnInCatalog(sparkSession)
+}
+}
Seq.empty[Row]
}
+ private def analyzeColumnInTempView(plan: LogicalPlan, sparkSession:
SparkSession): Unit = {
+val cacheManager = sparkSession.sharedState.cacheManager
+cacheManager.lookupCachedData(plan) match {
+ case Some(cachedData) =>
+val columnsToAnalyze = getColumnsToAnalyze(
+ tableIdent, cachedData.plan, columnNames, allColumns)
+cacheManager.analyzeColumnCacheQuery(sparkSession, cachedData,
columnsToAnalyze)
+ case _ =>
+val catalog = sparkSession.sessionState.catalog
+val db = tableIdent.database.getOrElse(catalog.getCurrentDatabase)
+throw new NoSuchTableException(db = db, table = tableIdent.identifier)
Review comment:
Since the existing tests expect this exception, I kept as it is:
https://github.com/apache/spark/blob/d6ee2f331db461c1f7a25e0ef17901f53d8b707e/sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala#L163
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson
SparkQA commented on issue #24159: [SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson URL: https://github.com/apache/spark/pull/24159#issuecomment-475092527 **[Test build #103741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103741/testReport)** for PR 24159 at commit [`ed3eea4`](https://github.com/apache/spark/commit/ed3eea4672b809da91904d2ffa4f48da5981dbe4). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangjiaochun commented on issue #24127: [SPARK-27186][CORE] optimize sortshuffle write
wangjiaochun commented on issue #24127: [SPARK-27186][CORE] optimize sortshuffle write URL: https://github.com/apache/spark/pull/24127#issuecomment-475091856 The underlying Iterator do nothing. so I think it's worth revising. first, ExternalSorter-instantiated is expensive operation, such as there is need to estimate the size of all the collection of the Instances. Also create tmp file is needless. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24047: [SPARK-25196][SQL] Extends the analyze column command for cached tables
SparkQA commented on issue #24047: [SPARK-25196][SQL] Extends the analyze column command for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-475091728 **[Test build #103752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103752/testReport)** for PR 24047 at commit [`c6f0fb5`](https://github.com/apache/spark/commit/c6f0fb5e7d0a9b5ede73ec058894c65ee238). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24047: [SPARK-25196][SQL] Extends the analyze column command for cached tables
AmplabJenkins removed a comment on issue #24047: [SPARK-25196][SQL] Extends the analyze column command for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-475091455 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9130/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24047: [SPARK-25196][SQL] Extends the analyze column command for cached tables
AmplabJenkins commented on issue #24047: [SPARK-25196][SQL] Extends the analyze column command for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-475091453 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer edited a comment on issue #24138: [MINOR][SQL]Put the grammar of database together, because this is good for maintenance and readability.
beliefer edited a comment on issue #24138: [MINOR][SQL]Put the grammar of database together, because this is good for maintenance and readability. URL: https://github.com/apache/spark/pull/24138#issuecomment-475090967 @srowen The test build not found. How to retrigger jenkins to build? The url 'https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4637/testReport' is 404. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
