[GitHub] [spark] dongjoon-hyun commented on issue #24860: [SPARK-28034][SQL][TEST] Port with.sql
dongjoon-hyun commented on issue #24860: [SPARK-28034][SQL][TEST] Port with.sql URL: https://github.com/apache/spark/pull/24860#issuecomment-510756243 Hi, @maropu and @wangyum , and @peter-toth . Since the followings seems to need more time, I'll merge this first. Please rebase and update those PRs. - https://github.com/apache/spark/pull/25109 ([SPARK-28343][SQL][TEST] PostgreSQL test should enable cartesian product ) - https://github.com/apache/spark/pull/25114 ([SPARK-28349][SQL] Add FALSE and SETMINUS to ansiNonReserved) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
dongjoon-hyun commented on a change in pull request #25129:
[SPARK-28159][ML][FOLLOWUP] fix typo
URL: https://github.com/apache/spark/pull/25129#discussion_r302835427
##
File path: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala
##
@@ -490,7 +490,7 @@ abstract class LDAModel private[ml] (
Vectors.zeros(k)
} else {
val (ids: List[Int], cts: Array[Double]) = vector match {
- case v: DenseVector => ((0 until v.size).toList, v.values)
+ case v: DenseVector => (List.range(0, v.size), v.values)
Review comment:
Please update the PR title or remove this from this PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25126: [SPARK-28306][SQL][FOLLOWUP] Fix NormalizeFloatingNumbers rule idempotence for equi-join with `<=>` predicates
dongjoon-hyun commented on issue #25126: [SPARK-28306][SQL][FOLLOWUP] Fix NormalizeFloatingNumbers rule idempotence for equi-join with `<=>` predicates URL: https://github.com/apache/spark/pull/25126#issuecomment-510753403 Could you fix the following UT failure, @yeshengm ? - org.apache.spark.sql.catalyst.optimizer.NormalizeFloatingPointNumbersSuite.normalize floating points in join keys (equal null safe) - idempotence This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL
dongjoon-hyun closed pull request #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL URL: https://github.com/apache/spark/pull/25059 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] chitralverma edited a comment on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base
chitralverma edited a comment on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base URL: https://github.com/apache/spark/pull/25122#issuecomment-510751215 @HyukjinKwon Thanks for your review comments, I've replied to them. I feel the suggested workaround([this](https://github.com/apache/spark/pull/25098#issuecomment-510283330) and [this](https://github.com/apache/spark/pull/25098#issuecomment-510283866)) will not work in this case as certain udf's like `sum` return `null` if aggregated on empty datasets ([SPARK-20346](https://issues.apache.org/jira/browse/SPARK-20346)). As per the guide in [SPARK-27921](https://issues.apache.org/jira/browse/SPARK-27921) , do you suggest I create a separate Jira for this and comment the problematic test cases for now ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql
HyukjinKwon closed pull request #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql
HyukjinKwon commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128#issuecomment-510751457 Merged to master. This cannot affect other tests other than the passed tests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] chitralverma commented on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base
chitralverma commented on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base URL: https://github.com/apache/spark/pull/25122#issuecomment-510751215 @HyukjinKwon Thanks for your review comments, I've replied to them. I feel the suggested workaround will not work in this case as certain udf's like `sum` return `null` if aggregated on empty datasets ([SPARK-20346](https://issues.apache.org/jira/browse/SPARK-20346)). As per the guide in [SPARK-27921](https://issues.apache.org/jira/browse/SPARK-27921) , do you suggest I create a separate Jira for this and comment the problematic test cases for now ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
AmplabJenkins removed a comment on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510749717 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
AmplabJenkins commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510749717 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
AmplabJenkins commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510749720 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107578/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
AmplabJenkins removed a comment on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510749720 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107578/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
SparkQA removed a comment on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510738003 **[Test build #107578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107578/testReport)** for PR 25129 at commit [`8adbf49`](https://github.com/apache/spark/commit/8adbf49406d785310066dda7ab65bbde88e81421). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
SparkQA commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510749526 **[Test build #107578 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107578/testReport)** for PR 25129 at commit [`8adbf49`](https://github.com/apache/spark/commit/8adbf49406d785310066dda7ab65bbde88e81421). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.s
AmplabJenkins removed a comment on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128#issuecomment-510747215 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107570/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.s
AmplabJenkins removed a comment on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128#issuecomment-510747208 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql
AmplabJenkins commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128#issuecomment-510747215 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107570/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql
AmplabJenkins commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128#issuecomment-510747208 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gczsjdy commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API
gczsjdy commented on a change in pull request #25007:
[SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API
URL: https://github.com/apache/spark/pull/25007#discussion_r302829737
##
File path:
core/src/main/java/org/apache/spark/shuffle/api/ShuffleMapOutputWriter.java
##
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.shuffle.api;
+
+import java.io.IOException;
+
+import org.apache.spark.annotation.Private;
+
+/**
+ * :: Experimental ::
+ * A top-level writer that returns child writers for persisting the output of
a map task,
+ * and then commits all of the writes as one atomic operation.
+ *
+ * @since 3.0.0
+ */
+@Private
+public interface ShuffleMapOutputWriter {
+
+ /**
+ * Creates a writer that can open an output stream to persist bytes for a
given chunk of
+ * a map task.
+ *
+ * The chunk corresponds to bytes in a partition that all share the same
reduce id, hence
+ * the given argument. This will not be called twice for the same partition
identifier.
+ * The partition identifier will be in the range of precisely 0 (inclusive)
to numPartitions
+ * (exclusive), where numPartitions was provided upon the creation of this
map output writer via
+ * {@link ShuffleWriteSupport#createMapOutputWriter(int, int, long, int)}.
+ */
+ ShufflePartitionWriter getPartitionWriter(int partitionId) throws
IOException;
+
+ /**
+ * Commits the writes done by all partition writers returned by all calls to
this object's
+ * {@link #getPartitionWriter(int)}.
+ *
+ * This should ensure that the writes conducted by this module's partition
writers are
+ * available to downstream reduce tasks. If this method throws any
exception, this module's
+ * {@link #abort(Throwable)} method will be invoked before propagating the
exception.
+ *
+ * This can also close any resources and clean up temporary state if
necessary.
+ */
+ void commitAllPartitions() throws IOException;
Review comment:
Shouldn't this return `Optional`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql
SparkQA removed a comment on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128#issuecomment-510715774 **[Test build #107570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107570/testReport)** for PR 25128 at commit [`78d636d`](https://github.com/apache/spark/commit/78d636d49efc416ac50367bb751b8819f263316b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql
SparkQA commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128#issuecomment-510746829 **[Test build #107570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107570/testReport)** for PR 25128 at commit [`78d636d`](https://github.com/apache/spark/commit/78d636d49efc416ac50367bb751b8819f263316b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #24759: [SPARK-27395][SQL][WIP] Improve EXPLAIN command
gatorsmile commented on issue #24759: [SPARK-27395][SQL][WIP] Improve EXPLAIN command URL: https://github.com/apache/spark/pull/24759#issuecomment-510745607 cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510744880 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510744880 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510744881 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107577/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510744881 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107577/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
SparkQA commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510744633 **[Test build #107577 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107577/testReport)** for PR 24793 at commit [`90eb0ac`](https://github.com/apache/spark/commit/90eb0acdd0d1dc82e702481cfd4ce34ffbafe217). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
SparkQA removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510733442 **[Test build #107577 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107577/testReport)** for PR 24793 at commit [`90eb0ac`](https://github.com/apache/spark/commit/90eb0acdd0d1dc82e702481cfd4ce34ffbafe217). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first.
AmplabJenkins commented on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first. URL: https://github.com/apache/spark/pull/24909#issuecomment-510744492 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first.
AmplabJenkins removed a comment on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first. URL: https://github.com/apache/spark/pull/24909#issuecomment-510744497 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107575/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first.
AmplabJenkins commented on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first. URL: https://github.com/apache/spark/pull/24909#issuecomment-510744497 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107575/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first.
AmplabJenkins removed a comment on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first. URL: https://github.com/apache/spark/pull/24909#issuecomment-510744492 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first.
SparkQA commented on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first. URL: https://github.com/apache/spark/pull/24909#issuecomment-510744161 **[Test build #107575 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107575/testReport)** for PR 24909 at commit [`8d0f3f9`](https://github.com/apache/spark/commit/8d0f3f9bee77bc2a6ca2f31d9ea91e72415dfea6). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first.
SparkQA removed a comment on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first. URL: https://github.com/apache/spark/pull/24909#issuecomment-510724481 **[Test build #107575 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107575/testReport)** for PR 24909 at commit [`8d0f3f9`](https://github.com/apache/spark/commit/8d0f3f9bee77bc2a6ca2f31d9ea91e72415dfea6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #25121: [SPARK-28356][SQL] Do not reduce the number of partitions for repartition in adaptive execution
cloud-fan commented on a change in pull request #25121: [SPARK-28356][SQL] Do
not reduce the number of partitions for repartition in adaptive execution
URL: https://github.com/apache/spark/pull/25121#discussion_r302825987
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -312,6 +312,16 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val REDUCE_POST_SHUFFLE_PARTITIONS_FOR_REPARTITION =
Review comment:
I think we can remove that test. Excluding repartition, the pre-shuffle num
partitions are always the same (200 by default).
We do need to have a test for repartition though.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #25108: [SPARK-28321][SQL] 0-args Java UDF should not be called only once
cloud-fan closed pull request #25108: [SPARK-28321][SQL] 0-args Java UDF should not be called only once URL: https://github.com/apache/spark/pull/25108 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #25121: [SPARK-28356][SQL] Do not reduce the number of partitions for repartition in adaptive execution
cloud-fan commented on a change in pull request #25121: [SPARK-28356][SQL] Do
not reduce the number of partitions for repartition in adaptive execution
URL: https://github.com/apache/spark/pull/25121#discussion_r302825851
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -312,6 +312,16 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val REDUCE_POST_SHUFFLE_PARTITIONS_FOR_REPARTITION =
Review comment:
let's rewrite the tests. We can use `spark.range` to create 2 relations with
different num partitions
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #25121: [SPARK-28356][SQL] Do not reduce the number of partitions for repartition in adaptive execution
cloud-fan commented on a change in pull request #25121: [SPARK-28356][SQL] Do
not reduce the number of partitions for repartition in adaptive execution
URL: https://github.com/apache/spark/pull/25121#discussion_r302825851
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -312,6 +312,16 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val REDUCE_POST_SHUFFLE_PARTITIONS_FOR_REPARTITION =
Review comment:
let's rewrite the tests. We can use `spark.range` to create 2 relations with
different num partitions
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #25108: [SPARK-28321][SQL] 0-args Java UDF should not be called only once
cloud-fan commented on issue #25108: [SPARK-28321][SQL] 0-args Java UDF should not be called only once URL: https://github.com/apache/spark/pull/25108#issuecomment-510742674 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base
AmplabJenkins removed a comment on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base URL: https://github.com/apache/spark/pull/25122#issuecomment-510740647 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107571/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base
SparkQA removed a comment on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base URL: https://github.com/apache/spark/pull/25122#issuecomment-510717123 **[Test build #107571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107571/testReport)** for PR 25122 at commit [`9be50d5`](https://github.com/apache/spark/commit/9be50d5d770556e9d98bf317fae8457f6749217e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base
AmplabJenkins removed a comment on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base URL: https://github.com/apache/spark/pull/25122#issuecomment-510740643 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base
AmplabJenkins commented on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base URL: https://github.com/apache/spark/pull/25122#issuecomment-510740643 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base
AmplabJenkins commented on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base URL: https://github.com/apache/spark/pull/25122#issuecomment-510740647 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107571/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL
AmplabJenkins removed a comment on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL URL: https://github.com/apache/spark/pull/25059#issuecomment-510740236 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107568/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base
SparkQA commented on issue #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base URL: https://github.com/apache/spark/pull/25122#issuecomment-510740519 **[Test build #107571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107571/testReport)** for PR 25122 at commit [`9be50d5`](https://github.com/apache/spark/commit/9be50d5d770556e9d98bf317fae8457f6749217e). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL
AmplabJenkins removed a comment on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL URL: https://github.com/apache/spark/pull/25059#issuecomment-510740232 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL
AmplabJenkins commented on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL URL: https://github.com/apache/spark/pull/25059#issuecomment-510740232 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL
AmplabJenkins commented on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL URL: https://github.com/apache/spark/pull/25059#issuecomment-510740236 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107568/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL
SparkQA removed a comment on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL URL: https://github.com/apache/spark/pull/25059#issuecomment-510706076 **[Test build #107568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107568/testReport)** for PR 25059 at commit [`b39fb62`](https://github.com/apache/spark/commit/b39fb625b18d5a07d3076406a73e0db8d519ae4b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
AmplabJenkins removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510739681 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL
SparkQA commented on issue #25059: [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL URL: https://github.com/apache/spark/pull/25059#issuecomment-510739815 **[Test build #107568 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107568/testReport)** for PR 25059 at commit [`b39fb62`](https://github.com/apache/spark/commit/b39fb625b18d5a07d3076406a73e0db8d519ae4b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
AmplabJenkins removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510739685 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107569/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
AmplabJenkins commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510739685 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107569/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
SparkQA removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510706078 **[Test build #107569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107569/testReport)** for PR 25041 at commit [`18dcf6d`](https://github.com/apache/spark/commit/18dcf6db991900fce533ca4bbc27f854959c3c14). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
AmplabJenkins commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510739681 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
SparkQA commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510739242 **[Test build #107569 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107569/testReport)** for PR 25041 at commit [`18dcf6d`](https://github.com/apache/spark/commit/18dcf6db991900fce533ca4bbc27f854959c3c14). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] chitralverma commented on a change in pull request #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base
chitralverma commented on a change in pull request #25122:
[SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF
test base
URL: https://github.com/apache/spark/pull/25122#discussion_r302821986
##
File path: sql/core/src/test/resources/sql-tests/results/udf/udf-pivot.sql.out
##
@@ -0,0 +1,494 @@
+-- Automatically generated by SQLQueryTestSuite
+-- Number of queries: 32
+
+
+-- !query 0
+create temporary view courseSales as select * from values
+ ("dotNET", 2012, 1),
+ ("Java", 2012, 2),
+ ("dotNET", 2012, 5000),
+ ("dotNET", 2013, 48000),
+ ("Java", 2013, 3)
+ as courseSales(course, year, earnings)
+-- !query 0 schema
+struct<>
+-- !query 0 output
+
+
+
+-- !query 1
+create temporary view years as select * from values
+ (2012, 1),
+ (2013, 2)
+ as years(y, s)
+-- !query 1 schema
+struct<>
+-- !query 1 output
+
+
+
+-- !query 2
+create temporary view yearsWithComplexTypes as select * from values
+ (2012, array(1, 1), map('1', 1), struct(1, 'a')),
+ (2013, array(2, 2), map('2', 2), struct(2, 'b'))
+ as yearsWithComplexTypes(y, a, m, s)
+-- !query 2 schema
+struct<>
+-- !query 2 output
+
+
+
+-- !query 3
+SELECT * FROM (
+ SELECT udf(year), course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 3 schema
+struct
+-- !query 3 output
+2012 15000 2
+2013 48000 3
+
+
+-- !query 4
+SELECT * FROM courseSales
+PIVOT (
+ udf(sum(earnings))
+ FOR year IN (2012, 2013)
+)
+-- !query 4 schema
+struct
+-- !query 4 output
+Java 2 3
+dotNET 15000 48000
+
+
+-- !query 5
+SELECT * FROM (
+ SELECT year, course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings)), udf(avg(earnings))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 5 schema
+struct
+-- !query 5 output
+2012 15000 7500.0 2 2.0
+2013 48000 48000.0 3 3.0
+
+
+-- !query 6
+SELECT * FROM (
+ SELECT udf(course) as course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 6 schema
+struct
+-- !query 6 output
+63000 5
+
+
+-- !query 7
+SELECT * FROM (
+ SELECT year, course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings)), udf(min(year))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 7 schema
+struct
+-- !query 7 output
+63000 20125 2012
+
+
+-- !query 8
+SELECT * FROM (
+ SELECT course, year, earnings, udf(s) as s
+ FROM courseSales
+ JOIN years ON year = y
+)
+PIVOT (
+ udf(sum(earnings))
+ FOR s IN (1, 2)
+)
+-- !query 8 schema
+struct
+-- !query 8 output
+Java 20122 nan
+Java 2013nan 3
+dotNET 201215000 nan
+dotNET 2013nan 48000
+
+
+-- !query 9
+SELECT * FROM (
+ SELECT course, year, earnings, s
+ FROM courseSales
+ JOIN years ON year = y
+)
+PIVOT (
+ udf(sum(earnings)), udf(min(s))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 9 schema
+struct
+-- !query 9 output
+2012 15000 1 2 1
+2013 48000 2 3 2
+
+
+-- !query 10
+SELECT * FROM (
+ SELECT course, year, earnings, s
+ FROM courseSales
+ JOIN years ON year = y
+)
+PIVOT (
+ udf(sum(earnings * s))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 10 schema
+struct
+-- !query 10 output
+2012 15000 2
+2013 96000 6
+
+
+-- !query 11
+SELECT 2012_s, 2013_s, 2012_a, 2013_a, c FROM (
+ SELECT year y, course c, earnings e FROM courseSales
+)
+PIVOT (
+ udf(sum(e)) s, udf(avg(e)) a
+ FOR y IN (2012, 2013)
+)
+-- !query 11 schema
+struct<2012_s:string,2013_s:string,2012_a:string,2013_a:string,c:string>
+-- !query 11 output
+15000 48000 7500.0 48000.0 dotNET
+2 3 2.0 3.0 Java
+
+
+-- !query 12
+SELECT firstYear_s, secondYear_s, firstYear_a, secondYear_a, c FROM (
+ SELECT year y, course c, earnings e FROM courseSales
+)
+PIVOT (
+ udf(sum(e)) s, udf(avg(e)) a
+ FOR y IN (2012 as firstYear, 2013 secondYear)
+)
+-- !query 12 schema
+struct
+-- !query 12 output
+15000 48000 7500.0 48000.0 dotNET
+2 3 2.0 3.0 Java
+
+
+-- !query 13
+SELECT * FROM courseSales
+PIVOT (
+ udf(abs(earnings))
+ FOR year IN (2012, 2013)
+)
+-- !query 13 schema
+struct<>
+-- !query 13 output
+org.apache.spark.sql.AnalysisException
+Aggregate expression required for pivot, but 'coursesales.`earnings`' did not
appear in any aggregate function.;
+
+
+-- !query 14
+SELECT * FROM (
+ SELECT year, course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings)), year
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 14 schema
+struct<>
+-- !query 14 output
+org.apache.spark.sql.AnalysisException
+Aggregate expression required for pivot, but
'__auto_generated_subquery_name.`year`' did not appear in any aggregate
function.;
+
+
+-- !query 15
+SELECT * FROM (
+ SELECT course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings))
+ FOR year IN (2012, 2013)
+)
+-- !query 15 schema
+struct<>
+--
[GitHub] [spark] AmplabJenkins removed a comment on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
AmplabJenkins removed a comment on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510737740 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12704/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
AmplabJenkins removed a comment on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510737737 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
SparkQA commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510738003 **[Test build #107578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107578/testReport)** for PR 25129 at commit [`8adbf49`](https://github.com/apache/spark/commit/8adbf49406d785310066dda7ab65bbde88e81421). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
AmplabJenkins commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510737737 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
AmplabJenkins commented on issue #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129#issuecomment-510737740 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12704/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng opened a new pull request #25129: [SPARK-28159][ML][FOLLOWUP] fix typo
zhengruifeng opened a new pull request #25129: [SPARK-28159][ML][FOLLOWUP] fix typo URL: https://github.com/apache/spark/pull/25129 ## What changes were proposed in this pull request? fix typo in spark-28159 `transfromWithMean` -> `transformWithMean` ## How was this patch tested? existing test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql
HyukjinKwon commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128#issuecomment-510736625 Thanks guys. This problem bugs me hard . Let me think more and try to make a fix or the list of workarounds probably within today. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
AmplabJenkins removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510736414 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107565/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
AmplabJenkins removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510736408 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #24937: [SPARK-28139][SQL] Add v2 ALTER TABLE implementation.
cloud-fan closed pull request #24937: [SPARK-28139][SQL] Add v2 ALTER TABLE implementation. URL: https://github.com/apache/spark/pull/24937 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
AmplabJenkins commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510736414 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107565/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
AmplabJenkins commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510736408 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
SparkQA removed a comment on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510703603 **[Test build #107565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107565/testReport)** for PR 25041 at commit [`b0af5c7`](https://github.com/apache/spark/commit/b0af5c7b4aef73076b411d981e01bf6f6f0baf77). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL
SparkQA commented on issue #25041: [SPARK-28133][SQL] Add acosh/asinh/atanh functions to SQL URL: https://github.com/apache/spark/pull/25041#issuecomment-510736120 **[Test build #107565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107565/testReport)** for PR 25041 at commit [`b0af5c7`](https://github.com/apache/spark/commit/b0af5c7b4aef73076b411d981e01bf6f6f0baf77). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gczsjdy commented on a change in pull request #25121: [SPARK-28356][SQL] Do not reduce the number of partitions for repartition in adaptive execution
gczsjdy commented on a change in pull request #25121: [SPARK-28356][SQL] Do not
reduce the number of partitions for repartition in adaptive execution
URL: https://github.com/apache/spark/pull/25121#discussion_r302819612
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala
##
@@ -43,7 +43,8 @@ import
org.apache.spark.util.collection.unsafe.sort.{PrefixComparators, RecordCo
*/
case class ShuffleExchangeExec(
override val outputPartitioning: Partitioning,
-child: SparkPlan) extends Exchange {
+child: SparkPlan,
+supportAdaptive: Boolean = true) extends Exchange {
Review comment:
+1
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gczsjdy commented on a change in pull request #25121: [SPARK-28356][SQL] Do not reduce the number of partitions for repartition in adaptive execution
gczsjdy commented on a change in pull request #25121: [SPARK-28356][SQL] Do not reduce the number of partitions for repartition in adaptive execution URL: https://github.com/apache/spark/pull/25121#discussion_r302819390 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/QueryStageExec.scala ## @@ -140,6 +140,8 @@ case class ShuffleQueryStageExec( case _ => } } + + def supportAdaptive: Boolean = plan.supportAdaptive Review comment: :nit -> `exchangeSupportAdaptive`? Or `canChangeNumPartition` `QueryStage` is already a part of adaptive execution This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gczsjdy commented on a change in pull request #25121: [SPARK-28356][SQL] Do not reduce the number of partitions for repartition in adaptive execution
gczsjdy commented on a change in pull request #25121: [SPARK-28356][SQL] Do not reduce the number of partitions for repartition in adaptive execution URL: https://github.com/apache/spark/pull/25121#discussion_r302819390 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/QueryStageExec.scala ## @@ -140,6 +140,8 @@ case class ShuffleQueryStageExec( case _ => } } + + def supportAdaptive: Boolean = plan.supportAdaptive Review comment: :nit -> `exchangeSupportAdaptive`? `QueryStage` is already a part of adaptive execution This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregat
HyukjinKwon commented on a change in pull request #25128:
[SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of
UDF as double in the failed test in udf-aggregate_part1.sql
URL: https://github.com/apache/spark/pull/25128#discussion_r302819302
##
File path:
sql/core/src/test/resources/sql-tests/inputs/udf/pgSQL/udf-aggregates_part1.sql
##
@@ -78,7 +78,7 @@ FROM (VALUES ('-Infinity'), ('Infinity')) v(x);
-- test accuracy with a large input offset
SELECT CAST(avg(udf(CAST(x AS DOUBLE))) AS int), CAST(udf(var_pop(CAST(x AS
DOUBLE))) AS decimal(10,3))
FROM (VALUES (10003), (10004), (10006), (10007)) v(x);
-SELECT CAST(avg(udf(CAST(x AS DOUBLE))) AS long), CAST(udf(var_pop(CAST(x AS
DOUBLE))) AS decimal(10,3))
+SELECT CAST(avg(udf(x)) AS long), CAST(udf(var_pop(CAST(x AS DOUBLE))) AS
decimal(10,3))
Review comment:
I am going to make a fix to address all of them soon .. this is just a quick
fix to make the tests run ..
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #24937: [SPARK-28139][SQL] Add v2 ALTER TABLE implementation.
cloud-fan commented on issue #24937: [SPARK-28139][SQL] Add v2 ALTER TABLE implementation. URL: https://github.com/apache/spark/pull/24937#issuecomment-510735076 Since it's a pretty big PR and I'd like to merge it first. @rdblue please send a followup PR to add docs to `ColumnChange` and support v2 session catalog, thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql
dongjoon-hyun commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128#issuecomment-510734812 Thank you for quick fix. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510734336 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510734345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12703/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510734345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12703/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510734336 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql
dongjoon-hyun commented on issue #25128: [SPARK-28270][test-maven][FOLLOW-UP][SQL][PYTHON][TESTS] Avoid cast input of UDF as double in the failed test in udf-aggregate_part1.sql URL: https://github.com/apache/spark/pull/25128#issuecomment-510734372 The Maven Jenkins result (2nd tigger) succeeded. ``` ... - pgSQL/int8.sql - pgSQL/case.sql - pgSQL/select.sql - pgSQL/aggregates_part2.sql - pgSQL/date.sql - pgSQL/boolean.sql - pgSQL/aggregates_part1.sql - pgSQL/int4.sql - pgSQL/int2.sql - pgSQL/float4.sql - ansi/interval.sql ... ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510733153 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12702/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins removed a comment on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510733150 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
SparkQA commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510733442 **[Test build #107577 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107577/testReport)** for PR 24793 at commit [`90eb0ac`](https://github.com/apache/spark/commit/90eb0acdd0d1dc82e702481cfd4ce34ffbafe217). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] carsonwang commented on a change in pull request #25121: [SPARK-28356][SQL] Do not reduce the number of partitions for repartition in adaptive execution
carsonwang commented on a change in pull request #25121: [SPARK-28356][SQL] Do
not reduce the number of partitions for repartition in adaptive execution
URL: https://github.com/apache/spark/pull/25121#discussion_r302817560
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -312,6 +312,16 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val REDUCE_POST_SHUFFLE_PARTITIONS_FOR_REPARTITION =
Review comment:
I initially didn't add this config. Later I found with that change, the
`test("Union two datasets with different pre-shuffle partition number")` won't
test what we want to test, because it was written using repartition. Probably I
can rewrite that test without using repartition.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510733153 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12702/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names
AmplabJenkins commented on issue #24793: [SPARK-27944][ML] Unify the behavior of checking empty output column names URL: https://github.com/apache/spark/pull/24793#issuecomment-510733150 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] chitralverma commented on a change in pull request #25122: [SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF test base
chitralverma commented on a change in pull request #25122:
[SPARK-28286][SQL][PYTHON][TESTS][WIP] Convert and port 'pivot.sql' into UDF
test base
URL: https://github.com/apache/spark/pull/25122#discussion_r302817447
##
File path: sql/core/src/test/resources/sql-tests/results/udf/udf-pivot.sql.out
##
@@ -0,0 +1,494 @@
+-- Automatically generated by SQLQueryTestSuite
+-- Number of queries: 32
+
+
+-- !query 0
+create temporary view courseSales as select * from values
+ ("dotNET", 2012, 1),
+ ("Java", 2012, 2),
+ ("dotNET", 2012, 5000),
+ ("dotNET", 2013, 48000),
+ ("Java", 2013, 3)
+ as courseSales(course, year, earnings)
+-- !query 0 schema
+struct<>
+-- !query 0 output
+
+
+
+-- !query 1
+create temporary view years as select * from values
+ (2012, 1),
+ (2013, 2)
+ as years(y, s)
+-- !query 1 schema
+struct<>
+-- !query 1 output
+
+
+
+-- !query 2
+create temporary view yearsWithComplexTypes as select * from values
+ (2012, array(1, 1), map('1', 1), struct(1, 'a')),
+ (2013, array(2, 2), map('2', 2), struct(2, 'b'))
+ as yearsWithComplexTypes(y, a, m, s)
+-- !query 2 schema
+struct<>
+-- !query 2 output
+
+
+
+-- !query 3
+SELECT * FROM (
+ SELECT udf(year), course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 3 schema
+struct
+-- !query 3 output
+2012 15000 2
+2013 48000 3
+
+
+-- !query 4
+SELECT * FROM courseSales
+PIVOT (
+ udf(sum(earnings))
+ FOR year IN (2012, 2013)
+)
+-- !query 4 schema
+struct
+-- !query 4 output
+Java 2 3
+dotNET 15000 48000
+
+
+-- !query 5
+SELECT * FROM (
+ SELECT year, course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings)), udf(avg(earnings))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 5 schema
+struct
+-- !query 5 output
+2012 15000 7500.0 2 2.0
+2013 48000 48000.0 3 3.0
+
+
+-- !query 6
+SELECT * FROM (
+ SELECT udf(course) as course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 6 schema
+struct
+-- !query 6 output
+63000 5
+
+
+-- !query 7
+SELECT * FROM (
+ SELECT year, course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings)), udf(min(year))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 7 schema
+struct
+-- !query 7 output
+63000 20125 2012
+
+
+-- !query 8
+SELECT * FROM (
+ SELECT course, year, earnings, udf(s) as s
+ FROM courseSales
+ JOIN years ON year = y
+)
+PIVOT (
+ udf(sum(earnings))
+ FOR s IN (1, 2)
+)
+-- !query 8 schema
+struct
+-- !query 8 output
+Java 20122 nan
+Java 2013nan 3
+dotNET 201215000 nan
+dotNET 2013nan 48000
+
+
+-- !query 9
+SELECT * FROM (
+ SELECT course, year, earnings, s
+ FROM courseSales
+ JOIN years ON year = y
+)
+PIVOT (
+ udf(sum(earnings)), udf(min(s))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 9 schema
+struct
+-- !query 9 output
+2012 15000 1 2 1
+2013 48000 2 3 2
+
+
+-- !query 10
+SELECT * FROM (
+ SELECT course, year, earnings, s
+ FROM courseSales
+ JOIN years ON year = y
+)
+PIVOT (
+ udf(sum(earnings * s))
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 10 schema
+struct
+-- !query 10 output
+2012 15000 2
+2013 96000 6
+
+
+-- !query 11
+SELECT 2012_s, 2013_s, 2012_a, 2013_a, c FROM (
+ SELECT year y, course c, earnings e FROM courseSales
+)
+PIVOT (
+ udf(sum(e)) s, udf(avg(e)) a
+ FOR y IN (2012, 2013)
+)
+-- !query 11 schema
+struct<2012_s:string,2013_s:string,2012_a:string,2013_a:string,c:string>
+-- !query 11 output
+15000 48000 7500.0 48000.0 dotNET
+2 3 2.0 3.0 Java
+
+
+-- !query 12
+SELECT firstYear_s, secondYear_s, firstYear_a, secondYear_a, c FROM (
+ SELECT year y, course c, earnings e FROM courseSales
+)
+PIVOT (
+ udf(sum(e)) s, udf(avg(e)) a
+ FOR y IN (2012 as firstYear, 2013 secondYear)
+)
+-- !query 12 schema
+struct
+-- !query 12 output
+15000 48000 7500.0 48000.0 dotNET
+2 3 2.0 3.0 Java
+
+
+-- !query 13
+SELECT * FROM courseSales
+PIVOT (
+ udf(abs(earnings))
+ FOR year IN (2012, 2013)
+)
+-- !query 13 schema
+struct<>
+-- !query 13 output
+org.apache.spark.sql.AnalysisException
+Aggregate expression required for pivot, but 'coursesales.`earnings`' did not
appear in any aggregate function.;
+
+
+-- !query 14
+SELECT * FROM (
+ SELECT year, course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings)), year
+ FOR course IN ('dotNET', 'Java')
+)
+-- !query 14 schema
+struct<>
+-- !query 14 output
+org.apache.spark.sql.AnalysisException
+Aggregate expression required for pivot, but
'__auto_generated_subquery_name.`year`' did not appear in any aggregate
function.;
+
+
+-- !query 15
+SELECT * FROM (
+ SELECT course, earnings FROM courseSales
+)
+PIVOT (
+ udf(sum(earnings))
+ FOR year IN (2012, 2013)
+)
+-- !query 15 schema
+struct<>
+--
[GitHub] [spark] AngersZhuuuu commented on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first.
AngersZh commented on issue #24909: [SPARK-28106][SQL] When Spark SQL use "add jar" , before add to SparkContext, check jar path exist first. URL: https://github.com/apache/spark/pull/24909#issuecomment-510733079 > Do you even need a real file system? I presume it would also fail if it can't read the FS so anything would work. Or will the exception be wrong? Add local jar path won't throw exception. Seems for remote jar file we do like this the same? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API
AmplabJenkins removed a comment on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#issuecomment-510731735 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107566/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API
AmplabJenkins removed a comment on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#issuecomment-510731731 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API
AmplabJenkins commented on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#issuecomment-510731735 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107566/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API
AmplabJenkins commented on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#issuecomment-510731731 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API
SparkQA removed a comment on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#issuecomment-510703613 **[Test build #107566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107566/testReport)** for PR 25007 at commit [`66aae91`](https://github.com/apache/spark/commit/66aae9152341d0c4ba7e26fcd94b51407906fb29). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API
SparkQA commented on issue #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#issuecomment-510731423 **[Test build #107566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107566/testReport)** for PR 25007 at commit [`66aae91`](https://github.com/apache/spark/commit/66aae9152341d0c4ba7e26fcd94b51407906fb29). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24798: [SPARK-27724][SQL] Implement REPLACE TABLE and REPLACE TABLE AS SELECT with V2
AmplabJenkins removed a comment on issue #24798: [SPARK-27724][SQL] Implement REPLACE TABLE and REPLACE TABLE AS SELECT with V2 URL: https://github.com/apache/spark/pull/24798#issuecomment-510731390 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24798: [SPARK-27724][SQL] Implement REPLACE TABLE and REPLACE TABLE AS SELECT with V2
AmplabJenkins removed a comment on issue #24798: [SPARK-27724][SQL] Implement REPLACE TABLE and REPLACE TABLE AS SELECT with V2 URL: https://github.com/apache/spark/pull/24798#issuecomment-510731394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107563/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24798: [SPARK-27724][SQL] Implement REPLACE TABLE and REPLACE TABLE AS SELECT with V2
AmplabJenkins commented on issue #24798: [SPARK-27724][SQL] Implement REPLACE TABLE and REPLACE TABLE AS SELECT with V2 URL: https://github.com/apache/spark/pull/24798#issuecomment-510731394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107563/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
