[GitHub] [spark] AmplabJenkins removed a comment on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
AmplabJenkins removed a comment on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587595676 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
AmplabJenkins removed a comment on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587595680 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23400/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27555: [SPARK-30802][ML] Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite
AmplabJenkins removed a comment on issue #27555: [SPARK-30802][ML] Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite URL: https://github.com/apache/spark/pull/27555#issuecomment-587595739 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27555: [SPARK-30802][ML] Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite
AmplabJenkins removed a comment on issue #27555: [SPARK-30802][ML] Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite URL: https://github.com/apache/spark/pull/27555#issuecomment-587595757 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23401/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
imback82 commented on a change in pull request #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#discussion_r380843983
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
##
@@ -470,9 +470,15 @@ trait CheckAnalysis extends PredicateHelper {
}
}
+val colsToDelete = mutable.Set.empty[Seq[String]]
Review comment:
Yes, working on it now!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27555: [SPARK-30802][ML] Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite
AmplabJenkins commented on issue #27555: [SPARK-30802][ML] Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite URL: https://github.com/apache/spark/pull/27555#issuecomment-587595739 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27555: [SPARK-30802][ML] Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite
AmplabJenkins commented on issue #27555: [SPARK-30802][ML] Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite URL: https://github.com/apache/spark/pull/27555#issuecomment-587595757 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23401/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
AmplabJenkins commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587595680 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23400/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
AmplabJenkins commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587595676 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table
AmplabJenkins removed a comment on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table URL: https://github.com/apache/spark/pull/27538#issuecomment-587595044 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table
AmplabJenkins removed a comment on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table URL: https://github.com/apache/spark/pull/27538#issuecomment-587595057 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118637/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table
AmplabJenkins commented on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table URL: https://github.com/apache/spark/pull/27538#issuecomment-587595044 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table
AmplabJenkins commented on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table URL: https://github.com/apache/spark/pull/27538#issuecomment-587595057 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118637/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
SparkQA commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587595090 **[Test build #118647 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118647/testReport)** for PR 27562 at commit [`797cd1e`](https://github.com/apache/spark/commit/797cd1e4771ba141f3e3f5071638b13d6c6fbd17). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table
SparkQA removed a comment on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table URL: https://github.com/apache/spark/pull/27538#issuecomment-587457097 **[Test build #118637 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118637/testReport)** for PR 27538 at commit [`cf28b07`](https://github.com/apache/spark/commit/cf28b073351798338cd8db8adb2c9433945de198). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
dongjoon-hyun commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587593642 Thank you for updating, @hvanhovell . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table
SparkQA commented on issue #27538: [SPARK-30785][SQL] Create table like should keep tracksPartitionsInCatalog same with source table URL: https://github.com/apache/spark/pull/27538#issuecomment-587594146 **[Test build #118637 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118637/testReport)** for PR 27538 at commit [`cf28b07`](https://github.com/apache/spark/commit/cf28b073351798338cd8db8adb2c9433945de198). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
dongjoon-hyun commented on a change in pull request #27562:
[SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to
non-existent table with same name
URL: https://github.com/apache/spark/pull/27562#discussion_r380840956
##
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
##
@@ -2641,4 +2641,12 @@ class DataFrameSuite extends QueryTest with
SharedSQLContext {
val idTuples = sampled.collect().map(row => row.getLong(0) ->
row.getLong(1))
assert(idTuples.length == idTuples.toSet.size)
}
+
+ test("SPARK-30811: CTE should not cause stack overflow when " +
Review comment:
Got it, @hvanhovell .
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
dongjoon-hyun commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587593560 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27623: [SPARK-30867][ML] Add FValueRegressionTest
AmplabJenkins removed a comment on issue #27623: [SPARK-30867][ML] Add FValueRegressionTest URL: https://github.com/apache/spark/pull/27623#issuecomment-587585231 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27623: [SPARK-30867][ML] Add FValueRegressionTest
AmplabJenkins removed a comment on issue #27623: [SPARK-30867][ML] Add FValueRegressionTest URL: https://github.com/apache/spark/pull/27623#issuecomment-587585239 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23399/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27623: [SPARK-30867][ML] Add FValueRegressionTest
AmplabJenkins commented on issue #27623: [SPARK-30867][ML] Add FValueRegressionTest URL: https://github.com/apache/spark/pull/27623#issuecomment-587585231 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27623: [SPARK-30867][ML] Add FValueRegressionTest
AmplabJenkins commented on issue #27623: [SPARK-30867][ML] Add FValueRegressionTest URL: https://github.com/apache/spark/pull/27623#issuecomment-587585239 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23399/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27623: [SPARK-30867][ML] Add FValueRegressionTest
SparkQA commented on issue #27623: [SPARK-30867][ML] Add FValueRegressionTest URL: https://github.com/apache/spark/pull/27623#issuecomment-587584230 **[Test build #118646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118646/testReport)** for PR 27623 at commit [`6b21229`](https://github.com/apache/spark/commit/6b21229f60f6067963a9fd57f9ec3fc012dc5420). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on issue #27527: [SPARK-30776][ML] Support FValueRegressionSelector for continuous features and continuous labels
huaxingao commented on issue #27527: [SPARK-30776][ML] Support FValueRegressionSelector for continuous features and continuous labels URL: https://github.com/apache/spark/pull/27527#issuecomment-587584326 > However, I think this PR is too large to track, I suggest to split it into three parts... I will close this PR and open smaller PRs. The comments are addressed in the new PR. Thanks a lot for reviewing! Please see https://github.com/apache/spark/pull/27623 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao closed pull request #27527: [SPARK-30776][ML] Support FValueRegressionSelector for continuous features and continuous labels
huaxingao closed pull request #27527: [SPARK-30776][ML] Support FValueRegressionSelector for continuous features and continuous labels URL: https://github.com/apache/spark/pull/27527 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
AmplabJenkins removed a comment on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587581937 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118644/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
AmplabJenkins removed a comment on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587581917 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder
AmplabJenkins removed a comment on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder URL: https://github.com/apache/spark/pull/27592#issuecomment-587581475 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118629/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
AmplabJenkins commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587581937 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118644/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
AmplabJenkins commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587581917 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder
AmplabJenkins commented on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder URL: https://github.com/apache/spark/pull/27592#issuecomment-587581462 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder
AmplabJenkins commented on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder URL: https://github.com/apache/spark/pull/27592#issuecomment-587581475 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118629/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao opened a new pull request #27623: [SPARK-30867][ML] Add FValueRegressionTest
huaxingao opened a new pull request #27623: [SPARK-30867][ML] Add FValueRegressionTest URL: https://github.com/apache/spark/pull/27623 ### What changes were proposed in this pull request? This is the very first PR for supporting continuous distribution features selectors. It adds the algorithm to compute fvalue for continuous features and continuous labels. This algorithm will be used for FValueRegressionSelector. ### Why are the changes needed? Current Spark only supports the selection of categorical features, while there are many requirements for the selection of continuous distribution features. I will add two new selectors: 1. FValueRegressionSelector for continuous features and continuous labels. 2. ANOVAFValueClassificationSelector for continuous features and categorical labels. I will use subtasks to add these two selectors: add FValueRegressionSelector on scala side - add FValueRegressionTest, this contains the algorithm to compute FValue - add FValueRegressionSelector using the above algorithm - add a common Selector, make FValueRegressionSelector and ChisqSelector to extend common selector add FValueRegressionSelector on python side add samples and doc do the same for ANOVAFValueClassificationSelector ### Does this PR introduce any user-facing change? Yes. ``` /** * @param dataset DataFrame of continuous labels and continuous features. * @param featuresCol Name of features column in dataset, of type `Vector` (`VectorUDT`) * @param labelCol Name of label column in dataset, of any numerical type * @return Array containing the SelectionTestResult for every feature against the label. */ SelectionTest.fValueRegressionTest(dataset: Dataset[_], featuresCol: String, labelCol: String) ``` ### How was this patch tested? Add Unit test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder
AmplabJenkins removed a comment on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder URL: https://github.com/apache/spark/pull/27592#issuecomment-587581462 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
SparkQA removed a comment on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587507255 **[Test build #118644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118644/testReport)** for PR 27041 at commit [`3718df9`](https://github.com/apache/spark/commit/3718df989c02641f864432b246313a158a11b9e6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder
SparkQA removed a comment on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder URL: https://github.com/apache/spark/pull/27592#issuecomment-587400790 **[Test build #118629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118629/testReport)** for PR 27592 at commit [`6d8eb75`](https://github.com/apache/spark/commit/6d8eb75f0c29962962f994d8f212fafae8577cfc). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder
SparkQA commented on issue #27592: [SPARK-30840][CORE][SQL] Add version property for ConfigEntry and ConfigBuilder URL: https://github.com/apache/spark/pull/27592#issuecomment-587580975 **[Test build #118629 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118629/testReport)** for PR 27592 at commit [`6d8eb75`](https://github.com/apache/spark/commit/6d8eb75f0c29962962f994d8f212fafae8577cfc). * This patch **fails from timeout after a configured wait of `400m`**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
SparkQA commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587581230 **[Test build #118644 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118644/testReport)** for PR 27041 at commit [`3718df9`](https://github.com/apache/spark/commit/3718df989c02641f864432b246313a158a11b9e6). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27622: [SPARK-27528][followup] improve migration guide
MaxGekk commented on a change in pull request #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#discussion_r380807952 ## File path: docs/sql-migration-guide.md ## @@ -87,7 +87,7 @@ license: | - In Spark version 2.4, when a spark session is created via `cloneSession()`, the newly created spark session inherits its configuration from its parent `SparkContext` even though the same configuration may exist with a different value in its parent spark session. Since Spark 3.0, the configurations of a parent `SparkSession` have a higher precedence over the parent `SparkContext`. The old behavior can be restored by setting `spark.sql.legacy.sessionInitWithConfigDefaults` to `true`. - - Since Spark 3.0, parquet logical type `TIMESTAMP_MICROS` is used by default while saving `TIMESTAMP` columns. In Spark version 2.4 and earlier, `TIMESTAMP` columns are saved as `INT96` in parquet files. To set `INT96` to `spark.sql.parquet.outputTimestampType` restores the previous behavior. + - Since Spark 3.0, parquet logical type `TIMESTAMP_MICROS` is used by default while saving `TIMESTAMP` columns. In Spark version 2.4 and earlier, `TIMESTAMP` columns are saved as `INT96` in parquet files. Note that, some SQL systems like Impala 2.x can only read `INT96` timestamps, you can set `spark.sql.parquet.outputTimestampType` as `INT96` to restore the previous behavior and keep interoperability. Review comment: I think you guard yourself so much. Impala has already supported the same as Spark: https://github.com/cloudera/Impala/blob/63ff49de748692758bc465b3787c4193de78936c/be/src/exec/parquet/parquet-common.h#L489-L501 And I am not sure that we should advertise other systems in Spark docs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision
AmplabJenkins removed a comment on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision URL: https://github.com/apache/spark/pull/27618#issuecomment-587558462 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118631/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision
SparkQA removed a comment on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision URL: https://github.com/apache/spark/pull/27618#issuecomment-587425129 **[Test build #118631 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118631/testReport)** for PR 27618 at commit [`7d812a3`](https://github.com/apache/spark/commit/7d812a3e5224b275b5633727d2a49459d16cc467). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision
AmplabJenkins removed a comment on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision URL: https://github.com/apache/spark/pull/27618#issuecomment-587558452 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision
AmplabJenkins commented on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision URL: https://github.com/apache/spark/pull/27618#issuecomment-587558452 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision
AmplabJenkins commented on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision URL: https://github.com/apache/spark/pull/27618#issuecomment-587558462 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118631/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision
SparkQA commented on issue #27618: [WIP][SQL] Convert dates to/from timestamps in microseconds precision URL: https://github.com/apache/spark/pull/27618#issuecomment-587557157 **[Test build #118631 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118631/testReport)** for PR 27618 at commit [`7d812a3`](https://github.com/apache/spark/commit/7d812a3e5224b275b5633727d2a49459d16cc467). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager
AmplabJenkins removed a comment on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager URL: https://github.com/apache/spark/pull/27539#issuecomment-587550688 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118638/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager
AmplabJenkins removed a comment on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager URL: https://github.com/apache/spark/pull/27539#issuecomment-587550670 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager
AmplabJenkins commented on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager URL: https://github.com/apache/spark/pull/27539#issuecomment-587550670 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager
AmplabJenkins commented on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager URL: https://github.com/apache/spark/pull/27539#issuecomment-587550688 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118638/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager
SparkQA removed a comment on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager URL: https://github.com/apache/spark/pull/27539#issuecomment-587467281 **[Test build #118638 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118638/testReport)** for PR 27539 at commit [`91e3a36`](https://github.com/apache/spark/commit/91e3a360f061546ef172a994b60a736baa1ea7d9). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager
SparkQA commented on issue #27539: [SPARK-30786] [CORE] Fix Block replication failure propogation issue in BlockManager URL: https://github.com/apache/spark/pull/27539#issuecomment-587549432 **[Test build #118638 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118638/testReport)** for PR 27539 at commit [`91e3a36`](https://github.com/apache/spark/commit/91e3a360f061546ef172a994b60a736baa1ea7d9). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-587538148 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-587538167 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23398/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-587538148 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-587538167 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23398/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
SparkQA commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-587537503 **[Test build #118645 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118645/testReport)** for PR 27495 at commit [`93eb3c5`](https://github.com/apache/spark/commit/93eb3c5f3d07da70468f55d49ef1853d663a8288). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor.
AmplabJenkins removed a comment on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor. URL: https://github.com/apache/spark/pull/26193#issuecomment-587522899 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23391/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor.
AmplabJenkins removed a comment on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor. URL: https://github.com/apache/spark/pull/26193#issuecomment-587522889 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor.
AmplabJenkins commented on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor. URL: https://github.com/apache/spark/pull/26193#issuecomment-587522889 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor.
AmplabJenkins commented on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor. URL: https://github.com/apache/spark/pull/26193#issuecomment-587522899 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23391/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2
AmplabJenkins removed a comment on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2 URL: https://github.com/apache/spark/pull/27609#issuecomment-587519390 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118636/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2
AmplabJenkins removed a comment on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2 URL: https://github.com/apache/spark/pull/27609#issuecomment-587519369 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2
AmplabJenkins commented on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2 URL: https://github.com/apache/spark/pull/27609#issuecomment-587519369 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2
AmplabJenkins commented on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2 URL: https://github.com/apache/spark/pull/27609#issuecomment-587519390 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118636/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2
SparkQA removed a comment on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2 URL: https://github.com/apache/spark/pull/27609#issuecomment-587447623 **[Test build #118636 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118636/testReport)** for PR 27609 at commit [`d1c7894`](https://github.com/apache/spark/commit/d1c78940d9ebb222dd69825db9bc03480167c8a9). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2
SparkQA commented on issue #27609: [WIP][SPARK-27733][CORE] Upgrade Avro to 1.9.2 URL: https://github.com/apache/spark/pull/27609#issuecomment-587518793 **[Test build #118636 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118636/testReport)** for PR 27609 at commit [`d1c7894`](https://github.com/apache/spark/commit/d1c78940d9ebb222dd69825db9bc03480167c8a9). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor.
SparkQA commented on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor. URL: https://github.com/apache/spark/pull/26193#issuecomment-587514793 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/23391/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
AmplabJenkins removed a comment on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587507799 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
AmplabJenkins removed a comment on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587507816 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23397/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
AmplabJenkins commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587507799 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
AmplabJenkins commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587507816 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23397/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
SparkQA commented on issue #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#issuecomment-587507255 **[Test build #118644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118644/testReport)** for PR 27041 at commit [`3718df9`](https://github.com/apache/spark/commit/3718df989c02641f864432b246313a158a11b9e6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ajithme commented on a change in pull request #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
ajithme commented on a change in pull request #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter URL: https://github.com/apache/spark/pull/27041#discussion_r380731555 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala ## @@ -176,9 +176,18 @@ private[hive] class HiveClientImpl( // is not set to builtin. When spark.sql.hive.metastore.jars is builtin, the classpath // has hive-site.xml. So, HiveConf will use that to override its default values. // 2: we set all spark confs to this hiveConf. -// 3: we set all entries in config to this hiveConf. +// 3: we take the conf passed as --hiveconf which would be set as system properties +// by org.apache.hive.service.server.HiveServer2.ServerOptionsProcessor.parse in +// org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main. +// 4: we set all entries in extraConfig to this hiveConf which have the highest precedence. +// To summarize, the order of precedence will be +// hadoopConf < sparkConf < overrideProps < extraConfig + +// not to lose command line overwritten properties +// make a copy overridden props so that it can be reinserted finally +val overriddenHiveProps = HiveConf.getConfSystemProperties.asScala Review comment: @cloud-fan updated. is the comment adequate now? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR edited a comment on issue #27620: [SPARK-30866][SS] FileStreamSource: Cache fetched list of files beyond maxFilesPerTrigger as unread files
HeartSaVioR edited a comment on issue #27620: [SPARK-30866][SS] FileStreamSource: Cache fetched list of files beyond maxFilesPerTrigger as unread files URL: https://github.com/apache/spark/pull/27620#issuecomment-587471848 The patch is actually very straightforward about how it works and how it helps (as the changeset except the test code is very small). I'll attach the test result for the use case of "initial load" in the section of "How was this patch tested?" sooner. I've already have screenshots of UI, but would like to run against latest master. EDIT: Just updated the description of PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
AmplabJenkins removed a comment on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587499251 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118634/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27620: [SPARK-30866][SS] FileStreamSource: Cache fetched list of files beyond maxFilesPerTrigger as unread files
HeartSaVioR commented on issue #27620: [SPARK-30866][SS] FileStreamSource: Cache fetched list of files beyond maxFilesPerTrigger as unread files URL: https://github.com/apache/spark/pull/27620#issuecomment-587504565 cc. @tdas @zsxwing @gaborgsomogyi This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27622: [SPARK-27528][followup] improve migration guide
AmplabJenkins removed a comment on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587502102 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
AmplabJenkins removed a comment on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587499233 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27622: [SPARK-27528][followup] improve migration guide
AmplabJenkins removed a comment on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587502112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118643/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27622: [SPARK-27528][followup] improve migration guide
AmplabJenkins commented on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587502112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118643/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27622: [SPARK-27528][followup] improve migration guide
AmplabJenkins commented on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587502102 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27622: [SPARK-27528][followup] improve migration guide
SparkQA commented on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587501899 **[Test build #118643 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118643/testReport)** for PR 27622 at commit [`b719294`](https://github.com/apache/spark/commit/b71929423721238fe0011c08a7aeadb66c0fb6ec). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27622: [SPARK-27528][followup] improve migration guide
SparkQA removed a comment on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587495601 **[Test build #118643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118643/testReport)** for PR 27622 at commit [`b719294`](https://github.com/apache/spark/commit/b71929423721238fe0011c08a7aeadb66c0fb6ec). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
SparkQA removed a comment on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587441765 **[Test build #118634 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118634/testReport)** for PR 27562 at commit [`797cd1e`](https://github.com/apache/spark/commit/797cd1e4771ba141f3e3f5071638b13d6c6fbd17). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
AmplabJenkins commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587499233 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
AmplabJenkins commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587499251 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118634/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name
SparkQA commented on issue #27562: [SPARK-30811][SQL][2.4] CTE should not cause stack overflow when it refers to non-existent table with same name URL: https://github.com/apache/spark/pull/27562#issuecomment-587498810 **[Test build #118634 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118634/testReport)** for PR 27562 at commit [`797cd1e`](https://github.com/apache/spark/commit/797cd1e4771ba141f3e3f5071638b13d6c6fbd17). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor.
SparkQA commented on issue #26193: [SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor. URL: https://github.com/apache/spark/pull/26193#issuecomment-587497744 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/23391/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27267: [SPARK-30556][SQL] Copy sparkContext.localproperties to child thread inSubqueryExec.executionContext
cloud-fan commented on a change in pull request #27267: [SPARK-30556][SQL] Copy
sparkContext.localproperties to child thread inSubqueryExec.executionContext
URL: https://github.com/apache/spark/pull/27267#discussion_r380719316
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala
##
@@ -164,4 +166,20 @@ object SQLExecution {
}
}
}
+
+ /**
+ * Wrap passed function to ensure necessary thread-local variables like
+ * SparkContext local properties are forwarded to execution thread
+ */
+ def withThreadLocalCaptured[T](
+ sparkSession: SparkSession, exec: ExecutionContext)(body: => T):
Future[T] = {
+val activeSession = sparkSession
+val sc = sparkSession.sparkContext
+val localProps = Utils.cloneProperties(sc.getLocalProperties)
+Future {
+ SparkSession.setActiveSession(activeSession)
+ sc.setLocalProperties(localProps)
Review comment:
the `localProps` is already a clone:
https://github.com/apache/spark/pull/27267/files#diff-ab49028253e599e6e74cc4f4dcb2e3a8R178
And I think the order doesn't matter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27622: [SPARK-27528][followup] improve migration guide
AmplabJenkins commented on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587496359 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23396/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27622: [SPARK-27528][followup] improve migration guide
AmplabJenkins removed a comment on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587496359 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23396/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27622: [SPARK-27528][followup] improve migration guide
AmplabJenkins commented on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587496353 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27622: [SPARK-27528][followup] improve migration guide
AmplabJenkins removed a comment on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587496353 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-587495714 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118630/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-587495714 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118630/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins removed a comment on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-587495701 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments
AmplabJenkins commented on issue #27495: [SPARK-28880][SQL] Support ANSI nested bracketed comments URL: https://github.com/apache/spark/pull/27495#issuecomment-587495701 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27621: [SPARK-26071][followup] Improve migration guide of disallowing map type map key
SparkQA removed a comment on issue #27621: [SPARK-26071][followup] Improve migration guide of disallowing map type map key URL: https://github.com/apache/spark/pull/27621#issuecomment-587484621 **[Test build #118641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118641/testReport)** for PR 27621 at commit [`7840774`](https://github.com/apache/spark/commit/784077423991aa8d02cbd112cca3fe037ceb8d2b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27622: [SPARK-27528][followup] improve migration guide
SparkQA commented on issue #27622: [SPARK-27528][followup] improve migration guide URL: https://github.com/apache/spark/pull/27622#issuecomment-587495601 **[Test build #118643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118643/testReport)** for PR 27622 at commit [`b719294`](https://github.com/apache/spark/commit/b71929423721238fe0011c08a7aeadb66c0fb6ec). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
