[GitHub] [spark] SparkQA commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
SparkQA commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576278836 **[Test build #117119 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117119/testReport)** for PR 27277 at commit [`a796d68`](https://github.com/apache/spark/commit/a796d68c67ff50ad89b50895b531c4a7ac6b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table
AmplabJenkins commented on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table URL: https://github.com/apache/spark/pull/27249#issuecomment-576278660 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table
AmplabJenkins commented on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table URL: https://github.com/apache/spark/pull/27249#issuecomment-576278671 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117109/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table
AmplabJenkins removed a comment on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table URL: https://github.com/apache/spark/pull/27249#issuecomment-576278660 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table
AmplabJenkins removed a comment on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table URL: https://github.com/apache/spark/pull/27249#issuecomment-576278671 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117109/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table
SparkQA removed a comment on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table URL: https://github.com/apache/spark/pull/27249#issuecomment-576185544 **[Test build #117109 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117109/testReport)** for PR 27249 at commit [`8e70ec1`](https://github.com/apache/spark/commit/8e70ec1e59b81a777c8c8efdc4d663c3fbb77f9d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table
SparkQA commented on issue #27249: [SPARK-30019][SQL] Add the owner property to v2 table URL: https://github.com/apache/spark/pull/27249#issuecomment-576277985 **[Test build #117109 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117109/testReport)** for PR 27249 at commit [`8e70ec1`](https://github.com/apache/spark/commit/8e70ec1e59b81a777c8c8efdc4d663c3fbb77f9d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
cloud-fan closed pull request #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
cloud-fan commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576276761 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand
cloud-fan commented on a change in pull request #27293:
[SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand
URL: https://github.com/apache/spark/pull/27293#discussion_r368546144
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala
##
@@ -65,7 +76,7 @@ object CommandUtils extends Logging {
// Calculate table size as a sum of the visible partitions. See
SPARK-21079
val partitions =
sessionState.catalog.listPartitions(catalogTable.identifier)
logInfo(s"Starting to calculate sizes for ${partitions.length}
partitions.")
- if (spark.sessionState.conf.parallelFileListingInStatsComputation) {
+ if (spark.sessionState.conf.parallelFileListingInCommands) {
Review comment:
shall we call the new method here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand
cloud-fan commented on a change in pull request #27293:
[SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand
URL: https://github.com/apache/spark/pull/27293#discussion_r368544554
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -1370,8 +1370,8 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
- val PARALLEL_FILE_LISTING_IN_STATS_COMPUTATION =
-
buildConf("spark.sql.statistics.parallelFileListingInStatsComputation.enabled")
+ val PARALLEL_FILE_LISTING_IN_COMMANDS =
+buildConf("spark.sql.parallelFileListingInCommands.enabled")
Review comment:
do we have to rename it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand
AmplabJenkins removed a comment on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand URL: https://github.com/apache/spark/pull/27293#issuecomment-576270361 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand
AmplabJenkins removed a comment on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand URL: https://github.com/apache/spark/pull/27293#issuecomment-576270368 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117113/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand
AmplabJenkins commented on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand URL: https://github.com/apache/spark/pull/27293#issuecomment-576270368 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117113/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand
AmplabJenkins commented on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand URL: https://github.com/apache/spark/pull/27293#issuecomment-576270361 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand
SparkQA removed a comment on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand URL: https://github.com/apache/spark/pull/27293#issuecomment-576208446 **[Test build #117113 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117113/testReport)** for PR 27293 at commit [`03ca0a3`](https://github.com/apache/spark/commit/03ca0a3907173411cc6d083adba51a9b9feaaca2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand
SparkQA commented on issue #27293: [SPARK-29938][SQL][FOLLOW-UP] Improve AlterTableAddPartitionCommand URL: https://github.com/apache/spark/pull/27293#issuecomment-576269291 **[Test build #117113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117113/testReport)** for PR 27293 at commit [`03ca0a3`](https://github.com/apache/spark/commit/03ca0a3907173411cc6d083adba51a9b9feaaca2). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class PathFilterIgnoreNonData(stagingDir: String) extends PathFilter with Serializable ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
AmplabJenkins removed a comment on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576266765 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
AmplabJenkins removed a comment on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576266773 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21882/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
AmplabJenkins commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576266765 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
AmplabJenkins commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576266773 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21882/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
AmplabJenkins removed a comment on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576265962 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117105/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
AmplabJenkins removed a comment on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576265948 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
SparkQA commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576266216 **[Test build #117118 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117118/testReport)** for PR 27265 at commit [`743981c`](https://github.com/apache/spark/commit/743981c8845fd364228e092d1b61666f5c39e9b6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
AmplabJenkins commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576265962 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117105/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
AmplabJenkins commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576265948 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
SparkQA removed a comment on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576164275 **[Test build #117105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117105/testReport)** for PR 27243 at commit [`bb21f16`](https://github.com/apache/spark/commit/bb21f16203ff1917fae60bcb2e7244aac0eaa8db). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
SparkQA commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576265360 **[Test build #117105 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117105/testReport)** for PR 27243 at commit [`bb21f16`](https://github.com/apache/spark/commit/bb21f16203ff1917fae60bcb2e7244aac0eaa8db). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint
AmplabJenkins removed a comment on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint URL: https://github.com/apache/spark/pull/27252#issuecomment-576261808 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117104/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint
AmplabJenkins removed a comment on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint URL: https://github.com/apache/spark/pull/27252#issuecomment-576261802 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint
AmplabJenkins commented on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint URL: https://github.com/apache/spark/pull/27252#issuecomment-576261802 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint
AmplabJenkins commented on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint URL: https://github.com/apache/spark/pull/27252#issuecomment-576261808 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117104/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint
SparkQA removed a comment on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint URL: https://github.com/apache/spark/pull/27252#issuecomment-576164274 **[Test build #117104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117104/testReport)** for PR 27252 at commit [`7dcfe91`](https://github.com/apache/spark/commit/7dcfe915087dbe274b470928600197745a645f5e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint
SparkQA commented on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint URL: https://github.com/apache/spark/pull/27252#issuecomment-576261199 **[Test build #117104 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117104/testReport)** for PR 27252 at commit [`7dcfe91`](https://github.com/apache/spark/commit/7dcfe915087dbe274b470928600197745a645f5e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
AmplabJenkins removed a comment on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576258034 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
AmplabJenkins removed a comment on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576258041 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117099/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
AmplabJenkins commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576258041 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117099/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
AmplabJenkins commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576258034 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
SparkQA commented on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576257378 **[Test build #117099 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117099/testReport)** for PR 27265 at commit [`763d64d`](https://github.com/apache/spark/commit/763d64df80c8330f971124d9b988581c1f6484c4). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
SparkQA removed a comment on issue #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table URL: https://github.com/apache/spark/pull/27265#issuecomment-576160962 **[Test build #117099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117099/testReport)** for PR 27265 at commit [`763d64d`](https://github.com/apache/spark/commit/763d64df80c8330f971124d9b988581c1f6484c4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite
AmplabJenkins removed a comment on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite URL: https://github.com/apache/spark/pull/27285#issuecomment-576256797 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117098/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite
AmplabJenkins removed a comment on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite URL: https://github.com/apache/spark/pull/27285#issuecomment-576256786 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite
AmplabJenkins commented on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite URL: https://github.com/apache/spark/pull/27285#issuecomment-576256786 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite
AmplabJenkins commented on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite URL: https://github.com/apache/spark/pull/27285#issuecomment-576256797 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117098/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite
SparkQA removed a comment on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite URL: https://github.com/apache/spark/pull/27285#issuecomment-576160944 **[Test build #117098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117098/testReport)** for PR 27285 at commit [`a7c071d`](https://github.com/apache/spark/commit/a7c071d9cce850376cbbe1dafe1b65d8ebdd9943). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite
SparkQA commented on issue #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite URL: https://github.com/apache/spark/pull/27285#issuecomment-576256159 **[Test build #117098 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117098/testReport)** for PR 27285 at commit [`a7c071d`](https://github.com/apache/spark/commit/a7c071d9cce850376cbbe1dafe1b65d8ebdd9943). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
AmplabJenkins commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576255654 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21881/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
AmplabJenkins removed a comment on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576255654 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21881/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
AmplabJenkins commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576255644 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
AmplabJenkins removed a comment on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576255644 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
SparkQA commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576255130 **[Test build #117117 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117117/testReport)** for PR 22878 at commit [`e59e58c`](https://github.com/apache/spark/commit/e59e58c847389a3c7d674745667f8be0f8f7fb0e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
xuanyuanking commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576253086 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
xuanyuanking commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576252991 The failed test `CSVSuite.SPARK-23786: warning should be printed if CSV header doesn't conform to schema` can pass locally. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins commented on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27237#issuecomment-576251543 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117096/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins removed a comment on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27237#issuecomment-576251521 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins removed a comment on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27237#issuecomment-576251543 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117096/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins commented on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27237#issuecomment-576251521 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
SparkQA removed a comment on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27237#issuecomment-576158036 **[Test build #117096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117096/testReport)** for PR 27237 at commit [`e39ef9a`](https://github.com/apache/spark/commit/e39ef9aa9b1023a6fff9013e3180adfeb0dd8bac). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
SparkQA commented on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27237#issuecomment-576250893 **[Test build #117096 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117096/testReport)** for PR 27237 at commit [`e39ef9a`](https://github.com/apache/spark/commit/e39ef9aa9b1023a6fff9013e3180adfeb0dd8bac). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27268: [SPARK-30553][DOCS] fix structured-streaming java example error
HeartSaVioR commented on issue #27268: [SPARK-30553][DOCS] fix structured-streaming java example error URL: https://github.com/apache/spark/pull/27268#issuecomment-576250811 I assume you are now having two example queries "before the fix" vs "after the fix". Once you run the query and ingest some data, you'll see the query plan in SQL tab, as you've added like `HashAggregate(keys=[window#11-T1ms, value#39]`. It would be enough to fill up the content and add below: ``` > Before the fix \``` \``` > After the fix \``` \``` ``` Please remove `\` above when pasting to the content of PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576245295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21880/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576245287 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576245287 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576245295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21880/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
SparkQA commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576244862 **[Test build #117116 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117116/testReport)** for PR 27278 at commit [`de6000c`](https://github.com/apache/spark/commit/de6000c1ab2f9857c8449ddc1853cf5507d622ac). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
zero323 commented on a change in pull request #27278:
[SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
URL: https://github.com/apache/spark/pull/27278#discussion_r368510992
##
File path: R/pkg/R/functions.R
##
@@ -1332,6 +1332,48 @@ setMethod("quarter",
column(jc)
})
+#' @details
+#' \code{percentile_approx} Returns the approximate percentile value of
+#' numeric column at the given percentage.
+#'
+#' @param percentage numeric percentage at which percentile should be computed
+#' All values should be between 0 and 1.
+#' If length equals to 1 resulting column is of type double,
+#' otherwise array/
+#' @param accuracy parameter (default: 1) is a positive numeric literal
which
+#' controls approximation accuracy at the cost of memory. Higher value of
accuracy
+#' yields better accuracy, 1.0/accuracy is the relative error of the
approximation.
+#'
+#' @rdname column_aggregate_functions
+#' @aliases percentile_approx percentile_approx,Column-method
+#' @note percentile_approx since 3.0.0
+setMethod("percentile_approx",
+ signature(x = "characterOrColumn", percentage = "numeric", accuracy
= "numeric"),
+ function(x, percentage, accuracy) {
Review comment:
I think this will requires switching to ellipsis in generic:
```r
setGeneric("percentile_approx", function(x, percentage, ...) {
standardGeneric("percentile_approx") })
setMethod("percentile_approx",
signature(x = "characterOrColumn", percentage = "numeric"),
function(x, percentage, accuracy = 1) {
...
```
or `ANY` in a `signature`
```r
setGeneric("percentile_approx",
function(x, percentage, accuracy = 1) {
standardGeneric("percentile_approx") })
setMethod("percentile_approx",
signature(x = "characterOrColumn", percentage = "numeric",
accuracy = "ANY"),
function(x, percentage, accuracy = 1) {
...
```
and none seemed very precise, but I don't have strong opinion about it.
Maybe the latter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
zero323 commented on a change in pull request #27278:
[SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
URL: https://github.com/apache/spark/pull/27278#discussion_r368511059
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -652,6 +652,122 @@ object functions {
*/
def min(columnName: String): Column = min(Column(columnName))
+ /**
+ * Aggregate function: Returns and array of the approximate percentile values
+ * of numeric column col at the given percentages.
+ *
+ * Each value of the percentage array must be between 0.0 and 1.0.
+ *
+ * The accuracy parameter is a positive numeric literal
+ * which controls approximation accuracy at the cost of memory.
+ * Higher value of accuracy yields better accuracy, 1.0/accuracy
+ * is the relative error of the approximation.
+ *
+ * @group agg_funcs
+ * @since 3.0.0
+ */
+ def percentile_approx(e: Column, percentage: Array[Double], accuracy: Long):
Column = {
+withAggregateFunction {
+ new ApproximatePercentile(
+e.expr, typedLit(percentage).expr, lit(accuracy).expr
+ )
+}
+ }
+
+ /**
+ * Aggregate function: Returns and array of the approximate percentile values
+ * of numeric column col at the given percentages.
+ *
+ * Each value of the percentage array must be between 0.0 and 1.0.
+ *
+ * The accuracy parameter is a positive numeric literal
+ * which controls approximation accuracy at the cost of memory.
+ * Higher value of accuracy yields better accuracy, 1.0/accuracy
+ * is the relative error of the approximation.
+ *
+ * @group agg_funcs
+ * @since 3.0.0
+ */
+ def percentile_approx(columnName: String, percentage: Array[Double],
accuracy: Long): Column = {
+percentile_approx(Column(columnName), percentage, accuracy)
+ }
+
+ /**
+ * Aggregate function: Returns and array of the approximate percentile values
+ * of numeric column col at the given percentages.
+ *
+ * Each value of the percentage array must be between 0.0 and 1.0.
+ *
+ * The accuracy parameter is a positive numeric literal
+ * which controls approximation accuracy at the cost of memory.
+ * Higher value of accuracy yields better accuracy, 1.0/accuracy
+ * is the relative error of the approximation.
+ *
+ * @group agg_funcs
+ * @since 3.0.0
+ */
+ def percentile_approx(e: Column, percentage: Seq[Double], accuracy: Long):
Column = {
+percentile_approx(e, percentage.toArray, accuracy)
+ }
+
+ /**
+ * Aggregate function: Returns and array of the approximate percentile values
+ * of numeric column col at the given percentages.
+ *
+ * Each value of the percentage array must be between 0.0 and 1.0.
+ *
+ * The accuracy parameter is a positive numeric literal
+ * which controls approximation accuracy at the cost of memory.
+ * Higher value of accuracy yields better accuracy, 1.0/accuracy
+ * is the relative error of the approximation.
+ *
+ * @group agg_funcs
+ * @since 3.0.0
+ */
+ def percentile_approx(columnName: String, percentage: Seq[Double], accuracy:
Long): Column = {
+percentile_approx(Column(columnName), percentage.toArray, accuracy)
+ }
+
+ /**
+ * Aggregate function: Returns the approximate percentile value of numeric
+ * column col at the given percentage.
+ *
+ * The value of percentage must be between 0.0 and 1.0.\
+ *
+ * The accuracy parameter is a positive numeric literal
+ * which controls approximation accuracy at the cost of memory.
+ * Higher value of accuracy yields better accuracy, 1.0/accuracy
+ * is the relative error of the approximation.
+ *
+ * @group agg_funcs
+ * @since 3.0.0
+ */
+ def percentile_approx(e: Column, percentage: Double, accuracy: Long): Column
= {
+withAggregateFunction {
+ new ApproximatePercentile(
+e.expr, lit(percentage).expr, lit(accuracy).expr
+ )
+}
+ }
+
+ /**
+ * Aggregate function: Returns the approximate percentile value of numeric
+ * column col at the given percentage.
+ *
+ * The value of percentage must be between 0.0 and 1.0.\
+ *
+ * The accuracy parameter is a positive numeric literal
+ * which controls approximation accuracy at the cost of memory.
+ * Higher value of accuracy yields better accuracy, 1.0/accuracy
+ * is the relative error of the approximation.
+ *
+ * @group agg_funcs
+ * @since 3.0.0
+ */
+ def percentile_approx(columnName: String, percentage: Double, accuracy:
Long): Column = {
Review comment:
Do you mean we should remove variants `(String, _, _) -> Column` and leave
only `(Column, _, _) -> Column`? Or rather replace all variants with `(Column,
Column, Column) -> Column`?
I'll be happy to do the former, but the latter is probably a bad idea:
- Aggregation by group gets confusing.
- Array variants get somewhat painful to use for anyone but Scala users ‒
building array li
[GitHub] [spark] AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576242409 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21879/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576242399 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576242409 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21879/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576242399 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
AmplabJenkins removed a comment on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576241707 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117107/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
zero323 commented on a change in pull request #27278:
[SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
URL: https://github.com/apache/spark/pull/27278#discussion_r368509689
##
File path: python/pyspark/sql/functions.py
##
@@ -582,6 +582,51 @@ def nanvl(col1, col2):
return Column(sc._jvm.functions.nanvl(_to_java_column(col1),
_to_java_column(col2)))
+@since(3.0)
+def percentile_approx(col, percentage, accuracy=1):
+"""Returns the approximate percentile value of numeric column col at the
given percentage.
+The value of percentage must be between 0.0 and 1.0.
+
+The accuracy parameter (default: 1)
+is a positive numeric literal which controls approximation accuracy at the
cost of memory.
+Higher value of accuracy yields better accuracy, 1.0/accuracy is the
relative error
+of the approximation.
+
+When percentage is an array, each value of the percentage array must be
between 0.0 and 1.0.
+In this case, returns the approximate percentile array of column col
+at the given percentage array.
+
+>>> df = (spark.range(0, 1000, 1, 1)
+... .withColumn("id", col("id") % 3)
+... .withColumn("value", randn(42) + col("id") * 10))
+>>> (df
+... .select(percentile_approx("value", [0.25, 0.5, 0.75],
100).alias("quantiles"))
+... .show(truncate=False))
++--+
+|quantiles |
++--+
+|[0.7264430125286507, 9.98975299938167, 19.335304783039014]|
Review comment:
Maybe `printSchema`? I think the main point is to show that schema can
differ from input to input.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
SparkQA removed a comment on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576182281 **[Test build #117107 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117107/testReport)** for PR 22878 at commit [`e59e58c`](https://github.com/apache/spark/commit/e59e58c847389a3c7d674745667f8be0f8f7fb0e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
AmplabJenkins removed a comment on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576241702 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
AmplabJenkins commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576241707 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117107/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
AmplabJenkins commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576241702 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro
SparkQA commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-576241544 **[Test build #117107 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117107/testReport)** for PR 22878 at commit [`e59e58c`](https://github.com/apache/spark/commit/e59e58c847389a3c7d674745667f8be0f8f7fb0e). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class AvroEncoderSuite extends SharedSparkSession ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
AmplabJenkins removed a comment on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576239635 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21878/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
AmplabJenkins removed a comment on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576239626 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
AmplabJenkins commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576239626 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
AmplabJenkins commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576239635 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21878/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped
AmplabJenkins removed a comment on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped URL: https://github.com/apache/spark/pull/27002#issuecomment-576238007 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117111/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] bettermouse commented on issue #27268: [SPARK-30553][DOCS] fix structured-streaming java example error
bettermouse commented on issue #27268: [SPARK-30553][DOCS] fix structured-streaming java example error URL: https://github.com/apache/spark/pull/27268#issuecomment-576239481 @HeartSaVioR Sorry.This is my first PR.Can you help me understand what `plan information` is. I think we just need update the documentation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
SparkQA commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576239251 **[Test build #117115 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117115/testReport)** for PR 27277 at commit [`9e5b209`](https://github.com/apache/spark/commit/9e5b20919b6ba95054b4541c2148bb2049a40e28). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped
AmplabJenkins removed a comment on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped URL: https://github.com/apache/spark/pull/27002#issuecomment-576238002 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped
SparkQA removed a comment on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped URL: https://github.com/apache/spark/pull/27002#issuecomment-576191992 **[Test build #117111 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117111/testReport)** for PR 27002 at commit [`a34d020`](https://github.com/apache/spark/commit/a34d0207b6b763f9ff9732cf5d6646c1d5359152). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27177: [SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
HyukjinKwon commented on a change in pull request #27177: [SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier URL: https://github.com/apache/spark/pull/27177#discussion_r368505186 ## File path: dev/deps/spark-deps-hadoop-3.2-hive-2.3 ## @@ -1,239 +1,239 @@ -HikariCP-2.5.1.jar -JLargeArrays-1.5.jar -JTransforms-3.1.jar -RoaringBitmap-0.7.45.jar -ST4-4.0.4.jar -accessors-smart-1.2.jar -activation-1.1.1.jar -aircompressor-0.10.jar -algebra_2.12-2.0.0-M2.jar -antlr-runtime-3.5.2.jar -antlr4-runtime-4.7.1.jar -aopalliance-1.0.jar -aopalliance-repackaged-2.6.1.jar -arpack_combined_all-0.1.jar -arrow-format-0.15.1.jar -arrow-memory-0.15.1.jar -arrow-vector-0.15.1.jar -audience-annotations-0.5.0.jar -automaton-1.11-8.jar -avro-1.8.2.jar -avro-ipc-1.8.2.jar -avro-mapred-1.8.2-hadoop2.jar -bonecp-0.8.0.RELEASE.jar -breeze-macros_2.12-1.0.jar -breeze_2.12-1.0.jar -cats-kernel_2.12-2.0.0-M4.jar -chill-java-0.9.3.jar -chill_2.12-0.9.3.jar -commons-beanutils-1.9.4.jar -commons-cli-1.2.jar -commons-codec-1.10.jar -commons-collections-3.2.2.jar -commons-compiler-3.0.15.jar -commons-compress-1.8.1.jar -commons-configuration2-2.1.1.jar -commons-crypto-1.0.0.jar -commons-daemon-1.0.13.jar -commons-dbcp-1.4.jar -commons-httpclient-3.1.jar -commons-io-2.4.jar -commons-lang-2.6.jar -commons-lang3-3.9.jar -commons-logging-1.1.3.jar -commons-math3-3.4.1.jar -commons-net-3.1.jar -commons-pool-1.5.4.jar -commons-text-1.6.jar -compress-lzf-1.0.3.jar -core-1.1.2.jar -curator-client-2.13.0.jar -curator-framework-2.13.0.jar -curator-recipes-2.13.0.jar -datanucleus-api-jdo-4.2.4.jar -datanucleus-core-4.1.17.jar -datanucleus-rdbms-4.1.19.jar -derby-10.12.1.1.jar -dnsjava-2.1.7.jar -dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar -ehcache-3.3.1.jar -flatbuffers-java-1.9.0.jar -generex-1.0.2.jar -geronimo-jcache_1.0_spec-1.0-alpha-1.jar -gson-2.2.4.jar -guava-14.0.1.jar -guice-4.0.jar -guice-servlet-4.0.jar -hadoop-annotations-3.2.0.jar -hadoop-auth-3.2.0.jar -hadoop-client-3.2.0.jar -hadoop-common-3.2.0.jar -hadoop-hdfs-client-3.2.0.jar -hadoop-mapreduce-client-common-3.2.0.jar -hadoop-mapreduce-client-core-3.2.0.jar -hadoop-mapreduce-client-jobclient-3.2.0.jar -hadoop-yarn-api-3.2.0.jar -hadoop-yarn-client-3.2.0.jar -hadoop-yarn-common-3.2.0.jar -hadoop-yarn-registry-3.2.0.jar -hadoop-yarn-server-common-3.2.0.jar -hadoop-yarn-server-web-proxy-3.2.0.jar -hive-beeline-2.3.6.jar -hive-cli-2.3.6.jar -hive-common-2.3.6.jar -hive-exec-2.3.6-core.jar -hive-jdbc-2.3.6.jar -hive-llap-common-2.3.6.jar -hive-metastore-2.3.6.jar -hive-serde-2.3.6.jar -hive-service-rpc-2.3.6.jar -hive-shims-0.23-2.3.6.jar -hive-shims-2.3.6.jar -hive-shims-common-2.3.6.jar -hive-shims-scheduler-2.3.6.jar -hive-storage-api-2.6.0.jar -hive-vector-code-gen-2.3.6.jar -hk2-api-2.6.1.jar -hk2-locator-2.6.1.jar -hk2-utils-2.6.1.jar -htrace-core4-4.1.0-incubating.jar -httpclient-4.5.6.jar -httpcore-4.4.12.jar -istack-commons-runtime-3.0.8.jar -ivy-2.4.0.jar -jackson-annotations-2.10.0.jar -jackson-core-2.10.0.jar -jackson-core-asl-1.9.13.jar -jackson-databind-2.10.0.jar -jackson-dataformat-yaml-2.10.0.jar -jackson-jaxrs-base-2.9.5.jar -jackson-jaxrs-json-provider-2.9.5.jar -jackson-mapper-asl-1.9.13.jar -jackson-module-jaxb-annotations-2.10.0.jar -jackson-module-paranamer-2.10.0.jar -jackson-module-scala_2.12-2.10.0.jar -jakarta.activation-api-1.2.1.jar -jakarta.annotation-api-1.3.5.jar -jakarta.inject-2.6.1.jar -jakarta.validation-api-2.0.2.jar -jakarta.ws.rs-api-2.1.6.jar -jakarta.xml.bind-api-2.3.2.jar -janino-3.0.15.jar -javassist-3.22.0-CR2.jar -javax.inject-1.jar -javax.jdo-3.2.0-m3.jar -javax.servlet-api-3.1.0.jar -javolution-5.5.1.jar -jaxb-api-2.2.11.jar -jaxb-runtime-2.3.2.jar -jcip-annotations-1.0-1.jar -jcl-over-slf4j-1.7.16.jar -jdo-api-3.0.1.jar -jersey-client-2.29.1.jar -jersey-common-2.29.1.jar -jersey-container-servlet-2.29.1.jar -jersey-container-servlet-core-2.29.1.jar -jersey-hk2-2.29.1.jar -jersey-media-jaxb-2.29.1.jar -jersey-server-2.29.1.jar -jline-2.14.6.jar -joda-time-2.10.5.jar -jodd-core-3.5.2.jar -jpam-1.1.jar -json-1.8.jar -json-smart-2.3.jar -json4s-ast_2.12-3.6.6.jar -json4s-core_2.12-3.6.6.jar -json4s-jackson_2.12-3.6.6.jar -json4s-scalap_2.12-3.6.6.jar -jsp-api-2.1.jar -jsr305-3.0.0.jar -jta-1.1.jar -jul-to-slf4j-1.7.16.jar -kerb-admin-1.0.1.jar -kerb-client-1.0.1.jar -kerb-common-1.0.1.jar -kerb-core-1.0.1.jar -kerb-crypto-1.0.1.jar -kerb-identity-1.0.1.jar -kerb-server-1.0.1.jar -kerb-simplekdc-1.0.1.jar -kerb-util-1.0.1.jar -kerby-asn1-1.0.1.jar -kerby-config-1.0.1.jar -kerby-pkix-1.0.1.jar -kerby-util-1.0.1.jar -kerby-xdr-1.0.1.jar -kryo-shaded-4.0.2.jar -kubernetes-client-4.6.4.jar -kubernetes-model-4.6.4.jar -kubernetes-model-common-4.6.4.jar -leveldbjni-all-1.8.jar -libfb303-0.9.3.jar -libthrift-0.12.0.jar -log4j-1.2.17.jar -logging-interceptor-3.12.6.jar -lz4-java-1.7.0.jar -machinist_2.12-0.6.8.jar -macro-compat_2.12-1.1.1.jar -mesos-1.4.0-shaded-protobuf.jar -metrics-core-
[GitHub] [spark] AmplabJenkins commented on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped
AmplabJenkins commented on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped URL: https://github.com/apache/spark/pull/27002#issuecomment-576238002 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped
AmplabJenkins commented on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped URL: https://github.com/apache/spark/pull/27002#issuecomment-576238007 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117111/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped
SparkQA commented on issue #27002: [SPARK-30346][CORE]Improve logging when events dropped URL: https://github.com/apache/spark/pull/27002#issuecomment-576237689 **[Test build #117111 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117111/testReport)** for PR 27002 at commit [`a34d020`](https://github.com/apache/spark/commit/a34d0207b6b763f9ff9732cf5d6646c1d5359152). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #27258: [SPARK-30547][SQL] Add unstable annotation to the CalendarInterval class
yaooqinn commented on a change in pull request #27258: [SPARK-30547][SQL] Add unstable annotation to the CalendarInterval class URL: https://github.com/apache/spark/pull/27258#discussion_r368502206 ## File path: common/unsafe/src/main/java/org/apache/spark/unsafe/types/CalendarInterval.java ## @@ -27,8 +29,21 @@ import static org.apache.spark.sql.catalyst.util.DateTimeConstants.*; /** - * The internal representation of interval type. + * The class representing calendar intervals. The calendar interval is stored internally in + * three components: + * + * an integer value representing the number of `months` in this interval, + * an integer value representing the number of `days` in this interval, + * a long value representing the number of `microseconds` in this interval. + * + * + * The `months` and `days` are not units of time with a constant length (unlike hours, seconds), so + * they are two separated fields from microseconds. One month may be equal to 28, 29, 30 or 31 days + * and one day may be equal to 23, 24 or 25 hours (daylight saving). + * + * @since 1.5.0 Review comment: checked the java doc about since https://www.oracle.com/technetwork/java/javase/documentation/index-137868.html#@since >When a class (or interface) is introduced, specify one since tag in its class description and no since tags in the members. Add a since tag only to members added in a later version than the class. ... > If a member changes from protected to public in a later release, the since tag would not change, even though it is now usable by any caller, not just subclassers. `1.5.0` should be fine I guess This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27177: [SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
HyukjinKwon commented on issue #27177: [SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier URL: https://github.com/apache/spark/pull/27177#issuecomment-576234897 Thanks for pinging me @dongjoon-hyun. I just saw this .. :-).. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
AmplabJenkins removed a comment on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576232673 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117102/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
AmplabJenkins commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576232673 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117102/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
AmplabJenkins removed a comment on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576232661 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
SparkQA removed a comment on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576164283 **[Test build #117102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117102/testReport)** for PR 27277 at commit [`4fc3f4b`](https://github.com/apache/spark/commit/4fc3f4b93c212639489dc4b5b6b3432c58c5cd5a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
SparkQA commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576232409 **[Test build #117102 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117102/testReport)** for PR 27277 at commit [`4fc3f4b`](https://github.com/apache/spark/commit/4fc3f4b93c212639489dc4b5b6b3432c58c5cd5a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema
AmplabJenkins commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576232661 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27265: [SPARK-30555][SQL] MERGE INTO insert action should only access columns from source table
cloud-fan commented on a change in pull request #27265: [SPARK-30555][SQL]
MERGE INTO insert action should only access columns from source table
URL: https://github.com/apache/spark/pull/27265#discussion_r368494661
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##
@@ -1326,13 +1326,20 @@ class Analyzer(
DeleteAction(resolvedDeleteCondition)
case UpdateAction(updateCondition, assignments) =>
val resolvedUpdateCondition =
updateCondition.map(resolveExpressionTopDown(_, m))
-UpdateAction(resolvedUpdateCondition,
resolveAssignments(assignments, m))
+// The update value can access columns from both target and source
tables.
+UpdateAction(
+ resolvedUpdateCondition,
+ resolveAssignments(assignments, m, resolveValuesWithSourceOnly =
false))
case o => o
}
val newNotMatchedActions = m.notMatchedActions.map {
case InsertAction(insertCondition, assignments) =>
val resolvedInsertCondition =
insertCondition.map(resolveExpressionTopDown(_, m))
Review comment:
good catch!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
