[GitHub] [spark] HeartSaVioR commented on pull request #24173: [SPARK-27237][SS] Introduce State schema validation among query restart

HeartSaVioR commented on pull request #24173: URL: https://github.com/apache/spark/pull/24173#issuecomment-675299208 retest this, please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HeartSaVioR commented on a change in pull request #29256: [SPARK-32456][SS] Check the Distinct by assuming it as Aggregate for Structured Streaming

HeartSaVioR commented on a change in pull request #29256: URL: https://github.com/apache/spark/pull/29256#discussion_r471960294 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala ## @@ -1106,6 +1107,54 @@ class StreamingQuerySuite

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29461: [DO-NOT-MERGE][SPARK-32456][SS][FOLLOWUP] Update doc to note about using SQL statement with streaming Dataset

AmplabJenkins removed a comment on pull request #29461: URL: https://github.com/apache/spark/pull/29461#issuecomment-675298637 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #25965: [SPARK-26425][SS] Add more constraint checks to avoid checkpoint corruption

AmplabJenkins removed a comment on pull request #25965: URL: https://github.com/apache/spark/pull/25965#issuecomment-675297890 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29461: [DO-NOT-MERGE][SPARK-32456][SS][FOLLOWUP] Update doc to note about using SQL statement with streaming Dataset

AmplabJenkins commented on pull request #29461: URL: https://github.com/apache/spark/pull/29461#issuecomment-675298637 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

AmplabJenkins removed a comment on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-675298265 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

AmplabJenkins removed a comment on pull request #28904: URL: https://github.com/apache/spark/pull/28904#issuecomment-675298132 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

AmplabJenkins removed a comment on pull request #28904: URL: https://github.com/apache/spark/pull/28904#issuecomment-675298123 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29360: [SPARK-32542][SQL] Add an optimizer rule to split an Expand into multiple Expands for aggregates

AmplabJenkins removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-675297814 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

AmplabJenkins removed a comment on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-675297409 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

AmplabJenkins removed a comment on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-675297631 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29360: [SPARK-32542][SQL] Add an optimizer rule to split an Expand into multiple Expands for aggregates

AmplabJenkins commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-675297814 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29453: [SPARK-31999][SQL][FOLLOWUP] Adds negative test cases with typos

AmplabJenkins removed a comment on pull request #29453: URL: https://github.com/apache/spark/pull/29453#issuecomment-675297371 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29457: [SPARK-32646][SQL] ORC predicate pushdown should work with case-insensitive analysis

AmplabJenkins removed a comment on pull request #29457: URL: https://github.com/apache/spark/pull/29457#issuecomment-675297236 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-675297983 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

AmplabJenkins removed a comment on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-675297424 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

AmplabJenkins removed a comment on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-675297620 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

AmplabJenkins removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-675297618 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

AmplabJenkins removed a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675297649 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29461: [DO-NOT-MERGE][SPARK-32456][SS][FOLLOWUP] Update doc to note about using SQL statement with streaming Dataset

SparkQA commented on pull request #29461: URL: https://github.com/apache/spark/pull/29461#issuecomment-675297860 **[Test build #127546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127546/testReport)** for PR 29461 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #25965: [SPARK-26425][SS] Add more constraint checks to avoid checkpoint corruption

AmplabJenkins removed a comment on pull request #25965: URL: https://github.com/apache/spark/pull/25965#issuecomment-675297882 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-675297983 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #25965: [SPARK-26425][SS] Add more constraint checks to avoid checkpoint corruption

AmplabJenkins commented on pull request #25965: URL: https://github.com/apache/spark/pull/25965#issuecomment-675297882 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

SparkQA removed a comment on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-675252986 **[Test build #127534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127534/testReport)** for PR 27694 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

AmplabJenkins commented on pull request #28904: URL: https://github.com/apache/spark/pull/28904#issuecomment-675298123 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29453: [SPARK-31999][SQL][FOLLOWUP] Adds negative test cases with typos

SparkQA removed a comment on pull request #29453: URL: https://github.com/apache/spark/pull/29453#issuecomment-675275574 **[Test build #127542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127542/testReport)** for PR 29453 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29360: [SPARK-32542][SQL] Add an optimizer rule to split an Expand into multiple Expands for aggregates

SparkQA removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-675233817 **[Test build #127526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127526/testReport)** for PR 29360 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

AmplabJenkins removed a comment on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-675297540 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

AmplabJenkins commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675297637 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

AmplabJenkins removed a comment on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675297225 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

SparkQA removed a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675273071 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

SparkQA removed a comment on pull request #28904: URL: https://github.com/apache/spark/pull/28904#issuecomment-675250393 **[Test build #127530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127530/testReport)** for PR 28904 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

AmplabJenkins removed a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675297293 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

SparkQA removed a comment on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675291618 **[Test build #127544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127544/testReport)** for PR 29456 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

AmplabJenkins removed a comment on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-675297527 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

AmplabJenkins commented on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-675298265 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

AmplabJenkins removed a comment on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-675297402 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

SparkQA removed a comment on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-675253028 **[Test build #127537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127537/testReport)** for PR 26935 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #25965: [SPARK-26425][SS] Add more constraint checks to avoid checkpoint corruption

SparkQA removed a comment on pull request #25965: URL: https://github.com/apache/spark/pull/25965#issuecomment-675253090 **[Test build #127538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127538/testReport)** for PR 25965 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

SparkQA removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-675227409 **[Test build #127524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127524/testReport)** for PR 28841 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

SparkQA removed a comment on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-675252940 **[Test build #127535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127535/testReport)** for PR 27649 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29453: [SPARK-31999][SQL][FOLLOWUP] Adds negative test cases with typos

AmplabJenkins removed a comment on pull request #29453: URL: https://github.com/apache/spark/pull/29453#issuecomment-675297359 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

AmplabJenkins removed a comment on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675297211 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

SparkQA removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-675250435 **[Test build #127532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127532/testReport)** for PR 28363 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29457: [SPARK-32646][SQL] ORC predicate pushdown should work with case-insensitive analysis

SparkQA removed a comment on pull request #29457: URL: https://github.com/apache/spark/pull/29457#issuecomment-675294654 **[Test build #127545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127545/testReport)** for PR 29457 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29457: [SPARK-32646][SQL] ORC predicate pushdown should work with case-insensitive analysis

AmplabJenkins removed a comment on pull request #29457: URL: https://github.com/apache/spark/pull/29457#issuecomment-675297229 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

AmplabJenkins commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-675297620 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR commented on pull request #29461: [DO-NOT-MERGE][SPARK-32456][SS][FOLLOWUP] Update doc to note about using SQL statement with streaming Dataset

HeartSaVioR commented on pull request #29461: URL: https://github.com/apache/spark/pull/29461#issuecomment-675297309 I'm marking this as draft as I'd like to see which is preferred - just document to warn about end users (this PR) vs collect and prevent some error-prone operations for

[GitHub] [spark] AmplabJenkins commented on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

AmplabJenkins commented on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-675297527 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

AmplabJenkins removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-675297607 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

AmplabJenkins commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-675297607 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

AmplabJenkins commented on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-675297411 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

SparkQA removed a comment on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-675252970 **[Test build #127536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127536/testReport)** for PR 27333 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

AmplabJenkins removed a comment on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-675297411 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29457: [SPARK-32646][SQL] ORC predicate pushdown should work with case-insensitive analysis

AmplabJenkins commented on pull request #29457: URL: https://github.com/apache/spark/pull/29457#issuecomment-675297229 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

SparkQA commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-675297170 **[Test build #127532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127532/testReport)** for PR 28363 at commit

[GitHub] [spark] SparkQA commented on pull request #29453: [SPARK-31999][SQL][FOLLOWUP] Adds negative test cases with typos

SparkQA commented on pull request #29453: URL: https://github.com/apache/spark/pull/29453#issuecomment-675297169 **[Test build #127542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127542/testReport)** for PR 29453 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

SparkQA removed a comment on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-675250414 **[Test build #127531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127531/testReport)** for PR 28422 at commit

[GitHub] [spark] SparkQA commented on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

SparkQA commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675297152 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins commented on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

AmplabJenkins commented on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-675297402 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

SparkQA commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-675297154 **[Test build #127536 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127536/testReport)** for PR 27333 at commit

[GitHub] [spark] SparkQA commented on pull request #25965: [SPARK-26425][SS] Add more constraint checks to avoid checkpoint corruption

SparkQA commented on pull request #25965: URL: https://github.com/apache/spark/pull/25965#issuecomment-675297180 **[Test build #127538 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127538/testReport)** for PR 25965 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29453: [SPARK-31999][SQL][FOLLOWUP] Adds negative test cases with typos

AmplabJenkins commented on pull request #29453: URL: https://github.com/apache/spark/pull/29453#issuecomment-675297359 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

SparkQA commented on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-675297182 **[Test build #127534 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127534/testReport)** for PR 27694 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

AmplabJenkins commented on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675297211 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

SparkQA commented on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-675297171 **[Test build #127531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127531/testReport)** for PR 28422 at commit

[GitHub] [spark] SparkQA commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

SparkQA commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-675297165 **[Test build #127524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127524/testReport)** for PR 28841 at commit

[GitHub] [spark] SparkQA commented on pull request #29360: [SPARK-32542][SQL] Add an optimizer rule to split an Expand into multiple Expands for aggregates

SparkQA commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-675297167 **[Test build #127526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127526/testReport)** for PR 29360 at commit

[GitHub] [spark] SparkQA commented on pull request #29457: [SPARK-32646][SQL] ORC predicate pushdown should work with case-insensitive analysis

SparkQA commented on pull request #29457: URL: https://github.com/apache/spark/pull/29457#issuecomment-675297166 **[Test build #127545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127545/testReport)** for PR 29457 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

AmplabJenkins commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675297293 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

SparkQA commented on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-675297150 **[Test build #127535 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127535/testReport)** for PR 27649 at commit

[GitHub] [spark] SparkQA commented on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

SparkQA commented on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-675297173 **[Test build #127537 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127537/testReport)** for PR 26935 at commit

[GitHub] [spark] SparkQA commented on pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

SparkQA commented on pull request #28904: URL: https://github.com/apache/spark/pull/28904#issuecomment-675297184 **[Test build #127530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127530/testReport)** for PR 28904 at commit

[GitHub] [spark] SparkQA commented on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

SparkQA commented on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675297168 **[Test build #127544 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127544/testReport)** for PR 29456 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29457: [SPARK-32646][SQL] ORC predicate pushdown should work with case-insensitive analysis

AmplabJenkins removed a comment on pull request #29457: URL: https://github.com/apache/spark/pull/29457#issuecomment-675295113 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HeartSaVioR opened a new pull request #29461: [DO-NOT-MERGE][SPARK-32456][SS][FOLLOWUP] Update doc to note about using SQL statement with streaming Dataset

HeartSaVioR opened a new pull request #29461: URL: https://github.com/apache/spark/pull/29461 ### What changes were proposed in this pull request? This patch proposes to update the doc (both SS guide doc and Dataset dropDuplicates method doc) to leave a note to check on using SQL

[GitHub] [spark] AmplabJenkins commented on pull request #29457: [SPARK-32646][SQL] ORC predicate pushdown should work with case-insensitive analysis

AmplabJenkins commented on pull request #29457: URL: https://github.com/apache/spark/pull/29457#issuecomment-675295113 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29457: [SPARK-32646][SQL] ORC predicate pushdown should work with case-insensitive analysis

SparkQA commented on pull request #29457: URL: https://github.com/apache/spark/pull/29457#issuecomment-675294654 **[Test build #127545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127545/testReport)** for PR 29457 at commit

[GitHub] [spark] gengliangwang commented on a change in pull request #29437: [SPARK-32621][SQL] 'path' option can cause issues while inferring schema in CSV/JSON datasources

gengliangwang commented on a change in pull request #29437: URL: https://github.com/apache/spark/pull/29437#discussion_r471954974 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/FileTable.scala ## @@ -34,13 +35,21 @@ import

[GitHub] [spark] cloud-fan commented on a change in pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

cloud-fan commented on a change in pull request #29452: URL: https://github.com/apache/spark/pull/29452#discussion_r471954231 ## File path: core/src/main/scala/org/apache/spark/scheduler/ExecutorDecommissionInfo.scala ## @@ -18,11 +18,21 @@ package org.apache.spark.scheduler

[GitHub] [spark] AmplabJenkins commented on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

AmplabJenkins commented on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675292135 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

AmplabJenkins removed a comment on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675292135 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

SparkQA commented on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675291618 **[Test build #127544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127544/testReport)** for PR 29456 at commit

[GitHub] [spark] cloud-fan edited a comment on pull request #29395: [3.0][SPARK-32518][CORE] CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

cloud-fan edited a comment on pull request #29395: URL: https://github.com/apache/spark/pull/29395#issuecomment-675290468 thanks, merging to 3.0! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #29395: [3.0][SPARK-32518][CORE] CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

cloud-fan closed pull request #29395: URL: https://github.com/apache/spark/pull/29395 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #29395: [3.0][SPARK-32518][CORE] CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

cloud-fan commented on pull request #29395: URL: https://github.com/apache/spark/pull/29395#issuecomment-675290468 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #29422: [SPARK-32613][CORE] Fix regressions in DecommissionWorkerSuite

cloud-fan closed pull request #29422: URL: https://github.com/apache/spark/pull/29422 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #29422: [SPARK-32613][CORE] Fix regressions in DecommissionWorkerSuite

cloud-fan commented on pull request #29422: URL: https://github.com/apache/spark/pull/29422#issuecomment-675289413 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan commented on pull request #29322: [SPARK-32511][SQL] Add dropFields method to Column class

cloud-fan commented on pull request #29322: URL: https://github.com/apache/spark/pull/29322#issuecomment-675287699 reopened This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #29458: [SPARK-32018][FOLLOWUP][Doc] Add migration guide for decimal value overflow in sum aggregation

cloud-fan commented on a change in pull request #29458: URL: https://github.com/apache/spark/pull/29458#discussion_r471947639 ## File path: docs/sql-migration-guide.md ## @@ -36,6 +36,10 @@ license: | - In Spark 3.1, NULL elements of structures, arrays and maps are

[GitHub] [spark] HeartSaVioR commented on a change in pull request #29256: [SPARK-32456][SS] Check the Distinct by assuming it as Aggregate for Structured Streaming

HeartSaVioR commented on a change in pull request #29256: URL: https://github.com/apache/spark/pull/29256#discussion_r471945148 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala ## @@ -1106,6 +1107,54 @@ class StreamingQuerySuite

[GitHub] [spark] HyukjinKwon commented on pull request #29459: [MINOR][INFRA] Rename master.yml to build_and_test.yml

HyukjinKwon commented on pull request #29459: URL: https://github.com/apache/spark/pull/29459#issuecomment-675283818 Thanks guys. Let me merge this after I cherry-pick Github Actions to other branches (at https://github.com/apache/spark/pull/29460)

[GitHub] [spark] HyukjinKwon commented on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

HyukjinKwon commented on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675283772 Thanks guys. Let me merge this after I cherry-pick Github Actions to other branches (at https://github.com/apache/spark/pull/29460)

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29459: [MINOR][INFRA] Rename master.yml to build_and_test.yml

AmplabJenkins removed a comment on pull request #29459: URL: https://github.com/apache/spark/pull/29459#issuecomment-675280417 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] viirya commented on a change in pull request #29437: [SPARK-32621][SQL] 'path' option can cause issues while inferring schema in CSV/JSON datasources

viirya commented on a change in pull request #29437: URL: https://github.com/apache/spark/pull/29437#discussion_r471933928 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/FileTable.scala ## @@ -34,13 +35,21 @@ import

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29459: [MINOR][INFRA] Rename master.yml to build_and_test.yml

AmplabJenkins removed a comment on pull request #29459: URL: https://github.com/apache/spark/pull/29459#issuecomment-675280413 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29459: [MINOR][INFRA] Rename master.yml to build_and_test.yml

AmplabJenkins commented on pull request #29459: URL: https://github.com/apache/spark/pull/29459#issuecomment-675280413 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29459: [MINOR][INFRA] Rename master.yml to build_and_test.yml

SparkQA removed a comment on pull request #29459: URL: https://github.com/apache/spark/pull/29459#issuecomment-675238599 **[Test build #127529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127529/testReport)** for PR 29459 at commit

[GitHub] [spark] SparkQA commented on pull request #29459: [MINOR][INFRA] Rename master.yml to build_and_test.yml

SparkQA commented on pull request #29459: URL: https://github.com/apache/spark/pull/29459#issuecomment-675279846 **[Test build #127529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127529/testReport)** for PR 29459 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #29256: [SPARK-32456][SS] Check the Distinct by assuming it as Aggregate for Structured Streaming

cloud-fan commented on a change in pull request #29256: URL: https://github.com/apache/spark/pull/29256#discussion_r471939568 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala ## @@ -1106,6 +1107,54 @@ class StreamingQuerySuite

<    4   5   6   7   8   9   10   >