[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

SparkQA commented on pull request #31680: URL: https://github.com/apache/spark/pull/31680#issuecomment-809088540 **[Test build #136638 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136638/testReport)** for PR 31680 at commit

[GitHub] [spark] SparkQA commented on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

SparkQA commented on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809088302 **[Test build #136637 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136637/testReport)** for PR 31984 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31987: [WIP][SPARK-34889][SS] Introduce MergingSessionsIterator merging elements directly which belong to the same session

AmplabJenkins removed a comment on pull request #31987: URL: https://github.com/apache/spark/pull/31987#issuecomment-809087002 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136627/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31989: [WIP][SPARK-34891][SS] Introduce state store manager for session window in streaming query

AmplabJenkins removed a comment on pull request #31989: URL: https://github.com/apache/spark/pull/31989#issuecomment-809086999 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136632/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

AmplabJenkins removed a comment on pull request #31985: URL: https://github.com/apache/spark/pull/31985#issuecomment-809087006 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41212/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

AmplabJenkins removed a comment on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809087005 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136623/

[GitHub] [spark] AmplabJenkins commented on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

AmplabJenkins commented on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809087005 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136623/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #31989: [WIP][SPARK-34891][SS] Introduce state store manager for session window in streaming query

AmplabJenkins commented on pull request #31989: URL: https://github.com/apache/spark/pull/31989#issuecomment-809086999 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136632/ -- This

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

SparkQA commented on pull request #31680: URL: https://github.com/apache/spark/pull/31680#issuecomment-809087041 **[Test build #136636 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136636/testReport)** for PR 31680 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31987: [WIP][SPARK-34889][SS] Introduce MergingSessionsIterator merging elements directly which belong to the same session

AmplabJenkins commented on pull request #31987: URL: https://github.com/apache/spark/pull/31987#issuecomment-809087002 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136627/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

AmplabJenkins commented on pull request #31985: URL: https://github.com/apache/spark/pull/31985#issuecomment-809087006 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41212/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

AmplabJenkins removed a comment on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809083608 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41206/

[GitHub] [spark] AmplabJenkins commented on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

AmplabJenkins commented on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809083608 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41206/ --

[GitHub] [spark] SparkQA commented on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

SparkQA commented on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809083569 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41206/ -- This is an automated message from the

[GitHub] [spark] MaxGekk closed pull request #31979: [SPARK-34879][SQL] HiveInspector supports DayTimeIntervalType and YearMonthIntervalType

MaxGekk closed pull request #31979: URL: https://github.com/apache/spark/pull/31979 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31901: [SPARK-34802][SQL] Move simplify expression rules before operator push down

AmplabJenkins removed a comment on pull request #31901: URL: https://github.com/apache/spark/pull/31901#issuecomment-809079486 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41208/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31987: [WIP][SPARK-34889][SS] Introduce MergingSessionsIterator merging elements directly which belong to the same session

AmplabJenkins removed a comment on pull request #31987: URL: https://github.com/apache/spark/pull/31987#issuecomment-809079495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41210/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

AmplabJenkins removed a comment on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809079487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

AmplabJenkins removed a comment on pull request #31985: URL: https://github.com/apache/spark/pull/31985#issuecomment-809079491 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136625/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31983: [SPARK-34882][SQL] Replace if with filter clause in RewriteDistinctAggregates

AmplabJenkins removed a comment on pull request #31983: URL: https://github.com/apache/spark/pull/31983#issuecomment-809079490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41209/

[GitHub] [spark] MaxGekk commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

MaxGekk commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809080557 +1, LGTM. Merging to master. Thank you @AngersZh . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #31958: [SPARK-34862][SQL] Support nested column in ORC vectorized reader

SparkQA commented on pull request #31958: URL: https://github.com/apache/spark/pull/31958#issuecomment-809080274 **[Test build #136635 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136635/testReport)** for PR 31958 at commit

[GitHub] [spark] SparkQA commented on pull request #31983: [SPARK-34882][SQL] Replace if with filter clause in RewriteDistinctAggregates

SparkQA commented on pull request #31983: URL: https://github.com/apache/spark/pull/31983#issuecomment-809080227 **[Test build #136634 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136634/testReport)** for PR 31983 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31983: [SPARK-34882][SQL] Replace if with filter clause in RewriteDistinctAggregates

AmplabJenkins commented on pull request #31983: URL: https://github.com/apache/spark/pull/31983#issuecomment-809079490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41209/ --

[GitHub] [spark] AmplabJenkins commented on pull request #31901: [SPARK-34802][SQL] Move simplify expression rules before operator push down

AmplabJenkins commented on pull request #31901: URL: https://github.com/apache/spark/pull/31901#issuecomment-809079486 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41208/ --

[GitHub] [spark] AmplabJenkins commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

AmplabJenkins commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809079487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins commented on pull request #31987: [WIP][SPARK-34889][SS] Introduce MergingSessionsIterator merging elements directly which belong to the same session

AmplabJenkins commented on pull request #31987: URL: https://github.com/apache/spark/pull/31987#issuecomment-809079495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41210/ --

[GitHub] [spark] AmplabJenkins commented on pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

AmplabJenkins commented on pull request #31985: URL: https://github.com/apache/spark/pull/31985#issuecomment-809079491 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136625/ -- This

[GitHub] [spark] MaxGekk commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

MaxGekk commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-80907 > Apache Spark master branch doesn't have Hive 1.2 @dongjoon-hyun Thank you for the information. @AngersZh Sorry, I wasn't aware of that it was removed. -- This

[GitHub] [spark] maropu commented on a change in pull request #31982: [SPARK-34881][SQL] New SQL Function: TRY_CAST

maropu commented on a change in pull request #31982: URL: https://github.com/apache/spark/pull/31982#discussion_r603013688 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/TryCast.scala ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache

[GitHub] [spark] sarutak commented on a change in pull request #31964: [SPARK-34872][SQL] quoteIfNeeded should quote a name which contains non-word characters

sarutak commented on a change in pull request #31964: URL: https://github.com/apache/spark/pull/31964#discussion_r603019469 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/package.scala ## @@ -148,10 +148,10 @@ package object util extends Logging

[GitHub] [spark] SparkQA removed a comment on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

SparkQA removed a comment on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809025371 **[Test build #136622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136622/testReport)** for PR 31979 at commit

[GitHub] [spark] SparkQA commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

SparkQA commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809070189 **[Test build #136622 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136622/testReport)** for PR 31979 at commit

[GitHub] [spark] HeartSaVioR edited a comment on pull request #31989: [WIP][SPARK-34891][SS] Introduce state store manager for session window in streaming query

HeartSaVioR edited a comment on pull request #31989: URL: https://github.com/apache/spark/pull/31989#issuecomment-809067233 Except the test suite, one more thing worths to address here is write amplification; we "blindly" replace all start times and all sessions. This could bring

[GitHub] [spark] SparkQA commented on pull request #31901: [SPARK-34802][SQL] Move simplify expression rules before operator push down

SparkQA commented on pull request #31901: URL: https://github.com/apache/spark/pull/31901#issuecomment-809069105 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41208/ -- This is an automated message from the

[GitHub] [spark] HeartSaVioR commented on pull request #31989: [WIP][SPARK-34891][SS] Introduce state store manager for session window in streaming query

HeartSaVioR commented on pull request #31989: URL: https://github.com/apache/spark/pull/31989#issuecomment-809067233 Except the test suite, one more thing worths to address here is write amplification; we "blindly" replace all start times and all sessions. This could bring unnecessary

[GitHub] [spark] SparkQA removed a comment on pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

SparkQA removed a comment on pull request #31985: URL: https://github.com/apache/spark/pull/31985#issuecomment-809042739 **[Test build #136625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136625/testReport)** for PR 31985 at commit

[GitHub] [spark] SparkQA commented on pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

SparkQA commented on pull request #31985: URL: https://github.com/apache/spark/pull/31985#issuecomment-809066739 **[Test build #136625 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136625/testReport)** for PR 31985 at commit

[GitHub] [spark] c21 commented on a change in pull request #31958: [SPARK-34862][SQL] Support nested column in ORC vectorized reader

c21 commented on a change in pull request #31958: URL: https://github.com/apache/spark/pull/31958#discussion_r603014404 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -838,6 +838,13 @@ object SQLConf { .intConf

[GitHub] [spark] SparkQA commented on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

SparkQA commented on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809064991 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41206/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #31983: [SPARK-34882][SQL] Replace if with filter clause in RewriteDistinctAggregates

SparkQA commented on pull request #31983: URL: https://github.com/apache/spark/pull/31983#issuecomment-809064604 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41209/ --

[GitHub] [spark] SparkQA commented on pull request #31901: [SPARK-34802][SQL] Move simplify expression rules before operator push down

SparkQA commented on pull request #31901: URL: https://github.com/apache/spark/pull/31901#issuecomment-809064539 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41208/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

SparkQA removed a comment on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809024472 **[Test build #136621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136621/testReport)** for PR 31979 at commit

[GitHub] [spark] SparkQA commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

SparkQA commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809063741 **[Test build #136621 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136621/testReport)** for PR 31979 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31983: [SPARK-34882][SQL] Replace if with filter clause in RewriteDistinctAggregates

AmplabJenkins removed a comment on pull request #31983: URL: https://github.com/apache/spark/pull/31983#issuecomment-809062638 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136626/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

AmplabJenkins removed a comment on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809062636 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41207/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

AmplabJenkins removed a comment on pull request #31680: URL: https://github.com/apache/spark/pull/31680#issuecomment-809062635 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136633/

[GitHub] [spark] AmplabJenkins commented on pull request #31983: [SPARK-34882][SQL] Replace if with filter clause in RewriteDistinctAggregates

AmplabJenkins commented on pull request #31983: URL: https://github.com/apache/spark/pull/31983#issuecomment-809062638 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136626/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

AmplabJenkins commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809062636 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41207/ --

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

AmplabJenkins commented on pull request #31680: URL: https://github.com/apache/spark/pull/31680#issuecomment-809062635 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136633/ -- This

[GitHub] [spark] SparkQA commented on pull request #31989: [WIP][SPARK-34891][SS] Introduce state store manager for session window in streaming query

SparkQA commented on pull request #31989: URL: https://github.com/apache/spark/pull/31989#issuecomment-809062214 **[Test build #136632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136632/testReport)** for PR 31989 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #31937: [SPARK-10816][SS] Support session window natively

HeartSaVioR commented on pull request #31937: URL: https://github.com/apache/spark/pull/31937#issuecomment-809062076 I filed 5 JIRA issues for all parts, and submitted 3 PRs which are not dependent to others. Remaining 2 parts depend on others and I'll deal with them once we merge

[GitHub] [spark] SparkQA commented on pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

SparkQA commented on pull request #31985: URL: https://github.com/apache/spark/pull/31985#issuecomment-809061510 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41205/ -- This is an automated message from the Apache

[GitHub] [spark] HeartSaVioR opened a new pull request #31989: [WIP][SPARK-34891][SS] Introduce state store manager for session window in streaming query

HeartSaVioR opened a new pull request #31989: URL: https://github.com/apache/spark/pull/31989 Introduction: this PR is a part of SPARK-10816 (`EventTime based sessionization (session window)`). Please refer #31937 to see the overall view of the code change. (Note that code diff could be

[GitHub] [spark] SparkQA commented on pull request #31988: [SPARK-34855][CORE] Avoid local lazy variable in SparkContext.getCallSite

SparkQA commented on pull request #31988: URL: https://github.com/apache/spark/pull/31988#issuecomment-809059717 **[Test build #136631 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136631/testReport)** for PR 31988 at commit

[GitHub] [spark] viirya commented on pull request #31988: [SPARK-34855][CORE] Avoid local lazy variable in SparkContext.getCallSite

viirya commented on pull request #31988: URL: https://github.com/apache/spark/pull/31988#issuecomment-809059452 cc @HyukjinKwon @srowen @lxian -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] viirya opened a new pull request #31988: [SPARK-34855][CORE] Avoid local lazy variable in SparkContext.getCallSite

viirya opened a new pull request #31988: URL: https://github.com/apache/spark/pull/31988 ### What changes were proposed in this pull request? `SparkContext.getCallSite` uses local lazy variable. In Scala 2.11, local lazy val requires synchronization so for large number

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

SparkQA commented on pull request #31680: URL: https://github.com/apache/spark/pull/31680#issuecomment-809059073 **[Test build #136630 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136630/testReport)** for PR 31680 at commit

[GitHub] [spark] SparkQA commented on pull request #31986: [SPARK-34888][SS] Introduce UpdatingSessionIterator adjusting session window on elements

SparkQA commented on pull request #31986: URL: https://github.com/apache/spark/pull/31986#issuecomment-809058962 **[Test build #136628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136628/testReport)** for PR 31986 at commit

[GitHub] [spark] SparkQA commented on pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

SparkQA commented on pull request #31985: URL: https://github.com/apache/spark/pull/31985#issuecomment-809058974 **[Test build #136629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136629/testReport)** for PR 31985 at commit

[GitHub] [spark] SparkQA commented on pull request #31987: [WIP][SPARK-34889][SS] Introduce MergingSessionsIterator merging elements directly which belong to the same session

SparkQA commented on pull request #31987: URL: https://github.com/apache/spark/pull/31987#issuecomment-809058939 **[Test build #136627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136627/testReport)** for PR 31987 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

AmplabJenkins removed a comment on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809058108 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136620/

[GitHub] [spark] tanelk commented on pull request #31973: [SPARK-34876][SQL] Fill defaultResult of non-nullable aggregates

tanelk commented on pull request #31973: URL: https://github.com/apache/spark/pull/31973#issuecomment-809058487 @HyukjinKwon , There is a failure on branch-2.4. I believe it is because `CountIf` exists since 3.0. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] AmplabJenkins commented on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

AmplabJenkins commented on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809058108 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136620/ -- This

[GitHub] [spark] HeartSaVioR opened a new pull request #31987: [WIP][SPARK-34889][SS] Introduce MergingSessionsIterator merging elements directly which belong to the same session

HeartSaVioR opened a new pull request #31987: URL: https://github.com/apache/spark/pull/31987 Introduction: this PR is a part of SPARK-10816 (`EventTime based sessionization (session window)`). Please refer #31937 to see the overall view of the code change. (Note that code diff could be

[GitHub] [spark] viirya commented on a change in pull request #31953: [SPARK-34855][CORE]spark context - avoid using local lazy val for callSite

viirya commented on a change in pull request #31953: URL: https://github.com/apache/spark/pull/31953#discussion_r603005164 ## File path: core/src/main/scala/org/apache/spark/SparkContext.scala ## @@ -2186,13 +2186,22 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] [spark] SparkQA removed a comment on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

SparkQA removed a comment on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809007618 **[Test build #136620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136620/testReport)** for PR 31984 at commit

[GitHub] [spark] SparkQA commented on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

SparkQA commented on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809053124 **[Test build #136620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136620/testReport)** for PR 31984 at commit

[GitHub] [spark] HeartSaVioR opened a new pull request #31986: [SPARK-34888][SS] Introduce UpdatingSessionIterator adjusting session window on elements

HeartSaVioR opened a new pull request #31986: URL: https://github.com/apache/spark/pull/31986 Introduction: this PR is a part of SPARK-10816 (`EventTime based sessionization (session window)`). Please refer #31937 to see the overall view of the code change. (Note that code diff could be

[GitHub] [spark] AngersZhuuuu commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

AngersZh commented on pull request #31680: URL: https://github.com/apache/spark/pull/31680#issuecomment-809051861 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31953: [SPARK-34855][CORE]spark context - avoid using local lazy val for callSite

HyukjinKwon commented on a change in pull request #31953: URL: https://github.com/apache/spark/pull/31953#discussion_r603003296 ## File path: core/src/main/scala/org/apache/spark/SparkContext.scala ## @@ -2186,13 +2186,22 @@ class SparkContext(config: SparkConf) extends

[GitHub] [spark] viirya commented on a change in pull request #31953: [SPARK-34855][CORE]spark context - avoid using local lazy val for callSite

viirya commented on a change in pull request #31953: URL: https://github.com/apache/spark/pull/31953#discussion_r603000582 ## File path: core/src/main/scala/org/apache/spark/SparkContext.scala ## @@ -2186,13 +2186,22 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] [spark] viirya commented on a change in pull request #31966: [SPARK-34638][SQL] Single field nested column prune on generator output

viirya commented on a change in pull request #31966: URL: https://github.com/apache/spark/pull/31966#discussion_r602999807 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -231,6 +231,27 @@ object

[GitHub] [spark] viirya commented on a change in pull request #31966: [SPARK-34638][SQL] Single field nested column prune on generator output

viirya commented on a change in pull request #31966: URL: https://github.com/apache/spark/pull/31966#discussion_r602999635 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -241,12 +262,69 @@ object

[GitHub] [spark] viirya commented on a change in pull request #31966: [SPARK-34638][SQL] Single field nested column prune on generator output

viirya commented on a change in pull request #31966: URL: https://github.com/apache/spark/pull/31966#discussion_r602999476 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -241,12 +262,69 @@ object

[GitHub] [spark] viirya commented on a change in pull request #31966: [SPARK-34638][SQL] Single field nested column prune on generator output

viirya commented on a change in pull request #31966: URL: https://github.com/apache/spark/pull/31966#discussion_r602998834 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -241,12 +262,69 @@ object

[GitHub] [spark] viirya commented on a change in pull request #31966: [SPARK-34638][SQL] Single field nested column prune on generator output

viirya commented on a change in pull request #31966: URL: https://github.com/apache/spark/pull/31966#discussion_r602997773 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -241,12 +262,69 @@ object

[GitHub] [spark] zhengruifeng commented on pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

zhengruifeng commented on pull request #31985: URL: https://github.com/apache/spark/pull/31985#issuecomment-809044289 @srowen @WeichenXu123 This is the last PR for LR supporting centering -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] zhengruifeng commented on a change in pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

zhengruifeng commented on a change in pull request #31985: URL: https://github.com/apache/spark/pull/31985#discussion_r602997003 ## File path: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala ## @@ -1863,21 +1899,125 @@ class

[GitHub] [spark] zhengruifeng commented on a change in pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

zhengruifeng commented on a change in pull request #31985: URL: https://github.com/apache/spark/pull/31985#discussion_r602996562 ## File path: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala ## @@ -1863,21 +1899,125 @@ class

[GitHub] [spark] SparkQA commented on pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

SparkQA commented on pull request #31985: URL: https://github.com/apache/spark/pull/31985#issuecomment-809042739 **[Test build #136625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136625/testReport)** for PR 31985 at commit

[GitHub] [spark] SparkQA commented on pull request #31983: [SPARK-34882][SQL] Replace if with filter clause in RewriteDistinctAggregates

SparkQA commented on pull request #31983: URL: https://github.com/apache/spark/pull/31983#issuecomment-809042768 **[Test build #136626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136626/testReport)** for PR 31983 at commit

[GitHub] [spark] zhengruifeng opened a new pull request #31985: [SPARK-34860][ML] Multinomial Logistic Regression with intercept support centering

zhengruifeng opened a new pull request #31985: URL: https://github.com/apache/spark/pull/31985 ### What changes were proposed in this pull request? 1, use new `MultinomialLogisticBlockAggregator` which support virtual centering 2, remove no-used `BlockLogisticAggregator`

[GitHub] [spark] SparkQA commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

SparkQA commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809041822 **[Test build #136624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136624/testReport)** for PR 31979 at commit

[GitHub] [spark] SparkQA commented on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

SparkQA commented on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809041800 **[Test build #136623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136623/testReport)** for PR 31984 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

AmplabJenkins removed a comment on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809041420 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

AmplabJenkins commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809041420 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] HyukjinKwon commented on pull request #31984: [SPARK-34884][SQL] Improve dynamic partition pruning evaluation

HyukjinKwon commented on pull request #31984: URL: https://github.com/apache/spark/pull/31984#issuecomment-809040667 cc @maryannxue FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

SparkQA commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809039750 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41203/ -- This is an automated message from the

[GitHub] [spark] HyukjinKwon commented on pull request #31859: [SPARK-34769][SQL]AnsiTypeCoercion: return closest convertible type among TypeCollection

HyukjinKwon commented on pull request #31859: URL: https://github.com/apache/spark/pull/31859#issuecomment-809038253 I just found out that I mistakenly assigned it to myself .. I removed it back now .. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

SparkQA commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809037801 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41204/ --

[GitHub] [spark] SparkQA commented on pull request #31979: [SPARK-34879][SQL] HiveInspector support DayTimeIntervalType and YearMonthIntervalType

SparkQA commented on pull request #31979: URL: https://github.com/apache/spark/pull/31979#issuecomment-809037424 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41203/ -- This is an automated message from the Apache

[GitHub] [spark] Ngone51 commented on pull request #31942: [SPARK-34834][NETWORK] Fix a potential Netty memory leak in TransportResponseHandler.

Ngone51 commented on pull request #31942: URL: https://github.com/apache/spark/pull/31942#issuecomment-809035511 I'm also confused with this part. I don't even see a place where the `resp.body()` (a.k.a `ManagedBuffer`) is referenced before the `TransportResponseHandler` handle the

[GitHub] [spark] zhengruifeng commented on pull request #31693: [SPARK-34858][SPARK-34448][ML] Binary Logistic Regression with intercept support centering

zhengruifeng commented on pull request #31693: URL: https://github.com/apache/spark/pull/31693#issuecomment-809034717 @srowen Thanks for reviewing and merging! I will send another PR for multinominal LR. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon commented on pull request #31976: [SPARK-34814][SQL] LikeSimplification should handle NULL

HyukjinKwon commented on pull request #31976: URL: https://github.com/apache/spark/pull/31976#issuecomment-809033091 cc @beliefer too FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon closed pull request #31976: [SPARK-34814][SQL] LikeSimplification should handle NULL

HyukjinKwon closed pull request #31976: URL: https://github.com/apache/spark/pull/31976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] HyukjinKwon commented on pull request #31976: [SPARK-34814][SQL] LikeSimplification should handle NULL

HyukjinKwon commented on pull request #31976: URL: https://github.com/apache/spark/pull/31976#issuecomment-809032964 Merged to master and branch-3.1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon closed pull request #31973: [SPARK-34876][SQL] Fill defaultResult of non-nullable aggregates

HyukjinKwon closed pull request #31973: URL: https://github.com/apache/spark/pull/31973 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] AngersZhuuuu commented on pull request #30212: [SPARK-33308][SQL] Refactor current grouping analytics

AngersZh commented on pull request #30212: URL: https://github.com/apache/spark/pull/30212#issuecomment-809028142 Gentle ping @cloud-fan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon commented on pull request #31973: [SPARK-34876][SQL] Fill defaultResult of non-nullable aggregates

HyukjinKwon commented on pull request #31973: URL: https://github.com/apache/spark/pull/31973#issuecomment-809028090 Merged to master, branch-3.1, branch-3.0 and branch-2.4 cc @cloud-fan, @maryannxue, @viirya FYI -- This is an automated message from the Apache Git Service. To

  1   2   3   >