[GitHub] [spark] AmplabJenkins removed a comment on pull request #29863: [SPARK-32877][SQL][TEST] Add test for Hive UDF complex decimal type

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698262534 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29863: [SPARK-32877][SQL][TEST] Add test for Hive UDF complex decimal type

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698262534 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29863: [SPARK-32877][SQL][TEST] Add test for Hive UDF complex decimal type

2020-09-24 Thread GitBox
SparkQA commented on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698261937 **[Test build #129074 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129074/testReport)** for PR 29863 at commit

[GitHub] [spark] ulysses-you opened a new pull request #29863: [SPARK-32877][SQL][TEST] Add test for Hive UDF complex decimal type

2020-09-24 Thread GitBox
ulysses-you opened a new pull request #29863: URL: https://github.com/apache/spark/pull/29863 ### What changes were proposed in this pull request? Add test to cover Hive UDF whose input contains complex decimal type. Add comment to explain why we can't make `HiveSimpleUDF`

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29862: [SPARK-32956][SQL] Ensure that the generated and existing headers are not duplicated in CSV DataSource

2020-09-24 Thread GitBox
HyukjinKwon commented on a change in pull request #29862: URL: https://github.com/apache/spark/pull/29862#discussion_r49425 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVUtils.scala ## @@ -93,6 +93,12 @@ object CSVUtils {

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29862: [SPARK-32956][SQL] Ensure that the generated and existing headers are not duplicated in CSV DataSource

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29862: URL: https://github.com/apache/spark/pull/29862#issuecomment-698251800 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] izchen commented on pull request #29862: [SPARK-32956][SQL] Ensure that the generated and existing headers are not duplicated in CSV DataSource

2020-09-24 Thread GitBox
izchen commented on pull request #29862: URL: https://github.com/apache/spark/pull/29862#issuecomment-698252565 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698251854 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698251842 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29862: [SPARK-32956][SQL] Ensure that the generated and existing headers are not duplicated in CSV DataSource

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29862: URL: https://github.com/apache/spark/pull/29862#issuecomment-698251280 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #29862: [SPARK-32956][SQL] Ensure that the generated and existing headers are not duplicated in CSV DataSource

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29862: URL: https://github.com/apache/spark/pull/29862#issuecomment-698251800 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698251842 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
SparkQA commented on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698251231 **[Test build #129071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129071/testReport)** for PR 29843 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29862: [SPARK-32956][SQL] Ensure that the generated and existing headers are not duplicated in CSV DataSource

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29862: URL: https://github.com/apache/spark/pull/29862#issuecomment-698251280 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698183820 **[Test build #129071 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129071/testReport)** for PR 29843 at commit

[GitHub] [spark] izchen opened a new pull request #29862: [SPARK-32956][SQL] Ensure that the generated and existing headers are not duplicated in CSV DataSource

2020-09-24 Thread GitBox
izchen opened a new pull request #29862: URL: https://github.com/apache/spark/pull/29862 ### What changes were proposed in this pull request? In [SPARK-16896](https://issues.apache.org/jira/browse/SPARK-16896), generate some new column headers to replace duplicate column headers or

[GitHub] [spark] AmplabJenkins commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698244613 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698244613 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698244013 **[Test build #129073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129073/testReport)** for PR 29800 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698234258 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698204073 **[Test build #129072 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129072/testReport)** for PR 29585 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698234258 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698234240 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

2020-09-24 Thread GitBox
SparkQA commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698234134 **[Test build #129072 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129072/testReport)** for PR 29585 at commit

[GitHub] [spark] yaooqinn commented on a change in pull request #29834: [SPARK-32963][SQL] empty string should be consistent for schema name in SparkGetSchemasOperation

2020-09-24 Thread GitBox
yaooqinn commented on a change in pull request #29834: URL: https://github.com/apache/spark/pull/29834#discussion_r494171538 ## File path: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkGetSchemasOperation.scala ## @@ -77,7 +77,8 @@

[GitHub] [spark] yaooqinn commented on a change in pull request #29834: [SPARK-32963][SQL] empty string should be consistent for schema name in SparkGetSchemasOperation

2020-09-24 Thread GitBox
yaooqinn commented on a change in pull request #29834: URL: https://github.com/apache/spark/pull/29834#discussion_r494171538 ## File path: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkGetSchemasOperation.scala ## @@ -77,7 +77,8 @@

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698225564 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698169231 **[Test build #129070 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129070/testReport)** for PR 29800 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698225550 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698225550 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698225413 **[Test build #129070 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129070/testReport)** for PR 29800 at commit

[GitHub] [spark] juliuszsompolski commented on a change in pull request #29834: [SPARK-32963][SQL] empty string should be consistent for schema name in SparkGetSchemasOperation

2020-09-24 Thread GitBox
juliuszsompolski commented on a change in pull request #29834: URL: https://github.com/apache/spark/pull/29834#discussion_r494158455 ## File path: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkGetSchemasOperation.scala ## @@ -77,7 +77,8 @@

[GitHub] [spark] juliuszsompolski commented on a change in pull request #29834: [SPARK-32963][SQL] empty string should be consistent for schema name in SparkGetSchemasOperation

2020-09-24 Thread GitBox
juliuszsompolski commented on a change in pull request #29834: URL: https://github.com/apache/spark/pull/29834#discussion_r494156528 ## File path: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkGetSchemasOperation.scala ## @@ -77,7 +77,8 @@

[GitHub] [spark] gaborgsomogyi commented on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

2020-09-24 Thread GitBox
gaborgsomogyi commented on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-698211449 @Tagar not sure what you mean when you say `this`. ACL possibility is available in the vanilla Kafka too: https://kafka.apache.org/documentation/#security_authz

[GitHub] [spark] tomvanbussel commented on pull request #29785: [SPARK-32901][CORE] Do not allocate memory while spilling UnsafeExternalSorter

2020-09-24 Thread GitBox
tomvanbussel commented on pull request #29785: URL: https://github.com/apache/spark/pull/29785#issuecomment-698206841 @Ngone51 Thanks for clarifying! It's both trying to allocate a new large pointer array and a new small pointer array. The allocation of the new large pointer array was

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698204779 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698204779 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] tomvanbussel commented on a change in pull request #29785: [SPARK-32901][CORE] Do not allocate memory while spilling UnsafeExternalSorter

2020-09-24 Thread GitBox
tomvanbussel commented on a change in pull request #29785: URL: https://github.com/apache/spark/pull/29785#discussion_r494140993 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java ## @@ -384,6 +401,28 @@ private void

[GitHub] [spark] SparkQA commented on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

2020-09-24 Thread GitBox
SparkQA commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698204073 **[Test build #129072 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129072/testReport)** for PR 29585 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698189706 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698158800 **[Test build #129068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129068/testReport)** for PR 29850 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698189696 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698189696 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
SparkQA commented on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698189298 **[Test build #129068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129068/testReport)** for PR 29850 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698184534 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698184534 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
SparkQA commented on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698183820 **[Test build #129071 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129071/testReport)** for PR 29843 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698169960 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698169960 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698169231 **[Test build #129070 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129070/testReport)** for PR 29800 at commit

[GitHub] [spark] beliefer commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
beliefer commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698167171 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] zhengruifeng commented on pull request #29852: [SPARK-21481][ML][FOLLOWUP][Trivial] HashingTF use util.collection.OpenHashMap instead of mutable.HashMap

2020-09-24 Thread GitBox
zhengruifeng commented on pull request #29852: URL: https://github.com/apache/spark/pull/29852#issuecomment-698162567 ping @huaxingao This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698159483 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698159535 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698159483 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698159535 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29861: [SPARK-32971][K8S][FOLLOWUP] Fix k8s-core module compilation in Scala 2.13

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29861: URL: https://github.com/apache/spark/pull/29861#issuecomment-698158592 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29861: [SPARK-32971][K8S][FOLLOWUP] Fix k8s-core module compilation in Scala 2.13

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29861: URL: https://github.com/apache/spark/pull/29861#issuecomment-698158585 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] xuanyuanking commented on a change in pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-24 Thread GitBox
xuanyuanking commented on a change in pull request #29756: URL: https://github.com/apache/spark/pull/29756#discussion_r494084571 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -861,15 +862,22 @@ class Analyzer(

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698158381 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
SparkQA commented on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698158800 **[Test build #129068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129068/testReport)** for PR 29850 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29861: [SPARK-32971][K8S][FOLLOWUP] Fix k8s-core module compilation in Scala 2.13

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29861: URL: https://github.com/apache/spark/pull/29861#issuecomment-698158585 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698158257 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698158345 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698158212 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698158264 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29860: URL: https://github.com/apache/spark/pull/29860#issuecomment-698158375 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-24 Thread GitBox
SparkQA commented on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698158844 **[Test build #129069 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129069/testReport)** for PR 29756 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698119769 **[Test build #129064 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129064/testReport)** for PR 29843 at commit

[GitHub] [spark] zhengruifeng commented on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
zhengruifeng commented on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698158347 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29860: URL: https://github.com/apache/spark/pull/29860#issuecomment-698158365 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698158205 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698158159 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29861: [SPARK-32971][K8S][FOLLOWUP] Fix k8s-core module compilation in Scala 2.13

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29861: URL: https://github.com/apache/spark/pull/29861#issuecomment-698100859 **[Test build #129061 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129061/testReport)** for PR 29861 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29860: URL: https://github.com/apache/spark/pull/29860#issuecomment-698090510 **[Test build #129060 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129060/testReport)** for PR 29860 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698158367 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698088612 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29860: URL: https://github.com/apache/spark/pull/29860#issuecomment-698158365 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698158335 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698158254 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698158257 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698086807 **[Test build #129058 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129058/testReport)** for PR 29800 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698158205 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698128830 **[Test build #129067 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129067/testReport)** for PR 29850 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-24 Thread GitBox
SparkQA removed a comment on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698100892 **[Test build #129062 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129062/testReport)** for PR 29756 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698158159 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698158254 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
AmplabJenkins commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698158367 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29861: [SPARK-32971][K8S][FOLLOWUP] Fix k8s-core module compilation in Scala 2.13

2020-09-24 Thread GitBox
SparkQA commented on pull request #29861: URL: https://github.com/apache/spark/pull/29861#issuecomment-698158003 **[Test build #129061 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129061/testReport)** for PR 29861 at commit

[GitHub] [spark] SparkQA commented on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-24 Thread GitBox
SparkQA commented on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698158002 **[Test build #129062 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129062/testReport)** for PR 29756 at commit

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

2020-09-24 Thread GitBox
SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698158001 **[Test build #129058 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129058/testReport)** for PR 29800 at commit

[GitHub] [spark] SparkQA commented on pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

2020-09-24 Thread GitBox
SparkQA commented on pull request #29860: URL: https://github.com/apache/spark/pull/29860#issuecomment-698157997 **[Test build #129060 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129060/testReport)** for PR 29860 at commit

[GitHub] [spark] SparkQA commented on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-24 Thread GitBox
SparkQA commented on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698157996 **[Test build #129064 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129064/testReport)** for PR 29843 at commit

[GitHub] [spark] SparkQA commented on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-24 Thread GitBox
SparkQA commented on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698158006 **[Test build #129067 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129067/testReport)** for PR 29850 at commit

[GitHub] [spark] c21 commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

2020-09-24 Thread GitBox
c21 commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r494083795 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/DisableUnnecessaryBucketedScan.scala ## @@ -0,0 +1,153 @@ +/* + * Licensed to

[GitHub] [spark] SparkQA commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

2020-09-24 Thread GitBox
SparkQA commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698157995 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] mridulm commented on a change in pull request #29855: [SPARK-32915][CORE] Network-layer and shuffle RPC layer changes to support push shuffle blocks

2020-09-24 Thread GitBox
mridulm commented on a change in pull request #29855: URL: https://github.com/apache/spark/pull/29855#discussion_r494083495 ## File path: common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java ## @@ -44,6 +51,71 @@ public static String

[GitHub] [spark] mridulm commented on a change in pull request #29855: [SPARK-32915][CORE] Network-layer and shuffle RPC layer changes to support push shuffle blocks

2020-09-24 Thread GitBox
mridulm commented on a change in pull request #29855: URL: https://github.com/apache/spark/pull/29855#discussion_r494003389 ## File path: common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java ## @@ -44,6 +51,71 @@ public static String

[GitHub] [spark] Ngone51 commented on pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

2020-09-24 Thread GitBox
Ngone51 commented on pull request #29860: URL: https://github.com/apache/spark/pull/29860#issuecomment-698155261 I remember we changed to showing plans one by one instead of side by side because of the TPCDS plans are usually too big to be shown in a good shape using side by side.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29857: [SPARK-32972][ML] Fix UTs of `mllib` module in Scala 2.13 except RandomForestRegressorSuite

2020-09-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-698151430 This is an automated message from the Apache Git Service. To respond to the message, please log on

<    1   2   3   4   5   6   >