[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s] Fix executor service account inconsistency for branch-2.4

2020-09-25 Thread GitBox
SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698424261 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #29863: [SPARK-32877][SQL][TEST] Add test for Hive UDF complex decimal type

2020-09-25 Thread GitBox
SparkQA commented on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698261937 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] sarutak commented on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

2020-09-25 Thread GitBox
sarutak commented on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-698678194 @c21 @imback82 @maropu @HyukjinKwon Any other feedback for this change? This is an automated message from

[GitHub] [spark] AmplabJenkins commented on pull request #29806: [SPARK-32187][PYTHON][DOCS] Doc on Python packaging

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29806: URL: https://github.com/apache/spark/pull/29806#issuecomment-698110509 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] holdenk commented on a change in pull request #29817: [SPARK-32850][CORE][K8S] Simplify the RPC message flow of decommission

2020-09-25 Thread GitBox
holdenk commented on a change in pull request #29817: URL: https://github.com/apache/spark/pull/29817#discussion_r494434701 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -166,17 +171,6 @@ private[spark] class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

2020-09-25 Thread GitBox
HeartSaVioR commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r494268346 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -467,6 +467,12 @@ class DataFrameReader

[GitHub] [spark] SparkQA removed a comment on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698523837 **[Test build #129084 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129084/testReport)** for PR 29533 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698101187 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid arguments num

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-698077548 **[Test build #129057 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129057/testReport)** for PR 29054 at commit

[GitHub] [spark] github-actions[bot] closed pull request #27604: [SPARK-30849][CORE][SHUFFLE]Fix application failed due to failed to get MapStatuses broadcast block

2020-09-25 Thread GitBox
github-actions[bot] closed pull request #27604: URL: https://github.com/apache/spark/pull/27604 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA removed a comment on pull request #29863: [SPARK-32877][SQL][TEST] Add test for Hive UDF complex decimal type

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698261937 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] Victsm commented on a change in pull request #29855: [SPARK-32915][CORE] Network-layer and shuffle RPC layer changes to support push shuffle blocks

2020-09-25 Thread GitBox
Victsm commented on a change in pull request #29855: URL: https://github.com/apache/spark/pull/29855#discussion_r494487660 ## File path: common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java ## @@ -44,6 +51,71 @@ public static String

[GitHub] [spark] SparkQA removed a comment on pull request #29859: [SPARK-32971][K8S][FOLLOWUP] Add `.toSeq` for Scala 2.13 compilation

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29859: URL: https://github.com/apache/spark/pull/29859#issuecomment-698075792 **[Test build #129056 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129056/testReport)** for PR 29859 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-698707878 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] holdenk commented on pull request #29471: [SPARK-32381][CORE][SQL] Move and refactor parallel listing & non-location sensitive listing to core

2020-09-25 Thread GitBox
holdenk commented on pull request #29471: URL: https://github.com/apache/spark/pull/29471#issuecomment-698493851 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] sunchao commented on a change in pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-25 Thread GitBox
sunchao commented on a change in pull request #29843: URL: https://github.com/apache/spark/pull/29843#discussion_r494467897 ## File path: external/kafka-0-10-sql/pom.xml ## @@ -79,6 +79,10 @@ kafka-clients ${kafka.version} + +

[GitHub] [spark] AmplabJenkins commented on pull request #29862: [SPARK-32956][SQL] Ensure that the generated and existing headers are not duplicated in CSV DataSource

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29862: URL: https://github.com/apache/spark/pull/29862#issuecomment-698251280 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

2020-09-25 Thread GitBox
dongjoon-hyun commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698512589 It seems that `SparkR` test fail. ``` KubernetesSuite: - Run SparkPi with no resources - Run SparkPi with a very long application name. - Use

[GitHub] [spark] cloud-fan commented on pull request #29798: [SPARK-32931][SQL] Unevaluable Expressions are not Foldable

2020-09-25 Thread GitBox
cloud-fan commented on pull request #29798: URL: https://github.com/apache/spark/pull/29798#issuecomment-698770919 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins commented on pull request #29797: [SPARK-32932][SQL] Do not use local shuffle reader on RepartitionByExpression when coalescing disabled

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29797: URL: https://github.com/apache/spark/pull/29797#issuecomment-698907651 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] srowen closed pull request #29833: [SPARK-32886][SPARK-31882][WEBUI][2.4] fix 'undefined' link in event timeline view

2020-09-25 Thread GitBox
srowen closed pull request #29833: URL: https://github.com/apache/spark/pull/29833 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29024: [SPARK-32001][SQL]Create JDBC authentication provider developer API

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29024: URL: https://github.com/apache/spark/pull/29024#issuecomment-698852755 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] srowen commented on a change in pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-25 Thread GitBox
srowen commented on a change in pull request #29868: URL: https://github.com/apache/spark/pull/29868#discussion_r494947145 ## File path: mllib/src/main/scala/org/apache/spark/ml/feature/FeatureHasher.scala ## @@ -91,8 +91,7 @@ class FeatureHasher(@Since("2.3.0") override val

[GitHub] [spark] SparkQA commented on pull request #29797: [SPARK-32932][SQL] Do not use local shuffle reader at final stage on DataWritingCommand

2020-09-25 Thread GitBox
SparkQA commented on pull request #29797: URL: https://github.com/apache/spark/pull/29797#issuecomment-698914035 **[Test build #129111 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129111/testReport)** for PR 29797 at commit

[GitHub] [spark] c21 commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

2020-09-25 Thread GitBox
c21 commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r494083795 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/DisableUnnecessaryBucketedScan.scala ## @@ -0,0 +1,153 @@ +/* + * Licensed to

[GitHub] [spark] dongjoon-hyun closed pull request #29853: [SPARK-32977][SQL][DOCS] Fix JavaDoc on Default Save Mode

2020-09-25 Thread GitBox
dongjoon-hyun closed pull request #29853: URL: https://github.com/apache/spark/pull/29853 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29862: [SPARK-32956][SQL] Ensure that the generated and existing headers are not duplicated in CSV DataSource

2020-09-25 Thread GitBox
HyukjinKwon commented on a change in pull request #29862: URL: https://github.com/apache/spark/pull/29862#discussion_r49425 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVUtils.scala ## @@ -93,6 +93,12 @@ object CSVUtils {

[GitHub] [spark] dongjoon-hyun commented on pull request #29861: [SPARK-32971][K8S][FOLLOWUP] Fix k8s-core module compilation in Scala 2.13

2020-09-25 Thread GitBox
dongjoon-hyun commented on pull request #29861: URL: https://github.com/apache/spark/pull/29861#issuecomment-698111394 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29843: [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1 and move to shaded client

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698120109 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan closed pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-25 Thread GitBox
cloud-fan closed pull request #29756: URL: https://github.com/apache/spark/pull/29756 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins commented on pull request #29867: [SPARK-32889][SQL][TESTS][FOLLOWUP][test-hadoop2.7][test-hive1.2] Skip special column names test in Hive 1.2

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29867: URL: https://github.com/apache/spark/pull/29867#issuecomment-698623561 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #29857: [SPARK-32972][ML] Fix UTs of `mllib` module in Scala 2.13 except RandomForestRegressorSuite

2020-09-25 Thread GitBox
dongjoon-hyun commented on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-698112690 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on a change in pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

2020-09-25 Thread GitBox
maropu commented on a change in pull request #29828: URL: https://github.com/apache/spark/pull/29828#discussion_r494010395 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/JsonSuite.scala ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache

[GitHub] [spark] LuciferYang removed a comment on pull request #29864: [SPARK-32987][MESOS] Pass all `mllib` module UTs in Scala 2.13

2020-09-25 Thread GitBox
LuciferYang removed a comment on pull request #29864: URL: https://github.com/apache/spark/pull/29864#issuecomment-698276483 cc @srowen @dongjoon-hyun to review this patch ~ thx This is an automated message from the Apache

[GitHub] [spark] zhengruifeng commented on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-25 Thread GitBox
zhengruifeng commented on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-698693191 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] juliuszsompolski commented on a change in pull request #29834: [SPARK-32963][SQL] empty string should be consistent for schema name in SparkGetSchemasOperation

2020-09-25 Thread GitBox
juliuszsompolski commented on a change in pull request #29834: URL: https://github.com/apache/spark/pull/29834#discussion_r494156528 ## File path: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkGetSchemasOperation.scala ## @@ -77,7 +77,8 @@

[GitHub] [spark] SparkQA commented on pull request #29795: [SPARK-32511][SQL] Add dropFields method to Column class

2020-09-25 Thread GitBox
SparkQA commented on pull request #29795: URL: https://github.com/apache/spark/pull/29795#issuecomment-698674623 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins commented on pull request #29866: [SPARK-32990][SQL] Migrate REFRESH TABLE to use UnresolvedTableOrView to resolve the identifier

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29866: URL: https://github.com/apache/spark/pull/29866#issuecomment-698617283 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR commented on pull request #29425: [SPARK-32350][FOLLOW-UP] Fix count update issue and partition the value list to a set of small batches for LevelDB writeAll

2020-09-25 Thread GitBox
HeartSaVioR commented on pull request #29425: URL: https://github.com/apache/spark/pull/29425#issuecomment-698303116 Sorry I still have several things in my plate and have been struggling with these things. You'd better ping @mridulm as he'd understand the patch well. @mridulm

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29844: [SPARK-27872][K8s] Fix executor service account inconsistency for branch-2.4

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-697043404 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698101187 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698088612 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] MLnick commented on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-25 Thread GitBox
MLnick commented on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698112434 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] dongjoon-hyun commented on pull request #29844: [SPARK-27872][K8s] Fix executor service account inconsistency for branch-2.4

2020-09-25 Thread GitBox
dongjoon-hyun commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698422594 ok to test This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29867: [SPARK-32889][SQL][TESTS][FOLLOWUP][test-hadoop2.7][test-hive1.2] Skip special column names test in Hive 1.2

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29867: URL: https://github.com/apache/spark/pull/29867#issuecomment-698623561 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #29795: [SPARK-32511][SQL] Add dropFields method to Column class

2020-09-25 Thread GitBox
cloud-fan commented on a change in pull request #29795: URL: https://github.com/apache/spark/pull/29795#discussion_r494745836 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -901,39 +901,125 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] srowen commented on pull request #29857: [SPARK-32972][ML] Fix UTs of `mllib` module in Scala 2.13 except RandomForestRegressorSuite

2020-09-25 Thread GitBox
srowen commented on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-698479253 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29833: [SPARK-32886][SPARK-31882][WEBUI][2.4] fix 'undefined' link in event timeline view

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29833: URL: https://github.com/apache/spark/pull/29833#issuecomment-698402321 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] srowen commented on pull request #29833: [SPARK-32886][SPARK-31882][WEBUI][2.4] fix 'undefined' link in event timeline view

2020-09-25 Thread GitBox
srowen commented on pull request #29833: URL: https://github.com/apache/spark/pull/29833#issuecomment-698397395 Jenkins retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] mridulm commented on a change in pull request #29855: [SPARK-32915][CORE] Network-layer and shuffle RPC layer changes to support push shuffle blocks

2020-09-25 Thread GitBox
mridulm commented on a change in pull request #29855: URL: https://github.com/apache/spark/pull/29855#discussion_r494003389 ## File path: common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java ## @@ -44,6 +51,71 @@ public static String

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698089064 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29869: [WIP][SPARK-32994][CORE] Update external accumulators before they entering into Spark listener event loop

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29869: URL: https://github.com/apache/spark/pull/29869#issuecomment-698726138 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

2020-09-25 Thread GitBox
HyukjinKwon commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-698117104 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] zhengruifeng commented on pull request #29852: [SPARK-21481][ML][FOLLOWUP][Trivial] HashingTF use util.collection.OpenHashMap instead of mutable.HashMap

2020-09-25 Thread GitBox
zhengruifeng commented on pull request #29852: URL: https://github.com/apache/spark/pull/29852#issuecomment-698162567 ping @huaxingao This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29857: [SPARK-32972][ML] Fix UTs of `mllib` module in Scala 2.13 except RandomForestRegressorSuite

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-698087732 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] nssalian commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

2020-09-25 Thread GitBox
nssalian commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698610691 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] srowen commented on a change in pull request #29852: [SPARK-21481][ML][FOLLOWUP][Trivial] HashingTF use util.collection.OpenHashMap instead of mutable.HashMap

2020-09-25 Thread GitBox
srowen commented on a change in pull request #29852: URL: https://github.com/apache/spark/pull/29852#discussion_r494390254 ## File path: mllib/src/main/scala/org/apache/spark/ml/feature/HashingTF.scala ## @@ -91,20 +90,13 @@ class HashingTF @Since("3.0.0") private[ml] (

[GitHub] [spark] AmplabJenkins commented on pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29860: URL: https://github.com/apache/spark/pull/29860#issuecomment-698090732 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on pull request #29756: [SPARK-32885][SS] Add DataStreamReader.table API

2020-09-25 Thread GitBox
cloud-fan commented on pull request #29756: URL: https://github.com/apache/spark/pull/29756#issuecomment-698755963 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon commented on pull request #29806: [SPARK-32187][PYTHON][DOCS] Doc on Python packaging

2020-09-25 Thread GitBox
HyukjinKwon commented on pull request #29806: URL: https://github.com/apache/spark/pull/29806#issuecomment-698110090 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] fqaiser94 commented on a change in pull request #29795: [SPARK-32511][SQL] Add dropFields method to Column class

2020-09-25 Thread GitBox
fqaiser94 commented on a change in pull request #29795: URL: https://github.com/apache/spark/pull/29795#discussion_r494698625 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala ## @@ -541,57 +541,105 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

2020-09-25 Thread GitBox
cloud-fan commented on a change in pull request #29860: URL: https://github.com/apache/spark/pull/29860#discussion_r494832113 ## File path: sql/core/src/test/scala/org/apache/spark/sql/PlanStabilitySuite.scala ## @@ -153,23 +154,93 @@ trait PlanStabilitySuite extends TPCDSBase

[GitHub] [spark] gatorsmile commented on a change in pull request #29056: [SPARK-31753][SQL][DOCS] Add missing keywords in the SQL docs

2020-09-25 Thread GitBox
gatorsmile commented on a change in pull request #29056: URL: https://github.com/apache/spark/pull/29056#discussion_r494808052 ## File path: docs/sql-ref-syntax-ddl-create-table-hiveformat.md ## @@ -36,6 +36,14 @@ CREATE [ EXTERNAL ] TABLE [ IF NOT EXISTS ] table_identifier

[GitHub] [spark] SparkQA commented on pull request #29797: [SPARK-32932][SQL] Do not use local shuffle reader on RepartitionByExpression when coalescing disabled

2020-09-25 Thread GitBox
SparkQA commented on pull request #29797: URL: https://github.com/apache/spark/pull/29797#issuecomment-698906945 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29797: [SPARK-32932][SQL] Do not use local shuffle reader on RepartitionByExpression when coalescing disabled

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29797: URL: https://github.com/apache/spark/pull/29797#issuecomment-698907651 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #29024: [SPARK-32001][SQL]Create JDBC authentication provider developer API

2020-09-25 Thread GitBox
gaborgsomogyi commented on a change in pull request #29024: URL: https://github.com/apache/spark/pull/29024#discussion_r494903467 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala ## @@ -23,12 +23,15 @@ import

[GitHub] [spark] viirya commented on pull request #29831: [SPARK-32351][SQL] Show partially pushed down partition filters in explain()

2020-09-25 Thread GitBox
viirya commented on pull request #29831: URL: https://github.com/apache/spark/pull/29831#issuecomment-699265145 cc @maropu too This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid argumen

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699272519 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid argumen

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699272523 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid arguments numbe

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699272519 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid arguments num

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699231076 **[Test build #129127 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129127/testReport)** for PR 29054 at commit

[GitHub] [spark] SparkQA commented on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid arguments number erro

2020-09-25 Thread GitBox
SparkQA commented on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699272432 **[Test build #129127 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129127/testReport)** for PR 29054 at commit

[GitHub] [spark] SparkQA commented on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid arguments number erro

2020-09-25 Thread GitBox
SparkQA commented on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699287405 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33746/

[GitHub] [spark] SparkQA commented on pull request #29872: [SPARK-32996][Web-UI] Handle empty ExecutorMetrics in ExecutorMetricsJsonSerializer

2020-09-25 Thread GitBox
SparkQA commented on pull request #29872: URL: https://github.com/apache/spark/pull/29872#issuecomment-699266269 **[Test build #129125 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129125/testReport)** for PR 29872 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29872: [SPARK-32996][Web-UI] Handle empty ExecutorMetrics in ExecutorMetricsJsonSerializer

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29872: URL: https://github.com/apache/spark/pull/29872#issuecomment-699197271 **[Test build #129125 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129125/testReport)** for PR 29872 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29855: [SPARK-32915][CORE] Network-layer and shuffle RPC layer changes to support push shuffle blocks

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-699207624 **[Test build #129126 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129126/testReport)** for PR 29855 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29855: [SPARK-32915][CORE] Network-layer and shuffle RPC layer changes to support push shuffle blocks

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-699271839 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29855: [SPARK-32915][CORE] Network-layer and shuffle RPC layer changes to support push shuffle blocks

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-699271839 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid argumen

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699257263 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid arguments numbe

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699257257 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid argumen

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699257257 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699168581 **[Test build #129124 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129124/testReport)** for PR 29875 at commit

[GitHub] [spark] SparkQA commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-25 Thread GitBox
SparkQA commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699274191 **[Test build #129124 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129124/testReport)** for PR 29875 at commit

[GitHub] [spark] SparkQA commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-25 Thread GitBox
SparkQA commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699296378 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33747/

[GitHub] [spark] HeartSaVioR commented on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

2020-09-25 Thread GitBox
HeartSaVioR commented on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-699269552 Worth noting that the issue is not just occurred in theory, but I've seen the case multiple times around community report, customers, etc. Probably we'd feel better to

[GitHub] [spark] AmplabJenkins commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699274719 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699274719 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid arguments number erro

2020-09-25 Thread GitBox
SparkQA commented on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699281169 **[Test build #129130 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129130/testReport)** for PR 29054 at commit

[GitHub] [spark] SparkQA commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-25 Thread GitBox
SparkQA commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699281142 **[Test build #129129 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129129/testReport)** for PR 29875 at commit

[GitHub] [spark] SparkQA commented on pull request #29872: [SPARK-32996][Web-UI] Handle empty ExecutorMetrics in ExecutorMetricsJsonSerializer

2020-09-25 Thread GitBox
SparkQA commented on pull request #29872: URL: https://github.com/apache/spark/pull/29872#issuecomment-699402597 **[Test build #129131 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129131/testReport)** for PR 29872 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid argumen

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699400125 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29872: [SPARK-32996][Web-UI] Handle empty ExecutorMetrics in ExecutorMetricsJsonSerializer

2020-09-25 Thread GitBox
AmplabJenkins commented on pull request #29872: URL: https://github.com/apache/spark/pull/29872#issuecomment-699266831 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29872: [SPARK-32996][Web-UI] Handle empty ExecutorMetrics in ExecutorMetricsJsonSerializer

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29872: URL: https://github.com/apache/spark/pull/29872#issuecomment-699266831 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan closed pull request #29798: [SPARK-32931][SQL] Unevaluable Expressions are not Foldable

2020-09-25 Thread GitBox
cloud-fan closed pull request #29798: URL: https://github.com/apache/spark/pull/29798 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] fhoering commented on pull request #29806: [SPARK-32187][PYTHON][DOCS] Doc on Python packaging

2020-09-25 Thread GitBox
fhoering commented on pull request #29806: URL: https://github.com/apache/spark/pull/29806#issuecomment-698877710 It would be nice to have K8s here indeed but I never deployed to K8s. So I will only do the small changes from above and let you open anther JIRA ticket for someone else to

[GitHub] [spark] SparkQA removed a comment on pull request #29797: [SPARK-32932][SQL] Do not use local shuffle reader on RepartitionByExpression when coalescing disabled

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29797: URL: https://github.com/apache/spark/pull/29797#issuecomment-698906945 **[Test build #129110 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129110/testReport)** for PR 29797 at commit

[GitHub] [spark] SparkQA commented on pull request #29857: [SPARK-32972][ML] Pass all UTs of `mllib` module in Scala 2.13

2020-09-25 Thread GitBox
SparkQA commented on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-698928101 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] wangyum commented on a change in pull request #29790: [SPARK-32914][SQL] Avoid calling dataType multiple times for each expression

2020-09-25 Thread GitBox
wangyum commented on a change in pull request #29790: URL: https://github.com/apache/spark/pull/29790#discussion_r494810795 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala ## @@ -3498,13 +3500,15 @@ object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29857: [SPARK-32972][ML] Pass all UTs of `mllib` module in Scala 2.13

2020-09-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-698937917 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29857: [SPARK-32972][ML] Pass all UTs of `mllib` module in Scala 2.13

2020-09-25 Thread GitBox
SparkQA removed a comment on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-698906903 **[Test build #129109 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129109/testReport)** for PR 29857 at commit

<    1   2   3   4   5   6   7   >