[GitHub] [spark] AmplabJenkins commented on pull request #28326: [SPARK-27340][SS] Alias on TimeWindow expression cause watermark metadata lost

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28326: URL: https://github.com/apache/spark/pull/28326#issuecomment-620006474 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-620005232 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28326: [SPARK-27340][SS] Alias on TimeWindow expression cause watermark metadata lost

2020-04-27 Thread GitBox
SparkQA commented on pull request #28326: URL: https://github.com/apache/spark/pull/28326#issuecomment-620005507 **[Test build #121905 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121905/testReport)** for PR 28326 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-620005232 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
gaborgsomogyi commented on a change in pull request #28368: URL: https://github.com/apache/spark/pull/28368#discussion_r415837441 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/DB2ConnectionProvider.scala ## @@ -48,7 +48,7 @@

[GitHub] [spark] SparkQA removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-619796131 **[Test build #121886 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121886/testReport)** for PR 28363 at commit

[GitHub] [spark] SparkQA commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

2020-04-27 Thread GitBox
SparkQA commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-620003866 **[Test build #121886 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121886/testReport)** for PR 28363 at commit

[GitHub] [spark] planga82 commented on a change in pull request #28351: [SPARK-31500][SQL] collect_set() of BinaryType returns duplicate elements

2020-04-27 Thread GitBox
planga82 commented on a change in pull request #28351: URL: https://github.com/apache/spark/pull/28351#discussion_r415835388 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/collect.scala ## @@ -139,6 +139,23 @@ case class

[GitHub] [spark] beliefer commented on a change in pull request #28194: [SPARK-31372][SQL][TEST] Display expression schema for double check.

2020-04-27 Thread GitBox
beliefer commented on a change in pull request #28194: URL: https://github.com/apache/spark/pull/28194#discussion_r415832820 ## File path: sql/core/src/test/scala/org/apache/spark/sql/ExpressionsSchemaSuite.scala ## @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] AmplabJenkins commented on pull request #28371: [SPARK-31535][SQL][FOLLOWUP] Simplify name conflict check in CTE resolution

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28371: URL: https://github.com/apache/spark/pull/28371#issuecomment-620001729 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28371: [SPARK-31535][SQL][FOLLOWUP] Simplify name conflict check in CTE resolution

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28371: URL: https://github.com/apache/spark/pull/28371#issuecomment-620001729 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] beliefer commented on a change in pull request #28194: [SPARK-31372][SQL][TEST] Display expression schema for double check.

2020-04-27 Thread GitBox
beliefer commented on a change in pull request #28194: URL: https://github.com/apache/spark/pull/28194#discussion_r415832429 ## File path: sql/core/src/test/scala/org/apache/spark/sql/ExpressionsSchemaSuite.scala ## @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] beliefer commented on a change in pull request #28194: [SPARK-31372][SQL][TEST] Display expression schema for double check.

2020-04-27 Thread GitBox
beliefer commented on a change in pull request #28194: URL: https://github.com/apache/spark/pull/28194#discussion_r415831986 ## File path: sql/core/src/test/resources/sql-functions/sql-expression-schema.md ## @@ -0,0 +1,331 @@ +## Summary + - Number of queries: 324 + -

[GitHub] [spark] tgravescs commented on pull request #28323: [SPARK-31551][CORE] Fix createSparkUser lost user's non-Hadoop credentials

2020-04-27 Thread GitBox
tgravescs commented on pull request #28323: URL: https://github.com/apache/spark/pull/28323#issuecomment-620001049 it's been a long time since I looked at this so I will have to go refresh my memory but I don't think using getUserName works in all cases. What kind of testing have

[GitHub] [spark] SparkQA commented on pull request #28371: [SPARK-31535][SQL][FOLLOWUP] Simplify name conflict check in CTE resolution

2020-04-27 Thread GitBox
SparkQA commented on pull request #28371: URL: https://github.com/apache/spark/pull/28371#issuecomment-62996 **[Test build #121904 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121904/testReport)** for PR 28371 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28371: [SPARK-31535][SQL][FOLLOWUP] Simplify name conflict check in CTE resolution

2020-04-27 Thread GitBox
cloud-fan commented on a change in pull request #28371: URL: https://github.com/apache/spark/pull/28371#discussion_r415829102 ## File path: sql/core/src/test/resources/sql-tests/results/cte-nested.sql.out ## @@ -64,10 +64,9 @@ WITH ) SELECT * FROM t2 -- !query schema

[GitHub] [spark] cloud-fan commented on pull request #28371: [SPARK-31535][SQL][FOLLOWUP] Simplify name conflict check in CTE resolution

2020-04-27 Thread GitBox
cloud-fan commented on pull request #28371: URL: https://github.com/apache/spark/pull/28371#issuecomment-61399 cc @peter-toth @xuanyuanking @dongjoon-hyun This is an automated message from the Apache Git Service. To

[GitHub] [spark] cloud-fan commented on a change in pull request #28371: [SPARK-31535][SQL][FOLLOWUP] Simplify name conflict check in CTE resolution

2020-04-27 Thread GitBox
cloud-fan commented on a change in pull request #28371: URL: https://github.com/apache/spark/pull/28371#discussion_r415829102 ## File path: sql/core/src/test/resources/sql-tests/results/cte-nested.sql.out ## @@ -64,10 +64,9 @@ WITH ) SELECT * FROM t2 -- !query schema

[GitHub] [spark] AmplabJenkins commented on pull request #28366: [WIP][SPARK-31365][SQL] Enable nested predicate pushdown per data sources

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-619998191 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28366: [WIP][SPARK-31365][SQL] Enable nested predicate pushdown per data sources

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-619998191 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan opened a new pull request #28371: [SPARK-31535][SQL][FOLLOWUP] Simplify name conflict check in CTE resolution

2020-04-27 Thread GitBox
cloud-fan opened a new pull request #28371: URL: https://github.com/apache/spark/pull/28371 ### What changes were proposed in this pull request? This is a followup of https://github.com/apache/spark/pull/28318, to make the code more readable: 1. add some comments to

[GitHub] [spark] SparkQA removed a comment on pull request #28366: [WIP][SPARK-31365][SQL] Enable nested predicate pushdown per data sources

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-619819648 **[Test build #121891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121891/testReport)** for PR 28366 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-619996212 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] SparkQA commented on pull request #28366: [WIP][SPARK-31365][SQL] Enable nested predicate pushdown per data sources

2020-04-27 Thread GitBox
SparkQA commented on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-619996923 **[Test build #121891 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121891/testReport)** for PR 28366 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-619996941 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-619996212 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] beliefer commented on a change in pull request #28081: [SPARK-31315][SQL] SQLQueryTestSuite: Display the total compile time for generated java code

2020-04-27 Thread GitBox
beliefer commented on a change in pull request #28081: URL: https://github.com/apache/spark/pull/28081#discussion_r415825048 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala ## @@ -1309,6 +1309,23 @@ object

[GitHub] [spark] prakharjain09 edited a comment on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

2020-04-27 Thread GitBox
prakharjain09 edited a comment on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-619994477 Recreated PR for https://github.com/apache/spark/pull/27864 to identify and fix test failures. Please refer to https://github.com/apache/spark/pull/27864 for more

[GitHub] [spark] prakharjain09 commented on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

2020-04-27 Thread GitBox
prakharjain09 commented on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-619994938 Can someone please enable Jenkins test runs on this. cc - @holdenk @dongjoon-hyun This is an

[GitHub] [spark] prakharjain09 commented on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

2020-04-27 Thread GitBox
prakharjain09 commented on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-619994477 Recreated PR for https://github.com/apache/spark/pull/27864 to identify and fix test failures. Please refer to https://github.com/apache/spark/pull/27864 for more

[GitHub] [spark] prakharjain09 opened a new pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

2020-04-27 Thread GitBox
prakharjain09 opened a new pull request #28370: URL: https://github.com/apache/spark/pull/28370 What changes were proposed in this pull request? After changes in SPARK-20628, CoarseGrainedSchedulerBackend can decommission an executor and stop assigning new tasks on it. We should also

[GitHub] [spark] cloud-fan commented on pull request #28364: [SPARK-31529][SQL][3.0] Remove extra whitespaces in formatted explain

2020-04-27 Thread GitBox
cloud-fan commented on pull request #28364: URL: https://github.com/apache/spark/pull/28364#issuecomment-619992324 thanks, merging to 3.0! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #27617: URL: https://github.com/apache/spark/pull/27617#issuecomment-619991663 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #27617: URL: https://github.com/apache/spark/pull/27617#issuecomment-619991663 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #27617: URL: https://github.com/apache/spark/pull/27617#issuecomment-619800492 **[Test build #121887 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121887/testReport)** for PR 27617 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28364: [SPARK-31529][SQL][3.0] Remove extra whitespaces in formatted explain

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28364: URL: https://github.com/apache/spark/pull/28364#issuecomment-619990215 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

2020-04-27 Thread GitBox
SparkQA commented on pull request #27617: URL: https://github.com/apache/spark/pull/27617#issuecomment-619990332 **[Test build #121887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121887/testReport)** for PR 27617 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28364: [SPARK-31529][SQL][3.0] Remove extra whitespaces in formatted explain

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28364: URL: https://github.com/apache/spark/pull/28364#issuecomment-619990215 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28364: [SPARK-31529][SQL][3.0] Remove extra whitespaces in formatted explain

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28364: URL: https://github.com/apache/spark/pull/28364#issuecomment-619812417 **[Test build #121888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121888/testReport)** for PR 28364 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-619988820 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-619988820 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28364: [SPARK-31529][SQL][3.0] Remove extra whitespaces in formatted explain

2020-04-27 Thread GitBox
SparkQA commented on pull request #28364: URL: https://github.com/apache/spark/pull/28364#issuecomment-619988845 **[Test build #121888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121888/testReport)** for PR 28364 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-619775791 **[Test build #121881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121881/testReport)** for PR 27019 at commit

[GitHub] [spark] SparkQA commented on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-04-27 Thread GitBox
SparkQA commented on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-619986648 **[Test build #121881 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121881/testReport)** for PR 27019 at commit

[GitHub] [spark] maropu commented on a change in pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
maropu commented on a change in pull request #28368: URL: https://github.com/apache/spark/pull/28368#discussion_r415800450 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/DB2ConnectionProvider.scala ## @@ -48,7 +48,7 @@

[GitHub] [spark] igreenfield commented on pull request #26624: [SPARK-8981][core] Add MDC support in Executor

2020-04-27 Thread GitBox
igreenfield commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-619976944 log4j.appender.console.layout.ConversionPattern=%d{/MM/dd HH:mm:ss} %p [%X{appId}] [%X{appName}] %c{3} - %m%n ```

[GitHub] [spark] AmplabJenkins commented on pull request #28349: [SPARK-30642][ML][PYSPARK] LinearSVC blockify input vectors

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28349: URL: https://github.com/apache/spark/pull/28349#issuecomment-619973961 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28349: [SPARK-30642][ML][PYSPARK] LinearSVC blockify input vectors

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28349: URL: https://github.com/apache/spark/pull/28349#issuecomment-619973961 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28349: [SPARK-30642][ML][PYSPARK] LinearSVC blockify input vectors

2020-04-27 Thread GitBox
SparkQA commented on pull request #28349: URL: https://github.com/apache/spark/pull/28349#issuecomment-619972322 **[Test build #121896 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121896/testReport)** for PR 28349 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28349: [SPARK-30642][ML][PYSPARK] LinearSVC blockify input vectors

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28349: URL: https://github.com/apache/spark/pull/28349#issuecomment-619855561 **[Test build #121896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121896/testReport)** for PR 28349 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v))

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #27803: URL: https://github.com/apache/spark/pull/27803#issuecomment-619971949 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v))

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #27803: URL: https://github.com/apache/spark/pull/27803#issuecomment-619971949 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28294: [SPARK-31519][SQL] Cast in having aggregate expressions returns the wrong result

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28294: URL: https://github.com/apache/spark/pull/28294#issuecomment-619970647 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28294: [SPARK-31519][SQL] Cast in having aggregate expressions returns the wrong result

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28294: URL: https://github.com/apache/spark/pull/28294#issuecomment-619970647 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v))

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #27803: URL: https://github.com/apache/spark/pull/27803#issuecomment-619779875 **[Test build #121883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121883/testReport)** for PR 27803 at commit

[GitHub] [spark] SparkQA commented on pull request #27803: [SPARK-31049][SQL] Support nested adjacent generators, e.g., explode(explode(v))

2020-04-27 Thread GitBox
SparkQA commented on pull request #27803: URL: https://github.com/apache/spark/pull/27803#issuecomment-619970194 **[Test build #121883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121883/testReport)** for PR 27803 at commit

[GitHub] [spark] SparkQA commented on pull request #28294: [SPARK-31519][SQL] Cast in having aggregate expressions returns the wrong result

2020-04-27 Thread GitBox
SparkQA commented on pull request #28294: URL: https://github.com/apache/spark/pull/28294#issuecomment-619969970 **[Test build #121903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121903/testReport)** for PR 28294 at commit

[GitHub] [spark] Ngone51 edited a comment on pull request #26624: [SPARK-8981][core] Add MDC support in Executor

2020-04-27 Thread GitBox
Ngone51 edited a comment on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-619962577 > about the pattern, it should be added but I think each will add what he needed and how he wanted it as it also supports adding local properties starting with MDC.

[GitHub] [spark] Ngone51 commented on pull request #26624: [SPARK-8981][core] Add MDC support in Executor

2020-04-27 Thread GitBox
Ngone51 commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-619962577 > about the pattern, it should be added but I think each will add what he needed and how he wanted it as it also supports adding local properties starting with MDC.

[GitHub] [spark] hvanhovell commented on a change in pull request #28351: [SPARK-31500][SQL] collect_set() of BinaryType returns duplicate elements

2020-04-27 Thread GitBox
hvanhovell commented on a change in pull request #28351: URL: https://github.com/apache/spark/pull/28351#discussion_r415776330 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/collect.scala ## @@ -139,6 +139,23 @@ case class

[GitHub] [spark] Ngone51 commented on a change in pull request #26624: [SPARK-8981][core] Add MDC support in Executor

2020-04-27 Thread GitBox
Ngone51 commented on a change in pull request #26624: URL: https://github.com/apache/spark/pull/26624#discussion_r415771934 ## File path: core/src/main/scala/org/apache/spark/executor/Executor.scala ## @@ -674,6 +677,18 @@ private[spark] class Executor( } } +

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28369: [SPARK-31527][SQL][TESTS] Add a benchmark test for datetime add/subtract interval operations

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28369: URL: https://github.com/apache/spark/pull/28369#issuecomment-619956476 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28369: [SPARK-31527][SQL][TESTS] Add a benchmark test for datetime add/subtract interval operations

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28369: URL: https://github.com/apache/spark/pull/28369#issuecomment-619956476 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28369: [SPARK-31527][SQL][TESTS] Add a benchmark test for datetime add/subtract interval operations

2020-04-27 Thread GitBox
SparkQA commented on pull request #28369: URL: https://github.com/apache/spark/pull/28369#issuecomment-619955813 **[Test build #121902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121902/testReport)** for PR 28369 at commit

[GitHub] [spark] yaooqinn commented on pull request #28369: [SPARK-31527][SQL][TESTS] Add a benchmark test for datetime add/subtract interval operations

2020-04-27 Thread GitBox
yaooqinn commented on pull request #28369: URL: https://github.com/apache/spark/pull/28369#issuecomment-619953714 cc @cloud-fan @dongjoon-hyun @maropu @HyukjinKwon many thank you for your valuable time. This is an automated

[GitHub] [spark] yaooqinn opened a new pull request #28369: [SPARK-31527][SQL][TESTS] Add a benchmark test for datetime add/subtract interval operations

2020-04-27 Thread GitBox
yaooqinn opened a new pull request #28369: URL: https://github.com/apache/spark/pull/28369 ### What changes were proposed in this pull request? With https://github.com/apache/spark/pull/28310, the operation of date +/- interval(m, d, 0) has been improved a lot, about 75% time cost is

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28215: [SPARK-31272][SQL] Support DB2 Kerberos login in JDBC connector

2020-04-27 Thread GitBox
gaborgsomogyi commented on a change in pull request #28215: URL: https://github.com/apache/spark/pull/28215#discussion_r415764707 ## File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DB2KrbIntegrationSuite.scala ## @@ -0,0 +1,89 @@ +/* + *

[GitHub] [spark] gaborgsomogyi commented on pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
gaborgsomogyi commented on pull request #28368: URL: https://github.com/apache/spark/pull/28368#issuecomment-619949875 cc @maropu @dongjoon-hyun @vanzin This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] igreenfield commented on pull request #26624: [SPARK-8981][core] Add MDC support in Executor

2020-04-27 Thread GitBox
igreenfield commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-619949637 Hi, @Ngone51 first thank for reviewing! about the pattern, it should be added but I think each will add what he needed and how he wanted it as it also supports adding

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
gaborgsomogyi commented on a change in pull request #28368: URL: https://github.com/apache/spark/pull/28368#discussion_r415758927 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/DB2ConnectionProvider.scala ## @@ -48,7 +48,7 @@

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
gaborgsomogyi commented on a change in pull request #28368: URL: https://github.com/apache/spark/pull/28368#discussion_r415758927 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/DB2ConnectionProvider.scala ## @@ -48,7 +48,7 @@

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
gaborgsomogyi commented on a change in pull request #28368: URL: https://github.com/apache/spark/pull/28368#discussion_r415758927 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/DB2ConnectionProvider.scala ## @@ -48,7 +48,7 @@

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
gaborgsomogyi commented on a change in pull request #28368: URL: https://github.com/apache/spark/pull/28368#discussion_r415756591 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/SecureConnectionProvider.scala ## @@ -26,6 +26,12

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
gaborgsomogyi commented on a change in pull request #28368: URL: https://github.com/apache/spark/pull/28368#discussion_r415758927 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/DB2ConnectionProvider.scala ## @@ -48,7 +48,7 @@

[GitHub] [spark] igreenfield commented on a change in pull request #26624: [SPARK-8981][core] Add MDC support in Executor

2020-04-27 Thread GitBox
igreenfield commented on a change in pull request #26624: URL: https://github.com/apache/spark/pull/26624#discussion_r415757489 ## File path: core/src/main/scala/org/apache/spark/executor/Executor.scala ## @@ -674,6 +677,18 @@ private[spark] class Executor( } } +

[GitHub] [spark] AmplabJenkins commented on pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28368: URL: https://github.com/apache/spark/pull/28368#issuecomment-619943464 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
gaborgsomogyi commented on a change in pull request #28368: URL: https://github.com/apache/spark/pull/28368#discussion_r415756591 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/SecureConnectionProvider.scala ## @@ -26,6 +26,12

[GitHub] [spark] SparkQA commented on pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
SparkQA commented on pull request #28368: URL: https://github.com/apache/spark/pull/28368#issuecomment-619943154 **[Test build #121901 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121901/testReport)** for PR 28368 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619942415 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619942415 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] gaborgsomogyi opened a new pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

2020-04-27 Thread GitBox
gaborgsomogyi opened a new pull request #28368: URL: https://github.com/apache/spark/pull/28368 ### What changes were proposed in this pull request? There is a race in secure JDBC connection providers. Namely multiple providers can read and/or write the exact same JVM security

[GitHub] [spark] cloud-fan commented on pull request #28356: [SPARK-31485][CORE][3.0] Avoid application hang if only partial barrier tasks launched

2020-04-27 Thread GitBox
cloud-fan commented on pull request #28356: URL: https://github.com/apache/spark/pull/28356#issuecomment-619942204 thanks, merging to 3.0! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA removed a comment on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619812405 **[Test build #121889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121889/testReport)** for PR 28362 at commit

[GitHub] [spark] SparkQA commented on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
SparkQA commented on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619940829 **[Test build #121889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121889/testReport)** for PR 28362 at commit

[GitHub] [spark] maropu commented on pull request #28215: [SPARK-31272][SQL] Support DB2 Kerberos login in JDBC connector

2020-04-27 Thread GitBox
maropu commented on pull request #28215: URL: https://github.com/apache/spark/pull/28215#issuecomment-619925244 I run it by myself and I checked it passed on my MacOS (Sierra, Docker Desktop community v2.3.0.0). This is an

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619923072 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619923072 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619897815 **[Test build #121900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121900/testReport)** for PR 28365 at commit

[GitHub] [spark] SparkQA commented on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
SparkQA commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619922643 **[Test build #121900 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121900/testReport)** for PR 28365 at commit

[GitHub] [spark] MaxGekk commented on pull request #28328: [SPARK-31553][SQL] Fix isInCollection for collection sizes above the optimisation threshold

2020-04-27 Thread GitBox
MaxGekk commented on pull request #28328: URL: https://github.com/apache/spark/pull/28328#issuecomment-619904603 I have update PR's description and added a column w/o optimization. I got the numbers by running the code: ```scala test("isInCollection benchmark") { def

[GitHub] [spark] AmplabJenkins commented on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619898728 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619898728 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28356: [SPARK-31485][CORE][3.0] Avoid application hang if only partial barrier tasks launched

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28356: URL: https://github.com/apache/spark/pull/28356#issuecomment-619898517 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28356: [SPARK-31485][CORE][3.0] Avoid application hang if only partial barrier tasks launched

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28356: URL: https://github.com/apache/spark/pull/28356#issuecomment-619898517 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
SparkQA commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619897815 **[Test build #121900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121900/testReport)** for PR 28365 at commit

[GitHub] [spark] SparkQA commented on pull request #28328: [SPARK-31553][SQL] Fix isInCollection for collection sizes above the optimisation threshold

2020-04-27 Thread GitBox
SparkQA commented on pull request #28328: URL: https://github.com/apache/spark/pull/28328#issuecomment-619897888 **[Test build #121899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121899/testReport)** for PR 28328 at commit

[GitHub] [spark] SparkQA commented on pull request #28356: [SPARK-31485][CORE][3.0] Avoid application hang if only partial barrier tasks launched

2020-04-27 Thread GitBox
SparkQA commented on pull request #28356: URL: https://github.com/apache/spark/pull/28356#issuecomment-619896941 **[Test build #121882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121882/testReport)** for PR 28356 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28356: [SPARK-31485][CORE][3.0] Avoid application hang if only partial barrier tasks launched

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28356: URL: https://github.com/apache/spark/pull/28356#issuecomment-619779844 **[Test build #121882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121882/testReport)** for PR 28356 at commit

[GitHub] [spark] MichaelChirico commented on a change in pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
MichaelChirico commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415704429 ## File path: R/pkg/R/DataFrame.R ## @@ -2587,18 +2589,18 @@ setMethod("join", if (is.null(joinType)) { sdf <-

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28328: [SPARK-31553][SQL] Fix isInCollection for collection sizes above the optimisation threshold

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28328: URL: https://github.com/apache/spark/pull/28328#issuecomment-619894210 This is an automated message from the Apache Git Service. To respond to the message, please log on

<    2   3   4   5   6   7   8   9   10   >