[GitHub] [spark] AmplabJenkins removed a comment on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

AmplabJenkins removed a comment on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#issuecomment-470789551 Can one of the admins verify this patch?

[GitHub] [spark] happyhua commented on issue #23827: [SPARK-26912][CORE][HISTORY] Allow setting permission for event_log

happyhua commented on issue #23827: [SPARK-26912][CORE][HISTORY] Allow setting permission for event_log URL: https://github.com/apache/spark/pull/23827#issuecomment-472411198 Permission of event_log should be configurable. Probably 775 permission as default is better. In our situation

[GitHub] [spark] AmplabJenkins commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled

AmplabJenkins commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled URL: https://github.com/apache/spark/pull/20433#issuecomment-472420986 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled

AmplabJenkins commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled URL: https://github.com/apache/spark/pull/20433#issuecomment-472420976 Merged build finished. Test PASSed.

[GitHub] [spark] attilapiros commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

attilapiros commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#discussion_r265162114 ## File path:

[GitHub] [spark] attilapiros commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

attilapiros commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#discussion_r265162114 ## File path:

[GitHub] [spark] attilapiros commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

attilapiros commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#discussion_r265162114 ## File path:

[GitHub] [spark] sujith71955 commented on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS"

sujith71955 commented on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS" URL: https://github.com/apache/spark/pull/24075#issuecomment-472459765 retest this please

[GitHub] [spark] SparkQA removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

SparkQA removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472341429 **[Test build #103430 has

[GitHub] [spark] AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472418864 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472418893 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

AmplabJenkins removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472418864 Merged build finished. Test FAILed.

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r265140605 ## File path: core/src/main/scala/org/apache/spark/deploy/ExternalShuffleService.scala ## @@

[GitHub] [spark] maropu commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

maropu commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-472433715 kindly ping @cloud-fan This is an automated message from the Apache Git

[GitHub] [spark] srowen commented on a change in pull request #23919: [MINOR][DOC] Documentation improvement: More detailed explanation of possible "deploy-mode"s

srowen commented on a change in pull request #23919: [MINOR][DOC] Documentation improvement: More detailed explanation of possible "deploy-mode"s URL: https://github.com/apache/spark/pull/23919#discussion_r265146573 ## File path: docs/submitting-applications.md ## @@

[GitHub] [spark] srowen commented on a change in pull request #23919: [MINOR][DOC] Documentation improvement: More detailed explanation of possible "deploy-mode"s

srowen commented on a change in pull request #23919: [MINOR][DOC] Documentation improvement: More detailed explanation of possible "deploy-mode"s URL: https://github.com/apache/spark/pull/23919#discussion_r265146185 ## File path: docs/submitting-applications.md ## @@

[GitHub] [spark] viirya commented on issue #24053: [SPARK-27126][SQL] Consolidate Scala and Java type deserializerFor

viirya commented on issue #24053: [SPARK-27126][SQL] Consolidate Scala and Java type deserializerFor URL: https://github.com/apache/spark/pull/24053#issuecomment-472440686 Yea, I think that's the right way to go. I realized we can make move to that direction during this refactoring and I

[GitHub] [spark] maropu commented on a change in pull request #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled

maropu commented on a change in pull request #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled URL: https://github.com/apache/spark/pull/20433#discussion_r265112136 ## File path:

[GitHub] [spark] ajithme commented on issue #24076: [SPARK-27142] Provide REST API for SQL level information

ajithme commented on issue #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#issuecomment-472413717 > @ajithme Thanks for the work. > To make it consistent with other API, I think we need to have at least two APIs >

[GitHub] [spark] AmplabJenkins removed a comment on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled

AmplabJenkins removed a comment on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled URL: https://github.com/apache/spark/pull/20433#issuecomment-472420986 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] [spark] AmplabJenkins removed a comment on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled

AmplabJenkins removed a comment on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled URL: https://github.com/apache/spark/pull/20433#issuecomment-472420976 Merged build finished. Test PASSed.

[GitHub] [spark] maropu commented on a change in pull request #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

maropu commented on a change in pull request #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#discussion_r265134944 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ##

[GitHub] [spark] srowen commented on a change in pull request #24057: [SPARK-26839][SQL] Work around classloader changes in Java 9 for Hive isolation

srowen commented on a change in pull request #24057: [SPARK-26839][SQL] Work around classloader changes in Java 9 for Hive isolation URL: https://github.com/apache/spark/pull/24057#discussion_r265142878 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala

[GitHub] [spark] ajithme commented on issue #24076: [SPARK-27142] Provide REST API for SQL level information

ajithme commented on issue #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#issuecomment-472435640 Ok i have updated the PR accordingly. Please review This is

[GitHub] [spark] DaveDeCaprio commented on a change in pull request #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager

DaveDeCaprio commented on a change in pull request #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager URL: https://github.com/apache/spark/pull/24028#discussion_r265142932 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala

[GitHub] [spark] DaveDeCaprio commented on a change in pull request #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager

DaveDeCaprio commented on a change in pull request #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager URL: https://github.com/apache/spark/pull/24028#discussion_r265142809 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r265152639 ## File path: core/src/main/scala/org/apache/spark/deploy/ExternalShuffleService.scala ## @@

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r265163265 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -245,4

[GitHub] [spark] shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#discussion_r265166923 ## File path:

[GitHub] [spark] shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#discussion_r265166923 ## File path:

[GitHub] [spark] gengliangwang commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information

gengliangwang commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#discussion_r265178647 ## File path: sql/core/src/main/scala/org/apache/spark/status/api/v1/SqlListResource.scala

[GitHub] [spark] gengliangwang commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information

gengliangwang commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#discussion_r265179697 ## File path: sql/core/src/main/scala/org/apache/spark/status/api/v1/SqlListResource.scala

[GitHub] [spark] gengliangwang commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information

gengliangwang commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#discussion_r265179417 ## File path: sql/core/src/main/scala/org/apache/spark/status/api/v1/SqlListResource.scala

[GitHub] [spark] gengliangwang commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information

gengliangwang commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#discussion_r265177205 ## File path: sql/core/src/main/scala/org/apache/spark/status/api/v1/SqlListResource.scala

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24049: [SPARK-27123][SQL] Improve CollapseProject to handle projects cross limit/repartition/sample

dongjoon-hyun commented on a change in pull request #24049: [SPARK-27123][SQL] Improve CollapseProject to handle projects cross limit/repartition/sample URL: https://github.com/apache/spark/pull/24049#discussion_r265189229 ## File path:

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r265189239 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -245,4

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r265189239 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -245,4

[GitHub] [spark] HeartSaVioR edited a comment on issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaDataConsumer

HeartSaVioR edited a comment on issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaDataConsumer URL: https://github.com/apache/spark/pull/22138#issuecomment-472479384 UPDATE: just added log message to log when Kafka consumer is created. * master:

[GitHub] [spark] HeartSaVioR edited a comment on issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaDataConsumer

HeartSaVioR edited a comment on issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaDataConsumer URL: https://github.com/apache/spark/pull/22138#issuecomment-472479384 UPDATE: just added log message to log when Kafka consumer is created. * master:

[GitHub] [spark] justinuang commented on issue #20303: [SPARK-23128][SQL] A new approach to do adaptive execution in Spark SQL

justinuang commented on issue #20303: [SPARK-23128][SQL] A new approach to do adaptive execution in Spark SQL URL: https://github.com/apache/spark/pull/20303#issuecomment-472482313 @carsonwang What happens when we call df.repartition(500) on a 10MB with AQE turned on? AQE will still

[GitHub] [spark] justinuang edited a comment on issue #20303: [SPARK-23128][SQL] A new approach to do adaptive execution in Spark SQL

justinuang edited a comment on issue #20303: [SPARK-23128][SQL] A new approach to do adaptive execution in Spark SQL URL: https://github.com/apache/spark/pull/20303#issuecomment-472482313 @carsonwang What happens when we call df.repartition(500) on a 10MB with AQE turned on? AQE will

[GitHub] [spark] cloud-fan commented on a change in pull request #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager

cloud-fan commented on a change in pull request #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager URL: https://github.com/apache/spark/pull/24028#discussion_r265207871 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ##

[GitHub] [spark] vanzin commented on issue #23827: [SPARK-26912][CORE][HISTORY] Allow setting permission for event_log

vanzin commented on issue #23827: [SPARK-26912][CORE][HISTORY] Allow setting permission for event_log URL: https://github.com/apache/spark/pull/23827#issuecomment-472499554 > Probably 775 permission as default is better. Absolutely not. That would be a security issue. > It

[GitHub] [spark] SparkQA removed a comment on issue #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown

SparkQA removed a comment on issue #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown URL: https://github.com/apache/spark/pull/24056#issuecomment-472391094 **[Test build #4617 has

[GitHub] [spark] happyhua edited a comment on issue #23827: [SPARK-26912][CORE][HISTORY] Allow setting permission for event_log

happyhua edited a comment on issue #23827: [SPARK-26912][CORE][HISTORY] Allow setting permission for event_log URL: https://github.com/apache/spark/pull/23827#issuecomment-472508692 First of all, not everyone uses Hadoop hdfs. In our scenario, we submit every sprark job from

[GitHub] [spark] happyhua edited a comment on issue #23827: [SPARK-26912][CORE][HISTORY] Allow setting permission for event_log

happyhua edited a comment on issue #23827: [SPARK-26912][CORE][HISTORY] Allow setting permission for event_log URL: https://github.com/apache/spark/pull/23827#issuecomment-472508692 First of all, not everyone uses Hadoop hdfs. In our scenario, we submit every sprark job from

[GitHub] [spark] jzhuge commented on a change in pull request #23848: [SPARK-26946][SQL] Identifiers for multi-catalog

jzhuge commented on a change in pull request #23848: [SPARK-26946][SQL] Identifiers for multi-catalog URL: https://github.com/apache/spark/pull/23848#discussion_r265234026 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/identifiers.scala ## @@

[GitHub] [spark] lwwmanning opened a new pull request #24083: [SPARK-24432] Support dynamic allocation without external shuffle service

lwwmanning opened a new pull request #24083: [SPARK-24432] Support dynamic allocation without external shuffle service URL: https://github.com/apache/spark/pull/24083 ## What changes were proposed in this pull request? This PR adds a limited version of dynamic allocation that does

[GitHub] [spark] cloud-fan commented on issue #24082: [SPARK-27123][SQL][FOLLOWUP] Use isRenaming check for limit too.

cloud-fan commented on issue #24082: [SPARK-27123][SQL][FOLLOWUP] Use isRenaming check for limit too. URL: https://github.com/apache/spark/pull/24082#issuecomment-472526749 LGTM This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function

AmplabJenkins removed a comment on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function URL: https://github.com/apache/spark/pull/24051#issuecomment-472529362 Test FAILed. Refer to this link for build results (access rights to CI

[GitHub] [spark] dongjoon-hyun commented on issue #24082: [SPARK-27123][SQL][FOLLOWUP] Use isRenaming check for limit too.

dongjoon-hyun commented on issue #24082: [SPARK-27123][SQL][FOLLOWUP] Use isRenaming check for limit too. URL: https://github.com/apache/spark/pull/24082#issuecomment-472529890 Retest this please. This is an automated

[GitHub] [spark] SparkQA commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

SparkQA commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-472535476 **[Test build #103444 has

[GitHub] [spark] SparkQA commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled

SparkQA commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled URL: https://github.com/apache/spark/pull/20433#issuecomment-472535580 **[Test build #103445 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-472538357 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #24044: [WIP][test-hadoop3.1] Test Hadoop 3.1 on jenkins

AmplabJenkins removed a comment on issue #24044: [WIP][test-hadoop3.1] Test Hadoop 3.1 on jenkins URL: https://github.com/apache/spark/pull/24044#issuecomment-472538308 Build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins removed a comment on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS"

AmplabJenkins removed a comment on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS" URL: https://github.com/apache/spark/pull/24075#issuecomment-472538240 Test PASSed. Refer to this link for build

[GitHub] [spark] AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-472538347 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager

AmplabJenkins removed a comment on issue #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager URL: https://github.com/apache/spark/pull/24028#issuecomment-472538268 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins removed a comment on issue #24044: [WIP][test-hadoop3.1] Test Hadoop 3.1 on jenkins

AmplabJenkins removed a comment on issue #24044: [WIP][test-hadoop3.1] Test Hadoop 3.1 on jenkins URL: https://github.com/apache/spark/pull/24044#issuecomment-472538331 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS"

AmplabJenkins removed a comment on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS" URL: https://github.com/apache/spark/pull/24075#issuecomment-472538233 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager

AmplabJenkins removed a comment on issue #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager URL: https://github.com/apache/spark/pull/24028#issuecomment-472538273 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] mccheah commented on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service

mccheah commented on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service URL: https://github.com/apache/spark/pull/24083#issuecomment-472539915 ok to test This is an automated message from

[GitHub] [spark] AmplabJenkins removed a comment on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service

AmplabJenkins removed a comment on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service URL: https://github.com/apache/spark/pull/24083#issuecomment-472537944 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins removed a comment on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service

AmplabJenkins removed a comment on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service URL: https://github.com/apache/spark/pull/24083#issuecomment-472541293 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service

AmplabJenkins removed a comment on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service URL: https://github.com/apache/spark/pull/24083#issuecomment-472541264 Merged build finished. Test PASSed.

[GitHub] [spark] HeartSaVioR edited a comment on issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaDataConsumer

HeartSaVioR edited a comment on issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaDataConsumer URL: https://github.com/apache/spark/pull/22138#issuecomment-472479384 UPDATE: just added log message to log when Kafka consumer is created. * master:

[GitHub] [spark] MaxGekk commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

MaxGekk commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472483976 @yanboliang Could you help me to regenerate expected values for `LogisticRegressionSuite`, please.

[GitHub] [spark] cloud-fan commented on a change in pull request #24049: [SPARK-27123][SQL] Improve CollapseProject to handle projects cross limit/repartition/sample

cloud-fan commented on a change in pull request #24049: [SPARK-27123][SQL] Improve CollapseProject to handle projects cross limit/repartition/sample URL: https://github.com/apache/spark/pull/24049#discussion_r265206012 ## File path:

[GitHub] [spark] cloud-fan commented on a change in pull request #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager

cloud-fan commented on a change in pull request #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager URL: https://github.com/apache/spark/pull/24028#discussion_r265208435 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ##

[GitHub] [spark] squito commented on a change in pull request #24057: [SPARK-26839][SQL] Work around classloader changes in Java 9 for Hive isolation

squito commented on a change in pull request #24057: [SPARK-26839][SQL] Work around classloader changes in Java 9 for Hive isolation URL: https://github.com/apache/spark/pull/24057#discussion_r265213623 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala

[GitHub] [spark] SparkQA commented on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function

SparkQA commented on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function URL: https://github.com/apache/spark/pull/24051#issuecomment-472495422 **[Test build #103434 has

[GitHub] [spark] SparkQA removed a comment on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function

SparkQA removed a comment on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function URL: https://github.com/apache/spark/pull/24051#issuecomment-472399853 **[Test build #103434 has

[GitHub] [spark] gagafunctor commented on a change in pull request #23983: [SPARK-26881][mllib] Heuristic for tree aggregate depth

gagafunctor commented on a change in pull request #23983: [SPARK-26881][mllib] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983#discussion_r265219047 ## File path: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala

[GitHub] [spark] SparkQA commented on issue #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown

SparkQA commented on issue #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown URL: https://github.com/apache/spark/pull/24056#issuecomment-472500740 **[Test build #4617 has

[GitHub] [spark] ajithme commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information

ajithme commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#discussion_r265221881 ## File path: sql/core/src/main/scala/org/apache/spark/status/api/v1/SqlListResource.scala

[GitHub] [spark] SparkQA removed a comment on issue #24024: [MINOR][CORE]spark.diskStore.subDirectories <= 0 should throw Exception

SparkQA removed a comment on issue #24024: [MINOR][CORE]spark.diskStore.subDirectories <= 0 should throw Exception URL: https://github.com/apache/spark/pull/24024#issuecomment-472396662 **[Test build #4618 has

[GitHub] [spark] SparkQA commented on issue #24024: [MINOR][CORE]spark.diskStore.subDirectories <= 0 should throw Exception

SparkQA commented on issue #24024: [MINOR][CORE]spark.diskStore.subDirectories <= 0 should throw Exception URL: https://github.com/apache/spark/pull/24024#issuecomment-472512888 **[Test build #4618 has

[GitHub] [spark] happyhua commented on issue #23827: [SPARK-26912][CORE][HISTORY] Allow setting permission for event_log

happyhua commented on issue #23827: [SPARK-26912][CORE][HISTORY] Allow setting permission for event_log URL: https://github.com/apache/spark/pull/23827#issuecomment-472516443 I wonder if you have used spark history server. So you want people have a cron job to chmod the event files

[GitHub] [spark] AmplabJenkins removed a comment on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

AmplabJenkins removed a comment on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#issuecomment-472516087 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions

AmplabJenkins commented on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions URL: https://github.com/apache/spark/pull/24025#issuecomment-472520302 Merged build finished. Test PASSed. This

[GitHub] [spark] dongjoon-hyun commented on issue #24082: [SPARK-27123][SQL][FOLLOWUP] Use isRenaming check for limit too.

dongjoon-hyun commented on issue #24082: [SPARK-27123][SQL][FOLLOWUP] Use isRenaming check for limit too. URL: https://github.com/apache/spark/pull/24082#issuecomment-472529664 Thank you so much for review and approval, @cloud-fan .

[GitHub] [spark] AmplabJenkins commented on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function

AmplabJenkins commented on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function URL: https://github.com/apache/spark/pull/24051#issuecomment-472529352 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins commented on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function

AmplabJenkins commented on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function URL: https://github.com/apache/spark/pull/24051#issuecomment-472529362 Test FAILed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function

AmplabJenkins removed a comment on issue #24051: [SPARK-26879][SQL] Standardize one-based column indexing for stack and json_tuple function URL: https://github.com/apache/spark/pull/24051#issuecomment-472529352 Merged build finished. Test FAILed.

[GitHub] [spark] mcheah commented on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service

mcheah commented on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service URL: https://github.com/apache/spark/pull/24083#issuecomment-472530317 @mccheah This is an automated message from

[GitHub] [spark] cloud-fan commented on issue #24012: [SPARK-26811][SQL] Add capabilities to v2.Table

cloud-fan commented on issue #24012: [SPARK-26811][SQL] Add capabilities to v2.Table URL: https://github.com/apache/spark/pull/24012#issuecomment-472530353 LGTM except https://github.com/apache/spark/pull/24012/files#r264765864

[GitHub] [spark] huaxingao opened a new pull request #24084: [SPARK-27153][PYTHON]add weightCol in python RegressionEvaluator

huaxingao opened a new pull request #24084: [SPARK-27153][PYTHON]add weightCol in python RegressionEvaluator URL: https://github.com/apache/spark/pull/24084 ## What changes were proposed in this pull request? add weightCol in python version of RegressionEvaluator and

[GitHub] [spark] SparkQA commented on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

SparkQA commented on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#issuecomment-472535259 **[Test build #103440 has

[GitHub] [spark] SparkQA commented on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS"

SparkQA commented on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS" URL: https://github.com/apache/spark/pull/24075#issuecomment-472535333 **[Test build #103441 has

[GitHub] [spark] SparkQA commented on issue #24084: [SPARK-27153][PYTHON]add weightCol in python RegressionEvaluator

SparkQA commented on issue #24084: [SPARK-27153][PYTHON]add weightCol in python RegressionEvaluator URL: https://github.com/apache/spark/pull/24084#issuecomment-472535188 **[Test build #103437 has

[GitHub] [spark] SparkQA commented on issue #24082: [SPARK-27123][SQL][FOLLOWUP] Use isRenaming check for limit too.

SparkQA commented on issue #24082: [SPARK-27123][SQL][FOLLOWUP] Use isRenaming check for limit too. URL: https://github.com/apache/spark/pull/24082#issuecomment-472535221 **[Test build #103438 has

[GitHub] [spark] AmplabJenkins commented on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service

AmplabJenkins commented on issue #24083: [SPARK-24432] Support dynamic allocation without external shuffle service URL: https://github.com/apache/spark/pull/24083#issuecomment-472535078 Can one of the admins verify this patch?

[GitHub] [spark] SparkQA commented on issue #24044: [WIP][test-hadoop3.1] Test Hadoop 3.1 on jenkins

SparkQA commented on issue #24044: [WIP][test-hadoop3.1] Test Hadoop 3.1 on jenkins URL: https://github.com/apache/spark/pull/24044#issuecomment-472535415 **[Test build #103442 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103442/testReport)** for PR

[GitHub] [spark] SparkQA commented on issue #24081: [SPARK-27151][SQL] ClearCacheCommand should be case-object to avoid copys

SparkQA commented on issue #24081: [SPARK-27151][SQL] ClearCacheCommand should be case-object to avoid copys URL: https://github.com/apache/spark/pull/24081#issuecomment-472535253 **[Test build #103439 has

[GitHub] [spark] SparkQA commented on issue #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager

SparkQA commented on issue #24028: [SPARK-26917][SQL] Further reduce locks in CacheManager URL: https://github.com/apache/spark/pull/24028#issuecomment-472535407 **[Test build #103443 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103443/testReport)** for

[GitHub] [spark] AmplabJenkins removed a comment on issue #24082: [SPARK-27123][SQL][FOLLOWUP] Use isRenaming check for limit too.

AmplabJenkins removed a comment on issue #24082: [SPARK-27123][SQL][FOLLOWUP] Use isRenaming check for limit too. URL: https://github.com/apache/spark/pull/24082#issuecomment-472538168 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS"

AmplabJenkins commented on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS" URL: https://github.com/apache/spark/pull/24075#issuecomment-472538233 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

AmplabJenkins removed a comment on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#issuecomment-472538163 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS"

AmplabJenkins commented on issue #24075: [SPARK-26176][SQL] Invalid column names validation is been added when we create a table using the Hive serde "STORED AS" URL: https://github.com/apache/spark/pull/24075#issuecomment-472538240 Test PASSed. Refer to this link for build results

[GitHub] [spark] AmplabJenkins removed a comment on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

AmplabJenkins removed a comment on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#issuecomment-472538157 Merged build finished. Test PASSed.

<    1   2   3   4   5   6   7   8   9   10   >