[GitHub] [spark] AmplabJenkins commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654874713 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

SparkQA commented on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-654873866 **[Test build #125223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125223/testReport)** for PR 29019 at commit

[GitHub] [spark] wangyum commented on pull request #29022: [SPARK-32207][SQL] Support 'F'-suffixed Float Literals

wangyum commented on pull request #29022: URL: https://github.com/apache/spark/pull/29022#issuecomment-654874240 ```sql postgres=# CREATE TABLE test_float2 as SELECT 1.0F, 2.0D, 1.0e2, 1.0e; SELECT 1 postgres=# \d test_float2; Table "public.test_float2" Column

[GitHub] [spark] SparkQA commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

SparkQA commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654873687 **[Test build #125224 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125224/testReport)** for PR 28683 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

cloud-fan commented on a change in pull request #28979: URL: https://github.com/apache/spark/pull/28979#discussion_r450875928 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala ## @@ -102,6 +105,25 @@ case class ScalaUDF( }

[GitHub] [spark] cloud-fan commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

cloud-fan commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654870731 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] cloud-fan commented on a change in pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

cloud-fan commented on a change in pull request #28983: URL: https://github.com/apache/spark/pull/28983#discussion_r450874248 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/UDAQuerySuite.scala ## @@ -119,6 +119,15 @@ object CountSerDeAgg extends

[GitHub] [spark] cloud-fan commented on a change in pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

cloud-fan commented on a change in pull request #28983: URL: https://github.com/apache/spark/pull/28983#discussion_r450873817 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/udaf.scala ## @@ -469,7 +471,8 @@ case class ScalaAggregator[IN, BUF,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

AmplabJenkins removed a comment on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-654868880 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

AmplabJenkins commented on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-654868880 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29012: [SPARK-31710][SQL][FOLLOWUP] Allow cast numeric to timestamp by default

AmplabJenkins removed a comment on pull request #29012: URL: https://github.com/apache/spark/pull/29012#issuecomment-654867840 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29012: [SPARK-31710][SQL][FOLLOWUP] Allow cast numeric to timestamp by default

AmplabJenkins removed a comment on pull request #29012: URL: https://github.com/apache/spark/pull/29012#issuecomment-654867823 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] cloud-fan commented on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

cloud-fan commented on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-654867835 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins commented on pull request #29012: [SPARK-31710][SQL][FOLLOWUP] Allow cast numeric to timestamp by default

AmplabJenkins commented on pull request #29012: URL: https://github.com/apache/spark/pull/29012#issuecomment-654867823 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR edited a comment on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR edited a comment on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-654853105 So if I understand correctly, what we want to confirm is that the (size+compression):memory ratio goes linearly on the number of tasks or not. Like short running

[GitHub] [spark] SparkQA removed a comment on pull request #29012: [SPARK-31710][SQL][FOLLOWUP] Allow cast numeric to timestamp by default

SparkQA removed a comment on pull request #29012: URL: https://github.com/apache/spark/pull/29012#issuecomment-654688957 **[Test build #125199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125199/testReport)** for PR 29012 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-654866771 That said, probably we also haven't check how much hybrid KV store will help on the smaller app event log (like 100mb or even smaller in uncompressed). If that turns out to

[GitHub] [spark] cloud-fan commented on pull request #28808: [SPARK-31975][SQL] Show AnalysisException when WindowFunction is used without WindowExpression

cloud-fan commented on pull request #28808: URL: https://github.com/apache/spark/pull/28808#issuecomment-654867130 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #28808: [SPARK-31975][SQL] Show AnalysisException when WindowFunction is used without WindowExpression

cloud-fan closed pull request #28808: URL: https://github.com/apache/spark/pull/28808 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] SparkQA commented on pull request #29012: [SPARK-31710][SQL][FOLLOWUP] Allow cast numeric to timestamp by default

SparkQA commented on pull request #29012: URL: https://github.com/apache/spark/pull/29012#issuecomment-654866560 **[Test build #125199 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125199/testReport)** for PR 29012 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

AmplabJenkins removed a comment on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-654864222 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

AmplabJenkins commented on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-654864222 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28999: [SPARK-32209][SQL] Re-use GetTimestamp in ParseToDate

AmplabJenkins removed a comment on pull request #28999: URL: https://github.com/apache/spark/pull/28999#issuecomment-654861466 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

SparkQA commented on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-654862573 **[Test build #125222 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125222/testReport)** for PR 28993 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28999: [SPARK-32209][SQL] Re-use GetTimestamp in ParseToDate

AmplabJenkins commented on pull request #28999: URL: https://github.com/apache/spark/pull/28999#issuecomment-654861466 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28808: [SPARK-31975][SQL] Show AnalysisException when WindowFunction is used without WindowExpression

AmplabJenkins commented on pull request #28808: URL: https://github.com/apache/spark/pull/28808#issuecomment-654860364 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28808: [SPARK-31975][SQL] Show AnalysisException when WindowFunction is used without WindowExpression

AmplabJenkins removed a comment on pull request #28808: URL: https://github.com/apache/spark/pull/28808#issuecomment-654860364 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

cloud-fan commented on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-654860601 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA removed a comment on pull request #28999: [SPARK-32209][SQL] Re-use GetTimestamp in ParseToDate

SparkQA removed a comment on pull request #28999: URL: https://github.com/apache/spark/pull/28999#issuecomment-654668593 **[Test build #125192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125192/testReport)** for PR 28999 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28808: [SPARK-31975][SQL] Show AnalysisException when WindowFunction is used without WindowExpression

SparkQA removed a comment on pull request #28808: URL: https://github.com/apache/spark/pull/28808#issuecomment-654649989 **[Test build #125185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125185/testReport)** for PR 28808 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

AmplabJenkins commented on pull request #28972: URL: https://github.com/apache/spark/pull/28972#issuecomment-654857824 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28999: [SPARK-32209][SQL] Re-use GetTimestamp in ParseToDate

SparkQA commented on pull request #28999: URL: https://github.com/apache/spark/pull/28999#issuecomment-654858374 **[Test build #125192 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125192/testReport)** for PR 28999 at commit

[GitHub] [spark] SparkQA commented on pull request #28808: [SPARK-31975][SQL] Show AnalysisException when WindowFunction is used without WindowExpression

SparkQA commented on pull request #28808: URL: https://github.com/apache/spark/pull/28808#issuecomment-654857834 **[Test build #125185 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125185/testReport)** for PR 28808 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

AmplabJenkins removed a comment on pull request #28972: URL: https://github.com/apache/spark/pull/28972#issuecomment-654857824 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

SparkQA commented on pull request #28972: URL: https://github.com/apache/spark/pull/28972#issuecomment-654856716 **[Test build #125221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125221/testReport)** for PR 28972 at commit

[GitHub] [spark] tgravescs commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

tgravescs commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-654856475 that is correct. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

AmplabJenkins removed a comment on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-654855031 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] tgravescs commented on pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

tgravescs commented on pull request #28972: URL: https://github.com/apache/spark/pull/28972#issuecomment-654855539 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] tgravescs commented on a change in pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

tgravescs commented on a change in pull request #28972: URL: https://github.com/apache/spark/pull/28972#discussion_r450859812 ## File path: core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala ## @@ -245,6 +245,7 @@ object ResourceProfile extends Logging { //

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

AmplabJenkins removed a comment on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-654855014 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

AmplabJenkins commented on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-654855014 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

SparkQA removed a comment on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-654759347 **[Test build #125207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125207/testReport)** for PR 29019 at commit

[GitHub] [spark] SparkQA commented on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

SparkQA commented on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-654854009 **[Test build #125207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125207/testReport)** for PR 29019 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-654853105 So if I understand correctly, what we want to confirm is that the (size+compression)-memory ratio goes linearly on the number of tasks or not. Like short running

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

AmplabJenkins removed a comment on pull request #28983: URL: https://github.com/apache/spark/pull/28983#issuecomment-654847252 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

AmplabJenkins commented on pull request #28983: URL: https://github.com/apache/spark/pull/28983#issuecomment-654847252 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

SparkQA commented on pull request #28983: URL: https://github.com/apache/spark/pull/28983#issuecomment-654846216 **[Test build #125220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125220/testReport)** for PR 28983 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins removed a comment on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654845354 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

SparkQA removed a comment on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654812553 **[Test build #125217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125217/testReport)** for PR 28683 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654845340 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

SparkQA commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654845031 **[Test build #125217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125217/testReport)** for PR 28683 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins removed a comment on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654845340 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] tgravescs commented on a change in pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

tgravescs commented on a change in pull request #28972: URL: https://github.com/apache/spark/pull/28972#discussion_r450850223 ## File path: core/src/main/scala/org/apache/spark/resource/ExecutorResourceRequests.scala ## @@ -54,6 +54,19 @@ class ExecutorResourceRequests()

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

AmplabJenkins removed a comment on pull request #28979: URL: https://github.com/apache/spark/pull/28979#issuecomment-654841740 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

AmplabJenkins commented on pull request #28979: URL: https://github.com/apache/spark/pull/28979#issuecomment-654841740 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

SparkQA removed a comment on pull request #28979: URL: https://github.com/apache/spark/pull/28979#issuecomment-654649950 **[Test build #125184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125184/testReport)** for PR 28979 at commit

[GitHub] [spark] SparkQA commented on pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

SparkQA commented on pull request #28979: URL: https://github.com/apache/spark/pull/28979#issuecomment-654839638 **[Test build #125184 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125184/testReport)** for PR 28979 at commit

[GitHub] [spark] erikerlandson commented on a change in pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

erikerlandson commented on a change in pull request #28983: URL: https://github.com/apache/spark/pull/28983#discussion_r450843199 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/UDAQuerySuite.scala ## @@ -156,20 +165,11 @@ abstract class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

AmplabJenkins removed a comment on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-654838383 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] tgravescs commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

tgravescs commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-654838940 thanks, for testing those combinations, that shows the number of jobs doesn't affect the memory usage much, but if I'm not mistaken in all the cases you have 400,000 tasks.

[GitHub] [spark] AmplabJenkins commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

AmplabJenkins commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-654838383 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

SparkQA commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-654837665 **[Test build #125219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125219/testReport)** for PR 28412 at commit

[GitHub] [spark] LantaoJin edited a comment on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin edited a comment on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-654836968 AFAIK, hive client (metastore clien) is not thread-safe. I don't find any documentation but I find a patch

[GitHub] [spark] LantaoJin commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-654836968 In my opinion, hive client (metastore clien) is not thread-safe. I don't find any documentation but I find a patch

[GitHub] [spark] tgravescs commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

tgravescs commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-654835055 test this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

AmplabJenkins removed a comment on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-654834498 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

AmplabJenkins commented on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-654834491 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

AmplabJenkins removed a comment on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-654834491 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29024: [WIP][SPARK-32001][SQL]Create JDBC authentication provider developer API

AmplabJenkins commented on pull request #29024: URL: https://github.com/apache/spark/pull/29024#issuecomment-654834092 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29024: [WIP][SPARK-32001][SQL]Create JDBC authentication provider developer API

AmplabJenkins removed a comment on pull request #29024: URL: https://github.com/apache/spark/pull/29024#issuecomment-654834092 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

AmplabJenkins removed a comment on pull request #28983: URL: https://github.com/apache/spark/pull/28983#issuecomment-654560221 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

SparkQA commented on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-654833433 **[Test build #125193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125193/testReport)** for PR 28993 at commit

[GitHub] [spark] SparkQA commented on pull request #29024: [WIP][SPARK-32001][SQL]Create JDBC authentication provider developer API

SparkQA commented on pull request #29024: URL: https://github.com/apache/spark/pull/29024#issuecomment-654833408 **[Test build #125218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125218/testReport)** for PR 29024 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

SparkQA removed a comment on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-654668595 **[Test build #125193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125193/testReport)** for PR 28993 at commit

[GitHub] [spark] cloud-fan commented on pull request #28882: [SPARK-31751][SQL]Serde property `path` overwrites hive table property location

cloud-fan commented on pull request #28882: URL: https://github.com/apache/spark/pull/28882#issuecomment-654833008 can we test non-hive compatible file sources like json? e.g. `CREATE TABLE t USING json LOCATION ...`. It seems like the serde property `path` will always be different from

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

AmplabJenkins removed a comment on pull request #28983: URL: https://github.com/apache/spark/pull/28983#issuecomment-654560215 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] erikerlandson commented on a change in pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

erikerlandson commented on a change in pull request #28983: URL: https://github.com/apache/spark/pull/28983#discussion_r450837156 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/UDAQuerySuite.scala ## @@ -354,6 +355,13 @@ abstract class UDAQuerySuite

[GitHub] [spark] gaborgsomogyi commented on pull request #29024: [WIP][SPARK-32001][SQL]Create JDBC authentication provider developer API

gaborgsomogyi commented on pull request #29024: URL: https://github.com/apache/spark/pull/29024#issuecomment-654832549 I've created a WIP PR because the MariaDB integration test fails on master as well so I can't tell this change is good or not: ``` 2020-07-07 12:43:13 8 [Warning]

[GitHub] [spark] gaborgsomogyi opened a new pull request #29024: [WIP][SPARK-32001][SQL]Create JDBC authentication provider developer API

gaborgsomogyi opened a new pull request #29024: URL: https://github.com/apache/spark/pull/29024 ### What changes were proposed in this pull request? At the moment only the baked in JDBC connection providers can be used but there is a need to support additional databases and use-cases.

[GitHub] [spark] cloud-fan commented on a change in pull request #28882: [SPARK-31751][SQL]Serde property `path` overwrites hive table property location

cloud-fan commented on a change in pull request #28882: URL: https://github.com/apache/spark/pull/28882#discussion_r450833763 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ## @@ -545,7 +545,14 @@ private[spark] class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28989: [WIP][SPARK-27194][SPARK-29302][SQL] Define a spark staging committer to resolve FileAlreadyExistingException

AmplabJenkins removed a comment on pull request #28989: URL: https://github.com/apache/spark/pull/28989#issuecomment-654818447 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28989: [WIP][SPARK-27194][SPARK-29302][SQL] Define a spark staging committer to resolve FileAlreadyExistingException

AmplabJenkins removed a comment on pull request #28989: URL: https://github.com/apache/spark/pull/28989#issuecomment-654818435 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28989: [WIP][SPARK-27194][SPARK-29302][SQL] Define a spark staging committer to resolve FileAlreadyExistingException

AmplabJenkins commented on pull request #28989: URL: https://github.com/apache/spark/pull/28989#issuecomment-654818435 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28989: [WIP][SPARK-27194][SPARK-29302][SQL] Define a spark staging committer to resolve FileAlreadyExistingException

SparkQA removed a comment on pull request #28989: URL: https://github.com/apache/spark/pull/28989#issuecomment-654738586 **[Test build #125205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125205/testReport)** for PR 28989 at commit

[GitHub] [spark] SparkQA commented on pull request #28989: [WIP][SPARK-27194][SPARK-29302][SQL] Define a spark staging committer to resolve FileAlreadyExistingException

SparkQA commented on pull request #28989: URL: https://github.com/apache/spark/pull/28989#issuecomment-654817444 **[Test build #125205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125205/testReport)** for PR 28989 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654813212 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins removed a comment on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654813212 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

SparkQA commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654812553 **[Test build #125217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125217/testReport)** for PR 28683 at commit

[GitHub] [spark] cloud-fan commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

cloud-fan commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-654811083 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] cloud-fan commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

cloud-fan commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-654809755 Your comments seem to rely on Hive internal details, it's better to have official document to prove it. BTW I'm not sure the hive client thread-unsafety comes from

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28965: [SPARK-32124][CORE][FOLLOW-UP] Use the invalid value Int.MinValue to fill the map index when the event logs from the old Spark

AmplabJenkins removed a comment on pull request #28965: URL: https://github.com/apache/spark/pull/28965#issuecomment-654809988 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29021: [WIP][SPARK-32201][SQL] More general skew join pattern matching

AmplabJenkins removed a comment on pull request #29021: URL: https://github.com/apache/spark/pull/29021#issuecomment-654809645 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28965: [SPARK-32124][CORE][FOLLOW-UP] Use the invalid value Int.MinValue to fill the map index when the event logs from the old Spark version

AmplabJenkins commented on pull request #28965: URL: https://github.com/apache/spark/pull/28965#issuecomment-654809988 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] frgomes commented on pull request #24969: [SPARK-28151][SQL] Fix MsSqlServerDialect Byte/Short/Float type mappings ( DRAFT)

frgomes commented on pull request #24969: URL: https://github.com/apache/spark/pull/24969#issuecomment-654809859 @srowen : Feedback: As suspected, the issue was between the keyboard and the chair. Maybe I was not casting properly, or maybe a typo or maybe some equality test somewhere...

[GitHub] [spark] AmplabJenkins commented on pull request #29021: [WIP][SPARK-32201][SQL] More general skew join pattern matching

AmplabJenkins commented on pull request #29021: URL: https://github.com/apache/spark/pull/29021#issuecomment-654809645 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28965: [SPARK-32124][CORE][FOLLOW-UP] Use the invalid value Int.MinValue to fill the map index when the event logs from the old Spark version

SparkQA commented on pull request #28965: URL: https://github.com/apache/spark/pull/28965#issuecomment-654809086 **[Test build #125216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125216/testReport)** for PR 28965 at commit

[GitHub] [spark] SparkQA commented on pull request #29021: [WIP][SPARK-32201][SQL] More general skew join pattern matching

SparkQA commented on pull request #29021: URL: https://github.com/apache/spark/pull/29021#issuecomment-654809085 **[Test build #125215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125215/testReport)** for PR 29021 at commit

[GitHub] [spark] LantaoJin commented on a change in pull request #29021: [WIP][SPARK-32201][SQL] More general skew join pattern matching

LantaoJin commented on a change in pull request #29021: URL: https://github.com/apache/spark/pull/29021#discussion_r450666963 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -730,6 +713,67 @@ class

[GitHub] [spark] HeartSaVioR commented on pull request #28965: [SPARK-32124][CORE][FOLLOW-UP] Use the invalid value Int.MinValue to fill the map index when the event logs from the old Spark version

HeartSaVioR commented on pull request #28965: URL: https://github.com/apache/spark/pull/28965#issuecomment-654807287 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28969: [SPARK-32150][BUILD] Upgrade to ZStd 1.4.5-4

AmplabJenkins removed a comment on pull request #28969: URL: https://github.com/apache/spark/pull/28969#issuecomment-654805323 Test FAILed. Refer to this link for build results (access rights to CI server needed):

<    5   6   7   8   9   10   11   12   13   14   >