[GitHub] [spark] AmplabJenkins commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-653932427 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28983: URL: https://github.com/apache/spark/pull/28983#issuecomment-653932519 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] erikerlandson commented on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

2020-07-05 Thread GitBox
erikerlandson commented on pull request #28983: URL: https://github.com/apache/spark/pull/28983#issuecomment-653932475 > Have you checked the existing predfined ones? tentatively this looks to be working in my repl testing. The unit tests appear to bypass the use of session builder a

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28926: [SPARK-32133][SQL] Forbid time field steps for date start/end in Sequence

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28926: URL: https://github.com/apache/spark/pull/28926#issuecomment-653932336 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28912: [SPARK-32057][SQL][test-hive1.2] ExecuteStatement: cancel and close should not transiently ERROR

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-653932319 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28882: [SPARK-31751][SQL]Serde property `path` overwrites hive table property location

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28882: URL: https://github.com/apache/spark/pull/28882#issuecomment-653932380 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28975: [SPARK-32148][SS] Fix stream-stream join issue on missing to copy reused unsafe row

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28975: URL: https://github.com/apache/spark/pull/28975#issuecomment-653932192 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28939: [SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28939: URL: https://github.com/apache/spark/pull/28939#issuecomment-653932289 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-653932335 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28963: [SPARK-32145][SQL][test-hive1.2] ThriftCLIService.GetOperationStatus should include exception's stack trace to the error messag

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28963: URL: https://github.com/apache/spark/pull/28963#issuecomment-653932249 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28912: [SPARK-32057][SQL][test-hive1.2] ExecuteStatement: cancel and close should not transiently ERROR

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-653932319 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28987: [SPARK-32162][PYTHON][TESTS] Improve error message of Pandas grouped map test with window

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28987: URL: https://github.com/apache/spark/pull/28987#issuecomment-653932156 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28882: [SPARK-31751][SQL]Serde property `path` overwrites hive table property location

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28882: URL: https://github.com/apache/spark/pull/28882#issuecomment-653932380 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28926: [SPARK-32133][SQL] Forbid time field steps for date start/end in Sequence

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28926: URL: https://github.com/apache/spark/pull/28926#issuecomment-653932336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] WinkerDu opened a new pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic parti…

2020-07-05 Thread GitBox
WinkerDu opened a new pull request #29000: URL: https://github.com/apache/spark/pull/29000 …tion overwrite mode ### What changes were proposed in this pull request? When using dynamic partition overwrite, each task has its working dir under staging dir like `stagingDir

[GitHub] [spark] AmplabJenkins commented on pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink to avoid memory issue

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28904: URL: https://github.com/apache/spark/pull/28904#issuecomment-653932363 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28965: [SPARK-32124][CORE][FOLLOW-UP] Use the invalid value Int.MinValue to fill the map index when the event logs from the old Spark

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28965: URL: https://github.com/apache/spark/pull/28965#issuecomment-653932256 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28964: [SPARK-32144][SQL] Retain EXTERNAL property in hive table properties

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28964: URL: https://github.com/apache/spark/pull/28964#issuecomment-653932246 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28859: [SPARK-32024][WEBUI] Update ApplicationStoreInfo.size during HistoryServerDiskManager initializing

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28859: URL: https://github.com/apache/spark/pull/28859#issuecomment-653932391 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28986: [SPARK-32160][CORE][PYSPARK] Disallow to create SparkContext in executors.

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28986: URL: https://github.com/apache/spark/pull/28986#issuecomment-653932163 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28983: URL: https://github.com/apache/spark/pull/28983#issuecomment-653932183 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-653932405 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28969: [SPARK-32150][BUILD] Upgrade to ZStd 1.4.5-4

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28969: URL: https://github.com/apache/spark/pull/28969#issuecomment-653932224 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28967: [SPARK-32149][SHUFFLE] Improve file path name normalisation at block resolution within the external shuffle service

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28967: URL: https://github.com/apache/spark/pull/28967#issuecomment-653932203 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28931: [SPARK-32103][YARN] Handle IPv6 host/port split in YarnRMClient

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28931: URL: https://github.com/apache/spark/pull/28931#issuecomment-653932298 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28970: [SPARK-31723][CORE][TEST] Reenable one test case in HistoryServerSuite

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28970: URL: https://github.com/apache/spark/pull/28970#issuecomment-653932191 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28990: [SPARK-32164][ML] GeneralizedLinearRegressionSummary optimization

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28990: URL: https://github.com/apache/spark/pull/28990#issuecomment-653932161 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28965: [SPARK-32124][CORE][FOLLOW-UP] Use the invalid value Int.MinValue to fill the map index when the event logs from the old Spark version

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28965: URL: https://github.com/apache/spark/pull/28965#issuecomment-653932256 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28964: [SPARK-32144][SQL] Retain EXTERNAL property in hive table properties

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28964: URL: https://github.com/apache/spark/pull/28964#issuecomment-653932246 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28963: [SPARK-32145][SQL][test-hive1.2] ThriftCLIService.GetOperationStatus should include exception's stack trace to the error message

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28963: URL: https://github.com/apache/spark/pull/28963#issuecomment-653932249 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28961: [SPARK-32143][SQL] Prevent a skewed join from producing too many partition splits

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28961: URL: https://github.com/apache/spark/pull/28961#issuecomment-653932265 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28969: [SPARK-32150][BUILD] Upgrade to ZStd 1.4.5-4

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28969: URL: https://github.com/apache/spark/pull/28969#issuecomment-653932224 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28987: [SPARK-32162][PYTHON][TESTS] Improve error message of Pandas grouped map test with window

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28987: URL: https://github.com/apache/spark/pull/28987#issuecomment-653932156 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28998: URL: https://github.com/apache/spark/pull/28998#issuecomment-653932104 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28988: [SPARK-32163][SQL] Nested pruning should work even with cosmetic variations

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28988: URL: https://github.com/apache/spark/pull/28988#issuecomment-653932145 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28988: [SPARK-32163][SQL] Nested pruning should work even with cosmetic variations

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28988: URL: https://github.com/apache/spark/pull/28988#issuecomment-653932145 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28989: [SPARK-27194][SPARK-29302][SQL] Define a spark staging committer to resolve FileAlreadyExistingException

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28989: URL: https://github.com/apache/spark/pull/28989#issuecomment-653931801 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28997: [SPARK-32172][CORE]Use createDirectory instead of mkdir

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28997: URL: https://github.com/apache/spark/pull/28997#issuecomment-653931784 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28992: [SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28992: URL: https://github.com/apache/spark/pull/28992#issuecomment-653932135 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28994: [SPARK-32170][CORE] Improve the speculation for the inefficient tasks by the task metrics.

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28994: URL: https://github.com/apache/spark/pull/28994#issuecomment-653931791 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28984: [WIP] [SPARK-32158] [SQL] Add options to toJSON function

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28984: URL: https://github.com/apache/spark/pull/28984#issuecomment-653931808 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #28967: [SPARK-32149][SHUFFLE] Improve file path name normalisation at block resolution within the external shuffle service

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28967: URL: https://github.com/apache/spark/pull/28967#issuecomment-653932203 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28983: URL: https://github.com/apache/spark/pull/28983#issuecomment-653932183 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28985: [SPARK-25262][K8S][2.4] Allow SPARK_LOCAL_DIRS to be tmpfs backed on K8S

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28985: URL: https://github.com/apache/spark/pull/28985#issuecomment-653931804 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-653932118 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28991: [SPARK-26533][SQL][test-hive1.2] Support query auto timeout cancel on thriftserver

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28991: URL: https://github.com/apache/spark/pull/28991#issuecomment-653931796 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #28986: [SPARK-32160][CORE][PYSPARK] Disallow to create SparkContext in executors.

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28986: URL: https://github.com/apache/spark/pull/28986#issuecomment-653932163 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28999: [MINOR][SQL] Re-use GetTimestamp in ParseToDate

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28999: URL: https://github.com/apache/spark/pull/28999#issuecomment-653932103 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28970: [SPARK-31723][CORE][TEST] Reenable one test case in HistoryServerSuite

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28970: URL: https://github.com/apache/spark/pull/28970#issuecomment-653932191 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28996: [SPARK-29358][SQL] Make unionByName optionally fill missing columns with nulls

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28996: URL: https://github.com/apache/spark/pull/28996#issuecomment-653932114 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28975: [SPARK-32148][SS] Fix stream-stream join issue on missing to copy reused unsafe row

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28975: URL: https://github.com/apache/spark/pull/28975#issuecomment-653932192 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28990: [SPARK-32164][ML] GeneralizedLinearRegressionSummary optimization

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28990: URL: https://github.com/apache/spark/pull/28990#issuecomment-653932161 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28999: [MINOR][SQL] Re-use GetTimestamp in ParseToDate

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28999: URL: https://github.com/apache/spark/pull/28999#issuecomment-653932103 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28989: [SPARK-27194][SPARK-29302][SQL] Define a spark staging committer to resolve FileAlreadyExistingException

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28989: URL: https://github.com/apache/spark/pull/28989#issuecomment-653932071 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins commented on pull request #28996: [SPARK-29358][SQL] Make unionByName optionally fill missing columns with nulls

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28996: URL: https://github.com/apache/spark/pull/28996#issuecomment-653932114 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-653932118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28984: [WIP] [SPARK-32158] [SQL] Add options to toJSON function

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28984: URL: https://github.com/apache/spark/pull/28984#issuecomment-653932081 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins commented on pull request #28991: [SPARK-26533][SQL][test-hive1.2] Support query auto timeout cancel on thriftserver

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28991: URL: https://github.com/apache/spark/pull/28991#issuecomment-653932070 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-652121062 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] AmplabJenkins commented on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28998: URL: https://github.com/apache/spark/pull/28998#issuecomment-653932104 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28931: [SPARK-32103][YARN] Handle IPv6 host/port split in YarnRMClient

2020-07-05 Thread GitBox
AmplabJenkins removed a comment on pull request #28931: URL: https://github.com/apache/spark/pull/28931#issuecomment-653221941 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] AmplabJenkins commented on pull request #28994: [SPARK-32170][CORE] Improve the speculation for the inefficient tasks by the task metrics.

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28994: URL: https://github.com/apache/spark/pull/28994#issuecomment-653932065 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins commented on pull request #28985: [SPARK-25262][K8S][2.4] Allow SPARK_LOCAL_DIRS to be tmpfs backed on K8S

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28985: URL: https://github.com/apache/spark/pull/28985#issuecomment-653932080 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins commented on pull request #28992: [SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28992: URL: https://github.com/apache/spark/pull/28992#issuecomment-653932134 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #27983: [SPARK-32105][SQL]Refactor current ScriptTransformationExec code

2020-07-05 Thread GitBox
SparkQA commented on pull request #27983: URL: https://github.com/apache/spark/pull/27983#issuecomment-653931985 **[Test build #124954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124954/testReport)** for PR 27983 at commit [`697dc81`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

2020-07-05 Thread GitBox
SparkQA commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-653932008 **[Test build #124957 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124957/testReport)** for PR 27066 at commit [`4315e92`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28961: [SPARK-32143][SQL] Prevent a skewed join from producing too many partition splits

2020-07-05 Thread GitBox
SparkQA commented on pull request #28961: URL: https://github.com/apache/spark/pull/28961#issuecomment-653931963 **[Test build #124939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124939/testReport)** for PR 28961 at commit [`3811ae9`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #28997: [SPARK-32172][CORE]Use createDirectory instead of mkdir

2020-07-05 Thread GitBox
AmplabJenkins commented on pull request #28997: URL: https://github.com/apache/spark/pull/28997#issuecomment-653932055 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SparkQA commented on pull request #28882: [SPARK-31751][SQL]Serde property `path` overwrites hive table property location

2020-07-05 Thread GitBox
SparkQA commented on pull request #28882: URL: https://github.com/apache/spark/pull/28882#issuecomment-653931950 **[Test build #124946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124946/testReport)** for PR 28882 at commit [`1496c66`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28859: [SPARK-32024][WEBUI] Update ApplicationStoreInfo.size during HistoryServerDiskManager initializing

2020-07-05 Thread GitBox
SparkQA commented on pull request #28859: URL: https://github.com/apache/spark/pull/28859#issuecomment-653931981 **[Test build #124947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124947/testReport)** for PR 28859 at commit [`ed05872`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

2020-07-05 Thread GitBox
SparkQA commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-653931993 **[Test build #124953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124953/testReport)** for PR 28363 at commit [`b648156`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink to avoid memory issue

2020-07-05 Thread GitBox
SparkQA commented on pull request #28904: URL: https://github.com/apache/spark/pull/28904#issuecomment-653931916 **[Test build #124945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124945/testReport)** for PR 28904 at commit [`11cb693`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

2020-07-05 Thread GitBox
SparkQA commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-653932007 **[Test build #124948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124948/testReport)** for PR 28841 at commit [`eb8b3e6`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28931: [SPARK-32103][YARN] Handle IPv6 host/port split in YarnRMClient

2020-07-05 Thread GitBox
SparkQA commented on pull request #28931: URL: https://github.com/apache/spark/pull/28931#issuecomment-653931939 **[Test build #124942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124942/testReport)** for PR 28931 at commit [`becec4e`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-05 Thread GitBox
SparkQA commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-653931962 **[Test build #124940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124940/testReport)** for PR 28957 at commit [`9ec81f2`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-07-05 Thread GitBox
SparkQA commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-653931995 **[Test build #124950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124950/testReport)** for PR 28804 at commit [`7766401`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-05 Thread GitBox
SparkQA commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-653931947 **[Test build #124951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124951/testReport)** for PR 28683 at commit [`dad8085`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource

2020-07-05 Thread GitBox
SparkQA commented on pull request #27366: URL: https://github.com/apache/spark/pull/27366#issuecomment-653932003 **[Test build #124956 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124956/testReport)** for PR 27366 at commit [`0a133ad`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28912: [SPARK-32057][SQL][test-hive1.2] ExecuteStatement: cancel and close should not transiently ERROR

2020-07-05 Thread GitBox
SparkQA commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-653931938 **[Test build #124944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124944/testReport)** for PR 28912 at commit [`4aaa34b`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28808: [SPARK-31975][SQL] Throw user facing error when use WindowFunction directly

2020-07-05 Thread GitBox
SparkQA commented on pull request #28808: URL: https://github.com/apache/spark/pull/28808#issuecomment-653931957 **[Test build #124949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124949/testReport)** for PR 28808 at commit [`3e116f5`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28926: [SPARK-32133][SQL] Forbid time field steps for date start/end in Sequence

2020-07-05 Thread GitBox
SparkQA commented on pull request #28926: URL: https://github.com/apache/spark/pull/28926#issuecomment-653931909 **[Test build #124943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124943/testReport)** for PR 28926 at commit [`b863eef`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-05 Thread GitBox
SparkQA commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-653932009 **[Test build #124952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124952/testReport)** for PR 28676 at commit [`c5f4803`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-07-05 Thread GitBox
SparkQA commented on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-653932018 **[Test build #124958 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124958/testReport)** for PR 27019 at commit [`464fbaa`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #25827: [SPARK-29128][SQL] Split predicate code in OR expressions

2020-07-05 Thread GitBox
SparkQA commented on pull request #25827: URL: https://github.com/apache/spark/pull/25827#issuecomment-653931994 **[Test build #124959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124959/testReport)** for PR 25827 at commit [`543c016`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-05 Thread GitBox
SparkQA commented on pull request #27428: URL: https://github.com/apache/spark/pull/27428#issuecomment-653931997 **[Test build #124955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124955/testReport)** for PR 27428 at commit [`73dc600`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28987: [SPARK-32162][PYTHON][TESTS] Improve error message of Pandas grouped map test with window

2020-07-05 Thread GitBox
SparkQA commented on pull request #28987: URL: https://github.com/apache/spark/pull/28987#issuecomment-653931855 **[Test build #124929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124929/testReport)** for PR 28987 at commit [`70da8b5`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28963: [SPARK-32145][SQL][test-hive1.2] ThriftCLIService.GetOperationStatus should include exception's stack trace to the error message

2020-07-05 Thread GitBox
SparkQA commented on pull request #28963: URL: https://github.com/apache/spark/pull/28963#issuecomment-653931904 **[Test build #124938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124938/testReport)** for PR 28963 at commit [`d72074a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28992: [SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

2020-07-05 Thread GitBox
SparkQA commented on pull request #28992: URL: https://github.com/apache/spark/pull/28992#issuecomment-653931838 **[Test build #124926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124926/testReport)** for PR 28992 at commit [`bc3542a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28975: [SPARK-32148][SS] Fix stream-stream join issue on missing to copy reused unsafe row

2020-07-05 Thread GitBox
SparkQA commented on pull request #28975: URL: https://github.com/apache/spark/pull/28975#issuecomment-653931875 **[Test build #124932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124932/testReport)** for PR 28975 at commit [`be34258`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28964: [SPARK-32144][SQL] Retain EXTERNAL property in hive table properties

2020-07-05 Thread GitBox
SparkQA commented on pull request #28964: URL: https://github.com/apache/spark/pull/28964#issuecomment-653931880 **[Test build #124937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124937/testReport)** for PR 28964 at commit [`30dc091`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28999: [MINOR][SQL] Re-use GetTimestamp in ParseToDate

2020-07-05 Thread GitBox
SparkQA commented on pull request #28999: URL: https://github.com/apache/spark/pull/28999#issuecomment-653931839 **[Test build #124922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124922/testReport)** for PR 28999 at commit [`87a3752`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28970: [SPARK-31723][CORE][TEST] Reenable one test case in HistoryServerSuite

2020-07-05 Thread GitBox
SparkQA commented on pull request #28970: URL: https://github.com/apache/spark/pull/28970#issuecomment-653931884 **[Test build #124933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124933/testReport)** for PR 28970 at commit [`057d543`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28965: [SPARK-32124][CORE][FOLLOW-UP] Use the invalid value Int.MinValue to fill the map index when the event logs from the old Spark version

2020-07-05 Thread GitBox
SparkQA commented on pull request #28965: URL: https://github.com/apache/spark/pull/28965#issuecomment-653931889 **[Test build #124936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124936/testReport)** for PR 28965 at commit [`9dfa5c0`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-05 Thread GitBox
SparkQA commented on pull request #28998: URL: https://github.com/apache/spark/pull/28998#issuecomment-653931822 **[Test build #124923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124923/testReport)** for PR 28998 at commit [`e06fb0c`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28996: [SPARK-29358][SQL] Make unionByName optionally fill missing columns with nulls

2020-07-05 Thread GitBox
SparkQA commented on pull request #28996: URL: https://github.com/apache/spark/pull/28996#issuecomment-653931886 **[Test build #124924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124924/testReport)** for PR 28996 at commit [`6afb8e8`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28939: [SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster

2020-07-05 Thread GitBox
SparkQA commented on pull request #28939: URL: https://github.com/apache/spark/pull/28939#issuecomment-653931887 **[Test build #124941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124941/testReport)** for PR 28939 at commit [`21720ac`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28988: [SPARK-32163][SQL] Nested pruning should work even with cosmetic variations

2020-07-05 Thread GitBox
SparkQA commented on pull request #28988: URL: https://github.com/apache/spark/pull/28988#issuecomment-653931843 **[Test build #124928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124928/testReport)** for PR 28988 at commit [`04f6bb6`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28986: [SPARK-32160][CORE][PYSPARK] Disallow to create SparkContext in executors.

2020-07-05 Thread GitBox
SparkQA commented on pull request #28986: URL: https://github.com/apache/spark/pull/28986#issuecomment-653931885 **[Test build #124930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124930/testReport)** for PR 28986 at commit [`aeed617`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-05 Thread GitBox
SparkQA commented on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-653931824 **[Test build #124925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124925/testReport)** for PR 28993 at commit [`fd65dd4`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

2020-07-05 Thread GitBox
SparkQA commented on pull request #28983: URL: https://github.com/apache/spark/pull/28983#issuecomment-653931864 **[Test build #124931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124931/testReport)** for PR 28983 at commit [`1a501d9`](https://github.com

<    5   6   7   8   9   10   11   >