[GitHub] SparkQA commented on issue #23555: [SPARK-26625] Add oauthToken to spark.kubernetes.authenticate.submission.oauthToken

SparkQA commented on issue #23555: [SPARK-26625] Add oauthToken to spark.kubernetes.authenticate.submission.oauthToken URL: https://github.com/apache/spark/pull/23555#issuecomment-454559944 Kubernetes integration test status success URL:

[GitHub] srowen commented on issue #17726: [SPARK-17928] [Mesos] No driver.memoryOverhead setting for mesos cluster mode

srowen commented on issue #17726: [SPARK-17928] [Mesos] No driver.memoryOverhead setting for mesos cluster mode URL: https://github.com/apache/spark/pull/17726#issuecomment-454563859 Merged to master This is an automated

[GitHub] SparkQA commented on issue #23518: [SPARK-26600] Update spark-submit usage message

SparkQA commented on issue #23518: [SPARK-26600] Update spark-submit usage message URL: https://github.com/apache/spark/pull/23518#issuecomment-454564171 **[Test build #4518 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4518/testReport)** for PR 23518

[GitHub] SparkQA commented on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350

SparkQA commented on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350 URL: https://github.com/apache/spark/pull/23544#issuecomment-454563861 **[Test build #101282 has

[GitHub] asfgit closed pull request #23208: [SPARK-25530][SQL] data source v2 API refactor (batch write)

asfgit closed pull request #23208: [SPARK-25530][SQL] data source v2 API refactor (batch write) URL: https://github.com/apache/spark/pull/23208 As this is a foreign pull request (from a fork), the diff has been sent to your commit mailing list, comm...@spark.apache.org

[GitHub] AmplabJenkins commented on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4

AmplabJenkins commented on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4 URL: https://github.com/apache/spark/pull/23553#issuecomment-454574872 Merged build finished. Test FAILed. This is an automated

[GitHub] SparkQA commented on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4

SparkQA commented on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4 URL: https://github.com/apache/spark/pull/23553#issuecomment-454574540 **[Test build #101271 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101271/testReport)** for PR

[GitHub] dbtsai commented on a change in pull request #23542: [WIP] [SPARK-25603][SQL] Pushing Down Nested Field projections

dbtsai commented on a change in pull request #23542: [WIP] [SPARK-25603][SQL] Pushing Down Nested Field projections URL: https://github.com/apache/spark/pull/23542#discussion_r248095908 ## File path:

[GitHub] SparkQA commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

SparkQA commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454591518 **[Test build #101275 has

[GitHub] SparkQA removed a comment on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

SparkQA removed a comment on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454509760 **[Test build #101275 has

[GitHub] vinooganesh commented on issue #23556: [SPARK-26626][SQL] Maximum size for repeatedly substituted aliases in SQL expressions

vinooganesh commented on issue #23556: [SPARK-26626][SQL] Maximum size for repeatedly substituted aliases in SQL expressions URL: https://github.com/apache/spark/pull/23556#issuecomment-454593819 @cloud-fan could we get you to take a look here? should hopefully be a quick review

[GitHub] dbtsai commented on a change in pull request #23542: [WIP] [SPARK-25603][SQL] Pushing Down Nested Field projections

dbtsai commented on a change in pull request #23542: [WIP] [SPARK-25603][SQL] Pushing Down Nested Field projections URL: https://github.com/apache/spark/pull/23542#discussion_r248107850 ## File path:

[GitHub] skonto commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS

skonto commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS URL: https://github.com/apache/spark/pull/23546#discussion_r248055353 ## File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ## @@ -330,6

[GitHub] AmplabJenkins commented on issue #19616: [SPARK-22404][YARN] Provide an option to use unmanaged AM in yarn-client mode

AmplabJenkins commented on issue #19616: [SPARK-22404][YARN] Provide an option to use unmanaged AM in yarn-client mode URL: https://github.com/apache/spark/pull/19616#issuecomment-454543430 Merged build finished. Test PASSed.

[GitHub] SparkQA commented on issue #19616: [SPARK-22404][YARN] Provide an option to use unmanaged AM in yarn-client mode

SparkQA commented on issue #19616: [SPARK-22404][YARN] Provide an option to use unmanaged AM in yarn-client mode URL: https://github.com/apache/spark/pull/19616#issuecomment-454543335 **[Test build #101277 has

[GitHub] AmplabJenkins removed a comment on issue #19616: [SPARK-22404][YARN] Provide an option to use unmanaged AM in yarn-client mode

AmplabJenkins removed a comment on issue #19616: [SPARK-22404][YARN] Provide an option to use unmanaged AM in yarn-client mode URL: https://github.com/apache/spark/pull/19616#issuecomment-454543430 Merged build finished. Test PASSed.

[GitHub] SparkQA removed a comment on issue #19616: [SPARK-22404][YARN] Provide an option to use unmanaged AM in yarn-client mode

SparkQA removed a comment on issue #19616: [SPARK-22404][YARN] Provide an option to use unmanaged AM in yarn-client mode URL: https://github.com/apache/spark/pull/19616#issuecomment-454537641 **[Test build #101277 has

[GitHub] gatorsmile commented on a change in pull request #23534: [SPARK-26610][PYTHON] Fix inconsistency between toJSON Method in Python and Scala.

gatorsmile commented on a change in pull request #23534: [SPARK-26610][PYTHON] Fix inconsistency between toJSON Method in Python and Scala. URL: https://github.com/apache/spark/pull/23534#discussion_r248056739 ## File path: python/pyspark/sql/dataframe.py ## @@ -109,15

[GitHub] pgandhi999 commented on a change in pull request #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

pgandhi999 commented on a change in pull request #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#discussion_r248059479 ## File path:

[GitHub] AmplabJenkins commented on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests

AmplabJenkins commented on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests URL: https://github.com/apache/spark/pull/23540#issuecomment-454548511 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins commented on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests

AmplabJenkins commented on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests URL: https://github.com/apache/spark/pull/23540#issuecomment-454548504 Merged build finished. Test PASSed.

[GitHub] SparkQA commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

SparkQA commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454548862 **[Test build #101278 has

[GitHub] AmplabJenkins removed a comment on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests

AmplabJenkins removed a comment on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests URL: https://github.com/apache/spark/pull/23540#issuecomment-454548504 Merged build finished. Test PASSed.

[GitHub] AmplabJenkins removed a comment on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests

AmplabJenkins removed a comment on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests URL: https://github.com/apache/spark/pull/23540#issuecomment-454548511 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] SparkQA removed a comment on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests

SparkQA removed a comment on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests URL: https://github.com/apache/spark/pull/23540#issuecomment-454449892 **[Test build #101270 has

[GitHub] erikerlandson commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS

erikerlandson commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS URL: https://github.com/apache/spark/pull/23546#discussion_r248062240 ## File path: docs/running-on-kubernetes.md ## @@ -400,7 +415,6 @@ There are

[GitHub] SparkQA commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

SparkQA commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454551185 **[Test build #101273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101273/testReport)** for PR 23551

[GitHub] AmplabJenkins removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

AmplabJenkins removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454551513 Merged build finished. Test FAILed. This is an automated

[GitHub] AmplabJenkins commented on issue #22502: [SPARK-25474][SQL]When the "fallBackToHdfsForStats= true", Size in bytes is coming as default size in bytes ( 8.0 EB)

AmplabJenkins commented on issue #22502: [SPARK-25474][SQL]When the "fallBackToHdfsForStats= true", Size in bytes is coming as default size in bytes ( 8.0 EB) URL: https://github.com/apache/spark/pull/22502#issuecomment-454551265 Merged build finished. Test FAILed.

[GitHub] SparkQA removed a comment on issue #22502: [SPARK-25474][SQL]When the "fallBackToHdfsForStats= true", Size in bytes is coming as default size in bytes ( 8.0 EB)

SparkQA removed a comment on issue #22502: [SPARK-25474][SQL]When the "fallBackToHdfsForStats= true", Size in bytes is coming as default size in bytes ( 8.0 EB) URL: https://github.com/apache/spark/pull/22502#issuecomment-454509781 **[Test build #101276 has

[GitHub] AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454551513 Merged build finished. Test FAILed. This is an automated message

[GitHub] AmplabJenkins commented on issue #22502: [SPARK-25474][SQL]When the "fallBackToHdfsForStats= true", Size in bytes is coming as default size in bytes ( 8.0 EB)

AmplabJenkins commented on issue #22502: [SPARK-25474][SQL]When the "fallBackToHdfsForStats= true", Size in bytes is coming as default size in bytes ( 8.0 EB) URL: https://github.com/apache/spark/pull/22502#issuecomment-454551271 Test FAILed. Refer to this link for build results

[GitHub] AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454551522 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] SparkQA removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

SparkQA removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454475121 **[Test build #101273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101273/testReport)** for PR

[GitHub] mccheah commented on issue #23556: [SPARK-26626][SQL] Maximum size for repeatedly substituted aliases in SQL expressions

mccheah commented on issue #23556: [SPARK-26626][SQL] Maximum size for repeatedly substituted aliases in SQL expressions URL: https://github.com/apache/spark/pull/23556#issuecomment-454554214 ok to test This is an automated

[GitHub] AmplabJenkins commented on issue #23556: [SPARK-26626][SQL] Maximum size for repeatedly substituted aliases in SQL expressions

AmplabJenkins commented on issue #23556: [SPARK-26626][SQL] Maximum size for repeatedly substituted aliases in SQL expressions URL: https://github.com/apache/spark/pull/23556#issuecomment-454556776 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] SparkQA commented on issue #23555: [SPARK-26625] Add oauthToken to spark.kubernetes.authenticate.submission.oauthToken

SparkQA commented on issue #23555: [SPARK-26625] Add oauthToken to spark.kubernetes.authenticate.submission.oauthToken URL: https://github.com/apache/spark/pull/23555#issuecomment-454556340 Kubernetes integration test starting URL:

[GitHub] skonto commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS

skonto commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS URL: https://github.com/apache/spark/pull/23546#discussion_r248068731 ## File path: docs/running-on-kubernetes.md ## @@ -400,7 +415,6 @@ There are several Spark

[GitHub] gatorsmile commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

gatorsmile commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454561158 @maropu We should improve our doc and document all these SQL metrics; otherwise this will be very hard for our end users to understand

[GitHub] erikerlandson commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS

erikerlandson commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS URL: https://github.com/apache/spark/pull/23546#discussion_r248073951 ## File path: docs/running-on-kubernetes.md ## @@ -400,7 +415,6 @@ There are

[GitHub] AmplabJenkins removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

AmplabJenkins removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454562484 Merged build finished. Test PASSed. This is an automated

[GitHub] AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454562484 Merged build finished. Test PASSed. This is an automated message

[GitHub] AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454562490 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] srowen closed pull request #17726: [SPARK-17928] [Mesos] No driver.memoryOverhead setting for mesos cluster mode

srowen closed pull request #17726: [SPARK-17928] [Mesos] No driver.memoryOverhead setting for mesos cluster mode URL: https://github.com/apache/spark/pull/17726 As this is a foreign pull request (from a fork), the diff has been sent to your commit mailing list, comm...@spark.apache.org

[GitHub] skonto commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS

skonto commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS URL: https://github.com/apache/spark/pull/23546#discussion_r248079323 ## File path: docs/running-on-kubernetes.md ## @@ -400,7 +415,6 @@ There are several Spark

[GitHub] SparkQA removed a comment on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4

SparkQA removed a comment on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4 URL: https://github.com/apache/spark/pull/23553#issuecomment-454459128 **[Test build #101271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101271/testReport)**

[GitHub] AmplabJenkins commented on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4

AmplabJenkins commented on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4 URL: https://github.com/apache/spark/pull/23553#issuecomment-454574879 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins removed a comment on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4

AmplabJenkins removed a comment on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4 URL: https://github.com/apache/spark/pull/23553#issuecomment-454574879 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins removed a comment on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4

AmplabJenkins removed a comment on issue #23553: [SPARK-23710][SQL] Upgrade built-in Hive to 2.3.4 URL: https://github.com/apache/spark/pull/23553#issuecomment-454574872 Merged build finished. Test FAILed. This is an

[GitHub] erikerlandson commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS

erikerlandson commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS URL: https://github.com/apache/spark/pull/23546#discussion_r248087151 ## File path: docs/running-on-kubernetes.md ## @@ -400,7 +415,6 @@ There are

[GitHub] AmplabJenkins commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

AmplabJenkins commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454592021 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] AmplabJenkins removed a comment on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

AmplabJenkins removed a comment on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454592015 Merged build finished. Test PASSed.

[GitHub] AmplabJenkins removed a comment on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

AmplabJenkins removed a comment on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454592021 Test PASSed. Refer to this link for build results (access rights

[GitHub] SparkQA commented on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests

SparkQA commented on issue #23540: [SPARK-26615][Core] Fixing transport server/client resource leaks in the core unittests URL: https://github.com/apache/spark/pull/23540#issuecomment-454547845 **[Test build #101270 has

[GitHub] srowen commented on a change in pull request #23534: [SPARK-26610][PYTHON] Fix inconsistency between toJSON Method in Python and Scala.

srowen commented on a change in pull request #23534: [SPARK-26610][PYTHON] Fix inconsistency between toJSON Method in Python and Scala. URL: https://github.com/apache/spark/pull/23534#discussion_r248058553 ## File path: python/pyspark/sql/dataframe.py ## @@ -109,15

[GitHub] SparkQA commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

SparkQA commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454562038 **[Test build #101281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101281/testReport)** for PR 23551 at

[GitHub] skonto commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS

skonto commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS URL: https://github.com/apache/spark/pull/23546#discussion_r248079323 ## File path: docs/running-on-kubernetes.md ## @@ -400,7 +415,6 @@ There are several Spark

[GitHub] AmplabJenkins removed a comment on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350

AmplabJenkins removed a comment on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350 URL: https://github.com/apache/spark/pull/23544#issuecomment-454571320 Merged build finished. Test PASSed.

[GitHub] SparkQA removed a comment on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350

SparkQA removed a comment on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350 URL: https://github.com/apache/spark/pull/23544#issuecomment-454563861 **[Test build #101282 has

[GitHub] AmplabJenkins commented on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350

AmplabJenkins commented on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350 URL: https://github.com/apache/spark/pull/23544#issuecomment-454571320 Merged build finished. Test PASSed.

[GitHub] AmplabJenkins commented on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350

AmplabJenkins commented on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350 URL: https://github.com/apache/spark/pull/23544#issuecomment-454571324 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] SparkQA commented on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350

SparkQA commented on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350 URL: https://github.com/apache/spark/pull/23544#issuecomment-454571233 **[Test build #101282 has

[GitHub] zsxwing commented on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350

zsxwing commented on issue #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350 URL: https://github.com/apache/spark/pull/23544#issuecomment-454574021 LGTM. Merging to master. Thanks!

[GitHub] AmplabJenkins commented on issue #23555: [SPARK-26625] Add oauthToken to spark.kubernetes.authenticate.submission.oauthToken

AmplabJenkins commented on issue #23555: [SPARK-26625] Add oauthToken to spark.kubernetes.authenticate.submission.oauthToken URL: https://github.com/apache/spark/pull/23555#issuecomment-454583088 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

AmplabJenkins commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454592015 Merged build finished. Test PASSed.

[GitHub] eatoncys commented on issue #23010: [SPARK-26012][SQL]Null and '' values should not cause dynamic partition failure of string types

eatoncys commented on issue #23010: [SPARK-26012][SQL]Null and '' values should not cause dynamic partition failure of string types URL: https://github.com/apache/spark/pull/23010#issuecomment-454605829 @cloud-fan Would you like to review it again, thanks.

[GitHub] HyukjinKwon commented on issue #18139: [SPARK-20787][PYTHON] PySpark can't handle datetimes before 1900

HyukjinKwon commented on issue #18139: [SPARK-20787][PYTHON] PySpark can't handle datetimes before 1900 URL: https://github.com/apache/spark/pull/18139#issuecomment-454613321 There're other possibilities like using pyarrow for it or pulling out one library dependent on pyarrow since we

[GitHub] SparkQA commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

SparkQA commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454616533 **[Test build #101278 has

[GitHub] SparkQA removed a comment on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

SparkQA removed a comment on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454548862 **[Test build #101278 has

[GitHub] AmplabJenkins removed a comment on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

AmplabJenkins removed a comment on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454616965 Test PASSed. Refer to this link for build results (access rights

[GitHub] AmplabJenkins commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched

AmplabJenkins commented on issue #22806: [SPARK-25250][CORE] : Late zombie task completions handled correctly even before new taskset launched URL: https://github.com/apache/spark/pull/22806#issuecomment-454616965 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] SparkQA commented on issue #23555: [SPARK-26625] Add oauthToken to spark.redaction.regex

SparkQA commented on issue #23555: [SPARK-26625] Add oauthToken to spark.redaction.regex URL: https://github.com/apache/spark/pull/23555#issuecomment-454618518 **[Test build #101279 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101279/testReport)** for

[GitHub] SparkQA removed a comment on issue #23555: [SPARK-26625] Add oauthToken to spark.redaction.regex

SparkQA removed a comment on issue #23555: [SPARK-26625] Add oauthToken to spark.redaction.regex URL: https://github.com/apache/spark/pull/23555#issuecomment-454552750 **[Test build #101279 has

[GitHub] maropu commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

maropu commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454620896 @gatorsmile yea, I like that idea. ok, I'll do that. This is an automated

[GitHub] jerryshao commented on issue #23509: [SPARK-26588][CORE] Idle executor should properly be removed when no job is su…

jerryshao commented on issue #23509: [SPARK-26588][CORE] Idle executor should properly be removed when no job is su… URL: https://github.com/apache/spark/pull/23509#issuecomment-454622245 Jenkins, retest this please. This is

[GitHub] jerryshao commented on issue #23509: [SPARK-26588][CORE] Idle executor should properly be removed when no job is su…

jerryshao commented on issue #23509: [SPARK-26588][CORE] Idle executor should properly be removed when no job is su… URL: https://github.com/apache/spark/pull/23509#issuecomment-454622531 The change itself looks fine to me. @tgravescs @vanzin would you please also take a look.

[GitHub] SparkQA commented on issue #23509: [SPARK-26588][CORE] Idle executor should properly be removed when no job is su…

SparkQA commented on issue #23509: [SPARK-26588][CORE] Idle executor should properly be removed when no job is su… URL: https://github.com/apache/spark/pull/23509#issuecomment-454622577 **[Test build #101284 has

[GitHub] wangyum commented on issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hadoop 3 profile

wangyum commented on issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hadoop 3 profile URL: https://github.com/apache/spark/pull/21588#issuecomment-454622405 Thanks @HyukjinKwon I have posted an email:

[GitHub] AmplabJenkins removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

AmplabJenkins removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454626257 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454626257 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

AmplabJenkins removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454626253 Merged build finished. Test PASSed. This is an automated

[GitHub] AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

AmplabJenkins commented on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454626253 Merged build finished. Test PASSed. This is an automated message

[GitHub] SparkQA removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels

SparkQA removed a comment on issue #23551: [SPARK-26622][SQL] Revise SQL Metrics labels URL: https://github.com/apache/spark/pull/23551#issuecomment-454562038 **[Test build #101281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101281/testReport)** for PR

[GitHub] liupc commented on a change in pull request #23445: [SPARK-26530]Validate heartheat arguments in HeartbeatReceiver

liupc commented on a change in pull request #23445: [SPARK-26530]Validate heartheat arguments in HeartbeatReceiver URL: https://github.com/apache/spark/pull/23445#discussion_r248132674 ## File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ##

[GitHub] maropu commented on a change in pull request #23010: [SPARK-26012][SQL]Null and '' values should not cause dynamic partition failure of string types

maropu commented on a change in pull request #23010: [SPARK-26012][SQL]Null and '' values should not cause dynamic partition failure of string types URL: https://github.com/apache/spark/pull/23010#discussion_r248134049 ## File path:

[GitHub] AmplabJenkins commented on issue #23550: [SPARK-26621][CORE]Use ConfigEntry for hardcoded configs for shuffle categories.

AmplabJenkins commented on issue #23550: [SPARK-26621][CORE]Use ConfigEntry for hardcoded configs for shuffle categories. URL: https://github.com/apache/spark/pull/23550#issuecomment-454630457 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] SparkQA commented on issue #23550: [SPARK-26621][CORE]Use ConfigEntry for hardcoded configs for shuffle categories.

SparkQA commented on issue #23550: [SPARK-26621][CORE]Use ConfigEntry for hardcoded configs for shuffle categories. URL: https://github.com/apache/spark/pull/23550#issuecomment-454630143 **[Test build #101285 has

[GitHub] AmplabJenkins removed a comment on issue #23539: [SPARK-26617][SQL] Cache manager locks

AmplabJenkins removed a comment on issue #23539: [SPARK-26617][SQL] Cache manager locks URL: https://github.com/apache/spark/pull/23539#issuecomment-454103291 Can one of the admins verify this patch? This is an automated

[GitHub] cloud-fan closed pull request #23521: [SPARK-26604][CORE] Clean up channel registration for StreamManager

cloud-fan closed pull request #23521: [SPARK-26604][CORE] Clean up channel registration for StreamManager URL: https://github.com/apache/spark/pull/23521 As this is a foreign pull request (from a fork), the diff has been sent to your commit mailing list, comm...@spark.apache.org

[GitHub] cloud-fan commented on issue #23539: [SPARK-26617][SQL] Cache manager locks

cloud-fan commented on issue #23539: [SPARK-26617][SQL] Cache manager locks URL: https://github.com/apache/spark/pull/23539#issuecomment-454634124 ok to test This is an automated message from the Apache Git Service. To

[GitHub] SparkQA commented on issue #23539: [SPARK-26617][SQL] Cache manager locks

SparkQA commented on issue #23539: [SPARK-26617][SQL] Cache manager locks URL: https://github.com/apache/spark/pull/23539#issuecomment-454634353 **[Test build #101286 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101286/testReport)** for PR 23539 at

[GitHub] AmplabJenkins commented on issue #23557: [SPARK-26629] [SS] Fixed error with multiple file stream in a query + restart on a batch that has no data for one file stream

AmplabJenkins commented on issue #23557: [SPARK-26629] [SS] Fixed error with multiple file stream in a query + restart on a batch that has no data for one file stream URL: https://github.com/apache/spark/pull/23557#issuecomment-454636566 Test PASSed. Refer to this link for build

[GitHub] AmplabJenkins commented on issue #23557: [SPARK-26629] [SS] Fixed error with multiple file stream in a query + restart on a batch that has no data for one file stream

AmplabJenkins commented on issue #23557: [SPARK-26629] [SS] Fixed error with multiple file stream in a query + restart on a batch that has no data for one file stream URL: https://github.com/apache/spark/pull/23557#issuecomment-454636564 Merged build finished. Test PASSed.

[GitHub] HeartSaVioR commented on a change in pull request #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

HeartSaVioR commented on a change in pull request #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query URL: https://github.com/apache/spark/pull/22952#discussion_r248146740 ## File path:

[GitHub] wangshuo128 commented on issue #23355: [SPARK-26418][SHUFFLE] Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService

wangshuo128 commented on issue #23355: [SPARK-26418][SHUFFLE] Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService URL: https://github.com/apache/spark/pull/23355#issuecomment-454645241 @Ngone51 @cloud-fan Yes, found that this pr is same

[GitHub] HeartSaVioR commented on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

HeartSaVioR commented on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query URL: https://github.com/apache/spark/pull/22952#issuecomment-454645347 @zsxwing > I feel the current behavior of rename is a bit weird. Let's say the source dir

[GitHub] HyukjinKwon commented on issue #23263: [SPARK-23674][ML] Adds Spark ML Events to Instrumentation

HyukjinKwon commented on issue #23263: [SPARK-23674][ML] Adds Spark ML Events to Instrumentation URL: https://github.com/apache/spark/pull/23263#issuecomment-454646029 Hey all, do you mind if I ask to take another look? I think I addressed all comments, and the changes here are not

[GitHub] HeartSaVioR edited a comment on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

HeartSaVioR edited a comment on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query URL: https://github.com/apache/spark/pull/22952#issuecomment-454645347 @zsxwing > I feel the current behavior of rename is a bit weird. Let's say the source

[GitHub] SparkQA commented on issue #23556: [SPARK-26626][SQL] Maximum size for repeatedly substituted aliases in SQL expressions

SparkQA commented on issue #23556: [SPARK-26626][SQL] Maximum size for repeatedly substituted aliases in SQL expressions URL: https://github.com/apache/spark/pull/23556#issuecomment-454610469 **[Test build #101280 has

[GitHub] zsxwing commented on a change in pull request #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

zsxwing commented on a change in pull request #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query URL: https://github.com/apache/spark/pull/22952#discussion_r238901099 ## File path:

<    1   2   3   4   5   6   7   8   9   10   >