[GitHub] [spark] attilapiros commented on issue #24072: [SPARK-27112] : Create a resource ordering between threads to resolve the deadlocks encountered …

attilapiros commented on issue #24072: [SPARK-27112] : Create a resource ordering between threads to resolve the deadlocks encountered … URL: https://github.com/apache/spark/pull/24072#issuecomment-473326618 LGTM This is an

[GitHub] [spark] srowen commented on issue #24085: [SPARK-26555][SQL] make ScalaReflection subtype checking thread safe

srowen commented on issue #24085: [SPARK-26555][SQL] make ScalaReflection subtype checking thread safe URL: https://github.com/apache/spark/pull/24085#issuecomment-473336597 @cloud-fan yeah I think something else was fixed and this is a different issue of the same type:

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

dongjoon-hyun commented on a change in pull request #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#discussion_r266046156 ## File path:

[GitHub] [spark] s1ck commented on a change in pull request #24107: [SPARK-27174][SQL] Add support for casting integer types to binary

s1ck commented on a change in pull request #24107: [SPARK-27174][SQL] Add support for casting integer types to binary URL: https://github.com/apache/spark/pull/24107#discussion_r266046555 ## File path:

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

dongjoon-hyun commented on a change in pull request #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#discussion_r266046563 ## File path:

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

dongjoon-hyun commented on a change in pull request #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#discussion_r266049288 ## File path:

[GitHub] [spark] skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management

skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management URL: https://github.com/apache/spark/pull/23599#discussion_r266049806 ## File path:

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

dongjoon-hyun commented on a change in pull request #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#discussion_r266049288 ## File path:

[GitHub] [spark] skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management

skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management URL: https://github.com/apache/spark/pull/23599#discussion_r266051394 ## File path: docs/running-on-kubernetes.md ## @@ -403,6 +403,36 @@ RBAC authorization and

[GitHub] [spark] skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management

skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management URL: https://github.com/apache/spark/pull/23599#discussion_r266051653 ## File path: docs/running-on-kubernetes.md ## @@ -403,6 +403,36 @@ RBAC authorization and

[GitHub] [spark] skonto commented on issue #23514: [SPARK-24902][K8s] Add PV integration tests

skonto commented on issue #23514: [SPARK-24902][K8s] Add PV integration tests URL: https://github.com/apache/spark/pull/23514#issuecomment-473355130 @shaneknapp do the default ones exist: ``` val HOST_PATH = sys.env.getOrElse("PVC_TESTS_HOST_PATH", "/home/jenkins/src/spark/tmp")

[GitHub] [spark] skonto edited a comment on issue #23514: [SPARK-24902][K8s] Add PV integration tests

skonto edited a comment on issue #23514: [SPARK-24902][K8s] Add PV integration tests URL: https://github.com/apache/spark/pull/23514#issuecomment-473355130 @shaneknapp do the default ones exist (should I trigger the tests): ``` val HOST_PATH =

[GitHub] [spark] arunmahadevan edited a comment on issue #23749: [SPARK-26841][SQL] Kafka timestamp pushdown

arunmahadevan edited a comment on issue #23749: [SPARK-26841][SQL] Kafka timestamp pushdown URL: https://github.com/apache/spark/pull/23749#issuecomment-473358007 IMO, we could adopt the more generic solution. If its just timestamp based filtering of start and end offsets maybe the

[GitHub] [spark] arunmahadevan commented on issue #23749: [SPARK-26841][SQL] Kafka timestamp pushdown

arunmahadevan commented on issue #23749: [SPARK-26841][SQL] Kafka timestamp pushdown URL: https://github.com/apache/spark/pull/23749#issuecomment-473358007 IMO, we could adopt the more generic solution. If its just timestamp based filtering of start and end offsets maybe the timestamp

[GitHub] [spark] skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management

skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management URL: https://github.com/apache/spark/pull/23599#discussion_r266063065 ## File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ## @@ -96,20

[GitHub] [spark] AmplabJenkins commented on issue #24103: [SPARK-27141][YARN] Use ConfigEntry for hardcoded configs for Yarn

AmplabJenkins commented on issue #24103: [SPARK-27141][YARN] Use ConfigEntry for hardcoded configs for Yarn URL: https://github.com/apache/spark/pull/24103#issuecomment-473365869 Can one of the admins verify this patch? This

[GitHub] [spark] AmplabJenkins commented on issue #24109: Add "completedStages" metircs for namespace=appStatus

AmplabJenkins commented on issue #24109: Add "completedStages" metircs for namespace=appStatus URL: https://github.com/apache/spark/pull/24109#issuecomment-473365798 Can one of the admins verify this patch? This is an

[GitHub] [spark] AmplabJenkins commented on issue #24108: [MINOR][SQL]Fix incorrect comment of SessionCatalog.

AmplabJenkins commented on issue #24108: [MINOR][SQL]Fix incorrect comment of SessionCatalog. URL: https://github.com/apache/spark/pull/24108#issuecomment-473365811 Can one of the admins verify this patch? This is an

[GitHub] [spark] AmplabJenkins commented on issue #24102: [SPARK-27171][SQL] Support Full-Partitons scan in limit for the first time

AmplabJenkins commented on issue #24102: [SPARK-27171][SQL] Support Full-Partitons scan in limit for the first time URL: https://github.com/apache/spark/pull/24102#issuecomment-473365885 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary

AmplabJenkins commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary URL: https://github.com/apache/spark/pull/24107#issuecomment-473365826 Can one of the admins verify this patch? This

[GitHub] [spark] AmplabJenkins removed a comment on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables

AmplabJenkins removed a comment on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-473370955 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] s1ck commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary

s1ck commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary URL: https://github.com/apache/spark/pull/24107#issuecomment-473316243

[GitHub] [spark] gengliangwang commented on issue #24067: [SPARK-27135][WebUI]Add ToolTip support for overflow text

gengliangwang commented on issue #24067: [SPARK-27135][WebUI]Add ToolTip support for overflow text URL: https://github.com/apache/spark/pull/24067#issuecomment-473319640 @sandeep-katta I took a look into this. I found that there is already a double click event for showing the full

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0

dongjoon-hyun commented on a change in pull request #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0 URL: https://github.com/apache/spark/pull/24106#discussion_r266039749 ## File path: dev/test-dependencies.sh ## @@ -33,7 +33,7 @@

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

dongjoon-hyun commented on a change in pull request #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#discussion_r266049288 ## File path:

[GitHub] [spark] SparkQA commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

SparkQA commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-473367945 **[Test build #103544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103544/testReport)** for PR

[GitHub] [spark] AmplabJenkins removed a comment on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table

AmplabJenkins removed a comment on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table URL: https://github.com/apache/spark/pull/24105#issuecomment-473368458 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins commented on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table

AmplabJenkins commented on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table URL: https://github.com/apache/spark/pull/24105#issuecomment-473368465 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table

SparkQA commented on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table URL: https://github.com/apache/spark/pull/24105#issuecomment-473368438 **[Test build #103530 has

[GitHub] [spark] AmplabJenkins commented on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table

AmplabJenkins commented on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table URL: https://github.com/apache/spark/pull/24105#issuecomment-473368458 Merged build finished. Test FAILed. This is

[GitHub] [spark] SparkQA removed a comment on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table

SparkQA removed a comment on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table URL: https://github.com/apache/spark/pull/24105#issuecomment-473367727 **[Test build #103530 has

[GitHub] [spark] cloud-fan commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary

cloud-fan commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary URL: https://github.com/apache/spark/pull/24107#issuecomment-473328705 Then I'm ok with supporting it. This is an

[GitHub] [spark] xuanyuanking opened a new pull request #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking opened a new pull request #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110 ## What changes were proposed in this pull request? This is a follow-up work for #22112's

[GitHub] [spark] skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management

skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management URL: https://github.com/apache/spark/pull/23599#discussion_r266050139 ## File path:

[GitHub] [spark] skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management

skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management URL: https://github.com/apache/spark/pull/23599#discussion_r266053512 ## File path: docs/running-on-kubernetes.md ## @@ -1018,6 +1047,13 @@ See the

[GitHub] [spark] skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management

skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management URL: https://github.com/apache/spark/pull/23599#discussion_r266050139 ## File path:

[GitHub] [spark] skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management

skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management URL: https://github.com/apache/spark/pull/23599#discussion_r266054978 ## File path:

[GitHub] [spark] xuanyuanking commented on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking commented on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-473352597 Test this please. Is Jenkins sleeping? :)

[GitHub] [spark] AmplabJenkins removed a comment on issue #24103: [SPARK-27141][YARN] Use ConfigEntry for hardcoded configs for Yarn

AmplabJenkins removed a comment on issue #24103: [SPARK-27141][YARN] Use ConfigEntry for hardcoded configs for Yarn URL: https://github.com/apache/spark/pull/24103#issuecomment-473366407 Can one of the admins verify this patch?

[GitHub] [spark] SparkQA commented on issue #24093: [SPARK-27161][SQL] improve the document of SQL keywords

SparkQA commented on issue #24093: [SPARK-27161][SQL] improve the document of SQL keywords URL: https://github.com/apache/spark/pull/24093#issuecomment-473367780 **[Test build #103534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103534/testReport)** for

[GitHub] [spark] AmplabJenkins removed a comment on issue #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0

AmplabJenkins removed a comment on issue #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0 URL: https://github.com/apache/spark/pull/24106#issuecomment-473366596 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #24103: [SPARK-27141][YARN] Use ConfigEntry for hardcoded configs for Yarn

AmplabJenkins commented on issue #24103: [SPARK-27141][YARN] Use ConfigEntry for hardcoded configs for Yarn URL: https://github.com/apache/spark/pull/24103#issuecomment-473367609 Can one of the admins verify this patch? This

[GitHub] [spark] AmplabJenkins removed a comment on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary

AmplabJenkins removed a comment on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary URL: https://github.com/apache/spark/pull/24107#issuecomment-473366381 Can one of the admins verify this patch?

[GitHub] [spark] SparkQA commented on issue #20303: [SPARK-23128][SQL] A new approach to do adaptive execution in Spark SQL

SparkQA commented on issue #20303: [SPARK-23128][SQL] A new approach to do adaptive execution in Spark SQL URL: https://github.com/apache/spark/pull/20303#issuecomment-473367983 **[Test build #103545 has

[GitHub] [spark] SparkQA commented on issue #24101: [CORE][MINOR] Correct the comment to show heartbeat interval is configurable

SparkQA commented on issue #24101: [CORE][MINOR] Correct the comment to show heartbeat interval is configurable URL: https://github.com/apache/spark/pull/24101#issuecomment-473367753 **[Test build #103531 has

[GitHub] [spark] SparkQA commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

SparkQA commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-473367864 **[Test build #103540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103540/testReport)**

[GitHub] [spark] AmplabJenkins removed a comment on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table

AmplabJenkins removed a comment on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table URL: https://github.com/apache/spark/pull/24105#issuecomment-473366609 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24109: Add "completedStages" metircs for namespace=appStatus

AmplabJenkins commented on issue #24109: Add "completedStages" metircs for namespace=appStatus URL: https://github.com/apache/spark/pull/24109#issuecomment-473367555 Can one of the admins verify this patch? This is an

[GitHub] [spark] SparkQA commented on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables

SparkQA commented on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-473367887 **[Test build #103538 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24108: [MINOR][SQL]Fix incorrect comment of SessionCatalog.

AmplabJenkins removed a comment on issue #24108: [MINOR][SQL]Fix incorrect comment of SessionCatalog. URL: https://github.com/apache/spark/pull/24108#issuecomment-473366370 Can one of the admins verify this patch? This is an

[GitHub] [spark] AmplabJenkins commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary

AmplabJenkins commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary URL: https://github.com/apache/spark/pull/24107#issuecomment-473367589 Can one of the admins verify this patch? This

[GitHub] [spark] SparkQA commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

SparkQA commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-473367903 **[Test build #103541 has

[GitHub] [spark] SparkQA commented on issue #24094: [SPARK-27162][SQL] Add new method getOriginalMap in CaseInsensitiveStringMap

SparkQA commented on issue #24094: [SPARK-27162][SQL] Add new method getOriginalMap in CaseInsensitiveStringMap URL: https://github.com/apache/spark/pull/24094#issuecomment-473367770 **[Test build #103533 has

[GitHub] [spark] SparkQA commented on issue #24088: [SPARK-27122][core] Jetty classes must not be return via getters in org.apache.spark.ui.WebUI

SparkQA commented on issue #24088: [SPARK-27122][core] Jetty classes must not be return via getters in org.apache.spark.ui.WebUI URL: https://github.com/apache/spark/pull/24088#issuecomment-473367779 **[Test build #103535 has

[GitHub] [spark] SparkQA commented on issue #24087: [SPARK-27096][SQL][FOLLOWUP] Do the correct validation of join types in R side and fix join docs for scala, python and r

SparkQA commented on issue #24087: [SPARK-27096][SQL][FOLLOWUP] Do the correct validation of join types in R side and fix join docs for scala, python and r URL: https://github.com/apache/spark/pull/24087#issuecomment-473367808 **[Test build #103536 has

[GitHub] [spark] SparkQA commented on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

SparkQA commented on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-473367673 **[Test build #103528 has

[GitHub] [spark] SparkQA commented on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

SparkQA commented on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#issuecomment-473367885 **[Test build #103539 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24102: [SPARK-27171][SQL] Support Full-Partitons scan in limit for the first time

AmplabJenkins removed a comment on issue #24102: [SPARK-27171][SQL] Support Full-Partitons scan in limit for the first time URL: https://github.com/apache/spark/pull/24102#issuecomment-473366414 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins removed a comment on issue #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0

AmplabJenkins removed a comment on issue #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0 URL: https://github.com/apache/spark/pull/24106#issuecomment-473366600 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table

SparkQA commented on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table URL: https://github.com/apache/spark/pull/24105#issuecomment-473367727 **[Test build #103530 has

[GitHub] [spark] SparkQA commented on issue #23788: [SPARK-27176][SQL] Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4

SparkQA commented on issue #23788: [SPARK-27176][SQL] Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4 URL: https://github.com/apache/spark/pull/23788#issuecomment-473367862 **[Test build #103543 has

[GitHub] [spark] SparkQA commented on issue #24073: [SPARK-27134][SQL] array_distinct function does not work correctly with columns containing array of array

SparkQA commented on issue #24073: [SPARK-27134][SQL] array_distinct function does not work correctly with columns containing array of array URL: https://github.com/apache/spark/pull/24073#issuecomment-473367833 **[Test build #103537 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table

AmplabJenkins removed a comment on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table URL: https://github.com/apache/spark/pull/24105#issuecomment-473366599 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0

SparkQA commented on issue #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0 URL: https://github.com/apache/spark/pull/24106#issuecomment-473367722 **[Test build #103529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103529/testReport)** for PR 24106

[GitHub] [spark] SparkQA commented on issue #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntegralColumn

SparkQA commented on issue #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntegralColumn URL: https://github.com/apache/spark/pull/23942#issuecomment-473367879 **[Test build #103542 has

[GitHub] [spark] SparkQA commented on issue #24100: [SPARK-27164] RDD.countApprox on empty RDDs schedules jobs which never complete

SparkQA commented on issue #24100: [SPARK-27164] RDD.countApprox on empty RDDs schedules jobs which never complete URL: https://github.com/apache/spark/pull/24100#issuecomment-473367789 **[Test build #103532 has

[GitHub] [spark] AmplabJenkins commented on issue #24102: [SPARK-27171][SQL] Support Full-Partitons scan in limit for the first time

AmplabJenkins commented on issue #24102: [SPARK-27171][SQL] Support Full-Partitons scan in limit for the first time URL: https://github.com/apache/spark/pull/24102#issuecomment-473367624 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins removed a comment on issue #24109: Add "completedStages" metircs for namespace=appStatus

AmplabJenkins removed a comment on issue #24109: Add "completedStages" metircs for namespace=appStatus URL: https://github.com/apache/spark/pull/24109#issuecomment-473366358 Can one of the admins verify this patch? This is

[GitHub] [spark] AmplabJenkins removed a comment on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

AmplabJenkins removed a comment on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-473367116 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #20303: [SPARK-23128][SQL] A new approach to do adaptive execution in Spark SQL

AmplabJenkins removed a comment on issue #20303: [SPARK-23128][SQL] A new approach to do adaptive execution in Spark SQL URL: https://github.com/apache/spark/pull/20303#issuecomment-473367234 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24108: [MINOR][SQL]Fix incorrect comment of SessionCatalog.

AmplabJenkins commented on issue #24108: [MINOR][SQL]Fix incorrect comment of SessionCatalog. URL: https://github.com/apache/spark/pull/24108#issuecomment-473367575 Can one of the admins verify this patch? This is an

[GitHub] [spark] AmplabJenkins removed a comment on issue #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntegralColumn

AmplabJenkins removed a comment on issue #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntegralColumn URL: https://github.com/apache/spark/pull/23942#issuecomment-473367087 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #23788: [SPARK-27176][SQL] Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4

AmplabJenkins removed a comment on issue #23788: [SPARK-27176][SQL] Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4 URL: https://github.com/apache/spark/pull/23788#issuecomment-473367187 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-473367184 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins removed a comment on issue #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntegralColumn

AmplabJenkins removed a comment on issue #23942: [SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntegralColumn URL: https://github.com/apache/spark/pull/23942#issuecomment-473367078 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

AmplabJenkins removed a comment on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-473367127 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables

AmplabJenkins commented on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-473370944 Merged build finished. Test FAILed. This is an

[GitHub] [spark] SparkQA commented on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables

SparkQA commented on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-473370926 **[Test build #103538 has

[GitHub] [spark] cloud-fan commented on a change in pull request #24012: [SPARK-26811][SQL] Add capabilities to v2.Table

cloud-fan commented on a change in pull request #24012: [SPARK-26811][SQL] Add capabilities to v2.Table URL: https://github.com/apache/spark/pull/24012#discussion_r266015572 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/FileTable.scala

[GitHub] [spark] Nimfadora commented on a change in pull request #23791: [SPARK-20597][SQL][SS][WIP] KafkaSourceProvider falls back on path as synonym for topic

Nimfadora commented on a change in pull request #23791: [SPARK-20597][SQL][SS][WIP] KafkaSourceProvider falls back on path as synonym for topic URL: https://github.com/apache/spark/pull/23791#discussion_r266027495 ## File path: docs/structured-streaming-kafka-integration.md

[GitHub] [spark] bdrillard commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro

bdrillard commented on issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878#issuecomment-473334726 @HyukjinKwon, to answer your question, when I mentioned access to "broader APIs", I meant 1. specifically the typed "[Java

[GitHub] [spark] skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management

skonto commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management URL: https://github.com/apache/spark/pull/23599#discussion_r266051969 ## File path: docs/running-on-kubernetes.md ## @@ -403,6 +403,36 @@ RBAC authorization and

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0

dongjoon-hyun commented on a change in pull request #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0 URL: https://github.com/apache/spark/pull/24106#discussion_r266057173 ## File path: dev/test-dependencies.sh ## @@ -33,7 +33,7 @@

[GitHub] [spark] srowen commented on a change in pull request #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0

srowen commented on a change in pull request #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0 URL: https://github.com/apache/spark/pull/24106#discussion_r266062296 ## File path: dev/test-dependencies.sh ## @@ -33,7 +33,7 @@

[GitHub] [spark] AmplabJenkins commented on issue #24109: Add "completedStages" metircs for namespace=appStatus

AmplabJenkins commented on issue #24109: Add "completedStages" metircs for namespace=appStatus URL: https://github.com/apache/spark/pull/24109#issuecomment-473366358 Can one of the admins verify this patch? This is an

[GitHub] [spark] AmplabJenkins commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary

AmplabJenkins commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary URL: https://github.com/apache/spark/pull/24107#issuecomment-473366381 Can one of the admins verify this patch? This

[GitHub] [spark] AmplabJenkins commented on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

AmplabJenkins commented on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-473366521 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24101: [CORE][MINOR] Correct the comment to show heartbeat interval is configurable

AmplabJenkins commented on issue #24101: [CORE][MINOR] Correct the comment to show heartbeat interval is configurable URL: https://github.com/apache/spark/pull/24101#issuecomment-473366587 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0

AmplabJenkins commented on issue #24106: [SPARK-27175][BUILD] Upgrade hadoop-3 to 3.2.0 URL: https://github.com/apache/spark/pull/24106#issuecomment-473366596 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] AmplabJenkins commented on issue #24102: [SPARK-27171][SQL] Support Full-Partitons scan in limit for the first time

AmplabJenkins commented on issue #24102: [SPARK-27171][SQL] Support Full-Partitons scan in limit for the first time URL: https://github.com/apache/spark/pull/24102#issuecomment-473366414 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins commented on issue #24103: [SPARK-27141][YARN] Use ConfigEntry for hardcoded configs for Yarn

AmplabJenkins commented on issue #24103: [SPARK-27141][YARN] Use ConfigEntry for hardcoded configs for Yarn URL: https://github.com/apache/spark/pull/24103#issuecomment-473366407 Can one of the admins verify this patch? This

[GitHub] [spark] AmplabJenkins commented on issue #24108: [MINOR][SQL]Fix incorrect comment of SessionCatalog.

AmplabJenkins commented on issue #24108: [MINOR][SQL]Fix incorrect comment of SessionCatalog. URL: https://github.com/apache/spark/pull/24108#issuecomment-473366370 Can one of the admins verify this patch? This is an

[GitHub] [spark] AmplabJenkins commented on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table

AmplabJenkins commented on issue #24105: [SPARK-27173][SQL]Remove （lz4，brotli，zstd） for hive parquet table URL: https://github.com/apache/spark/pull/24105#issuecomment-473366609 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

AmplabJenkins commented on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-473366527 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] xuanyuanking commented on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking commented on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-473368995 cc @cloud-fan @gatorsmile.

[GitHub] [spark] SparkQA removed a comment on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables

SparkQA removed a comment on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-473367887 **[Test build #103538 has

[GitHub] [spark] xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking edited a comment on issue #24110: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24110#issuecomment-473352597 Test this please.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables

AmplabJenkins removed a comment on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-473370944 Merged build finished. Test FAILed. This

[GitHub] [spark] AmplabJenkins commented on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables

AmplabJenkins commented on issue #24047: [SPARK-25196][SQL] Extends Analyze commands for cached tables URL: https://github.com/apache/spark/pull/24047#issuecomment-473370955 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] cloud-fan commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary

cloud-fan commented on issue #24107: [SPARK-27174][SQL] Add support for casting integer types to binary URL: https://github.com/apache/spark/pull/24107#issuecomment-473319954 what's the current behavior in Spark when casting int to binary? fail or return null?

  1   2   3   4   5   6   7   8   >