[jira] [Updated] (SPARK-40591) ignoreCorruptFiles results data loss

2022-09-27 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-40591: - Attachment: image-2022-09-28-09-20-21-693.png > ignoreCorruptFiles results data loss >

[jira] [Created] (SPARK-40591) ignoreCorruptFiles results data loss

2022-09-27 Thread Kent Yao (Jira)
Kent Yao created SPARK-40591: Summary: ignoreCorruptFiles results data loss Key: SPARK-40591 URL: https://issues.apache.org/jira/browse/SPARK-40591 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-39960) Upgrade mysql-connector-java to 8.0.30

2022-08-03 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-39960. -- Resolution: Fixed Issue resolved by pull request 37387 [https://github.com/apache/spark/pull/37387]

[jira] [Assigned] (SPARK-39960) Upgrade mysql-connector-java to 8.0.30

2022-08-03 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-39960: Assignee: BingKun Pan > Upgrade mysql-connector-java to 8.0.30 >

[jira] [Created] (SPARK-39930) Cache Hints

2022-08-01 Thread Kent Yao (Jira)
Kent Yao created SPARK-39930: Summary: Cache Hints Key: SPARK-39930 URL: https://issues.apache.org/jira/browse/SPARK-39930 Project: Spark Issue Type: New Feature Components: SQL

[jira] [Assigned] (SPARK-37825) Make spark beeline be able to handle javaOpts

2022-05-26 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-37825: Assignee: Kent Yao > Make spark beeline be able to handle javaOpts >

[jira] [Resolved] (SPARK-37825) Make spark beeline be able to handle javaOpts

2022-05-26 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-37825. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35116

[jira] [Assigned] (SPARK-39254) Upgrade ZSTD-JNI to 1.5.2-3

2022-05-26 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-39254: Assignee: Yang Jie > Upgrade ZSTD-JNI to 1.5.2-3 > > >

[jira] [Resolved] (SPARK-39254) Upgrade ZSTD-JNI to 1.5.2-3

2022-05-26 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-39254. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36634

[jira] [Resolved] (SPARK-39246) Implement Groupby.skew

2022-05-23 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-39246. -- Fix Version/s: 3.4.0 Assignee: zhengruifeng Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-39240) Source and binary releases using different tool to generates hashes for integrity

2022-05-20 Thread Kent Yao (Jira)
Kent Yao created SPARK-39240: Summary: Source and binary releases using different tool to generates hashes for integrity Key: SPARK-39240 URL: https://issues.apache.org/jira/browse/SPARK-39240 Project:

[jira] [Assigned] (SPARK-39221) sensitive information is not redacted correctly on thrift job/stage page

2022-05-19 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-39221: Assignee: Kent Yao > sensitive information is not redacted correctly on thrift job/stage page >

[jira] [Resolved] (SPARK-39221) sensitive information is not redacted correctly on thrift job/stage page

2022-05-19 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-39221. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36592

[jira] [Updated] (SPARK-39221) sensitive information is not redacted correctly on thrift job/stage page

2022-05-18 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-39221: - Summary: sensitive information is not redacted correctly on thrift job/stage page (was: sensitive

[jira] [Created] (SPARK-39221) sensitive information is not redacted correctly

2022-05-18 Thread Kent Yao (Jira)
Kent Yao created SPARK-39221: Summary: sensitive information is not redacted correctly Key: SPARK-39221 URL: https://issues.apache.org/jira/browse/SPARK-39221 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-39194) Add a pre resolution builder for spark session extensions

2022-05-16 Thread Kent Yao (Jira)
Kent Yao created SPARK-39194: Summary: Add a pre resolution builder for spark session extensions Key: SPARK-39194 URL: https://issues.apache.org/jira/browse/SPARK-39194 Project: Spark Issue

[jira] [Assigned] (SPARK-39138) Add ANSI general value specification and function -user

2022-05-13 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-39138: Assignee: Kent Yao > Add ANSI general value specification and function -user >

[jira] [Resolved] (SPARK-39138) Add ANSI general value specification and function -user

2022-05-13 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-39138. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36497

[jira] [Created] (SPARK-39174) Catalogs loading swallows missing classname for ClassNotFoundException

2022-05-13 Thread Kent Yao (Jira)
Kent Yao created SPARK-39174: Summary: Catalogs loading swallows missing classname for ClassNotFoundException Key: SPARK-39174 URL: https://issues.apache.org/jira/browse/SPARK-39174 Project: Spark

[jira] [Resolved] (SPARK-39041) Mapping Spark Query ResultSet/Schema to TRowSet/TTableSchema directly

2022-05-12 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-39041. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36373

[jira] [Assigned] (SPARK-39041) Mapping Spark Query ResultSet/Schema to TRowSet/TTableSchema directly

2022-05-12 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-39041: Assignee: Kent Yao > Mapping Spark Query ResultSet/Schema to TRowSet/TTableSchema directly >

[jira] [Created] (SPARK-39138) Add ANSI general value specification and function -user

2022-05-10 Thread Kent Yao (Jira)
Kent Yao created SPARK-39138: Summary: Add ANSI general value specification and function -user Key: SPARK-39138 URL: https://issues.apache.org/jira/browse/SPARK-39138 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-36391) When fetch chunk throw NPE, improve the error message

2022-05-06 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-36391: Assignee: angerszhu > When fetch chunk throw NPE, improve the error message >

[jira] [Assigned] (SPARK-36098) Group exception messages in core/storage

2022-05-06 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-36098: Assignee: dgd_contributor (was: Apache Spark) > Group exception messages in core/storage >

[jira] [Updated] (SPARK-36111) Support casting of timestamp without time zone to numeric type

2022-05-06 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36111: - Fix Version/s: (was: 3.3.0) > Support casting of timestamp without time zone to numeric type >

[jira] [Assigned] (SPARK-39068) Make thriftserver and sparksql-cli support in-memory catalog

2022-05-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-39068: Assignee: Kent Yao > Make thriftserver and sparksql-cli support in-memory catalog >

[jira] [Resolved] (SPARK-39068) Make thriftserver and sparksql-cli support in-memory catalog

2022-05-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-39068. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36407

[jira] [Created] (SPARK-39068) Make thriftserver and sparksql-cli support in-memory catalog

2022-04-29 Thread Kent Yao (Jira)
Kent Yao created SPARK-39068: Summary: Make thriftserver and sparksql-cli support in-memory catalog Key: SPARK-39068 URL: https://issues.apache.org/jira/browse/SPARK-39068 Project: Spark Issue

[jira] [Created] (SPARK-39055) Fix documentation 404 page

2022-04-28 Thread Kent Yao (Jira)
Kent Yao created SPARK-39055: Summary: Fix documentation 404 page Key: SPARK-39055 URL: https://issues.apache.org/jira/browse/SPARK-39055 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-39041) Mapping Spark Query ResultSet/Schema to TRowSet/TTableSchema directly

2022-04-27 Thread Kent Yao (Jira)
Kent Yao created SPARK-39041: Summary: Mapping Spark Query ResultSet/Schema to TRowSet/TTableSchema directly Key: SPARK-39041 URL: https://issues.apache.org/jira/browse/SPARK-39041 Project: Spark

[jira] [Resolved] (SPARK-38922) TaskLocation.apply throw NullPointerException

2022-04-20 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-38922. -- Fix Version/s: 3.1.3 3.0.4 3.3.0 3.2.2

[jira] [Created] (SPARK-38922) TaskLocation.apply throw NullPointerException

2022-04-16 Thread Kent Yao (Jira)
Kent Yao created SPARK-38922: Summary: TaskLocation.apply throw NullPointerException Key: SPARK-38922 URL: https://issues.apache.org/jira/browse/SPARK-38922 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-38799) Fix scala license declaration

2022-04-05 Thread Kent Yao (Jira)
Kent Yao created SPARK-38799: Summary: Fix scala license declaration Key: SPARK-38799 URL: https://issues.apache.org/jira/browse/SPARK-38799 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-38778) Replace http with https for project url in pom

2022-04-03 Thread Kent Yao (Jira)
Kent Yao created SPARK-38778: Summary: Replace http with https for project url in pom Key: SPARK-38778 URL: https://issues.apache.org/jira/browse/SPARK-38778 Project: Spark Issue Type:

[jira] [Created] (SPARK-38777) Add `bin/spark-submit --kill / --status` support for yarn

2022-04-03 Thread Kent Yao (Jira)
Kent Yao created SPARK-38777: Summary: Add `bin/spark-submit --kill / --status` support for yarn Key: SPARK-38777 URL: https://issues.apache.org/jira/browse/SPARK-38777 Project: Spark Issue

[jira] [Resolved] (SPARK-38695) ORC can not surport the dataType,such as char or varchar

2022-03-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-38695. -- Resolution: Duplicate This issue has been resolved and duplicated, I will close it. Feel free to

[jira] [Commented] (SPARK-38695) ORC can not surport the dataType,such as char or varchar

2022-03-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515222#comment-17515222 ] Kent Yao commented on SPARK-38695: -- do you see this error in spark 3.2.1 too? It shall be resolved

[jira] [Commented] (SPARK-38631) Arbitrary shell command injection via Utils.unpack()

2022-03-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512296#comment-17512296 ] Kent Yao commented on SPARK-38631: -- [~hyukjin.kwon] Hi, shall we update the fixed versions? >

[jira] [Commented] (SPARK-38587) Validating new location for rename command should use formatted names

2022-03-23 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511152#comment-17511152 ] Kent Yao commented on SPARK-38587: -- resolved by https://github.com/apache/spark/pull/35895 >

[jira] [Resolved] (SPARK-38587) Validating new location for rename command should use formatted names

2022-03-23 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-38587. -- Fix Version/s: 3.1.3 3.0.4 3.3.0 3.2.2

[jira] [Created] (SPARK-38587) Validating new location for rename command should use formatted names

2022-03-17 Thread Kent Yao (Jira)
Kent Yao created SPARK-38587: Summary: Validating new location for rename command should use formatted names Key: SPARK-38587 URL: https://issues.apache.org/jira/browse/SPARK-38587 Project: Spark

[jira] [Created] (SPARK-38500) Add ASF License header to all Service Provider configuration files

2022-03-10 Thread Kent Yao (Jira)
Kent Yao created SPARK-38500: Summary: Add ASF License header to all Service Provider configuration files Key: SPARK-38500 URL: https://issues.apache.org/jira/browse/SPARK-38500 Project: Spark

[jira] [Created] (SPARK-38446) Deadlock between ExecutorClassLoader and FileDownloadCallback caused by Log4j

2022-03-08 Thread Kent Yao (Jira)
Kent Yao created SPARK-38446: Summary: Deadlock between ExecutorClassLoader and FileDownloadCallback caused by Log4j Key: SPARK-38446 URL: https://issues.apache.org/jira/browse/SPARK-38446 Project: Spark

[jira] [Commented] (SPARK-38403) `running-on-kubernetes` page render bad in v3.2.1(latest) website

2022-03-02 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17500553#comment-17500553 ] Kent Yao commented on SPARK-38403: -- I guess it has been fixed via

[jira] [Created] (SPARK-37889) Log4j2 MarkerFilter can not filter unnecessary thrift errors

2022-01-12 Thread Kent Yao (Jira)
Kent Yao created SPARK-37889: Summary: Log4j2 MarkerFilter can not filter unnecessary thrift errors Key: SPARK-37889 URL: https://issues.apache.org/jira/browse/SPARK-37889 Project: Spark Issue

[jira] [Created] (SPARK-37877) Support clear shuffle dependencies eagerly for thrift server

2022-01-12 Thread Kent Yao (Jira)
Kent Yao created SPARK-37877: Summary: Support clear shuffle dependencies eagerly for thrift server Key: SPARK-37877 URL: https://issues.apache.org/jira/browse/SPARK-37877 Project: Spark Issue

[jira] [Assigned] (SPARK-37863) Add submitTime for Spark Application

2022-01-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-37863: Assignee: Apache Spark > Add submitTime for Spark Application >

[jira] [Resolved] (SPARK-37863) Add submitTime for Spark Application

2022-01-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-37863. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35162

[jira] [Assigned] (SPARK-37841) BasicWriteTaskStatsTracker should not try get status for a skipped file

2022-01-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-37841: Assignee: Kent Yao > BasicWriteTaskStatsTracker should not try get status for a skipped file >

[jira] [Resolved] (SPARK-37841) BasicWriteTaskStatsTracker should not try get status for a skipped file

2022-01-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-37841. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35132

[jira] [Created] (SPARK-37841) BasicWriteTaskStatsTracker should not try get status for a skipped file

2022-01-07 Thread Kent Yao (Jira)
Kent Yao created SPARK-37841: Summary: BasicWriteTaskStatsTracker should not try get status for a skipped file Key: SPARK-37841 URL: https://issues.apache.org/jira/browse/SPARK-37841 Project: Spark

[jira] [Created] (SPARK-37826) Use zstd codec name in ORC file names for hive orc impl

2022-01-06 Thread Kent Yao (Jira)
Kent Yao created SPARK-37826: Summary: Use zstd codec name in ORC file names for hive orc impl Key: SPARK-37826 URL: https://issues.apache.org/jira/browse/SPARK-37826 Project: Spark Issue Type:

[jira] [Updated] (SPARK-37825) Make spark beeline be able to handle javaOpts

2022-01-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-37825: - Issue Type: Improvement (was: Bug) > Make spark beeline be able to handle javaOpts >

[jira] [Created] (SPARK-37825) Make spark beeline be able to handle javaOpts

2022-01-05 Thread Kent Yao (Jira)
Kent Yao created SPARK-37825: Summary: Make spark beeline be able to handle javaOpts Key: SPARK-37825 URL: https://issues.apache.org/jira/browse/SPARK-37825 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-37820) Replace ApacheCommonBase64 with JavaBase64 for fucntions

2022-01-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-37820: - Summary: Replace ApacheCommonBase64 with JavaBase64 for fucntions (was: Replace ApacheCommonBase64

[jira] [Updated] (SPARK-37820) Replace ApacheCommonBase64 with JavaBase64 for string fucntions

2022-01-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-37820: - Summary: Replace ApacheCommonBase64 with JavaBase64 for string fucntions (was: Replace

[jira] [Created] (SPARK-37820) Replace ApacheCommonBase64 with JavaBase64 for

2022-01-05 Thread Kent Yao (Jira)
Kent Yao created SPARK-37820: Summary: Replace ApacheCommonBase64 with JavaBase64 for Key: SPARK-37820 URL: https://issues.apache.org/jira/browse/SPARK-37820 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-37694) Disallow delete resources in spark sql cli

2021-12-20 Thread Kent Yao (Jira)
Kent Yao created SPARK-37694: Summary: Disallow delete resources in spark sql cli Key: SPARK-37694 URL: https://issues.apache.org/jira/browse/SPARK-37694 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-37661) SparkSQLCLIDriver will use hive defaults to resolve warehouse dir

2021-12-16 Thread Kent Yao (Jira)
Kent Yao created SPARK-37661: Summary: SparkSQLCLIDriver will use hive defaults to resolve warehouse dir Key: SPARK-37661 URL: https://issues.apache.org/jira/browse/SPARK-37661 Project: Spark

[jira] [Resolved] (SPARK-35414) Completely fix the broadcast timeout issue in AQE

2021-12-10 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-35414. -- Target Version/s: 3.2.0 Assignee: (was: Yu Zhong) Resolution: Duplicate AQE

[jira] [Resolved] (SPARK-37530) Spark reads many paths very slow though newAPIHadoopFile

2021-12-03 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-37530. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34792

[jira] [Assigned] (SPARK-37530) Spark reads many paths very slow though newAPIHadoopFile

2021-12-03 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-37530: Assignee: Kent Yao > Spark reads many paths very slow though newAPIHadoopFile >

[jira] [Created] (SPARK-37532) RDD name could be very long and memory costly

2021-12-02 Thread Kent Yao (Jira)
Kent Yao created SPARK-37532: Summary: RDD name could be very long and memory costly Key: SPARK-37532 URL: https://issues.apache.org/jira/browse/SPARK-37532 Project: Spark Issue Type:

[jira] [Created] (SPARK-37530) Spark reads many paths very slow though newAPIHadoopFile

2021-12-02 Thread Kent Yao (Jira)
Kent Yao created SPARK-37530: Summary: Spark reads many paths very slow though newAPIHadoopFile Key: SPARK-37530 URL: https://issues.apache.org/jira/browse/SPARK-37530 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-37452) Char and Varchar breaks backward compatibility between v3 and v2

2021-11-29 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-37452. -- Fix Version/s: 3.2.1 3.3.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-37452) Char and Varchar breaks backward compatibility between v3 and v2

2021-11-29 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-37452: Assignee: Kent Yao > Char and Varchar breaks backward compatibility between v3 and v2 >

[jira] [Updated] (SPARK-37481) Disappearance of skipped stages mislead the bug hunting

2021-11-28 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-37481: - Description: # ## With FetchFailedException and Map Stage Retries When rerunning spark-sql shell

[jira] [Created] (SPARK-37481) Disappearance of skipped stages mislead the bug hunting

2021-11-28 Thread Kent Yao (Jira)
Kent Yao created SPARK-37481: Summary: Disappearance of skipped stages mislead the bug hunting Key: SPARK-37481 URL: https://issues.apache.org/jira/browse/SPARK-37481 Project: Spark Issue Type:

[jira] [Created] (SPARK-37452) Char and Varchar breaks backward compatibility between v3 and v2

2021-11-24 Thread Kent Yao (Jira)
Kent Yao created SPARK-37452: Summary: Char and Varchar breaks backward compatibility between v3 and v2 Key: SPARK-37452 URL: https://issues.apache.org/jira/browse/SPARK-37452 Project: Spark

[jira] [Created] (SPARK-37215) Support Application Timeouts on YARN

2021-11-05 Thread Kent Yao (Jira)
Kent Yao created SPARK-37215: Summary: Support Application Timeouts on YARN Key: SPARK-37215 URL: https://issues.apache.org/jira/browse/SPARK-37215 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-37098: - Fix Version/s: (was: 3.1.3) > Alter table properties should invalidate cache >

[jira] [Commented] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433614#comment-17433614 ] Kent Yao commented on SPARK-37098: -- Issuse resolved by [https://github.com/apache/spark/pull/34365] >

[jira] [Resolved] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-37098. -- Fix Version/s: 3.3.0 3.2.1 3.1.3 Resolution: Fixed >

[jira] [Assigned] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-37098: Assignee: XiDuo You > Alter table properties should invalidate cache >

[jira] [Resolved] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-03 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-36659. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33904

[jira] [Assigned] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-03 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-36659: Assignee: Kent Yao > Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

[jira] [Created] (SPARK-36662) special timestamps values support for path filters - modifiedBefore/modifiedAfter

2021-09-03 Thread Kent Yao (Jira)
Kent Yao created SPARK-36662: Summary: special timestamps values support for path filters - modifiedBefore/modifiedAfter Key: SPARK-36662 URL: https://issues.apache.org/jira/browse/SPARK-36662 Project:

[jira] [Created] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-02 Thread Kent Yao (Jira)
Kent Yao created SPARK-36659: Summary: Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config Key: SPARK-36659 URL: https://issues.apache.org/jira/browse/SPARK-36659 Project: Spark

[jira] [Created] (SPARK-36634) Support access and read parquet file by column index

2021-09-01 Thread Kent Yao (Jira)
Kent Yao created SPARK-36634: Summary: Support access and read parquet file by column index Key: SPARK-36634 URL: https://issues.apache.org/jira/browse/SPARK-36634 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36611) Remove unused listener in HiveThriftServer2AppStatusStore

2021-08-30 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407054#comment-17407054 ] Kent Yao commented on SPARK-36611: -- issue resolved by https://github.com/apache/spark/pull/33866 >

[jira] [Resolved] (SPARK-36611) Remove unused listener in HiveThriftServer2AppStatusStore

2021-08-30 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-36611. -- Fix Version/s: 3.3.0 Resolution: Fixed > Remove unused listener in

[jira] [Created] (SPARK-36477) Infering schema from JSON file shall respect ignoreCorruptFiles and handle IOE

2021-08-11 Thread Kent Yao (Jira)
Kent Yao created SPARK-36477: Summary: Infering schema from JSON file shall respect ignoreCorruptFiles and handle IOE Key: SPARK-36477 URL: https://issues.apache.org/jira/browse/SPARK-36477 Project:

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-06 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Description:   h2. A test case Use bin/spark-sql with local mode and all other default settings with

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-06 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Description:   h2. A test case Use bin/spark-sql with local mode and all other default settings with

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-06 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Attachment: image-2021-08-06-17-57-15-765.png > Demote BroadcastJoin causes performance regression and

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-06 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Attachment: screenshot-1.png > Demote BroadcastJoin causes performance regression and increases OOM

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Description:     !image-2021-08-06-11-24-34-122.png! was:   !image-2021-08-06-11-24-34-122.png!

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Description:   !image-2021-08-06-11-24-34-122.png! was:!image-2021-08-06-11-24-34-122.png! >

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Attachment: image-2021-08-06-11-24-34-122.png > Demote BroadcastJoin causes performance regression and

[jira] [Updated] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36443: - Description: !image-2021-08-06-11-24-34-122.png! (was: !image-2021-08-06-11-19-00-105.png!) > Demote

[jira] [Created] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-08-05 Thread Kent Yao (Jira)
Kent Yao created SPARK-36443: Summary: Demote BroadcastJoin causes performance regression and increases OOM risks Key: SPARK-36443 URL: https://issues.apache.org/jira/browse/SPARK-36443 Project: Spark

[jira] [Created] (SPARK-36421) Validate all SQL configs to prevent from wrong use for ConfigEntry

2021-08-04 Thread Kent Yao (Jira)
Kent Yao created SPARK-36421: Summary: Validate all SQL configs to prevent from wrong use for ConfigEntry Key: SPARK-36421 URL: https://issues.apache.org/jira/browse/SPARK-36421 Project: Spark

[jira] [Updated] (SPARK-36421) Validate all SQL configs to prevent from wrong use for ConfigEntry

2021-08-04 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36421: - Issue Type: Bug (was: Improvement) > Validate all SQL configs to prevent from wrong use for

[jira] [Updated] (SPARK-36421) Validate all SQL configs to prevent from wrong use for ConfigEntry

2021-08-04 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36421: - Priority: Minor (was: Major) > Validate all SQL configs to prevent from wrong use for ConfigEntry >

[jira] [Updated] (SPARK-36414) Disable timeout for BroadcastQueryStageExec in AQE

2021-08-04 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36414: - Description: This reverts SPARK-31475, as there are always more concurrent jobs running in AQE mode,

[jira] [Created] (SPARK-36414) Disable timeout for BroadcastQueryStageExec in AQE

2021-08-04 Thread Kent Yao (Jira)
Kent Yao created SPARK-36414: Summary: Disable timeout for BroadcastQueryStageExec in AQE Key: SPARK-36414 URL: https://issues.apache.org/jira/browse/SPARK-36414 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36414) Disable timeout for BroadcastQueryStageExec in AQE

2021-08-04 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36414: - Attachment: image-2021-08-04-18-53-44-879.png > Disable timeout for BroadcastQueryStageExec in AQE >

[jira] [Resolved] (SPARK-36213) Normalize PartitionSpec for DescTable with PartitionSpec

2021-07-21 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-36213. -- Fix Version/s: 3.1.3 3.2.0 3.0.4 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-36213) Normalize PartitionSpec for DescTable with PartitionSpec

2021-07-21 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-36213: Assignee: Kent Yao > Normalize PartitionSpec for DescTable with PartitionSpec >

[jira] [Updated] (SPARK-36213) Normalize PartitionSpec for DescTable with PartitionSpec

2021-07-20 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-36213: - Description: !image-2021-07-20-16-26-09-456.png! (was: !image-2021-07-20-01-03-49-573.png!) >

<    4   5   6   7   8   9   10   11   12   13   >