[jira] [Updated] (SPARK-32121) ExternalShuffleBlockResolverSuite failed on Windows

2020-06-30 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-32121: -- Issue Type: Bug (was: Test) > ExternalShuffleBlockResolverSuite failed on Windows >

[jira] [Created] (SPARK-32121) ExternalShuffleBlockResolverSuite failed on windows

2020-06-28 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-32121: - Summary: ExternalShuffleBlockResolverSuite failed on windows Key: SPARK-32121 URL: https://issues.apache.org/jira/browse/SPARK-32121 Project: Spark Issue Type:

[jira] [Updated] (SPARK-32121) ExternalShuffleBlockResolverSuite failed on Windows

2020-06-28 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-32121: -- Summary: ExternalShuffleBlockResolverSuite failed on Windows (was: ExternalShuffleBlockResolverSuite

[jira] [Updated] (SPARK-34146) "*" should not crash in SparkGetSchemasOperation

2021-01-17 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-34146: -- Description: HiveServer2 treat "*" as list all databases, but spark will crashed when handle global

[jira] [Created] (SPARK-34146) "*" should not crash in SparkGetSchemasOperation

2021-01-17 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-34146: - Summary: "*" should not crash in SparkGetSchemasOperation Key: SPARK-34146 URL: https://issues.apache.org/jira/browse/SPARK-34146 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-34146) "*" should not throw Exception in SparkGetSchemasOperation

2021-01-17 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-34146: -- Summary: "*" should not throw Exception in SparkGetSchemasOperation (was: "*" should not crash in

[jira] [Updated] (SPARK-34146) "*" should not crash in SparkGetSchemasOperation

2021-01-17 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-34146: -- Description: HiveServer2 treat "*" as list all databases, but spark will throw `Exception` when

[jira] [Commented] (SPARK-35310) Bump Breeze from 1.0 to 1.2

2021-05-12 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343090#comment-17343090 ] Cheng Pan commented on SPARK-35310: --- Had a try, but failed, seeĀ [diverging implicit expansion on

[jira] [Updated] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-35223: -- Description: !image-2021-04-25-22-26-52-006.png! (was: !image-2021-04-25-22-24-00-868.png!) > Add

[jira] [Updated] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-35223: -- Attachment: image-2021-04-25-22-26-52-006.png > Add IssueNavigationLink for IDEA >

[jira] [Updated] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-35223: -- Affects Version/s: (was: 3.1.1) 3.2.0 > Add IssueNavigationLink for IDEA >

[jira] [Created] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-35223: - Summary: Add IssueNavigationLink for IDEA Key: SPARK-35223 URL: https://issues.apache.org/jira/browse/SPARK-35223 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-35310) Bump Breeze from 1.0 to 1.2

2021-05-04 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-35310: - Summary: Bump Breeze from 1.0 to 1.2 Key: SPARK-35310 URL: https://issues.apache.org/jira/browse/SPARK-35310 Project: Spark Issue Type: Dependency upgrade

[jira] [Created] (SPARK-36466) Table in unloaded catalog referenced by view should load correctly

2021-08-10 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-36466: - Summary: Table in unloaded catalog referenced by view should load correctly Key: SPARK-36466 URL: https://issues.apache.org/jira/browse/SPARK-36466 Project: Spark

[jira] [Updated] (SPARK-36466) Table in unloaded catalog referenced by view should load correctly

2021-08-10 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-36466: -- Affects Version/s: (was: 3.1.2) > Table in unloaded catalog referenced by view should load

[jira] [Created] (SPARK-37675) Fix calculate approxChunkSize when bitmaps isEmpty

2021-12-17 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-37675: - Summary: Fix calculate approxChunkSize when bitmaps isEmpty Key: SPARK-37675 URL: https://issues.apache.org/jira/browse/SPARK-37675 Project: Spark Issue Type:

[jira] [Updated] (SPARK-37675) Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

2021-12-17 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-37675: -- Summary: Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block (was: Fix

[jira] [Commented] (SPARK-37648) Spark catalog and Delta tables

2021-12-19 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17462361#comment-17462361 ] Cheng Pan commented on SPARK-37648: --- This issue has been fixed in Apache Kyuubi (Incubating),

[jira] [Comment Edited] (SPARK-37648) Spark catalog and Delta tables

2021-12-19 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17462361#comment-17462361 ] Cheng Pan edited comment on SPARK-37648 at 12/20/21, 2:42 AM: -- We have a

[jira] [Created] (SPARK-37793) Invalid LocalMergedBlockData cause task hang

2021-12-30 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-37793: - Summary: Invalid LocalMergedBlockData cause task hang Key: SPARK-37793 URL: https://issues.apache.org/jira/browse/SPARK-37793 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-37574) Simplify fetchBlocks w/o retry

2021-12-07 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-37574: - Summary: Simplify fetchBlocks w/o retry Key: SPARK-37574 URL: https://issues.apache.org/jira/browse/SPARK-37574 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-37847) PushBlockStreamCallback should check isTooLate first to avoid NPE

2022-01-09 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-37847: - Summary: PushBlockStreamCallback should check isTooLate first to avoid NPE Key: SPARK-37847 URL: https://issues.apache.org/jira/browse/SPARK-37847 Project: Spark

[jira] [Created] (SPARK-37925) Update document to mention the workaround for YARN-11053

2022-01-16 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-37925: - Summary: Update document to mention the workaround for YARN-11053 Key: SPARK-37925 URL: https://issues.apache.org/jira/browse/SPARK-37925 Project: Spark Issue

[jira] [Created] (SPARK-38411) Use UTF-8 to read event log

2022-03-03 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-38411: - Summary: Use UTF-8 to read event log Key: SPARK-38411 URL: https://issues.apache.org/jira/browse/SPARK-38411 Project: Spark Issue Type: Bug Components:

[jira] [Comment Edited] (SPARK-38703) High GC and memory footprint after switch to ZSTD

2022-04-01 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515907#comment-17515907 ] Cheng Pan edited comment on SPARK-38703 at 4/1/22 12:55 PM: SPARK-34390 may

[jira] [Commented] (SPARK-38703) High GC and memory footprint after switch to ZSTD

2022-04-01 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515907#comment-17515907 ] Cheng Pan commented on SPARK-38703: --- SPARK-34390 may helps, our benchmark of 1T TPC-DS shows the

[jira] [Comment Edited] (SPARK-38703) High GC and memory footprint after switch to ZSTD

2022-04-01 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515907#comment-17515907 ] Cheng Pan edited comment on SPARK-38703 at 4/1/22 12:56 PM: SPARK-34390 may

[jira] [Commented] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

2022-04-02 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516280#comment-17516280 ] Cheng Pan commented on SPARK-30602: --- Hi [~mshen] [~mridulm80] Why your reporter shows that Push-based

[jira] [Comment Edited] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

2022-04-02 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516280#comment-17516280 ] Cheng Pan edited comment on SPARK-30602 at 4/2/22 12:10 PM: Hi [~mshen]

[jira] [Created] (SPARK-38138) Materialize QueryPlan subqueries

2022-02-08 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-38138: - Summary: Materialize QueryPlan subqueries Key: SPARK-38138 URL: https://issues.apache.org/jira/browse/SPARK-38138 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45089) Remove obsolete repo of DB2 JDBC driver

2023-09-05 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-45089: - Summary: Remove obsolete repo of DB2 JDBC driver Key: SPARK-45089 URL: https://issues.apache.org/jira/browse/SPARK-45089 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-45670) SparkSubmit does not support --total-executor-cores when deploying on K8s

2023-10-25 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-45670: - Summary: SparkSubmit does not support --total-executor-cores when deploying on K8s Key: SPARK-45670 URL: https://issues.apache.org/jira/browse/SPARK-45670 Project: Spark

[jira] [Created] (SPARK-45292) Remove Guava from shared classes from IsolatedClientLoader

2023-09-24 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-45292: - Summary: Remove Guava from shared classes from IsolatedClientLoader Key: SPARK-45292 URL: https://issues.apache.org/jira/browse/SPARK-45292 Project: Spark Issue

[jira] [Created] (SPARK-45389) Correct MetaException matching rule on getting partition metadata

2023-09-30 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-45389: - Summary: Correct MetaException matching rule on getting partition metadata Key: SPARK-45389 URL: https://issues.apache.org/jira/browse/SPARK-45389 Project: Spark

[jira] [Created] (SPARK-45388) Eliminate unnecessary reflection invocation in Hive shim classes

2023-09-30 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-45388: - Summary: Eliminate unnecessary reflection invocation in Hive shim classes Key: SPARK-45388 URL: https://issues.apache.org/jira/browse/SPARK-45388 Project: Spark

[jira] [Created] (SPARK-45358) Remove shim classes for Hive prior 2.0.0

2023-09-27 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-45358: - Summary: Remove shim classes for Hive prior 2.0.0 Key: SPARK-45358 URL: https://issues.apache.org/jira/browse/SPARK-45358 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-45004) Adding extension for Spark SQL authorization with Ranger-Hive policies

2023-09-25 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768581#comment-17768581 ] Cheng Pan commented on SPARK-45004: --- [~Ahmadkhani] The Apache Kyuubi community maintains such an

[jira] [Created] (SPARK-45646) Remove hardcoding time variables prior to Hive 2.0

2023-10-24 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-45646: - Summary: Remove hardcoding time variables prior to Hive 2.0 Key: SPARK-45646 URL: https://issues.apache.org/jira/browse/SPARK-45646 Project: Spark Issue Type:

[jira] [Created] (SPARK-44938) Change default value of spark.sql.maxSinglePartitionBytes to 128m

2023-08-23 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-44938: - Summary: Change default value of spark.sql.maxSinglePartitionBytes to 128m Key: SPARK-44938 URL: https://issues.apache.org/jira/browse/SPARK-44938 Project: Spark

[jira] [Created] (SPARK-44811) Upgrade Guava to 32+

2023-08-14 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-44811: - Summary: Upgrade Guava to 32+ Key: SPARK-44811 URL: https://issues.apache.org/jira/browse/SPARK-44811 Project: Spark Issue Type: Dependency upgrade

[jira] [Created] (SPARK-39313) V2ExpressionUtils.toCatalystOrdering should fail if V2Expression can not be translated

2022-05-27 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39313: - Summary: V2ExpressionUtils.toCatalystOrdering should fail if V2Expression can not be translated Key: SPARK-39313 URL: https://issues.apache.org/jira/browse/SPARK-39313

[jira] [Created] (SPARK-39160) Remove workaround for ARROW-1948

2022-05-12 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39160: - Summary: Remove workaround for ARROW-1948 Key: SPARK-39160 URL: https://issues.apache.org/jira/browse/SPARK-39160 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-39601) AllocationFailure should not be treated as exitCausedByApp when driver is shutting down

2022-06-25 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39601: - Summary: AllocationFailure should not be treated as exitCausedByApp when driver is shutting down Key: SPARK-39601 URL: https://issues.apache.org/jira/browse/SPARK-39601

[jira] [Created] (SPARK-39607) DataSourceV2: Distribution and ordering support V2 function in writing

2022-06-26 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39607: - Summary: DataSourceV2: Distribution and ordering support V2 function in writing Key: SPARK-39607 URL: https://issues.apache.org/jira/browse/SPARK-39607 Project: Spark

[jira] [Created] (SPARK-39439) Suppress error log for in-progress event log not found

2022-06-10 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39439: - Summary: Suppress error log for in-progress event log not found Key: SPARK-39439 URL: https://issues.apache.org/jira/browse/SPARK-39439 Project: Spark Issue Type:

[jira] [Created] (SPARK-39403) Add SPARK_SUBMIT_OPTS in spark-env.sh.template

2022-06-07 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39403: - Summary: Add SPARK_SUBMIT_OPTS in spark-env.sh.template Key: SPARK-39403 URL: https://issues.apache.org/jira/browse/SPARK-39403 Project: Spark Issue Type:

[jira] [Created] (SPARK-39472) Refactor InMemoryTableCatalog classes hierarchy

2022-06-14 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39472: - Summary: Refactor InMemoryTableCatalog classes hierarchy Key: SPARK-39472 URL: https://issues.apache.org/jira/browse/SPARK-39472 Project: Spark Issue Type: Test

[jira] [Resolved] (SPARK-39472) Refactor InMemoryTableCatalog classes hierarchy

2022-06-14 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan resolved SPARK-39472. --- Resolution: Not A Problem > Refactor InMemoryTableCatalog classes hierarchy >

[jira] [Created] (SPARK-39538) Convert CaseInsensitiveStringMap#logger to static

2022-06-20 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39538: - Summary: Convert CaseInsensitiveStringMap#logger to static Key: SPARK-39538 URL: https://issues.apache.org/jira/browse/SPARK-39538 Project: Spark Issue Type:

[jira] [Created] (SPARK-39079) Catalog name should not contain dot

2022-04-30 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39079: - Summary: Catalog name should not contain dot Key: SPARK-39079 URL: https://issues.apache.org/jira/browse/SPARK-39079 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-39128) Log cost time for Get FileStatus in HadoopTableReader

2022-05-08 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39128: - Summary: Log cost time for Get FileStatus in HadoopTableReader Key: SPARK-39128 URL: https://issues.apache.org/jira/browse/SPARK-39128 Project: Spark Issue Type:

[jira] [Created] (SPARK-40886) Bump Jackson Databind 2.13.4.2

2022-10-23 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-40886: - Summary: Bump Jackson Databind 2.13.4.2 Key: SPARK-40886 URL: https://issues.apache.org/jira/browse/SPARK-40886 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-40887) Allow Spark on K8s to integrate w/ Log Service

2022-10-23 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-40887: - Summary: Allow Spark on K8s to integrate w/ Log Service Key: SPARK-40887 URL: https://issues.apache.org/jira/browse/SPARK-40887 Project: Spark Issue Type: New

[jira] [Created] (SPARK-40747) Support setting driver log url using env vars other than Standalone mode

2022-10-11 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-40747: - Summary: Support setting driver log url using env vars other than Standalone mode Key: SPARK-40747 URL: https://issues.apache.org/jira/browse/SPARK-40747 Project: Spark

[jira] [Commented] (SPARK-40767) Compile Spark example module will hang

2022-10-12 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17616232#comment-17616232 ] Cheng Pan commented on SPARK-40767: --- it's too slow on my side !image-2022-10-12-14-57-55-988.png! >

[jira] [Updated] (SPARK-40767) Compile Spark example module will hang

2022-10-12 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-40767: -- Attachment: image-2022-10-12-14-57-55-988.png > Compile Spark example module will hang >

[jira] [Created] (SPARK-39977) Remove unnecessary guava exclusion from jackson-module-scala

2022-08-04 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39977: - Summary: Remove unnecessary guava exclusion from jackson-module-scala Key: SPARK-39977 URL: https://issues.apache.org/jira/browse/SPARK-39977 Project: Spark

[jira] [Created] (SPARK-40997) K8s resource name prefix should start w/ alphanumeric

2022-11-02 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-40997: - Summary: K8s resource name prefix should start w/ alphanumeric Key: SPARK-40997 URL: https://issues.apache.org/jira/browse/SPARK-40997 Project: Spark Issue Type:

[jira] [Created] (SPARK-41788) Move InsertIntoStatement to basicLogicalOperators

2022-12-30 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-41788: - Summary: Move InsertIntoStatement to basicLogicalOperators Key: SPARK-41788 URL: https://issues.apache.org/jira/browse/SPARK-41788 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-41451) Avoid using empty abbrevMarker in StringUtils.abbreviate

2022-12-08 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-41451: - Summary: Avoid using empty abbrevMarker in StringUtils.abbreviate Key: SPARK-41451 URL: https://issues.apache.org/jira/browse/SPARK-41451 Project: Spark Issue

[jira] [Created] (SPARK-41667) Expose env var SPARK_DRIVER_POD_NAME in Driver Pod

2022-12-21 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-41667: - Summary: Expose env var SPARK_DRIVER_POD_NAME in Driver Pod Key: SPARK-41667 URL: https://issues.apache.org/jira/browse/SPARK-41667 Project: Spark Issue Type:

[jira] [Created] (SPARK-41376) Executor netty direct memory check should respect spark.shuffle.io.preferDirectBufs

2022-12-04 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-41376: - Summary: Executor netty direct memory check should respect spark.shuffle.io.preferDirectBufs Key: SPARK-41376 URL: https://issues.apache.org/jira/browse/SPARK-41376

[jira] [Created] (SPARK-41458) Correctly transform the SPI services for Yarn Shuffle Service

2022-12-08 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-41458: - Summary: Correctly transform the SPI services for Yarn Shuffle Service Key: SPARK-41458 URL: https://issues.apache.org/jira/browse/SPARK-41458 Project: Spark

[jira] [Created] (SPARK-41210) Window based executor failure tracking mechanism

2022-11-20 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-41210: - Summary: Window based executor failure tracking mechanism Key: SPARK-41210 URL: https://issues.apache.org/jira/browse/SPARK-41210 Project: Spark Issue Type: New

[jira] [Created] (SPARK-40983) Remove Hadoop requirements for zstd mention in Parquet compression codec

2022-10-31 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-40983: - Summary: Remove Hadoop requirements for zstd mention in Parquet compression codec Key: SPARK-40983 URL: https://issues.apache.org/jira/browse/SPARK-40983 Project: Spark

[jira] [Created] (SPARK-41089) Relocate Netty native arm64 libs

2022-11-09 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-41089: - Summary: Relocate Netty native arm64 libs Key: SPARK-41089 URL: https://issues.apache.org/jira/browse/SPARK-41089 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-41093) Remove netty-tcnative-classes from Spark dependencyList

2022-11-09 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-41093: - Summary: Remove netty-tcnative-classes from Spark dependencyList Key: SPARK-41093 URL: https://issues.apache.org/jira/browse/SPARK-41093 Project: Spark Issue

[jira] [Created] (SPARK-41136) Shorten graceful shutdown time of ExecutorPodsSnapshotsStoreImpl to prevent blocking shutdown process

2022-11-14 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-41136: - Summary: Shorten graceful shutdown time of ExecutorPodsSnapshotsStoreImpl to prevent blocking shutdown process Key: SPARK-41136 URL: https://issues.apache.org/jira/browse/SPARK-41136

[jira] [Updated] (SPARK-39601) AllocationFailure should not be treated as exitCausedByApp when driver is shutting down

2022-11-14 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-39601: -- Description: I observed some Spark Applications successfully completed all jobs but failed during

[jira] [Updated] (SPARK-39601) AllocationFailure should not be treated as exitCausedByApp when driver is shutting down

2022-11-14 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-39601: -- Description: I observed some Spark Applications successfully completed all jobs but failed during

[jira] [Updated] (SPARK-39601) AllocationFailure should not be treated as exitCausedByApp when driver is shutting down

2022-11-14 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-39601: -- Description: I observed some Spark Applications successfully completed all jobs but failed during

[jira] [Created] (SPARK-42906) Resource name prefix should start with an alphabetic character

2023-03-23 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-42906: - Summary: Resource name prefix should start with an alphabetic character Key: SPARK-42906 URL: https://issues.apache.org/jira/browse/SPARK-42906 Project: Spark

[jira] [Created] (SPARK-42813) Print application info when waitAppCompletion is false

2023-03-15 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-42813: - Summary: Print application info when waitAppCompletion is false Key: SPARK-42813 URL: https://issues.apache.org/jira/browse/SPARK-42813 Project: Spark Issue Type:

[jira] [Commented] (SPARK-41952) Upgrade Parquet to fix off-heap memory leaks in Zstd codec

2023-02-19 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691012#comment-17691012 ] Cheng Pan commented on SPARK-41952: --- Fix on Spark side is feasible, I'm working on this. > Upgrade

[jira] [Created] (SPARK-43150) Remove workaround for PARQUET-2160

2023-04-14 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43150: - Summary: Remove workaround for PARQUET-2160 Key: SPARK-43150 URL: https://issues.apache.org/jira/browse/SPARK-43150 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43211) Remove Hadoop2 support in IsolatedClientLoader

2023-04-20 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43211: - Summary: Remove Hadoop2 support in IsolatedClientLoader Key: SPARK-43211 URL: https://issues.apache.org/jira/browse/SPARK-43211 Project: Spark Issue Type:

[jira] [Created] (SPARK-43208) IsolatedClassLoader should close barrier class InputStream after reading

2023-04-19 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43208: - Summary: IsolatedClassLoader should close barrier class InputStream after reading Key: SPARK-43208 URL: https://issues.apache.org/jira/browse/SPARK-43208 Project: Spark

[jira] [Created] (SPARK-43193) Remove workaround for HADOOP-12074

2023-04-19 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43193: - Summary: Remove workaround for HADOOP-12074 Key: SPARK-43193 URL: https://issues.apache.org/jira/browse/SPARK-43193 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43186) Remove workaround for FileSinkDesc

2023-04-19 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43186: - Summary: Remove workaround for FileSinkDesc Key: SPARK-43186 URL: https://issues.apache.org/jira/browse/SPARK-43186 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43187) Remove workaround for MiniKdc's BindException

2023-04-19 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43187: - Summary: Remove workaround for MiniKdc's BindException Key: SPARK-43187 URL: https://issues.apache.org/jira/browse/SPARK-43187 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-43191) Replace reflection w/ direct calling for Hadoop CallerContext

2023-04-19 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43191: - Summary: Replace reflection w/ direct calling for Hadoop CallerContext Key: SPARK-43191 URL: https://issues.apache.org/jira/browse/SPARK-43191 Project: Spark

[jira] [Created] (SPARK-43195) Remove unnecessary serializable wrapper in HadoopFSUtils

2023-04-19 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43195: - Summary: Remove unnecessary serializable wrapper in HadoopFSUtils Key: SPARK-43195 URL: https://issues.apache.org/jira/browse/SPARK-43195 Project: Spark Issue

[jira] [Created] (SPARK-43171) Support dynamic changing unix user in Pod

2023-04-17 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43171: - Summary: Support dynamic changing unix user in Pod Key: SPARK-43171 URL: https://issues.apache.org/jira/browse/SPARK-43171 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43202) Replace reflection w/ direct calling for YARN Resource API

2023-04-19 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43202: - Summary: Replace reflection w/ direct calling for YARN Resource API Key: SPARK-43202 URL: https://issues.apache.org/jira/browse/SPARK-43202 Project: Spark Issue

[jira] [Created] (SPARK-43200) Remove Hadoop 2 reference in docs

2023-04-19 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43200: - Summary: Remove Hadoop 2 reference in docs Key: SPARK-43200 URL: https://issues.apache.org/jira/browse/SPARK-43200 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-43320) Directly call Hive 2.3.9 API

2023-04-28 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43320: - Summary: Directly call Hive 2.3.9 API Key: SPARK-43320 URL: https://issues.apache.org/jira/browse/SPARK-43320 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43319) Remove usage of deprecated DefaultKubernetesClient

2023-04-28 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43319: - Summary: Remove usage of deprecated DefaultKubernetesClient Key: SPARK-43319 URL: https://issues.apache.org/jira/browse/SPARK-43319 Project: Spark Issue Type:

[jira] [Created] (SPARK-43248) Unnecessary serialize/deserialize of Path on parallel gather partition stats

2023-04-24 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43248: - Summary: Unnecessary serialize/deserialize of Path on parallel gather partition stats Key: SPARK-43248 URL: https://issues.apache.org/jira/browse/SPARK-43248 Project:

[jira] [Created] (SPARK-42949) Simplify code for NAAJ

2023-03-28 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-42949: - Summary: Simplify code for NAAJ Key: SPARK-42949 URL: https://issues.apache.org/jira/browse/SPARK-42949 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-44489) Make InsertIntoDataSourceDirCommand extends DataWritingCommand

2023-07-19 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-44489: - Summary: Make InsertIntoDataSourceDirCommand extends DataWritingCommand Key: SPARK-44489 URL: https://issues.apache.org/jira/browse/SPARK-44489 Project: Spark

[jira] [Resolved] (SPARK-44489) Make InsertIntoDataSourceDirCommand extends DataWritingCommand

2023-07-19 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan resolved SPARK-44489. --- Resolution: Not A Problem > Make InsertIntoDataSourceDirCommand extends DataWritingCommand >

[jira] [Updated] (SPARK-43880) Organize hadoop-cloud in standard maven project structure

2023-05-30 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-43880: -- Parent: SPARK-43197 Issue Type: Sub-task (was: Improvement) > Organize hadoop-cloud in

[jira] [Created] (SPARK-43880) Organize hadoop-cloud in standard maven project structure

2023-05-30 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43880: - Summary: Organize hadoop-cloud in standard maven project structure Key: SPARK-43880 URL: https://issues.apache.org/jira/browse/SPARK-43880 Project: Spark Issue

[jira] [Created] (SPARK-43751) Document for unbase64 behavior change

2023-05-23 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43751: - Summary: Document for unbase64 behavior change Key: SPARK-43751 URL: https://issues.apache.org/jira/browse/SPARK-43751 Project: Spark Issue Type: Documentation

[jira] [Created] (SPARK-44525) Improve error message when Invoke method is not found

2023-07-24 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-44525: - Summary: Improve error message when Invoke method is not found Key: SPARK-44525 URL: https://issues.apache.org/jira/browse/SPARK-44525 Project: Spark Issue Type:

[jira] [Created] (SPARK-44727) Improve the error message for dynamic allocation conditions

2023-08-08 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-44727: - Summary: Improve the error message for dynamic allocation conditions Key: SPARK-44727 URL: https://issues.apache.org/jira/browse/SPARK-44727 Project: Spark Issue

[jira] [Created] (SPARK-44677) Drop legacy Hive-based ORC file format

2023-08-04 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-44677: - Summary: Drop legacy Hive-based ORC file format Key: SPARK-44677 URL: https://issues.apache.org/jira/browse/SPARK-44677 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44669) Parquet/ORC files written using Hive Serde should has file extension

2023-08-03 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-44669: - Summary: Parquet/ORC files written using Hive Serde should has file extension Key: SPARK-44669 URL: https://issues.apache.org/jira/browse/SPARK-44669 Project: Spark

[jira] [Created] (SPARK-44180) DistributionAndOrderingUtils should apply ResolveTimeZone

2023-06-25 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-44180: - Summary: DistributionAndOrderingUtils should apply ResolveTimeZone Key: SPARK-44180 URL: https://issues.apache.org/jira/browse/SPARK-44180 Project: Spark Issue

[jira] [Created] (SPARK-44204) Add missing recordHiveCall for getPartitionNames

2023-06-26 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-44204: - Summary: Add missing recordHiveCall for getPartitionNames Key: SPARK-44204 URL: https://issues.apache.org/jira/browse/SPARK-44204 Project: Spark Issue Type: Bug

  1   2   >