[jira] [Created] (SPARK-43533) Enable MultiIndex test for IndexesTests.test_difference

Haejoon Lee created SPARK-43533: --- Summary: Enable MultiIndex test for IndexesTests.test_difference Key: SPARK-43533 URL: https://issues.apache.org/jira/browse/SPARK-43533 Project: Spark Issue

[jira] [Assigned] (SPARK-43532) Upgrade `jdbc` related test dependencies

[ https://issues.apache.org/jira/browse/SPARK-43532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43532: - Assignee: BingKun Pan > Upgrade `jdbc` related test dependencies >

[jira] [Resolved] (SPARK-43532) Upgrade `jdbc` related test dependencies

[ https://issues.apache.org/jira/browse/SPARK-43532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43532. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41194

[jira] [Commented] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723338#comment-17723338 ] Yuming Wang commented on SPARK-43526: - Why do you prefer shuffle hash join? > when shuffle hash

[jira] [Commented] (SPARK-43509) Support creating multiple sessions for Spark Connect in PySpark

[ https://issues.apache.org/jira/browse/SPARK-43509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723336#comment-17723336 ] Snoot.io commented on SPARK-43509: -- User 'grundprinzip' has created a pull request for this issue:

[jira] [Updated] (SPARK-43461) Skip compiling useless files when making distribution

[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43461: Fix Version/s: 3.5.0 > Skip compiling useless files when making distribution >

[jira] [Resolved] (SPARK-43461) Skip compiling useless files when making distribution

[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43461. -- Resolution: Fixed Issue resolved by pull request 41141 https://github.com/apache/spark/pull/41141 >

[jira] [Assigned] (SPARK-43461) Skip compiling useless files when making distribution

[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43461: Assignee: Yuming Wang > Skip compiling useless files when making distribution >

[jira] [Assigned] (SPARK-43531) Enable more parity tests for Pandas UDFs.

[ https://issues.apache.org/jira/browse/SPARK-43531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43531: - Assignee: Takuya Ueshin > Enable more parity tests for Pandas UDFs. >

[jira] [Resolved] (SPARK-43531) Enable more parity tests for Pandas UDFs.

[ https://issues.apache.org/jira/browse/SPARK-43531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43531. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41193

[jira] [Updated] (SPARK-43488) bitmap function

[ https://issues.apache.org/jira/browse/SPARK-43488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yiku123 updated SPARK-43488: Description: maybe spark need to have some bitmap functions？ example  like bitmapBuild

[jira] [Created] (SPARK-43532) Upgrade `jdbc` related test dependencies

BingKun Pan created SPARK-43532: --- Summary: Upgrade `jdbc` related test dependencies Key: SPARK-43532 URL: https://issues.apache.org/jira/browse/SPARK-43532 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-43524) Memory leak in Spark UI

[ https://issues.apache.org/jira/browse/SPARK-43524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43524. - Resolution: Duplicate > Memory leak in Spark UI > --- > >

[jira] [Updated] (SPARK-43521) Support CREATE TABLE LIKE FILE for PARQUET

[ https://issues.apache.org/jira/browse/SPARK-43521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43521: Issue Type: New Feature (was: Bug) > Support CREATE TABLE LIKE FILE for PARQUET >

[jira] [Created] (SPARK-43531) Enable more parity tests for Pandas UDFs.

Takuya Ueshin created SPARK-43531: - Summary: Enable more parity tests for Pandas UDFs. Key: SPARK-43531 URL: https://issues.apache.org/jira/browse/SPARK-43531 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-43525) Enhance ImportOrderChecker rules for `group.scala`

[ https://issues.apache.org/jira/browse/SPARK-43525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43525: Assignee: BingKun Pan > Enhance ImportOrderChecker rules for `group.scala` >

[jira] [Resolved] (SPARK-43525) Enhance ImportOrderChecker rules for `group.scala`

[ https://issues.apache.org/jira/browse/SPARK-43525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43525. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41185

[jira] [Resolved] (SPARK-43528) Support duplicated field names in createDataFrame with pandas DataFrame.

[ https://issues.apache.org/jira/browse/SPARK-43528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43528. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41190

[jira] [Assigned] (SPARK-43528) Support duplicated field names in createDataFrame with pandas DataFrame.

[ https://issues.apache.org/jira/browse/SPARK-43528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43528: Assignee: Takuya Ueshin > Support duplicated field names in createDataFrame with pandas

[jira] [Resolved] (SPARK-43527) Fix catalog.listCatalogs in PySpark

[ https://issues.apache.org/jira/browse/SPARK-43527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43527. -- Fix Version/s: 3.5.0 3.4.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-43527) Fix catalog.listCatalogs in PySpark

[ https://issues.apache.org/jira/browse/SPARK-43527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43527: Assignee: Ruifeng Zheng > Fix catalog.listCatalogs in PySpark >

[jira] [Resolved] (SPARK-43360) Scala Connect: Add StreamingQueryManager API

[ https://issues.apache.org/jira/browse/SPARK-43360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43360. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41039

[jira] [Assigned] (SPARK-43360) Scala Connect: Add StreamingQueryManager API

[ https://issues.apache.org/jira/browse/SPARK-43360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43360: Assignee: Wei Liu > Scala Connect: Add StreamingQueryManager API >

[jira] [Created] (SPARK-43530) Protobuf: Read descriptor file only once at the compile time

Raghu Angadi created SPARK-43530: Summary: Protobuf: Read descriptor file only once at the compile time Key: SPARK-43530 URL: https://issues.apache.org/jira/browse/SPARK-43530 Project: Spark

[jira] [Created] (SPARK-43529) Support general expressions as OPTIONS values

Daniel created SPARK-43529: -- Summary: Support general expressions as OPTIONS values Key: SPARK-43529 URL: https://issues.apache.org/jira/browse/SPARK-43529 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-43528) Support duplicated field names in createDataFrame with pandas DataFrame.

Takuya Ueshin created SPARK-43528: - Summary: Support duplicated field names in createDataFrame with pandas DataFrame. Key: SPARK-43528 URL: https://issues.apache.org/jira/browse/SPARK-43528 Project:

[jira] [Resolved] (SPARK-42958) Refactor `CheckConnectJvmClientCompatibility` to compare client and avro

[ https://issues.apache.org/jira/browse/SPARK-42958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42958. --- Fix Version/s: 3.5.0 Assignee: Yang Jie Resolution: Fixed >

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Svyatoslav Semenyuk updated SPARK-43514: Environment: Scala version: 2.12.17 Test examples were executed inside Zeppelin

[jira] [Resolved] (SPARK-43043) Improve the performance of MapOutputTracker.updateMapOutput

[ https://issues.apache.org/jira/browse/SPARK-43043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-43043. -- Fix Version/s: 3.4.1 Resolution: Done > Improve the performance of

[jira] [Assigned] (SPARK-43359) DELETE from Hive table result in INTERNAL error

[ https://issues.apache.org/jira/browse/SPARK-43359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43359: - Assignee: BingKun Pan > DELETE from Hive table result in INTERNAL error >

[jira] [Resolved] (SPARK-43359) DELETE from Hive table result in INTERNAL error

[ https://issues.apache.org/jira/browse/SPARK-43359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43359. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41172

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723216#comment-17723216 ] Svyatoslav Semenyuk commented on SPARK-43514: - We applied "current workaround" to

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Svyatoslav Semenyuk updated SPARK-43514: Affects Version/s: 3.3.2 (was: 3.3.1) > Unexpected

[jira] [Resolved] (SPARK-43520) Upgrade mysql-connector-java from 8.0.32 to 8.0.33

[ https://issues.apache.org/jira/browse/SPARK-43520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43520. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41182

[jira] [Assigned] (SPARK-43520) Upgrade mysql-connector-java from 8.0.32 to 8.0.33

[ https://issues.apache.org/jira/browse/SPARK-43520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43520: - Assignee: BingKun Pan > Upgrade mysql-connector-java from 8.0.32 to 8.0.33 >

[jira] [Assigned] (SPARK-38469) Use error classes in org.apache.spark.network

[ https://issues.apache.org/jira/browse/SPARK-38469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38469: Assignee: Bo Zhang > Use error classes in org.apache.spark.network >

[jira] [Resolved] (SPARK-38469) Use error classes in org.apache.spark.network

[ https://issues.apache.org/jira/browse/SPARK-38469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38469. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41140

[jira] [Resolved] (SPARK-43512) Update stateStoreOperationsBenchmark to allow rocksdb jni upgrade

[ https://issues.apache.org/jira/browse/SPARK-43512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43512. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41175

[jira] [Assigned] (SPARK-43512) Update stateStoreOperationsBenchmark to allow rocksdb jni upgrade

[ https://issues.apache.org/jira/browse/SPARK-43512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43512: - Assignee: Anish Shrigondekar > Update stateStoreOperationsBenchmark to allow rocksdb

[jira] [Updated] (SPARK-43512) Update stateStoreOperationsBenchmark to allow rocksdb jni upgrade

[ https://issues.apache.org/jira/browse/SPARK-43512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43512: -- Issue Type: Test (was: Task) > Update stateStoreOperationsBenchmark to allow rocksdb jni

[jira] [Updated] (SPARK-43512) Update stateStoreOperationsBenchmark to allow rocksdb jni upgrade

[ https://issues.apache.org/jira/browse/SPARK-43512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43512: -- Affects Version/s: 3.5.0 (was: 3.4.0) > Update

[jira] [Commented] (SPARK-43522) Creating struct column occurs error 'org.apache.spark.sql.AnalysisException [DATATYPE_MISMATCH.CREATE_NAMED_STRUCT_WITHOUT_FOLDABLE_STRING]'

[ https://issues.apache.org/jira/browse/SPARK-43522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723158#comment-17723158 ] Jia Fan commented on SPARK-43522: - https://github.com/apache/spark/pull/41187 > Creating struct column

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: (was: image-2023-05-16-21-23-33-611.png) > when shuffle hash join is enabled, q95

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: (was: image-2023-05-16-21-22-44-532.png) > when shuffle hash join is enabled, q95

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: (was: image-2023-05-16-21-20-18-727.png) > when shuffle hash join is enabled, q95

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: (was: application_1684208757063_0028_90.html) > when shuffle hash join is enabled, q95

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: application_1684208757063_0028_90.html > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Description: Testing with 5TB dataset, the performance of q95 in tpcds deteriorates when shuffle hash join

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Description: Testing with 5TB dataset, the performance of q95 in tpcds deteriorates when shuffle hash join

[jira] [Updated] (SPARK-43527) Fix catalog.listCatalogs in PySpark

[ https://issues.apache.org/jira/browse/SPARK-43527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43527: -- Summary: Fix catalog.listCatalogs in PySpark (was: Fix catalog.listCatalogs) > Fix

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-16-21-28-11-514.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-16-21-28-44-163.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Description: Testing with 5TB dataset, the performance of q95 in tpcds deteriorates when shuffle hash join

[jira] [Created] (SPARK-43527) Fix catalog.listCatalogs

Ruifeng Zheng created SPARK-43527: - Summary: Fix catalog.listCatalogs Key: SPARK-43527 URL: https://issues.apache.org/jira/browse/SPARK-43527 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Description: Testing with 5TB dataset, the performance of q95 in tpcds deteriorates when shuffle hash join

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Description: Testing with 5TB dataset, the performance of q95 in tpcds deteriorates when shuffle hash join

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Description: Testing with 5TB dataset, the performance of q95 in tpcds deteriorates when shuffle hash join

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Description: Testing with 5TB dataset, the performance of q95 in tpcds deteriorates when shuffle hash join

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-16-21-24-09-182.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Description: Testing with 5TB dataset, the performance of q95 in tpcds deteriorates when shuffle hash join

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-16-21-23-35-237.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-16-21-23-33-611.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-16-21-22-16-170.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-16-21-22-44-532.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-16-21-21-35-493.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-16-21-20-18-727.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Description: Testing with 5TB dataset, the performance of q95 in tpcds deteriorates when shuffle hash join

[jira] [Created] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

caican created SPARK-43526: -- Summary: when shuffle hash join is enabled, q95 performance deteriorates Key: SPARK-43526 URL: https://issues.apache.org/jira/browse/SPARK-43526 Project: Spark Issue

[jira] [Assigned] (SPARK-39281) Speed up Timestamp type inference of legacy format in JSON/CSV data source

[ https://issues.apache.org/jira/browse/SPARK-39281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-39281: Assignee: Jia Fan > Speed up Timestamp type inference of legacy format in JSON/CSV data source >

[jira] [Resolved] (SPARK-39281) Speed up Timestamp type inference of legacy format in JSON/CSV data source

[ https://issues.apache.org/jira/browse/SPARK-39281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-39281. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41091

[jira] [Commented] (SPARK-43504) [K8S] Mounts the hadoop config map on the executor pod

[ https://issues.apache.org/jira/browse/SPARK-43504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723109#comment-17723109 ] Nikita Awasthi commented on SPARK-43504: User 'turboFei' has created a pull request for this

[jira] [Updated] (SPARK-43524) Memory leak in Spark UI

[ https://issues.apache.org/jira/browse/SPARK-43524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amine Bagdouri updated SPARK-43524: --- Description: We have a distributed Spark application running on Azure HDInsight using Spark

[jira] [Created] (SPARK-43525) Enhance ImportOrderChecker rules for `group.scala`

BingKun Pan created SPARK-43525: --- Summary: Enhance ImportOrderChecker rules for `group.scala` Key: SPARK-43525 URL: https://issues.apache.org/jira/browse/SPARK-43525 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-43518) Convert `_LEGACY_ERROR_TEMP_2029` to INTERNAL_ERROR

[ https://issues.apache.org/jira/browse/SPARK-43518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-43518: Assignee: BingKun Pan > Convert `_LEGACY_ERROR_TEMP_2029` to INTERNAL_ERROR >

[jira] [Resolved] (SPARK-43518) Convert `_LEGACY_ERROR_TEMP_2029` to INTERNAL_ERROR

[ https://issues.apache.org/jira/browse/SPARK-43518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-43518. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41179

[jira] [Created] (SPARK-43524) Memory leak in Spark UI

Amine Bagdouri created SPARK-43524: -- Summary: Memory leak in Spark UI Key: SPARK-43524 URL: https://issues.apache.org/jira/browse/SPARK-43524 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-43523) Memory leak in Spark UI

Amine Bagdouri created SPARK-43523: -- Summary: Memory leak in Spark UI Key: SPARK-43523 URL: https://issues.apache.org/jira/browse/SPARK-43523 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-43302) Make Python UDAF an AggregateFunction

[ https://issues.apache.org/jira/browse/SPARK-43302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723061#comment-17723061 ] ASF GitHub Bot commented on SPARK-43302: User 'cloud-fan' has created a pull request for this

[jira] [Commented] (SPARK-43518) Convert `_LEGACY_ERROR_TEMP_2029` to INTERNAL_ERROR

[ https://issues.apache.org/jira/browse/SPARK-43518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723059#comment-17723059 ] ASF GitHub Bot commented on SPARK-43518: User 'panbingkun' has created a pull request for this

[jira] [Assigned] (SPARK-43457) [PYTHON][CONNECT] user agent should include the OS and Python versions

[ https://issues.apache.org/jira/browse/SPARK-43457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43457: Assignee: Niranjan Jayakar > [PYTHON][CONNECT] user agent should include the OS and

[jira] [Resolved] (SPARK-43457) [PYTHON][CONNECT] user agent should include the OS and Python versions

[ https://issues.apache.org/jira/browse/SPARK-43457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43457. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41138

[jira] [Updated] (SPARK-43522) Creating struct column occurs error 'org.apache.spark.sql.AnalysisException [DATATYPE_MISMATCH.CREATE_NAMED_STRUCT_WITHOUT_FOLDABLE_STRING]'

[ https://issues.apache.org/jira/browse/SPARK-43522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heedo Lee updated SPARK-43522: -- Description: When creating a struct column in Dataframe, the code that ran without problems in

[jira] [Updated] (SPARK-43522) Creating struct column occurs error 'org.apache.spark.sql.AnalysisException [DATATYPE_MISMATCH.CREATE_NAMED_STRUCT_WITHOUT_FOLDABLE_STRING]'

[ https://issues.apache.org/jira/browse/SPARK-43522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heedo Lee updated SPARK-43522: -- Description: When creating a struct column in Dataframe, the code that ran without problems in

[jira] [Updated] (SPARK-43522) Creating struct column occurs error 'org.apache.spark.sql.AnalysisException [DATATYPE_MISMATCH.CREATE_NAMED_STRUCT_WITHOUT_FOLDABLE_STRING]'

[ https://issues.apache.org/jira/browse/SPARK-43522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heedo Lee updated SPARK-43522: -- Description: When creating a struct column in Dataframe, the code that ran without problems in

[jira] [Created] (SPARK-43522) Creating struct column occurs error 'org.apache.spark.sql.AnalysisException [DATATYPE_MISMATCH.CREATE_NAMED_STRUCT_WITHOUT_FOLDABLE_STRING]'

Heedo Lee created SPARK-43522: - Summary: Creating struct column occurs error 'org.apache.spark.sql.AnalysisException [DATATYPE_MISMATCH.CREATE_NAMED_STRUCT_WITHOUT_FOLDABLE_STRING]' Key: SPARK-43522 URL:

[jira] [Created] (SPARK-43521) Support CREATE TABLE LIKE FILE for PARQUET

melin created SPARK-43521: - Summary: Support CREATE TABLE LIKE FILE for PARQUET Key: SPARK-43521 URL: https://issues.apache.org/jira/browse/SPARK-43521 Project: Spark Issue Type: Bug