[jira] [Assigned] (SPARK-42330) Assign name to _LEGACY_ERROR_TEMP_2175

[ https://issues.apache.org/jira/browse/SPARK-42330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42330: Assignee: Koray Beyaz > Assign name to _LEGACY_ERROR_TEMP_2175 >

[jira] [Resolved] (SPARK-42330) Assign name to _LEGACY_ERROR_TEMP_2175

[ https://issues.apache.org/jira/browse/SPARK-42330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-42330. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Created] (SPARK-44652) Raise error when only one df is None

Amanda Liu created SPARK-44652: -- Summary: Raise error when only one df is None Key: SPARK-44652 URL: https://issues.apache.org/jira/browse/SPARK-44652 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44651) Make do-release-docker.sh compatible with Mac m2

Yuming Wang created SPARK-44651: --- Summary: Make do-release-docker.sh compatible with Mac m2 Key: SPARK-44651 URL: https://issues.apache.org/jira/browse/SPARK-44651 Project: Spark Issue Type:

[jira] [Created] (SPARK-44650) `spark.executor.defaultJavaOptions` Check illegal java options

dzcxzl created SPARK-44650: -- Summary: `spark.executor.defaultJavaOptions` Check illegal java options Key: SPARK-44650 URL: https://issues.apache.org/jira/browse/SPARK-44650 Project: Spark Issue

[jira] [Resolved] (SPARK-40664) Union in query can remove cache from the plan

[ https://issues.apache.org/jira/browse/SPARK-40664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-40664. - Resolution: Not A Problem > Union in query can remove cache from the plan >

[jira] [Resolved] (SPARK-44572) Clean up unused installers ASAP

[ https://issues.apache.org/jira/browse/SPARK-44572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44572. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42292

[jira] [Assigned] (SPARK-44572) Clean up unused installers ASAP

[ https://issues.apache.org/jira/browse/SPARK-44572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44572: - Assignee: Ruifeng Zheng > Clean up unused installers ASAP >

[jira] [Created] (SPARK-44649) Runtime Filter supports passing equivalent creation side expressions

jiaan.geng created SPARK-44649: -- Summary: Runtime Filter supports passing equivalent creation side expressions Key: SPARK-44649 URL: https://issues.apache.org/jira/browse/SPARK-44649 Project: Spark

[jira] [Commented] (SPARK-44265) Built-in XML data source support

[ https://issues.apache.org/jira/browse/SPARK-44265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750555#comment-17750555 ] Snoot.io commented on SPARK-44265: -- User 'sandip-db' has created a pull request for this issue:

[jira] [Updated] (SPARK-41636) DataSourceStrategy#selectFilters returns predicates in non-deterministic order

[ https://issues.apache.org/jira/browse/SPARK-41636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jia Fan updated SPARK-41636: Affects Version/s: 3.4.1 3.4.0 > DataSourceStrategy#selectFilters returns

[jira] [Commented] (SPARK-41636) DataSourceStrategy#selectFilters returns predicates in non-deterministic order

[ https://issues.apache.org/jira/browse/SPARK-41636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750547#comment-17750547 ] Snoot.io commented on SPARK-41636: -- User 'Hisoka-X' has created a pull request for this issue:

[jira] [Commented] (SPARK-41636) DataSourceStrategy#selectFilters returns predicates in non-deterministic order

[ https://issues.apache.org/jira/browse/SPARK-41636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750545#comment-17750545 ] Snoot.io commented on SPARK-41636: -- User 'Hisoka-X' has created a pull request for this issue:

[jira] [Commented] (SPARK-44572) Clean up unused installers ASAP

[ https://issues.apache.org/jira/browse/SPARK-44572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750544#comment-17750544 ] Snoot.io commented on SPARK-44572: -- User 'zhengruifeng' has created a pull request for this issue:

[jira] [Resolved] (SPARK-44645) Update assertDataFrameEqual docs error example output

[ https://issues.apache.org/jira/browse/SPARK-44645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44645. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44645) Update assertDataFrameEqual docs error example output

[ https://issues.apache.org/jira/browse/SPARK-44645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44645: Assignee: Amanda Liu > Update assertDataFrameEqual docs error example output >

[jira] [Assigned] (SPARK-42730) Update Spark Standalone Mode - Starting a Cluster Manually

[ https://issues.apache.org/jira/browse/SPARK-42730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42730: Assignee: Junyao Huang > Update Spark Standalone Mode - Starting a Cluster Manually >

[jira] [Resolved] (SPARK-42730) Update Spark Standalone Mode - Starting a Cluster Manually

[ https://issues.apache.org/jira/browse/SPARK-42730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42730. -- Fix Version/s: 3.5.0 4.0.0 3.4.2 Resolution:

[jira] [Resolved] (SPARK-44488) Support deserializing long fields into `Metadata` object

[ https://issues.apache.org/jira/browse/SPARK-44488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44488. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42083

[jira] [Assigned] (SPARK-44488) Support deserializing long fields into `Metadata` object

[ https://issues.apache.org/jira/browse/SPARK-44488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44488: Assignee: Richard Chen > Support deserializing long fields into `Metadata` object >

[jira] [Resolved] (SPARK-44643) __repr__ broken for Row when the field is empty Row

[ https://issues.apache.org/jira/browse/SPARK-44643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44643. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44643) __repr__ broken for Row when the field is empty Row

[ https://issues.apache.org/jira/browse/SPARK-44643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44643: Assignee: Takuya Ueshin > __repr__ broken for Row when the field is empty Row >

[jira] [Assigned] (SPARK-44620) Make `ResolvePivot` retain the `Plan_ID_TAG`

[ https://issues.apache.org/jira/browse/SPARK-44620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44620: Assignee: Ruifeng Zheng > Make `ResolvePivot` retain the `Plan_ID_TAG` >

[jira] [Resolved] (SPARK-44620) Make `ResolvePivot` retain the `Plan_ID_TAG`

[ https://issues.apache.org/jira/browse/SPARK-44620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44620. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-44648) Set up memory limits for analyze in Python.

Takuya Ueshin created SPARK-44648: - Summary: Set up memory limits for analyze in Python. Key: SPARK-44648 URL: https://issues.apache.org/jira/browse/SPARK-44648 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-44368) Support partition operation on dataframe in Spark Connect Go Client

[ https://issues.apache.org/jira/browse/SPARK-44368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44368: - Assignee: BoYang > Support partition operation on dataframe in Spark Connect Go Client

[jira] [Resolved] (SPARK-44368) Support partition operation on dataframe in Spark Connect Go Client

[ https://issues.apache.org/jira/browse/SPARK-44368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44368. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 13

[jira] [Commented] (SPARK-44503) Support PARTITION BY and ORDER BY clause for table arguments

[ https://issues.apache.org/jira/browse/SPARK-44503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750505#comment-17750505 ] Daniel commented on SPARK-44503: Here is where we call the "eval" method on each input row that the UDTF

[jira] [Comment Edited] (SPARK-44503) Support PARTITION BY and ORDER BY clause for table arguments

[ https://issues.apache.org/jira/browse/SPARK-44503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750505#comment-17750505 ] Daniel edited comment on SPARK-44503 at 8/2/23 11:51 PM: - Here is where we call

[jira] [Commented] (SPARK-42730) Update Spark Standalone Mode - Starting a Cluster Manually

[ https://issues.apache.org/jira/browse/SPARK-42730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750498#comment-17750498 ] Junyao Huang commented on SPARK-42730: -- Thanks [~gurwls223] . I linked my PR to this issues. feel

[jira] [Resolved] (SPARK-44424) Reattach to existing execute in Spark Connect (python client)

[ https://issues.apache.org/jira/browse/SPARK-44424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44424. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44424) Reattach to existing execute in Spark Connect (python client)

[ https://issues.apache.org/jira/browse/SPARK-44424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44424: Assignee: Hyukjin Kwon > Reattach to existing execute in Spark Connect (python client) >

[jira] [Updated] (SPARK-44641) Results duplicated when SPJ partial-cluster and pushdown enabled but conditions unmet

[ https://issues.apache.org/jira/browse/SPARK-44641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated SPARK-44641: -- Description: Adding the following test case in KeyGroupedPartitionSuite demonstrates the problem.  

[jira] [Created] (SPARK-44647) Support SPJ when join key is subset of partition keys

Szehon Ho created SPARK-44647: - Summary: Support SPJ when join key is subset of partition keys Key: SPARK-44647 URL: https://issues.apache.org/jira/browse/SPARK-44647 Project: Spark Issue Type:

[jira] [Created] (SPARK-44646) Migrate Log4j 2.x in Spark 3.4.1 to Logback

Yu Tian created SPARK-44646: --- Summary: Migrate Log4j 2.x in Spark 3.4.1 to Logback Key: SPARK-44646 URL: https://issues.apache.org/jira/browse/SPARK-44646 Project: Spark Issue Type: Brainstorming

[jira] [Updated] (SPARK-44645) Update assertDataFrameEqual docs error example output

[ https://issues.apache.org/jira/browse/SPARK-44645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amanda Liu updated SPARK-44645: --- Summary: Update assertDataFrameEqual docs error example output (was: Update assertDataFrame docs

[jira] [Created] (SPARK-44645) Update assertDataFrame docs error example output

Amanda Liu created SPARK-44645: -- Summary: Update assertDataFrame docs error example output Key: SPARK-44645 URL: https://issues.apache.org/jira/browse/SPARK-44645 Project: Spark Issue Type:

[jira] [Created] (SPARK-44644) Improve error messages for creating Python UDTFs with pickling errors

Allison Wang created SPARK-44644: Summary: Improve error messages for creating Python UDTFs with pickling errors Key: SPARK-44644 URL: https://issues.apache.org/jira/browse/SPARK-44644 Project: Spark

[jira] [Resolved] (SPARK-44626) Followup on streaming query termination when client session is timed out for Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-44626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44626. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44626) Followup on streaming query termination when client session is timed out for Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-44626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44626: Assignee: Bo Gao > Followup on streaming query termination when client session is timed

[jira] [Assigned] (SPARK-44636) Leave no dangling iterators in Spark Connect Scala

[ https://issues.apache.org/jira/browse/SPARK-44636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-44636: - Assignee: Alice Sayutina (was: Herman van Hövell) > Leave no dangling

[jira] [Resolved] (SPARK-44636) Leave no dangling iterators in Spark Connect Scala

[ https://issues.apache.org/jira/browse/SPARK-44636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44636. --- Fix Version/s: 3.5.0 Resolution: Fixed > Leave no dangling iterators in

[jira] [Assigned] (SPARK-44636) Leave no dangling iterators in Spark Connect Scala

[ https://issues.apache.org/jira/browse/SPARK-44636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-44636: - Assignee: Herman van Hövell > Leave no dangling iterators in Spark Connect

[jira] [Created] (SPARK-44643) __repr__ broken for Row when the field is empty Row

Takuya Ueshin created SPARK-44643: - Summary: __repr__ broken for Row when the field is empty Row Key: SPARK-44643 URL: https://issues.apache.org/jira/browse/SPARK-44643 Project: Spark Issue

[jira] [Resolved] (SPARK-44637) ExecuteRelease needs to synchronize

[ https://issues.apache.org/jira/browse/SPARK-44637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44637. --- Fix Version/s: 3.5.0 Assignee: Juliusz Sompolski Resolution: Fixed

[jira] [Updated] (SPARK-43754) Spark Connect Session & Query lifecycle

[ https://issues.apache.org/jira/browse/SPARK-43754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-43754: -- Epic Name: connect-query-lifecycle (was: sc-query-lifecycle) > Spark Connect Session

[jira] [Updated] (SPARK-43754) Spark Connect Session & Query lifecycle

[ https://issues.apache.org/jira/browse/SPARK-43754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-43754: -- Epic Name: sc-query-lifecycle (was: sc-session-lifecycle) > Spark Connect Session &

[jira] [Created] (SPARK-44642) ExecutePlanResponseReattachableIterator should release all after error

Juliusz Sompolski created SPARK-44642: - Summary: ExecutePlanResponseReattachableIterator should release all after error Key: SPARK-44642 URL: https://issues.apache.org/jira/browse/SPARK-44642

[jira] [Updated] (SPARK-44642) ExecutePlanResponseReattachableIterator should release all after error

[ https://issues.apache.org/jira/browse/SPARK-44642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-44642: -- Epic Link: SPARK-43754 > ExecutePlanResponseReattachableIterator should release all

[jira] [Updated] (SPARK-44636) Leave no dangling iterators in Spark Connect Scala

[ https://issues.apache.org/jira/browse/SPARK-44636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-44636: -- Epic Link: SPARK-43754 > Leave no dangling iterators in Spark Connect Scala >

[jira] [Updated] (SPARK-44641) Results duplicated when SPJ partial-cluster and pushdown enabled but conditions unmet

[ https://issues.apache.org/jira/browse/SPARK-44641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated SPARK-44641: -- Description: Adding the following test case in KeyGroupedPartitionSuite demonstrates the problem.  

[jira] [Created] (SPARK-44641) Results duplicated when SPJ partial-cluster and pushdown enabled but conditions unmet

Szehon Ho created SPARK-44641: - Summary: Results duplicated when SPJ partial-cluster and pushdown enabled but conditions unmet Key: SPARK-44641 URL: https://issues.apache.org/jira/browse/SPARK-44641

[jira] [Updated] (SPARK-44640) Improve error messages for Python UDTF returning non iterable

[ https://issues.apache.org/jira/browse/SPARK-44640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-44640: - Summary: Improve error messages for Python UDTF returning non iterable (was: Improve error

[jira] [Created] (SPARK-44640) Improve error messages for invalid Python UDTF return type

Allison Wang created SPARK-44640: Summary: Improve error messages for invalid Python UDTF return type Key: SPARK-44640 URL: https://issues.apache.org/jira/browse/SPARK-44640 Project: Spark

[jira] [Created] (SPARK-44639) Add option to use Java tmp dir for RocksDB state store

Adam Binford created SPARK-44639: Summary: Add option to use Java tmp dir for RocksDB state store Key: SPARK-44639 URL: https://issues.apache.org/jira/browse/SPARK-44639 Project: Spark Issue

[jira] [Updated] (SPARK-44561) Fix AssertionError when converting UDTF output to a complex type

[ https://issues.apache.org/jira/browse/SPARK-44561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44561: -- Fix Version/s: (was: 4.0.0) > Fix AssertionError when converting UDTF output to a complex

[jira] [Assigned] (SPARK-44561) Fix AssertionError when converting UDTF output to a complex type

[ https://issues.apache.org/jira/browse/SPARK-44561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-44561: - Assignee: (was: Allison Wang) > Fix AssertionError when converting UDTF output to

[jira] [Assigned] (SPARK-44559) Improve error messages for Python UDTF arrow type casts

[ https://issues.apache.org/jira/browse/SPARK-44559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-44559: - Assignee: Allison Wang > Improve error messages for Python UDTF arrow type casts >

[jira] (SPARK-44561) Fix AssertionError when converting UDTF output to a complex type

[ https://issues.apache.org/jira/browse/SPARK-44561 ] Takuya Ueshin deleted comment on SPARK-44561: --- was (Author: ueshin): Issue resolved by pull request 42191 https://github.com/apache/spark/pull/42191 > Fix AssertionError when converting

[jira] (SPARK-44559) Improve error messages for Python UDTF arrow type casts

[ https://issues.apache.org/jira/browse/SPARK-44559 ] Allison Wang deleted comment on SPARK-44559: -- was (Author: allisonwang-db): Resolved by [https://github.com/apache/spark/pull/42191] > Improve error messages for Python UDTF arrow type

[jira] [Reopened] (SPARK-44561) Fix AssertionError when converting UDTF output to a complex type

[ https://issues.apache.org/jira/browse/SPARK-44561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang reopened SPARK-44561: -- > Fix AssertionError when converting UDTF output to a complex type >

[jira] [Resolved] (SPARK-44559) Improve error messages for Python UDTF arrow type casts

[ https://issues.apache.org/jira/browse/SPARK-44559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang resolved SPARK-44559. -- Fix Version/s: 3.5.0 Target Version/s: 3.5.0 Resolution: Fixed Resolved by

[jira] [Commented] (SPARK-44559) Improve error messages for Python UDTF arrow type casts

[ https://issues.apache.org/jira/browse/SPARK-44559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750419#comment-17750419 ] Allison Wang commented on SPARK-44559: -- Resolved by [https://github.com/apache/spark/pull/42191] >

[jira] [Updated] (SPARK-44638) Unable to read from JDBC data sources when using custom schema containing varchar

[ https://issues.apache.org/jira/browse/SPARK-44638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Said updated SPARK-44638: - Description: When querying the data from JDBC databases with custom schema containing varchar

[jira] [Updated] (SPARK-44638) Unable to read from JDBC data sources when using custom schema containing varchar

[ https://issues.apache.org/jira/browse/SPARK-44638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Said updated SPARK-44638: - Description: When querying the data from JDBC databases with custom schema containing varchar

[jira] [Created] (SPARK-44638) Unable to read from JDBC data sources when using custom schema containing varchar

Michael Said created SPARK-44638: Summary: Unable to read from JDBC data sources when using custom schema containing varchar Key: SPARK-44638 URL: https://issues.apache.org/jira/browse/SPARK-44638

[jira] [Updated] (SPARK-44637) ExecuteRelease needs to synchronize

[ https://issues.apache.org/jira/browse/SPARK-44637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-44637: -- Epic Link: SPARK-43754 > ExecuteRelease needs to synchronize >

[jira] [Created] (SPARK-44637) ExecuteRelease needs to synchronize

Juliusz Sompolski created SPARK-44637: - Summary: ExecuteRelease needs to synchronize Key: SPARK-44637 URL: https://issues.apache.org/jira/browse/SPARK-44637 Project: Spark Issue Type:

[jira] [Updated] (SPARK-38506) Push partial aggregation through join

[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38506: Description: Please see

[jira] [Updated] (SPARK-44247) Upgrade Arrow to 13.0.0

[ https://issues.apache.org/jira/browse/SPARK-44247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44247: -- Affects Version/s: 4.0.0 (was: 3.5.0) > Upgrade Arrow to 13.0.0 >

[jira] [Updated] (SPARK-44609) ExecutorPodsAllocator doesn't create new executors if no pod snapshot captured pod creation

[ https://issues.apache.org/jira/browse/SPARK-44609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alibi Yeslambek updated SPARK-44609: Description: There’s a following race condition in ExecutorPodsAllocator when running a

[jira] [Updated] (SPARK-44634) Encoders.bean does no longer support nested beans with type arguments

[ https://issues.apache.org/jira/browse/SPARK-44634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giambattista Bloisi updated SPARK-44634: Description: Hi,   while upgrading a project from spark 2.4.0 to 3.4.1 version,

[jira] [Updated] (SPARK-44634) Encoders.bean does no longer support nested beans with type arguments

[ https://issues.apache.org/jira/browse/SPARK-44634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giambattista Bloisi updated SPARK-44634: Description: Hi,   while upgrading a project from spark 2.4.0 to 3.4.1 version,

[jira] [Created] (SPARK-44636) Leave no dangling iterators in Spark Connect Scala

Alice Sayutina created SPARK-44636: -- Summary: Leave no dangling iterators in Spark Connect Scala Key: SPARK-44636 URL: https://issues.apache.org/jira/browse/SPARK-44636 Project: Spark Issue

[jira] [Updated] (SPARK-44609) ExecutorPodsAllocator doesn't create new executors if no pod snapshot captured pod creation

[ https://issues.apache.org/jira/browse/SPARK-44609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alibi Yeslambek updated SPARK-44609: Component/s: Scheduler > ExecutorPodsAllocator doesn't create new executors if no pod

[jira] [Updated] (SPARK-44609) ExecutorPodsAllocator doesn't create new executors if no pod snapshot captured pod creation

[ https://issues.apache.org/jira/browse/SPARK-44609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alibi Yeslambek updated SPARK-44609: Component/s: Kubernetes (was: Scheduler) > ExecutorPodsAllocator

[jira] [Commented] (SPARK-44581) ShutdownHookManager get wrong hadoop user group information

[ https://issues.apache.org/jira/browse/SPARK-44581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750229#comment-17750229 ] liang yu commented on SPARK-44581: -- I found that the ShutDownHook Manager will start a new Thread when

[jira] [Updated] (SPARK-44635) Handle shuffle fetch failures in decommissions

[ https://issues.apache.org/jira/browse/SPARK-44635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Zhang updated SPARK-44635: - Description: Spark's decommission feature supports migration of shuffle data. However shuffle data

[jira] [Created] (SPARK-44635) Handle shuffle fetch failures in decommissions

Bo Zhang created SPARK-44635: Summary: Handle shuffle fetch failures in decommissions Key: SPARK-44635 URL: https://issues.apache.org/jira/browse/SPARK-44635 Project: Spark Issue Type:

[jira] [Created] (SPARK-44634) Encoders.bean does no longer support nested beans with type arguments

Giambattista Bloisi created SPARK-44634: --- Summary: Encoders.bean does no longer support nested beans with type arguments Key: SPARK-44634 URL: https://issues.apache.org/jira/browse/SPARK-44634

[jira] [Commented] (SPARK-44562) Add OptimizeOneRowRelationSubquery in batch of Subquery

[ https://issues.apache.org/jira/browse/SPARK-44562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750203#comment-17750203 ] ASF GitHub Bot commented on SPARK-44562: User 'wangyum' has created a pull request for this

[jira] [Assigned] (SPARK-44562) Add OptimizeOneRowRelationSubquery in batch of Subquery

[ https://issues.apache.org/jira/browse/SPARK-44562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44562: --- Assignee: Yuming Wang > Add OptimizeOneRowRelationSubquery in batch of Subquery >

[jira] [Resolved] (SPARK-44562) Add OptimizeOneRowRelationSubquery in batch of Subquery

[ https://issues.apache.org/jira/browse/SPARK-44562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44562. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42180

[jira] [Updated] (SPARK-44633) pandas-on-Spark Dataframe.between_time fails when timestamp fields are present

[ https://issues.apache.org/jira/browse/SPARK-44633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tewei Luo updated SPARK-44633: -- Attachment: image.png > pandas-on-Spark Dataframe.between_time fails when timestamp fields are

[jira] [Updated] (SPARK-44633) pandas-on-Spark Dataframe.between_time fails when timestamp fields are present

[ https://issues.apache.org/jira/browse/SPARK-44633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tewei Luo updated SPARK-44633: -- Description: I tried to execute the between_time() method of a pandas-on-Spark dataframe that has a

[jira] [Resolved] (SPARK-44631) Remove session-based directory when the isolated session cache is evicted

[ https://issues.apache.org/jira/browse/SPARK-44631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44631. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44631) Remove session-based directory when the isolated session cache is evicted

[ https://issues.apache.org/jira/browse/SPARK-44631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44631: Assignee: Hyukjin Kwon > Remove session-based directory when the isolated session cache

[jira] [Created] (SPARK-44633) pandas-on-Spark Dataframe.between_time fails when timestamp fields are present

Tewei Luo created SPARK-44633: - Summary: pandas-on-Spark Dataframe.between_time fails when timestamp fields are present Key: SPARK-44633 URL: https://issues.apache.org/jira/browse/SPARK-44633 Project:

[jira] [Resolved] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44627. -- Resolution: Not A Problem I have checked with the reporter offline. This issue only exists in the

[jira] [Commented] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750147#comment-17750147 ] Kent Yao commented on SPARK-44627: -- Similar to SPARK-44280？ >

[jira] [Assigned] (SPARK-44280) Add convertJavaTimestampToTimestamp in JDBCDialect API

[ https://issues.apache.org/jira/browse/SPARK-44280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44280: --- Assignee: Mingkang Li > Add convertJavaTimestampToTimestamp in JDBCDialect API >

[jira] [Resolved] (SPARK-44280) Add convertJavaTimestampToTimestamp in JDBCDialect API

[ https://issues.apache.org/jira/browse/SPARK-44280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44280. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41843

[jira] [Updated] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhao updated SPARK-44627: - Description: When the resultSet exists a timestmp column and it's value is null, but column define is

[jira] [Updated] (SPARK-44572) Clean up unused installers ASAP

[ https://issues.apache.org/jira/browse/SPARK-44572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44572: -- Summary: Clean up unused installers ASAP (was: Clean up unused installer ASAP) > Clean up

[jira] [Updated] (SPARK-44572) Clean up unused installer ASAP

[ https://issues.apache.org/jira/browse/SPARK-44572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44572: -- Summary: Clean up unused installer ASAP (was: Clean up unused files ASAP) > Clean up unused

[jira] [Updated] (SPARK-43043) Improve the performance of MapOutputTracker.updateMapOutput

[ https://issues.apache.org/jira/browse/SPARK-43043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-43043: - Fix Version/s: 3.5.0 (was: 3.4.1) > Improve the performance of

[jira] [Assigned] (SPARK-44630) Revert SPARK-43043 Improve the performance of MapOutputTracker.updateMapOutput

[ https://issues.apache.org/jira/browse/SPARK-44630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44630: Assignee: Dongjoon Hyun > Revert SPARK-43043 Improve the performance of

[jira] [Resolved] (SPARK-44630) Revert SPARK-43043 Improve the performance of MapOutputTracker.updateMapOutput

[ https://issues.apache.org/jira/browse/SPARK-44630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44630. -- Fix Version/s: 3.4.2 Resolution: Fixed Issue resolved by pull request 42285

[jira] [Updated] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhao updated SPARK-44627: - Priority: Minor (was: Major) >

[jira] [Commented] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750115#comment-17750115 ] Min Zhao commented on SPARK-44627: -- !image-2023-08-02-14-01-54-447.png! it only update isNull to true,

  1   2   >