[jira] [Updated] (SPARK-44032) Remove threeten-extra exclusion in enforceBytecodeVersion rule

2023-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44032: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Test) > Remove threeten-extra

[jira] [Updated] (SPARK-44032) Remove threeten-extra exclusion in enforceBytecodeVersion rule

2023-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44032: -- Affects Version/s: 4.0.0 (was: 3.4.0) > Remove threeten-extra

[jira] [Updated] (SPARK-44032) Remove threeten-extra exclusion in enforceBytecodeVersion rule

2023-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44032: -- Description: We can remove `threeten-extra` library exclusion rule because Apache Spark

[jira] [Updated] (SPARK-44032) Remove threeten-extra exclusion in enforceBytecodeVersion rule

2023-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44032: -- Description: Apache Spark 4.0 (was: Remove the exclusion of threeten-extra when orc-core

[jira] [Updated] (SPARK-44032) Remove threeten-extra exclusion in enforceBytecodeVersion rule

2023-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44032: -- Parent: (was: SPARK-44022) Issue Type: Test (was: Sub-task) > Remove

[jira] [Updated] (SPARK-44032) Remove threeten-extra exclusion in enforceBytecodeVersion rule

2023-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44032: -- Summary: Remove threeten-extra exclusion in enforceBytecodeVersion rule (was: Remove the

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Description: A Spark job ran successfully with Spark 3.2.x and 3.3.x. But after upgrading to

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Description: A Spark job ran successfully with Spark 3.2.x and 3.3.x. But after upgrading to

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Description: A Spark job ran successfully with Spark 3.2.x and 3.3.x. But after upgrading to

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Description: A Spark job ran successfully with Spark 3.2.x and 3.3.x. But after upgrading to

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Summary: After upgrading to Spark 3.4.1 and 3.5.0 we receive RuntimeException "is not valid

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Description: A Spark job ran successfully with Spark 3.2.x and 3.3.x. But after upgrading to

[jira] [Updated] (SPARK-44032) Remove the bytecode version exclusion of threeten-extra when orc-core fixes the violation

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44032: --- Labels: pull-request-available (was: ) > Remove the bytecode version exclusion of

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Description: A Spark job ran successfully with Spark 3.2.x and 3.3.x. But after upgrading to

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Description: A Spark job ran successfully with Spark 3.2.x and 3.3.x. But after upgrading to

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Description: A Spark job ran successfully with Spark 3.2.x and 3.3.x. But after upgrading to

[jira] [Resolved] (SPARK-45618) Remove BaseErrorHandler

2023-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45618. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43468

[jira] [Assigned] (SPARK-45618) Remove BaseErrorHandler

2023-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45618: - Assignee: L. C. Hsieh > Remove BaseErrorHandler > --- > >

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Description: A Spark job ran successfully with Spark 3.2.x and 3.3.x. But after upgrading to

[jira] [Updated] (SPARK-44790) XML: to_xml

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44790: --- Labels: pull-request-available (was: ) > XML: to_xml > --- > >

[jira] [Updated] (SPARK-45644) After upgrading to Spark 3.4.1 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adi Wehrli updated SPARK-45644: --- Component/s: SQL > After upgrading to Spark 3.4.1 we receive RuntimeException "is not valid >

[jira] [Created] (SPARK-45644) After upgrading to Spark 3.4.1 we receive RuntimeException "is not valid external type"

2023-10-23 Thread Adi Wehrli (Jira)
Adi Wehrli created SPARK-45644: -- Summary: After upgrading to Spark 3.4.1 we receive RuntimeException "is not valid external type" Key: SPARK-45644 URL: https://issues.apache.org/jira/browse/SPARK-45644

[jira] [Updated] (SPARK-45533) Use `j.l.r.Cleaner` instead of `finalize` for `RocksDBIterator/LevelDBIterator`

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45533: --- Labels: pull-request-available (was: ) > Use `j.l.r.Cleaner` instead of `finalize` for >

[jira] [Comment Edited] (SPARK-45533) Use `j.l.r.Cleaner` instead of `finalize` for `RocksDBIterator/LevelDBIterator`

2023-10-23 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778917#comment-17778917 ] Min Zhao edited comment on SPARK-45533 at 10/24/23 4:12 AM: Thank you very

[jira] [Commented] (SPARK-45533) Use `j.l.r.Cleaner` instead of `finalize` for `RocksDBIterator/LevelDBIterator`

2023-10-23 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778917#comment-17778917 ] Min Zhao commented on SPARK-45533: -- Thank you very much. [~LuciferYang]  The close method of RocksDB

[jira] [Updated] (SPARK-45643) Replace `s.c.mutable.MapOps#transform` with `s.c.mutable.MapOps#mapValuesInPlace`

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45643: --- Labels: pull-request-available (was: ) > Replace `s.c.mutable.MapOps#transform` with >

[jira] [Commented] (SPARK-45642) Fix `FileSystem.isFile & FileSystem.isDirectory is deprecated`

2023-10-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778915#comment-17778915 ] Yang Jie commented on SPARK-45642: -- [~panbingkun] I move this one out of `Drop Scala 2.12 and make

[jira] [Updated] (SPARK-45642) Fix `FileSystem.isFile & FileSystem.isDirectory is deprecated`

2023-10-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45642: - Parent: (was: SPARK-45314) Issue Type: Improvement (was: Sub-task) > Fix

[jira] [Created] (SPARK-45643) Replace `s.c.mutable.MapOps#transform` with `s.c.mutable.MapOps#mapValuesInPlace`

2023-10-23 Thread Yang Jie (Jira)
Yang Jie created SPARK-45643: Summary: Replace `s.c.mutable.MapOps#transform` with `s.c.mutable.MapOps#mapValuesInPlace` Key: SPARK-45643 URL: https://issues.apache.org/jira/browse/SPARK-45643 Project:

[jira] [Updated] (SPARK-45551) Enhance PySpark testing utils

2023-10-23 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-45551: Summary: Enhance PySpark testing utils (was: Enchance PySpark testing utils) > Enhance PySpark

[jira] [Created] (SPARK-45642) Fix `FileSystem.isFile & FileSystem.isDirectory is deprecated`

2023-10-23 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-45642: --- Summary: Fix `FileSystem.isFile & FileSystem.isDirectory is deprecated` Key: SPARK-45642 URL: https://issues.apache.org/jira/browse/SPARK-45642 Project: Spark

[jira] [Updated] (SPARK-45641) Display the start time of an app following the Total Uptime segment on AllJobsPage

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45641: --- Labels: pull-request-available (was: ) > Display the start time of an app following the

[jira] [Updated] (SPARK-45527) Task fraction resource request is not expected

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45527: --- Labels: pull-request-available (was: ) > Task fraction resource request is not expected >

[jira] [Updated] (SPARK-45640) Fix flaky ProtobufCatalystDataConversionSuite

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45640: --- Labels: pull-request-available (was: ) > Fix flaky ProtobufCatalystDataConversionSuite >

[jira] [Created] (SPARK-45641) Display the start time of an app following the Total Uptime segment on AllJobsPage

2023-10-23 Thread Kent Yao (Jira)
Kent Yao created SPARK-45641: Summary: Display the start time of an app following the Total Uptime segment on AllJobsPage Key: SPARK-45641 URL: https://issues.apache.org/jira/browse/SPARK-45641 Project:

[jira] [Created] (SPARK-45640) Fix flaky ProtobufCatalystDataConversionSuite

2023-10-23 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-45640: --- Summary: Fix flaky ProtobufCatalystDataConversionSuite Key: SPARK-45640 URL: https://issues.apache.org/jira/browse/SPARK-45640 Project: Spark Issue Type:

[jira] [Closed] (SPARK-44117) Spark Connect GA

2023-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-44117. - > Spark Connect GA > > > Key: SPARK-44117 > URL:

[jira] [Updated] (SPARK-44443) Use PartitionEvaluator API in CoGroupExec, DeserializeToObjectExec, ExternalRDDScanExec

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-3: --- Labels: pull-request-available (was: ) > Use PartitionEvaluator API in CoGroupExec,

[jira] [Resolved] (SPARK-45547) Validate Vectors with built-in function

2023-10-23 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45547. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43380

[jira] [Assigned] (SPARK-45547) Validate Vectors with built-in function

2023-10-23 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-45547: - Assignee: Ruifeng Zheng > Validate Vectors with built-in function >

[jira] [Assigned] (SPARK-45619) Apply the observed metrics to Observation object.

2023-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45619: Assignee: Takuya Ueshin > Apply the observed metrics to Observation object. >

[jira] [Resolved] (SPARK-45619) Apply the observed metrics to Observation object.

2023-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45619. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43469

[jira] [Updated] (SPARK-45639) Support Python data source in DataFrameReader

2023-10-23 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45639: - Description: Allow users to read from a Python data source using

[jira] [Updated] (SPARK-45639) Support Python data source in DataFrameReader

2023-10-23 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45639: - Description: Allow users to read from a Python data source using

[jira] [Updated] (SPARK-45639) Support Python data source in DataFrameReader

2023-10-23 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45639: - Description: Allow users to read from a Python data source using

[jira] [Created] (SPARK-45639) Support Python data source in DataFrameReader

2023-10-23 Thread Allison Wang (Jira)
Allison Wang created SPARK-45639: Summary: Support Python data source in DataFrameReader Key: SPARK-45639 URL: https://issues.apache.org/jira/browse/SPARK-45639 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45524) Initial support for Python data source read API

2023-10-23 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45524: - Description: Add API for data source and data source reader and add Catalyst + execution

[jira] [Resolved] (SPARK-45623) Move `mllib` and `mllib-local` to separate test groups.

2023-10-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45623. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43432

[jira] [Updated] (SPARK-45637) Time window aggregation in separate streams followed by stream-stream join not returning results

2023-10-23 Thread Andrzej Zera (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Zera updated SPARK-45637: - Description: According to documentation update (SPARK-42591) resulting from SPARK-42376, Spark

[jira] [Created] (SPARK-45637) Time window aggregation in separate streams followed by stream-stream join not returning results

2023-10-23 Thread Andrzej Zera (Jira)
Andrzej Zera created SPARK-45637: Summary: Time window aggregation in separate streams followed by stream-stream join not returning results Key: SPARK-45637 URL: https://issues.apache.org/jira/browse/SPARK-45637

[jira] [Commented] (SPARK-45533) Use `j.l.r.Cleaner` instead of `finalize` for `RocksDBIterator/LevelDBIterator`

2023-10-23 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778729#comment-17778729 ] Min Zhao commented on SPARK-45533: -- I read about their implementation, using java.lang.ref.cleaner for

[jira] [Updated] (SPARK-45636) Upgrade jersey to 2.41

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45636: --- Labels: pull-request-available (was: ) > Upgrade jersey to 2.41 > -- >

[jira] [Created] (SPARK-45636) Upgrade jersey to 2.41

2023-10-23 Thread Yang Jie (Jira)
Yang Jie created SPARK-45636: Summary: Upgrade jersey to 2.41 Key: SPARK-45636 URL: https://issues.apache.org/jira/browse/SPARK-45636 Project: Spark Issue Type: Improvement Components:

[jira] [Assigned] (SPARK-45627) Fix `symbol literal is deprecated`

2023-10-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-45627: Assignee: BingKun Pan > Fix `symbol literal is deprecated` > --

[jira] [Resolved] (SPARK-45627) Fix `symbol literal is deprecated`

2023-10-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45627. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43487

[jira] [Assigned] (SPARK-45631) Broken backward compatibility in PySpark: StreamingQueryListener due to the addition of onQueryIdle

2023-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-45631: Assignee: Jungtaek Lim > Broken backward compatibility in PySpark:

[jira] [Resolved] (SPARK-45631) Broken backward compatibility in PySpark: StreamingQueryListener due to the addition of onQueryIdle

2023-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-45631. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-45635) Cleanup unused import for PySpark testing

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45635: --- Labels: pull-request-available (was: ) > Cleanup unused import for PySpark testing >

[jira] [Created] (SPARK-45635) Cleanup unused import for PySpark testing

2023-10-23 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-45635: --- Summary: Cleanup unused import for PySpark testing Key: SPARK-45635 URL: https://issues.apache.org/jira/browse/SPARK-45635 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-45634) Remove `get_dtype_counts` from Pandas API on Spark

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45634: --- Labels: pull-request-available (was: ) > Remove `get_dtype_counts` from Pandas API on

[jira] [Created] (SPARK-45634) Remove `get_dtype_counts` from Pandas API on Spark

2023-10-23 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-45634: --- Summary: Remove `get_dtype_counts` from Pandas API on Spark Key: SPARK-45634 URL: https://issues.apache.org/jira/browse/SPARK-45634 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-45633) Make SQL test group depend on `analyze_udtf`

2023-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45633: Assignee: Ruifeng Zheng > Make SQL test group depend on `analyze_udtf` >

[jira] [Resolved] (SPARK-45633) Make SQL test group depend on `analyze_udtf`

2023-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45633. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43486

[jira] [Updated] (SPARK-45627) Fix `symbol literal is deprecated`

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45627: --- Labels: pull-request-available (was: ) > Fix `symbol literal is deprecated` >

[jira] [Assigned] (SPARK-45511) SPIP: State Data Source - Reader

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45511: -- Assignee: (was: Apache Spark) > SPIP: State Data Source - Reader >

[jira] [Assigned] (SPARK-45511) SPIP: State Data Source - Reader

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45511: -- Assignee: Apache Spark > SPIP: State Data Source - Reader >

[jira] [Assigned] (SPARK-45511) SPIP: State Data Source - Reader

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45511: -- Assignee: (was: Apache Spark) > SPIP: State Data Source - Reader >

[jira] [Assigned] (SPARK-45511) SPIP: State Data Source - Reader

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45511: -- Assignee: Apache Spark > SPIP: State Data Source - Reader >

[jira] [Assigned] (SPARK-45511) SPIP: State Data Source - Reader

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45511: -- Assignee: (was: Apache Spark) > SPIP: State Data Source - Reader >

[jira] [Assigned] (SPARK-45631) Broken backward compatibility in PySpark: StreamingQueryListener due to the addition of onQueryIdle

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45631: -- Assignee: (was: Apache Spark) > Broken backward compatibility in PySpark:

[jira] [Assigned] (SPARK-45631) Broken backward compatibility in PySpark: StreamingQueryListener due to the addition of onQueryIdle

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45631: -- Assignee: Apache Spark > Broken backward compatibility in PySpark:

[jira] [Assigned] (SPARK-45511) SPIP: State Data Source - Reader

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45511: -- Assignee: Apache Spark > SPIP: State Data Source - Reader >

[jira] [Assigned] (SPARK-45631) Broken backward compatibility in PySpark: StreamingQueryListener due to the addition of onQueryIdle

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45631: -- Assignee: (was: Apache Spark) > Broken backward compatibility in PySpark:

[jira] [Assigned] (SPARK-45631) Broken backward compatibility in PySpark: StreamingQueryListener due to the addition of onQueryIdle

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45631: -- Assignee: Apache Spark > Broken backward compatibility in PySpark:

[jira] [Assigned] (SPARK-45209) Flame Graph Support For Executor Thread Dump Page

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45209: -- Assignee: Apache Spark > Flame Graph Support For Executor Thread Dump Page >

[jira] [Assigned] (SPARK-45209) Flame Graph Support For Executor Thread Dump Page

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45209: -- Assignee: (was: Apache Spark) > Flame Graph Support For Executor Thread Dump

[jira] [Assigned] (SPARK-45209) Flame Graph Support For Executor Thread Dump Page

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45209: -- Assignee: Apache Spark > Flame Graph Support For Executor Thread Dump Page >

[jira] [Assigned] (SPARK-45209) Flame Graph Support For Executor Thread Dump Page

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45209: -- Assignee: (was: Apache Spark) > Flame Graph Support For Executor Thread Dump

[jira] [Assigned] (SPARK-44752) XML: Update Spark Docs

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-44752: -- Assignee: (was: Apache Spark) > XML: Update Spark Docs > --

[jira] [Assigned] (SPARK-44752) XML: Update Spark Docs

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-44752: -- Assignee: Apache Spark > XML: Update Spark Docs > -- > >

[jira] [Updated] (SPARK-45633) Make SQL test group depend on `analyze_udtf`

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45633: --- Labels: pull-request-available (was: ) > Make SQL test group depend on `analyze_udtf` >

[jira] [Created] (SPARK-45633) Make SQL test group depend on `analyze_udtf`

2023-10-23 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-45633: - Summary: Make SQL test group depend on `analyze_udtf` Key: SPARK-45633 URL: https://issues.apache.org/jira/browse/SPARK-45633 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45632) Table cache should avoid unnecessary ColumnarToRow when enable AQE

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45632: --- Labels: pull-request-available (was: ) > Table cache should avoid unnecessary

[jira] [Created] (SPARK-45632) Table cache should avoid unnecessary ColumnarToRow when enable AQE

2023-10-23 Thread XiDuo You (Jira)
XiDuo You created SPARK-45632: - Summary: Table cache should avoid unnecessary ColumnarToRow when enable AQE Key: SPARK-45632 URL: https://issues.apache.org/jira/browse/SPARK-45632 Project: Spark

[jira] [Resolved] (SPARK-45628) improve vector_to_array for .mllib.linalg.SparseVector

2023-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45628. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43480

[jira] [Assigned] (SPARK-45628) improve vector_to_array for .mllib.linalg.SparseVector

2023-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45628: Assignee: Ruifeng Zheng > improve vector_to_array for .mllib.linalg.SparseVector >

[jira] [Updated] (SPARK-45631) Broken backward compatibility in PySpark: StreamingQueryListener due to the addition of onQueryIdle

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45631: --- Labels: pull-request-available (was: ) > Broken backward compatibility in PySpark:

[jira] [Commented] (SPARK-45631) Broken backward compatibility in PySpark: StreamingQueryListener due to the addition of onQueryIdle

2023-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778531#comment-17778531 ] Jungtaek Lim commented on SPARK-45631: -- Will submit a PR shortly. > Broken backward compatibility

[jira] [Created] (SPARK-45631) Broken backward compatibility in PySpark: StreamingQueryListener due to the addition of onQueryIdle

2023-10-23 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-45631: Summary: Broken backward compatibility in PySpark: StreamingQueryListener due to the addition of onQueryIdle Key: SPARK-45631 URL:

[jira] [Updated] (SPARK-45630) Replace `s.c.mutable.MapOps#retain` with `s.c.mutable.MapOps#filterInPlace`

2023-10-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45630: --- Labels: pull-request-available (was: ) > Replace `s.c.mutable.MapOps#retain` with

[jira] [Created] (SPARK-45630) Replace `s.c.mutable.MapOps#retain` with `s.c.mutable.MapOps#filterInPlace`

2023-10-23 Thread Yang Jie (Jira)
Yang Jie created SPARK-45630: Summary: Replace `s.c.mutable.MapOps#retain` with `s.c.mutable.MapOps#filterInPlace` Key: SPARK-45630 URL: https://issues.apache.org/jira/browse/SPARK-45630 Project: Spark