[jira] [Created] (SPARK-48652) Casting Issue in Spark SQL: String Column Compared to Integer Value Yields Empty Results

2024-06-18 Thread Abhishek Singh (Jira)
Abhishek Singh created SPARK-48652: -- Summary: Casting Issue in Spark SQL: String Column Compared to Integer Value Yields Empty Results Key: SPARK-48652 URL: https://issues.apache.org/jira/browse/SPARK-48652

[jira] [Updated] (SPARK-48573) Upgrade ICU version

2024-06-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48573: --- Labels: pull-request-available (was: ) > Upgrade ICU vers

[jira] [Updated] (SPARK-48573) Upgrade ICU version

2024-06-18 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48573: -- Parent: (was: SPARK-46837) Issue Type: Bug (was: Sub-task) > Upgrade

[jira] [Updated] (SPARK-48573) Upgrade ICU version

2024-06-18 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48573: -- Epic Link: SPARK-46830 > Upgrade ICU vers

[jira] [Updated] (SPARK-48573) Upgrade ICU version

2024-06-18 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48573: -- Summary: Upgrade ICU version (was: TBD) > Upgrade ICU vers

[jira] [Updated] (SPARK-48651) Document configuring different JDK for Spark on YARN

2024-06-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48651: --- Labels: pull-request-available (was: ) > Document configuring different JDK for Sp

[jira] [Created] (SPARK-48651) Document configuring different JDK for Spark on YARN

2024-06-18 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-48651: - Summary: Document configuring different JDK for Spark on YARN Key: SPARK-48651 URL: https://issues.apache.org/jira/browse/SPARK-48651 Project: Spark Issue Type

[jira] [Assigned] (SPARK-48280) Improve collation testing surface area using expression walking

2024-06-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48280: -- Assignee: (was: Apache Spark) > Improve collation testing surface area us

[jira] [Assigned] (SPARK-48280) Improve collation testing surface area using expression walking

2024-06-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48280: -- Assignee: Apache Spark > Improve collation testing surface area using express

[jira] [Resolved] (SPARK-48459) Implement DataFrameQueryContext in Spark Connect

2024-06-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48459. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46789 [https

[jira] [Assigned] (SPARK-48459) Implement DataFrameQueryContext in Spark Connect

2024-06-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48459: Assignee: Hyukjin Kwon > Implement DataFrameQueryContext in Spark Conn

[jira] [Updated] (SPARK-48650) Display correct call site from IPython Notebook

2024-06-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48650: --- Labels: pull-request-available (was: ) > Display correct call site from IPython Noteb

[jira] [Created] (SPARK-48650) Display correct call site from IPython Notebook

2024-06-18 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-48650: --- Summary: Display correct call site from IPython Notebook Key: SPARK-48650 URL: https://issues.apache.org/jira/browse/SPARK-48650 Project: Spark Issue Type

[jira] [Resolved] (SPARK-48342) [M0] Parser support

2024-06-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48342. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46665 [https

[jira] [Assigned] (SPARK-48342) [M0] Parser support

2024-06-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48342: --- Assignee: David Milicevic > [M0] Parser supp

[jira] [Resolved] (SPARK-48585) Make `JdbcDialect.classifyException` throw out the original exception

2024-06-18 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-48585. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46937 [https

[jira] [Assigned] (SPARK-48585) Make `JdbcDialect.classifyException` throw out the original exception

2024-06-18 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-48585: Assignee: BingKun Pan > Make `JdbcDialect.classifyException` throw out the original except

[jira] [Resolved] (SPARK-48647) Refine the error message for YearMonthIntervalType in df.collect

2024-06-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48647. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47004 [https

[jira] [Updated] (SPARK-48649) Add "ignoreInvalidPartitionPaths" and "spark.sql.files.ignoreInvalidPartitionPaths" configs to allow ignoring invalid partition paths

2024-06-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48649: --- Labels: pull-request-available (was: ) > Add "ignoreInvalidPartit

[jira] [Resolved] (SPARK-48644) Do a length check and throw COLLECTION_SIZE_LIMIT_EXCEEDED error in Hex.hex

2024-06-17 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48644. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47001 [https

[jira] [Assigned] (SPARK-48648) Make tags properly threadlocal

2024-06-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48648: Assignee: Hyukjin Kwon > Make tags properly threadlo

[jira] [Resolved] (SPARK-48648) Make tags properly threadlocal

2024-06-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48648. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47005 [https

[jira] [Resolved] (SPARK-47910) Memory leak when interrupting shuffle write using zstd compression

2024-06-17 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-47910. - Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-47910) Memory leak when interrupting shuffle write using zstd compression

2024-06-17 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-47910: --- Assignee: JacobZheng > Memory leak when interrupting shuffle write us

[jira] [Updated] (SPARK-48649) Add "ignoreInvalidPartitionPaths" and "spark.sql.files.ignoreInvalidPartitionPaths" configs to allow ignoring invalid partition paths

2024-06-17 Thread Ivan Sadikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sadikov updated SPARK-48649: - Description: When having a table directory with invalid partitions such as: {code:java} table

[jira] [Created] (SPARK-48649) Add "ignoreInvalidPartitionPaths" and "spark.sql.files.ignoreInvalidPartitionPaths" configs to allow ignoring invalid partition paths

2024-06-17 Thread Ivan Sadikov (Jira)
URL: https://issues.apache.org/jira/browse/SPARK-48649 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 4.0.0 Reporter: Ivan Sadikov -- This message was sent by Atlassian J

[jira] [Updated] (SPARK-48648) Make tags properly threadlocal

2024-06-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48648: --- Labels: pull-request-available (was: ) > Make tags properly threadlo

[jira] [Created] (SPARK-48648) Make tags properly threadlocal

2024-06-17 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48648: Summary: Make tags properly threadlocal Key: SPARK-48648 URL: https://issues.apache.org/jira/browse/SPARK-48648 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-48647) Refine the error message for YearMonthIntervalType in df.collect

2024-06-17 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-48647: - Summary: Refine the error message for YearMonthIntervalType in df.collect Key: SPARK-48647 URL: https://issues.apache.org/jira/browse/SPARK-48647 Project: Spark

[jira] [Resolved] (SPARK-48643) Narrow down default regex of sql options redaction.

2024-06-17 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang resolved SPARK-48643. --- Resolution: Won't Fix > Narrow down default regex of sql options redact

[jira] [Assigned] (SPARK-48497) Add user guide for batch data source write API

2024-06-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48497: Assignee: Allison Wang > Add user guide for batch data source write

[jira] [Resolved] (SPARK-48497) Add user guide for batch data source write API

2024-06-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48497. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46833 [https

[jira] [Assigned] (SPARK-48567) Pyspark StreamingQuery lastProgress and friend should return actual StreamingQueryProgress

2024-06-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48567: Assignee: Wei Liu > Pyspark StreamingQuery lastProgress and friend should return act

[jira] [Resolved] (SPARK-48567) Pyspark StreamingQuery lastProgress and friend should return actual StreamingQueryProgress

2024-06-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48567. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46921 [https

[jira] [Commented] (SPARK-48472) Enable reflect expressions with collated strings

2024-06-17 Thread Mihailo Aleksic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17855742#comment-17855742 ] Mihailo Aleksic commented on SPARK-48472: - Working on this > Enable reflect expressi

[jira] [Commented] (SPARK-48463) MLLib function unable to handle nested data

2024-06-17 Thread Chhavi Bansal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17855714#comment-17855714 ] Chhavi Bansal commented on SPARK-48463: --- Thank you [~weichenxu123] for reopening the ticket

[jira] [Resolved] (SPARK-48642) False reported SparkOutOfMemoryError caused by killing task on spilling

2024-06-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48642. --- Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution

[jira] [Assigned] (SPARK-48642) False reported SparkOutOfMemoryError caused by killing task on spilling

2024-06-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48642: - Assignee: Cheng Pan > False reported SparkOutOfMemoryError caused by killing t

[jira] [Assigned] (SPARK-48566) [Bug] Partition indices are incorrect when UDTF analyze() uses both select and partitionColumns

2024-06-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-48566: - Assignee: Daniel > [Bug] Partition indices are incorrect when UDTF analyze() uses b

[jira] [Resolved] (SPARK-48566) [Bug] Partition indices are incorrect when UDTF analyze() uses both select and partitionColumns

2024-06-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-48566. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46918 [https

[jira] [Resolved] (SPARK-47579) Spark core: Migrate logInfo with variables to structured logging framework

2024-06-17 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47579. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46951 [https

[jira] [Updated] (SPARK-48177) Upgrade `Parquet` to 1.14.1

2024-06-17 Thread Fokko Driesprong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated SPARK-48177: - Summary: Upgrade `Parquet` to 1.14.1 (was: Upgrade `Parquet` to 1.14.0) > Upgr

[jira] [Resolved] (SPARK-48603) Update *ParquetReadSchemaSuite to cover type widen capability

2024-06-17 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48603. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46959 [https

[jira] [Assigned] (SPARK-48603) Update *ParquetReadSchemaSuite to cover type widen capability

2024-06-17 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-48603: Assignee: Cheng Pan > Update *ParquetReadSchemaSuite to cover type widen capabil

[jira] [Resolved] (SPARK-48641) Upgrade curator to 5.7.0

2024-06-17 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48641. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46998 [https

[jira] [Updated] (SPARK-48644) Do a length check and throw COLLECTION_SIZE_LIMIT_EXCEEDED error in Hex.hex

2024-06-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48644: --- Labels: pull-request-available (was: ) > Do a length check and th

[jira] [Assigned] (SPARK-48620) Fix internal raw data leak in YearMonthIntervalType

2024-06-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48620: -- Assignee: Apache Spark > Fix internal raw data leak in YearMonthIntervalT

[jira] [Assigned] (SPARK-48620) Fix internal raw data leak in YearMonthIntervalType

2024-06-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48620: -- Assignee: (was: Apache Spark) > Fix internal raw data l

[jira] [Resolved] (SPARK-48572) Fix DateSub, DateAdd, WindowTime, TimeWindow and SessionWindow expressions

2024-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48572. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46943 [https

[jira] [Assigned] (SPARK-48572) Fix DateSub, DateAdd, WindowTime, TimeWindow and SessionWindow expressions

2024-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48572: --- Assignee: Mihailo Milosevic > Fix DateSub, DateAdd, WindowTime, TimeWin

[jira] [Updated] (SPARK-48642) False reported SparkOutOfMemoryError caused by killing task on spilling

2024-06-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48642: --- Labels: pull-request-available (was: ) > False reported SparkOutOfMemoryError cau

[jira] [Updated] (SPARK-48643) Narrow down default regex of sql options redaction.

2024-06-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48643: --- Labels: pull-request-available (was: ) > Narrow down default regex of sql opti

[jira] [Resolved] (SPARK-48600) Fix FrameLessOffsetWindowFunction expressions implicit casting

2024-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48600. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46987 [https

[jira] [Assigned] (SPARK-48600) Fix FrameLessOffsetWindowFunction expressions implicit casting

2024-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48600: --- Assignee: Mihailo Milosevic > Fix FrameLessOffsetWindowFunction expressions impli

[jira] [Updated] (SPARK-48643) Narrow down default regex of sql options redaction.

2024-06-17 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-48643: -- Description: The current *`spark.sql.redaction.options.regex`* configuration is too loose, resulting

[jira] [Created] (SPARK-48643) Narrow down default regex of sql options redaction.

2024-06-17 Thread Zhen Wang (Jira)
Zhen Wang created SPARK-48643: - Summary: Narrow down default regex of sql options redaction. Key: SPARK-48643 URL: https://issues.apache.org/jira/browse/SPARK-48643 Project: Spark Issue Type

[jira] [Created] (SPARK-48642) False reported SparkOutOfMemoryError caused by killing task on spilling

2024-06-17 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-48642: - Summary: False reported SparkOutOfMemoryError caused by killing task on spilling Key: SPARK-48642 URL: https://issues.apache.org/jira/browse/SPARK-48642 Project: Spark

[jira] [Assigned] (SPARK-48610) Remove ExplainUtils.processPlan synchronize

2024-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48610: --- Assignee: Ziqi Liu > Remove ExplainUtils.processPlan synchron

[jira] [Resolved] (SPARK-48610) Remove ExplainUtils.processPlan synchronize

2024-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48610. - Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-48557) Support scalar subquery with group-by on column equal to constant

2024-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48557. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46902 [https

[jira] [Assigned] (SPARK-48557) Support scalar subquery with group-by on column equal to constant

2024-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48557: --- Assignee: Jack Chen > Support scalar subquery with group-by on column equal to const

[jira] [Resolved] (SPARK-48627) Perf improvement for binary to HEX_DISCRETE strings

2024-06-16 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48627. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46984 [https

[jira] [Assigned] (SPARK-48627) Perf improvement for binary to HEX_DISCRETE strings

2024-06-16 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-48627: Assignee: Kent Yao > Perf improvement for binary to HEX_DISCRETE stri

[jira] [Resolved] (SPARK-48577) Replace invalid byte sequences in UTF8Strings

2024-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48577. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46899 [https

[jira] [Resolved] (SPARK-48633) Upgrade scalacheck to 1.18.0

2024-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48633. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46992 [https

[jira] [Resolved] (SPARK-48587) Avoid storage amplification when accessing sub-Variant

2024-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48587. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46941 [https

[jira] [Updated] (SPARK-48640) Perf improvement for format hex from byte array

2024-06-16 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-48640: - Summary: Perf improvement for format hex from byte array (was: Perf improvement for format hex

[jira] [Updated] (SPARK-48640) Perf improvement for format hex

2024-06-16 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-48640: - Priority: Minor (was: Critical) > Perf improvement for format

[jira] [Updated] (SPARK-48640) Perf improvement for format hex

2024-06-16 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-48640: - Parent: SPARK-48624 Issue Type: Sub-task (was: Improvement) > Perf improvement for format

[jira] [Created] (SPARK-48640) Perf improvement for format hex

2024-06-16 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-48640: --- Summary: Perf improvement for format hex Key: SPARK-48640 URL: https://issues.apache.org/jira/browse/SPARK-48640 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-48615) Perf improvement for parsing hex string

2024-06-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-48615: Assignee: Kent Yao > Perf improvement for parsing hex str

[jira] [Resolved] (SPARK-48615) Perf improvement for parsing hex string

2024-06-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-48615. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46972 [https

[jira] [Created] (SPARK-48639) Add Origin to RelationCommon in protobuf defnition

2024-06-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48639: Summary: Add Origin to RelationCommon in protobuf defnition Key: SPARK-48639 URL: https://issues.apache.org/jira/browse/SPARK-48639 Project: Spark Issue

[jira] [Assigned] (SPARK-48555) Support Column type for several SQL functions in scala and python

2024-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48555: Assignee: Ron Serruya > Support Column type for several SQL functions in sc

[jira] [Resolved] (SPARK-48555) Support Column type for several SQL functions in scala and python

2024-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48555. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46901 [https

[jira] [Resolved] (SPARK-47777) Add spark connect test for python streaming data source

2024-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46906 [https

[jira] [Assigned] (SPARK-47777) Add spark connect test for python streaming data source

2024-06-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-4: Assignee: Chaoqin Li > Add spark connect test for python streaming data sou

[jira] [Resolved] (SPARK-48597) Distinguish the streaming nodes from the text representation of logical plan

2024-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48597. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46953 [https

[jira] [Assigned] (SPARK-48597) Distinguish the streaming nodes from the text representation of logical plan

2024-06-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48597: --- Assignee: Jungtaek Lim > Distinguish the streaming nodes from the text representat

[jira] [Updated] (SPARK-48574) Fix support for StructTypes with collations

2024-06-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48574: --- Labels: pull-request-available (was: ) > Fix support for StructTypes with collati

[jira] [Commented] (SPARK-48638) Native QueryExecution information for the dataframe

2024-06-16 Thread Sem Sinchenko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17855443#comment-17855443 ] Sem Sinchenko commented on SPARK-48638: --- I'm working on the implementation of that logic

[jira] [Updated] (SPARK-48638) Native QueryExecution information for the dataframe

2024-06-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48638: --- Labels: pull-request-available (was: ) > Native QueryExecution informat

[jira] [Created] (SPARK-48638) Native QueryExecution information for the dataframe

2024-06-16 Thread Martin Grund (Jira)
Martin Grund created SPARK-48638: Summary: Native QueryExecution information for the dataframe Key: SPARK-48638 URL: https://issues.apache.org/jira/browse/SPARK-48638 Project: Spark Issue

[jira] [Updated] (SPARK-48637) On-demand shuffle migration peer refresh during decommission

2024-06-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48637: --- Labels: pull-request-available (was: ) > On-demand shuffle migration peer refresh dur

[jira] [Created] (SPARK-48637) On-demand shuffle migration peer refresh during decommission

2024-06-16 Thread wuyi (Jira)
wuyi created SPARK-48637: Summary: On-demand shuffle migration peer refresh during decommission Key: SPARK-48637 URL: https://issues.apache.org/jira/browse/SPARK-48637 Project: Spark Issue Type

[jira] [Created] (SPARK-48636) Event driven block manager decommissioner

2024-06-16 Thread wuyi (Jira)
wuyi created SPARK-48636: Summary: Event driven block manager decommissioner Key: SPARK-48636 URL: https://issues.apache.org/jira/browse/SPARK-48636 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-48634) Avoid statically initialize threadpool at ExecutePlanResponseReattachableIterator

2024-06-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48634: -- Assignee: Apache Spark > Avoid statically initialize threadp

[jira] [Assigned] (SPARK-48634) Avoid statically initialize threadpool at ExecutePlanResponseReattachableIterator

2024-06-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48634: -- Assignee: (was: Apache Spark) > Avoid statically initialize threadp

[jira] [Resolved] (SPARK-48302) Preserve nulls in map columns in PyArrow Tables

2024-06-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48302. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46837 [https

[jira] [Assigned] (SPARK-48302) Preserve nulls in map columns in PyArrow Tables

2024-06-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48302: Assignee: Ian Cook > Preserve nulls in map columns in PyArrow Tab

[jira] [Updated] (SPARK-11150) Dynamic partition pruning

2024-06-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-11150: --- Labels: pull-request-available release-notes (was: release-notes) > Dynamic partit

[jira] [Assigned] (SPARK-48611) Log TID for input split in HadoopRDD and NewHadoopRDD

2024-06-14 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-48611: -- Assignee: Cheng Pan > Log TID for input split in HadoopRDD and NewHadoop

[jira] [Resolved] (SPARK-48611) Log TID for input split in HadoopRDD and NewHadoopRDD

2024-06-14 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-48611. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46966 [https

[jira] [Reopened] (SPARK-48463) MLLib function unable to handle nested data

2024-06-14 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reopened SPARK-48463: Assignee: Weichen Xu > MLLib function unable to handle nested d

[jira] [Updated] (SPARK-48635) Assign classes to join type errors and as-of join error

2024-06-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48635: --- Labels: pull-request-available (was: ) > Assign classes to join type err

[jira] [Updated] (SPARK-48635) Assign classes to join type errors and as-of join error

2024-06-14 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-48635: Summary: Assign classes to join type errors and as-of join error (was: Assign classes to join type

[jira] [Updated] (SPARK-48635) Assign classes to join type errors and as-of join error _LEGACY_ERROR_TEMP_3217

2024-06-14 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-48635: Summary: Assign classes to join type errors and as-of join error _LEGACY_ERROR_TEMP_3217

[jira] [Updated] (SPARK-48635) Assign classes to join type errors and as-of join error _LEGACY_ERROR_TEMP_3217

2024-06-14 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-48635: Description: job type errors: LEGACY_ERROR_TEMP[1319, 3216] as-of join error: _LEGACY_ERROR_TEMP_3217

[jira] [Updated] (SPARK-48635) Assign classes to join type errors and as-of join error _LEGACY_ERROR_TEMP_3217

2024-06-14 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-48635: Description: job type errors: _LEGACY_ERROR_TEMP_[1319, 3216] as-of join error: > Assign clas

[jira] [Updated] (SPARK-48635) Assign classes to join type errors and as-of join error _LEGACY_ERROR_TEMP_3217

2024-06-14 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-48635: Description: job type errors: _LEGACY_ERROR_TEMP_[1319, 3216] as-of join error: _LEGACY_ERROR_TEMP_3217

  1   2   3   4   5   6   7   8   9   10   >